world bank document · 1 using an absolute poverty line fixed at 60 percent of median equivalent...

Document of

The World Bank FOR OFFICIAL USE ONLY

Report No: ICR00004446

IMPLEMENTATION COMPLETION AND RESULTS REPORT

8426-HR

ON A

LOAN IN THE AMOUNT OF EUR70 MILLION (US$ 95.55 MILLION EQUIVALENT)

TO THE REPUBLIC OF CROATIA

FOR A SOCIAL PROTECTION SYSTEM MODERNIZATION PROJECT (P145171)

May 25, 2018

Social Protection & Jobs Global Practice

Europe And Central Asia Region

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

(Exchange Rate Effective May 25, 2018)

Currency Unit = Croatian Kuna

(HRK)

HRK 6.30 = US$1

US$ 1.17 = EUR 1

FISCAL YEAR

January 1 – December 31

Regional Vice President: Cyril E Muller

Country Director: Arup Banerji

Senior Global Practice Director: Michal J. Rutkowski

Practice Manager: Cem Mete

Task Team Leader(s): Ivan Drabek, Zoran Anusic

ICR Main Contributor: Peter Pojarski

ABBREVIATIONS AND ACRONYMS

CDCI Central Disability Certification Institute

CES Croatian Employment Service

CHII Croatian Health Insurance Institute

CPII Croatian Pension Insurance Institute

CSW Centers for Social Welfare

DLI Disbursement Linked Indicator

EEP Eligible Expenditure Program

EFC Error Fraud and Corruption

ESF European Social Fund

ERDF European Regional Development Fund

EU European Union

GDP Gross Domestic Product

GMB Guaranteed Minimum Benefit

ICR Implementation Completion and Results Report IDCPREPD Institute for Disability Certification, Professional Rehabilitation and

Employment of Persons with Disabilities

IPF Investment Project Financing

IT Information Technology

LI Labor Inspectorate

M&E Monitoring and Evaluation

MDFYSP Ministry of Demography, Family, Youth and Social Policy

MIS Management Information System

MLPS Ministry of Labor and Pensions Systems

MOF Ministry of Finance

MOH Ministry of Health

MPA Ministry of Public Administration

MSPY Ministry of Social Policy and Youth

NRP National Reform Program

OSS One Stop Shop

PAD Project Appraisal Document

PAO Public Administration Office

PDI PDO-Level Results Indicator

PDO Project Development Objective

PIU Project Implementation Unit

POM Project Operational Manual

PPA Project Preparation Advance

StAP Statistically Assisted Profiling

SP Social Protection

SPSM Social Protection System Modernization

SWA Social Welfare Act

TA Technical Assistance

TABLE OF CONTENTS

DATA SHEET ....................................................................... ERROR! BOOKMARK NOT DEFINED.

I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 5

A. CONTEXT AT APPRAISAL .........................................................................................................5

B. SIGNIFICANT CHANGES DURING IMPLEMENTATION .............................................................. 10

II. OUTCOME .................................................................................................................... 11

A. RELEVANCE OF PDOs ............................................................................................................ 11

B. ACHIEVEMENT OF PDOs (EFFICACY) ...................................................................................... 12

C. EFFICIENCY ........................................................................................................................... 14

D. JUSTIFICATION OF OVERALL OUTCOME RATING .................................................................... 15

E. OTHER OUTCOMES AND IMPACTS ......................................................................................... 16

III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 16

A. KEY FACTORS DURING PREPARATION ................................................................................ 16

B. KEY FACTORS DURING IMPLEMENTATION ............................................................................. 17

IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 18

A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................ 18

B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 19

C. BANK PERFORMANCE ........................................................................................................... 20

D. RISK TO DEVELOPMENT OUTCOME ....................................................................................... 22

V. LESSONS AND RECOMMENDATIONS ............................................................................. 22

ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 24

ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ....................... 410

ANNEX 3. PROJECT COST BY COMPONENT ......................................................................... 432

ANNEX 4. EFFICIENCY ANALYSIS ......................................................................................... 443

ANNEX 5. BORROWER COMPLETION NOTE ........................................................................ 476

ANNEX 6. DISBURSEMENT-LINKED INDICATORS WITH PDO-LEVEL INDICATORS .................. 587

The World Bank Social Protection System Modernization Project ( P145171 )

Page 1 of 59

DATA SHEET

BASIC INFORMATION

Product Information

Project ID Project Name

P145171 Social Protection System Modernization Project

Country Financing Instrument

Croatia Investment Project Financing

Original EA Category Revised EA Category

Partial Assessment (B) Partial Assessment (B)

Organizations

Borrower Implementing Agency

Ministry of Finance Ministry of Demographics, Family, Youth and Social

Policy, Ministry of Labor and Pension System

Project Development Objective (PDO) Original PDO

The project aims to improve the efficiency and effectiveness of Croatia’s social protection system.


Page 2 of 59

FINANCING

Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$)

World Bank Financing IBRD-84260

95,550,000 9,176,574 7,425,402

Total 95,550,000 9,176,574 7,425,402

Non-World Bank Financing

Borrower 0 0 0

Total 0 0 0

Total Project Cost 95,550,000 9,176,574 7,425,402

KEY DATES

Approval Effectiveness MTR Review Original Closing Actual Closing

19-Sep-2014 24-Dec-2014 31-Dec-2018 30-Nov-2017

RESTRUCTURING AND/OR ADDITIONAL FINANCING

Date(s) Amount Disbursed (US$M) Key Revisions

21-Nov-2017 7.43 Change in Loan Closing Date(s) Cancellation of Financing Reallocation between Disbursement Categories Change in Implementation Schedule

KEY RATINGS

Outcome Bank Performance M&E Quality

Unsatisfactory Moderately Satisfactory Modest

RATINGS OF PROJECT PERFORMANCE IN ISRs

No. Date ISR Archived DO Rating IP Rating Actual

Disbursements (US$M)

01 25-Dec-2014 Satisfactory Satisfactory 0


Page 3 of 59

02 30-Jun-2015 Satisfactory Satisfactory 1.06

03 15-Feb-2016 Satisfactory Moderately Satisfactory 2.00

04 22-Nov-2016 Moderately

Unsatisfactory Moderately Unsatisfactory 6.48

05 20-Jul-2017 Unsatisfactory Unsatisfactory 7.43

06 30-Nov-2017 Unsatisfactory Unsatisfactory 7.43

SECTORS AND THEMES

Sectors

Major Sector/Sector (%)

Social Protection 100


Public Administration - Social Protection 50

Themes

Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Social Development and Protection 100


Social Safety Nets 26

Social Insurance and Pensions 12

Social protection delivery systems 62

ADM STAFF

Role At Approval At ICR

Regional Vice President: Laura Tuck Cyril E Muller

Country Director: Mamta Murthi Arup Banerji

Senior Global Practice Director: Arup Banerji Michal J. Rutkowski

Practice Manager: Andrew Mason Cem Mete

Task Team Leader(s): Emil Daniel Tesliuc, Ivan Drabek Ivan Drabek, Zoran Anusic

ICR Contributing Author: Peter Ivanov Pojarski


Page 4 of 59


Page 5 of 59

I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES

A. CONTEXT AT APPRAISAL

Context A. Country Context 1. At the time of appraisal, Croatia had just gone through five consecutive years of recession since the outbreak of the global financial crisis in 2008. By 2013, projected real gross domestic product (GDP) was 12 percentage points lower than in 2008, with a contraction of 1 percent in 2013 alone. With monetary policy constrained by the need to keep the kuna-euro exchange rate stable, progress with structural reforms and attracting foreign direct investment had to be accelerated. 2. The prolonged crisis led to a rise in poverty. Although Croatia was classified as a high-income country with an average GDP of US$13,920,1 poverty had increased by over 4 percentage points from 13.3 percent in 2008 to 18 percent in 2012. Household Budget Survey data suggested that the poverty incidence had increased further to 20 percent in 2013, due almost entirely to deterioration in living standards as job losses and real wage reduction depressed household consumption. Government efforts to protect social assistance benefits and old-age pensions during the crisis partially mitigated the impact on poverty and shared prosperity. However, overall consumption still declined substantially between 2008 and 2011 (by 4.6 percent) and distribution fell disproportionally on low-consumption households, that is, the bottom 40 percent whose mean consumption declined by 3.5 percent. The share of spending on categorical social assistance benefits or entitlements was high, while the share of poverty-focused programs was disproportionally low, even after the reform of the Guaranteed Minimum Benefit (GMB), which had taken place not long before the project appraisal. After five years of recession, the demand for social protection (SP) was on the rise, while the resources to finance it were scarce. 3. Croatia joined the European Union (EU) on July 1, 2013, and additional funds became available for social protection, thus allowing the budget space to meet some of the unmet demand for SP spending. B. Sectoral and Institutional Context 4. Institutional complexity. At the time of appraisal, Croatia operated a complex SP system, which was costly to administer. Three ministries were setting policies and four were administering SP programs. The Ministry of Labor and Pensions Systems (MLPS) covered employment and pension policies, administered through two specialized agencies and their networks: the Croatian Pension Insurance Institute (CPII) and the Croatian Employment Service (CES). SP measures for war veterans were set and administered by the Ministry for War Veterans. The Ministry of Social Policy and Youth (MSPY)2 had policy authority over most social assistance programs, but administered only some of them, along with social services, through its network of Centers for Social Welfare (CSW). The Ministry of Public Administration (MPA) was the designated ministry for establishing the one-stop shop (OSS) within the network of public administration offices (PAOs). A child allowance program was administered by the CPII, while birth grants and maternity and paternity

1 using an absolute poverty line fixed at 60 percent of median equivalent consumption in 2008. 2 In 2016, the MSPY was restructured and renamed to Ministry for Demography, Family, Youth and Social Policy (MDFYSP).


Page 6 of 59

allowances were administered by the Croatian Health Insurance Institute (CHII). Each of these agencies had its own network of offices across Croatia. This fragmented administration created costs for beneficiaries applying for benefits at multiple agencies, augmented the risk of error and fraud in the SP system, and hindered policy making, particularly in the area of social assistance. 5. Program complexity and design. Efforts to consolidate some cash transfer programs and improve targeting had been realized through a new Social Welfare Law adopted in 2013. The law introduced a new GMB, which consolidated four different social assistance benefits (the social allowance program, the extended unemployment benefit, the homeland war veterans’ benefit, and the disabled World War II support allowance) and applied means testing with changed equivalence scale to protect the most vulnerable (single elderly and single parents). The Government was advancing a set of reforms to address key weaknesses of the SP system: consolidating major social assistance benefits and the unemployment benefit under the OSS network; harmonizing disability certification; developing a system to prevent, detect and correct error and fraud in high-budget and risk prone SP programs and the labor market; advancing the deinstitutionalization agenda; and developing tools that could target the activation measures to those at risk of becoming long-term unemployed. These reforms were outlined in the first National Reform Program (NRP)3 developed by Croatia as a member of the EU. 6. Error, Fraud and Corruption (EFC). Croatia was seen to be losing significant amounts of money due to ineffective institutions and mechanisms to combat EFC in the SP and labor system. Croatia operated a number of cash benefits with high a-priori risk of EFC, including disability pensions and allowances, income and means-tested programs (the child allowance and the GMB programs), and income replacement programs. The delivery system of these benefits relied on preventive measures to avoid EFC; detection and deterrence efforts were minimal. The capacity of the SP system to detect or deter fraud was low. An exception was the Labor Inspectorate (LI) under the MLPS, which had an adequate number of labor inspectors for a country of the size of Croatia. 7. Areas of advancing reforms. The consolidation of the administration of major social assistance benefits and the unemployment benefit was planned to continue through the announced introduction of the OSS network. This network was supposed to expand gradually, to first include the GMB, then the child allowance and non-contributory maternity/paternity allowances, and finally the unemployment benefit. Disability benefits in Croatia were among the highest in the EU,4 with certification spread across six sectors/agencies with no harmonized methodology. This was expected to be corrected through the introduction of a Central Disability Certification Institute (CDCI) in 2014. Creation of the CDCI within the Institute for Disability Certification, Professional Rehabilitation and Employment of Persons with Disabilities (ICDPREPD) was expected to result in faster, more transparent, and more efficient certification process and service for clients. 8. The legacy of state policies and practices in social care in Croatia was based on a continued reliance on institutional care for vulnerable people. A de-institutionalization master plan was approved in 2010. The corresponding operational plan was approved by ministerial decision on June 18, 2014, setting out Croatia’s goals for deinstitutionalization, transformation plans for existing institutions and roles for important stakeholders involved in the process.

3 Government of Croatia. 2014 National Reform Program. 4 64 percent above EU 27 average.


Page 7 of 59

9. Finally, Croatia experienced significant employment challenges, but its employment service (CES) could provide more efficiently tailored services to the unemployed. A new Social Welfare Act (SWA) introduced measures to encourage work-able social assistance beneficiaries to transition from assistance to work.

Theory of Change (Results Chain)

Activities Outputs Outcomes/PDOs

By introducing OSS, reduced private costs for

social assistance beneficiaries and

unemployed by 15%

CDCI operating independently and percentage of cases

certified by IDCPREPD’s fulltime certifying officers

increased from 0% to at least 90%

EFC risk-prone programs have strengthened

information systems and oversight and control

procedures

715 beneficiaries previously living in State and non-State Institutions or Family homes moved to family-type care

environments

Active labormarket policies tailored to support hard-to-employ

beneficiaries

Direct Project Beneficiaries:Core Sector Indicator: Number of

people affected by the project

A more efficient national social protection system

A more effective national social protection system

The administration of the GMB, Child allowances, Maternity/

Paternity/ Birth Grant for non-contributory programs and the

UnemploymentBenefit programs are

transferred to the OSS

Disability certificationsprocessed centrally, unified

disability certification procedures applied, regional office network and independent IT system with

interfaces to OSS and other agencies

EFC risk-prone programs have strengthened information systems,

and oversight and control procedures including detection of error and fraud using risk-based

investigation, data matching

Number of beneficiariesliving in State and non-

State Institutions orFamily homes moved to

family-type careenvironments

Statisticallyassisted profiling introduced

HIG

HER

LEVEL O

BJEC

TIVES

Assu

re min

imu

m livin

g stand

ard fo

r the p

oo

rest;.A

ssists Fiscal Co

nso

lidatio

n; A

ssist Social In

tegration

and

Family

Rein

tegration

.

Croatia SPSM Project PDO: to improve the efficiency and effectiveness of Croatia’s social protection

Intermediate outcomes

reducing fragmentation in the

delivery ofsocial assistance cash

transfers; harmonizing certification

procedures for disability pensions

andallowances; and

reducing error and fraud in social

protection programs.

reducing the private costs for

beneficiaries leading to increased uptake

ofbenefits; reducing

the number of children and adults

in residential institutions; and

supportingactivation of work

able social assistance

beneficiaries.

Critical assumptions: 1. Sufficient capacity exists/is built to implement the program; 2. Policy and reform

direction is maintained

Note: SPSM = Social Protection System Modernization Project; PDO = Project Development Objective

Project Development Objectives (PDOs) 10. The PDO was to improve the efficiency and effectiveness of Croatia’s social protection system. 11. The efficiency element was to be achieved through reducing fragmentation in the delivery of social assistance cash transfers, harmonizing certification procedures for disability pensions and allowances, and reducing error and fraud in SP programs. The effectiveness aspect was to be achieved by reducing the private costs for beneficiaries leading to increased uptake of benefits, reducing the number of children and adults in residential institutions, and supporting activation of work-able social assistance beneficiaries.


Page 8 of 59

Key Expected Outcomes and Outcome Indicators 12. The PDO had two parts: (a) a more efficient national social protection system, and (b) a more effective national social protection system. 13. The key expected outcome indicators were:

(a) Reduce private costs for social assistance beneficiaries and unemployed by at least 15 percent compared to the 2014 baseline; (b) Percentage of cases certified by IDCPREPD’s5 full-time certifying officers increased from 0 percent to at least 90 percent in 2018; (c) Effective system to reduce error, fraud and corruption in risk-prone social protection programs in place; (d) About 715 children without parental care and children and adults with disabilities moved from institutions to family-type living environments; and (e) Cumulative fiscal savings of at least 0.5 percent of GDP resulting from the one-stop-shop, a reduction in error and fraud and the unification of disability assessment over the project lifetime.

Components

14. The project had three components, addressing five thematic areas. The project components were:

• Component 1: Improving Efficiency and Effectiveness of the Social Protection System;

• Component 2: Investments and Technical Assistance to Support Improvements in Efficiency and Effectiveness of the Social Protection System; and

• Component 3: Project Management Support. 15. The project addressed five thematic areas, and the thematic areas received support through results-based financing (DLI-related disbursements under Component 1) and/or through investment support (Component 2). The five thematic areas were: (a) OSS; (b) CDCI; (c) EFC; (d) de-institutionalization; and (e) activation. The first four out of the five thematic areas received both results-based and investment support, and coincide with the four results areas supported through disbursement-linked indicators (DLIs) disbursements. The fifth thematic area, activation, was supported only through technical assistance (TA) (Component 2 investments), without linking it to the DLIs. The Project Appraisal Document (PAD) interchangeably uses ‘Thematic Areas’ 1 through 4, and ‘Results Areas’ A through D when referring to

5 Institute for Disability Certification, Professional Rehabilitation and Employment of Persons with Disability


Page 9 of 59

the thematic areas covered by the DLIs. To avoid confusion, and as Thematic Areas 1 through 4 coincide with Results Areas A through D, from this point onwards this Implementation Completion and Results Report (ICR) will only use the term ‘thematic area’. Component 1: Improving Efficiency and Effectiveness of the Social Protection System (IBRD planned DLI-linked disbursements EUR 50 million, actual DLI-linked disbursements EUR 4 million) 16. This component used a results-based approach. Payments made under this component were received by the Ministry of Finance (MOF) as reimbursement for expenditures made in the Eligible Expenditure Program (EPP) (in this case, the GMB). Payments were triggered by achievement of pre-specified Disbursement Linked Indicators (DLIs), as well as evidence that sufficient expenditures had been made under the EEP. The DLIs were chosen to support the Government’s SP reform in four thematic areas. Thematic Area 1 (Results Area A): Consolidation of Benefit Administration and Simplification of Procedures (Planned DLI disbursements EUR 20 million, actual DLI disbursement EUR 0) 17. A key element of the Government’s reform was the consolidation of cash administration for social welfare programs, family policy programs and unemployment benefits through a single delivery network, an OSS. The OSS was to transform the experience of social assistance beneficiaries by providing a single service point to apply for and receive a range of cash benefits. The OSS was expected to reduce the time and cost of application for beneficiaries and create the opportunity of introducing a cap to the total amount of social assistance benefits paid to one household, generating potential for savings and reducing work disincentives associated with the provision of multiple need-based social assistance benefits. Each DLI was priced at EUR 5 million, reflecting the difficulty of this reform and its importance for Croatia’s fiscal consolidation plan. Thematic Area 2 (Results Area B): Unifying and Harmonizing the Certification of Disability (Planned DLI disbursements EUR 4 million, actual DLI disbursement 2 million) 18. Another major reform supported by the project was the creation of a CDCI within the IDCPREPD, which was expected to result in a faster, more transparent, and more efficient certification process and service to clients/applicants. Instead of several certifications per applicant, depending on the type of certification, by the end of the project all applicants were expected to be undergoing one certification, ensuring horizontal equity (all persons with disabilities to be certified using a single method). Individual costs of disability certification would thus fall and client satisfaction would rise accordingly. Each DLI was priced at EUR 2 million, reflecting the largely institutional nature of the reform. Thematic Area 3 (Results Area C): Reducing Error, Fraud and Corruption (Planned DLI disbursements EUR 9 million, actual DLI disbursement EUR 0) 19. The Government was committed to generating additional fiscal savings by developing a comprehensive system to reduce the level of error and fraud in the SP system. Each DLI was priced at EUR 2 million, reflecting the largely technical nature of the reform, with a EUR 3 million disbursement for the final year DLI at which point the system would be fully strengthened and best practice, risk-based inspections are being conducted.


Page 10 of 59

Thematic Area 4 (Results Area D): De-institutionalization of Vulnerable Children and Adults. (Planned DLI disbursements EUR 17 million, actual DLI disbursement EUR 2 million) 20. The MSPY’s reform plans also encompassed a process of deinstitutionalization in the provision of social services. Deinstitutionalization referred to a needed shift in the balance of service provision for vulnerable children and adults from institutional care to a range of services provided by a variety of service providers in family-type environments outside of the institutions, including return to the biological family, adoption or foster care, organized housing or other community services. Each DLI for this area was priced at EUR 5 million because they represented higher-risk outcome indicators. Component 2: Investments and Technical Assistance to Support Improvements in Efficiency and Effectiveness of the Social Protection System (IBRD planned EUR 19.4 million, actual EUR 1.83 million) 21. This component used a regular Investment Project Financing (IPF) approach to support all five thematic areas, including TA that was necessary to meet the DLIs. Specific support was also included to leverage EU funds in the areas of deinstitutionalization and activation. The component also supported Thematic Area 5, which was not supported under Component 1. 22. Thematic Area 5: Activation – This thematic area only received support from Component 2, and had no DLIs associated with it. Out of planned financing of EUR 0.40 million, the thematic area utilized EUR 0.24 million. Under this thematic area, the project supported the CES in developing and evaluating a statistically assisted profiling model to identify the clients at risk of long-term unemployment. Component 3: Project Management Support (IBRD planned EUR 0.6 million; actual EUR 0.49 million) 23. This component also (like Component 2) used a regular IPF financing. A Project Implementation Unit (PIU) in the MSPY was the core of project management. Operating expenses for the PIU were to be shared between the MSPY and the loan.

B. SIGNIFICANT CHANGES DURING IMPLEMENTATION 24. After a period of two subsequent elections in the country6, followed by political uncertainty with regard to the objectives of the project, on July 24, 2017, the Government sent to the World Bank a letter requesting early closure of the project and cancellation of the undisbursed amount of the loan. In preparation for early closure, the project was restructured on November 21, 2017 to:

(a) Change the loan closing date(s) – cutting 13 months off project implementation, for an early closing in November 2017 (moved the project closing date from December 31, 2018 to November 30, 2017);

(b) Cancel financing – cancelling EUR 63.28 million out of EUR 70.00 million original loan amount;

6 Croatia had three governments during the period of project preparation and implementation. The first Government, with

which the project was prepared, lasted until November 2015; the second Government was in place from February 2016 until October 2016; and the third Government (at ICR) came in place in October 2016 and was in place at the time of this ICR

preparation.


Page 11 of 59

(c) Reallocate between disbursement categories; and

(d) Change the implementation schedule – to reflect the early closure.

25. In addition, one early change may have contributed to issues in subsequent phases of implementation – this was the moving of the PIU under the Project Preparation Advance (PPA) from the MPA to the MSPY.

Rationale for Changes and Their Implication on the Original Theory of Change

26. The reasons for the early closing of the project were largely policy-related – for example, the change of the

strategic vision regarding the OSS for social benefits and the decision not to proceed with it. The implications of these

changes on the theory of change were significant – practically, the full achievement of the PDO was made impossible.

Some partial elements were still achieved, following the original theory of change logic.

27. The moving of the PIU from the MPA to the MSPY may have signaled problems with commitment and coordination

and delayed the implementation, which was supposed to start under the PPA.

II. OUTCOME

A. RELEVANCE OF PDOs

The PDO relevance at closing was Modest.

Assessment of Relevance of PDOs and Rating 28. The initial PDO prospects were optimistic. At appraisal, the Government’s NRP included the objectives of assuring a minimum living standard for the poorest, facilitating access to social welfare rights, and improving the system of awarding social benefits (including structural administrative reforms). All five aspects (thematic areas) of the project were cited in the NRP and the World Bank’s support under the proposed project was explicitly referenced. The NRP was the first prepared by Croatia as a member of the EU and as part of the yearly cycle of economic policy coordination at the EU level. Two of the pillars in the Croatia Country Partnership Strategy FY14-17 provided the framework for the proposed operation. Pillar 1 related to public finance and within this, Theme 1 aims to promote fiscal consolidation. The project could deliver fiscal savings that could amount to 0.5 percent of GDP during its lifetime (2014-2018). It was also seen as support for Pillar 3, which relates to maximizing the benefits of EU membership by assisting the authorities to prepare for and access European funds for activation and de-institutionalization. 29. However, at closing the situation had changed in terms of the desired outputs and outcomes. Although the Government’s Completion Note for the project (see annex 5) states that the project objectives are still relevant, that relevance had been limited by the reduction in scope and the exclusion of major activities from the project. The Government that came to power in October 20167 changed its technical approach and removed the OSS from its priorities. The removal of the OSS blurred the direction of the reform agenda and diminished the relevance of the project objectives. Still, it should be noted that the revised NRP that was developed in 2017 generally confirmed the administrative and

7 See footnote 6.


Page 12 of 59

efficiency aspects by stating that that “the reform of the social protection system will be carried out with an emphasis on the area of social welfare and social benefits system with the aim of increasing its efficiency, transparency and fairness, as well as ensuring its sustainability.” NRP 2017 also talks about plans to consolidate SP benefits, improving targeting and increasing coverage, although the integration of the SP system with other systems is now seen only in a limited form, in the area of data and information sharing.

B. ACHIEVEMENT OF PDOs (EFFICACY)

30. Efficacy of the project is rated at the border between Negligible and Modest. The project disbursed only about 9 percent of its original commitment, barely passing the threshold to be considered as having had any meaningful implementation. Nevertheless, the small disbursement still managed to affect some of the results areas, supporting some limited areas of the outcomes and catalyzing others.

Assessment of Achievement of Each Objective/Outcome

PDO Part 1: Make the National Social Protection System More Efficient

31. This part of the PDO was supported by three of the five project thematic areas, as listed in the following paragraphs. Efficiency was supposed to be achieved by reducing fragmentation in the delivery of social assistance cash transfers, harmonizing certification procedures for disability pensions and allowances, and reducing error and fraud in the SP programs.

32. Thematic Area 1: Consolidation of Social Assistance Cash Benefits Administration and Simplification of Procedures under One-Stop-Shop. The planned disbursement for achievement of the DLIs under this thematic area for the duration of the project was EUR 20 million. None of the four DLIs, nor the PDO indicator, under this thematic area was achieved. The project disbursed EUR 0 against the DLIs for this thematic area. Out of EUR 12 million allocated for this thematic area under the project’s investment component, only about EUR 0.64 million was spent. The OSS for social benefit administration was prioritized by the Government at appraisal (2012-2015) as one of the longer term fiscal consolidation measures and as such was supported by the project. However, the next Government (2016) decided to abandon the concept of consolidation of social benefit administration and simplification of procedures under the OSS and to proceed with a ‘light’ approach that would keep the administration unchanged and focus solely on strengthening the links among the existing information systems for the various social benefits as well as on eventual consolidation only of those social benefits that are not directly demographic.8 This new direction was reflected in the latest NRP. 33. Thematic Area 2: Unifying, Improving and Simplifying Disability Certification Process. Out of planned DLI disbursements of EUR 4 million, actual DLI disbursement under this thematic area was EUR 2 million. Under the investment component, out of EUR 4 million, only EUR 0.3 million was spent. The creation of a CDCI within the IDCPREPD had an objective to establish a faster, more transparent, and more efficient certification process and service to clients/applicants. Positive developments were noted during the initial phase of project implementation: one DLI was achieved (EUR 2 million), and project activities were launched for establishing the CDCI, including business processes design, one information technology (IT) procurement, and training. Later, however, the Government challenged the disability certification unification concept and IDCPREPD’s effectiveness, removed the disability certification reform from

8 One of the current priorities of the Croatian Government is curbing the depopulation tendency by supporting various ‘demographic’ measures aimed at increasing the birth rate, supporting families with children, and so on.


Page 13 of 59

the NRP, and put on hold the project component implementation in 2016. The PDO indicator associated with this component was not achieved. 34. Thematic Area 3: Reducing Error, Fraud and Corruption in Social Benefits. The project did not disburse any funds out of the EUR 9 million planned DLI disbursements under this thematic area. Under the investment component, out of EUR 2.25 million allocated for this thematic area, only EUR 0.34 million was utilized. The project aimed to develop a comprehensive system to reduce the level of EFC with the objective of generating additional fiscal savings. The TA financed a study tour to the United Kingdom and supported the development of a strategy for combating EFC in social benefits which was approved by the Government in November 2015. An action plan was developed subsequently and approved by the World Bank team in late 2016. However, it was not adopted by the new Government. The World Bank team was informed by the MLPS that they intend to implement the activities described in the draft action plan using the EU funds instead of the World Bank loan. However, during the ICR preparation no progress could be verified on any action from the draft action plan.

PDO Part 2: Make the National Social Protection System More Effective

35. This part of the PDO was covered by two thematic areas – on deinstitutionalization of vulnerable children and adults; and on work activation of the inactive and unemployed. Effectiveness was to be achieved by increasing the uptake of benefits, reducing the number of children and adults in residential institutions, and supporting activation of work able social assistance beneficiaries. 36. Thematic Area 4: Deinstitutionalization of Vulnerable Children and Adults. The project disbursed EUR 2 million against the DLIs for this thematic area out of the planned EUR 17 million. Of EUR 0.75 million allocated for de-institutionalization under the investment component, about EUR 0.36 million has been used. The PDO indicator associated with this component was not achieved. The first DLI, which entailed production, adoption and publication of the operational plan for deinstitutionalization was met early in the project implementation. However, the second, third and fourth DLI, which entailed the de-institutionalization of 210, 225 and 280 people, respectively, to family-type environments with adequate support and net de-institutionalization of 50 percent have not been achieved. Eventually, the line ministry decided to withdraw from the implementation of the project’s de-institutionalization sub-component and cancel all the existing consultant contracts in June 30, 2017. 37. Thematic Area 5: Activation. This thematic area only received support from the investment Component 2 and had no DLI’s associated with it. Out of planned financing of EUR 0.40 million, the thematic area benefitted from disbursed financing of EUR 0.24 million. Under this thematic area, the project successfully supported the CES in developing and evaluating a statistically assisted profiling model to identify the clients at risk of long-term unemployment. The pilot started in March 2017 and an impact evaluation has been carried out by the CES, using a randomized impact evaluation design. This thematic area successfully implemented most of the activities envisioned under the project.

Justification of Overall Efficacy Rating 38. As seen from the details above, the operation was barely implemented and thus largely did not achieve its intended outcomes. The project failed to meet the majority of its PDO indicators, and did not disburse on the majority of its DLIs related to structural policy improvements. There was, however, some limited progress on a few items from the implementation and policy agenda. Therefore, the project’s efficacy is at the borderline between negligible and modest.


Page 14 of 59

39. Out of the five PDO indicators, two achieved no progress whatsoever, including the indicator on the OSS, where the project envisaged the largest amounts of investments. The other three indicators had some positive movement towards the targets but those were piecemeal or unconfirmed. To mention the positives, some progress is noted on the disability certification. Numerically, progress is also seen on the deinstitutionalization indicator but that cannot be classified as genuine progress as the number of deinstitutionalized persons is offset by an even larger number of people who entered the institutions during the same period (the indicator does not look at the net difference and does not capture this). The activation activities – albeit small - also brought some value added. 40. The other aspect of looking at outcomes is the meeting of the DLI requirements and the disbursements towards the DLIs. Although the DLIs are largely output-based, they are closely connected to the achievement of the outcomes and can be looked at as partial proxies. The project only met the conditions on 2 (two) out of 14 (fourteen) DLIs and disbursed only EUR 4 million out of the planned EUR 50 million.

C. EFFICIENCY

Assessment of Efficiency and Rating

41. Efficiency was Modest.

Original Economic Analysis vs Actual Developments

42. The original project’s economic and financial analysis looked at several aspects of the project’s development

impact on efficiency, as well as on fiscal savings from the implementation of various policies, and leveraging EU funds.

43. The analysis of efficiency as part of the development impact was reviewed earlier in this ICR as part of the

assessment of outcome, as it is part of the PDO. Expected efficiency gains from improved targeting, OSS, increased

coverage, administrative savings from better disability certification, and so on did not materialize. This has already been

assessed as part of the Efficacy rating. Additional economic analysis is not possible given the low level of implementation.

44. With regards to the leveraging of EU funds through support for preparation of European Social Fund (ESF) and

European Regional Development Fund (ERDF) project proposals in the deinstitutionalization area and through the

activation work, the project seems to have had some positive effects, including a positive fiscal impact by helping the

Government to generate non-deficit creating spending/investments by better utilizing EU funds. However, because of the

overall economic improvements and the general inflow of EU funds governed by the country’s EU membership, it is

difficult to say what part of those outcomes could be attributed to the project. While there is no quantitative measure of

this contribution, it is likely to have been negligible to modest.

Efficiency Aspects of Design and Implementation. Administrative Costs.

45. The design included elements with high risk – the materialization of these risks, and the blockage of some of the

design elements by the Government led to significantly reduced efficiency. Delayed implementation and the non-

implementation of important design elements also played an important role in reducing the efficiency.


Page 15 of 59

46. Preparation. Overall, the preparation process for the project was following a reasonable timeline. After

identification of the project in February-March 2013, a Project Concept Review was conducted in December 2013.

Because of the complexity of the project task and the work in an EU country, the team sought further advice and input

from a technical Quality Enhancement Review in April 2014. Appraisal was conducted in June 2014, followed by Board

approval on September 19, 2014, and signing of the Loan Agreement on September 25, 2014. Effectiveness was declared

on December 24, 2014. A PPA agreement was signed in April 2014, before project appraisal. The PPA, however, could not

be utilized as some of the activities it was supposed to finance were delayed and started at the time the project was

already approved.

47. Cancellation. A significant portion of the loan was cancelled – more than 90 percent of the original loan amount.

This cancellation affected mainly the substantive Components 1 and 2, thus leading to the inability of the project to fully

meet its development objectives. While the impact was significantly reduced, the cost of project management remained

quite high and the sunk costs for project preparation cannot be recovered or compensated.

48. Administrative Costs. The component that was planned to provide the bulk of financing – Component 1

(Improving efficiency and effectiveness of the social protection system) – was reduced from EUR 50 million to EUR 4 million

because of the early closure, cancellation and inability of the project to meet the disbursement requirements from all but

two of the DLIs. Component 2, which was supposed to provide investments and TA for meeting the DLIs was also cut from

EUR 19.4 million to EUR 1.83 million. These reductions inevitably adversely affected the expected efficiency gains from

the project. These two substantive components disbursed cumulatively only 8.4 percent of the initially planned (at

appraisal) financing. At the same time, the component for Project Management spent EUR 0.49 million of the planned

EUR 0.60 million, which is more than 80 percent of the planned management cost at appraisal. Proportional to the

disbursement under the original financing, the project was supposed to have EUR 12,000 of management costs for every

million Euros disbursed, while in reality the project had more than EUR 84,000 of administrative expenses for every million

of substantive financing utilized. This makes the project management approximately seven times less efficient in

administrative terms than planned9.

D. JUSTIFICATION OF OVERALL OUTCOME RATING

49. Overall Outcome of the project is rated Unsatisfactory. The operation’s objectives were promising at the beginning, but only modestly relevant at closing. There were significant shortcomings in the operation’s progress towards achievement of the objectives, and although some limited progress on specific activities was noted, the overall objectives were barely achieved. Efficiency was also modest, and below expectation. The shortcomings were caused by the materialization of political risks that affected the design. These developments practically removed a key results area, and led to an early closing and the cancellation of more than 90 percent of the originally-planned financing, which affected most of the development impact areas.

9 Originally the plan was to establish the PIU in the MPA, and all members should have been public servants, with no additional administrative cost to be financed through the project. However, in June 2014, the Government decided to shift this task to the MSPY. As there was limited capacity in the MSPY to establish a PIU, in coordination with the World Bank a decision was made that PIU members would be hired as consultants, and all their expenses would be covered from the project management component.


Page 16 of 59

E. OTHER OUTCOMES AND IMPACTS

Institutional Strengthening

50. There were some institutional development and strengthening impacts. Coordination among various

Government institutions was initiated and improved in the areas related to disability certification and activation.

III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME

51. The project was approved on September 19, 2014, and became effective on December 24, 2014. The project had a relatively good implementation start, with activities immediately initiated in all the project areas after project approval. However, the initial progress was followed by a slowdown in the implementation in anticipation of the parliamentary elections (held in November 2015). A period of political changes that followed, with snap elections in 2016, halted the main project-supported reforms and almost all activities, except for the activation sub-component. The Government that entered in office in October 201610 took several months to review the project, and eventually decided that its main objectives and activities did not fit with the Government’s new priorities. Most importantly, the Government decided not to introduce the OSS approach in its original format, and requested that it be removed from the project. In the months that followed the World Bank project team and management explored opportunities with the Government to restructure the project. However, these efforts were not successful and in early 2017, two years after effectiveness, the Croatian Government decided to request the closing of the project. Notably at appraisal, the OSS had been the largest activity contributing towards meeting the project’s objective of improving efficiency of the country’s SP system. After intense discussions on possible restructuring of the remainder of the project, in June 201711 the Government decided to close the project early, and to completely cancel all remaining funds. Thus, the project was closed 13 months earlier from the original schedule, after having disbursed only 9 percent of its original amount.

A. KEY FACTORS DURING PREPARATION

52. Overall, at the time of preparation analysis, the objectives of the project seemed realistic and clearly formulated. The Results Framework had appropriate indicators and a plan for monitoring, although, because of the premature project closure, this plan cannot be fully evaluated. 53. However, the project design was complex and involved many areas of intervention and a mixed instrument – IPF with a results-based component. The choice of instrument may have been particularly important for the development of the project. At the concept stage, the mixed, more complex instrument was chosen over a simple, traditional IPF without the results-based part. Using the larger instrument with DLIs was seen as an entry point for a more comprehensive reform of the SP system. In retrospect, the choice of a simpler IPF could have allowed more flexibility to restructure or respond to political and implementation challenges.

54. Related to the complexity of design, there was also an issue with the large number of participating stakeholders. While there was one PIU in place, it was difficult for it to control and influence all the various authorizing environments of all the participating institutions. Steps were taken to ensure readiness for implementation – a PPA in the amount of EUR 890,000 was approved to accelerate the project preparation. However, the PPA was never properly utilized as it took

10 See footnote 6. 11 Officially, the decision was communicated to the World Bank in July 2017.


Page 17 of 59

longer than planned to create a PIU and in the preparation stage the PIU was moved from one ministry to another.12 Therefore, the activities planned to be undertaken under the PPA were moved to the project implementation phase. The actual project became effective eight months after the PPA signing. 55. The political risk was also underestimated, and once the Government’s commitment to some of the policy reforms dropped, the objectives’ relevance was diminished and the project was discontinued.

B. KEY FACTORS DURING IMPLEMENTATION

56. The ICR analysis concludes that there were no major factors outside of Government or implementing entities’ controls that may have influenced the limited implementation that took place. The macroeconomic situation of Croatia during the period was improving and the country’s stability and stance also marked achievements, including joining the EU in mid-2013, only months before the start of active project implementation. While there were two elections13 during the life of the project, they did not mark any major political instability, in the sense that the country kept its main priorities as an EU member. While the changes in government eventually resulted in the Government pulling its support from a major project activity (the OSS), this cannot be considered a factor outside Government control. 57. In terms of the factors subject to World Bank control, they are also summarized in other sections further in the text, including the stable implementation support, the candid reporting and the proactive search for solutions to restructure the project. For example, between January and June 2017, when it was already clear that the OSS activities were to be removed, the World Bank was open to discuss options for continuing the project through restructuring and change of the implementation modality. One such feasible opportunity was for the MLPS to take over the implementation of a refocused project, preserving the areas of CDCI and EFC. 58. Finally, but perhaps most importantly, there were a number of factors subject to Government and implementing entities control, which influenced implementation and provided additional explanation for the suboptimal outcome of the project. These factors are summarized in the following paragraphs by project thematic area, to supplement the story about the non-achievement of the PDO.

Consolidation of Social Benefit Administration and Simplification of Procedures (One Stop Shop)

59. The Government's strategy at the time of appraisal was to create a single administrative point within the network of PAOs for the citizens for exercising their rights to benefits. However, the next Government (2016) had doubts about the concept of placing the OSS into the network of PAOs because it abandoned the idea of creating a single administrative point for citizens in the PAOs. In addition, new leadership of the MSPY opposed the idea of moving the administration of the GMB away from the CSWs. 60. As explained earlier, the Government decided to abandon the concept of the OSS as originally envisioned by the project and to cancel all project activities under this thematic area. Moreover, the SWA was amended to take out the provisions related to transferring the administration of the GMB to the PAOs.

Central Disability Certification Institute (CDCI)

12 See footnote 9 for details. 13 See footnote 6.


Page 18 of 59

61. This thematic area achieved limited implementation progress during the life of the project. Initial activities focused on funding urgent IT procurement in the city of Zagreb, design of IDCPREPD’s business processes, training in unified disability certification methodology, and some small office renovations. In the third quarter of 2016, the Government requested that the project freezes the investment activities under the CDCI sub-component until completing a review of business practices and constraints of the IDCPREPD (CDCI is an integral part of the IDCPREPD). Options for improvement and possible amendments to the planned project activities were to be discussed following the review. However, the review was never initiated and the IDCPREPD engaged in self-development of fragmented business processes (certification, human resource system, office management, and other functions) that differed significantly from the business process design prepared in the initial stage of the SPSM Project.

Reducing EFC in the SP System 62. The Strategy for Combating EFC in Social Benefits was approved by the Government in November 2015 and an action plan was prepared by the working group led by the MLPS and the MDFYSP, reviewed and approved by the World Bank in late 2016. The action plan was not adopted as of the time of this ICR, and the new Government had not appointed a new head and members of the working group. The Government shared plans to implement the activities described in the draft action plan using the EU funds instead of the World Bank loan funds. However, the ICR review did not find convincing evidence that the process was progressing.

De-institutionalization of Vulnerable Children and Adults

63. The national deinstitutionalization team, comprising three consultants hired under the project who were responsible for the coordination and planning of the process of deinstitutionalization and transformation of social welfare institutions, helped prepare transformation plans for 15 social welfare institutions, of which 4 were approved. Preparation of individual transformation plans was one of the preconditions for submission of project proposals by the social welfare institutions for the ESF and ERDF financing. Approvals of the individual transformation plans were delayed, which contributed to slow utilization of the EU funds allocated for this purpose. Three institutions that have advanced the transformation processes and had their transformation plans approved have received financing from the EU funds to support the implementation of the transformation plans.

Activation

64. The activation activity is perhaps the one marking best progress, with the pilot of statistically-assisted profiling (StAP) in the CES having started in March 2017. The pilot was phased, with four groups starting to implement the pilot during the life of the project. The feedback from the CES counsellors indicated that the StAP was easy to use, and the tool was well appreciated.

IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME

A. QUALITY OF MONITORING AND EVALUATION (M&E) Rating: Modest

M&E Design


Page 19 of 59

65. The operation’s theory of change was relatively clear from the onset, and adequate indicators were identified to

monitor progress toward the PDOs. However, because of the project’s complex design, planned M&E arrangements did

not seem fully sufficient to provide the needed data on results. Results monitoring was a task of the MSPY, which reported

on budget execution of the EEP and lead the reporting on all activities under the project. Project Implementation Reports

were sent to the World Bank semi-annually. These reports described progress against each of the five thematic areas

covered by the project. The reports also contained information on fiduciary functions. A format for these reports was

agreed with the counterparts and set out in the Project Operational Manual (POM). Separately from that, some thematic

areas were supposed to have their own M&E arrangements, such as EFC, the LI, and the deinstitutionalization. However,

because of the limited implementation, these arrangements also largely remained unutilized.

M&E Implementation

66. The actual delivery of the M&E reports and systems’ outputs, as described earlier, was not systematic and mostly

at the World Bank’s request. Several M&E reports in the form of Excel files with the numbers of beneficiaries of social

welfare institutions which showed the flow of beneficiaries were provided throughout the project implementation. Those

were not on quarterly basis, but rather at the World Bank’s request, ahead of missions. Overall, because of the project’s

early cancellation, most of the planned M&E activities were either not conducted, or conducted in а reduced format (for

example, some impact assessment work on Activation was initiated, but was only at initial stage of implementation at

the time of the ICR).

M&E Utilization

67. Because of the limited implementation, not much was done in terms of analyzing the data collected under the

various thematic areas except perhaps in the area of activation. In the activation area, some project-supported

information data tools were implemented and utilized on the ground (for example, piloting the StAP14).

Justification of Overall Rating of Quality of M&E

68. The overall quality of the M&E system was modest. Overall, the M&E system was not developed in much detail.

The part of the M&E system that concerned the general monitoring of project implementation at the PIU level seems to

have been functional and sufficient – the PIU was diligent in collecting data and reporting at its own level. However, the

elements of monitoring the thematic areas had never been fully developed before the project was prematurely closed.

B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE

Environmental Compliance

14 The Statistically Assisted Profiling (StAP) is based on a tool that estimates the probability of employment in the next 12 months for newly registered unemployed persons and categorises newly registered unemployed persons into risk groups based on estimated probability of employment. StAP was piloted in two regional offices.


Page 20 of 59

69. Because of the expected limited refurbishment activities, the project was rated as environmental Category B according to the World Bank categorization. The project triggered the policies on Environmental Assessment (OP 4.01) and Physical Cultural Resources (OP 4.11). Due to the early project closure and the limited disbursements, there was practically no implementation of activities that would require the application of the environmental and cultural safeguards. No environmental issues were registered during the project implementation and the environmental part of the POM remained only theoretical.

Social Compliance

70. Because of the limited amount of disbursements and the limited number of implemented activities and policy developments (including the failure to implement the OSS), much of what was planned during appraisal in terms of positive social developments did not materialize. On the other hand, there were also no visible net negative impacts. The project did not trigger OP 4.12 on Involuntary Resettlement. No land acquisition, displacement of occupants (legal or illegal), or restriction of access to resources or income streams took place during project implementation. The project did not finance any new construction or any significant refurbishments of existing buildings, with the exception of small rehabilitation works. 71. Some positive social effects can be assumed as a result of the deinstitutionalization activities, where children deprived of parental care and adults and children with development disabilities were provided with family-type living environment. The activities implemented under the activation agenda may have also benefitted the target groups by providing them better opportunities to return to sustainable living.

Procurement

72. Procurement under the project was carried out in accordance with the applicable Procurement and Consultant Guidelines and as agreed in the relevant project documents, that is, the Loan Agreement and the Procurement Plan. A post review was not conducted because all implemented contracts were subject to prior review. In the final implementation support documents procurement was rated satisfactory.

Financial Management

73. The project underwent regular financial management implementation support and supervision and the last review (before closing) took place in January 2017. The PIU established under the MSPY (later renamed MDFYSP) had adequate capacity for the project financial management with proper staffing and project management. All project-related audited financial statements and management recommendation letters for the whole project duration were found acceptable and were submitted on time. The PIU was also reporting on a semester basis to the World Bank in the form of interim unaudited financial reports. Such reports have been delivered on time and were found acceptable by the World Bank.

C. BANK PERFORMANCE Rating: Moderately Satisfactory

Quality at Entry


Page 21 of 59

74. In terms of analytical foundation, the project entry was solid. The project built on previous and existing operations

in the sector and analytical work, including the Social Assistance chapter of the Public Finance Review. An NRP TA task15

and the ERDPL-216 complemented the project’s focus on the efficiency of the sector by analyzing and supporting

parametric reforms that aimed to increase the share of SP programs that were means-tested. The deinstitutionalization

element was founded on the activities from the earlier Social Welfare Development Project,17 which among others

focused on strengthening the community services for vulnerable people. The activation work was based on regional

programmatic knowledge and advisory service task on activation and skills18.

75. One issue of the quality at entry was the adoption of a complex design framework that includes five thematic

areas, numerous counterparts, and a mixed instrument of IPF with results-based financing. According to records,

simplified options were considered at the time of the concept review (for example a simple IPF focused on a smaller

number of areas), however, it was eventually decided by management to pursue a broader and deeper, albeit riskier,

intervention.

76. Another issue at entry was the underestimating of the risks, and more specifically – the stakeholder risks and the

design risks. The PAD correctly notes that there could be parts of the administration resisting the reform, and especially

the OSS. The PAD also notes that the negotiating Government is politically in exit mode with elections nearing. The

complexity of the design is also recognized, with the OSS being a key element of that complexity. The team, however,

assessed these risks relatively optimistically, as Substantial rather than High, and declared them easy to mitigate. Overall

risk was therefore assessed as Moderate. Unfortunately, these risks, although with not a high probability of materializing,

had the potential to disrupt the whole project – which became clear once they indeed materialized.

Quality of Supervision

77. Supervision was hands-on and proactive throughout the life of the project. The project at all times had tsk team

leaders based in the field. Dialogue was close and comprehensive, without interruptions. The World Bank provided

needed advice and guidance in all areas, including procurement and financial management. Once it became clear that the

stakeholder risk was materializing, the World Bank team proactively analyzed the situation and discussed options for

restructuring. Meetings and consultations with the Government were carried out, and suggestions for restructuring and

partial cancellation were proposed. While eventually the Government chose not to restructure but to outright cancel,

there is ample evidence that various restructuring options were discussed and put on the table by the World Bank. Once

the Government made its final choice, the World Bank team efficiently supported the closing and cancellation of the

remaining loan funds.

Justification of Overall Rating of Bank Performance

78. While the World Bank underestimated some risks at the design phase, the overall quality at entry was based on

sound analytical ground and good dialogue with the counterparts. After project approval, supervision was field-based and

15 National Reform Program Technical Assistance (P147844). 16 Second Economic Recovery Development Policy Loan (P127665) approved by the World Bank’s Board of Directors on April 29, 2014. 17 Social Welfare Development Project (P069937) approved by the World Bank’s Board of Directors on June 15, 2005 and closed on March 31, 2011. 18 Activation and Skills for Employability and Protection Technical Assistance (P143351).


Page 22 of 59

proactive. The unfortunate materialization of the risks was swiftly identified and options were proposed before the final

decision on early closure was taken. Therefore, the overall World Bank performance is assessed as Moderately

Satisfactory.

D. RISK TO DEVELOPMENT OUTCOME 79. The risk to the development outcome of the project is High. First, a significant portion of the project’s development outcome was not achieved at all. Second, without the project, the development objectives have low chances of being achieved any time soon – the national level environment seems to have deprioritized the objectives set up in the project. While improving the overall efficiency and effectiveness of the SP remain on the agenda, it has been pushed aside because of their overall unpopular nature (for example, stricter controls, means testing, less inclusion errors) and have been blocked by various administrative hurdles.

V. LESSONS AND RECOMMENDATIONS

Policy-level Lessons

• Policy support under results-based approach should choose straightforward and clear policy steps. A clear and manageable set of policies should be chosen under the DLIs. Each policy change should be concrete and should be building on an already firm foundation. Planning policy changes without first ensuring that their prerequisites had taken place, may lead to slow or unsuccessful implementation.

Design and Implementation Lessons

• Overall complexity of project design. Project implementation included five ministries and a number of agencies under those ministries.19 A number of consecutive Croatian Governments have had a history of weak coordination and despite the fact that the DPM and the Minister of Social Policy and Youth had overall responsibility for all those ministries, in reality the project was overly complex in design, which contributed to slow start and non-utilization of the PPA. While limiting a project to one ministry is not always feasible, it is important to carefully consider during preparation the number of stakeholders and the existing coordination mechanisms among them, and assess the possible communication and subordination issues that might affect project implementation. Projects with complex network of stakeholders need to include appropriate activities and mechanisms to address the institutional set up. Particularly in a politically volatile environment adopting institutional arrangements for ambitious reforms with multiple stakeholders is a challenge. Related to that, when designing a project that includes policy components, it should be taken into account that even higher-income EU countries may lack some specific capacities needed in the areas of reform. These needs should be analyzed in advance, and addressed in the project design.

• Streamlined implementation accountabilities and reform in place. Future design of a project with strong reform aspect should be based on reform activities that commence before the start of project implementation (as was the case with the CDCI, for example). Another aspect of the design risk relates to the fact that this was a mixed investment/results based instrument with a strong reform component (OSS), whereby the actual implementation of the reform started only at the stage of project implementation.

19 MSPY, MLPS, MPA, MOH, MOF, CPII, CHII, CES, CSW, CDCI and LI.


Page 23 of 59

Risk Assessment Lessons

• Stakeholder risks should be assessed more conservatively, with more mitigation measures put in place. There are two aspects that should be taken into consideration when assessing the stakeholder risks: (a) the capacity of the stakeholders that are responsible for implementation; and (b) the political readiness and power of the stakeholders that are responsible for pushing through policy reforms. The implementing stakeholders should possess specific capacity to perform the implementation tasks, and where capacity gaps are identified, respective capacity-building measures should be planned. On the political side, the policy reforms should have a clear institutional champion and sufficient political backup in various parts of the political spectrum (both the Government in power and the opposition). .


Page 24 of 59

.

ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS

A. RESULTS INDICATORS A.1 PDO Indicators

Indicator Name Unit of Measure Baseline Original Target Formally Revised

Target

Actual Achieved at Completion

One Stop Shop: Reduce private costs for social assistance beneficiaries and unemployed by 15% from baseline (of X%,month 2014)

Percentage 0.00 15.00 15.00 0.00

01-Jan-2014 31-Dec-2018 31-Dec-2018 15-Nov-2017

Comments (achievements against targets):


Target


Central Disability Certification Institute: Percentage of cases certified by IDCPREPD's full-time certifying officers

Percentage 0.00 90.00 90.00 77.00

01-Jan-2014 31-Dec-2018 31-Dec-2018 15-Nov-2017


Page 25 of 59

increased from 0% to 90%



Target


Reduced Error and Fraud: EFC risk-prone programs have strengthened information systems, and oversight and control procedures

Text Piecemeal, partial EFC actions aimed at a limited set of Social Protection Programs

EFC risk-prone programs have strengthened information systems, and oversight and control procedures

EFC risk-prone programs have strengthened information systems and oversight and control procedures

Piecemeal, partial EFC actions aimed at a limited set of Social Protection Programs

01-Jan-2014 31-Dec-2018 31-Dec-2018 15-Nov-2017



Target


De-institutionalization: Number of vulnerable children and adults moved from institutions to family-type care environments

Number 0.00 715.00 715.00 1374.00

01-Jan-2014 31-Dec-2018 31-Dec-2018 15-Nov-2017


Page 26 of 59



Target


Fiscal Savings realized Percentage 0.00 0.50 0.50 0.00

01-Jan-2014 31-Dec-2018 31-Dec-2018 15-Nov-2017


A.2 Intermediate Results Indicators

Component: Component 1: Improving Efficiency and Effectiveness of the Social Protection System


Target


Direct project beneficiaries Number 951690.00 951690.00 951690.00 951690.00

01-Jan-2014 31-Dec-2018 31-Dec-2018 15-Nov-2016

Female beneficiaries Percentage 475845.00 475845.00 475845.00 475845.00

01-Jan-2014 31-Dec-2018 31-Dec-2018 15-Nov-2016



Page 27 of 59


Target


DLIs 1-4: OSS in place Text GMB, Child Allowance, non-contributory Mat/Pat allowance and Birth Grants, and Unemployment Benefits administered through different networks

GMB, Child Allowance, non-contributory Mat/Pat allowance and Birth Grants, and Unemployment Benefits administered through the OSS

GMB, Child Allowance, non-contributory Mat/Pat allowance and Birth Grants, and Unemployment Benefits administered through the OSS

GMB, Child Allowance, non-contributory Mat/Pat allowance and Birth Grants, and Unemployment Benefits administered through different networks

01-Jan-2014 31-Dec-2018 31-Dec-2018 15-Nov-2017



Target


DLIs 5-6 CDCI operating independently

Text No CDCI CDCI operating independently

CDCI operating independently

CDCI established and operating

01-Jan-2014 31-Dec-2018 31-Dec-2018 15-Nov-2017



Target


DLIs 7-10 Reduced Error and Text Piecemeal, partial EFC EFC risk-prone EFC risk-prone Piecemeal, partial EFC


Page 28 of 59

Fraud actions aimed at a

limited set of Social Protection Programs

programs have strengthened information systems and oversight and control procedures

programs have strengthened information systems and oversight and control procedures

actions aimed at a limited set of Social Protection Programs

01-Jan-2014 31-Dec-2018 31-Dec-2018 15-Nov-2017



Target


DLIs 11-14: Deinstitutionalization

Number 0.00 715.00 715.00 1374.00

01-Jan-2014 31-Dec-2018 31-Dec-2018 15-Nov-2017


Component: Component 2: Investments and Technical Assistance to Support Improvements in Efficiency and Effectiveness of the Social Protection System


Target


DLIs 1-4: OSS in place Text GMB, Child Allowance, non-contributory Mat/Pat allowance and Birth Grants, and Unemployment

GMB, Child Allowance, non-contributory Mat/Pat allowance and Birth Grants, and Unemployment




Page 29 of 59

Benefits administered through different networks

Benefits administered through the OSS

Benefits administered through the OSS

Benefits administered through different networks

01-Jan-2014 31-Dec-2018 31-Dec-2018 15-Nov-2017



Target


DLIs 5-6 CDCI operating independently

Text No CDCI CDCI operating independently

CDCI operating independently

CDCI established and operating

01-Jan-2014 31-Dec-2018 31-Dec-2018 15-Nov-2017



Target


DLIs 7-10 Reduced Error and Fraud

Text Piecemeal, partial EFC actions aimed at a limited set of Social Protection Programs



Piecemeal, partial EFC actions aimed at a limited set of Social Protection Programs

01-Jan-2014 31-Dec-2018 31-Dec-2018 15-Nov-2017


Page 30 of 59



Target


DLIs 11-14: Deinstitutionalization

Number 0.00 715.00 715.00 1374.00

01-Jan-2014 31-Dec-2018 31-Dec-2018 15-Nov-2017



Target


Activation: Statistically assisted profiling introduced

Text No statistically assisted profiling of unemployed

Statistically assisted profiling introduced

Statistically assisted profiling introduced

Pilot is being implemented and evaluated.

01-Jan-2014 31-Dec-2018 31-Dec-2018 15-Nov-2017


Component: Component 3: Project Management Support


Page 31 of 59

B. KEY OUTPUTS BY COMPONENT

Objective/Outcome 1: Improve the Efficiency of the Social Protection System

Outcome Indicators

1. Reduce private costs for social assistance beneficiaries and unemployed by 15% from baseline (the PAD lists this one as an indicator of the effectiveness of the SP system. However, this ICR considers this indicator as an indicator of efficiency of the system, as it relates to the achievement of the system results while at the same time reducing the costs to the beneficiaries and the public.) 2. Percentage of cases certified by IDCPREPD’s fulltime certifying officers increased from 0% to at least 90% 3. EFC risk-prone programs have strengthened information systems, and oversight and control procedures including detection of error and fraud using risk-based investigation, data matching. 4. Cumulative fiscal savings of at least 0.5% of GDP

Intermediate Results Indicators

1. OSS in place: The administration of the GMB, child allowances, maternity/paternity/birth grant for noncontributory programs, and the unemployment benefit programs are transferred to the OSS. 2. CDCI operating independently including disability certifications processed centrally, unified disability certification procedures applied, regional office network, and independent IT system with interfaces to the OSS and other agencies. 3. EFC risk-prone programs have strengthened information systems, and oversight and control procedures including detection of error and fraud using risk-based investigation, data matching.

Key Outputs by Component (linked to the achievement of the Objective/Outcome 1)

1. Preparation of operational investment plan for development of the one-stop shop (OSS) 2. Design of the business processes and the business software of the Institute for Professional Rehabilitation and Disability Certification


Page 32 of 59

3. Social protection system’s unified transaction center - one-stop shop (OSS) implementation analysis 4. Various trainings for the CDCI 5. TAs for the establishment and for the design, developing, and implementation of management information system one-stop shop 6. Minor refurbishments for the CDCI

Objective/Outcome 2: Improve the Effectiveness of the Social Protection System

Outcome Indicators

1. Number of beneficiaries living in State and non-State Institutions or Family homes moved from institutions to family-type care environments.

Intermediate Results Indicators

1. De-institutionalization: Number of beneficiaries living in State and non- State Institutions or Family homes moved to family-type care environments 2. Activation: Statistically assisted profiling introduced 3. Direct project beneficiaries; of which female.

Key Outputs by Component (linked to the achievement of the Objective/Outcome 2)

1. Development, implementation, communication and preparation of scale up of statistically assisted profiling model; 2. Various TA to the Deinstitutionalization Operational Program implementing team; 3. TA for development of statistical profiling model.


Page 33 of 59

Detailed Outputs

Components/Activities Total (EUR)

Cancelled Loan Amount on July 28,

2017

Final Loan Amount

Achieved %

Disbursed Planned

Component 1: Improving Efficiency and Effectiveness of the Social Protection System (IBRD EUR 50 million)

4,000,000.00 50,000,000.00 46,000,000.00 4,000,000.00 100.00

Component 2: Investment and Technical Assistance to Support Improvements in Efficiency and Effectiveness of the Social Protection System (IBRD EUR 19.4 million)

1,833,849.95 19,400,000.00

Thematic Area 1: OSS 641,353.14 12,000,000.00 17,277,234.82 2,722,765.18 85.29

Thematic Area 2: CDCI 259,723.37 4,000,000.00

Thematic Area 3: EFC 335,357.34 2,250,000.00

Thematic Area 4: Deinstitutionalization 360,323.66 750,000.00

Thematic Area 5: Activation 237,092.44 400,000.00

Component 3 (IBRD EUR 0.6 million) *488,350.65a 600,000.00

Project Management *454,442.35a

Other (including study tour, training, conference) 33,908.30

Total Expenditure *6,322,200.61a 70,000,000.00 63,277,234.82 6,722,765.18 94.04

Note: a. The above amount will be increased by EUR 21,029.83 for the remaining eligible expenditure of the project (remaining audit cost).

Loan Account Balance (EUR) Project’s Operating Account

Balance (HRK) DA Balance in Loan

Currency (EUR) Note

Amount in currency 6,722,765.18 2,981,934.63 400,56457

Remaining acceptable costs

(156,550.00) (21,028.83) Audit cost Balance after final payment 6,722,765.18 2,825,384.63 379,535.74 -—


Page 34 of 59

Consultant Services - Firms

No. Thematic

Area Contract

Description Consultant's Name

Contract Signing

Date

Start Date of Services

Completion Date of Services

Contract End Date

Explanatory Notes

1 OSS Preparation of operational investment plan for development of the OSS (firm) (PPA)

JV: DELOITTE SAVJETODAVNE USLUGE d.o.o. and CONTRAST MANAGEMENT CONSULTING d.o.o.

11-25-2014 12-01-2014 12-01-2014 03-25-2017 IMPLEMENTED/ DISBURSED

2 CDCI Design of the business processes and the business software of the Institute for Professional Rehabilitation and Disability Certification (firm) (PPA)

JV: ERICSSON NIKOLA TESLA d.d., BBRZ AUSTRIA and INFODOM d.o.o.


3 Activation Development, implementation, communication, and preparation of scale-up of StAP

JV: SELECTIO LTD, TECHED CONSULTING SERVICE LTD AND INSTITUTE FOR LABOUR MARKET DEVELOPMENT


4 PIU Financial statements audit of the SPSMP

HLB INŽENJERSKI BIRO D.O.O.

06-15-2016 06-15-2016 15-06-2016 12-31-2018 ACTIVE


Page 35 of 59

No. Thematic

Area Contract

Description Consultant's Name

Contract Signing

Date

Start Date of Services


Contract End Date

Explanatory Notes

5 OSS SP system’s unified transaction center - OSS implementation analysis

DELOITTE SAVJETODAVNE USLUGE d.o.o.

09-01-2016 09-01-2016 09-01-2016 03-31-2017 Following 4th session SPSMP Steering Committee decision to cancel project’s thematic area: (a) Consolidation of social benefit administration and simplification of procedures. IMPLEMENTED/ DISBURSED

Note: SPSMP = Social Protection System Modernization Project.

Consultant Services - Individuals

No. Thematic Area Contract Description Contract

Signing Date Start Date of

Services


Explanatory Notes

1 Deinstitutionalization TA to the deinstitutionalization operational program implementing team—deinstitutionalization and transformation process planning and coordination adviser (PCA)(PPA phase 1 + phase 2 - loan)

02-09-2015 02-16-2015 12-31-2018 Contract cancelled on 30.06.2017, following 4th session SPSMP Steering Committee decision to cancel project's thematic area: (a) Deinstitutionalization of vulnerable children and adults and related loan activities in this thematic area. DISBURSED


Page 36 of 59



Services


Explanatory Notes

2 Deinstitutionalization TA to the deinstitutionalization operational program implementing team—deinstitutionalization and transformation financial planning and coordination adviser (FPCA)(PPA phase 1 + phase 2 - loan)


3 Deinstitutionalization TA to the deinstitutionalization operational program implementing team—deinstitutionalization and transformation process foster care adviser (FCA)(PPA phase 1 + phase 2 - loan)


4 PIU TA for PIU-financial management assistant (FMA)-phase1 + phase2

11-10-2014 11-17-2014 12-31-2018 Contract cancelled on 31.11.2017, pursuant to new project closing date. DISBURSED


Page 37 of 59



Services


Explanatory Notes

5 PIU TA for PIU-procurement assistant (PA)-phase1 + phase2

11-10-2014 11-17-2014 12-31-2018 Contract cancelled on 30.07.2017, due to change of consultant’s employment status and inability to fulfill obligations as per contract. DISBURSED

6 CDCI Training in unified disability certification methodology-education and training adviser for the CDCI: Condition after transplantation of solid organs (kidney, liver, pancreas) and work capacity (the specifics of the assessment of patients with solid organ transplants)

11-03-2015 11-06-2015 11-07-2015 IMPLEMENTED/ DISBURSED

7 CDCI Training in unified disability certification methodology-education and training adviser for CDCI: Children with developmental disabilities and children with traumatic experiences specific estimation


8 CDCI Training in unified disability certification methodology-education and training adviser for CDCI: Educational rehabilitation role and competences in CDCI processes


9 CDCI Training in unified disability certification methodology-



Page 38 of 59



Services


Explanatory Notes

education and training adviser for CDCI: Cancer patients work capacity

10 OSS TA—OSS establishment, implementation, and education supervisor


11 OSS TA for the design, developing and implementation of management information system one-stop shop (MISSOS)—development of technical specifications, evaluation of bids, and MISSOS implementation supervision IT adviser

09-14-2015 09-14-2015 02-10-2016 Contract cancelled on 10.02.2016, due to change of consultant’s employment status and inability to fulfill obligations as per contract. DISBURSED

12 OSS TA for the design, developing, and implementation of management information system one stop shop (MISSOS)—development of technical specifications, evaluation of bids, and MISSOS implementation supervision IT adviser

07-26-2016 07-26-2016 04-30-2018 Contract cancelled on 30.06.2017, following 4th session SPSMP Steering Committee decision to cancel project's thematic area: (a) Consolidation of social benefit administration and simplification of procedures. DISBURSED

13 Activation TA for development of StAP model: Calibration, communication, and revision of statistical predictive model for profiling



Page 39 of 59



Services


Explanatory Notes

14 Activation TA for development of StAP model: Expert validation and support of statistical predictive model for profiling


15 Activation TA for development of StAP model: Impact evaluation of StAP


16 Deinstitutionalization TA for the implementation of the deinstitutionalization and transformation process: EU funds adviser (EUFA)

08-17-2015 08-17-2015 06-30-2018 Contract cancelled on 30.06.2017, following 4th session SPSMP Steering Committee decision to cancel project's thematic area: (a) Deinstitutionalization of vulnerable children and adults and related Loan activities in this thematic area. DISBURSED

17 PIU TA for PIU-financial management adviser

05-15-2015 05-18-2015 12-31-2018 Contract cancelled on 31.11.2017, pursuant to new project closing date. DISBURSED

18 OSS/CDCI TA for OSS/CDCI facilities design adviser (FDA) for renovation of the office space for PAOs and for IDCPREPD


19 PIU TA for PIU-TA 05-15-2015 05-18-2015 12-31-2018 Contract cancelled on 31.11.2017, pursuant to new project closing date. DISBURSED


Page 40 of 59



Services


Explanatory Notes

20 PIU TA for PIU-translation assistant 05-15-2015 05-18-2015 12-31-2018 Contract cancelled on 31.11.2017, pursuant to new project closing date. DISBURSED

Goods

No. Thematic Area Contract Description Contract Signing

Date

Contractual Delivery

Date

Estimated Delivery

Date

Explanatory Notes

1 OSS/CDCI IT equipment for the first phase of OSS development (firm) + IT equipment for initial operations and piloting of the new IPRDCED’s information system (firm) (PPA)


2 CDCI/PIU Procurement of IT equipment (personal computers) for CDCI and PIU


3 EFC Procurement of the IT equipment for LI 11-04-2015 03-04-2016 03-04-2016 IMPLEMENTED/ DISBURSED

4 Activation Upgrade of management information system (MIS) to integrate profiling and monitoring dashboard



Page 41 of 59

ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION

A. TASK TEAM MEMBERS

Name Role

Preparation

Emil Tesliuc, Ivan Drabek Task Team Leader(s)

Penny Williams Team Member

Zoran Anusic Team Member

Alessandra Marini Team Member

Antonia G. Viyachka Procurement Specialist

Lamija Marijanovic Financial Management Specialist

Vera Dugandzic Social Safegaurds Specialist

Natasa Vetma Environmental Safeguards Specialist

Supervision/ICR

Ivan Drabek, Zoran Anusic Task Team Leader(s)

Antonia G. Viyachka Procurement Specialist

Lamija Marijanovic Financial Management Specialist

Vera Dugandzic Social Safeguards Specialist

Natasa Vetma Environmental Safeguards Specialist

Abla Safir Team Member

Ruzica Jugovic Team Member

B. STAFF TIME AND COST

Stage of Project Cycle Staff Time and Cost

No. of staff weeks US$ (including travel and consultant costs)

Preparation


Page 42 of 59

FY13 2.500 56,038.17

FY14 34.372 223,564.08

FY15 23.422 130,613.19

FY16 7.048 47,186.69

FY18 .100 205.96

Total 67.44 457,608.09

Supervision/ICR

FY13 1.700 8,253.76

FY14 .430 3,224.40

FY15 15.198 84,543.07

FY16 15.779 41,844.70

FY17 29.342 74,293.34

FY18 15.652 64,865.13

Total 78.10 277,024.40


Page 43 of 59

ANNEX 3. PROJECT COST BY COMPONENT

Components Amount at Approval

(EUR, millions)

Actual at Project Closing (EUR,

millions)

Percentage of Approval (%)

Improving efficiency and effectiveness of the social protection system

50.00 4.00 8.00

Investments and Technical Assistance to support improvements in efficiency and effectiveness of the social protection system

19.40 1.83 9.40

Project Management Support 0.60 0.49 81.70

Total 70.00 6.32 9.03


Page 44 of 59

ANNEX 4. EFFICIENCY ANALYSIS

I. Original Economic Analysis: Update Project’s development impact 1. Improved targeting of needs-based social assistance programs. The project was expected to complement the parametric reforms led by the Second Economic Recovery Development Policy Loan and supported by the NRP aimed to improve the targeting of selected cash benefits (child allowance program) and policies (for example, the child tax allowance). The impact of the parametric reforms on improved targeting depended both on the parametric reforms being legislated, and the implementation arrangements (in particular, on compliance with program rules). International studies suggested that the contribution of the two factors to the final outcome was comparable in size. The project was designed to support a number of administrative reforms to complement and expand the impact of the parametric reforms, such as introduction of the OSS (expected to help prevent erroneous claims from being processed), development of the system to combat EFC (to reduce the incidence of erroneous or fraudulent claims in disability pensions, the social assistance programs to be administered under the OSS, and the instances of shadow labor relationships) and the support to the CES to strengthen activation of the most vulnerable inactive and unemployed, including SP beneficiaries. A simulation of the introduction of means-tests for the child allowance program indicated a possible reduction in the number of beneficiaries by 17 percent. However, the dropping of the OSS and the non-introduction of strict means testing in child allowances prevented the expected economic result. 2. Increased take-up and coverage of social assistance programs. The introduction of the OSS was expected to reduce exclusion error and improve program take-up. This would be triggered by three factors: better information for applicants, lower private costs of application and recertification, and higher benefit take-up. As key cash benefits were supposed to be processed in one place, the claimants would have been better informed on the full array of benefits they were entitled to, thus reducing the exclusion error due to lack of information on available benefit programs and their eligibility criteria. Second, the OSS was expected to reduce the overall private costs for applicants, and this could have had a significant role in improving take-up. Third, the trip to the OSS would have been less costly and more worthwhile for beneficiaries of multiple benefits as the total sum to be cashed in would have increased, while the private application costs would have stayed constant. International comparisons suggested that the second and third factor determined the low take up for means-tested programs, often as low as 40–60 percent of the total eligible beneficiaries. The large segments of potential beneficiaries, who do not apply, do so because the cash benefit they could receive is too low compared to the cost of application. These measures could have increased the number of beneficiaries of social welfare programs and the budget for such programs, with most of this budget accruing to the poor and vulnerable beneficiaries, and contributing to a further reduction in poverty rate. A micro-simulation model of program take-up suggested that the coverage of the child allowance and GMB programs could have increased gradually, by up to 4 percent and 35 percent, respectively, by 2018, through the introduction of the OSS. This would have possibly resulted in a gradual increase of the child allowance and GMB program budget by up to 3.5 percent and 38 percent in 2018. However, the failure to implement the OSS invalidated all the expected efficiency benefits described in the original economic analysis from the time of appraisal. 3. Greater horizontal equity in disability certification, medium-term administrative costs savings and


Page 45 of 59

stronger gate-keeping of new claims. The introduction of harmonized disability certification criteria and procedures were expected to introduce horizontal equity across different categories and diagnoses. In addition, the introduction of a single disability certification unit and criteria should have consolidated the various certification bodies under one institution generating administrative costs savings in the process. Rolling out of the pension model across all disability programs (from about 50 percent of the caseload in the present to at least 90 percent after harmonization) could have further reduced error and fraud in disability allowance programs. In particular, stronger gate-keeping through harmonized certification could bring fiscal savings of up to 0.2 percent of GDP over the project period, ceteris paribus. The implementation progress in this area was limited, therefore leading to modest results compared to the original expectations. The MLPS focused predominantly on funding office space renovation, and the remaining CDCI activities (business process implementation, training) were taken off the project list, and remained to possibly be financed through EU funds. 4. Fiscal savings from reduced EFC rate. The development of a modern, comprehensive system to control EFC in the SP benefit stream was expected to improve compliance and generate fiscal savings estimated at up to 0.5 percent of GDP for the duration of the project (over a four-year period). This was seen to make the SP system less invasive for the clients (as only clients with high risk of EFC would be investigated, and many of the investigative techniques such as data matching or other type of intelligence gathering would not be invasive) and would create trust that the SP ministries are able to transfer cash benefits to the right beneficiary, at the right time and in the right amount, and will minimize the waste of public resources. The small progress on EFC under the project, however, has prevented these results from materializing in full. 5. Overall cost savings under the project. Over the four years of its implementation, the project was expected to generate cumulative fiscal savings of at least 0.5 percent of GDP. The savings were to be generated by the introduction of stronger gate-keeping in the CDCI and the strengthening of EFC, and were expected to reduce spending by at least 0.7 percent of GDP (0.2 percent from CDCI gate keeping, and 0.5 percent from the overall EFC activities), that would compensate an estimated increase in costs of about 0.2 percent of GDP due to the increase in coverage of the GMB and child allowances and the administrative costs involved in setting up the OSS, CDCI, EFC and deinstitutionalization. As a result of the limited project implementation in the above areas, these projected outcomes were not achieved. 6. Social inclusion of children and disabled adults currently in institutions. The project was planned to help the Croatian Government use EU funds to finance the cost of de-institutionalization of about 715 children without parental care, children and adults with disabilities, and mentally ill adults. In the long run, this was expected to lead to savings as some institutions could be closed while the de-institutionalized individuals require lower staff/beneficiary ratios, lead more independent lives and adults may contribute to the economy through employment. During the project life, more than 1300 individuals were deinstitutionalized, but no institutions were closed and in the majority of cases new individuals were accepted in the institutions to replace the ones who left. Therefore, the economic effect of the deinstitutionalization is negligible. 7. Facilitating activation of out-of-work social assistance beneficiaries. The project had a relatively successful contribution to the design and implementation of activities that facilitate the activation of the out of work, including social assistance beneficiaries to resume employment, part of full time, in combination with social assistance support or not. The activities contributed to better prioritization and targeting of active labor market programs to those who have the highest probability of becoming long-term unemployed. This, in the long term, will have a positive efficiency impact by reducing the number of social assistance beneficiaries, helping


Page 46 of 59

discouraged unemployed, and better matching people to jobs thus ensuring higher economic efficiency of the labor market. 8. The project planned to leverage EU funds through support for preparation of European fund project proposals in the deinstitutionalization area and through the activation work. As such, the project was expected to have a positive fiscal impact by helping the Government to generate non-deficit creating spending/investments. Some progress was made in this area. Three institutions advanced the transformation processes, had their transformation plans approved, and successfully signed their project proposals for financing by the EU funds in the amount of HRK 50 million (approximately EUR 6.7 million). Estimated Poverty Reduction Impact 9. Simulated poverty reduction impact of the OSS. The poverty headcount was expected to fall from an estimated 14.4 percent before the introduction of the OSS to 14 percent after the introduction of the OSS. However, the OSS was not introduced and any expected poverty impacts from the OSS did not materialize. II. Original Component Costs and Administrative Costs 10. As shown in annex 3, and as explained in detail in the main text of the ICR, the financing for practically all components of the project was drastically reduced. The component that was planned to provide the bulk of financing – Component 1 (Improving efficiency and effectiveness of the social protection system) – was reduced from EUR 50 million to EUR 4 million because of the early closure, cancellation and inability of the project to meet the disbursement requirements from all but two of the DLIs. Component 2 that was supposed to provide investments and TA for meeting the DLIs was also cut from EUR 19.40 million to EUR 1.83 million. These reductions inevitably adversely affected the expected efficiency gains from the project. These two substantive components disbursed cumulatively only 8.4 percent of the initially planned (at appraisal) financing. At the same time, the component for project management spent EUR 0.49 million of planned EUR 0.6 million, which is more than 80 percent of the planned management cost at appraisal. Proportional to the disbursement of the substantive financing, the project was supposed to have EUR 12,000 of management costs for every million euro disbursed, while in reality the project had more than EUR 84,000 of administrative expenses for every million of substantive financing utilized. This makes the project management approximately seven times less efficient in administrative terms than planned. III. Cancellation of a Part of the Loan Amount

11. As explained in the sections on outcome evaluation of this ICR, a significant portion of the loan was cancelled—more than 90 percent of the original loan amount. This cancellation affected mainly the substantive Components 1 and 2, thus leading to the inability of the project to fully meet its development objectives. While the impact was significantly reduced, the project management cost remained quite high and the sunk costs for project preparation cannot be recovered or compensated.


Page 47 of 59

ANNEX 5. BORROWER COMPLETION NOTE

This annex contains the Government of Croatia Completion Note provided to the World Bank on May 8, 2018. The Completion Note has been translated from Croatian, and is hereby presented in its full, unabbreviated version.

Contribution to the ICR

Social Protection System Modernization Project, Loan No. IBRD8426-HR

Following the Act on the Ratification of the Loan Agreement between the Republic of Croatia and the International Bank for Reconstruction and Development (IBRD) for the Social Protection System Modernization Project (Official Gazette, International Agreements, 8/2014) of December 19, 2014, the Loan Agreement for the Social Protection System Modernization Project, IBRD Loan No. 8426-HR, signed on September 25, 2014, shall come into effect. The primary objective of the Social Protection System Modernization Project (the Project) was to improve the efficiency and cost effectiveness of Croatia’s social protection system. For that purpose, the Republic of Croatia received a loan for the Project in the amount of EUR 70 million. The Project aimed at supporting the reforms of the Croatian Government regarding the social protection system, with a particular focus on five thematic areas: (a) consolidating social assistance cash benefits administration through a single delivery network and procedure simplification; (b) unifying and harmonizing the certification of disability; (c) reducing error, abuse, and corruption; (d) deinstitutionalization of children and adults; and (e) activation of social protection beneficiaries in the work market.

On the basis of accomplishing 14 disbursement-linked indicators (DLIs) defined under the first four thematic areas, a withdrawal of funds from the loan to the state budget had been planned up to the total amount of EUR 50 million, within the scope of Component 1. Within the scope of Component 2 of the Project, investments and technical assistance to support improvements and cost effectiveness of the social protection system had been planned in the total amount of EUR 19.4 million. Along with the activities related to the four thematic areas, this component also ensured support for the fifth thematic area—activation in the work market. For Project management, EUR 0.6 million had been planned within the scope of Component 2. The initial loan closing date was December 31, 2018. Pursuant to the Amendment to the Loan Agreement of November 22, 2017, the closing date for all Project activities for the remaining components is November 30, 2017. Until the loan closing date, 9.60 percent of the loan (of 14.16 percent that had been stipulated in the Loan Agreement) had been used exclusively for investments and technical assistance (Component 2). During the Project, only two indicators within the scope of Component 1, on the basis of which the withdrawal of funds from the loan is approved (DLI), had been achieved. Thus, in May 2015, the terms and conditions for the withdrawal of the total amount of EUR 4 million had been met, while the said amount was paid to the state budget in May 2016. Apart from the two DLIs mentioned above, no other DLI had been achieved until the loan closing date.

Loan recipients and thematic area managers were Ministry of Demography, Family, Youth and Social Policy20 (MDFYSP), Ministry of Public Administration, Ministry of Labour and Pension System (MLPS),

20 Pursuant to Article 2 of the Act on the Organization and Scope of Ministries and Other Central Government Bodies, Official Gazette No. 93/2016, the Ministry of Social Policy and Youth had been renamed the Ministry of Demography, Family, Youth, and Social Policy.


Page 48 of 59

Institute for Expert Evaluation, Professional Rehabilitation and Employment of Persons with Disabilities (ZVPRZOI), and the Croatian Employment Service (CES). Other institutions involved were: Croatian Pension Insurance Institute (CPII) and the Croatian Health Insurance Institute (CHII).

Overview of Project activities/conclusions by the institutions involved per thematic area is presented in the following paragraphs.

1. Single delivery network - SDN (Beneficiary: Ministry of Public Administration)

One of the nine priorities of the Croatian Government’s strategy for long-term fiscal consolidation reforms, in accordance with the NRP,21 was to consolidate the processing of cash benefits for social welfare programs, family support programs, and unemployment benefits in an SDN, a one-stop shop (OSS).

Following the parliamentary elections in 2015, the new National Reform Program (NRP)22 established a need for redefining the proposal on the structure of the SDN. Also, since the Ministry of Public Administration implemented and planned several projects for the development and reform of public administration (establishing single service points, introducing e-signatures, e-business, and so on), along with the problems of poorly suited infrastructure of public administration offices (PAOs), especially with regard to office space and the lack of human resources, the Croatian Government arrived at a new proposal for the concept and the relocation of the SDN from PAOs.

In late 2016 and early 2017, by introducing the new NRP, the Croatian Government decided to abandon the concept of the consolidation of social assistance cash benefits administration and procedure simplification within the scope of the SDN system of social protection, as initially planned within the Project, and to pursue the approach that would keep the administration unchanged while focusing exclusively on strengthening the ties between existing information systems for different social benefits. This new course is reflected in the most recent NRP.23 At a meeting held on April 27, 2017, the Project’s Management Board unanimously decided to support the decision of the MDOMSP to cancel all activities within two thematic areas, the establishment of the SDN and deinstitutionalization, which had been carried out according to the initial Project draft. This decision of the MDFYSP also established a new course for administration simplification and for better control over benefits, in line with the NRP, according to which, with the help of European Union (EU) legal experts and EU funds, benefits would be consolidated and the existing information system SocSkrb would be improved while physically locating the SDN, as the initial model planned would be abandoned.

Key activities planned/implemented through the Project

In this thematic area, the loan was used to finance the following activities of the Project for investments and technical assistance and education: technical assistance for the preparation and design of investment and action plans for the development of the social protection system SDN/OSS; technical assistance for monitoring the training process, implementing and establishing the SDN; technical assistance for monitoring the design, development, and implementation of the information system for managing the

21 National Reform Program, April 2014. 22 National Reform Program, April 2016. 23 National Reform Program, April 2017.


Page 49 of 59

administration and payment of benefits at the SDN; implementation analysis of the SDN social protection system - OSS; Office Space Design Counsellor within the network of the PAOs for establishing the SDN and the ZVPRZOI for establishing the single expert evaluation body (SEEB); procurement of IT equipment for the PAOs for phase one of SDN development.

Conclusion

The Ministry of Public Administration finds the following positive effects of the Project:

• All prerequisites for establishing the SDN have been defined.

o A detailed independent analysis has been performed regarding the current status of implementation of selected social benefits (Guaranteed Minimum Benefit [GMB], maternity benefits, CA, and UB) from the perspective of their respective institutions (CSS, CPII, CHII, and CES), PAOs, and FINA, including drafting the summary list of the cash benefits administration processes and determining the status of all resources involved (human resources, infrastructure, and the IT system).

o Three concept models for establishing the SDN have been specified and developed (traditional model, e-services model, and the hybrid model).

o Financial parameters required for implementing the SDN concept have been quantified.

• The developed methodology and approach may have a long-term effect because they represent a starting point for the development of the single service point concept as the ultimate objective for improving the efficiency of public services.

The Ministry of Public Administration finds a negative effect in the failure to finish the Project (implementation), which demotivates all involved stakeholders, especially the employees of the institutions involved in the Project who were hoping to improve the effectiveness of their work, relieving them of the burden of managing the financial segment of benefits, as had been planned.

The ministry still plans the single service point project, which would bring public administration services to places of residence and work of citizens and companies, the implementation of which is expected within three years. People would receive services from PAOs, FINA, and post offices, which will receive cases and forward them to competent agencies. HRK 113.8 million was planned for the establishment of the system and the Project itself, of which up to 85 percent would be grants from the European Social Fund (ESF). IT equipment procured within the Project, used in PAOs, will be used in the implementation of the single service point project.

As an institution involved in this Project area, the CHII points out that the proposal to consolidate the exercising of cash benefits rights, which are disbursed from the state budget, into the social protection system is still an extremely useful measure. Establishing a single administrative body that would take over the task of managing the rights to cash benefits, which are now managed by several competent institutions, would enable beneficiaries to exercise their rights in a much simpler and more accessible


Page 50 of 59

way. Also, the CHII finds that this would contribute to a more significant degree of data control and monitoring on all levels, which is also important. The CHII would thus continue to manage only the benefits disbursed from the compulsory health insurance fund, which would significantly facilitate administration as well as financial management. Consequently, even though this thematic area had been abandoned in the Project, it finds the potential effort to achieve initial objectives as positive.

2. Standardization and harmonization of the expert evaluation process - SEEB (Beneficiary: ZVPRZOSI)

The ZVPRZOI (hereinafter referred to as the Institute) was established by the Act on Professional Rehabilitation and Employment of Persons with Disabilities (OG 157/13 and 52/14) and the Act on the Single Expert Evaluation Body (OG 85/14). The Institute started its operation on February 7, 2014. The Institute is the legal successor of the Fund for Professional Rehabilitation and Employment of Persons with Disabilities that operated from its establishment in 2007 until February 14, 2014.

The Institute became fully functional in all three areas of operation on January 1, 2015.


In this thematic area, the loan was used to finance the following activities of the Project for investments and technical assistance (Component 2) and education: technical assistance for defining business processes to be applied in the SEEB application; technical assistance for the education and training activities of the SEEB (four consultants)—specificities of the evaluation of patients with transplanted solid organs, specificities of the evaluation of children presenting developmental disabilities and children that have experienced trauma, the role and competencies of education rehabilitators in the expert evaluation process, and work capacity of oncology patients; and procurement of IT equipment for the ZVPRZOSI.

Other activities of the ZVPRZOSI

Various activities were carried out during the previous period from January 1, 2015, to November 30, 2017, first with regard to the organization of periodic workshops based on issues encountered during business activities, that is, during the expert evaluation process. The primary education of new expert evaluators is an important factor for further operation because a primary lack of medical doctors in the Republic of Croatia forced the Institute hire independent contractors who are not primarily employed as expert evaluators. To enable expert evaluation in all systems, the Expert Evaluation Sector carried out education activities from its very inception, such as the following:

(a) A seminar on the rights to welfare (a category 2 course) was held on February 13, 2015, which consisted of lectures and work simulations in groups divided by themes.

(b) Another workshop was held on May 29, 2015, on harmonizing criteria when adopting findings and opinions, during which a working group answered previously received questions posed by expert evaluators, followed by all field offices giving presentations regarding interesting (complex) cases.

(c) A consultation and education of expert evaluators in the field of medicine was held from November 5 to 7, 2015, in Lovran, financed through the Project’s resources. It covered four


Page 51 of 59

topics: (i) specificities of the evaluation of children presenting developmental disabilities and children that have experienced trauma; (ii) work capacity of oncology patients; (iii) condition of patients following transplantation of solid organs (kidney, liver, and pancreas) and their work capacity; and (iv) the role and competencies of education rehabilitators in the expert evaluation process.

(d) BBRZ organized a study visit to Linz, with the participation of representatives from field offices, during which the working of the Austrian model of professional rehabilitation was shown as well as the possibilities of professional rehabilitation.

(e) A workshop was held on February 5, 2016, on professional rehabilitation and expert evaluation of functional capacities.

(f) A workshop was held on November 10, 2017, on (i) harmonizing the criteria of medical and functional expert evaluation, (ii) operation in accordance with the new regulation on expert evaluation methodologies—detected shortcomings and guidelines presented, (iii) MIA application; and (iv) the criteria for referring a patient for treatment to the Professional Rehabilitation Centre.

The application for expert evaluation has been implemented in the central office in late 2015 (it concerns dealing with complaints and revising cases involving war veterans).

The use of the application on the first level started gradually in field offices and separate places of work as of June 2017. As of November 2017, the application is being used in all organizational units of the Institute.

Conclusion

Through implementing the activities from Thematic Area 2 regarding the standardization and harmonization of the expert evaluation process, an SEEB has been established as an organizational unit of the ZVPRZOI, that is, the Expert Evaluation Sector.

The expert evaluation as such is carried out pursuant to the Regulation on Expert Evaluation Methodologies (this regulation came into force on July 20, 2017, Official Gazette No. 67/2017).

The Regulation on Expert Evaluation Methodologies has been amended once (October 15, 2015) for its harmonization with other legal acts, and in July 2017 (July 20, 2017), a new Regulation on Expert Evaluation Methodologies came into force. The new regulation is still aimed at closing the gaps between different expert evaluation methodologies and achieving the primary objective of an SEEB, that is, for the expert evaluation process to be carried out following a uniform methodology within the SEEB and enabling users to exercise their rights in various systems (rights in the area of welfare, pension insurance, education, professional rehabilitation, and employment of persons with disabilities; rights based on maternity and parental subsidies; and rights regarding the protection of military and civilian victims of war) based on a single document with findings and opinions.


Page 52 of 59

The aforementioned activities have been precisely defined through the Project and deadlines have been set for their implementation, while the implementation as such continued even after the Project ended. The Project had a positive impact in every way, and it should be noted that the SEEB Project is a national project subject to mandatory improvements by the Institute because the most vulnerable members of society depend on it.

The fundamental goal of this project and the future work of the Institute is for a single document with findings and opinions to be sufficient to exercise rights in all systems. To achieve this, investments in IT components should continue, as well as the exchange of data with cooperating systems, and the continuous harmonization of legislation.

3. Prevention of error, fraud, and corruption (EFC) (Beneficiary: MLPS/MDFYSP)

Following the Decision of the Government of the Republic of Croatia, class: 022-03/15-04/506, file No.: 50301-04/12-15-2 of November 5, 2015, by which the strategy for the reduction of EFC in the area of social protection in the Republic of Croatia was adopted for the period 2015–2020 (the Strategy), members of the working group from the MLPS, Labor Inspectorate of the MLPS, independent sector for audit and supervision of medical expert evaluations of the MLPS, CDCI, CPII, and MDFYSP participated in the development of the final proposal of the action plan. The action plan was made public in the e-consultancy procedure organized for the interested public in November/December 2016. The action plan established the proposed deadlines for the implementation of activities, sources of funding/co-funding, and means of monitoring and performance indicators. All submitted comments were reviewed and statements were sent for drafting a report, which was planned to be published in February 2017. Due to parliamentary elections and the induction of a new government in the first quarter of 2017, it was impossible to meet certain requirements. Because the action plan was not adopted and approved by the Government of the Republic of Croatia, there was no legal basis to start intensive work on the realization of activities from the action plan.


In this thematic area, the loan was used to finance the following activities of the Project for investments and technical assistance and education: the procurement of IT equipment for the Labor Inspectorate for the support of EFC prevention activities.

As part of the Project, representatives of the involved institutions participated in presentations, workshops, meetings, and discussions, all of which focused on the issue of misspending state budget funds. Officials in charge of carrying out supervision in the social protection system became aware that they should focus more on material issues during supervision, instead of focusing on beneficiary’s rights protection and the quality of offered services, as used to be the case.

Conclusion

It has become well known and recognized that the service in charge of supervision in the social welfare system of the MDFYSP is understaffed, so the new restructuring carried out in September 2017 saw the appointment of several officials to the Supervision and Petitions Sector, of which 10 are inspectors, 7 conduct administrative supervision of social welfare centres, and 7 work on petitions and complaints filed


Page 53 of 59

by beneficiaries and other citizens. The process of resolving petitions now makes it possible to get access to documents pertaining to each individual beneficiary in the system, thus enabling the control of recognized rights and warning social welfare centers in case it is established that the procedure for ensuring their rights has not been followed. Administrative supervision or inspection will be initiated if errors keep occurring in certain institutions. It is believed that the area of error and abuse prevention is a challenge that has to be faced in the future in the social welfare system, in particular, and all other systems ensuring social protection in general.

The Labor Inspectorate of the MLPS believes that the Project has had a positive impact because it enabled the procurement of IT equipment that facilitated the work of inspectors. In addition, numerous meetings were held with representatives of other government authorities on the topic of information exchange, and it was precisely established which data other government bodies should use to carry out a more effective supervision of undeclared work. Following these meetings, the Labor Inspectorate has developed a plan for linking its database with those of other government bodies, taking into account the available funds.

Even before taking part in the Social Protection System Modernization Project as one of the key managers in Thematic Area 3, the CPII conducted important activities and achieved important results in the area of error and abuse prevention. Namely, since 2010 the CPII has been continuously implementing the Project for the prevention of unlawful payments, along with other activities, such as exchanging information with the Ministry of Public Administration to report deaths, and so on. All of this has produced significant results in this area. However, even though the Project has not been implemented, the CPII’s participation in it has been a very positive experience because contact with other institutions has been established and ideas, activities, practices, and results obtained in the area of error and abuse prevention with other systems and states have been exchanged, a part of which will surely be implemented in the future, regardless of the Project having ended. Despite the Project’s end, the CPII will continue to plan and conduct activities for the prevention of unlawful payments, that is, errors and abuse. It is found that it would be particularly useful in the long run to link the databases of other institutions to compare data and establish new criteria for benchmarking and control to detect errors and abuse.

The positive outcomes of the preparation of the strategy and action plan arise from the problem analysis itself, the achieved cooperation, and the linking of various different systems. Based on this, further cooperation between the institutions in this area can be expected.

4. Deinstitutionalization of children and adults (Beneficiary: MDFYSP)

The goal of this Project area was to reduce the number of children and adults in state and non-state homes and family homes and to provide them with care in the community, biological, and foster families.


In this thematic area the loan was used to finance the following activities of the Project for investments and technical assistance (Component 2) and education: technical assistance for implementing the operational plan for the deinstitutionalization of beneficiaries of social welfare homes (three advisers)— one adviser for planning and coordination of the deinstitutionalization and transformation process, one adviser for financial planning and coordination of the deinstitutionalization and transformation process,


Page 54 of 59

one foster care adviser within the deinstitutionalization and transformation process; technical assistance/adviser for EU funds in the deinstitutionalization and transformation process.

Conclusion

The Project contributed to the intensification of the process of deinstitutionalizing persons with disabilities and their integration into the community. The organized housing service with assistance is currently being used by 647 beneficiaries who were deinstitutionalized during the previous five-year period. By the time the Project was closed, a total of 717 persons with disabilities and a total of 839 children and young people without adequate parental care as well as children and young people with behavioral problems were deinstitutionalized.

Examples of good practice in this process and the provision of additional resources in the ESF and the European Regional Development Fund (ERDF) ensured the sustainability of the deinstitutionalization process and its continuation. To intensify the deinstitutionalization process, both the ESF and the ERDF provided the funds that are considered sufficient for the process to be carried out properly. The basis was the operational plan for the deinstitutionalization and transformation of social welfare homes and other legal entities performing social care activities in the Republic of Croatia 2014—2016. About 32 homes established by the Republic of Croatia were included in the operational plan because a practical example was required to show that the transformation and deinstitutionalization process was possible and that it was efficient and that the services were ultimately more cost-efficient if the process is implemented and completed in a useful and desirable manner. This also includes vacating certain real estate that is costly and the maintenance of which is demanding and in turn organizing the beneficiaries’ lives in smaller housing communities. This required continuous and coordinated action in several areas: harmonizing the activity planning and funding/state budget and EU funds; harmonizing the legal regulations and also expanding the community service network. The MDFYSP shall include all social service providers into the process by adopting a new operational plan for 2017–2020 to ensure the goals defined by the previous operational plan are achieved and as many beneficiaries as possible are deinstitutionalized, which will, in turn, enable them to exercise the fundamental human right to live in a community. For the program period 2014–2020, the ERDF has provided funds in the amount of EUR 66,470,588.24 for persons with disabilities, 15 percent of which is financed by the state budget, and 85 percent through the fund. EUR 17,647,058.82 has been provided from the same fund for children and young people, and the same amount has been provided for the CSS as important stakeholders in the process. The ERDF has provided funds in the amount of EUR 101,764,705.90 in total. Furthermore, the ESF has provided EUR 35,294,117.65 for ensuring the staff required for the provision of services in the community for persons with disabilities, while the total funds from the ESF, including funds provided for children and young people, amount to EUR 51,764,705.89. Institutions have already withdrawn a portion of the funds in an amount exceeding HRK 60 million, of which more than HRK 50 million are intended for homes for persons with disabilities. The tenders are permanently open, the institutions submit their applications when they are ready, and the applications need to be compatible and intended solely for the purpose of the transformation and deinstitutionalization process.

5. Activation in the work market (Beneficiary: CES)


Page 55 of 59

The Project supported the enhancement of CES’s efficiency so that it could serve its users and promote the activation in the work market, especially for sensitive members of the population.

The statistically assisted profiling (StAP) should help the CES direct its services and interventions considering the needs of unemployed persons as early as possible during their unemployment period, aiming to harmonize the criteria used for categorizing unemployed persons taking into account their needs, that is, aiming to diminish the differences in the categorization and treatment of unemployed persons considering the different discretionary assessments made by different counsellors.


The loan was used to finance the following activities of the Project for investments and technical assistance (Component 2) and education: technical assistance for calibrating, communicating and reviewing the StAP model for the CES; technical assistance for developing, implementing, communicating, and preparing the extension of the StAP model for the CES; technical assistance for peer evaluation and support to the StAP model; technical assistance for the impact assessment of the StAP model; upgrade service of the management information system (MIS) for the integration of StAP and the development of a web application for monitoring.

The StAP is based on a tool that estimates the probability of employment in the next 12 months for newly registered unemployed persons and categorizes newly registered unemployed persons into risk groups based on estimated probability of employment. The integral part of the StAP is a set of recommendations on conducting individual consultations and defining a professional plan, taking into account the segmentation of unemployed persons. These recommendations should contribute to the channelling of services and interventions with regard to the unemployed persons’ needs; early interventions for persons facing obstacles in the work market; establishing of the treatment of unemployed persons based on objective criteria, that is, reducing the differences in the approaches of individual counsellors; managing of the relatively large workload of the counsellors in Croatia (the number of unemployed persons per counsellor).

Piloting of the StAP has been performed in the Regional CES Office in Zagreb (with several branches) and in the Field Office in Križevci between March 13 and September 30, 2017, using the experimental design of ‘stepped wedge’, which comprises the counsellors (that is, their newly registered unemployed protégés) being gradually introduced to the treatment.

Conclusion

An evaluation of the StAP effects is under way, which should examine the effects of the StAP on the outcomes of the process (correspondence of recommendations and real-life practice of service provision) and its effects on the outcomes of unemployed persons on the work market (employment after 6 and 12 months, deletion from the register of unemployed persons due to noncompliance or checking out, entry into active employment policy measures). Considering that the monitoring of outcomes and treatment of all registered unemployed persons included in the piloting takes six months, a complete evaluation of the effects may not commence sooner than April 2018.


Page 56 of 59

Based on the supervision report of the pilot implementation of the StAP, that is, qualitative and quantitative (survey) data on the attitudes and experience of the counsellors using StAP, it can be concluded that the majority of counsellors prefer StAP in comparison to the standard categorization procedure as it makes the process easier (faster) due to the integration of information on unemployed persons as well as the statistical estimation of employability, which is a welcome starting point for a personal assessment based on an interview with the unemployed person and familiarity with the local work market. A greater deal of support to the StAP was expressed by male and female counsellors with less work experience. About 37 percent of the 142 interviewed counsellors claimed that the standard categorization was complicated, while 21 percent of the interviewed counsellors called the StAP complicated. The negative aspects of the introduction of the StAP include an estimated time pressure for conducting quality categorization and professional plan, considering the fact that a greater number of individual counselling sessions takes more time. Counsellors whose files mostly contain groups of people with lower employability estimated that they would be unable to completely follow the recommendations on the frequency of personal consultations during unemployment. Counsellors who specialize in persons with disabilities assessed the statistical estimates made by the StAP and the related recommendations on the frequency of personal consultations as inadequate for their group of clients.

The most significant long-term result of the Project is the implementation of the StAP at the national level, which is reflected by the fact that the majority of Croatian counsellors passed the StAP training. Permanent implementation of a statistical tool and its related functionalities into the CES information system are also worth mentioning. The intended future applications of the StAP, extending beyond the directly affected processes of treating unemployed persons, are improving the quality of data collected in the CES information system, monitoring the coverage and composition of the users of active employment policy measures with regard to the segments of user employability, and monitoring and allocating available resources to regional and local offices, taking into account the local differences in the composition of the newly registered unemployed persons per employability segments.

Project Management

The MDFYSP established a Management Board to manage, direct, and coordinate the Project and integrate the Project activities into the economic and general legal reforms. The Management Board consisted of the representatives of the following institutions: MDFYSP, Ministry of Public Administration, MLPS, Ministry of Health, Ministry of Finance, ZVPRZOI, CPII, CHII, and CES. The Management Board President was the Loan Manager. During the implementation of the Project, a total of three different Loan Managers were in charge of preparing, implementing, and controlling all activities of the Project, and in 2016 itself, there were two reappointments to the Loan Manager position. The Management Board members changed with the same pace.

A Project Implementation Unit (PIU) was established at the MDPMSP, under direct control of the Loan Manager, which is responsible for the preparation, implementation, and management of the Project, with a focus on the procurement of goods and services during Project implementation and facilitating the conduct of all financial transactions within the Project. The PIU consisted of two government officials in the positions of Project Coordinator/Procurement Expert and Financial Management Expert. Apart from the Government officials mentioned earlier, the PIU also consisted of employees financed from the loan as part of the technical assistance for the PIU: a Technical Assistant for Procurement, a Technical Assistant


Page 57 of 59

for Finance, a Technical Assistant in the PIU, and a Translator. An audit of the Project’s financial statements was also conducted within the Project management process.


Page 58 of 59

ANNEX 6. DISBURSEMENT-LINKED INDICATORS WITH PDO-LEVEL INDICATORS

PDO Objective: DLIs with tentative indication of “target timing” PDO-Level Results

Indicators (PDIs)

Target Timing =

Year 1

Target Timing =

Year 2

Target Timing =

Year 3

Target Timing =

Year 4

I. One Stop Shop

1. The administration

of the GMB program

has been transferred

to the OSS (output),

EUR 5 million

2. The

administration of

the Child

Allowance

program has been

transferred to the

OSS (output),

EUR 5 million

3.The administration

of the non-

contributory

Maternity/Paternity

allowances and Birth

Grants have been

transferred to the OSS

(output),

EUR 5 million

4. The

administration of

the

Unemployment

Benefit program

has been

transferred to the

OSS (output),

EUR 5 million

PDI A. Reduce private

costs for social assistance

beneficiaries and

unemployed by 15% from

baseline

II. Central

Disability

Certification

Institute (CDCI)

5. CDCI legally

established (Law on

CDCI adopted),

adequately staffed,

and unified disability

certification

methodology

adopted by the

Government

(according to the

Law) (output), EUR

2 million

6. CDCI operating

independently

including: disability

certifications

processed centrally,

unified disability

certification

procedures applied,

regional office

network and

independent IT system

with interfaces to OSS

and other agencies

(output), EUR 2

million

PDI B1. Percentage of

cases certified by

IDCPREPD’s full-time

certifying officers

increased from 0% to at

least 90% (December

2018)

III. Reduce Error

and Fraud

7. The EFC Strategy

is approved and

disseminated by

Government, and the

Action Plan is

adopted and

disseminated by

MLPS and MSPY

(input), EUR 2

million

8. Minimum

effective EFC

capacity in place

in (a) MSPY; and

(b) MLPS or a

combination of

MLPS and either

or both of any of

its implementing

agencies and

CDCI (output),

EUR 2 million

9. Inspections of risk-

prone programs** are

carried out to correct

the list of irregularities

resulting from the

cross-checks between

the MIS of these

programs and the

databases of the Tax

Administration, CPII,

CES, MLPS, MSPY

and the Civil Registry

(output), EUR 2

million

10. Inspections of

risk-prone

programs* are

carried out to

correct the

beneficiary cases

with higher risk

scores, determined

through analytical

risk profiling.

(output), EUR 3

million

PDI C. Reduced Error

and Fraud: EFC risk-

prone programs* have

strengthened information

systems, and oversight and

control procedures in place,

including detection of error

and fraud using risk-based

investigation and data

matching.

IV. De-

institutionalization

11. Operational Plan

for Transformation

and

Deinstitutionalization

of Social Welfare

Homes 2014-2016

adopted by MSPY

and published

(output), EUR 2

million

12. 210

beneficiaries

living in State and

non-State

institutions or

family homes

moved from

institutions to

family-type

environments

(outcome), EUR 5

million

13. 225 beneficiaries

living in State and

non-State institutions

or family homes

moved from

institutions to family-

type environments

(outcome), EUR 5

million

14. 280

beneficiaries

living in State and

non-State

institutions or

family homes

moved from

institutions to

family-type

environments

(outcome), EUR 5

million

PDI D. De-

institutionalization. 715

beneficiaries living in State

and non-State institutions or

family homes moved from

institutions to family-type

care environments

Disbursements EUR 11 million EUR 12 million EUR 14 million EUR 13 million


Page 59 of 59

world bank document · 1 using an absolute poverty line fixed at 60 percent of median equivalent...

Documents