analytical techniques and considerations for...

Analytical techniques and considerations for evaluating Performance of SPL Programs

Welfare, Living Standards and Poverty Assessments

Ruslan Yemtsov

Global Practice for “Social Protection and Labor”

April 25, 2016

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Outline

1. Basic welfare concepts• Welfare: income, consumption, assets• Poverty• Vulnerability

2. How to measure performance of social protection?• Definitions: mapping of SPL• Performance indicators

• Pensions

• SSN

• Examples• Global benchmarking

2

Levels of welfare analysis

• Country welfare/living standards

• Welfare/living standards of population groups (deciles, quintiles, urban/rural..)

• Composition of welfare/income

• Policies and programs and their effect on welfare/poverty

• Indicators of welfare• Labor market indicators: employment, wages, productivity• Asset indicators: financial wealth, housing wealth, consumer durables,

productive assets• Access indicators: coverage by social services, distances to services, out-of–

pocket cost of services, access to social protection• Risk/opportunities indicators: unemployment statistics, catastrophic health care

spending, hazard rates for natural disasters. Human opportunities indices (HOI)• Inclusion/voice/political power indicators• Subjective well being indicators• Poverty and its dimensions

3

4,989

6,364 6,4917,430

6,358

19,585

25,510

31,899

39,545

42,486

0

5,000

10,000

15,000

20,000

25,000

30,000

35,000

40,000

45,000

50,000

1965 1970 1975 1980 1985 1990 1995 2000 2005 2010

PP

P c

on

stan

t 2

00

5 in

tern

atio

nal

$United States

Income Growth of the Bottom 40%1967-2011: 0.55% p.a.2000-2011: -1.41% p.a.

GDP per capita

Mean income of bottom 40%

4

Concerns on the growth of inequality that has happened in many economies…

5

Source: Lipton (2014).

Outline



• Pensions

• SSN


6

Poverty rates for different countries

7

0

100

200

300

National povert

y lin

e (

$/m

onth

at 2005 P

PP

)

3 4 5 6 7 Log consumption per person at 2005 PPP

Note: Fitted values use a lowess smoother with bandwidth=0.8

8

Where $- a day is coming from:National poverty lines plotted

against mean consumption using consumption PPPs for 2005

OLS elasticity=0.66

Source: Chen and Ravallion (2009)

AbsoluteIsrelative

Global poverty

* Preliminary

51

5958

4948

43

35

29

19 18

0

10

20

30

40

50

60

70

1981 1984 1987 1990 1993 1996 1999 2002 2005 2008 2010*

Popula

tion l

ivin

g o

n l

ess

than

$1.2

5 a

day

, 2005 P

PP

(%

)

Sub-Saharan

Africa

World

9

Non-monetary poverty:InsecurityPoor healthLow education or illiteracyLack of basic servicesSocial exclusionLack of freedom & voice/ lack of

empowermentPoor nutritional status*

10

Monetary poverty:

Income poverty

Consumption poverty

Dynamics of household welfare:• Duration in poverty• Chronic vs. transient poverty

Other dimensions of poverty

Vulnerability to poverty• Risk-induced vulnerability

* Spans across dimensions

Non-monetary poverty

concerns SP policy

11

Households move up and down in the distribution, some experience large swings

In this case with poverty line at 600 every one was close or crossed the line, but just one household stayed poor and one –non-poor

Other hh are “vulnerable” to risk of falling into poverty

But how reliable are these observations?

12

Concept of vulnerability : Welfare over time

HH1: always poor; HH3- usually poor; HH5 –transient poor; HH1- occasionally poor, HH2-never poor

400

500

600

700

800

900

1000

1100

1200

Period 1 Period 2 Period 3 Period 4

Expenditure trajectories over survey periods

(5 random households)

HH1

HH2

HH3

HH4

HH5

Vulnerability to poverty:

13

Vulnerable

The problem with this definition is that easily 60-80 % are becoming eligible for assistance=> Policy choices

TH

E W

OR

LD

BA

NK

’S S

OC

IAL

PR

OT

EC

TIO

N A

ND

LA

BO

R S

TR

AT

EG

Y 2

01

2-2

02

2

Who are the vulnerable groups?

A. Population groups with high exposure to risks, low resilience against shocks, and inability to cope with shocks

B. Groups living in disaster-prone areas, being exposed to diseases, living in remote areas, not owning assets

C. Groups that in any society have challenges: orphans, single mothers, disabled

D. A,B,C are correctPopula

tion gr

oups with

h...

Groups l

iving i

n disa

ster..

.

Groups t

hat in a

ny socie

t...

A,B,C

are

corr

ect

0% 0%0%0%

CLICKER QUESTION

30

Outline



• Pensions

• SSN


15

Performance of SPL ProgramsSocial Protection can be Quantified, Monitored and EvaluatedIndicators include:

• Inventory (statutory) information: what is out there• Input indicators (budgets)• Key performance (output) indicators• Indicators of impact

• What are programs? How are they financed?

• What are the objective of the programs? Their functions?

• How are programs/ schemes doing in • Covering those in need• Providing adequate support• Preserving incentives for work and contribute• Avoiding leakages and maintaining reasonable administrative costs

16

Social protection: questions to be answeredDiagnostic

1. What’s in place? Mapping national social

protection systems

Overview of existing schemes, benefits provided, people

covered and for what? Resources invested in

social protection & from which sources

Diagnostic

2. What’s missing? Uncovered population

in need for social protection: who they

are? What are the needs?

Options for extension?

Impacts & assessment

3. Performance of existing provision?

Evaluate effectiveness & efficiency of existing

schemes and national social protection system

Impacts & Simulation

4.Long impacts? (assessing poverty, labour

market and economic impacts)

17

Categorisation of schemes & benefits- Statistical view (ILO)

Benefits features Schemes/ programs definition

SP system

Public

Private

Contributory

Non contributory

Contributory

Non contributory

Older persons Old age Survivors*

Family / child benefit

Active age Sickness Disability/ Invalidity Unemployment Employment injury Maternity Survivors*

ALMP Housing * Other social assistance*

Health care benefit

Function

Cash

In-kind

Type of benefit

periodic

Lump-sum/ ad hoc

periodic

Lump-sum/ ad hoc

Periodicity

Means-tested

Not means tested

Means-tested

Not means tested

Means-tested

Not means tested

Means-tested or not

Other criteria Poverty/livelihood

– protection against destitution

Risk management – prevention

Promotion

Statutory

Non statutory

Public

Private

Contributory

Non contributory

Contributory

Non contributory

18

Practice- driven approach (WBG)Social

Protection and Labor

Social Insurance (contribution based)

Old-age and survivor’s pensions

Disability

Paid Maternity leave

Temporary sick leave

Funeral allowance

Disaster insurance

Social Assistance (non-contributory)

Cash benefits

Family benefits

Child care allowance

Social pension

Birth grant

Public works

In-kind benefits

Housing

Day care services

Food based

School and sup feeding

Medical

Labor market services

Unemployment

Measures/programs and support

administered by NES

Private Charities,

Remittances, Intra family

1 2 3 4

19

TH

E W

OR

LD

BA

NK

’S S

OC

IAL

PR

OT

EC

TIO

N A

ND

LA

BO

R S

TR

AT

EG

Y 2

01

2-2

02

2

Are functions of social protection the same as stages of life cycle?

A. Yes, these are all concepts of Social protection stasistics

B. Yes, they are the same, every function (child, working age, old) corresponds to a stage in life cycle

C. No, they are different: functions have evolved historically in the development of social protection, life cycles is a theoretical concept

D. Yes and no, “functions” are detailed descriptions of risks, they do not have one-to-one correspondents with life cycle. Yes,

these

are all c

oncept..

.

Yes, th

ey are

the sa

me, e

...

No, they are

diff

erent:

fu...

Yes and n

o, “fu

nctio

ns” ..

.

0% 0%0%0%

CLICKER QUESTION

30

Social spending pattern varies by countries

0

2,000

4,000

6,000

8,000

10,000

12,000

14,000

16,000

0%

5%

10%

15%

20%

25%

GN

I per

cap

ita

Perc

ent

of

GD

P

Direct Transfers EducationHealth Contributory PensionsNon-contributory Pensions Other Social SpendingGNI per capita (2011 PPP, right axis)

Composition of Social Spending(as a share of GDP)

Source: Armenia (Younger and Khachatryan, 2015); Georgia (Cancho and Bondarenko, 2015); Russia (Lopez-Calva et al, 2015); Brazil

(Higgins and Pereira, 2014); Indonesia (Jellema et al, 2015), Mexico (Scott, 2014); Peru (Jaramillo, 2014), Uruguay (Bucheli et al, 2014);

South Africa (Inchauste et al, 2014); and Sri Lanka (Arunatilake et al, 2015).21

Outline



• Pensions

• SSN


22

Reminder: pension systems performance - CAS

Sustainability

Adequacy Coverage

SSN and Pensions system performance similarities and differences

Pensions• Pension – provide adequate income

replacement for those unable to work in a sustainable and predictable way

• 3 equally important performance indicators:• Coverage of everyone (insurance),

including contributions/affiliations• Adequacy to replace lost income• Sustainability to ensure predictable

protection

• Additional indicators:• Environment design ,performance

SSN• SSN- provide necessary income

support for those who need it as long as they need it to avoid poverty and deprivation

• Main performance outcome –poverty reduction/elimination

• Ways to get there:• Cover those in need (poor and

vulnerable) – coverage based on receipts of benefits

• Cover only those in need (targeting)

• Provide enough money to get out of poverty (adequacy)

24Common: administrative efficiency

What is coverage?

Coverage has two dimensions:

1) scope of coverage i.e. the number of SP programs / areas to which different population groups have access and

2) extent of coverage i.e. the share of persons covered within the target group of different SP programs.

We are interested in the extent of coverage, which in turns can be:

a) legal (statutory) coverage, i.e. groups covered by statutory schemes for a given social protection function/branch in national legislation

b) effective coverage, i.e. the proportion of persons covered within the whole population or target group:

i. protected persons, i.e. the number of persons who have benefits guaranteed but are not necessarily currently receiving them (e.g. in contributory schemes those actually contributing/affiliated); and

ii. actual beneficiaries, i.e. the proportion of the population affected by a certain contingency who actually receive the respective benefit

» The distinction between direct and indirect beneficiaries versus

direct beneficiaries

25

Coverage

It is the percentage of the population receiving a benefit

or service

Coverage =Number of beneficiaries in the total population (or group ∗)

Total population (or group)

Group refers to subsamples of the population, i.e.: income groups (quintiles,

poor, non poor), geographical groups (rural vs urban),

26

TH

E W

OR

LD

BA

NK

’S S

OC

IAL

PR

OT

EC

TIO

N A

ND

LA

BO

R S

TR

AT

EG

Y 2

01

2-2

02

2

27

RECAP: Global coverage

7570

65

46

32

22

1217

9

21

22 43

13 13

2327

4633

SS Africa MENA SAR LAC EAP ECA

No transfer Only social insurance

Only social assistance Labor market programs

Per

cen

t

Source: World Bank ASPIRE database

% of households receiving transfers

Low coverage concentrated among:

low-income countries and fragile contexts

poor populations and vulnerable groups, including women

informal sector

Meeting the coverage challenge

Sustainability -- fiscally and institutionally

Adequacy – appropriate levels of support, inclusive

TH

E W

OR

LD

BA

NK

’S S

OC

IAL

PR

OT

EC

TIO

N A

ND

LA

BO

R S

TR

AT

EG

Y 2

01

2-2

02

2

What is the objective of Social Protection in terms of coverage?

A. Cover 100% of populations

B. Cover everyone in need and when they are in need

C. Cover the poor and vulnerable

D. Cover everyone who is willing and capable to contribute.

Cover 100% o

f popula

tions

Cover eve

ryone in

need a..

Cover the p

oor and vuln

...

Cover eve

ryone w

ho is w

il...

0% 0%0%0%

CLICKER QUESTION

30

Graph 2: Venn diagrams

39.71%

1.45%

0.10%

1.35%

27.52%

6.78% 0.49%

No SP transfers: 22.60%

All social insurance

All labor market programs

All social assistance

Program overlap is the share of households receiving 2 or

more programs: example of Russia

Source: Rosstsat, Income Survey , 2014

Why coverage is not enough?

What is adequacy?

Because what is relevant is “access to adequate and predictable benefits in case of need” Common Principle: Taken together, cash and in kind benefits should secure protection against poverty, vulnerability and social exclusion and enable a decent standard of living

How to measure? Benefit level

– Combined across programs (household survey vs. administrative data)

– Minimum standards (poverty line)

– Consumption /income

– International benchmarks

NEEDS

GAPS & ISSUES

Solutions

30

ADEQUACY of Benefits

It is the share of the transfers with respect to total

consumption or income of a group

Adequacy=Amount of transfers received by a group

Total income or consumption of thebeneficiaries in the group

Group refers to subsamples of the beneficiaries that belong to specific groups, i.e.: income

groups (quintiles, poor, non poor), geographical groups (rural vs urban).

31

Immustration SSN adequacy for the poor varies by

country

Between 2010--2013

4032

3026

2424

2322

2019

181717

151414

1313

1211

109

7

0 5 10 15 20 25 30 35 40 45

Georgia (2011)

Average Social Pension (63)

Average UCT (34)

Average PW (10)

Mexico (2010)

Colombia (2012)

Nepal (2010)

Swaziland (2010)

Djibouti (2012)

Argentina (2010)

Honduras (2011)

Average Other SSN (53)

Average CCT (15)

Malawi (2010)

Belarus (2010)

Bangladesh (2010)

Philippines (2013)

Jordan (2010)

Moldova (2010)

Madagascar (2010)

Turkey (2012)

Jamaica (2010)

Senegal (2011)

Child A

Pensions

SA

TH

E W

OR

LD

BA

NK

’S S

OC

IAL

PR

OT

EC

TIO

N A

ND

LA

BO

R S

TR

AT

EG

Y 2

01

2-2

02

2

What determines adequacy of Social Protection transfer?

A. Available budgetary resources and number of beneficiaries

B. Statutory provisions (laws and regulations)

C. Politics and lobbying by different groups

D. Economic situation in the country, inflation, employment, wages

E. Depending on the program and time period, any of the above. Avail

able b

udgeta

ry re

so...

Statu

tory

pro

visions (

law

...

Politics

and lobbyin

g by d

...

Econom

ic sit

uation in

th...

Depending o

n the p

rogr

..

0% 0% 0%0%0%

CLICKER QUESTION

30

Distribution of Benefits

(Benefits incidence)

It is the proportion of benefits received by a

group

Benefits incidence =Amount of benefits received by a group

Program total benefits

34

MARKET INCOME

DISPOSABLE INCOME

PLUS DIRECT TRANSFERS MINUS DIRECT TAXES

PLUS INDIRECT SUBSIDIES MINUS INDIRECT TAXES

POST-FISCAL or CONSUMABLE INCOME

PLUS MONETIZED VALUE OF PUBLIC SERVICES: EDUCATION & HEALTH

FINAL INCOME

Static Distributional Analysis : Income

Concepts

35

-0.20

-0.18

-0.16

-0.14

-0.12

-0.10

-0.08

-0.06

-0.04

-0.02

0.00

Chang e i n G in i f r om m ar k e t t o f i na l i n c om e

Sources: Armenia (Younger et al, 2014), Brazil (Higgins and Pereira, 2014), Bolivia (Paz Arauco

et al, 2014), El Salvador (Beneke et al, 2015), Ethiopia (Woldehanna et al, 2014), Georgia

(Cancho and Bondarenko, 2015), Guatemala (Cabrera et al, 2014), Indonesia (Afkar et al,

2015), Mexico (Scott, 2014), Peru (Jaramillo, 2014), Russia (Lopez Calva et al, 2015), Uruguay

(Bucheli et al, 2014), South Africa (Inchauste et al, 2015), and Sri Lanka (Arunatilake et al,

2014).

0.51

0.57

0.57

0.65

0.49

0.57

0.47

0.43

0.42

0.38

0.36

0.33

0.33

0.31

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7

Mexico

United States

Spain

Poland

Japan

Sweden

Switzerland

Gini post-tax and transfers

G I NI I nequa l i t y be f o re and a f t e r f i s ca l

po l i c y ( l a t es t ava i l ab le yea r ) , s e l ec t ed

O ECD c oun t r i es

Sources: For Mexico, CEQ; for other OECD countries, Janet Gornick, based on LIS

microdata ca 2000-mid2000s, reproduced in The Economist, Nov 2013. 36

How a government collects and spends money can

influence inequality

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Cum

ula

tive

pro

po

rtio

n o

f m

arket

in

com

e/ t

ran

sfer

Cumulative proportion of the population

Progressive transfer

in relative terms

(NOT pro-poor)

Pre-transfer

Lorenz curve

Absolutely progressive

transfer (pro-poor)

37

Measuring progressivity

Example of distributional analysis by population

groups

38

Distributional Impact of Different Components of the Tax and Benefit System by

Market (Pre-fiscal) Income Deciles, Percent of Disposable Income, 2014

Source: World Bank based on HSE-RLMS data for 2014.

-60

-40

-20

0

20

40

60

80

100

poorest 2 3 4 5 6 7 8 9 richestSocial Contributions Direct Taxes Indirect Taxes

Pensions Direct Transfers Total

TH

E W

OR

LD

BA

NK

’S S

OC

IAL

PR

OT

EC

TIO

N A

ND

LA

BO

R S

TR

AT

EG

Y 2

01

2-2

02

2

What is a progressive transfer?

A. A transfer that reduces inequality

B. A transfer that goes only to the poor

C. A transfer that goes in equal share to everyone in the society

D. A transfer that is not reducing incentives for work.

A transf

er that r

educes i

...

A transf

er that g

oes only

..

A transf

er that g

oes in eq...

A transf

er that i

s not r

ed...

0% 0%0%0%

CLICKER QUESTION

30

Social protection Impact

Simulated change (%) on poverty HC/Gap/Gini due to SPL programs

• Poverty headcount reduction • Poverty gap reduction• Inequality reduction (Gini)

HC reduction=Poverty headcount pre transfer− poverty headcount post transfer)

Poverty headcount pre transfer

40

Source: ASPIRE

41

Thank you!!

42

analytical techniques and considerations for...

Documents