Impact EvaluationImpact Evaluation

Low-Income Housing Low-Income Housing Finance in Sri LankaFinance in Sri Lanka

Binh T. NguyenBinh T. Nguyen

Independent Evaluation Department Independent Evaluation Department

Asian Development BankAsian Development Bank

17 Oct 201017 Oct 2010

ContentsContents

Overview of ADB Assistance in Low-Income Overview of ADB Assistance in Low-Income HousingHousing

Case Project: Urban Development and Low-Case Project: Urban Development and Low-Income Housing Project in Sri LankaIncome Housing Project in Sri Lanka

The Impact EvaluationThe Impact Evaluation

The Importance of Housing for Achieving the Millennium Development Goals

Millennium Development Goals (MDGs) Role of Housing in Achieving MDGs

Goal 1: Eradicate extreme poverty and hunger

Residential activities can provide job opportunities and income and thereby allow urban poor to invest in food and other basic needs.

Improved housing conditions raise worker productivity. Residential activities improve a nation’s wealth (e.g. taxes and

savings) and allow governments and agencies to invest in social oriented programs to reduce poverty.

Goal 2: Achieve universal primary education

Improved, and access to, housing in appropriate locations lowers absenteeism from school.

Improved, and access to, housing increases educational productivity. Secure tenure allows parents to engage in income-generation

activities allowing them to cater for educational expenses. Goal 3: Promote gender equality and empower women

Secure tenure contributes to household stability and provides women with peaceful atmosphere to engage in economic-generating activities.

Good housing reduces stress and contributes to women’s productivity. Goal 4: Reduce child mortality

Good housing and related services (e.g. water, electricity, and sanitation) reduces the risk of diseases among children.

Goal 5: Improve maternal health

Improved housing lowers the need for health services for women. Secure tenure reduces stress among slum dwellers, especially

women. Safeguard procreation and nurturing of the young.

Goal 6: Combat HIV/AIDS, malaria and other diseases

Access to housing reduces homelessness and risks of social vices associated with street people.

Good housing brings comfort, reduces overcrowding and limits the transmission of communicable diseases (e.g. tuberculosis), it facilitates and enhances care-giving.

Health conditions depend largely on good living environment. Goal 7: Ensure environmental sustainability

Good housing conditions and related services contribute to a good environment.

Use of environmentally friendly building materials, including energy-efficient materials, contributes to environmental protection.

Good housing and urban design are cornerstones for mitigating ecological footprints of settlements and reducing vulnerability to climate change.

Goal 8: Develop a global partnership for development

Partnership between national government and international development agencies creates synergy and reduces duplication of programs.

Partnership between national government and international development agencies for housing ensures realistic policies and programs, and sharing of best practices.

Programs that involve partnerships among national governments, international development agencies, local communities and slum dwellers have a better chance of long-term sustenance.

Source: Tibaijuka, Ana Kajumulo, “Building Prosperity: Housing and Economic Development”, UN-Habitat and Earthscan, London and Sterling, Virginia, 2009.

ADB Loans and Grants for LIUHADB Loans and Grants for LIUH

Year of Year of ApprovaApprova

ll

LIUH Loans & GrantsLIUH Loans & GrantsAll ADB Loans & All ADB Loans &

GrantsGrants

Amount Amount

NumbeNumberr

Amount Amount

($, M)($, M)% of All % of All LoansLoans

NumbeNumberr

Amount Amount

($, M)($, M)

1966-1966-19691969 - -

- - - -

21 21 99.68 99.68

1970-1970-19791979

4 4

98.80 98.80 2.452.45

239 239 4,040.57 4,040.57

1980-1980-19891989

7 7

269.60 269.60 2.412.41

262 262 11,206.53 11,206.53

1990-1990-19991999

9 9

612.00 612.00 1.921.92

323 323 31,915.58 31,915.58

2000-2000-20102010

19 19

346.30 346.30 0.390.39

2,046 2,046 88,968.94 88,968.94

TotalTotal 39 39

1,326.7 1,326.7 0.940.94

2,891 2,891

141,513.21 141,513.21

Project Performance RatingsProject Performance Ratings

RatingRating NumberNumber %%

Highly Highly SuccessfulSuccessful

11 4.54.5

SuccessfulSuccessful 1313 59.159.1

Partly SuccessfulPartly Successful 44 18.218.2

Unsuccessful Unsuccessful 11 4.54.5

Not RatedNot Rated 33 13.613.6

Total Total 2222 100100

Case ProjectCase ProjectLoan 1632-SRI: Urban Development Loan 1632-SRI: Urban Development

and Housing Projectand Housing ProjectBasic Data:Basic Data:

Approved 1998 Completed 2005Approved 1998 Completed 2005

Total cost $102.99M ADB loan $67.02M Govt/Banks $35.97MTotal cost $102.99M ADB loan $67.02M Govt/Banks $35.97M

Four Components: Four Components:

(i)(i) Urban Development: roads, traffic improvement, water Urban Development: roads, traffic improvement, water supply, drainage, etc.supply, drainage, etc.

(ii)(ii) Community Development: basic infra, tenure regulations Community Development: basic infra, tenure regulations

(iii)(iii) Housing Finance: housing loans to low-income householdsHousing Finance: housing loans to low-income households

(iv)(iv) Institutional Development: training on staff skills in municipal Institutional Development: training on staff skills in municipal management, environment management, etc.management, environment management, etc.

Project Expenditure ($, Million)Project Expenditure ($, Million)

  ComponentComponent ADBADB GovtGovt Total Total

PercenPercentt

Urban Urban InfrastructureInfrastructure

40.2840.28 23.623.6 63.8863.88 62.062.0

Community Community DevelopmentDevelopment

1.411.41 1.61.6 3.013.01 2.92.9

Housing Housing FinanceFinance

19.919.933

77 26.9326.93 26.126.1

Institutional Institutional DevelopmentDevelopment

3.533.53 3.773.77 7.37.3 7.17.1

ChargesCharges 1.871.87 00 1.871.87 1.81.8

TotalTotal 67.067.022

35.9735.97 102.99102.99 100100

Housing Finance ComponentHousing Finance Component

Objectives:Objectives:

• Increase access of low-income households (LIHs) to Increase access of low-income households (LIHs) to market-based housing finance through the formal market-based housing finance through the formal sector;sector;

• Facilitate improvements of housing conditions and Facilitate improvements of housing conditions and quality of life; andquality of life; and

• Promote formal banking sector interest in financing Promote formal banking sector interest in financing low-income housing market segment. low-income housing market segment.

LIHs = households with monthly income below the 55LIHs = households with monthly income below the 55thth income percentile, i.e., below Rs12,500 (appr. $200) income percentile, i.e., below Rs12,500 (appr. $200) per month. per month.

Housing Finance Component Housing Finance Component (Cont.)(Cont.)

Basic Data:Basic Data:

• Total Amount = $26.93M Total Amount = $26.93M

ADB loan = $19.93M PCIs = $7MADB loan = $19.93M PCIs = $7M

• Total Borrowers = 28,378Total Borrowers = 28,378

• PCIs = 7 participating credit institutions PCIs = 7 participating credit institutions

- Housing Finance Development Corporation = Housing Finance Development Corporation = 68.6%68.6%

- 3 Regional Development Banks = 27.9%3 Regional Development Banks = 27.9%

- 3 Commercial Banks (BOC, Hatton, National) = 3 Commercial Banks (BOC, Hatton, National) = 3.5%3.5%

Housing Finance Component Housing Finance Component (Cont.)(Cont.)

Loan Disbursement by ProvinceLoan Disbursement by Province

Province Province

NumbeNumber r

of of Loans Loans

(%) (%)

Loan Loan Amount Amount

(SLRs (SLRs million)million)

(%) (%)

Western Western 5,097 5,097 18.0 18.0 604.6 604.6 25.1 25.1

Southern Southern 8,109 8,109 28.6 28.6 561.2 561.2 23.3 23.3

Central Central 7,051 7,051 24.9 24.9 500.0 500.0 20.8 20.8

SabaragamuSabaragamuwa wa

2,154 2,154 7.6 7.6 216.8 216.8 9.0 9.0

North North Western Western

1,658 1,658 5.8 5.8 182.5 182.5 7.5 7.5

North Central North Central 2,506 2,506 8.8 8.8 179.5 179.5 7.5 7.5

UVA UVA 1,420 1,420 5.0 5.0 125.3 125.3 5.2 5.2

North and North and East East

383 383 1.3 1.3 38.3 38.3 1.6 1.6

Total Total 28,378 28,378 100.0 100.0 2,408.2 2,408.2 100.0 100.0

Housing Finance Component Housing Finance Component (Cont.)(Cont.)

Loan Disbursement by Income Loan Disbursement by Income Group Group

Income Group Income Group

(SLRs per (SLRs per month) month)

NumbeNumber r

of of Loans Loans

(%) (%) SLRsSLRs

MillionMillion (%) (%)

less than 2,500 less than 2,500 362 362 1.3 1.3 12.6 12.6 0.5 0.5

2,501–5,000 2,501–5,000 4,198 4,198 14.8 14.8 219.6 219.6 9.1 9.1

5,001–7,500 5,001–7,500 9,630 9,630 33.9 33.9 719.6 719.6 29.9 29.9

7,501–10,000 7,501–10,000 8,948 8,948 31.5 31.5 825.8 825.8 34.3 34.3

10,001–12,50010,001–12,500 5,240 5,240 18.5 18.5 630.6 630.6 26.2 26.2

Total Total 28,378 28,378 100.0 100.0 2,408.2 2,408.2 100.0 100.0

Housing Finance Loan Housing Finance Loan (Cont.)(Cont.) Loan Disbursement by PurposeLoan Disbursement by Purpose

Purpose Purpose NumberNumber

of of Loans Loans

(%) (%)

Construction of New House Construction of New House and Extensions and Extensions

19,751 19,751 69.6 69.6

Renovation of Existing Renovation of Existing Houses Houses

6,130 6,130 21.6 21.6

Purchase of Land for New Purchase of Land for New House Construction House Construction

2,128 2,128 7.5 7.5

Service Connections Service Connections 312 312 1.1 1.1

Purchase of House Purchase of House 57 57 0.2 0.2

Total Total 28,378 28,378 100.0 100.0

Why This Project Was Chosen?Why This Project Was Chosen?

• Findings of the evaluation will provide insights Findings of the evaluation will provide insights and lessons for urban development operations and lessons for urban development operations guidelines being prepared by the Urban COPguidelines being prepared by the Urban COP

• The project had clear assignment rule: The project had clear assignment rule: Households with income below the 55Households with income below the 55thth income income percentile (this was confirmed during the percentile (this was confirmed during the Reconnaissance Mission in preparing for the Reconnaissance Mission in preparing for the evaluation)evaluation)

• The project appeared to be the best among ADB The project appeared to be the best among ADB housing projects in terms of baseline data: loan housing projects in terms of baseline data: loan applicants were required to submit a detailed applicants were required to submit a detailed household profile, and these are kept in PCIshousehold profile, and these are kept in PCIs

The Impact EvaluationThe Impact Evaluation

Objectives: Objectives: In addition to assessing the extent to which In addition to assessing the extent to which the housing finance component met its stated objectives the housing finance component met its stated objectives by usingby using the standard evaluation criteria, this impact the standard evaluation criteria, this impact evaluation will:evaluation will:

• Empirically assess the welfare change of the Empirically assess the welfare change of the beneficiaries that can be attributed to the housing beneficiaries that can be attributed to the housing finance componentfinance component

• Identify factors (social, economic, project design and Identify factors (social, economic, project design and implementation) influencing the project outcomesimplementation) influencing the project outcomes

• Propose a sensible set of outcome indicators and Propose a sensible set of outcome indicators and benchmarks that can be used in future project benchmarks that can be used in future project design, monitoring and evaluationdesign, monitoring and evaluation

Evaluation FrameworkEvaluation Framework

Hypothesis: Hypothesis:

Improved housing conditions will lead to increased Improved housing conditions will lead to increased labor productivity, income, and overall social well-labor productivity, income, and overall social well-being of the project beneficiaries.being of the project beneficiaries.

Logic Model:Logic Model:

Inputs Inputs Activities Activities Outputs Outputs Outcomes Outcomes ImpactsImpacts

Impact IndicatorsImpact Indicators

Follow IADB study (2008) and Field and Kremer (2006)Follow IADB study (2008) and Field and Kremer (2006)

Household-level outcomes:Household-level outcomes:• Housing quality index (HQI),Housing quality index (HQI),[1][1]

• Per capita household consumption expenditure (per year),Per capita household consumption expenditure (per year),• Household completeness (presence of spouse and formally Household completeness (presence of spouse and formally

married),married),• Occupation ratio (percent of working household members),Occupation ratio (percent of working household members),• School attendance (of school age children), andSchool attendance (of school age children), and• Nourishment ratio (percent of children under 6 who are not Nourishment ratio (percent of children under 6 who are not

under-nourished).under-nourished).

[1][1] , where , where aaii equals unity if the house has condition equals unity if the house has condition ii, and zero, and zero

otherwise; andotherwise; and i i runs through the seven dwelling quality indicators: potable runs through the seven dwelling quality indicators: potable water access, sewage connection, electricity connection, walls, floors, ceilings, water access, sewage connection, electricity connection, walls, floors, ceilings, and overcrowding problems (more than 2 persons living per room).and overcrowding problems (more than 2 persons living per room).

i

iaHQI7

Impact Indicators (Cont.)Impact Indicators (Cont.)

Community-level outcomes:Community-level outcomes:

• Poverty rate (percent of households below Poverty rate (percent of households below the poverty line),the poverty line),

• Housing shortage (percent of households Housing shortage (percent of households without a house),without a house),

• Loan default ratio, Loan default ratio,

• Crime rates, andCrime rates, and

• Net migration (difference between migration Net migration (difference between migration in and out).in and out).

Estimation MethodsEstimation Methods

Regression Discontinuity Design:Regression Discontinuity Design:

Before the treatment After the treatmentBefore the treatment After the treatment

+++++++++++ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

********************

******************

***************

**********

Y

Participants Non-participants

+++++++++++ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

****************************

***************

*******************

************

Y

Participants Non-participants

Cutoff ScoreCutoff Score

RDD (cont.)RDD (cont.)

The program impact is estimated by the mean difference The program impact is estimated by the mean difference in outcomes for persons above and below the cut-off point. in outcomes for persons above and below the cut-off point.

Probability of BorrowingProbability of Borrowing

Before treatment:Before treatment:

After treatment:After treatment:

.55 income

.55 income

Expected OutcomeExpected Outcome

Before Treatment:Before Treatment:

After Treatment:After Treatment:

.55 income

.55 income

Fuzzy Regression Discontinuity Fuzzy Regression Discontinuity DesignDesign

Estimator:Estimator:

where where EE[.] is the expectation operation; [.] is the expectation operation; xx* is the cutoff (i.e., the 55th income * is the cutoff (i.e., the 55th income percentile); percentile); YYii and and XXii are the outcome and forcing (treatment-determining) are the outcome and forcing (treatment-determining) variable of household variable of household ii, respectively; and is the assignment variable , respectively; and is the assignment variable with 1[.] being the index function taking value 1 if the condition in the square with 1[.] being the index function taking value 1 if the condition in the square brackets is correct, and zero otherwise. The denominator represents the jump in brackets is correct, and zero otherwise. The denominator represents the jump in borrowing probability due to treatment assignment.borrowing probability due to treatment assignment.

We will follow We will follow Imbens and Lemieux (2008)Imbens and Lemieux (2008)[1][1] to use the local linear regression to use the local linear regression estimation method, where both the difference in the outcome (numerator) and estimation method, where both the difference in the outcome (numerator) and the difference in the borrowing probability (denominator) will be estimated by the difference in the borrowing probability (denominator) will be estimated by the fitted values of the fitted values of YY and and WW at both sides of the cutoff. at both sides of the cutoff.

[1][1] Imbens, G. and T. Lemieux. 2008. Regression discontinuity designs: a guide to practice. Imbens, G. and T. Lemieux. 2008. Regression discontinuity designs: a guide to practice. Journal of Journal of EconometricsEconometrics, vol. 142 (2): 615 – 635. , vol. 142 (2): 615 – 635.

] x X | W[ E lim - ] x X | W[ E lim

] x X |Y [ E lim - ] x X |Y [ E lim

i i* x↓x

i i* x↑x

i i* x↓x

i i* x↑x

*][1 xXW ii

Other Estimators: PSM and DDOther Estimators: PSM and DD

The RDD estimator gives an unbiased estimate The RDD estimator gives an unbiased estimate of the project impact. However, it only estimates of the project impact. However, it only estimates the project effect near the cutoff. the project effect near the cutoff.

If the project effect is constant, this poses no If the project effect is constant, this poses no problem. problem.

To give an estimate of the overall average To give an estimate of the overall average treatment effect, we will also use the propensity treatment effect, we will also use the propensity score matching (PSM) method combined with a score matching (PSM) method combined with a double/single difference estimator, pending data double/single difference estimator, pending data availability and quality. availability and quality.

Data RequirementData Requirement

Province Province TreatmenTreatmen

ttComparisoCompariso

nn

CommunitCommunityy

SurveySurvey FGDFGD KIIKII

SouthernSouthern 495495 495495 1010 1010

CentralCentral 420420 420420 1010 1010

WesternWestern 300300 300300 1010 1010

North CentralNorth Central 150150 150150 1010 1010

SabaragamuSabaragamuwawa 135135 135135 1010 1010

TotalTotal1,5001,500 1,5001,500 5050 5050

101000

Household SampleHousehold Sample

TimeTime Treatment GroupTreatment Group Comparison GroupComparison Group

19981998 T0 = 1,500 households with T0 = 1,500 households with income below the cutoff; income below the cutoff; households actually households actually borrowed will be drawn from borrowed will be drawn from client profiles in PCIs’ client profiles in PCIs’ databases; households that databases; households that did not borrow will be drawn did not borrow will be drawn from household profiles kept from household profiles kept in the village statistics in the village statistics archivesarchives

C0 = 1,500 households with C0 = 1,500 households with income above cutoff drawn from income above cutoff drawn from household profiles in the village household profiles in the village statistics archivesstatistics archives

20102010 T1 = the same 1,500 T1 = the same 1,500 households of T0 to be households of T0 to be surveyed for the studysurveyed for the study

C1 = the same 1,500 households C1 = the same 1,500 households of C0 to be surveyed for the of C0 to be surveyed for the study study

Implementation PlanImplementation Plan

MilestonesMilestones OCTOCT NOVNOV DECDEC JANJAN FEBFEB MARMAR APRAPR MAYMAY JUNJUN JULJULAUAUGG SEPSEP

PreparationsPreparations                                       

Data CollectionData Collection                                       

DataData AnalysisAnalysis                                       

Drafting ReportDrafting Report                                       

Review/RevisionReview/Revision                                       

CompletionCompletion                                       

Thank you.Thank you.

For more information:For more information: http://www.adb.org/Evaluation/resources.asphttp://www.adb.org/Evaluation/resources.asp