m&e in ppps · 2014-01-17 · management center (pimac), korea development institute nov. 2013...
TRANSCRIPT
![Page 1: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/1.jpg)
Public and Private Infrastructure Investment Management Center (PIMAC),
Korea Development Institute
PIMAC Nov. 2013
Korean Evaluation System - M&E in PPPs -
Soojin Park
![Page 2: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/2.jpg)
2
The Role of PIMAC stipulated by the National Finance Act and its decrees
1) Evaluator and/or Government Agency in Public Investment Management (PIM) :
Carry out ex-ante evaluation such as PFS (preliminary feasibility study)
Carry out intermediate evaluations such as RSF (re-assessment study of feasibility) and RDF (re-assessment of demand forecast)
Carry out ex-post evaluation such as IEBP (in-depth evaluation of budgetary program)
2) Researcher
Support for new initiatives of better PIM
Policy studies on PIM
Missions of PIMAC (Fiscal investment)
![Page 3: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/3.jpg)
Overview of Korean PPP Part-01
![Page 4: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/4.jpg)
Amendment
Jan. 2005
Revision
Jan. 1999
PPP Act, which was introduced in 1994 has gone through several
revisions.
PPP Legal Framework in Korea (I)
Enactment
Aug. 1994
『The Private Capital Inducement Promotion Act』
『The Act on Private Participation in Infrastructure 』
· Unsolicited proposals, Minimum Revenue Guarantee
『The Act on Private Participation in Infrastructure 』
· Diversified PPP-eligible Facility Types (35 -> 44)
· Introduction of BTL Scheme
Unsolicited proposals and MRG in 1999
- MRG abolished (unsolicited in 2006/ solicited in 2009)
BTL in 2005 : promoted its use in educational facilities, military residences,
environmental facilities, etc.
4
![Page 5: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/5.jpg)
5
Investments through PPP increased from 1995 to 2007, since then it
continues to decrease.
MRG was abolished in 2006, and financial crisis occurred in 2008
As considerable level of infrastructure are equipped, growth ratio of PPP investment
began to decrease.
Track Record of Korean PPP - History
bil USD Number
(*) All the figures are contract basis and thus real investment balance may be different
![Page 6: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/6.jpg)
6
Track Record of Korean PPP – Snap Shot
As of 2012, Korea had 633 PPP Projects either in construction or in
operation status, total value being 85 bil USD.
209 number of BOT and BOO projects with 60.4 bil USD of investment
424 BTL projects with 24.5 bil USD of investment cost.
mil USD
BTL BTO & BOO
mil USD
60.4 bil USD 24.5 bil USD
(*) All the figures are contract basis and thus real investment balance may be different
![Page 7: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/7.jpg)
7
PPP has alleviated the government’s burden on infrastructure
investment from 5% to 36% per year. ( Average: 21% per year )
The promotion of PPP has helped ease constraints on the government’s financial
resources, enabling it to secure resources for sectors other than SOC.
PPP’s Contribution to Fiscal Budget
(*) All the figures are contract basis and thus real investment balance may be different
bil USD % of PPP
![Page 8: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/8.jpg)
Monitoring & Evaluation in PPP Part-02
![Page 9: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/9.jpg)
9
Monitoring & Evaluation by Project Cycles
Feasibility Test
Vfm test
Revised Vfm test
Macro level Project level
Intermediate & Ex-post
management
BTO/ BTL Performance
evaluation
Comprehensive ex-post evaluation • Macro-lev. evaluation • Ex-post VFM • Satisfaction survey
Ensuring macro economic sustainability and better PPP policies
Continuous Reporting
& Improvement
Feedback
Infra-info DB
Feedback &
Input
![Page 10: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/10.jpg)
Fiscal Soundness Management Part-03
![Page 11: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/11.jpg)
11
Macroeconomic Fiscal Risk Management
PPPs can accelerate infrastructure provision by mitigating the
problem of limited public resources
Government can benefit from the efficiency of the private sector and
invest in other areas by saving the expenditure
However, it is not appropriate to increase PPPs without limit
What is the sustainable level of PPPs?
How to measure the government debt caused by PPPs?
How to manage a PPP liability ceiling?
– Who evaluates the ceiling?
– When and how often evaluated?
– How to report to and get approval from the National Assembly?
Key Questions
![Page 12: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/12.jpg)
12
Macroeconomic Fiscal Risk Management
Necessity of setting up a safeguard ceiling for PPPs
– May have a negative impact on fiscal soundness and stability
– Debt which should be paid by next generations (no free lunch)
United Kingdom
– Annual government payment for PFI : about 2% of total government
expenditure
– PFI investment : 10% to 15% of total public capital investment
Korea : Setting a safeguard ceiling on government payment
– Annul government payment : 2% of total government budget
expenditure
– PPP investment : 10% to 15% of total public investment
![Page 13: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/13.jpg)
13
Simulation 4
• Examines whether total amount of public financing of signed and planned BTO plus BTL
projects exceeds the 2% ceiling.
• In Scenario 1, still less than 2%, but, in Scenario 2, higher than 2% .
Forecast on Ratio of Public Financing of Signed and
Planned BTO plus BTL Projects: Scenario 1 (selected)
Assumptions: 1. Calculation of amount of public financing of BTO: Based on agreements+Planned 2. Estimated period of BTL projects in Scenario 1: 10
years (3.8 tri.(2005), 7.3 tri.(2006), 5.5 tri.(2007), 5.0 tri.(2008) to be invested each year in 2009-2015) 3. Estimated government expenditure = (247 trillion
won in 2007, adjusted for 4% inflation)
Forecast on Ratio of Public Financing of Signed and
Planned BTO plus BTL Projects: Scenario 2
0.7%
0.6%0.5%
0.4%0.3%
0.2%0.1% 0.1%0.0%
0.8%0.8%0.9%0.9%0.9%1.0%1.0%1.0%1.0%
1.5%
1.8%
2.2%
1.5%
1.3%
1.6%1.6% 1.6% 1.6%
1.1%1.1% 1.1%
0.0%
0.5%
1.0%
1.5%
2.0%
2.5%
2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 2028 2029 2030 2031 2032 2033 2034 2035 2036
Ratio of Public Financing of BTO Ratio of Public Financing of BTL Combined Ratio of BTO and BTL
1.9%
1.7%1.5%
1.4%
1.2%
1.0%
0.7%
0.5%
0.3%
2.0%2.1%2.1%
2.2%2.2%2.3%
2.4%2.5%
2.5%
1.5%
1.8%
2.2%
1.5%
1.3%
1.7%
2.0%
2.3%
2.5%
2.2%
2.4%
2.6%
0.0%
0.5%
1.0%
1.5%
2.0%
2.5%
3.0%
2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 2028 2029 2030 2031 2032 2033 2034 2035 2036
Ratio of Public Financing of BTO Ratio of Public Financing of BTL Combined Ratio of BTO anc BTL
Source: Jay-Hyung Kim (2008)
Macroeconomic Fiscal Risk Management
![Page 14: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/14.jpg)
14
Simulation Results
Macroeconomic Fiscal Risk Management
The amount of public financing for signed and planned BTO projects
is not yet at a level that can affect the stability of fiscal management.
If the government carries out BTL projects for a limited amount of 5
trillion Won as in Scenario 1, there would be no fiscal problem.
However, if the government carries out BTL projects for more than a
decade or longer at a cost of annual 10 trillion won in Scenario 2, it
may harm the government fiscal sustainability.
Scenario 1 adopted due to recommendation and market condition
![Page 15: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/15.jpg)
Feasibility & Vfm test Part-04
![Page 16: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/16.jpg)
Phase 1: Feasibility study (Decision to Invest)
The cost- benefit analysis is conducted to determine feasibility of the project from a national economy perspective.
Phase 2: Value for Money Assessment (Decision on PFI)
The government payment of PSC (Public Sector Comparator) is compared against that of PFI (Private Finance Initiative) to decide whether the PFI achieves VfM.
Phase 3: Formulation of PFI alternatives
Based on the results of phase 2, an appropriate PFI alternatives is formulated.
The level of project cost, user fee, subsidy scale, etc. are suggested by the government.
Bonus points (10% Phase 4: Award bonus points to the initial proponent max.)
awarded to the initial proponent are estimated based on the results of VfM tests
and the quality of the proposal.
Feasibility & VFM Test
![Page 17: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/17.jpg)
17
Economic Feasibility
Estimate Benefit and Cost
Discount to PV
Ratio Analysis
Bench Marks (Comparable)
Benefit Estimation Cost Estimation
PV(Benefit) PV(Cost)
Social Discount Rate
CBR NPV IRR Payback
Period
![Page 18: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/18.jpg)
Guidelines for Pre-Feasibility Study and Value for Money (VFM) test
Guidelines for RFP preparation
Guidelines for Standard Output Specification by facility
Guidelines for Tender Evaluation
Guidelines for Standard Concession Agreement
Guidelines for Refinancing
Objectivity, Consistency and Independence
Implementations in Transparency, Objectivity and Consistency as well
as Professional expertise are key elements
Independent PIMAC has developed PPP Implementation Guidelines
and updated reflecting changes in market conditions
![Page 19: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/19.jpg)
Project Performance Evaluation in
Regular Basis
Part-05
![Page 20: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/20.jpg)
Performance evaluation
Issues in PPP facilities
Acceptable service quality
Operational and cost efficiency
Cooperation between competent authority and private investor
Evaluation procedure (SOP)
Evaluation Planning
Operation Data Collection
Data / Document Review
Field Evaluation User Satisfaction
Survey Reporting
& Feedback
Preliminary Evaluation Field Evaluation
![Page 21: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/21.jpg)
Performance evaluation
Evaluation Criteria
Degree of User Satisfaction : Safety, Customer Service, Kindness, etc.
Cooperation with Competent Authority : Feedback and Improvement,
Timely Reporting, VMS information support
Cost and Operational Efficiency : Operation Cost Management,
Organization Management, Public Relationship
Cost Efficiency
User Satisfaction
Cooperation Between Competent Authority and Project Company
Balanced Evaluation considering Trade-offs
![Page 22: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/22.jpg)
Performance Evaluation
For fair performance evaluation, the competent authority must form a performance evaluation committee consisting of government officials, the project company (SPC or operator), and experts of the relevant field.
The project company should be allowed to first submit a self-evaluation report, which is reviewed by the competent authority.
The performance evaluation committee can decide whether to conduct an additional independent evaluation by a third party.
Performance Check List
Performance evaluation
![Page 23: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/23.jpg)
User Satisfaction Survey
The satisfaction survey is conducted by each project company and submitted to the competent authorities.
It is taken yearly or quarterly as specified in the standard performance quality requirement.
Each evaluation item is given an evaluation grade(e.g., grade A-D) and then a score according to the grade. Weights are given to evaluation items to calculate final evaluation result.
Evaluation Criteria
Performance evaluation
![Page 24: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/24.jpg)
Performance evaluation
![Page 25: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/25.jpg)
Comprehensive Ex-post evaluation
-Macro level evaluation
-Ex-post VFM
-Stakeholder satisfaction survey
Part-04
![Page 26: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/26.jpg)
Welfare Assessment (Based on 14 road construction projects)
It is noted that it took 2 years less to complete 14 road construction projects
using PPP procurement compared to those with TIPs (Traditional Investment
Procurements) as of 2006.
Socio-economic benefits (i.e. travel time reductions) caused by early completion
of the projects are estimated about 1.45 billion USD as of 2006.
Early Realization of Socio-economic Benefits
(W billion)
Comprehensive Ex-post evaluation ; Macro-level evaluation
As of 2006 1 year delay 2 year delay
3 year delay
4 year delay
623 1,455 2,472 3,301
![Page 27: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/27.jpg)
Economic Growth Impact Assessment
The analysis of the ripple effects that the promotion of PPP projects could have
on the economy was conducted using KDI Macroeconomic Model.
Given that the government could have used the saved budget in other areas
thanks to the promotion of PPP projects, it is expected to have had the effect of
increasing the GDP by 0.198% in 2008.
Estimation of Growth Impact of PPP using the KDI Macroeconomic Model
Comprehensive Ex-post evaluation ; Macro-level evaluation
![Page 28: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/28.jpg)
Ex-post VFM Test (BTO projects)
Ex-post VFM tests for BTO projects, for which concession agreements were
signed as of 2008, show that ex-post VFM has increased compared to ex-ante
VFM due to savings in construction and operational costs.
Realized VFM Increase in BTO using Ex-post VFM Test (%)
In the case of 11
BTO projects, the
difference between
the ex-ante VFM
and ex-post VFM
figures shows an
additional VFM
increase of 16.32%
(W 142.5 billion).
Comprehensive Ex-post Evaluation ; Ex-post VFM Test
![Page 29: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/29.jpg)
Outline of survey (BTO projects)
In 2006, PIMAC of KDI began a survey of major stakeholders of BTO road projects currently in operation, such as competent authorities, project companies, and experts, about user satisfaction, project performance, and other issues.
To find out the level of user satisfaction, interviews and e-mail survey were conducted with considerable amount of users, and a face-to face survey was conducted with numerable public officials, project company employees, and experts related to BTO projects.
Outline of Survey on BTO Projects
Comprehensive Ex-post Evaluation ; Satisfaction Survey
![Page 30: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/30.jpg)
Perception about Infrastructure Expansion by Utilizing Private Capital
Do BTO roads shorten travel time compared to alternative roads?
Are the Toll Levels on BTO roads Appropriate?
Results of User Survey
The results of the survey show that different groups of stakeholders have different perceptions about the performance of BTO projects.
In the user survey, those who use BTO roads were found to be largely satisfied with the services despite the high tolls.
Although the BTO roads provide shorter travel time compared with alternative roads, the reduction of tolls appeared to be the most important task to increase user satisfaction levels.
Comprehensive Ex-post Evaluation ; Satisfaction Survey
![Page 31: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/31.jpg)
Satisfaction Level with the Performance
Results of survey of the project companies, competent authorities, and experts
While project companies and experts have a very positive perception of the results of BTO projects, competent authorities have a somewhat negative perception.
This can be attributed to the financial burdens caused by subsidies and MRGs and the additional administrative burdens from higher tolls and civil complaints.
Did the BTO projects improve VFM?
Are the Risks and Responsibilities properly distributed? Are BTO projects Attaining the Expected Benefits?
Comprehensive Ex-post Evaluation ; Stakeholder Satisfaction Survey
![Page 32: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/32.jpg)
Infrainfo DB System Part-11
![Page 33: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/33.jpg)
Infrainfo DB System
Development of DB System
The integrated DB system is essential for monitoring and evaluation.
Without the integrated DB system, the PPP data was managed by each
procuring authorities or municipal governments.
PIMAC began to develop integrated data base system (Infrainfo DB system) in
2011, and initiated the system in 2012.
![Page 34: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/34.jpg)
Min. of Strategy & Finance
PIMAC
Min. of Land, Transport & Maritime Affairs
Min. of Culture, Sports & Tourism
Min. of Education, Science & Technology
Min. of Environment
Min. of Health & Welfare
Min. of Natl. Defense, etc.
Procuring Ministries
DB system support
In the line of continuous monitoring process, all the status changes of PPP
projects should be reported to MOSF and the information be delivered to
information center for updates to system (Infrainfo System).
Infrainfo DB System
Ex-post management
Infra-info System
Project Implementation
Reporting
Information delivery
Information input
Support with expertise
Feedback
Comprehensive Ex-post Evaluation
Feedback
Continuous monitoring & improvement
![Page 35: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/35.jpg)
Intermediate & Ex-post Management Part-12
![Page 36: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/36.jpg)
Even with successful institutional settings of PPP, Korea is now
facing new controversial issues and challenges that need to be
resolved in order to move forward to advanced stage.
So far, most of the government's efforts focused on improving the PPP
procurement process from project initiation to the construction stage.
Relatively little attention has been paid to the ex-post operational phase.
PPP Projects typically involve significant investment by the private
sector over a long period of time, and substantial changes of the
business environment or policy objectives may cause dispute.
Disputes are costly, time-consuming and lead to a breakdown of the
project.
It is important to monitor and resolve conflict as quickly, efficiently and formatively
as possible.
Intermediate & ex-post management
![Page 37: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/37.jpg)
BTO BTL Q&A Total
2007 20 69 45 134
2008 29 170 101 300
2009 35 180 118 333
2010 44 104 143 291
Number of Inquiries and Consultancy by PIMAC
PIMAC is acting as an help desk for inquiry and consultancy;
Scope of Supporting Work : Reviews for request for proposals, concession
agreement, financial model, refinancing, Q&A
Intermediate & ex-post management
![Page 38: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/38.jpg)
PPP projects undergo concession change for various reasons
throughout the project period.
Main reasons for change of the concession agreement include:
(1) Change in toll fee
(2) Change in total project cost
(3) Change in corporate tax
(4) Change incurred by alteration of the PPP Act
(5) Change in investor and refinancing
(6) Change concerning port labor union policy
(7) Recalculation of government payment (by adjusting to macroeconomic shocks)
Intermediate & ex-post management
![Page 39: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/39.jpg)
Building capacity of government officials through training and education
to reduce occurrence of disputes
Training provides sound knowledge of PPP rules and regulations, comprehensive
and thorough understanding of standard. It improves communications with
contractors, stakeholders/users.
Revise Standard Concession Agreement (“SCA”) to provide detailed
guidelines for initial concession design. Fairness and transparency in
PPP projects, leading to reduce the disputes, can be enhanced by
developing SCA
Weakness of concession agreement result from their deficient design
Create a formal PPP dispute resolution committee within the government
to manage and assist operational PPP projects
The Committee must be neutral, unbiased and independent.
Preemptive efforts to monitor and reduce disputes
Intermediate & ex-post management
![Page 40: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/40.jpg)
A PPP Dispute Resolution Committee has been newly established
under the Minister of Strategy and Finance as a means to resolve any
dispute involving PPP projects.
Newly created as Article 44 of the revised PPP Act (November 2011)
The Committee must be neutral, unbiased and independent in order to administer mediation, viable alternatives to transnational arbitration or litigation.
The Committee is composed of no more than nine members including one chairperson, of whom one or more shall represent: ① the government, ② the project company, ③ the public interest.
The Committee shall submit a written draft of mediation within 90 days
from the date of request for dispute resolution. This may further extend
up to 60 days.
Dispute resolution committee
Intermediate & ex-post management
![Page 41: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/41.jpg)
Conclusion & Lessons learned Part-13
![Page 42: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/42.jpg)
Vfm test
Comprehensive ex-post evaluation
Macroeconomic fiscal risk management
Continuous Reporting
& Improvement
Monitoring and evaluation results are fed back to enhance the quality
of service and to ensure the macroeconomic sustainability.
Plays a very important role in achieving development goals!
BTO/ BTL Performance
evaluation
Infra-info DB
Intermediate & Ex-post
management
Conclusion & Lessons learned
![Page 43: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/43.jpg)
Project level
DB setup and update
Macro level
Conclusion & Lessons learned
Developing monitoring and evaluation system is not an one-off job! The system development itself is a never-ending journey.
In the early stages, the effort should focus on project level evaluation and monitoring.
As number of PPP projects accumulates and project level evaluation system matures, the focus may move forward to comprehensive (macro) level.
The project information should continuously be reported to control tower and updated to a database, which is an important asset for next step.
The journey of monitoring and
evaluation system development
• Feedback and system improvement • Dispute resolution by cases
![Page 44: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/44.jpg)
Conclusion & Lessons learned
Trade-off attributes should be carefully considered to achieve the goal
of development when designing evaluation and monitoring criteria
The quality of facility can be represented by user satisfaction survey
The cost efficiency can be measured by ex-post VFM
VFM
User
Satisfaction
Evaluation drive direction 1
1) Choice with more priority to quality
2) Choice with more priority to cost savings
Evaluation drive direction 2
![Page 45: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/45.jpg)
Institution with independence and capable expertise is a prerequisite
for monitoring and evaluation Professional expertise is needed to setup Standard Operation Procedure of
evaluation and monitoring
Operational experience and fiscal policy direction should be reflected in developing Evaluation Criteria, which can’t be the same for all PPP facilities, and thus properly revised for the practical application
The evaluation report should be reliable and independent so that third party audience can trust
PPP is not a free gift! It may shift burden to next generation.
The liability should be properly measured and controlled to keep fiscal soundness.
Monitoring and evaluation itself is not enough! The results should be duly reflected into future projects or PPP system.
With monitoring, various field issues and disputes should be properly and timely resolved to make projects moving forward.
Conclusion & Lessons learned
![Page 46: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/46.jpg)
Conclusion & Lessons learned
Preventive efforts for continuous field monitoring and problem
solving are by far more important than ex-post evaluation.
Continuous field monitoring system can help solving problems in a timely manner.
Every stakeholder should recognize themselves as project improver through monitoring and suggestion.
To achieve this goal, education is one of most critical factors for success.
Field monitoring
Reporting Ex-post
Evaluation
Feedback with incentive or penalty
Problem Solving
Education
![Page 47: M&E in PPPs · 2014-01-17 · Management Center (PIMAC), Korea Development Institute Nov. 2013 PIMAC Korean Evaluation System - M&E in PPPs - ... Monitoring & Evaluation by Project](https://reader033.vdocument.in/reader033/viewer/2022041815/5e5a28f771d21b380f5b7e48/html5/thumbnails/47.jpg)