partnership for growth

PARTNERSHIP FOR GROWTH

THE PHILIPPINES – UNITED STATES (2011–2015) Mid-Term Evaluation Final Report

February 2015

February 4, 2015

Final Report For


MID-TERM EVALUATION

PHILIPPINES

Optimal Solutions Group, LLC

Christabel Dadzie

Peter Murrell

Tiernan Mennen

Mark Turner

Table of Contents

Table of Contents ........................................................................................................................................ 3

List of Figures .............................................................................................................................................. 8

Acknowledgements ..................................................................................................................................... 9

List of Acronyms ....................................................................................................................................... 10

Executive Summary .................................................................................................................................. 12

Purpose of Evaluation ............................................................................................................................. 12

Methodology ........................................................................................................................................... 13

Findings, Conclusions, and Recommendations ...................................................................................... 14

1. Introduction ....................................................................................................................................... 19

1.1 Background and Context of the PFG Initiative ................................................................................. 19

1.2 Background of the Philippines PFG Initiative ............................................................................ 19

1.3 Philippines PFG Joint Country Action Plan (JCAP)................................................................... 20

1.4 Purpose and Scope of the Evaluation .......................................................................................... 20

1.5 Evaluation Questions .................................................................................................................. 21

Terms Used in the Evaluation Report ................................................................................................. 21

2. Methodology ...................................................................................................................................... 23

2.1 Methods of Data Collection and Data Analysis ................................................................................ 23

2.2 Data Analysis .................................................................................................................................... 25

2.3 Evaluation/ Study Limitations .......................................................................................................... 26

3. General Evaluation Findings ........................................................................................................... 28

3.1 Overall Advantages and Disadvantages of the PFG ......................................................................... 29

3.2 Conclusions and Lessons Learned .................................................................................................... 32

3.3 Course Corrections and Recommendations ...................................................................................... 33

4. Cross-Cutting Question 1: What are the advantages and/or disadvantages of the PFG whole-of-government approach to development assistance? ........................................................................... 34

4.1 Findings............................................................................................................................................. 35



5. Cross-Cutting Question 2: To what extent has the Partnership for Growth affected the workload on national government and U.S. government staff, as compared to the workload created by traditional forms of development assistance delivery? ..................................................................... 41

5.1 Findings............................................................................................................................................. 41

5.2 Conclusions and Lessons Learned .............................................................................................. 45

5.3 Course Corrections and Recommendations ................................................................................ 46

6. Cross-Cutting Question 3: What contributions has “non-assistance” made to the PFG process, and how can it be utilized moving forward? .......................................................................................... 47

6.1 Findings............................................................................................................................................. 47



7. Country-Specific Question 1: For each of the constraints, are the goal-level commitments set forth in the JCAP alone capable of achieving the constraint-level objectives and outcomes? ........... 53

7.1 Findings............................................................................................................................................. 53



8. Country-Specific Question 2: Is quantitative and objectively verifiable information being used to manage JCAP implementation in order to achieve and measure results? ...................................... 60

8.1 Findings............................................................................................................................................. 61



9. Country-Specific Question 3: At the midterm, are the performances of the selected PFG interventions on target and creating the necessary outputs to achieve the desired outcomes? ......... 70

9.1 Findings............................................................................................................................................. 71



ANNEXES ............................................................................................................................................... 100

Annex 1: Evaluation Statement of Work .............................................................................................. 100

Annex 2: Work Plan ............................................................................................................................... 121

Annex 3: Advantages and Disadvantages for Data Collection Methods for this Evaluation ........... 122

Annex 4: Updated Methodology and Evaluation Plan Report ........................................................... 125

List of Acronyms ..................................................................................................................................... 127

A. Project Purpose ............................................................................................................................... 128

i. Partnership for Growth.................................................................................................................. 128

ii. PFG Mid-Term Evaluation ........................................................................................................... 128

iii. The Updated Methodology and Evaluation Plan for PFG the Philippines ................................... 129

B. Constraints to the Philippines’s Development at Baseline .......................................................... 130

i. Constraint 1 “Regulatory Quality”: Domestic Investment Climate .............................................. 130

ii. Constraint 2 “Rule of Law and Anti-Corruption Measures”: Governance ................................... 136

iii. Constraint 3 “Fiscal Performance”: Fiscal Space ......................................................................... 139

iv. Analysis of Data Collected with Semi-structured Interviews ....................................................... 142

v. Analysis of Interview Data ........................................................................................................... 144

vi. Analysis of Data Collected with Confidential Online Surveys ..................................................... 144

vii. Cross-cutting Questions ............................................................................................................ 146

Respondent Characteristics ............................................................................................................... 146

Advantages and Disadvantages of PFG’s WGA, CCQ1 .................................................................. 146

Changes in Workload of National and USG Personnel as a Result of PFG, CCQ2 ......................... 147

Non-assistance (CCQ3)..................................................................................................................... 147

viii. Country-specific Questions ....................................................................................................... 147

Quantitative information used to manage JCAP implementation (CSQ2) ....................................... 147

ix. Pre-Field Visit Data Needs and Analytical Guide ........................................................................ 148

Cross-cutting Questions .................................................................................................................... 148

Philippines Country-Specific Questions ........................................................................................... 149

Annex 1: Supporting Tables for Constraint-Level Indicators ............................................................ 151

Annex 2: Table Shells for Confidential Online Survey Results .......................................................... 155

Annex 3: Confidential Online Surveys ................................................................................................. 163

Annex 4: Interview Guides ................................................................................................................... 177

Stakeholder Types ................................................................................................................................. 177

Semi-structured Interviews ................................................................................................................... 178

Annex 5: List of References.................................................................................................................... 203

List of Tables

Table 2.1: Stakeholder interview information ............................................................................................ 24 Table 2.2: Semi-structured interview guides by research question and respondent type .................. 24 Table 2.3: Web-based survey by response rate and government affiliation ........................................ 24 Table 4.1: In your opinion, has the WGA led to change in the way the USG delivers development assistance in Philippines? ............................................................................................................................ 36 Table 4.2: Have any changes led to operational efficiency in project design and implementation? .......... 37 Table 5.1: Respondents by gender .............................................................................................................. 44 Table 6.1: What contribution (if any) has non-assistance made to the PFG process, in relation to the Philippines? ................................................................................................................................................. 47 Table 7.1: USG and GPH interviewee responses about JCAP importance and familiarity with JCAP ..... 54 Table 7.2: USG and GPH interviewee responses on whether the JCAP is out of date ............................... 54 Table 8.1: PFG implementation: Summary status of USAID project M&E plans ..................................... 65 Table 8.2: Implementers actively using an M&E plan ............................................................................... 66 Table 8.3: Implementers evaluated by indicators in M&E ......................................................................... 67 Table 9.1: Individual responses about the status of PFG-selected interventions at mid-term .................... 71 Table 9.2: PFG-selected goals and projects ................................................................................................ 73 Table 9.3: Status of projects and activities under Goal 1: Facilitate Trade and Investment ....................... 75 Table 9.4: Status of projects and activities under Goal 2: Reduce Regulatory Bottlenecks, Entry Barriers, and Discriminatory Provisions to Investment ............................................................................................. 81 Table 9.5: Status of projects and activities under Goal 7: Support efforts to improve judicial efficiency . 84 Table 9.6: Activities defined by the JCAP to be implemented as part of Goal 7 ....................................... 87 Table 9.7: Status of tasks, activities, and deliverables under Goal 8: Strengthen anti-corruption institutions (USAID i3Project and USDOJ) .................................................................................................................. 89 Table 9.8: Status on projects and activities under the tax effort strengthened goal for the FPI project ...... 93 Table 9.9: Overall progress according to the PFG Philippines scorecard ................................................... 94 Table 9.10: Facilitate trade and investment scorecard data ........................................................................ 95 Table 9.11: Status of weak governance constraint according to scorecard ................................................. 96 Table 9.12: Scorecard data for fiscal space constraint ................................................................................ 97 UMEP Report Annex Table 1: Constraint-level Indicators, Constraint 1 .................................................................................... 130 Table 2: Constraint-level Indicators, Constraint 2 .................................................................................... 136 Table 3: Constraint-level Indicators, Constraint 3 .................................................................................... 139 Table 4: Semi-structured Interview Guides by Research Question .......................................................... 144 Table 5: Online Survey Items by Research Question ............................................................................... 144 Table 6: Cross-cutting Questions .............................................................................................................. 148 Table 7: The Philippines Country-Specific Questions ......................................................................... 149 Table 8: Foreign direct investment, net inflows (% of GDP) ................................................................... 151 Table 9: Gross capital formation (% of GDP) .......................................................................................... 151 Table 10: Global Competitiveness Index, Rank ....................................................................................... 151 Table 11: Ease of Doing Business Rank ................................................................................................... 151

Table 12: Starting a Business, Rank ......................................................................................................... 152 Table 13: Government Effectiveness, Percentile Rank............................................................................. 152 Table 14: Rule of Law, Percentile Rank ................................................................................................... 152 Table 15: Control of Corruption, Percentile Rank .................................................................................... 153 Table 16: Regulatory Quality, Percentile Rank ........................................................................................ 153 Table 17: Tax revenue (% of GDP) .......................................................................................................... 153 Table 18: General government net lending/borrowing ............................................................................. 153 Table 19: Total investment ....................................................................................................................... 154

List of Figures

Figure 3.1: Did the PFG bring changes, as compared with other approaches to development assistance intended to affect economic growth? .......................................................................................................... 29 Figure 3.2: Is the PFG meeting its goal of advancing economic growth in the Philippines? ..................... 29 Figure 5.1: Change in workload resulting from PFG involvement: USG and GPH ................................... 42 Figure 5.2: Change in workload resulting from the PFG by task ............................................................... 43 Figure 5.3: Change in workload resulting from the PFG by ask – USG and GPH ..................................... 43 Figure 5.4: Changes in workload by gender ............................................................................................... 44 Figure 5.5: Perceived effectiveness of the PFG compared with traditional development assistance approaches................................................................................................................................................... 45 Figure 6.1: Survey responses regarding whether non-assistance is being used in the PFG Philippines Initiative ...................................................................................................................................................... 48 Figure 6.2: Have you seen non-assistance tools being used in the PFG activity with which you are or were involved? (Based on El Salvador evaluation survey responses only) ......................................................... 49 Figure 6.3: Examples of non-assistance in the Philippines PFG Initiative, provided by interview and survey respondents ...................................................................................................................................... 49 Figure 8.1: Are the appropriate indicators being used to allow for transparent, accountable, and fact-based monitoring of the PFG? (Based on survey responses only) ........................................................................ 64 Figure 8.2: Are the appropriate indicators being used to allow for transparent, accountable, and fact-based monitoring of the PFG? (Based on survey responses only) ........................................................................ 64 UMEP Report Annex Figure 1: Foreign direct investment, net inflows (% of GDP) .................................................................. 132 Figure 2: Gross capital formation (% of GDP) ......................................................................................... 132 Figure 3: Global Competitiveness Index, Rank ........................................................................................ 133 Figure 4: Ease of Doing Business Rank .................................................................................................... 134 Figure 5: Starting a Business, Rank .......................................................................................................... 135 Figure 6: Government Effectiveness, Percentile Rank ............................................................................. 136 Figure 7: Rule of Law, Percentile Rank .................................................................................................... 137 Figure 8: Control of Corruption, Percentile Rank ..................................................................................... 138 Figure 9: Regulatory Quality, Percentile Rank ......................................................................................... 139 Figure 10: Tax revenue (% of GDP) ......................................................................................................... 140 Figure 11: General government net lending/borrowing (% of GDP)........................................................ 141 Figure 12: Total investment (% of GDP) .................................................................................................. 141

Partnership for Growth Initiative Mid-term Evaluation Report for The Philippines 9

Acknowledgements

The development and success of this evaluation was a result of a collaborative effort between the authors and many other people, who we would like to acknowledge. First, a depth of gratitude goes to the USAID points of contact in Washington D.C., Ryder Rogers and Brian Levey who reviewed the reports and provided valuable feedback to the team. Special thanks goes to our points of contact in the Philippines, Gerald Briton and Gil Dy-Liacco, who spent many hours providing documentation, responding to questions of the evaluators, and organizing meetings, including high level meetings and site visits, for the evaluation to be successful.

We are very grateful for the time, information, and insights provided by our interviewees, including government officials, implementers, and independent experts, whose contributions are at the heart of this evaluation.

The authors are also grateful to the support team at Optimal –Joanne Perodin and Omotunde Okesanya for their tireless efforts on this evaluation, and to the hard work of our local consultants, Aubrey Bualat-Trinidad and Kenmore Espinoza whose local knowledge expertise and contributions were instrumental to the success of the evaluation.


List of Acronyms

AEC

ASEAN

CA

COR

CSO

DOF

DOJ

ASEAN Economic Community

Association of South East Asian Nations

Constraints Analysis

Contracting Officer’s Representative

Civic Society Organization

Department of Finance

Department of Justice

ED United States Government Economic Department

FDI Foreign Direct Investment

FPI

GPH

Facilitating Public Investment

Government of the Philippines

GDP Gross Domestic Product

I3 Integrity for Investments Initiative

INL International Narcotics and Law Enforcement Affairs

INVEST

IP

JCAP

Investment Enabling Environment

Intellectual Property

Joint Country Action Plan

LOAs Lines of Action

M&E Monitoring and Evaluation

MCC Millennium Challenge Corporation

NEDA

NGO

National Economic Development Authority

Non-Governmental Organization

NSW National Single Window

OPDAT

OTA

PFG

Office of Overseas Prosecutorial Development, Assistance & Training

U.S. Treasury Office of Technical Assistance

Partnership for Growth

PDP GPH’s Philippines Development Program

RKC Revised Kyoto Convention

SIMM Scaling Innovations in Mobile Money


SOW Statement of Work

STRIDE Science, Technology, Research and Innovation for Development

TOC Theory of Change

TPP

UMEP

Trans-pacific partnerships

Updated Methodology and Evaluation Plan

U.S. United States

USAID United States Agency for International Development

USG United States Government

USTR United States Trade Representative

WCO

WGA

WTO

World Customs Organization

Whole of Government Approach

World Trade Organization


Executive Summary

The Partnership for Growth (PFG) aims to achieve accelerated, sustained, and broad-based economic growth in partner countries, including El Salvador and the Philippines, through bilateral agreements between the United States Government (USG) and the partnering countries’ national governments. Using principles set forth in President Barack Obama’s September 2010 Presidential Policy Directive on Global Development, PFG requires rigorous, joint analyses of countries’ individual constraints to growth to develop joint action plans to address the most pressing of these constraints and to establish high-level mutual accountability for the goals and activities selected to alleviate them.

The countries of El Salvador, the Philippines, Tanzania, and Ghana were selected as the first group of countries in which the United States (U.S.) and partner governments would attempt to structure new PFG initiatives, with selection based, in part, on each country’s record of accomplishment in implementing ongoing Millennium Challenge Compacts (MCC).

In November 2011, USG and the Government of the Philippines (GPH) signed a Joint Country Action Plan (JCAP) for a 5-year period for implementing two primary constraints, each with three sub-constraints, to Philippine economic growth. Each constraint was accompanied by a set of goals, policies, and actions to relieve them. The agreed-upon actions are designed to lead to goal and policy achievement, which, in turn, will mitigate the effects of currently binding constraints and, therefore, accelerate and sustain the Philippines’ rate of broad-based, inclusive economic growth.

Purpose of Evaluation

Optimal Solutions Group, LLC (Optimal) was contracted to conduct a mid-term evaluation of the PFG initiative in the Philippines. According to the evaluation statement of work (SOW), this evaluation serves two purposes.

First, the evaluation analyzes whether the PFG process demonstrates improvements over pre-PFG assistance approaches. In particular, the evaluation team examined the extent to which PFG’s whole-of-government approach (WGA) and constraints analysis (CA) led to a change in the way USG delivered development assistance.

The second purpose of the evaluation consists of two parts:

(i) determining whether PFG efforts “have been developed in such a way as to allow for the eventual determination of their impact in addressing the identified constraints and desired outcomes.”

(ii) determining whether “measurable elements of PFG are moving in the right direction, are considered necessary and sufficient to achieve PFG goals, and are contributing to national interests through the integration and coordination of work done by both governments.”1

The evaluation was guided by the following six evaluation questions as outlined in the SOW.2

1 Evaluation Statement of Work, 2013. pgs. 7–8


Cross-Cutting Questions

1) What are the advantages and/or disadvantages of the PFG whole-of-government approach to development assistance?

2) To what extent has PFG affected the workload on national government and U.S. government staff, as compared to the workload created by traditional forms of development-assistance delivery?

3) What contributions has “non-assistance” made to the PFG process, and how can it be utilized moving forward?

Country-Specific Questions

1) For each of the constraints, are the goal-level commitments set forth in the JCAP capable of achieving the constraint-level objectives and outcomes?

2) Is quantitative and objectively verifiable information being used to manage JCAP implementation in order to achieve and measure results?

3) At the midterm, are the performances of the selected PFG interventions on target and creating the necessary outputs to achieve the desired outcomes?

Methodology

The evaluation team employed a mixed-methods approach for data collection and analysis of the Philippines PFG activities. The study included an extensive desk review of all available project documentation and monitoring data; an evaluability assessment and selection of goals for an in-depth case study; interviews with PFG architects3 and leadership in Washington, D.C.; a site visit to the Philippines to conduct individual semi-structured, face-to-face interviews; telephone interviews with stakeholders who were not available in person; and an online survey administered to USG and GPH PFG staff.

The evaluation team conducted a qualitative research evaluation, which reviewed, collated, and analyzed the perceptions gathered. Using a mixed-methods approach, quantitative approaches were also used to analyze and verify information provided by PFG in situations when documentation and data were available for such analysis. The evaluation team employed triangulation to verify findings from various vantage points. Upon completion of the analysis, the team developed this report, which details the findings and conclusions, and provides recommendations and course corrections that could be employed by the Philippines PFG partners at midterm. These findings, conclusions, and recommendations are provided in response to the six evaluation questions specified in the SOW.

2 Evaluation Statement of Work, 2013. pgs. 9–10. 3 Architects refer to individuals who were part of designing and mapping out the overall PFG initiative, which led to the selection of the four countries that then initiated and implemented their own initiatives. See annex 4 (Stakeholder Types) of this report.


Findings, Conclusions, and Recommendations

Overall Advantages and Disadvantages of PFG

In general, the evaluation team found that PFG has aided in the positive movement of policy reforms in the Philippines. The initial PFG planning through the CA activity laid a good foundation for implementing the overall initiative. Additionally, PFG increased leverage for implementing USG development activities in the Philippines.

Cross-Cutting Question 1: What are the advantages and/or disadvantages of the PFG Whole-of-Government Approach to development assistance?

Findings – Advantages of the PFG WGA within USG and GPH PFG Staff

• The CA and JCAP development framed WGA dynamics. • WGA has led to a change in the way USG programs’ development assistance is conducted in the

Philippines. • WGA has focused human and operational resources to increasing consistency and coherence in

the programming and policy objectives of PFG. • WGA has improved coordination between USG and GPH. • WGA has led to a more consistent USG message on key GPH reforms.

Challenges/Disadvantages of the PFG/WGA

• The number of agencies to which the WGA is relevant in terms of their daily activities and decision-making is highly limited.

• There has been some dissatisfaction because of a perceived delay of PFG project implementation. • There is confusion about decision-making authority within the PFG system, limiting WGA

implementation. • WGA requires greater staff time to be effective.

Conclusions

• PFG WGA represents a positive change in development assistance compared with previous approaches.

• WGA is most likely to be effective if purposefully incorporated into the design of the initiative and backed by high-level leadership.

• WGA promoted collaboration within and across governments. • WGA coordination is time-consuming.

Recommendations

• Provide stronger mechanisms for inter-goal collaboration through a management team. • Develop performance measures and collect better data to track the coordination of activities in

promoting effectiveness. o Develop a set agenda for coordination meetings. o Introduce accountability measures.


o Identify and inform staff and key stakeholders of responsibilities for the overall management of PFG coordination.

o Establish (and update) an overarching work plan to track progress on coordination. o Identify and track activities or actions that demonstrate coordination.

Cross-Cutting Question 2: To what extent has Partnership for Growth affected the workload of national government and U.S. government staff, as compared to the workload created by traditional forms of development-assistance delivery?

Findings

• PFG has resulted in an increase in the workload of USG and GPH staff. • PFG has prompted a significant increase in workload focused on coordination and

communication, both intra- and inter-governmental. • There was no discernible difference in workload by gender. • Increased workload is associated with perceived effectiveness of PFG and WGA.

Conclusions

• PFG has unambiguously increased the workloads of both USG and GPH staff.

Recommendations

• Identify management staffing that could assist in reducing the workload of PFG staff across the initiative.

• Identify and promote the use of limited resources more efficiently to reduce the overall workload.

Cross-Cutting Question 3: What contributions has “non-assistance” made to the PFG process, and how can it be utilized moving forward?

Findings

• The concept of non-assistance and examples of its application is unclear to many PFG stakeholders (particularly lower-level technical staff). Knowledge about non-assistance is varied.

• Whereas non-assistance likely facilitated broader reforms in the Philippines, few specific positive results can be directly attributed to these efforts.

• Aligning USG foreign policy goals with development goals through the WGA has the potential to increase the impact of “non-assistance” and to advance those goals, but it might also reduce the effectiveness of development aid.

• Not enough is known about the business opportunities created by PFG and the value of changing public perception as a result of PFG non-assistance activities.

Conclusions

• The lack of familiarity of USG and GPH program managers and implementers with the concept of non-assistance may mean that opportunities to employ non-assistance will be missed.


• Embassy leadership, particularly within the United States Agency for International Development (USAID), provided the clearest example of the successful use of non-assistance in the Philippines.

• The value of promoting PFG goals, attained and pending, is not always evident.

Recommendations

• PFG participants need to become more knowledgeable about the diversity of forms non-assistance takes and its value should be made known to a broad array of beneficiaries.

• Develop a management and reporting system for promoting non-assistance activities. • Improve reporting and public awareness on non-assistance activities. • Increase staffing whose assignments are geared toward identifying and working on non-assistance

activities.

Country-Specific Question 1: For each of the constraints, are the goal-level commitments set forth in the JCAP capable of achieving the constraint-level objectives and outcomes?

Findings

• The JCAP for the Philippines PFG is outdated and has not been updated in response to implementation changes, even though it is supposed to be a living document.

• An overall formal theory of change linking activities, goals, and constraints was not fully developed.

• Additional goals and activities were included in the Philippines JCAP that were not emphasized in the CA.

• A number of JCAP goals were not being fully implemented. • During the development of the JCAP, there were proactive efforts to absorb the views of a broad

cross-section of society; nevertheless, there were differences of opinion on whether the constraints analysis and the JCAP development process were inclusive.

Conclusions

• Without a fully articulated overall theory of change, it is not possible to provide a theoretically grounded assessment of whether or not achieving goals included in the JCAP can have the desired effect on constraint-level objectives and outcomes.

• In lieu of a theory of change, it is difficult to determine whether the JCAP goals were chosen partially based upon bureaucratic considerations, organizational convenience, or the judgments of influential decision makers.

• The ambitious goals in the JCAP are unlikely to be accomplished given the resources currently available for PFG activities.

• Even though broad and ambitious, the JCAP did not satisfy some influential elements of society (e.g., civil society, independent experts, business community).

Recommendations

• Clarify the status of the JCAP and update components of the guiding document as necessary. • Better align the JCAP with available resources.


• Be more proactive in involving civil society and the private sector at the implementation stage.

Country Specific Question 2: Is quantitative and objectively verifiable information being used to manage JCAP implementation in order to achieve and measure results? Findings

• The implementation of the JCAP—viewed as a comprehensive integrated program—is not being adequately managed.

• The monitoring and evaluation (M&E) framework required by the JCAP, which would ensure standardized M&E across the PFG initiative, has not been finalized.

• The M&E plans of individual USAID projects are comprehensive, complete, and ready to be used to manage activities, but have only recently been finalized, so they are not in use for projects that are already underway.

• Quantitative information is being collected at the project and activity level, but it is not (yet) being assembled, analyzed, and used to manage JCAP implementation.

• Individual activities have M&E plans and indicators, but these are not mapped to the overall M&E of PFG implementation.

Conclusions

• JCAP implementation—viewed as a comprehensive integrated program—is not being managed. • Since the overall M&E framework required by the JCAP has not been finalized, there is no

explicit framework within which overall JCAP implementation might be monitored and evaluated.

• At the individual project level, during the time of the evaluation team’s fact-finding in-country, it would have been difficult to authoritatively use quantitative and objectively verifiable information for management because the M&E plans of most individual USAID projects had not been finalized at that point. However, recent approvals should aid the program in moving forward in terms of M&E.

• Although quantitative information is being collected at the project level, it is not (yet) being used to manage JCAP implementation.

Recommendations

• A decision should be made about whether PFG JCAP implementation should be managed as a single program.

• PFG staff and implementers should be provided with guidance on how the detailed indicators in project M&E plans are to be used as quantitative management tools.


Country-Specific Question 3: At the midterm, are the performances of the selected PFG interventions on target and creating the necessary outputs to achieve the desired outcomes?

Findings

• Stakeholders’ perceptions varied in terms of whether PFG is on target. Many stated that government-to-government negotiations, collaboration on project design, and procurement timelines delayed the initiation of projects and activities.

• Implementation at the project level is generally on target, given when projects were initiated. • Most USG staff feel that PFG is on target. Most GPH staff cite delays in procurement. • The annual PFG scorecards indicate macro-level progress in the Philippines that is less indicative

of PFG progress. • PFG is not far enough along at this point to effectively measure progress.

Conclusions

• The PFG process takes time, particularly if initiatives are starting from scratch, as was the case in the Philippines.

• Monitoring systems for the PFG Philippines initiative need to be augmented rapidly in order to accurately measure progress toward PFG objectives.

Recommendations

• The PFG Philippines management team should define “midterm” and “on target” under PFG as these apply to each particular country and context, taking into consideration the analytical, consensus-building, design, and joint-implementation process.

• PFG Philippines leadership should facilitate the rapid completion of a comprehensive M&E framework that links outputs to outcomes for all JCAP goals, and also appropriately links these goals to the constraints.

• Reporting and monitoring from guidelines provided in one overarching standardized M&E framework needs to begin and occur regularly at the technical sub-committee level for each constraint, with regular reporting and reviews of all constraints by the steering committee.

• Conduct an annual strategic review of PFG progress across USG and GPH with a pre-established date, both as a means to review progress and to incentivize implementation.

Overall Summary and Conclusion

Overall, the evaluation team found the PFG Philippines initiative to have made progress in developing a true partnership, where both governments have a seat at the table in decision making. PFG has led to a complete shift in how development assistance is performed in the Philippines and has led to positive policy reforms, with perceptions of improved economic growth. However, the absence of an explicit M&E framework within which overall JCAP implementation is monitored and evaluated has made it a challenge to determine the performance of PFG in the Philippines at midterm. Updating the overall monitoring system will aid in appropriate tracking of the initiative, and, eventually, will provide the ability to determine the impact of PFG,


1. Introduction

1.1 Background and Context of the PFG Initiative

The Partnership for Growth (PFG) aims to achieve accelerated, sustained, and broad-based economic growth in partner countries, including the Philippines, El Salvador, Ghana, and Tanzania, through bilateral agreements between the United States Government (USG) and the partnering countries’ national governments. Using principles set forth in President Barack Obama’s September 2010 Presidential Policy Directive on Global Development, the PFG requires rigorous, joint analyses of countries’ individual constraints to growth, joint action plans to address the most pressing of these constraints, and high-level mutual accountability for the goals and activities selected to alleviate them.

The countries of El Salvador, the Philippines, Tanzania, and Ghana were selected as the first group of countries in which the United States and partner governments would attempt to structure new PFG initiatives with selection based, in part, on each country’s record of accomplishment in implementing ongoing Millennium Challenge Compacts.

1.2 Background of the Philippines PFG Initiative

Despite improved economic growth in recent years, domestic investment is meager and the Philippines’ Gross Domestic Products (GDP) continues to decline. In comparison to its East and Southeast Asian neighbors, development performance has not been outstanding. In addition, the country has experienced a growing trade deficit since 20014, and although the economy grew by 7.6 percent in 2010, growth almost halved to 3.9 percent in 2011. Economic performance is reflected in the increasing levels of poverty and underemployment that remain a challenge for the Philippines, with 42 million Filipinos surviving on less than $2 a day5. The main issues affecting the economic development of this region are: widespread corruption, and weak legal and regulatory systems. Additional issues include relatively low levels of foreign direct investment (FDI), which reflect deficiencies in the investment environment; low marks for infrastructure, quality of public institutions, and corruption. Consequently, about 10 percent of Filipinos work abroad due to lack of job opportunities at home6.

To achieve further improvements, the GPH is committed to sustainable growth in economic development and poverty reduction schemes.

Under the government of President Benigno S. Aquino III, a new order of rule, under the “Social Contract with the Filipino People,” is being instituted. This contract articulates a commitment to transformational leadership, institutional reform, economic stability, and inclusive growth. In line with this contract, the Philippine Development Plan (PDP) 2011-2016 is addressing key constraints to growth in the Philippines while focusing on broad issues of good governance and anti-corruption. Specifically, the PDP was

4 Philippines Balance of Trade. Trading Economics. June 2014. (http://www.tradingeconomics.com/philippines/balance-of-trade) 5 Poverty headcount ratio at $2 a day (PPP) (% of population). World Bank, Development Research Group. June 2014. (http://data.worldbank.org/indicator/SI.POV.2DAY) 6 “Built on Dreams, Grounded in Reality: Economic Policy Reform in the Philippines” <http://asiafoundation.org/resources/pdfs/FRONT.pdf>

http://www.tradingeconomics.com/philippines/balance-of-trade

http://data.worldbank.org/indicator/SI.POV.2DAY

http://asiafoundation.org/resources/pdfs/FRONT.pdf


designed to address poverty, create employment opportunities, and achieve the GPH’s vision of inclusive growth.

1.3 Philippines PFG Joint Country Action Plan (JCAP)

In November 2011, the GPH and USG signed a 5-year commitment to undertake the PFG demonstrated through a JCAP, with the period of performance from 2011 through 2016. The development of the JCAP was to provide an implementation strategy based on the findings from analyzing and defining the two main binding constraints to growth in the Philippines – weak governance and narrow fiscal space. These were further grouped into sub-constraints, goals, activities and projects, in the PFG initiative. These findings also align with the targets of the GPH’s PDP, discussed previously.

The Philippines’ PFG performance is tracked through indicators. These indicators measure three inter-related development interventions namely (referred to as sub-constraints for this evaluation): Regulatory Quality, Rule of Law and Anti-Corruption, and Fiscal Space/Performance. To ensure that the JCAP is being implemented effectively and that the desired outcomes are being achieved, the program is subject to a rigorous, transparent monitoring and evaluation (M&E) process. Using an evidence-based M&E framework, the GPH and USG are to work together to identify targets and track progress on macro and sectorial-level indicators to best demonstrate performance against benchmarks. This framework is expected to cover the following:

1. Process for selecting the benchmarks and indicators associated with each objective; 2. Relevant entity tasked with collecting M&E data to present to the Steering Committee (Technical

Sub-Committees, an office within USAID, or the PFG Secretariat); and 3. Expectations or plans to collect M&E information from all USG agencies and the GPH.

This process should include meaningful participation by civil society and the private sector, and approved by the PFG Steering Committee. The framework will be drafted by April 2011 and approved by the PFG Steering Committee. This evaluation explores the progress of the Philippines PFG initiative to inform stakeholders in alignment with the statement of work (SOW) discussed in detail subsequently.

1.4 Purpose and Scope of the Evaluation

The evaluation of the PFG initiative in the Philippines serves two purposes, per the statement of work (SOW). First, the evaluation will determine whether the PFG process demonstrates improvements over pre-PFG assistance approaches. Second, the evaluation will determine if PFG efforts were “developed in such a way as to allow for the eventual determination of their impact in addressing the identified constraints and desired outcomes,” and whether or not measurable efforts are “moving in the right direction, are considered necessary and sufficient to achieve PFG goals, and are contributing to national interests through the integration and coordination of work done by both governments.”


1.5 Evaluation Questions

As outlined in the evaluation SOW,7 six principal questions guided the evaluation. The evaluation team also developed sub-questions, as appropriate, which further probed specific topics to evoke in-depth responses to the evaluation questions. The six questions are listed below.

Cross-Cutting Questions

4) What are the advantages and/or disadvantages of the PFG whole-of-government approach (WGA) to development assistance?

5) To what extent has the PFG affected the workload of national government and U.S. government staff as compared with the workload created by traditional forms of development-assistance delivery?

6) What contributions has “non-assistance” made to the PFG process, and how can it be utilized moving forward?

Country-Specific Questions—Philippines

4) For each of the constraints, are the goal-level commitments set forth in the JCAP capable of achieving the constraint-level objectives and outcomes?

5) Is quantitative and objectively verifiable information being used to manage JCAP implementation in order to achieve and measure results?

6) At the midterm, are the performances of the selected PFG interventions on target and creating the necessary outputs to achieve the desired outcomes?

Per the SOW, the findings and conclusions of the evaluation will:

• help decision makers determine whether PFG indicates an improved model for providing assistance and whether it portends a higher probability of achieving desired development results.

• tangible input to the national government and USG entities for identifying obstacles and optimizing PFG implementation in the field, allowing for country program course corrections where feasible and needed in order to enhance the likelihood of achieving sustainable, cost-effective, and measurable results.8

Terms Used in the Evaluation Report

For the purpose of this evaluation, the team is using the following terms, in order to align with the requirements in the SOW:

• Constraints – these refer (as referenced in the JCAP and SOW) to the two main constraints that have been identified by the Philippines to be tackled in order to remove economic constraints

7 PFG Mid-term Evaluation Statement of Work, pgs. 7–8. 8 Ibid.


• Sub-constraints – these refer to the three topic areas that have been identified beyond the two main constraints. These are also the expected outcomes. They are: (1) regulatory quality improved; (2) rule of law and anti-corruption measures strengthened; and (3) fiscal performance improved. Sub-constraint (1) and (3) are related and both tackled within the private sector; while sub-constraint (2) is assessed within government and references improvements to government.

• Goals – given that the SOW requests a goal-selection process, the 17 indicators provided and viewed as outputs for the Philippines PFG initiative will be referred to as goals. Activities that are conducted to address these goals will be referred to as activities and projects.


2. Methodology

The evaluation team employed a variety of data collection approaches to gather information that would respond to the six key evaluation questions for the PFG mid-term evaluation. The overall evaluation utilized a mixed-method approach to data collection and analysis to ensure that the findings gathered would provide useful information for future PFG initiatives and evaluations. The mixed-methods approach was useful in providing comprehensive data for reporting by leveraging both qualitative and quantitative findings. The majority of information gathered was qualitative in nature, but when possible, the evaluation team verified qualitative findings by triangulating through third-party sources and quantitative data.

2.1 Methods of Data Collection and Data Analysis

The evaluation team began the data collection and review process with preliminary information gathering and an extensive desk review. The team then conducted semi-structured interviews and an online survey administered to GPH and USG PFG staff. The evaluation team also traveled to the Philippines for 2 weeks, from June 30, 2014 through July 15, 2014, where they conducted 58 interviews. These interviews were followed by 14 in-person or phone interviews in Washington, DC, with U.S.-based stakeholders and stakeholders who were unavailable in the Philippines during the field visit. Fiscal space and rule of law subject-matter experts conducted the interviews.

The remainder of this section is a summary of the data collection and analysis approaches undertaken in this evaluation. A detailed description of the methodology is provided through a separately developed report —the Updated Methodology and Evaluation Plan (UMEP), provided in Annex 4. All data collection and analysis activities followed the United States Agency for International Development (USAID) Evaluation Policy issued in January 2011.9 Desk review: The evaluation team conducted a desk review based on the program documentation received primarily from USAID for the PFG initiative and gathered publicly available documentation. All reviewed information was categorized per the SOW’s guidance in a desk review matrix and used to identify key themes for each of the six evaluation questions (see annexes VII and VIII for a list of documents referenced). Semi-structured interviews: The evaluation team conducted semi-structured interviews with key PFG stakeholders, including leadership, architects, program managers, implementers, and independent experts affiliated with the PFG initiative in both the United States and in the Philippines. USAID provided an initial list of stakeholders from which the evaluation team selected interviewees. GPH also provided names of their staff who are affiliated with PFG. The original list had a total of 130 stakeholders from USG (29), GPH (49) and partners (52). The interviews were semi-structured in nature, thus allowing for an open framework, conversational communication, and probing questions. Table 2.1 below lists the number of intended respondents versus the actual number of respondents and response rates by

9 USAID, January 2011. Evaluation Policy. http://www.usaid.gov/sites/default/files/documents/1868/USAIDEvaluationPolicy.pdf


government, affiliation, and respondent’s primary role. The overall response rate was 55 percent, the response rate for GPH respondents was 31 percent, and the response rate for USG respondents was 65 percent. All interviewees signed a consent form that confirmed maintaining confidentiality throughout the evaluation process.

Table 2.1: Stakeholder interview information Stakeholder type Government of the Philippines United States Government

# of intended respondents

# of respondents # of intended respondents

# of respondents

Leadership 37 15 (41%) 23 19 (83%) Architects 4 0 (0%) 14 4 (29%) Program managers/CORs 7 3 (43%) 6 5 (83%) Government subtotal 48 18 (38%) 43 28 (65%) Government total 46 (51%) Stakeholder Type Independent stakeholders # of intended respondents # of respondents Implementers 31 27 (87%) Experts 10 3 (30%) Total 41 30 (73%) Total number of respondents 76 (58%)

Table 2.2 lists which questions were asked of each stakeholder type, and annex IV contains all the semi-structured interview guides. Table 2.2: Semi-structured interview guides by research question and respondent type

Primary PFG Role/Interview Guide CCQ1 CCQ2 CCQ3 CSQ1 CSQ2 CSQ3 High-level leadership X Leadership X X X X X X Architect X X X X X Program Managers/CORs X X X X X X Implementers X X X Independent Experts X X X X X

NOTE: CCQ stands for cross-cutting question and CSQ stands for country-specific question. Web-based survey: Similarly, the web-based survey, which included 27 questions, was administered directly to PFG staff (GPH and USG). However, high-level leadership was not surveyed. Table 2.3 provides the number of stakeholders who received the survey and the number of respondents by government affiliation.

Table 2.3: Web-based survey by response rate and government affiliation Government

affiliation # of intended respondents # of respondents Response rate

GPH 86 11 13%

USG 43 30 70%

Total 1291 41 32%


1 The original stakeholders list had 138 contacts. From this list, six (6) addresses from the GPH and three (3) from the USG were non-working addresses. This discrepancy left the team with 129 working addresses, three (3) of which had out-of-office auto replies.

Out of the 27 survey questions, three requested information regarding the respondent’s role in PFG, 13 were applicable to multiple evaluation questions, and 11 were applicable to Cross-Cutting Question 2 (CCQ2), concerning workload changes. The overall response rate was 32 percent, with a notable variance in response rates between roles and affiliation. For example, the USG response rate was 70 percent (30 responses out of 43 solicitations), whereas the GPH response rate was 13 percent (11 responses out of 86 solicitations). Annex IV lists the questions included in the web-based survey.

2.2 Data Analysis

The data gathered were analyzed using a mixed-methods approach. The evaluation team, in collaboration with USAID, determined the analysis techniques that were most suitable for answering the six evaluation questions. The team took the necessary steps to ensure that the data analysis produced accurate results and identified any limitations to the analysis. As previously indicated, the evaluation team developed a UMEP report prior to conducting data analysis. The report detailed the purpose of the evaluation, questions required by the SOW, data quality assurance procedures, data entry, data cleaning procedures, types of analyses, and the limitations of the data analysis.

Although the evaluation team undertook quantitative analysis, where possible, the methods in this evaluation were largely qualitative and comparative. When possible, interview and web-based survey data were cross-referenced with information obtained during desk reviews in order to understand and verify empirical evidence. Specifically, interviews were used to gather information about stakeholders’ perceptions as they applied to the SOW’s six evaluation questions. These data were distilled into themes that underscored the applicable evaluation questions. All data collection tools were tested by the internal team, and members of USAID staff both in Washington D.C. and the Philippines. Changes were made to the tools based on the feedback received and they were approved by USAID prior to use.

The evaluation team recognized the importance of producing valid and properly documenting findings. To this end, the team used a multi-pronged process to ensure that data were analyzed in an accurate and unbiased manner, particularly the data from the semi-structured interviews. This quality control approach involved:

• training coders; • using a layered coding process—interview summary notes were reviewed by two primary

reviewers and a senior reviewer; • conducting daily coder meetings; and • performing periodic senior-review spot checks (through evaluation and subject-matter experts) to

reconcile inconsistencies. The evaluation team also used different data verification methods that included validating summary notes and triangulation. Validating summary notes involved checking the accuracy of the notes and augmenting the written summary notes used by the coders. The evaluation team also validated the accuracy of data by triangulating across multiple data sources and modes when possible. The evaluation team’s triangulation methods consisted of collecting the same type of information from different sources using different methods. Data collected from the desk reviews, semi-structured interviews, and web-based surveys of


various stakeholders were compared as much as possible in order to strengthen the reliability of the analysis. Throughout the evaluation, the data received were carefully collated, reviewed continuously, summarized, and synthesized. This systematic approach helped to identify trends and establish the authenticity, validity, and reliability of the reported findings.

Semi-structured interviews were coded by evaluation question, question-specific themes, and perceived sentiment. Web-based survey results were aligned with the interview results by respondent and evaluation question. The team also used Microsoft Excel to code interview findings and Microsoft Excel and Stata to conduct online survey-data cleaning and quantitative analysis. It is important to note that for the stakeholder interviews—which provided the majority of information concerning PFG design, initiation, and implementation—the evaluation team developed modular interview guides for PFG staff and key stakeholders: high-level leadership, leadership, architects, program managers or CORs, implementers, and independent experts, as stated previously. However, according to the data provided by USAID on stakeholder roles, sometimes a stakeholder may fulfill more than one role. For example, a staff member who was considered leadership because he or she played a managerial role in the PFG structure could also be a program manager or contracting officer’s representative (COR), or a program manager or COR could also be an implementer. Despite the overlap, only one interview guide could be used for the hour-long interviews, so a guide was selected based on a respondent’s primary role as understood by the evaluation team. During analysis, the evaluation team’s approach included counting stakeholders more than once, as many stakeholders fulfilled more than one role. Therefore, quantitative counts provided for leadership, goal leads, and implementers would reflect a “double count” if needed.

2.3 Evaluation/ Study Limitations

Identified study limitations include the following.

Lack of counterfactual: Assessing PFG’s performance without a readily available counterfactual or a valid metric of PFG’s alignment with the WGA was a limitation to the overall study. Respondents with years of experience were able to compare PFG with more traditional approaches to development, but respondents with less experience may not have had a point of reference with which to compare. Also, the perceptions of those who had more experience and were able to make a comparison may have reflected recall bias. In addition, a valid metric that gauges a program’s alignment with the WGA has not been developed and, as a result, programs are designated as either PFG or not PFG.

Perceptions gathered through semi-structured interviews: Interviews were structured to address evaluation questions, which provided flexibility to further explore themes. Although this approach was a strength, it also presented a weakness in that all responses were subjective, and because the interviews were the majority of the data collection, the majority of the findings were based on perceptions and not necessarily facts.

The online survey was relatively narrow in scope and included only a limited number of questions: The online survey, which was administered to PFG architects, leadership, and program managers or CORs, afforded respondents an opportunity to relay their candid impressions of the PFG. To promote a high response rate, the online survey was as short and specific as allowable. Although respondents were


provided with an opportunity to elaborate if they wished, questions were largely closed ended and did not provide the detail that an interview would. The dichotomous (Yes/No) and ordinal (Good/Average/Bad) nature of most of the variables allowed for quantification and direct comparison. However, these advantages were partially offset by the fact that the interviewer directed and limited the lines of inquiry. As with the interviews, although they provided another dimension for comparison in developing findings, these responses were based on perception.

Data verification: Ideally, evaluations should be based on “analyzed facts, evidence and data supported by strong qualitative or quantitative evidence and not anecdotes, hearsay or people’s opinions.”10 With information collected from over 100 sources (41 online survey respondents and 73 interviewees), the evaluation team has confidence in the strength of the qualitative data collected and analyzed. Multiple information sources facilitated cross-validation, allowing the evaluation team to make statements that are supported by the opinions of multiple respondents (for instance, as demonstrated in the interview guides in Annex 4, the majority of the same questions were asked across stakeholder types). However, even the online survey was no more than the quantification of respondents’ subjective opinions. More rigorous PFG monitoring data for each of the goals and interventions would have been a good source to affirm (or contradict) these perceptions. However, the documents received did not always provide the required level of rigor for each goal and corresponding interventions. Although the lack of good performance data limits the evaluation’s findings, empirically-based evidence was used to corroborate perceptions whenever possible.

10 USAID, November 2012. How-To Note: Preparing Evaluation Reports. http://www.usaid.gov/sites/default/files/documents/1870/How-to-Note_Preparing-Evaluation-Reports.pdf


3. General Evaluation Findings

Analyses of interviews with key PFG Philippine stakeholders and data from survey respondents indicated a number of distinct overall findings about the PFG Philippines initiative at midterm. The overall findings in this chapter provide input beyond the specific six evaluation questions and thus provide an opportunity to capture context, a summary of responses, and unanticipated advantages and disadvantages not otherwise captured by the evaluation questions. Chapters 4 through 10 provide an in-depth discussion of the answers to the six evaluation questions. According to the JCAP, which laid out how the PFG should be implemented, the following are the timelines for the main PFG Philippines’ activities, which were reviewed at midterm for this evaluation:

• December 2011: Composition and terms of reference for PFG technical working groups agreed upon.

• December 2011: PFG targets established and overall monitoring and evaluation (M&E) framework developed for the Philippines.

• January 2012: PFG Technical Working Groups organized. • January 2012: Programs designed, small activities initiated. • February 2012: Procurement for major programs undertaken (depending on funding availability). • Q2-2012: PFG projects rolled out. • 2012-2015: Regular M&E of program activities.11

The evaluation determined that PFG Philippines has not met all of the above targets within the initially envisioned time frame. After careful analysis and discussions with various stakeholders, the evaluation team determined that the time frame in the JCAP for program procurement and implementation was unrealistic and that much of the delay in programming, particularly with regard to USAID, was reasonable given the normal time for procurement and the lack of pre-existing or “legacy” USAID programs that would fall under the areas envisioned in the constraints analysis. PFG Philippines started from scratch and reinvigorated dialogue between the U.S. government and the Philippines government on development objectives and program design, and completely reoriented the emphasis within the USAID program in the Philippines. This process understandably took time; consequently, new program design was not finished until at least a year into the PFG. By most accounts, USAID project procurement was accelerated to get the PFG back on track. Measurement of PFG at midterm progress was challenging. This challenge was caused by a lack of an overall PFG M&E framework and a corresponding lack of integration of overall PFG objectives, activities, indicators, and goals of individual programs, by USAID and other USG agencies working with GPH counterparts. Most USG and GPH projects collect M&E data of some sort, but it is currently not being collected or aggregated for PFG purposes. The evaluation team was not able to analyze the progress of many JCAP goals in depth because of the lack of systematic, organized data on the implementation of programs related to each goal and the consequent inability to conceptually link these data to overall PFG 11 PFG Philippines JCAP 2011.


progress toward addressing the constraints from the constraints analysis (CA). This applies to specific USAID programs, but also to the policy, non-assistance, and programming work of other USG agencies and related advances by GPH.

3.1 Overall Advantages and Disadvantages of the PFG

Based on data collected through the online survey, the evaluation team found that respondents’ perceived the PFG to be an improvement over other development assistance strategies. As shown in figure 3.1, a total of 77 percent of respondents believed that the PFG was an improvement over other forms of development assistance: 47 percent saw it as “an improvement” and 30 percent saw it as a “significant improvement.” Figure 3.1: Did the PFG bring changes, as compared with other approaches to development assistance intended to affect economic growth?

Online survey respondents also shared their views on whether the PFG initiative is meeting its ultimate goal of advancing the economy in the Philippines. As illustrated in figure 3.2 below, 70 percent of respondents agreed that PFG is meeting its goal, with 12 percent stating that they strongly agree, and 58 percent stating that they agree that the PFG has been a catalyst for promoting economic growth. Figure 3.2: Is the PFG meeting its goal of advancing economic growth in the Philippines?


Through interviews, the team obtained further insight into the main purpose of the evaluation: determining whether the PFG was an improvement over other approaches for development assistance and whether it was meeting its goal in the Philippines. In addressing these central issues, interviewees frequently mentioned four topics: policy reforms, the CA, the JCAP, and additional leverage provided by the program. Below is a summary of the findings related to each of these topics. Advantages of the PFG Finding 1: The PFG was a key instrument that promoted a change in the USG’s development approach in the Philippines.

Different sets of stakeholders cited the PFG process as the key event that allowed the USG and GPH to reset the focus of development programming in the Philippines. It was the first time the GPH was able to discuss priorities with USAID, as previous priorities and programs had been established more or less unilaterally with a focus on the Mindanao region. The timing of the PFG also followed the start of the new Aquino administration and provided an opportunity for the U.S. Embassy to develop a program to support the GPH’s new reforms. Finding 2: The consultative nature of the CA and JCAP process developed ownership of and commitment to key approaches and reforms among important Philippines stakeholders.

The CA and JCAP process, which involved GPH counterparts in the analysis, decision making, and structure of programming, has increased government buy-in and participation. This rigorous analysis and more cooperative decision-making process was complicated and time consuming, but there was a general consensus among the leadership within the USG and GPH that it generated momentum for reforms and for government commitment to PFG-related projects. Finding 3: Initial PFG planning through the CA created a positive technical and empirical basis for programming decisions. Twelve out of 17 USG respondents who were part of the CA process shared that the CA was a positive, objective, and analytical approach to development programming that allowed the USG and GPH to identify priorities and create sustainable programs that otherwise might not have been the focus of the PFG if this analytical process had not been undertaken. Finding 4: The PFG has demonstrated potential for positively influencing policy reforms in the Philippines.

A number of members of PFG leadership and project interviewees discussed the role the PFG process has had in promoting key policy reforms in the Philippine government. Policy reform, including required legislation, is central to many PFG goals. Through a combination of coordinated messaging by the United States, an inclusive development programming process with the GPH, and the combined efforts of multiple USG partners, the PFG has supported many positive policy reform efforts by the GPH. The policy reform process is still ongoing on many of these issues, but based on the team’s analysis, the PFG has improved the likelihood of reforms being enacted.


Disadvantages and challenges of the PFG Finding 5: PFG-WGA raised the GPH’s expectations too high only to have the GPH discover later on that WGA came with no resources.

The PFG was announced with much fanfare and with an exceedingly ambitious JCAP that covered a large array of activities. A reader of the JCAP who was not conversant with the intricacies of funding mechanisms and corresponding restrictions could have easily been convinced that large amounts of additional funds would be available for the implementation of the JCAP. A large amount of additional funds was not available; a fact that some stakeholders were later surprised to learn. Moreover, the concept of non-assistance led some stakeholders to conclude that the PFG meant that U.S. policy changes to help the Philippines would be forthcoming as part of the PFG aid process. Naturally, there was some confusion on the GPH side about what policy changes could be implemented, given a lack of detailed knowledge of the constraints that inhibit policy change in the United States. Over time, this confusion has led to some disappointments within the GPH about the PFG process.

Finding 6: Participation is often limited by USG agencies’ restrictive mandates.

The PFG’s potential to change the way the USG conducts foreign development assistance is often overstated because most USG agencies do not have an institutional mandate to spend funds overseas. The number of USG agencies other than USAID that can effectively contribute to the PFG is limited, and not all that can are willing. Many USG respondents indicated that unless there is funding to allow for greater technical contributions by other USG agencies, their participation would remain limited. Finding 7: The consultative nature of PFG WGA created perceived delays in programming.

The consultative process that the USG and GPH carried out upon initiation of the PFG resulted in increased coordination and commitment, but also required more staff time and created delays in the start of procurement for specific programs. Most PFG-related USAID programs started more than a year into the PFG, and most are only in the first phase of monitoring results. Increased coordination and planning with the GPH added time to the design phase of projects. A greatly accelerated USAID program procurement cycle allowed the PFG to get closer to its original schedule. The delay caused by the PFG coordination was accentuated in the Philippines as the focus of development programming was reconfigured, and there were few existing “legacy” projects that could be rolled into the PFG. Nevertheless, most respondents felt that the time it took to build relationships and jointly design new programs has had greater benefits than costs. Finding 8: The CA system lends itself to disagreements.

Given the objectivity of the CA, the PFG identified issues that are politically sensitive and caused conflict between or within government agencies, particularly on the Philippine side. The CA focused on a tighter set of activities than the preferences of individuals, bureaucratic necessities, and political processes would have. Such factors can lead to the CA being subverted or disregarded, damaging the integrity of the process at the beginning. As a result, the JCAP was also sidetracked by disagreements and an inability to address more intractable issues.


Finding 9: The objective, empirical approach of the PFG was limited by a complex political and programmatic landscape and lack of dedicated funding.

The PFG is meant to use empirical analysis to design an objective, comprehensive set of programs, but it is only one of many, often competing, USG or GPH efforts (Country Development Cooperation Strategies, Philippines Development Plan, Climate Change initiative; Congressional earmarks, etc.) and often does not have the full political support needed for implementation. As stated by one respondent, “PFG does not change the fact that development assistance is earmarked to death.” The political support that does exist currently could wane with a change of administration and stall the program. Further, a lack of PFG-specific funding also makes the initiative dependent on political will and the priorities of prevailing projects (and earmarks), which can wane with a change of administration or other circumstances, thereby stalling the initiative. It is important to note, however, that according to USAID, the PFG enabled the Mission to obtain more non-earmarked funds to support scaled-up programs to implement the core activities identified by the JCAP, which could promote sustainability.

Already there is evidence that the theoretical framework of the CA (and JCAP) is breaking down, resulting in a business-as-usual approach by many PFG stakeholders. The JCAP avoided important issues that were confronted in the CA, while the JCAP itself has not been completely implemented, with certain goals being inadequately addressed or reconfigured to fit preferred programming. These problems and PFG progress, more generally, are not adequately monitored nor is there an appropriate framework for doing so. While not a panacea to these issues, associated funding would increase the urgency of adhering to the structure of the PFG and reporting on results.

3.2 Conclusions and Lessons Learned

PFG has been an extremely important mechanism for resetting USG’s engagement with development assistance in the Philippines after years of focus on Mindanao. It has also allowed USAID and other USG actors to engage with the GPH in a collaborative process to set priorities for reforms and technical assistance, and it has improved coordination across USG efforts on reforms that benefit the Philippines. Implementing the process does, however, require more time for both the USG and GPH. The analytical process of both the CA and the JCAP, and the political process of agreement on the design of programs, resulted in delays in the anticipated schedule for programming and other targeted activities.

Despite the positive contributions it has made toward collaboration and consensus-building, PFG Philippines has not been completely developed and implemented on several primary levels. First, the development theoretical framework (or “theory of change”) as captured by the CA has not been fully preserved, and other development topics have been inserted into the PFG based on subsequent analysis.. There is no clear instruction for PFG country programs on how to incorporate changes in country context, or revise a CA or JCAP midstream. In the case of the Philippines, the revisions to the CA and JCAP were not done formally and did not entail a new, robust empirical analysis. Second, monitoring of the progress and results of PFG implementation is behind schedule. Because of the absence of a PFG-wide M&E framework, determining what USG-led and GPH-led goals and activities are being implemented effectively, and whether or not PFG as a whole is on track, is a difficult endeavor.


3.3 Course Corrections and Recommendations

The overall findings on the PFG initiative lead to five general recommendations. Recommendations are expanded upon further in the subsequent detailed findings if they were applicable to an evaluation question. Overall, the evaluation team recommends the following:

• USG and GPH leadership should finalize and implement an M&E framework for the overall PFG initiative that articulates how progress at the project, activity, and goal levels contributes to constraint-level indicators. Much of these data already exist and are being tracked by GPH institutions or individual USG projects, but they have not been articulated into a management tool for overall PFG tracking.

• The PFG M&E framework should be finalized and used to guide all individual project M&E plans.

• The Philippines PFG should conduct annual strategic reviews (as defined in the JCAP) that chart JCAP implementation progress and make corrections to under-achieving programs.

• The JCAP should be updated to reflect actual programming. The theoretical basis of the PFG—the CA and JCAP—has not been adhered to during implementation. Given that the JCAP is supposed to be a living document, applicable portions should be updated to reflect the current development theory, given that changes have been made to the overall constraints being tackled.


4. Cross-Cutting Question 1: What are the advantages and/or disadvantages of the PFG whole-of-government approach to development assistance?

The WGA, initially utilized within fragile and emerging countries,12 was motivated by the evolving threat of terrorism and the desire to mitigate risks and adverse outcomes by ensuring that information is shared between and among agencies.13 As evidenced by the PFG, the concept of the WGA has now been leveraged more broadly within the international development arena. The WGA is defined as follows, as one researcher explained: “The slogans of ‘joined-up-government’ and ‘whole-of-government’ provided new labels for the old doctrine of coordination in the study of public administration. The ‘joined-up-government’ was presented as the opposite of ‘departmentalism,’ ‘tunnel vision,’ and ‘vertical silos.’”14 The WGA can also be defined as an aspiration to achieve horizontal and vertical coordination between stakeholders in a particular policy area in order to use scarce resources better and to eliminate situations when different policies undermine each other. Similarly, the Australian Management Advisory Committee’s Connecting Government report (2004) says of the WGA in the Australian public service: “Whole of government denotes public services agencies working across portfolio boundaries to achieve a shared goal and an integrated government response to a particular issue.”15,16 The WGA has been utilized widely due to its several benefits. For example, the WGA has been recognized as a system for enhancing “the quality of services and benefit[ting] participating organizations by offering better processes, improved relationships, and greater capacity to respond to local needs, as well as more efficient use of resources.”17 The following are the three main components needed to demonstrate an effective WGA:

• multiple organizations or agencies that share an interest or objectives; • leadership that promotes WGA within management and coordination; and • accountability mechanisms for fostering the approach.

The evaluation team analyzed the advantages and disadvantages of the WGA as a core component of the PFG initiative in the Philippines, as well as the PFG’s alignment with the WGA. According to the evaluation SOW specific to the PFG:

Within the U.S. government, the term “whole of government” reflects efforts to align each agency’s activities to achieve a common objective. PFG calls upon the U.S.

12 Organization for Economic Cooperation and Development. Whole of Government Approaches to Fragile States Report (2006). 13 Hammond, T. (2004). Why Is the Intelligence Community So Difficult to Redesign? Paper presented at the SOG-conference, University of British Colombia, Vancouver, June 15–17. 14 Christensen, T., & Laegreid, P. (2007). The Whole-of-Government Approach to Public Sector Reform. Public Administration Review, 67(6): 1059–1066. 15 Australian Management Advisory Committee’s Connecting Government report (2004). 16 Ling, T. (2002). Delivering Joined-Up Government in the UK: Dimensions, Issues and Problems. Public Administration, 80(4): 615–642. 17 Humpage, L. (2005). Experimenting with a ‘Whole of Government’ Approach: Indigenous Capacity Building in New Zealand and Australia. Policy Studies, 26(1): 47–66.


Government (USG) and partner countries to be more comprehensive and creative in our development work—to reach beyond aid to all the instruments that both governments can bring to bear to connect and amplify the impact of current investments and unlock growth potential. USG commitments under PFG are comprised of both assistance and non-assistance tools that, undertaken in close coordination with partner countries, will maximize our impact and success. In addition to those actions already identified by the interagency and partner countries, additional non-assistance activities should be considered over the life of PFG for a sustained and focused effort.18

Information about the advantages and disadvantages of the PFG’s WGA were collected through semi-structured interviews primarily conducted with PFG staff members, including leadership and goal leads from the USG and GPH. Additional documentation that could have added to staff perceptions, such as PFG meeting summaries and attendance sheets to corroborate the advantages and disadvantages of such interagency cooperation, were not readily available. The evaluation team also administered an online survey, which allowed respondents to provide their impressions of the WGA, but largely in the context of the additional workload that interagency cooperation requires.

4.1 Findings

Advantages of the WGA

Finding 1: The CA and JCAP development framed WGA dynamics.

The review of the CA itself was not part of SOW for the mid-term evaluation. However, the CA does provide context for the development of the PFG and the establishment of its principles, targeted constraints, related goals, and projects/ activities. Among architects and leaders involved with the initial stages of the PFG process, the CA was often mentioned as necessary for promoting the WGA. Three out of four USG Philippine architects identified the CA as an essential part of the PFG system, promoting the initiation of the WGA. Interviewees who explicitly commented on CA suggested that it was a very useful exercise, explaining that beginning the PFG process with an objective technical analysis of the constraints to economic growth gave the process credibility. A USG leader explained that the fact that CA process was established with counterparts was good progress for initiating inter-country collaboration and coordination. This process led to increased buy-in and collaboration between the USG and the Philippines. Further, another USG leader provided input on how the CA process better shaped the thinking and methodology of USG development practice. The interviewee explained that the CA helped focus the attention of both governments on a finite number of central development objectives. The CA process also helped channel the direction of USG development assistance in the Philippines.

18 Statement of Work – Partnership for Growth (PFG) Mid-term Evaluation: El Salvador and The Philippines, September 2013, pg. 10.


Finding 2: The WGA has led to a change in the way USG programs conduct development assistance in the Philippines.

Nineteen respondents, within the GPH and USG, believed that the introduction of the WGA represents a new paradigm in how development assistance is designed and implemented in the Philippines. Table 4.1 illustrates the sentiments of the USG and GPH leadership and architects. Of the 38 respondents that were asked if the WGA led to changes in development-assistance delivery, 19 answered “yes” and four answered “no,” demonstrating that the majority of leadership believed that the WGA introduced a new and different approach to how development is provided. A USG leader stated, “PFG was a development that forced us to work together toward a common objective and also forced us to work closely together. This [cooperation] was different than previously.”

Table 4.1: In your opinion, has the WGA led to change in the way the USG delivers development assistance in Philippines?

Interview responses

USG GPH Overall total respondents

Leadership Architects Leadership Architects

Yes 10 (53%) 3 (75%) 6 (40%) N/A 19 (50%)

No 3 (16%) 1 (25%) 0 N/A 4 (11%)

No response 6 (32%) 0 9 (60%) N/A 15 (39%)

Total 19 4 15 N/A 38 Some respondents had more tempered views on whether the PFG itself changed development assistance modalities. They cited individual agendas as key factors to informing change, rather than PFG-specific processes. Others felt that since no additional infrastructure was included to implement the PFG, not much changed in terms of a new system of delivering development.

Finding 2: The WGA has effectively focused human and operational resources to increasing consistency and coherence in the programming and policy objectives of the PFG.

As demonstrated in table 4.2 below, PFG staff—leadership, architects, and program managers—mostly believe that the WGA has led to changes in management and implementation, and has helped promote operational efficiency within the USG team in the Philippines and the GPH team working under the PFG. Sixteen percent of USG leaders, 75 percent of architects, and 40 percent of program managers interviewed found the WGA to have led to improvements in their work. For example, a USG architect expressed that the PFG WGA led to improved coordination and better dialogue between USG agencies that had previously worked in isolation. Within the USG, the introduction of the WGA and the “forced” coordination that it requires led to a renewed focus on how work should be done, and given that all stakeholders were following a shared system of working, greater efficiency has been attained. There were a few staff members who did not believe that the WGA led to increased operational efficiency. One USG leader stated, “This approach has not necessarily created more efficiency, depending on the definition of it, but it has reduced duplication of mixed messages.” It is important to note that while stakeholders did share the opinion that there was an increase in operational efficiency, they also noted that the coordination


aspects of the WGA led to an increased workload for USG PFG staff, particularly during the initiation processes.19

Table 4.2: Have any changes led to operational efficiency in project design and implementation?

Interview responses

USG GPH Overall total respondents Leadership Architects

Program manager/

CORs Leadership Architects

Program manager/

CORs

Yes 3 (16%) 3 (75%) 2 (40%) 2 (13%) N/A 1 (33%) 11 (24%)

No 2 (11%) 1 (25%) 1 (20%) 1 (7%) N/A 1 (33%) 6 (13%)

No response 14 (74%) 0 2 (40%) 12 (80%) N/A 1 (33%) 29(63%)

Total 19 4 5 15 N/A 3 46

Similarly, within the GPH, 13 percent of leaders and 33 percent of program managers interviewed found the WGA to have led to improvements in the operational efficiency of their work. The increased efficiencies have been attributed to the WGA’s ability to promote coordination within and among the two governments. For instance, a GPH leader shared that there has been an increase in efficiency because of the new system, leading to having all relevant stakeholders at the decision-making table, which provides more comprehensive perspectives than would be the norm. These comprehensive perspectives lead to better inputs for more efficient implementation. Finding 3: The WGA has improved coordination between the USG and GPH.

The WGA under PFG has brought multiple stakeholders together to discuss challenges and resources. This coordination has resulted in several opportunities for targeted USG agencies to assist their GPH counterparts. A GPH leader noted that the WGA has led to inter-agency coordination between the GPH and USG. Examples of this coordination included USAID projects and leadership supporting staff from the GPH Commission on Audit to do a 4-month fellowship at the General Accounting Office in Washington, DC. A GPH leader shared that as a result of the PFG, USAID leadership is now better connected to the leadership within the GPH Department of Finance (DOF), and that USAID even held joint annual meetings with the DOF National Economic Development Authority (NEDA). Multiple members of the USG and GPH agencies also discussed the collaboration between USAID, the U.S. Department of Justice (U.S. DOJ), and other USG agencies in Washington, DC, including the White House, to ensure that the GPH ombudsman was able to conduct a study tour in the United States to learn best practices and gain experience related to corruption prosecution. According to most people interviewed, this program would not have been possible without PFG.

Finding 4: The WGA has led to a more consistent USG message on key GPH reforms.

Various USG staff cited that increased coordination under the PFG has led to consistent messaging to GPH counterparts on key reforms. Through more regular coordination and program design meetings,

19 Changes in workload are discussed in more detail in response to Evaluation Question 2 in the next section.


USG staff across agencies were able to identify key policy and institutional reform issues in the GPH that each were working on simultaneously (and therefore could be duplicating efforts without coordination). As a result of increased collaboration, they were also able to discuss any differences in approach. Through these discussions, a consistent message to the GPH was crafted on what USG expects in terms of development programming, which has led to more impactful advocacy.

Challenges or Disadvantages of the WGA

Finding 6: The number of agencies to which the WGA is relevant in terms of their daily activities and decision making is highly limited.

The effects of the WGA tend to be overstated, perhaps because there is a dissonance between the powerful theory and practicalities on the ground. While most USG and GPH staff felt that the WGA was effective, many struggled to cite concrete examples. Upon further questioning, most revealed that their interaction with the USG was limited to one or two agencies—either USAID or MCC—and the implementers associated with specific projects. A few have interactions with multiple agencies, but the extent of technical assistance coming from these interactions is limited. This situation is a natural result of the limitations placed on most USG agencies on the use of their budgeted funds. When the PFG was a large emphasis within the highest levels of the USG, there was pressure on agencies to contribute personnel and funds. This emphasis has declined over time and cooperating with the PFG has become less important for USG agencies (other than USAID).

Finding 7: There has been some dissatisfaction because of a perceived delay of PFG project implementation.

GPH staff—leadership and program managers, in particular—expressed frustration about a perceived delay in program implementation, stemming largely from the increased coordination and sign-off needed from multiple agencies as part of the WGA. Delays included interruption to project initiation due to needed agreement on project design before projects could be procured. Furthermore, across the leadership and program managers both in the USG and GPH, it was acknowledged that the WGA promotes coordination; however, respondents also explained that coordination was sometimes not possible because of the time it took to get all the needed stakeholders together to discuss pertinent issues, thereby causing delays in implementation. Finding 8: There is some confusion about decision-making authority within the PFG system, limiting WGA implementation.

There was general confusion about who is in charge of the PFG, who makes decisions, and how decisions are then implemented. This confusion is almost certainly a direct result of the slow implementation of procedures for the management of the PFG and the minimal management structures in place. The evaluation team provides further detail and evidence about this issue below, under Country-Specific Question 2.

Finding 9: The WGA requires greater staff time to be effective.

Although respondents agreed about the benefits of increased coordination, they also shared the challenge that coordination presents in terms of staff time. Six members of USG leadership expressed this concern, with one saying, “Overall, WGA requires more effort, and there is probably some benefit to it that was


not there before PFG.” It is important to note, however, that while PFG staff did discuss the increased time required for coordination, they were also quick to mention that the extra time commitment was worth it (as discussed previously). The evaluation team provides detailed input on staff time and changes in workload as a result of the PFG in the subsequent chapter.


Overall, WGA was viewed as a positive approach within the PFG initiative that improved the way development assistance is implemented. The WGA has also led to increased coordination both within the USG and GPH, and between the two governments. This increased coordination, while time consuming, has led to improved operational efficiency because when all relevant stakeholders gather to identify a solution, the outcome tends to be comprehensive and relevant. WGA has also promoted better programing because of the increased buy-in of all stakeholders during project design and implementation.

While WGA has been largely positive, it also has challenges that include the increased staff time needed to effectively implement WGA, and the fact that it is sometimes challenging to gather and coordinate all needed stakeholders, which could lead to implementation delays.


Provide stronger mechanisms for inter-goal collaboration through a management team. Now that most goals (and corresponding activities) are underway, synergy across goals can be increased. The evaluation team believes that the PFG initiative would be beneficial if its entire leadership and implementation staff are made aware of how other goals (and corresponding activities) feed into the larger PFG initiative. Consequently, the evaluation team recommends that a small bilateral PFG management, coordination, and M&E unit be put in place (referenced as the PFG Secretariat in the JCAP). Currently, PFG’s management, to the extent that it exists, is housed within USAID, but without a specific overall point of contact for operational activities; within the GPH, a similar system does not exist. Given the vastness of the initiative, there may be value in establishing a defined management team with members from both the USG and GPH who would oversee the PFG initiative. One of the main tasks of such a management team would be identifying and facilitating opportunities for synergy and coordination, and to track the status of individual PFG projects and of the PFG overall (the management team is referenced, as applicable, in other recommendations in this report).

Develop performance measures and collect better data to track the coordination of activities in promoting effectiveness. Based on its analysis, the evaluation team concluded that the WGA has led to increased coordination within the Philippines PFG initiative, which has ultimately led to efficiencies. However, the evaluation team was not able to identify direct performance measures that gauge the quality, scope, and depth of coordination. For an initiative such as the PFG, it is possible, particularly over time, that frustration among management and staff could arise if indicators cannot be identified. Consequently, the evaluation team recommends that PFG Philippines management develop performance indicators that define WGA coordination activities and track how these activities have led to enhanced effectiveness and efficiency.

The evaluation team recognizes that there are challenges inherent in measuring the performance of a concept such as “coordination.” For example, when personnel are gathered, one might attempt to use


attendance as a performance measure; however, participation in meetings does not lead to improved outcomes (in fact, the increased time for attending meetings was identified as a disadvantage of the PFG WGA), so it would not serve as a good measure. The PFG teams both within the USG and GPH will need to collaboratively identify key indicators that measure how the teams have been more effective. The following examples of potentially useful indicators could be considered:

• Develop a set agenda for coordination meetings: Ensure that agenda items are reviewed and tracked during each meeting, and action items are reviewed at the onset of each meeting and tracked in terms of progress or regress.

• Introduce accountability measures: Establish repercussions for not participating in these meetings or not following through on agenda or action items.

• Identify and inform staff and key stakeholders of responsibilities for the overall management of PFG coordination: Ensure that all PFG staff members are aware of the management personnel within each government (and across governments). Staff should also be aware of their respective responsibilities related to fostering coordination activities.

• Establish (and update) an overarching work plan to track progress on coordination: The PFG initiative uses scorecards to track indicators of project performance by tracking goals and activities. However, based on the findings, it is evident that the PFG initiative is more than a compilation of programs, and instead a government-to-government initiative. Therefore, in addition to tracking the progress of projects, it would be key to track the progress of the non-project initiatives such as coordination activities, publicity of coordination, and other activities such as non-assistance, which will be discussed in a subsequent section. Such tracking could occur through PFG group listservs or newsletters.

• Identify and track activities or actions that demonstrate coordination: As a result of the PFG initiative, several activities could promote coordination. Meetings such as technical committees and scorecard meetings should be included in reporting on coordination activities.

As discussed in the previous recommendation, a newly established bilateral PFG management team could be assigned to facilitate the tasks indicated in this recommendation.


5. Cross-Cutting Question 2: To what extent has the Partnership for Growth affected the workload on national government and U.S. government staff, as compared to the workload created by traditional forms of development assistance delivery?

In responding to whether or not the PFG has led to an increase in workload on its staff, the evaluation team gathered data through interviews and the web-based survey. The findings gathered were based on perceptions from respondents. Performance indicators on time usage or timesheets pre-PFG through PFG implementation would provide a more accurate assessment of how the PFG has affected the USG20 and GPH workload as compared with traditional forms of development assistance delivery, but such data are not collected. Moreover, it may be difficult for respondents to compare the PFG workload to the workload pre-PFG, due to recall bias for those with more than 3 years of involvement with the PFG or possibly no point of comparison for those with less than 3 years of PFG involvement. Nevertheless, the data gathered via semi-structured interviews and web-based surveys provide insightful information about changes in government staff workloads attributable to the PFG and the perceived effectiveness of changes in workload, as compared with traditional forms of development assistance delivery.

It is important to note that a major limitation of the data collection and analysis in response to this question was the low response rates to the web-based survey. As mentioned in the Methodology section previously, the response rates were primarily low; among the GPH stakeholders who received the survey, only 14 percent completed it.

5.1 Findings

Finding 1: The PFG has resulted in an increase in the workload of USG and GPH staff.

As illustrated in figure 5.1 below, respondents confirmed that introduction of the PFG has led to an increase in the workload for USG and GPH staff. Among USG respondents, 32 percent indicated that their workloads increased somewhat and 29 percent indicated that their workloads increased significantly. Among GPH respondents, 42 percent indicated that their workload stayed about the same, and 58 percent indicated that their workload increased somewhat. None of the respondents reported a decrease in workload resulting from involvement in PFG.

20 Note that USG staff represent staff from both Philippines Mission and staff in Washington, DC. This combined representation could potentially skew the findings of the USG team given that Washington, DC, staff may have other PFG country responsibilities outside of the Philippines.


Figure 5.1: Change in workload resulting from PFG involvement: USG and GPH

Finding 2: The PFG has prompted a significant increase in workload focused on coordination and communication, both intra- and inter-governmental.

PFG activities were categorized into the following: 1) coordination with colleagues within my government; 2) coordination with colleagues in other (partner) governments; 3) monitoring progress of PFG tasks; 4) communicating about PFG with my superiors and senior leadership in my government; 5) managing PFG activities; 6) designing and/or procuring PFG activities; 7) other administrative tasks. These findings are provided overall (USG and GPH combined–figure 5.2), then USG and GPH separately (figure 5.3).

According to figure 5.2 below, the increased workload due to PFG activities was distributed fairly evenly across all of the range of activities; “coordination with colleagues within my government” constituted the largest increase in workload in comparison with the other types of PFG activities. Secondarily, communicating on PFG with superiors and senior leadership within the government accounted for large shares of the increased workload due to PFG.


Figure 5.2: Change in workload resulting from the PFG by task

In comparing USG to GPH staff, it appears that the PFG has prompted the USG to expend more hours on intra-government coordination and communication as compared with the GPH (see figure 5.3). GPH staff have expended relatively more hours, due to the PFG, in monitoring progress and managing PFG activities. Figure 5.3: Change in workload resulting from the PFG by ask – USG and GPH


Finding 3: There was no discernible difference in workload by gender.

Table 5.1 below provides a breakdown of respondents by gender, and affiliation (USG only, GPH only, and USG and GPH combined). The evaluation team used these data to review changes in workload by gender. Table 5.1: Respondents by gender

As illustrated in figure 5.4, the evaluation found that there was no discernible difference in workload attributed to PFG by gender. Fourteen percent of female respondents reported that workload increased significantly, whereas 24 percent of male respondents reported that workload increased significantly. Figure 5.4: Changes in workload by gender

Finding 4: Increased workload is associated with perceived effectiveness of the PFG and WGA.

Similar to innovations in other industries, the PFG has required the “innovators” and “early adopters” in the USG and GPH to invest upfront, resulting in significant increases in workload and possibly crowding out traditional forms of development-assistance delivery. These significant increases in workload may not be sustainable if the PFG does not result in higher levels of efficiency and effectiveness, or at least may be perceived to improve the efficiency and effectiveness, as compared with traditional forms of development-assistance delivery. It is still too early to assess the PFG’s efficiency, but this analysis provides insights to stakeholders’ perceived effectiveness.

Affiliation Male Female Total

USG 23 8 31 GPH 6 6 12 Total 29 14 43


Figure 5.5 illustrates that PFG stakeholders who invested more time in PFG activities perceived that PFG was a significant improvement compared to traditional development assistance approaches. For example, more than 40 percent of PFG stakeholders who reported that PFG was a significant improvement over traditional approaches also dedicated 16 or more hours per week to PFG activities. In comparison, less than 30 percent of PFG stakeholders who reported that PFG was a significant improvement over traditional approaches worked less than 16 hours per week on PFG activities. Figure 5.5: Perceived effectiveness of the PFG compared with traditional development assistance approaches


Overall, the PFG was viewed as an improvement compared with traditional development approaches. This perception was supported by stakeholders who invested more time in PFG activities. PFG staff, particularly USG staff involved in communication and coordination within the government, have generally experienced an increase in workload. Members of the USG who experienced a significant increase in workload due to the PFG spent these hours on intra-government coordination and communication. Though a significant increase in workload was not observed among GPH staff, those whose workload somewhat increased were involved in monitoring progress and managing PFG activities.



Identify management staffing that could assist in reducing the workload of PFG staff across the Initiative. Based on the evaluation findings, it is evident that the PFG has created a heavier workload for USG and GPH staff. Effort should be made to ensure that sufficient resources are available to manage PFG activities without detracting from other worthwhile commitments. For example, if staff members spend 3 to 6 hours per week on PFG, then 3 to 6 hours per week on other, non-PFG responsibilities may be neglected. It would be prudent to have, from the start, sufficient staff to manage the daily functions of the PFG, coordinate across agencies and governments, and facilitate the monitoring of PFG data to track progress.

Identify and promote the use of limited resources more efficiently to reduce overall workload. The PFG—and the WGA, in particular—represent a relatively new approach to development assistance. Encouraging staff closest to the project to identify processes that work well and those that do not could help the USG and its partners to use their human capital more productively in the future. Identifying promising processes (i.e., those that appear to work well) involves mapping inputs to desired outcomes, first theoretically, and then empirically. Although this would be best accomplished at project inception and iteratively reevaluated, the identification of promising processes can and should be an ongoing process. By adequately tracking activities that lead to increased workloads as the PFG progresses, areas can be identified for redirecting workload or teams can be reassigned to create a more balanced workload among PFG staff.


6. Cross-Cutting Question 3: What contributions has “non-assistance” made to the PFG process, and how can it be utilized moving forward?

USAID’s Policy Framework (2011–2015) defines non-assistance as “growth-oriented technical and scientific collaborations with agencies in the federal scientific community and efforts to raise awareness in the U.S. business community of commercial opportunities in developing countries.” Furthermore, non-assistance includes “policy levers, financing tools such as insurance products and guarantees from the U.S. Export-Import Bank and the Overseas Private Investment Corporation.”21 Particular to this evaluation, USAID defined non-assistance tools as those including “diplomatic engagement, convening authority, and other forms of non-monetized assistance to engage both governmental and non-governmental stakeholders in support of catalytic policy change and development priorities.”22

The evaluation team sought to identify whether non-assistance, as defined above, has made any contributions to the PFG initiative in the Philippines and how it can be utilized moving forward. Leadership, program managers, and independent experts within the USG and GPH were asked about non-assistance during semi-structured interviews, and examples were sought through the online survey. Documentation received from USAID prior to the evaluation team’s fieldwork in the Philippines was also carefully reviewed for evidence of non-assistance activities, but none were found. This meant that information gathered could only be based on perceptions gathered during fieldwork.

6.1 Findings

Finding 1: The concept of non-assistance and examples of its application is unclear to many PFG stakeholders (particularly lower-level technical staff). Knowledge about non-assistance is varied.

Stakeholder perceptions were gathered during the interview process as detailed in table 6.1 below.

Table 6.1: What contribution (if any) has non-assistance made to the PFG process, in relation to the Philippines?

Responses

USG GPH Overall total respondents Leadership Architects

Program manager/

CORs Leadership Architects

Program manager/

CORs

Yes 8 (42%)

3 (75%) 2 (40%)

8 (53%)

N/A 1 (33%)

22 (48%)

No 3 (16%)

1 (25%) 2 (40%)

1 (7%)

N/A 1 (33%)

8 (17%)

No response

8 (42%)

0 1 (20%)

6 (40%)

N/A 1 (33%)

16 (35%)

Total 19 4 5 15 N/A 3 46

21 http://www.usaid.gov/sites/default/files/documents/1870/USAID%20Policy%20Framework%202011-2015.PDF. 22 Provided by USAID evaluation POCs via e-mail to the evaluation team.


When stakeholders were asked about the contribution non-assistance has made to the PFG process in the Philippines, the majority of responses outside of high-level leadership, particularly on the USG side, indicated a lack of familiarity with the concept. Stakeholder responses included “not familiar,” “explain to me what this is,” and “I’m curious, what is it?” Those who were familiar with the concept of non-assistance could not provide many examples to demonstrate how it has contributed to the PFG. Another common answer provided to the non-assistance question, once the concept was explained, was that non-assistance was unrealistic as everything “involves money in some way.”

The question was answered differently by USG and GPH leadership, as demonstrated in table 6.1 above—eight out of 19 USG leadership and eight out of 15 GPH leadership staff members were aware of the contributions of non-assistance and non-assistance activities implemented as part of the PFG initiative. One member of the USG leadership team, for example, shared that non-assistance activities are announced through a report that is submitted to USG Washington. A GPH leadership official shared that they hear about non-assistance activities during budgetary meetings and at the Philippines Development Forum annual meetings, where USAID reports on non-assistance activities.

Findings from the online survey were similar to the interview findings in that more USG personnel were aware of non-assistance activities than GPH personnel were. As illustrated in figure 6.1 below, when asked if they have seen non-assistance tools being used, 74 percent of USG staff said “yes” and 42 percent of GPH staff said “yes.” Within GPH, 33 percent believed that non-assistance projects did not exist within the PFG Philippines initiative, whereas a much smaller number–10 percent of USG staff–believed those types of projects do not exist in the PFG Philippines initiative.

Figure 6.1: Survey responses regarding whether non-assistance is being used in the PFG Philippines Initiative


Figure 6.2: Have you seen non-assistance tools being used in the PFG activity with which you are or were involved? (Based on El Salvador evaluation survey responses only)

Finding 2: While non-assistance likely facilitated broader reforms in the Philippines, few specific positive results can be directly attributed to these efforts.

USG leadership officials explained that non-assistance activities are primarily originated and facilitated through the economic section of the embassy. The most frequently cited form of non-assistance involved diplomatic engagement, primarily through the former ambassador23 and USAID Mission Director. Most respondents referenced that the USAID Mission Director represented the United States as the main initiator of non-assistance activities in the Philippines. USG Mission Manila also actively leverages visits of congressional delegations and high-ranking officials as part of “non-assistance” efforts. Three GPH respondents identified non-assistance activities spearheaded by the economic section, however, they expressed that there was a need for more U.S. DOJ involvement in non-assistant activities.

Other examples of non-assistance activities are provided in figure 6.2 below.

Figure 6.3: Examples of non-assistance in the Philippines PFG Initiative, provided by interview and survey respondents

• Use of USG and USAID convening power authority to engage high-level stakeholders within GPH for policy discussions (and changes).

• U.S.-Philippines Bilateral Strategic Dialogues. • The State Department Economic Section has been working with GPH officials to advance trans-pacific

partnerships (TPP) and reform. • United States Trade Representatives engage with GPH counterparts on trade issues when the two

governments have complementary interests. • The Judges Swapping Program, through which U.S. judges come to the Philippines, provides peer

exchange. • U.S. assistance led the Philippines to be taken off the Intellectual Property watch list. • Better Than Cash Alliance Advocacy, which supports current thrust of government to move toward a

digitized economy. • Periodic sectorial and overall partnership joint reviews scheduled at senior-management levels on a

23 Note there is a new United States Ambassador to the Philippines, hence his involvement in the PFG has thus far been limited.


regular basis. • Endorsement of Philippine participation in U.S.-led initiatives (Open Government). • Engagement by U.S. Department of Justice (U.S. DOJ) with ombudsman on asset forfeiture framework

and other activities. • High-level visits from Washington, DC, to discuss needs with Philippine counterparts. • Working with national competitiveness authority. • Advocacy versus proposed trade-impeding measures such as 100 percent Customs pre-inspection.

Finding 3: Aligning USG foreign policy goals with development goals through the WGA has the potential to increase the impact of “non-assistance” and to advance those goals, but it might also reduce the effectiveness of development aid.

Non-assistance involves a blurring of lines between policy processes and aid programs, which can have both positive and negative consequences. By using the WGA to involve USG policy-makers through non-assistance, U.S. policy is brought to bear at relevant and important junctures. Some USG respondents were appreciative of the way in which the PFG program allowed them to increase their presence in the Philippine dialog on reform. However, at the same time, some GPH respondents expressed concerns that misunderstandings might result when the PFG agenda was being used to forward policy proposals in which the interests of the United States and the Philippines were not exactly aligned. The evaluation team also has concerns that the blurring of the lines between policy and aid under the rubric of non-assistance could lessen the effectiveness of aid for the recipient country. For example, preparation for the implementation of trade agreements and preparation for trade negotiations are two areas in which non-assistance is a natural tool. However, Philippine respondents were acutely aware that USG interests in these areas were not perfectly aligned with those of the GPH and this could lead to public perceptions—perhaps not accurate—that aid was being used to promote the USG agenda. Such perceptions would damage the credibility of the aid program in general.

The very concept of non-assistance might also have led to some disagreements between USG and GPH policy makers, particularly because some GPH officials think that non-assistance can lead to U.S. policy changes that would benefit the Philippines. However, these changes would not be easily implemented because they would require congressional approval. Certainly, there were instances in which GPH officials were under the impression that the United States would make policy changes as part of the aid program, when, in fact, such policy changes would have to be the product of processes completely separate from aid programs, some involving the U.S. Congress.

Finding 4: Not enough is known about the business opportunities created by the PFG and the value of changing public perception as a result of PFG non-assistance activities.

Respondents generally expressed a lack of clarity about non-assistance activities and did not understand that within their individual projects, non-assistance activities could be identified and initiated, possibly through WGA activities (that is, discussions among multiple USG partners). Three program managers said that they had no role in non-assistance at their level (table 6.1). Furthermore, the public affairs and public information system did not seem to have played a role in publicizing non-assistance activities to the extent that PFG partners and other stakeholders could understand and appreciate them.



The lack of familiarity of USG and GPH program managers and implementers with the concept of non-assistance tools may mean that opportunities to employ non-assistance will be missed. Within the Philippines PFG staff, the term “non-assistance” was not generally known. Only members of high-level leadership had a clear understanding of the term and were able to provide concrete examples of non-assistance activities. Other staff, particularly technical staff, believed that their role did not contribute to non-assistance activities. This confusion of the term could lead to missed opportunities for identifying and implementing non-assistance activities within the PFG initiative.

Embassy leadership, particularly within USAID, provided the clearest example of the successful use of non-assistance in the Philippines. The USAID Mission Director was cited as the most visible proponent contributing to non-assistance activities within PFG in the Philippines. Within the State Department, the U.S. Embassy’s Economic Section was also mentioned as the initiator and facilitator of most of the non-assistance activities.

The value of promoting PFG goals, attained and pending, is not always evident. Opportunities exist to leverage non-assistance activities to promote a positive adjustment of expectations for the investor community, foreign and domestic, and the citizens of the Philippines.


PFG participants need to become more knowledgeable about the diversity of forms non-assistance takes and its value should be shared with a broad array of beneficiaries. Given its importance within the PFG approach, the evaluation team recommends that the PFG, possibly through the PFG management team suggested in a previous recommendation, should conduct training(s) or other knowledge-sharing and communication activities related to non-assistance, and create awareness through trainings as new staff members are brought in. Within this training, there should be a concerted effort to identify past examples of non-assistance and activities that may lend themselves to non-assistance in the future. Such training would aid PFG staff in identifying opportunities for non-assistance and effectively translating examples of non-assistance to the larger stakeholders within their goals and projects/or activities.

Develop a management and reporting system for promoting non-assistance activities. From the findings, the evaluation team concluded that there was no structured reporting system even within the PFG initiative for capturing progress on non-assistance activities. This lack of a reporting system parallels with the lack of overall monitoring of JCAP implementation emphasized elsewhere in this report. While it is evident that given the nature of some non-assistance activities (including security), not all non-assistance activities can be discussed with a larger audience, it would be useful to include in PFG periodic reporting information on completed non-assistance activities to raise awareness about what non-assistance is and how it can be identified and leveraged within various PFG projects to benefit the entire initiative. Within this structure, it would be beneficial to clearly establish the role of the Manila U.S. Embassy’s Economic Section as the convener of non-assistance so that PFG staff know to contact the Economic Section staff when they identify potential opportunities for non-assistance. This clarity in roles will foster streamlined implementation, reporting, and quality control of activities. The evaluation team also believes that the Public Affairs Office within the USG could be instrumental in raising public awareness concerning non-assistance activities as they relate to the PFG.


Improve reporting and public awareness on non-assistance activities. The evaluation team recommends increased public awareness and reporting of PFG’s goals to improve the potential investment climate (promoting growth), decrease negative perceptions of rule-of-law activities (e.g., perceptions of judicial corruption), and set the expectations of the general public in terms of what the PFG initiative can and cannot do within non-assistance activities. An expectation that the PFG will produce a better regulatory environment or judicial climate (to promote both domestic and foreign investment) in the near future increases the attractiveness of the initiative and long-term revenue streams.

Increase staffing whose assignments are geared toward identifying and working on non-assistance activities. Based on the findings about non-assistance activities and the need to bring more awareness to various stakeholders, the evaluation team recommends that PFG Philippines consider increasing staffing geared toward non-assistance. Increased staffing will be needed because workloads will invariably increase if attempts to leverage non-assistance are increased in the future. The team is not necessarily suggesting that additional staff need to be hired. Within the existing structure, it would be useful to designate an individual (or individuals) who would be responsible for identifying non-assistance opportunities, especially related to public information and stakeholder mobilization within the PFG initiative. Also, it would be beneficial to work with the relevant stakeholders to highlight and report on such activities in a uniform manner. In the previous section, in response to Cross-Cutting Question 1, the evaluation team suggested having a core management team devoted to PFG activities. The evaluation team believes that identifying areas for promoting non-assistance could also be a task for the core management team, in conjunction with the existing Economic Section of the Embassy. This work would include some of the ideas provided above, such as facilitating knowledge sharing among new PFG staff (or reinforcement within existing staff) on the importance of non-assistance and identifying ways to conduct and promote non-assistance.


7. Country-Specific Question 1: For each of the constraints, are the goal-level commitments set forth in the JCAP alone capable of achieving the constraint-level objectives and outcomes?

Philippines PFG JCAP

As stated previously, GPH and USG agreed on a 5-year PFG JCAP in late 2011 to run through 2016. The JCAP was developed to provide an implementation strategy based on the findings from identifying and defining the main binding constraints to growth in the Philippines: weak governance and narrow fiscal space. These two constraints were further grouped into sub-constraints, goals, activities, and projects in the PFG initiative. The JCAP aligned with the targets of the GPH’s Philippines Development Program discussed previously. The JCAP listed measures (embodied in activities and projects) to be undertaken as part of PFG, indicators to monitor the progress of JCAP implementation, and an organizational structure for implementation.

7.1 Findings

Finding 1: The JCAP for the Philippines PFG is outdated and has not been updated in response to events even though it is intended to be a living document.

The initial goal was for the JCAP to be a living document—one that would be updated with economic developments, projects to be implemented, and lessons learned. The document itself stated that “the JCAP remains flexible and is intended to be reviewed annually, in order to make on-course correction and adjustment for subsequent implementation.”24 However, the JCAP has not been reviewed and updated, and events have causes the document to become outdated. The Philippine economy has performed much better than those working on the CA had anticipated, suggesting that the initial CA might need to be revisited because of events since it was first written. For example, the rate of gross domestic product (GDP) growth shifted from 3.64 percent in 2011 to 6.81 percent in 2012 and 7.16 percent in 2013. Currently, USG-led projects have been designed and are being implemented, with experience gained on which activities are feasible and progressing. Most important, the priorities of the USAID Mission, the prime implementer of USG PFG, has changed and are reflected in the development of a new Country Development Cooperation Strategy, which describes USAID’s program to support PFG. This strategy includes a broader set of activities than those that were originally the focus of the CA and embodied in the JCAP. For example, the implementation of programs under the Cities Development Initiative—currently a key emphasis of USAID Philippines—casts a different perspective on how the JCAP is being implemented and how it should be monitored.

In interviews, it was difficult to find a respondent who would say that the JCAP was at the center of their project design and implementation, and was an important part of their decision making (see table 7.1 below).

24 PFG Philippines JCAP, 2011, p. 14.


Table 7.1: USG and GPH interviewee responses about JCAP importance and familiarity with JCAP

Response Implementers Experts Architects Leadership Managers/ CORs Total

JCAP is central 0 0 1 (25%) 4 (12%) 0 5 (6%)

Familiar with JCAP 17 (63%) 1 (33%) 3 (75%) 16 (47%) 6 (75%) 43 (57%)

Not familiar with JCAP

10 (37%) 1 (33%) 0 6 (18%) 2 (25%) 19 (25%)

No response or question not asked

0 1 (33%) 0 8 (23%) 0 9 (12%)

Total 27 3 4 34 8 76

Some respondents said that they were guided by other documents, most often the Philippine Development Plan. Respondents within USG were open about the fact that the JCAP was now out of date and needed to be updated (table 7.2); however, at the same time, there were few respondents willing to seriously suggest that the JCAP should be updated. Among the GPH and the implementers of projects, knowledge of the JCAP varied. It is notable that newer staff members knew least about the JCAP and how it should be guiding PFG. This discrepancy is an indication that the initial influence of the JCAP is waning rapidly.

Table 7.2: USG and GPH interviewee responses on whether the JCAP is out of date Response Experts Leadership Total JCAP is out of date 2 (67%) 6 (18%) 8 (22%) JCAP should be maintained in its current state

0 3 (9%) 3 (8%)

No response or question not asked 1 (33%) 25 (73%) 26 (70%) Total 3 34 37

Finding 2: A formal theory of change linking activities, goals, and constraints was not fully developed.

The overarching idea behind the PFG process was that identifying key constraints would provide the basis for a clearly articulated theory of change (TOC), identifying the goals that need to be accomplished to loosen those constraints, and that identifying those activities (or projects) would be crucial in reaching those goals. In attaining such an objective, it would be logical to develop a TOC or logical framework that would ensure that activities are mapped to goals and to constraints. The TOC is also important to ensure that all agencies associated with PFG have an aligned set of goals and are aware of outputs that will lead to PFG projected outcomes. In its review of the PFG initiative, the evaluation team found that, although partial and implicit TOCs can be found in the JCAP and subsequent project SOWs, no formal theory of change was ever articulated for PFG as a whole. It is evident that constraints were initially mapped to goals, but goals were not adequately mapped to activities (projects), which could mean that projects would not always be relevant to PFG. A comprehensive alignment is also crucial given the inclusion of new goals and activities.


There are several consequences of the absence of an overall detailed PFG TOC:

• Those initiating projects are not provided with clear guidance about which projects are the most important to implement for the success of the JCAP and the relief of the crucial constraints. For example, the Science, Technology, Research and Innovation for Development (STRIDE) and Scaling Innovations in Mobile Money (SIMM) projects are both important Mission initiatives. STRIDE focuses on higher education, and the JCAP mentions that in this area of activity, “the primary focus will be to help improve regulation of institutions to improve the quality of post-secondary education.” The evaluation team found no clear explanation of why a focus on higher education follows from the CA and why the focus on regulating post-secondary institutions followed from the focus on higher education. Similarly, the evaluation team could find little in the JCAP and CA to justify the inclusion of the SIMM project under PFG. This lack of explanation could be justified by the fact that the SIMM project was already being developed before the JCAP was finalized; therefore, it constitutes a bridge project. Nonetheless, a follow-up on a project after SIMM, E-PESO, is already being developed under PFG, and the comments above apply to this new project.

• Given that an overall uniform TOC does not exist, it is difficult for outsiders to understand the selection process for goals and activities. This difficulty is particularly concerning with an initiative such as PFG, which had the potential to make the aid process much more politicized and to directly involve administrators and politicians from two governments, with goals that could not be expected to completely coincide. As one expert commented, “Sometimes the cause of a problem is the activities to which the current government is committed, and therefore relying on the government itself to identify both the problems and the solutions would likely not yield desirable results.”

• This situation could also mean that some key goals and activities that would lead to constraints-level outcomes could be overlooked when formulating the list of projects to be implemented under PFG. For example, improved regulation of the financial industry does not appear in the list of activities emphasized in the JCAP, but there is little analysis that suggests why it was omitted, and why competition policy, for example, was emphasized.

• Having no overall defined TOC to link activities, goals, and constraints means that there is no clear guidance on how project-level accomplishments link to the attainment of goals that relieve the constraints. Hence, it is difficult to draw a link between M&E of the implementation of detailed elements of projects, and M&E of the more macro indicators that summarize progress of PFG and implementation of the JCAP.

• An explicit overall TOC might have brought more attention to the interactions between activities that are aimed at different goals, which has implications for the successful implementation of the WGA.

The overarching question for this section of the evaluation is, “For each of the constraints, are the goal-level commitments set forth in the JCAP alone capable of achieving the constraint-level objectives and outcomes?” This particular finding suggests that the JCAP does not provide a solid basis to directly answer this question.


Finding 3: Additional goals and activities were included in the Philippines JCAP that were not emphasized in the CA.

The evaluation team was not able to fully determine why human capital was included as a constraint in the JCAP. In particular, Goal 5 under the JCAP regulatory constraint was not explicitly recommended as a goal in the CA. Indeed, the CA concluded that, “Based on this assessment of available data and evidence, human capital, while a critical constraint to some growth and emerging industries in the service sector, does not, on balance, appear to be a binding constraint to growth in the Philippines.”25 Additionally, it added that “the supply of educated workers currently outstrips demand.” Moreover, the JCAP noted the following in commenting on the CA: “Although significant drags on more inclusive economic growth in their own right, deeper analysis revealed that the infrastructure and human capital constraints are largely the consequence of limited fiscal space.” Certainly, this constraint does not seem to foreshadow the goal in the JCAP—“enhance human capacity for economic growth”—by which “PFG seeks to provide assistance to strengthen higher-education institutions in the country to serve as a key driver for innovative, sustained economic growth.”

Certainly, there was more of an emphasis on inclusive economic growth in the JCAP than in the CA, and this increased emphasis was one reason offered by USAID leadership when asked about the change. Nevertheless, although a focus on tertiary education might seem relevant to an upper-middle-income country that is approaching high income, it does not seem so relevant to inclusivity in a lower-middle-income country such as the Philippines. USAID leadership also stated that it undertook additional research and analysis during JCAP preparation that justified the inclusion of the human capital activities in the JCAP based on the argument that there was an imbalance between the skills imparted by the higher education sector and those needed by industry, services, and commerce. While not doubting the argument that such higher education problems were a constraint to growth, the evaluation team did not find evidence to warrant the change to the level of a binding constraint at the level of the binding constraints identified by the CA. Thus, in the final analysis, the evaluation team would have to conclude that this goal was introduced into the JCAP in a process external to the PFG’s structured approach. This conclusion does not mean that the evaluation team felt that the addition of human capital to the JCAP was an incorrect judgment; rather the evaluation team notes that this judgment was made outside of the PFG framework of the CA-driven identification of constraints and has not been fully justified in any documents made available to the evaluation team.

The addition of an extra—and highly funded—goal weakened the analytics of the mapping from the constraint-level objectives and outcomes to the JCAP. Furthermore it potentially stripped other, more targeted interventions of needed resources.

25 Partnership for Growth, Philippines Constraints Analysis, 2011, pg. 48


Finding 4: A number of JCAP goals were not being fully implemented.

The evaluation team found that a number of JCAP goals were not being fully implemented. If the CA and JCAP are to be seen as an integrated program, all goals should be addressed. Instead, certain goals received minimal or no funding, while other goals that were not addressed by the CA were introduced into the JCAP. For example, under the weak governance constraint, JCAP Goals 13–14, which focused on strengthening ethics and public accountability measures(activities laid out in the JCAP), have not been addressed by subsequent USAID or USG programming where such programming was envisaged in the JCAP. The JCAP Annex B states that the DOJ Office of Overseas Prosecutorial Development, Assistance & Training(DOJ/OPDAT) will “work with DOJ/Office of Inspector General and DOJ/Office of Professional Responsibility to develop comparative models for procedures for prosecutor discipline, reflecting U.S. experience and DOJ/OPDAT experience in support in this area overseas” and that USAID will “provide [technical assistance] to justice sector agencies to support the internal affairs unit to effectively enforce professional standards and discipline,” in addition to “[providing technical assistance] to the Justice Research and Training Institute, including support for evaluations and assessments.” Neither of these activities appeared in the progress reports or the SOW of subsequent USAID or DOJ projects. Given the lack of information on JCAP implementation management, the evaluation team was unable to determine if this change was because of a shift in priorities or a realignment of funding to other programs.

Finding 5: During the development of the JCAP, there were proactive efforts to absorb the views of a broad cross-section of society; nevertheless, there were differences of opinion on whether the CA and the JCAP development process were inclusive.

Respondents characterized the CA as a rather secretive process that involved a narrow set of experts (from USG and GPH), but not broader representation. Even some members of USAID felt that they were not fully involved in the process, as much as would have been appropriate. Many of the civil society and business community interviewed by the evaluation team also felt they were not directly involved at the CA stage of the development of PFG. However, CA did absorb the results of a study produced by one important group in the private sector, the American Chamber of Commerce in the Philippines, whose 2010 study, Arangkada Philippines 2010: A Business Perspective, detailed the concerns of members of the private sector and was styled as an advocacy report.

After that, in the formulation of the JCAP, there were efforts to reach out to civil society and the private sector. The team that developed the JCAP included a broad cross-section of officials from USG and GPH. There were forums in which representatives of civil society and the private sector could express their concerns. Respondents differed on the degree to which this effort was effective, in terms of response and in terms of how much the JCAP reflected a broad cross-section of views. One GPH respondent assured the evaluation team that civil society organizations were fully supportive of the JCAP, while another expressed the opinion that civil society was not involved actively in the PFG process, although the latter view was a minority one. Another respondent referred to “huge consultations,” and inviting “stakeholders to meetings to talk about constraints analysis” and another to “meetings with leaders, roundtables with civic society organizations (includes media, NGOs, etc.).” One respondent, who would have been more well-versed in economic aspects than in rule of law activities, felt that civil society had the biggest input in the conceptualization phase of PFG and the JCAP, but felt there was less involvement once the implementation began.


A number of independent and civil society experts expressed doubts about the ability of the JCAP goals and subsequent projects to address the constraints. They cited politically sensitive issues such as land reform, public procurement reform, and foreign ownership of land as important issues that the government is not willing to address. The evaluation team also identified a number of important issues from the CA that were not addressed in the JCAP goals, including robust support to civil society and private-sector monitoring of needed reforms.

Some GPH agencies use elaborate mechanisms to integrate civil society and private sector input into policy decisions, such as budgetary decisions, but it was not clear how much these particular mechanisms carried over into the implementation of PFG projects, especially since implementers would naturally be more attuned to the requirements of USAID and other U.S. funding agencies than GPH. One expert reported that there was no engagement of civil society in the official meetings of PFG, but several projects naturally interact with the private sector and with consumers, such as SIMM, where the private sector is a major implementer and one of the goals of the project is to attract consumers to the service being offered. In such projects, catering to the needs of the private sector are almost a given, and implementers do a follow-up after implementation by getting feedback from those affected by projects.


Without a fully articulated theory of change that guides the overall implementation of the PFG, it is not possible to provide a theoretically grounded assessment of whether or not achieving goals included in the JCAP can have the desired effect on constraint-level objectives and outcomes. The JCAP does not provide a rationale for choosing to emphasize the 17 goals that are emphasized under the JCAP. Without such a stated rationale, it would be speculative to argue for or against the inclusion of any goals and, by extension, their associated activities or projects.

In lieu of a theory of change, it is difficult to determine whether the JCAP goals were chosen partially based upon bureaucratic considerations or the organizational convenience of decision makers. The clearest example of this consideration is the addition of the important human capital goal under the regulatory quality constraint; a decision that contradicted the CA.

It becomes harder to revise a JCAP that is outdated without a fully developed theory of change. The Philippines provides an example of a situation in which economic events have moved much faster than was expected at the time the JCAP was formulated. If the JCAP contained a comprehensive theory of change, perhaps revision would be much easier, since the relationship of goal choices to underlying conditions would be much more explicit.

The ambitious goals in the JCAP are unlikely to be accomplished given the resources currently available for PFG activities. This unlikely outcome inevitably leads to challenges to implementation in which decisions must be made about which parts of broad goals to pursue, and then later in M&E, where failure is more or less bound to arise because the plan is ultimately unattainable.

Even though broad and ambitious, the JCAP did not satisfy some influential elements of society (e.g., civil society, independent experts, business community). Some representatives of these groups felt that the CA and JCAP formulation, especially the former, did not include adequate input from broad cross-sections of society, thus defeating the purpose of the expansiveness of the JCAP.



Clarify the status of the JCAP and update as necessary. The evaluation team recommends that outdated elements of the JCAP should be updated to solidify the PFG process moving forward. The updates need not be complete and comprehensive, but could be a simple clarification of changes and inclusions. The evaluation team recommends that this process be spearheaded by the bilateral Steering Committee so that there is adequate buy-in from both governments on the changes to be included. Furthermore, on the operational side, the PFG management team recommended previously could work as the coordinator for bringing together the needed stakeholders for this activity, tracking the changes made, and reporting on the final outcomes.

Better align the JCAP with available resources. Credibility can be enhanced by making sure that what was promised in the JCAP can be delivered with the resources available from USG and GPH under the WGA. One revision to the Philippines JCAP might then be to prune the long list of activities and overambitious timelines that it envisages to correspond to what has been implemented up to the present. This process would aid in ensuring that the current indicators for projects and goals can lead to attaining the constraints-level outcomes.

Be more proactive in involving civil society and the private sector at the implementation stage. Not all project implementers were cognizant of the need to involve the broader society when considering how their projects would be implemented. Sometimes a narrow technical focus caused implementers to ignore the broader society even though the implemented activities could have a widespread effect.

The PFG management team could leverage already existing PFG meetings to train program implementers on the need to include more civil society or private sector activities (as applicable). Further, the PFG M&E team should ensure that publicizing PFG activities and involving the broader beneficiary body is a performance measurement for PFG implementation, and identify accountability strategies for enforcing this activity.


8. Country-Specific Question 2: Is quantitative and objectively verifiable information being used to manage JCAP implementation in order to achieve and measure results?

This evaluation question contains several conceptually separate elements. First, there is the distinction between the implementation of the JCAP and the implementation of its individual constituent activities. It is entirely possible that, even if the constituent activities are being individually managed using appropriate information, the implementation of the JCAP itself—as a comprehensive program—is not being managed. Thus, the evaluation team addresses below managing the implementation of the JCAP and managing the implementation of the individual programs. Second, this question addresses what information is being used to manage the JCAP and its activities. There are three subparts to this element of the question: To what extent is a system of management in place? Is quantitative and objectively verifiable information being collected? And finally, is that information being used as a tool of active management? The findings below address all of these issues.

Management Structures of the Philippines PFG JCAP

Progress on the Philippines’ PFG performance is supposed to be tracked through indicators. These indicators measure three inter-related development interventions (referred to as sub-constraints for this evaluation), namely regulatory quality; rule of law and anti-corruption; and fiscal space or performance. To ensure that the JCAP is being implemented effectively and that the desired outcomes are being achieved, the initiative envisages a rigorous, transparent M&E process. Using an evidence-based M&E framework, the JCAP envisioned USG and GPH working together to identify targets and track progress on macro and sectorial-level indicators to best demonstrate performance against benchmarks. This M&E framework was expected to cover:

4. A process for selecting the benchmarks and indicators associated with each objective; 5. A relevant entity tasked with collecting M&E data to present to the Steering Committee

(Technical Sub-Committees, an office within USAID, or the PFG Secretariat); and 6. Expectations or plans to collect M&E information from all USG and GPH agencies.

This process was to include meaningful participation by civil society and the private sector, and was to be approved by the PFG Steering Committee. According to the JCAP, the goal was to establish the framework by December 2011.26 Based on the documents reviewed, the evaluation team found that the first of the three Steering Committee meetings did not occur until April 2012. Additionally, none of the reports from the three Steering Committee meetings included a discussion of this M&E framework other than project and activity updates.

26 Partnership for Growth Joint Country Action Plan, 2011, pg. 13


Other meetings or management activities were referenced in documents and interviews. However, evidence related to these meetings has not been made available to the evaluation team. These meetings and activities include:

• a rules-of-engagement workshop involving GPH and USG; • information about meetings of the USG Interagency Coordinating Team proposed in the JCAP; • information about meetings of the PFG Secretariat; • meetings of the Steering Committee to identify specific policy areas for discussion with the

USG; • agendas and minutes of the Technical Subcommittee on Regulatory Quality meetings that

occurred subsequent to the July 5, 2012 meeting (interviewees indicated that several meetings had taken place); and

• agendas and minutes of the Technical Subcommittee on Fiscal Space meetings subsequent to the June 21, 2012 meeting (interviewees indicated that several meetings had taken place).

8.1 Findings

Finding 1: The implementation of the JCAP—viewed as a comprehensive integrated program—is not being adequately managed.

The JCAP document articulated a very ambitious program for managing implementation: • A PFG Joint Steering Committee would meet every 6 months, provide policy direction, and

oversee the overall progress of the PFG JCAP. The committee would have members from a large number of agencies: it would assess the progress on both GPH and USG commitments, and assess collaboration among participating agencies and civil society organizations.

• There would be three technical working groups; one for each of the major constraints identified in the JCAP—regulatory quality, rule of law and anti-corruption, and fiscal performance—with these groups providing advisory support in pursuit of program goals and objectives, resolving implementation issues, and preparing and reviewing technical documents required for Steering Committee deliberation.

• A PFG Secretariat would provide administrative support to the Steering Committee, such as undertaking follow-up actions in support of Steering Committee decisions and serving as the liaison between the Steering Committee and the technical working groups.

The goal was to convene the Steering Committee by November 2011 and the technical working groups by January 2012.

These were ambitions plans, with follow-through being less than commensurate. No PFG staff interviewed stated that the management structure was in place and functioning as proposed. Also, the Steering Committee has only met once every year and has not fully met its function of overseeing the whole progress of PFG. The evaluation team could find no record of any major decisions of the Steering Committee that had significantly altered the direction of JCAP implementation, which would suggest that the committee was playing an active PFG management role. The technical subcommittees have met more times but still infrequently, and the meetings were not well attended. The subcommittees have mainly acted as forums to hear ad hoc reports on the implementation of individual programs, without being active in the management of PFG activities within the areas of competence of the working groups. Similarly, the


evaluation team was not able to identify any major decisions by the technical working groups that would suggest that the groups were playing an active role. Finally, the evaluation team did not find any information that would suggest that a Secretariat exists and is playing a management role that would spur or track actions by the Steering Committee or the technical working groups.

Finding 2: The M&E framework required by the JCAP has not been finalized and as a result, indicator tracking is not occurring at the initiative level

The JCAP also planned for GPH and USG to work together to establish an evidence-based M&E framework by December 2011. The intention was to have a standardized M&E tracking system and to involve the PFG Steering Committee intimately in monitoring the overall implementation progress and ensuring the proper monitoring and evaluation of activities. Monitoring was supposed to occur at three levels: for PFG as a whole, for the three constraints considered individually, and for all activities, conducted by USG and GPH.

In February 2012, the Philippines-U.S. Partnership for Growth Amplified Monitoring and Evaluation Plan, the document to be used to begin implementing this M&E process at the highest JCAP level, was produced. This document is evidently an interim one, because it is not the JCAP-mentioned framework, but rather announces the intention to develop such a framework: “to ensure that the JCAP is being implemented…using an evidence-based Monitoring and Evaluation (M&E) framework… [it is] expected to cover the following:

1. a process for selecting the benchmarks and indicators associated with each objective; 2. a relevant entity tasked with collecting M&E data to present to the Steering Committee

(Technical Sub-Committees, an office within USAID, or the PFG Secretariat); and 3. expectations or plans to collect M&E information from all USG agencies and GPH.

…The framework will be drafted by April 2011 [sic] and approved by the PFG Steering Committee.”27,28

According to USAID Philippines, the framework—which was forecasted for December 2011 by the JCAP and then for April 2012 by the interim document—exists and is under review. However, the evaluation team requested it but did not receive it, which raises the question as to how the overall PFG initiative is being monitored in a systematic structure given that PFG implementation has already been underway for more than 2 years.

The February 2012 document also stated that “the GPH and the USG intend to conduct a high-level review of JCAP implementation each November…This review should focus on progress towards alleviating the constraints and achieving the overarching PFG goal of accelerated and inclusive economic growth.” Based on interview responses, this review did not take place in 2012 or in 2013..

There is one scorecard of indicators that has been adopted by PFG Philippines to indicate overall progress, the latest report of which was on April 14, 2014. This scorecard uses 10 high-level indicators that report on outcomes and are available from public sources. The scorecard, which is a useful tool for measuring the overall progress of the Philippines, states its purpose is to measure contribution but not

27 Philippines-U.S. Partnership for Growth Amplified Monitoring and Evaluation Plan, 2011, pg. 1. 28 Note that the quoted April 2011 date is obviously a mistake; the assumption is that this date was a typo and 2012 was intended.


attribution of PFG to these higher level results. The scorecard also indicates that “indicators for outputs and outcomes for specific PFG interventions/programs will be linked to the higher-level outcomes of PFG.”29 However, the evaluation team did not find any indicators that explicitly linked to scorecard higher-level outcomes. As a result, the scorecard and related undeveloped indicators do not serve an M&E or a results-based management purpose for PFG Philippines. The indicators from the scorecard are too high-level to serve this purpose, and there are no linked indicators with established targets so that actual performance can be compared with expectations. An articulation of how PFG USG and GPH activities under the JCAP contribute to these high-level indicators and the inclusion of a more diverse set of direct and mid-level indicators would be more instructive. Additionally, monitoring of the progress on the three individual constraints (i.e., regulatory quality, rule of law and anti-corruption, and fiscal performance) shows similar characteristics. None of the technical subcommittees have issued standards or guidance on M&E. Within individual technical subcommittee meetings, there have been ad hoc reports produced on the progress of individual projects and activities, but this reporting process has not reached a level of formality in which it could be described as systematic monitoring. Furthermore, there are no scorecards for the individual constraints that disaggregate indicators beyond what is in the scorecard, as previously mentioned. This finding is surprising given that the JCAP document, early on in the PFG process, provided guidance on how tracking can be done at the constraint level.

Finding 3: PFG stakeholders felt that, in some instances, better indicators could have been selected to monitor PFG, but they did not readily provide alternatives.

When respondents were asked if the indicators prescribed for PFG are generally appropriate for monitoring the initiative, the following was found: Among USG leadership, 57 percent indicated that “the best indicators are being used,’’ and 43 percent said that the indicators being used were some of the best (see figure 8.1). Among non-leadership in USG, 13 percent stated that the best indicators are being used, 33 percent stated that some of the best indicators are used, 29 percent were not sure how to judge the appropriateness of the indicators being used, 17 percent were unaware of the indicators being used, and 8 percent did not provide answers.

29 Philippines – U.S. Partnership for Growth Scorecard as of April 14, 2014


Figure 8.1: Are the appropriate indicators being used to allow for transparent, accountable, and fact-based monitoring of the PFG? (Based on survey responses only)

As shown in figure 8.2, 50 percent of GPH leadership staff stated that the best indicators were being used, while 50 percent stated that they were not sure how to judge the appropriateness of the indicators being used. None of the leadership staff within GPH who responded to the survey indicated that the current indicators were not the best. However, among GPH non-leadership staff, 10 percent expressed that the best indicators were being used, 30 percent stated that only some of the best indicators were being used, 30 percent were not sure how to judge the appropriateness of indicators being used, and 30 percent were unaware of the indicators being used.

Figure 8.2: Are the appropriate indicators being used to allow for transparent, accountable, and fact-based monitoring of the PFG? (Based on survey responses only)

Overall, only 12 (three from GPH staff, and nine from USG staff) of the 43 survey respondents provided suggestions for alternative indicators, and few of these suggestions were clearly defined. Some examples of suggestions provided were “qualitative indicators,” and “control of corruption cases.”


Finding 4: The M&E plans of individual USAID projects are comprehensive, complete, and ready to be used to manage activities, but many were recently approved and thus they have not been used to manage JCAP implementation.30

The USAID-funded projects are only one part of PFG, but as one of the largest USG-funded elements, their use of M&E plans to capture quantitative data for managing results is essential. As a result, the evaluation team closely reviewed USAID project M&E and found that every USAID PFG project currently has a complete M&E (or performance management) plan. This is largely because USAID project contracts have a clause indicating that an M&E plan will be completed within 60 days of the beginning of the project, and to the knowledge of the evaluation team, this requirement was satisfied in all cases. Project implementers regard this step as completing their contractual requirements. Once completed, these M&E plans are submitted to the USAID contracting officer’s representatives (program managers). The review phase for the M&E plans seems to have been a thorough process, one which has proceeded while the evaluation has evolved. At the time the evaluation team was in-country, a number of respondents from projects indicated some uncertainty about whether their M&E plans had been finalized. Not having final approval on these M&E plans meant that implementers were in doubt about whether performance on the indicators in these plans can be used to make management decisions on their projects. It also indicated that the PFG initiative is not yet using the full complement of quantitative and objectively verifiable data to measure results. As shown in table 8.1, the evaluation team received a full, updated table of the status of USAID M&E plans shortly after the field portion of the evaluation concluded, confirming that plans had been finalized and, by now, approved. Table 8.1: PFG implementation: Summary status of USAID project M&E plans31

Status OEd OEDG OEECC OH PRM Total Approved M&E Plans 6 12 9 7 2 36

PRM review good practice but not required/old guidance

2 (JOBS, IDEA)

5 (INVEST, TRADE,

SIMM, P2PE, Strengthening

CSOs)

5 (3MPA, RARE, CREST, Abuan,

ECOFISH)

4 (PRISM,

LuzonHealth, ROMP, HLGP)

2 (SELP,

EXPAND)

No PRM review/new guidance

- 2 (ATiP,

Elections Fiji)

- - -

With PRM clearance

4 (BASA, MyDev, EDGE,

STRIDE)

5 (TAPP,

PhilAm Fund, i3, ENGAGE,

JUSTICE)

4 (B-WISER, AMCCAP, BAWP, Be

Secure)

3 (CHANGE, IMPACT,

MindanaoHealth)

-

Awaiting final draft M&E Plan

2 (COMPETE,

FPI)

1 (B-

LEADERS)

1 (Visayas Health)

1 (WPS)

5

Under review 1 2 3

30 At the time of the evaluation team’s visit in June 2014, there were a limited number of approved M&E plans. Subsequent communications with USAID have indicated that almost all USAID project M&E plans for active projects are now finalized. 31 The initial summary plan was received from USAID on August 25, 2014, and later updated and provided on October 10, 2014.


Status OEd OEDG OEECC OH PRM Total

by C/AOR 1 (Logistics)

2 (CMSU, HPDP2)

by PRM Awaiting first draft M&E Plan

Not yet due 1

(GEM 3 extension)

1

(Status not yet determined; awaiting M&E plan submission of C/AOR)

1 (A3B to Land

Conflict)

1

Total 6 17 10 10 3 46

Finding 5: Quantitative information is being collected at the project and activity level, but it is not (yet) being assembled, analyzed, and used to manage JCAP implementation.

The amount of information being collected at the project level is impressive (given that most implementers seem to be already guided by their—albeit as yet unofficial—M&E plans. Table 8.1 above provides detail on monitoring status). For example, the Investment Enabling Environment (INVEST) project collects data on 36 indicators derived from a systematic and articulated results framework, and these indicators are integrated into a well-organized logical framework. The indicators include baseline values against which performance can be measured. The indicators are collected both at the highest level relevant to the project (e.g., number of business start-ups) and at the most micro level relevant to the project (e.g., number of days spent on technical assistance for the Business Permits and Licensing System in partner cities). Furthermore, between the high-level macro indicators and the rather traditional performance indicators, there are also real measures of the micro impact of the project, for example, the results from a client satisfaction index based on exit interviews conducted on the permits and licensing system of partner cities where INVEST has provided technical assistance. It must be stressed that this example—the M&E plan of INVEST—is typical of the M&E plans that have been developed by projects funded by USAID in the Philippines.

The evaluation team asked implementers through interviews whether their work was guided by M&E plans. The responses are provided in table 8.2 below:

Table 8.2: Implementers actively using an M&E plan Response Implementers Guided by M&E plan 24 (89%) Not guided by M&E plan 0 No response 3 (11%) Total 27

Once quantitative and objective information is collected, the next issue is to identify whether or not the information gathered is used for management; concerning the latter, the issue is clouded and the evidence mixed. For example, most implementers felt that the management within their projects was guided by their M&E plans, as evidenced in table 8.2 above. The evaluation team interviewed implementers who


felt that the information they collected for the M&E indicators was used actively in the process of judging their performance. If these implementers fell behind on their targets, they would have to explain how they would address the problem and then detail an action plan to explain how they would improve performance. On the other hand, some implementers felt that the comprehensive quantitative information they submitted to USAID was not integrated into the detailed management of their activities. These implementers were either not sure how their performance was judged or felt that USAID was more interested in the high points of activities, such as publicity-generating events.

GPH agencies are also actively tracking progress using indicators and government statistics, including USG-supported projects. These data are widely available, integrated into government reports, and used for management, but the data gathered has not also been utilized by PFG to measure progress or to elaborate on the scorecard, according to findings from the evaluation, evidenced in table 8.3

Table 8.3: Implementers evaluated by indicators in M&E Implementers Actively evaluated by M&E plan indicators 2 (44%) Not sure how performance is evaluated/use their own indicators 22 (44%)

No response 3 (11%) Total 27

Not surprisingly, projects vary in the degree to which quantitative and objectively verifiable information is used to make internal decisions about where to focus efforts and to assess whether those efforts have led to success. In general, project managers seem to be imbued with the ethos of applying quantitative methods to improve the efficiency of implementation. International data sources are used to guide activities, and data from different types of surveys and observations of processes are used to assess outputs of activities.

Interestingly, the use of quantitative and objectively verifiable information to make decisions seems to become less important at higher levels and becomes more aggregate in nature, reaching the level of policy. Program managers and implementers were asked to provide information about how decisions about a major change in direction on their projects would occur within USAID, and none articulated a data-driven process. In contrast, it was not difficult to find examples of cases where major elements of projects were driven by decisions that could only be characterized as political or ad hoc in nature.

Finding 6: Individual activity M&E plans and indicators are not mapped to the overall M&E of PFG implementation.

It is not surprising, given the standard procedures of M&E design within USAID, that the M&E plans of different projects have indicators in common and indicators that can be aggregated upward into more composite measures of performance. Thus, there is great potential for synergy across the different projects. However, the evaluation team has not been made aware of any systematic process by which the indicator reports of different projects are being aggregated and made into composite reports that detail the progress in the 17 areas of activities listed in the JCAP, the three constraints of the JCAP, or the PFG program itself. This lack of communication means that a great opportunity is being missed because quantitative and objective information is being collected, but not aggregated and synthesized.



Based on the evaluation team’s findings, quantitative and objectively verifiable information is not being used universally to manage JCAP implementation. The majority of activities on the USG side are being monitored using USAID M&E procedures, as they were funded by USAID, but it is not clear whether this monitoring translates into active management. Objective quantitative data on progress from other USG agencies was also generally lacking. In addition, there was no reporting on PFG-specific progress from GPH, or efforts by USG to document how far GPH has advanced in its PFG commitments. In sum, the use of quantitative and objectively verifiable information to manage JCAP implementation at higher levels is minimal.

JCAP implementation—viewed as a comprehensive integrated program—is not being managed. The JCAP itself had ambitious plans for how the program was going to be managed, and not surprisingly these timelines and plans have not come into fruition. Further, the Steering Committee has only met sporadically and has not met its function of overseeing the whole progress of PFG. The technical subcommittees have acted as forums to hear ad hoc reports on the implementation of individual programs, but they have not been active in the management of PFG activities.

Since the overall M&E framework required by the JCAP has not been finalized, there is no explicit framework within which overall JCAP implementation might be monitored and evaluated. Thus, it is not surprising that the assessment of JCAP implementation called for by the Steering Committee in November 2013 did not take place. This lack of framework means it is difficult to compare performance across projects and to aggregate the results of activities up to the constraint level.

At the individual project level, during the time of the evaluation team’s fact-finding visit in-country, it would have been difficult to authoritatively use quantitative and objectively verifiable information for management since the M&E plans of most individual USAID projects had not been finalized at that point. However, since then, by October 2014, a significant number of plans received final approval. These plans are comprehensive and complete, and ready to be used to manage activities, but at the moment their official status is unclear, so implementers are not sure whether or not the plans are being used to manage their projects.

It is widely believed that at least some M&E indicators could be improved, but alternatively, improved measures were not easily provided. Few interviewees and only one in five survey respondents felt that the indicators currently being used were as good as alternative indicators that were potentially better. Only a few respondents, mostly leadership, could provide specifics on what a proposed superior indicator might look like.

Although individual activities have M&E plans and indicators, they are not mapped into a process for M&E of overall PFG implementation. This finding follows almost automatically from the previous conclusions in the absence of an overall coordination of M&E of all projects and in the absence of a strong M&E at the overall JCAP level.

Although quantitative information is being collected at the project level, it is not (yet) being used to manage JCAP implementation. Implementers and managers did not provide a consistent picture of how information is being used by the funders to manage the implementation of activities.



A decision should be made concerning the most appropriate system through which PFG-JCAP implementation should be managed. Deciding not to manage the program, qua program, and instead treat it as an umbrella to roughly describe an overall set of projects would not mean that PFG is declining. Rather, it would be a different type of program than outlined in the initial signing statements and the JCAP. It would be a way to describe a thoughtful and somewhat integrated approach to the design of an array of related USAID programs. On the other hand, if the ambitions of the JCAP are to be executed, then the Steering Committee, the technical subcommittees, and the Secretariat need to be more active, meet more frequently, and track the outcomes of decisions made during these meetings. The evaluation team also recommends that the M&E framework required by the JCAP be finalized quickly and publicized, with the overall framework providing guidance on tracking at all three levels, as stipulated by the JCAP.

Provide clear guidance on the role of developed indicators to the overall initiative, and include program managers and implementation team management in review of indicators that may not be appropriate. Based on findings from the survey, it was determined that not all the indicators assigned to PFG initiatives are appropriate. However, only a few respondents were able to provide appropriate alternatives. Therefore the evaluation team recommends that during the review of the JCAP and development of an M&E framework for PFG, as previously discussed, it would be useful to review the existing indicators with the program managers and implementation team management to ensure that the indicators are appropriate and realistic, their interdependencies are identified, and the fact that they can lead to appropriate tracking of PFG progress and impact. Guidance should also be provided on how activity-level indicators, goal-level indicators, and scorecards relate to the constraint-level indicators. Having all PFG partners and implementation teams understand these links would facilitate the achievement of desired PFG outcomes. USAID should clarify how the detailed indicators in project M&E plans are to be used as quantitative management tools. At the moment, project managers are uncertain about the implications of not meeting M&E targets, and how their targets feed into overall PFG progress. Therefore, the evaluation team recommends that the technical subcommittees provide clearer guidance (and possibly training) for effective program tracking to ensure that implementation is in line with JCAP expectations.


9. Country-Specific Question 3: At the midterm, are the performances of the selected PFG interventions on target and creating the necessary outputs to achieve the desired outcomes?

This evaluation question inquires whether selected PFG interventions are on target. The evaluation team sought answers to this question by reviewing whether the overall PFG intervention is perceived to be on target, through interviews; and then conducting an extensive review of six selected goals to respond to this question.

Documented data sources for these case studies include a review of M&E plans, as available; project reporting through quarterly and annual reports; scorecards. To gather further information, the evaluation team interviewed not only program managers of the selected goals, but also various implementers and independent experts affiliated with these goals. The findings based on these evaluation strategies are provided in this chapter.

This question about the progress of PFG Philippines at midterm produced a strong reaction from most stakeholders. USG respondents, in particular, felt it was far too early to be conducting a mid-term evaluation, as projects had only recently begun implementation. The evaluation team found the definition of “mid-term” for PFG unclear. The beginning of PFG could be designated as either the day the JCAP was signed, or the day specific PFG projects began implementation. If PFG progress is measured by specific projects, then the interventions are uniformly behind schedule as most are 5-year projects that have recently begun or just finished their first year of implementation. However, if PFG progress is measured more broadly, then the consultative design phase must be taken into account, in which case, the agreed-upon design and procurement of projects by Year 2 is a success. Others have emphasized that as a holistic approach to USG interventions and GPH collaboration, PFG progress should be measured based on the success of implementing key reforms and not just through USAID projects. Regardless of the definition of “mid-term,” the lack of a comprehensive M&E framework for the entire PFG initiative in the Philippines meant that USG and GPH are not regularly reporting on or have defined progress. This lack of a framework hampered the evaluation team’s ability to measure whether PFG was “on-target” at mid-term.

Ten out of 19 respondents from USG PFG leadership interviews indicated that PFG-selected goals and corresponding activities are on target (see table 9.1 below). Also, GPH leadership officials (3 positive, and 3 negative) who responded to this question were split in terms of who thought that PFG interventions were on target or behind, particularly due to perceived delays in procurement, as shown in table 9.1. Most respondents (22 out of 42) did not respond to the question primarily due to their disbelief that PFG was at the mid-term and so status of projects could be judged). Ultimately, there is no rigorous results-based way of measuring this assertion, and the data gathered is primarily based on interviewee perceptions, as stated above.

As discussed in response to the previous evaluation question, the overall M&E framework for PFG is undeveloped, individual project M&E plans have only recently been finalized and approved, and there is no systematic measurement of progress by the Steering Committee or technical subcommittees. Moreover, USG or USAID projects and activities are not always precisely aligned with the 17 JCAP


goals, such that many projects contain elements that contribute to a number of goals, while some goals are not addressed at all. It was challenging for the evaluation team to systematically access and review data because, as mentioned previously in Country-Specific Question 2, there is no precise PFG framework and M&E plan even for the pre-selected goals. Results at the goal level are mostly anecdotal or based on activities described in quarterly progress reports that provide little frame of reference about what was supposed to have been accomplished.

Table 9.1: Individual responses about the status of PFG-selected interventions at mid-term

Interview responses

USG GPH Overall total respondents Leadership Program

manager/CORs Leadership Program manager/CORs

Yes (on target)

10 (53%) 4 (80%) 3 (20%) 1 (33%) 18 (43%)

No (not on target)

0 0 3 (20%) 1 (33%) 4 (9%)

No response 9 (47%) 1 (20%) 9 (60%) 1 (33%) 20 (48%)

Total 19 5 15 3 42

The PFG scorecard is another mechanism for demonstrating whether or not activities are on target. The scorecard for the Philippines reports mostly on macro-level indicators for the entire country, such as GDP growth, the value of exports of goods and services, and the control of corruption index, among others.

Nine out of 10 of these indicators demonstrated improvement between 2011 and 2013; however, it is unclear whether or not these improvements can be attributed to PFG interventions. The creation and use of a corresponding M&E framework and mid-level indicators would have allowed PFG to attribute success more effectively to scorecard progress.

9.1 Findings

Finding 1: Government-to-government negotiations, collaboration on project design, and procurement timelines have delayed the initiation of projects and activities.

Initiating and implementing PFG is an inherently slower process than initiating and implementing other development modalities because each project requires high-level commitments from two governments. The PFG process, from the CA to the JCAP, requires more coordination, agenda setting, and consensus building. As a result, it takes longer than more unilateral development assistance programming. Most respondents indicated that while the PFG process delayed implementation, the benefits of coordination outweighed the costs.

Finding 2: Implementation at the project level is generally on target, given when projects were initiated: USAID’s PFG projects were mostly initiated between 6 months and one and a half years ago, despite PFG officially starting in November 2011 (over two and a half years ago). For example, the JUSTICE project—one of the activities under the goals selected for further review that addresses judicial efficiency under the weak governance constraint—was one of the first USAID projects launched, but it did not begin implementation until October 2012. At the time of the mid-term evaluation in July 2014, the project had been in implementation a little over two and a half years but did not have an approved M&E


plan. Similarly, the Facilitating Public Investment (FPI) project, the most important post-PFG-initiated contribution to the fiscal space constraint, had a signed contract in place on August 22, 2013 and has experienced (rather standard) delays in implementation due to challenges with hiring key personnel.

PFG Philippines was initiated at the same time as an extensive reorientation of USAID programming to focus on structural reforms that utilized the PFG CA approach as its foundation. The joint analytical process across USG and GPH, and the design of various programs and activities by all actors to address the binding constraints, was an important yet time-consuming intervention that, by most accounts, delayed procurement and, therefore, implementation of new programs. Once the design and consultation process was completed, most USAID staff felt that procurement was faster than usual. At midterm, almost all programs are in full implementation mode.

The evaluation team investigated progress for a subset of JCAP pre-selected goals.32 Progress was difficult to measure vis-à-vis the JCAP because of a lack of an overarching PFG M&E framework and regular reporting with established yearly or mid-term targets. In addition, tracking was challenging because the match between JCAP goals and projects is not explicit and can only be identified by comparing detailed activities within projects. As stated previously, of the existing 47 USAID projects, 31 now have finalized M&E plans;33 however, the evaluation team did not get information on how indicators within these projects are being systematically tracked. During the evaluation process, the evaluation team aligned JCAP goals with specific projects by reviewing quarterly and annual progress reports in an attempt to roughly identify whether activities have either started, are delayed, or have been completed. These reports do not contain comprehensive data, nor do they systematically report on indicators that allow for a more quantitative analysis. Furthermore, they often do not contain information on specific activities.

Below are the findings from the review of the six goals selected for a more in-depth study.34

Selected goals for improving regulatory quality sub-constraint

The statement of work required that the evaluation team conduct an in-depth study of selected PFG goals as part of this evaluation. Table 9.2 below provides the sub-constraints, goals, and the corresponding activities that were selected for the study. The detailed findings follow subsequently.

32 Mid-Term PFG Philippines Evaluability Report, June 2014. 33 As noted earlier, most of the M&E plans are close to being finalized and approved by USAID, and many are being used in draft form by the projects to collect data for eventual reporting. 34 IMPORTANT NOTE: The following analysis is based on all documents supplied to the evaluation team after repeated requests. At the time of this report’s development, there seem to be a number of missing documents that might have affected this assessment. During the process of reviewing this evaluation, the evaluation team will again make efforts to procure the missing documents, meaning that the following analysis might be amended because of information discovered during that process.


Table 9.2: PFG-selected goals and projects Sub-Constraint Total No. of

Goals Selected Goals

Improving regulatory quality 5

Goal 1: Facilitate Trade and Investment, TRADE TAPP, SIMM, INVEST, and PHILAM Goal 2: Reduce Regulatory Bottlenecks, involving the projects by COMPETE, SIMM, TAPP, INVEST, and PHILAM

Rule of law and anticorruption 9

Goal 7: Support Efforts to Improve Judicial Efficiency Component 1 of the JUSTICE project; multiple justice sector counterparts; USAID and DOJ actors Goal 8: Strengthen Anti-Corruption Institutions Component 1 of the I3 project; multiple institutional partners; USAID and DOJ actors Goal 12: Strengthen Corporate Governance—I3 Component 3; public agencies and private-sector firms and associations; USAID and DOJ

Narrow fiscal space 3 Goal 15: Increase Fiscal Space by Expanding the Tax Base, involving the project FPI

Goal 1: Facilitate Trade and Investment

The CA highlighted governance as a primary binding constraint to growth, and split this constraint into three subject areas—regulatory quality, control of corruption and political stability, and absence of violence. Goal 1 falls under regulatory quality, which the CA elaborated as having the following facets:

• a weak regulatory environment caused by poor governance, which has played a key role in deterring public and private investment in the country;

• restrictions on foreign investment, limiting access to the Philippine market and, therefore, suppressing business creation and labor demand across the economy; and

• regulatory inconsistency, poor policy implementation and enforcement, and regulatory capture, which inhibit business entry and activity.

Thus, improving regulatory quality became one of the three main areas of activities, or sub-constraints, emphasized in the JCAP. In introducing this area, the JCAP, quite naturally, repeated—in slightly changed form—the three bullets outlined in the CA. Under improving regulatory quality, there were five goals, the first of which was to facilitate trade and investment. In a counterintuitive manner, the name does not define what is under this goal because, in any reasonably functioning market economy, any practical policy will facilitate trade and investment in some way. However, the JCAP defined this goal as being composed of the following:

• facilitated compliance with World Customs Organization (WCO) Revised Kyoto Convention (RKC) provisions and other international trade regimes, including World Trade Organization (WTO) customs valuation agreement and regional trade agreements, which is a sub-part of the general aim of harmonizing local customs rules that conform to international standards;

• trade facilitation through greater use of automated systems and electronic means to reduce paper bureaucracy and person-to-person interactions at the Bureau of Customs;


• improved coordination between government departments under the National Single Window (NSW), an obligation under the Association of South East Asian Nations (ASEAN) Economic Blueprint, which is a sub-part of the general aim of meeting the country’s commitments under the ASEAN Economic Blueprint;

• trade and investment-related policy reforms needed for eligibility into multilateral and bilateral trade agreements such as the Trans-Pacific Partnership agreement; and

• improved customs compliance through more sophisticated targeting and risk management systems.

As is clear from the above list, the name for this goal—“facilitate trade and investment”—might be more accurately stated as “facilitate international trade through compliance with international agreements.” Whatever the economic and policy logic behind this emphasis, it was not prefigured in the CA, which chose to focus on other aspects of regulatory quality in its analysis, its emphases, and its recommendations. During its interviews with PFG-Philippines leaders and architects, the evaluation team was not made aware of the influences—whether USG or GPH—that led to the particular emphases of Goal 1. Nevertheless, it must be noted that this area is another where the JCAP seemed to stray away from the precision of the CA.

The following projects are in areas of activity that could be classified under the rubric of “facilitate trade and investment,” given the generality of the name for this goal and the broad nature of the activities under the projects: COMPETE: 2013–2018; INVEST: 2011–2013; PHIL AM FUND: 2013–2018; SIMM: 2012–2014; TAPP: 2012–2016; and TRADE: 2013–2018.

Using the available documentation, the evaluation team found that after more closely matching the detailed activities listed under Goal 1 of the JCAP (summarized above) and the detailed reporting on activities of each of the projects, only the TRADE project had any activities that could be counted under this goal.

The following tasks were listed in the statement of work for TRADE:

• Task #1: Implement discrete priority activities to directly assist the Philippines in meeting its commitments under the ASEAN Economic Community (AEC) Blueprint and advance the country’s trans-pacific partnerships (TPP) readiness.

• Task #2: Strengthen the institutional capacity of trade-related agencies. • Task #3: Support customs modernization and trade facilitation measures laid down under GPH’s

obligations under the WCO’s RKC and the Philippine-U.S. Trade Facilitation Protocol. • Task #4: Support the full implementation of NSW and its links to the ASEAN Single Window. • Task #5: Support actions that conform the legal framework for competition to international

norms. • Task #6: Strengthen the capacity of regulatory agencies, including the DOJ’s Office of

Competition. • Task #7: Reinforce public consultation mechanisms for trade liberalization. • Task #8: Build independent capacities for trade policy analysis.

Of these eight tasks, all but Task #6 clearly relate to the activities listed under Goal 1 in the JCAP.


The TRADE project began in May 2013; therefore, when this evaluation began, the team had less than 1 year of activities on which it could base its judgment of Goal 1. The evaluation team based its analysis of activities and deliverables on the SOW, the draft M&E plan (updated August 2013), and three quarterly progress reports (January–March 2014, October–December 2013, and May–September 2013), and the team gathered a list of relevant activities from that documentation. The evaluation team also used the responses from USAID leadership, managers (both USAID and GPH), and implementers working on this goal. These activities are listed in table 9.3 below. From the various progress reports, the evaluation team was able to ascertain whether the activities were ongoing, delayed, or completed, or whether there was no information on the activities (in which case the presumption was that the activity has not yet started).

Activity toward meeting Goal 1 (Facilitate Trade and Investment) is only in its earliest phases. Table 9.3 reflects 10 months of implementation of activities (that is 28 months after PFG officially began and more than 2 years after the JCAP forecasted that projects would start). The list of project accomplishments in table 9.3 includes a combination of completed and ongoing activities that would be normal for such an early stage in a project’s implementation. There are really two issues that arise in qualifying an answer to the evaluation question, “Are the performances of the selected PFG interventions on target?” The first is whether or not the start date of the relevant projects was on target given when PFG began. The second is whether or not the interventions envisaged in the project SOWs are on target given when the project started. The answer to the first issue for Goal 1 (Facilitate Trade and Investment) is that the start date of the relevant project seems to have been quite delayed relative to when PFG started and relative to the following statement in the JCAP: “Q2-2012: PFG projects [to be] rolled out.” The answer to the second question for Goal 1 is that there is no evidence indicating that the relevant interventions were delayed in implementation once the relevant project began official operation.

Table 9.3: Status of projects and activities under Goal 1: Facilitate Trade and Investment

Goal 1: Facilitate Trade and Investment Task 1: Assist the meeting of the country’s commitments under the AEC Blueprint and the various WTO agreements, and advance the country’s readiness to accede to the TPP. Support the holding of a series of CAEC workshops and roundtables to map out and implement the AEC Compliance Agenda.

Delayed Provide advice on operational approach to compliance (e.g., mobilize inter-agency CAEC subcommittees focused on addressing specific and critical compliance deficiencies). Convene a technical group of legal experts, trade economists, and CAEC officers to conduct a legal gap analysis of AEC Blueprint measures (e.g., Foreign Direct Investment [FDI] ownership restrictions), and identify legislative or administrative options for CAEC to close the gap.

Completed

Assist CAEC agencies in implementing the AEC Compliance Agenda. This implementation entails providing policy advice and empirical basis for addressing the unimplemented AEC Blueprint measures, via briefings, memoranda, and policy papers for the CAEC Chair (DTI Undersecretary), other CAEC officials, and other policy makers as may be requested.

In progress

Work with DTI, PCCI, EDC (through its AEC Committee) and other groups to conduct regular policy forums and roundtables on AEC, including one that will involve the Secretaries of CAEC agencies.

Delayed

Provide assistance to DTI and CAEC in the conduct of evidence-based and gender-sensitive analyses of opportunities and challenges (i.e., benefit-cost analysis) of full AEC membership and associated commitments, including identification of measures to maximize the country’s net benefits from AEC membership.

In progress


Goal 1: Facilitate Trade and Investment Analysis of the pattern of PH trade in goods and services with other ASEAN member countries with a view toward identifying economic opportunities along with possible risks or threats.

Completed

Assessment of overall economic impacts of AEC on PH (including GDP, employment, prices, sectorial, and gender implications). Delayed

Assess the current WTO compliance level of PH based on the last Trade Policy Review; develop a list of compliance issues at the statutory and implementation levels; suggest a plan to improve compliance, and assist the DTI and TRM to implement the plan.

In progress

Assess the country’s state of readiness in meeting requirements of new generation trade agreements; i.e., determine (1) policy deficiencies and (2) necessary measures and options to overcome these deficiencies.

Delayed

Undertake empirical analyses of costs and benefits of Philippine membership in new generation trade agreements, such as TPP.

In progress

Promote increased public awareness and support for PH participation in TPP. In progress

Identify and harness partner organizations (private sector and civil society) for TPP advocacy, and support partner initiatives to promote TPP readiness by helping lobby for needed policy reforms.

In progress

Conduct discussion forums on TPP to widen the advocacy constituency for TPP membership. In progress

Produce a monthly “TPP Updates” bulletin that summarizes key developments pertaining to TPP within the past month.

In progress

Assist the GPH in adopting reforms toward TPP readiness and eventual membership, including through work with advocacy partners.

In progress

Provide advice and technical support (including empirical evidence) to policy makers, including legislators, pertaining to relevant policy reforms needed for TPP readiness (e.g., foreign ownership restrictions, competition policies).

In progress

Provide technical assistance to DTI on substantive preparations for the hosting of the APEC 2015 meeting, as requested, particularly on trade facilitation measures for small and medium enterprises.

In progress

Task 2: Strengthen the institutional capacity of trade-related agencies. Analyze the existing structures and inter-agency mechanisms for trade policy formulation and recommend appropriate measures to achieve greater coordination, consistency, coherence and clear accountabilities.

Completed

Undertake a training needs assessment on the key agencies involved in trade policy and negotiations (i.e., TRM Committee agencies including DTI, NEDA, DA, DOT, etc.), including a stocktaking of the nature and gaps in existing training activities for these agencies.

Delayed

Support the formulation of a training program design on trade policy and negotiations for officers and staff of trade policy-making and negotiating agencies, and identification of options for a sustainable training program or institution based on an assessment of previous GPH experiences.

In progress

Support implementation of a DTI-approved training program or institution (Trade Policy Institute or Academy).

In progress

Support conduct of training programs guided by the adopted training program design. In progress Support training on the above for trade-related agencies under existing available training programs such as the PCCI i-MUST PLUS. Completed

Support an internship program at DTI that will draw promising students to trade policy work in government (interns provided with a modest stipend).

In progress

Support the convening by DTI (through the Undersecretary for Policy and Planning) of regular meetings of trade policy experts to share knowledge generated by existing research of experts (including those commissioned by the TRADE project) on topics relevant to the trade and investment work of DTI.

In progress

Expand the network and pipeline of trade policy experts by starting a research program for experts In progress


Goal 1: Facilitate Trade and Investment from various academic and research institutions.

Develop a market access simulation model that CAEC can use to simulate improvements in market access of alternative trade negotiating positions.

In progress

Task 3: Support Customs modernization and trade facilitation measures laid down under GPH’s obligations under the WCO’s RKC and the Philippine-U.S. Trade Facilitation Protocol (PUSTFP).

Assess current trade facilitation regime for consistency with AEC, PUSTFP, WTO and TPP, and identify compliance gaps and areas of inconsistency.

In progress

Assist BOC on a peer review of the draft CMTA bill to review its consistency with the relevant provisions of AEC, PUSTFP, WTO and TPP. This assistance includes:

In progress

Conducting a gap analysis on the existing version of the bill; In progress Formulation of a compliance plan to ensure CMTA’s consistency with AEC, PUSTFP, WTO, and TPP; and In progress

Drafting of implementing rules and regulations, and supporting implementing orders. In progress

Assist BOC in shepherding the CMTA bill through Congress. This will entail training of the responsible unit within BOC with (1) sustainable capacity with respect to provisions of AEC, PUSTFP, etc., and (2) expertise in legislative processes.

In progress

Assist BOC with the preparation and issuance of appropriate implementing orders to supplement CMTA to ensure full consistency with AEC, PUSTFP, WTO, and TPP readiness.

In progress

Assist BOC in implementing urgent provisions of the RKC, AEC and PUSTFP, which can be effected under the current TCCP, starting with (for Y2):

In progress

Full utilization of advance ruling regime and improved requisite appeal system; In progress

Adoption of a rationalized de minimis regime; and In progress Establishment of pre-arrival cargo clearance process for authorized companies. In progress Conduct a series of end-to-end cargo processing and release studies under selected ports (Manila ports, Cebu, Davao, Batangas, Cagayan de Oro (CDO), Iloilo). The studies will: Document shipment processing and release procedures (starting from the time importer submits import entry until cargo is received by importer) and review processes by value-added service providers or VASPs, trade-regulatory agencies, BOC, port operators, truckers, etc. Review will be done at various port districts, covering both regulated and non-regulated cargo classification; Identify efficiencies and inefficiencies (e.g., bottlenecks) of cargo processing and release procedures; Develop procedural and structural improvements for improved operational efficiency. The improvements will be used as input for the prospective re-engineering plan that will create BOC model district offices in Cities Development Initiative (CDI) cities.

In progress

Work with the BOC and the PCCI to develop and implement an international standard ATA Carnet system. The ATA Carnet is an international customs document that allows the holder to import goods without payment of duties and taxes, including value-added taxes, for up to 1 year. In progress

Assist BOC in providing an efficient and effective cargo clearance process for cargoes entered under temporary and conditionally free importation, through: In progress

Streamlined cargo release and raw materials liquidation process and bonds management system; and In progress

Application of IT in the operation of Customs bonded warehouses. In progress Assist BOC in identifying further improvements to their website toward: Expanding the availability to the public of rulings, orders, etc.; Achieving a more transparent “business-centric” and “entrepreneur-friendly” character; and Improving navigability and general user-friendliness.

Completed


Goal 1: Facilitate Trade and Investment Task 4: Support the full implementation of the National Single Window (NSW) and its links to the ASEAN Single Window. Support efforts to improve the permit processing system of the most critical (up to 15) NSW-member TRGAs. The project will identify the TRGAs that are frequently transacted for trade related permits, licenses, and other authorizations. The following agencies are likely candidates: NFA, SRA, BIR, BPS, BOC, BOI-BETP, PEZA, FDA, BAI, BPI, BIS, BOQ, FPA, NMIS, NTC and BFAR.

In progress

Provide policy advice to the GPH on strengthening the coordination mechanism for NSW (e.g., by exploring alternatives on the organization and operation of the NSW Steering Committee).

Completed

Assist DTI, along with other trade related agencies, in establishing a National Trade Repository (NTR) through: Development of an NTR Action Plan, considering, among others: (1) the IT infrastructure (process, software, hardware, etc.) to be employed; (2) the organization of the NTR implementing body; (3) attributes of trade data to be maintained by the NTR in terms of types, coverage, quality, frequency of updates, access models (e.g. public or open, regulated, restricted, etc.) and data models; and (4) NTR sustainability strategies. Implementation of the NTR action plan.

In progress

Task 5: Support actions that conform to the legal framework for competition to international norms. Motivate the formation of a broad-based coalition of advocates to support the development of a national competition policy framework for the Philippines. In progress

Increase the technical expertise and knowledge of legislators and critical technical staff in both Houses of Congress (particularly the Chiefs of Staff of the Senate, and the relevant Committee Secretariats of the Lower House) to defend and successfully pass the proposed competition law.

In progress

Identify constituencies who can be tapped and developed as allies for competition law and policy advocacy; and establish a baseline of consumer awareness and perceptions on state of competition in the Philippine economy by designing and conducting a sample survey.

In progress

Task 6: Reinforce public consultation mechanisms for trade liberalization. Strengthen and systematize the project’s advocacy process (design, implementation, monitoring and networking-linkages). Delayed

Identify relevant stakeholders and stakeholder groups, and harness key groups to assist in the project’s advocacies. In progress

Identify component champions and thought leaders. In progress Develop key messages for stakeholder clusters. In progress Provide training for government and private-sector partners and component champions on trade policy advocacy. Delayed

Conduct media training sessions for private-sector partners and component champions. In progress Support and strengthen the DTI OCOV Secretariat with public outreach experts. In progress Develop a Code of Consultations to guide public outreach on DTI-GPH’s trade and investment policy. Delayed

Develop a registry of stakeholders (that can be segmented and analyzed according to key and common concerns and generate a feedback structure). Completed

Engage communication channels that establish presence and reach a variety of stakeholders. In progress Publish select IEC materials from knowledge generated by all components. In progress Assist DTI in the production of a monthly “Trade, Industry and Investment Indicators” factsheet summarizing key trade and investment indicators and important developments. In progress

Provide communications support for PCCI regional forums. In progress Strengthen media monitoring capacity in DTI. In progress Engage media in developing in-depth international trade reports (e.g., via international trade workshops for journalists). In progress

Formulate an AEC Communication and Advocacy Plan, including key PH messages on AEC and a multimedia strategy for communicating AEC to key stakeholders and the general public. In progress

Undertake baseline surveys on public awareness, perceptions and sentiments on AEC, WTO, and TPP, and on competition policy. In progress


Goal 1: Facilitate Trade and Investment Assist CAEC in conducting policy forums on AEC, including one that will involve the Secretaries of CAEC agencies. Delayed

Assist in conducting briefing sessions for legislators and legislative staff for key measures, particularly CMTA and Competition Policy Law, and for advocacy for TPP. In progress

Provide communications support for the project’s initiatives on AEC, RKC and PUSTFP compliance, customs modernization, and NSW or ASW. No information

Promote increased public awareness and support for PH participation in TPP. No information Identify and harness partner organizations (private sector and civil society) for TPP advocacy and support partner initiatives to promote TPP readiness. No information

Conduct discussion forums on TPP to widen the advocacy constituency for TPP membership. No information

Goal 2: Reduce Regulatory Bottlenecks, Entry Barriers, and Discriminatory Provisions to Investment

Goal 2 also falls under regulatory quality, which the CA described as having the following facets:

• a weak regulatory environment caused by poor governance, which has played a key role in deterring public and private investment in the country;

• restrictions on foreign investment limit access to the Philippine market and, therefore, suppression of business creation and labor demand across the economy; and

• regulatory inconsistency, poor policy implementation and enforcement, and regulatory capture, which also inhibit business entry and activity.

Thus, improving regulatory quality became one of the three main areas of activities, or sub-constraints, emphasized in the JCAP. In introducing this area, the JCAP repeated—in slightly changed form—the three bullets outlined in the CA. Under Improving Regulatory Quality, there were five goals, the first of which was to “facilitate trade and investment.” The JCAP defined this goal to compose the following:

• reducing the cost of doing business by streamlining business procedures; • improving the investment climate through regulatory reforms; • ensuring all import regulations are consistent with internationally accepted standards and policies; • reducing barriers to entry; • separating dual operational and regulatory functions of government agencies; • supporting public-private partnerships; • improving the provision of transport infrastructure; • reducing the high cost of electricity and promoting access to affordable power; • supporting regulatory reform initiatives particularly in port, water, and transport sectors; and • supporting investment policy conferences, reverse trade missions, or international buyer

programs.

The JCAP specifically stated that activities should be aimed at growth in priority sectors, including tourism, agribusiness, infrastructure, logistics, mining, and manufacturing.

As is clear from the list above, the set of activities is broad and heterogeneous, activities that nevertheless fall in the area of “reduce regulatory bottlenecks, entry barriers, and discriminatory provisions to


investment,” the target for Goal 2. The JCAP did not discuss why these particular areas were chosen over others.

The following projects are in areas of activity that could be classified under the rubric “reduce regulatory bottlenecks, entry barriers, and discriminatory provisions to investment,” given the generality of the name for this goal and the broad nature of the activities under the projects: COMPETE 2013–2018; INVEST 2011–2013; SIMM 2012–2014; TAPP 2012–2016; and TRADE 2013–2018.

Using the available documentation, the evaluation team found that after more closely matching the detailed activities listed under Goal 2 of the JCAP (summarized above) and the detailed reporting on activities of each of the projects, the COMPETE, SIMM, INVEST, and TRADE projects had activities that could be counted under this goal.

The following should be noted about the limited amount of time that some of these projects have been in existence and the limited documentation on some of them.

• The COMPETE project began in April 5, 2013; therefore, when this evaluation began, the COMPETE project had been running for only slightly over a year. Moreover, the evaluation team did not receive any documentation on COMPETE activities in 2014, so there is little evidence that can be used to gauge the performance of this project.

• The TRADE project began in May 2013; therefore, when this evaluation began, the evaluation team had less than 1 year of activities on which it could base its judgment of the implementation of activities under this project.

• The SIMM project began much earlier (April 26, 2012), but most of its activities were not classifiable under this particular goal within the JCAP.

• The INVEST project began on November 3, 2011; therefore, its project design was not thoroughly integrated into PFG. A considerable part of that project does not fall under the set of activities indicated in the JCAP. Moreover, the evaluation team received a limited amount of documentation on that project.

Activity toward meeting Goal 2 (reduce regulatory bottlenecks, entry barriers, and discriminatory provisions to investment) is listed in the set of project accomplishments in table 9.4 below. In making its judgments about the status of individual activities, the evaluation team analyzed all project documents made available plus the responses from USAID leadership, managers (both USG and GPH), and implementers working on this goal. Activities relevant to this goal are either in their earliest phases (for COMPETE and TRADE) or only part of the activities that were undertaken by SIMM and INVEST. Moreover, table 9.4 reflects the status of only a highly limited set of activities of the INVEST project relevant to Goal 2 because of the lack of documentation made available to the evaluation team. Given these caveats, the list of project accomplishments in table 9.4 contains a mixture of completed and ongoing activities that would be normal for projects at the various stages of implementation of the four projects that are relevant to Goal 2. Note also that the lack of indications of completed interventions is due to the fact that COMPETE and TRADE began only in 2013 and the projects that started earlier—INVEST and SIMM—have fewer activities related to this goal.

There are two main issues that arise when formulating an answer to the question, “Are the performances of the selected PFG interventions on target?” The first is whether the start date of the relevant projects


was on target given when PFG began. The second is whether the interventions envisaged in the project SOWs are on target given when the project started. The answer to the first question for Goal 2 (increase fiscal space by expanding the tax base) is somewhat mixed. The start dates of the two major projects under this goal (COMPETE and TRADE) seem to have been quite delayed relative to when PFG started and relative to the statement in the JCAP: “Q22012: PFG projects [to be] rolled out.” The start dates of SIMM and INVEST were much earlier, but both of these seem to be bridge projects with activities (and interventions) not entirely closely aligned with PFG. The answer to the second question for Goal 2 is that there is no evidence indicating that the relevant interventions were delayed in implementation once the relevant projects began official operation.

Table 9.4: Status of projects and activities under Goal 2: Reduce Regulatory Bottlenecks, Entry Barriers, and Discriminatory Provisions to Investment Goal 2: Reduce Regulatory Bottlenecks, Entry Barriers, and Discriminatory Provisions to Investment Goal Project Activity Status COMPETE Support policy issuances to address regulatory, institutional, and legal issues to

promote open, transparent, and competitive process. In progress

COMPETE Institutionalize rational procurement process for PPP pipeline. In progress COMPETE Seamless transport infra planning. In progress COMPETE Budgeting and implementation of priority infrastructure. In progress COMPETE Develop policy cross ownership (draft a competition policy to eliminate cross

ownership). In progress

COMPETE Aggregation and competitive power supply procurement by electric cooperatives (reduce energy costs through competitive bidding).

In progress

COMPETE Capacity building on retail competition and open access for the manufacturing sector.

No information

COMPETE Introducing competitive mechanisms for captive market. Completed COMPETE Assist in the creation of an independent market operator. In progress COMPETE Assessing effectiveness of WESMs in ensuring competitive supply. No information COMPETE Technical Support to MINDA in promoting investments in power. In progress COMPETE Conduct research and analysis on generation plants as economic zones. Delayed COMPETE Electricity demand forecasting for NEDA. Completed COMPETE Strategic marketing and institutional strengthening of secondary gateways. In progress COMPETE Gateway tourism development areas action plans. In progress COMPETE Policy issuance and training program and advance passenger information

system. No information

COMPETE Expansion of and institutionalization of New Convergence Programs. Completed COMPETE Expansion of the DOT-DPWH convergence program. Completed COMPETE Implementation of DOT-DSWD One-Step Project MOU. In progress COMPETE Expanding gateway capacities. No information COMPETE Bohol tourism recovery program. No information COMPETE Terms of reference on the bidding for shared service facilities. No information COMPETE Inclusive agri-business summit. No information COMPETE Support increased exports through improved compliance with SPS standards. Completed COMPETE Diagnose constraints to lower transport and logistics costs for agricultural

exports. In progress

COMPETE Maximizing the utilization of the Batangas Port as growth driver for Batangas City and surrounding region.

In progress

COMPETE Introduce Chassis Ro-Ro and ASEAN Ro-Ro linkages to help reduce transport costs.

In progress

COMPETE Re-drafting of the EO on the Cha-Ro policy. In progress COMPETE Launch of the Davao-GenSan-Bitung RORO link. In progress


Goal 2: Reduce Regulatory Bottlenecks, Entry Barriers, and Discriminatory Provisions to Investment Goal Project Activity Status INVEST Assessment report on the current status of business permits and licensing system

(BPLS) reforms. In progress

INVEST Assessment on the streamlined processes for new and renewing business applications.

No information

INVEST Report on the profile of business applicants in the three cities. No information INVEST Report on the Conduct of the Customer Relations Workshop. No information INVEST Information strategic system plan (ISSP) for each of the three cities. No information INVEST Assessment report on the Business One-Stop Shop (BOSS) in three cities. In progress INVEST Action plan of BOSS reforms in three cities. No information INVEST Assessment report on the BOSS (reforms in three cities). No information INVEST Institutional Study on NERBAC, BOSS, and the Philippine Business Register

(PBR). No information

INVEST Assessment report on inspection processes in the target cities. No information INVEST Study on risk-based inspection. No information INVEST Study on benchmarking of inspection fees. No information INVEST Action plan on inspection reforms for implementation in target cities. No information INVEST Report on the assessment workshops on the setting up of business-friendly

inspection systems in the three cities. No information

INVEST Report on the assessment of the reformed inspection systems. No information INVEST Report on the assessment of PBR Phase 1. No information INVEST Roadmap for PBR implementation. No information INVEST SEC connectivity to PBR. No information INVEST Connectivity of three partner cities to the PBR. No information INVEST Report on the technical assistance to the TWG in BPLS automation. No information INVEST Training manuals on the BPLS automation. No information INVEST Assessment of BPLS standards in the DILG-DTI Joint Memorandum Circular

No.1 Series of 2010. No information

INVEST Conduct of the report card survey on BPLS in three cities for 2013. No information INVEST Recommendations on the streamlining of construction permits. In progress INVEST Recommendations on the streamlining processes for operating hotel

establishments. In progress

INVEST Report on the requirements of the cities for the reformulation of the LIIC. In progress INVEST Study on the inconsistency of incentives provided in national laws and local

applications. No information

INVEST Report on the training on managing risks and responsibilities in joint implementation of development projects.

No information

INVEST Report of fund sourcing of public-private arrangement forged. No information SIMM Conduct consultations with DBM, BSP, COA the pilot government agencies, as

well as with the EMIs. Completed

SIMM Prepare a program for the government agencies. The program would describe how best to obtain buy-in from each agency, and how best to educate the agency executives and employees on mobile money. It will also include employee enrollment and activation of their mobile money accounts.

Completed

SIMM Review government policies and regulations for any potential roadblocks limiting or disallowing EMI participation in G2P and CCT program design and disbursement rules.

Completed

SIMM Perform comparison and cost benefit analysis of e-money versus current CCT distribution.

Completed

SIMM Conduct individual consultations with DBM, DA, BSP, DSWD, LBP, and MRPIs on G2P and facilitate joint discussion between the key actors to discuss the overall vision and benefits of m-money, while addressing constraints and barriers to entry.

Completed


Goal 2: Reduce Regulatory Bottlenecks, Entry Barriers, and Discriminatory Provisions to Investment Goal Project Activity Status SIMM Finalize plan of action that may result in written agreements between the

organizations, as well as program for preparing areas for CCT and social subsidy distribution by building an m-money ecosystem in these areas and activating m-money adoption among CCT beneficiaries.

Not pursued

SIMM Coordinate with the BSP, DTI, and NTC on m-money consumer protection issues.

Completed

SIMM In consultation with MRPIs, prepare messages and materials for consumer protection education to highlight consumer rights related to m-money.

Completed

SIMM Participate in webinars with DAI on e-money and improving the global regulatory environment.

Completed

TRADE Motivate the formation of a broad-based coalition of advocates to support the development of a national competition policy network for the Philippines.

In progress

TRADE Increase the technical expertise and knowledge of legislators and critical technical staff in both Houses of Congress (particularly the Chiefs of Staff of the Senate, and the relevant Committee Secretariats of the Lower House) to defend and successfully pass the proposed competition law.

In progress

TRADE Identify constituencies who can be tapped and developed as allies for competition law and policy advocacy; and establish a baseline of consumer awareness and perceptions on state of competition in the Philippine economy by designing and conducting a sample survey.

In progress

TRADE Examine the economic rationale for classifying specific industries as public utilities or public service facilities to inform initiatives to amend existing laws that are hindering reforms to promote competition and attract investments.

In progress

TRADE Increase the capability of sector regulators to conduct evidence-based research to assess the economic costs of lack of or inadequate competition in key economic sectors.

In progress

TRADE Design and conduct stakeholder survey via focus group discussions on intensity and quality of competition in key economic sectors.

In progress

TRADE Formulate a policy reform agenda to rationalize and address (1) the conflicts that arise when government-owned-and-controlled corporations, and other government agencies are mandated with conflicting developmental and regulatory roles, and (2) the role of state trading enterprises (e.g., NFA, PITC).f

Delayed

TRADE Develop and conduct short courses on competition law and policy for selected regulatory agencies, including sessions on evidence-based research and regulation.

In progress

Selected Goals for Rule of Law and Anti-Corruption Sub-Constraint

Goal 7: Support Efforts to Improve Judicial Efficiency

The CA highlighted improving judicial efficiency as a key activity for creating a more favorable investment climate through improved and timelier resolution of commercial disputes. The JCAP set this specific goal and outlined key activities to be supported by USAID and the U.S. DOJ’s Office of Overseas Prosecutorial Development, Assistance and Training (DOJ-OPDAT) working with counterparts in GPH. USAID subsequently designed a program working with the GPH Supreme Court and provided a 5-year grant to the American Bar Association Rule of Law Initiative for the JUSTICE Project. Component 1—Judicial Efficiency of the JUSTICE Project contains 10 sub-components:

1. developing and installing eCourts; 2. adding enhancements to the Case Management Information System at the Court of Appeals;


3. building the capacity of the Court Management Office (under the Supreme Court Office of the Court Administrator) and the judicial supervisors to utilize the National eCourt Program;

4. updating the Automated Case Management Information System at the Sandiganbayan (SB CMIS);

5. determining whether Quezon City Pilot Rules for Speedier Litigation should be amended and adopted nationwide (including revisions of rules of civil procedures);

6. helping the judiciary determine whether to expand jurisdiction of small claims courts; 7. conducting a broader nationwide information campaign to highlight the availability of small

claims courts to settle contract disputes; 8. improving the efficiency of the courts through a one-time case decongestion effort; 9. determining lower court judges’ case-carrying capacities; and 10. analyzing case delay and court congestion.

According to respondents associated with implementing this goal, progress has been relatively good since the project began in October 2012. Activities were carried out under all sub-components, except 7 and 10, which were scheduled to begin later in 2014.

The evaluation team based its analysis of activities and deliverables on the JUSTICE SOW, three quarterly progress reports (Q2 2013, Q1 2014, Q2 2104), the 2013 annual report, the Year 1 and 2 work plans, and the draft M&E plan (updated May 2014); the team gathered a list of relevant activities from that documentation. These activities are listed in table 9.5. From the various progress reports, the evaluation team was able to ascertain whether the activities were ongoing, delayed, or complete, or whether there was no information on the activities (in which case the presumption was that the activity has not yet started).

Table 9.5: Status of projects and activities under Goal 7: Support efforts to improve judicial efficiency

Goal 7: Support Efforts to Improve Judicial Efficiency (USAID JUSTICE Project) Year 1 1.1 Develop and install eCourts in 12 economic centers of the Philippines Stable and operational eCourt installed in Quezon City (QC) courts. In progress Establish project management team and project implementation management, performance and risk monitoring system. In progress eCourt roll-out initiated in 12 economic centers (upon Supreme Court approval and system stabilization). In progress 1.2 Adding enhancements to CMIS features at the CA Study impact of CA CMIS. In progress CA CMIS enhancements identified. In progress 1.4 Updating the automated CMIS at the Sandiganbayan (SB) Organize overall structure for project implementation, monitoring, and risk management. In progress Conduct business process mapping in Sandiganbayan and identify process issues and requirements. In progress Prototype of enhanced SB CMIS presented to the judiciary. In progress 1.5 Determine whether Quezon City Pilot Rules for Speedier Litigation should be amended and adopted nationwide (including revision of Rules of Civil Procedure) Judges and lawyers trained on the judicial affidavit rule. Completed


Goal 7: Support Efforts to Improve Judicial Efficiency (USAID JUSTICE Project) Review of QC practice guidelines commenced In progress

Stakeholder meeting/s for Mindanao and Visayas leading up to the National Conference for Revision of Rules of Civil Procedure. Discontinued Print and distribute criminal sentencing guidelines. In progress 1.6 Help the judiciary determine whether to expand jurisdiction of small claims courts

Review of small claims procedures and performance for possible increased jurisdictional amount, including an analysis of backlog, demand, relevant case types, and access. In progress 1.8 Improving the efficiency of the courts through a one-time case decongestion effort First draft detailed implementation plan. Completed Pilot decongestion program implemented. Completed Inventory report of dormant cases of target courts. In progress 1.9 Determination of case-carrying capacity for lower court judges Recommendations on the case carrying capacity of judges. In progress 1.10 Analysis of case delay and court congestion

Scope, approach, and methodology for the study of indicators and targets for judicial efficiency appropriate to changing conditions. Delayed Year 2 1.1 Develop and install eCourts Validate Quezon City (QC) eCourt data. In progress Develop final enhanced eCourt software suitable for use in roll-out of eCourt. In progress Finalize list of target courts, implementation plan for eCourt roll-out (referred to as “eCIP”). In progress Establish eCourt support systems and capacities. In progress

Roll-out eCourt in target court stations (continues to Year 5; steps are sequential and will be implemented for each court selected for implementation in Year 2). Delayed 1.2 Adding enhancements to CMIS features at the CA Study impact of Court of Appeals Case Management Information System (CA CMIS) (continue from Q4 Year 1). In progress Identify CA CMIS enhancements. In progress Develop and finalize SRS for CA CMIS enhancements. Delayed Develop and test the enhanced program with users. Delayed

1.3 Building the capacity of the Court Management Office (CMO) and the judicial supervisors to utilize the National eCourt Program Assess CMO’s current programs and capacity to monitor trial courts’ performance and needs. Delayed Develop eCourt module for CMO. Delayed Assist CMO in developing court performance standards and a monitoring plan for trial courts. In progress 1.4 Updating Sandiganbayan CMIS Organize overall structure for project implementation, monitoring, and risk management (continue from Year 1). In progress

Conduct business process mapping in Sandiganbayan, and identify process issues and requirements, including IT components. In progress Develop and finalize upgrade specifications for SB CMIS in coordination with SB workgroups. Delayed Develop and test the enhanced program with users (continue to Year 3). Delayed

1.5 Determine whether Quezon City Pilot Rules for Speedier Litigation should be amended and adopted nationwide (including revision of Rules of Civil Procedure)


Goal 7: Support Efforts to Improve Judicial Efficiency (USAID JUSTICE Project) Review results of pilot implementation of the Practice Guidelines in Quezon City (continue from Q3 Year 1). In progress Review QC Practice Guidelines. Delayed Print and distribute Criminal Sentencing Guidelines. Delayed Develop Official Court Forms for inclusion in the Revised Rules of Civil Procedure. In progress 1.6 Help the judiciary determine whether to expand jurisdiction of small claims courts

Develop approach and methodology to review performance of small claims and study the potential of expanding the jurisdictional amount for the application of small claims procedures. In progress Conduct follow-up training for judges and clerks of court branches regarding the new small claims procedures. Delayed

1.7 Conduct a broader nationwide information campaign to highlight the availability of small claims courts to settle contract disputes

Develop a communication strategy and plan for small claims court outreach to serve as a blueprint for its public information campaign drive. Delayed Conduct information campaign dissemination on small claims. Delayed 1.8 Improving the efficiency of the courts through a one-time case decongestion effort Conduct meetings of the Ad Hoc Oversight and Monitoring Committee. In progress Finalize Manual of Procedure on Case Decongestion. Completed Recruit lawyers and legal researchers to participate in the case decongestion activity. In progress Engage, train, and utilize private courier services for the delivery of notices and orders from the target courts. In progress Complete the implementation of the case decongestion activity in Quezon City (continued from Year 1). In progress Conduct stakeholders’ workshops in the areas where the case decongestion activity will be implemented. In progress Conduct the case decongestion activity in 50 additional target courts. Delayed Implement a communication plan for the case decongestion activity through sub sub-grantee. In progress

Submit interim report to the Ad Hoc Committee and Supreme Court Committee on Delay and Congestion in Lower Courts. In progress 1.9 Determination of case-carrying capacity for lower court judges Develop recommendations on the case-carrying capacity of judges. In progress 1.10 Analysis of case delay and court congestion Gather data and analyses rendered on case congestion and delay. Delayed As with all of the PFG projects, the JUSTICE Project was delayed by the PFG design phase but came online shortly thereafter. Despite this initial delay, the activities associated with Goal 7 on judicial efficiency have made progress over the past one and a half years of implementation. According to the review of quarterly and annual reports and responses from USAID leadership, managers (both USAID and GPH), and implementers working on this goal, this project is on target.

The evaluation team did not, however, find any evidence of the U.S. DOJ-OPDAT contributions discussed in the JCAP (see table 9.6). Discussions with DOJ in the Philippines indicated a focus only on the work with the ombudsman under Goal 8. The activities assigned to DOJ in the JCAP were not conducted as far as the evaluation team could determine.


Table 9.6: Activities defined by the JCAP to be implemented as part of Goal 7 Joint Country Action Plan – Partnership for Growth – Philippines, 2012–2016,

Summary Matrix of Activities, Proposed Actions Joint Actions Follow-On USG Activities

Strengthening the Rule of Law and Anti-corruption Enforcement (ii) Assist Philippine justice-sector

agencies (e.g., judiciary, prosecutors, public defenders and private law practitioners) in revising laws and court procedures to (a) enhance tools for criminal and civil investigations; (b) improve investigator-prosecutor cooperation; (c) promote the efficient disposition of cases through continuous trials, effective preliminary investigations, and reduced interlocutory appeals.

DOJ-OPDAT • Identify gaps in current criminal justice laws, regulations, and

practices to reduce procedural gaps and inconsistencies (Lead agency: U.S. DOJ-OPDAT/INL).

• Provide expertise and feedback on draft legislation to address deficiencies (Lead agency: U.S. DOJ-OPDAT/INL).

• Organize programs and skills-training activities focused on strengthening criminal procedure tools (Lead agency: U.S. DOJ-OPDAT/INL).

• Support the review and reform of Criminal Procedure Rules to promote the fair and effective administration of justice (Lead agency: U.S. DOJ-OPDAT/INL).

• Promote better and more efficient use of existing procedural tools, including plea bargaining (consensual resolution of cases) and pretrial procedures and mechanisms (Lead agency: U.S. DOJ-OPDAT/INL).

• Support the development of appropriate internal policies and in-service skills training at PDOJ to reduce delay (Lead agency: U.S. DOJ-OPDAT/INL).

Goal 8: Strengthen Anti-Corruption Institutions

The CA highlighted corruption as a key constraint to economic growth and assisted GPH to effectively enforce anti-corruption laws as a critical step. The JCAP outlined a number of joint activities under the goal of “strengthen anti-corruption institutions” to include working with the ombudsman, Commission on Audit, and Justice Sector Coordinating Council. USAID and U.S. DOJ-OPDAT were identified as the lead USG institutions. USAID subsequently designed a program and procured a contract for the 5-year Integrity for Investments Initiative (I3) Project, implemented by Deloitte Consulting. I3 implementation, which began on February 20, 2013, was also delayed during the PFG consultation and design phase. I3 has three components, each of which primarily addresses one of the JCAP goals under the weak governance constraint. Component 1, increasing GPH effectiveness in enforcing anti-corruption laws, is the most closely aligned with Goal 8. Per project documents, Component 1 is broken down into four sub-tasks:

• Task 1: Help the anti-corruption enforcement agencies define, prioritize, and advocate the legal changes needed to effectively detect, investigate, and prosecute corruption cases;

• Task 2: Support anti-corruption agencies to improve the management of anti-corruption investigations and prosecutions;

• Task 3: Help the anti-corruption enforcement agencies build their skills in detecting, auditing, investigating, and prosecuting corruption; and

• Task 4: Seek ways to promote and facilitate inter-agency cooperation among the anti-corruption institutions.


At the time of the evaluation, the I3 project was not using an approved M&E plan,35 so it was not possible to track progress versus expected results. The evaluation team instead cross-referenced the SOW, the work plan for Year 1, quarterly and annual progress reports, and in-person interviews with implementers, the COR for the project at USAID, and GPH counterparts. Based on this analysis, the evaluation team concluded that the I3 project was put together and procured quite rapidly after the PFG design period, and has made some progress in its first year and 5 months of operation. Project leadership had to be changed during the first year of implementation, but since that time, according to GPH and USG respondents, the project has improved results.

The I3 Year 1 Work Plan received from USAID was ambitious in its support to strengthen the OMB, COA and other anti-corruption institutions. The Work Plan detailed 33 activities over four tasks and seven sub-tasks under Component 1, and laid out the expected implementation schedule from February to September 2013 through a GANTT chart in the annexes. The I3 February–September 2013 Annual Report received by the evaluation team focused on project start up and administrative tasks and deliverables, and indicated problems in adjusting the project to counterpart GPH mandates and changes in expectations. It also indicated delay in implementation due to the replacement of both the chief of party and the M&E advisor. The annual report did not track progress according to the Work Plan GANTT chart and most activities cited in the work plan were not addressed in the annual report. Given the absence of other reporting and M&E documents, this work plan was the most accurate document the evaluation team could use to identify progress in Year 1.

The evaluation team also received an unapproved Year 2 Work Plan (in Excel format) and two Quarterly Reports (October-December 2013 Quarterly Report, January-March 2014 Quarterly Report). The Quarterly Reports do not clearly track with the Work Plan chart, but there is more information on the progress of technical activities than in the Year 1 Annual Report. Nonetheless, it is difficult to track progress through the series of bulleted activity descriptions in the Quarterly Reports in comparison to expected progress from the Work Plan GANTT chart. The evaluation team did not receive an updated GANTT chart for each proposed activity detailing progress. Instead, the most recent Quarterly Report for January to March 2014 contained a table listing the number of project activities that were on and off track (see exhibit 9.1). According to this table I3 no activities are delayed, off-track or cancelled.

As of March 31, 2014 Comp 1

Comp 2

Comp 3

Cross Cutting Special Activities Fund Total

Activities initiated 17 - 3 - - 20 Ongoing activities On-Track 8 3 0 1 1 13

Ongoing activities Off-Track - - - - - -

Activities completed 5 - 3 - - 8 Totals 30 3 6 1 1 41

According to the Year 1 annual report, it appears that there have been significant gaps in implementation in Year 1, however, the project has made considerably more progress in the first two quarters of Year 2 and most activities are reportedly on schedule. The lack of reporting according to an approved M&E plan

35 The I3 M&E plan was subsequently approved by USAID and is now in use by the project to monitor and report on results.


make it difficult to conclude whether the project is “on-target” and expected progress has been achieved. Key GPH counterparts, while encouraged by recent progress, expressed frustration at the rate of implementation, particularly, the delay in procuring key consultants.

In order to determine progress and whether or not this goal is on-target, and given the lack of a comprehensive reporting document against the annual work plans or an M&E plan, the evaluation team developed a chart (see table 9.7 below) cross-referencing the activities and deliverables in each annual work plan with the reported results in annual and quarterly progress reports. This chart pertains to the USAID I3 chart only and does not track other activities outlined in the JCAP for this goal, as there is no such document that reports on this. From the various progress reports, the evaluation team was able to ascertain whether the activities were in progress, delayed, or completed, or whether there was no information on the activities (in which case the presumption could possibly be that the activity has not yet started).

Table 9.7: Status of tasks, activities, and deliverables under Goal 8: Strengthen anti-corruption institutions (USAID i3Project and USDOJ)

Goal 8: Strengthen Anti-Corruption Institutions (USAID i3Project and U.S. DOJ) Component One: Increasing GPH Effectiveness in Enforcing Anticorruption Laws Help the anti-corruption enforcement agencies define, prioritize, and advocate legal changes needed to effectively detect, investigate, and prosecute corruption cases Analyze the legal framework. Deliverable: Initial assessment report with recommendations. Completed Run test scenarios/policy simulations. Deliverable: Scenarios based on inputted variables for discussion and strategic decision making in formulating stronger and viable legislation. Completed Equip counterparts with tools and data. Deliverable: Workshops with COA, OMB on legal changes. Completed

Support anticorruption agencies to improve the management of anticorruption investigations and prosecutions Initial, overall rapid assessment of anti-corruption enforcement management systems and processes. Deliverable: Anti-corruption Enforcement Effectiveness Assessment Report In progress Engage with OMB and COA in joint, in-depth needs and resources assessments and develop improvement plans. Deliverable: Blueprint for organizational and work flow management confirmation and/or redesign, recommendations, and assessments. In progress Assist anti-corruption institutions to improve case management/tracking systems Develop recommendations to support case management system. Evaluate current case management systems and processes. Deliverable: Blueprint for organizational and work flow management confirmation and/or redesign. In progress Start initial design of upgrades to optimize CMS utilization. Deliverable: Detailed technical specs for improving/establishing case management/tracking systems. In progress

Help the anti-corruption enforcement agencies to improve records management systems and evidence security Conduct an assessment and develop standards and/or principles for managing records, or to validate standards/principles employed to date with respect to records management systems and evidence security. Deliverable: Records management assessment and document management and security plan In progress


Goal 8: Strengthen Anti-Corruption Institutions (USAID i3Project and U.S. DOJ) Develop clear documentation in records management procedures. Deliverable: Documentation on case management procedures, record keeping, evidence security, and statistics. In progress Enhance the ability of anti-corruption institutions to implement systems for monitoring the performance of auditors, investigators, and prosecutors Establish collaborative process with OMB/COA to receive critical input and to determine internal benchmarks that will determine “success”. In progress

Assess the current system for setting departmental and individual employee performance evaluation/monitoring at the COA and OMB. In progress Make recommendations for improvements. Deliverable: Assessment report with recommendations for improvement to monitoring/evaluation of individual employees and departments. In progress Help COA to enhance audit operations Enhancing ability to detect fraud in infrastructure projects. Deliverable: Training workshop on fraud in infrastructure projects. No information Conduct a review of past trainings for Certified Fraud Examiners (CFE). No information Improve security and situational awareness for investigators, auditors, and prosecutors Deliverable: Personal security training for COA, OMB officers. Completed Review functional structures in support of a comprehensive organizational development assessment at COA with a focus on fraud and corruption. No information Support the preparation of COA peer review/professional audit standards. No information Help the Anti-corruption Enforcement Agencies Build Their Skills in Detecting, Auditing, Investigating, and Prosecuting Corruption Train new enforcement staff of anticorruption agencies Conduct training assessment (“analysis”). Deliverable: Training needs assessment report. No information Design curricula and identify relevant case studies (“design”). Deliverable: Course outline for trainings. No information Develop training modules (“development”). Deliverable: Case studies and other training materials No information Train OMB/COA trainers with new approach (“implementation”). Deliverable: Training of trainers at COA and OMB. No information Implement training. Deliverable: Training program for enforcement staff (auditors, investigators, prosecutors). No information Evaluate training. Provide advanced or specialized training in detection, investigation, and prosecution skills Support to OMB “special panels”. Deliverable: Institute a narrowly tailored training program that offers specialized courses in the elements of collaborative case build up with the intention of improving the overall strategic thinking of panel members in the pursuit of high-level corruption. No information Work with “whistleblowers,” including non-governmental organizations (NGOs), such as Transparency and Accountability Network or the Hills Program at the Asian Institute of Management. Deliverable: Training to NGO and CSO representatives. No information Design specialized curricula and case studies for OMB and COA. Deliverable: Specialized training program for enforcement staff. No information Provide technical assistance to the Office of the Ombudsman on bank secrecy laws. Implement training. Evaluate training.


Goal 8: Strengthen Anti-Corruption Institutions (USAID i3Project and U.S. DOJ) Further develop the training capacity of existing training units in the anticorruption agencies to conduct their own basic and routine training and to account for the use of donor funds Assess training units at OMB and COA . Deliverable: Assessment report of training units at OMB and COA. Delayed Capacity building of COA. Seek Ways to Promote and Facilitate Inter-Agency Cooperation Among the Anti-corruption Institutions Consider and assess the most efficient means for communication and coordination between agencies. In progress Create protocols for communication and coordination between agencies. Deliverable: Draft protocols for interagency coordination and communication for OMB and COA. In progress Training and facilitated networking. Deliverable: Facilitated networking sessions. In progress Provide technical assistance in the delivery of inter-agency training on fraud awareness: procurement. In progress Facilitate interagency data sharing, policies protocols, and platforms in support of anti-corruption audits and investigations. In progress Based on the team’s analysis of the USAID I3, project support for the various partner anti-corruption institutions has proceeded, but information about what exactly has been done and whether JCAP goals are being met is incomplete. Despite the relatively rapid procurement of the I3 project after finalization of the JCAP, there is not ample evidence that it has accomplished the goals or produced the outputs envisioned in the first 2 years of operation. The lack of a project M&E plan and reporting on defined quantitative targets under the plan makes it difficult to more accurately assess progress.

A review of DOJ-OPDAT reports and discussions with the DOJ representative in Manila indicated that PFG implementation is on track and the following agreed-upon activities in support of the ombudsman are being carried out:

• assisting with drafting and supporting the passage of money-laundering amendments that address bank secrecy and freezing orders;

• assisting with the creation, policies, interagency agreements, and technical capacity of a Philippine-seized or forfeited asset management authority; and

• reviewing and providing assistance with developing and drafting new procedures on the use of electronic evidence and forensic accounting and document fraud detection and providing training on same.

Goal 12: Strengthen Corporate Governance

Goal 12 under the weak governance constraint was included in the initial PFG mid-term evaluability assessment report. However, due to timing issues, interview priorities, and a lack of project activities, this goal was not evaluated further.


Narrow Fiscal Space Sub-Constraint

Goal 15: Increase Fiscal Space by Expanding the Tax Base, Involving the Project FPI The CA highlighted “fiscal space” as a primary binding constraint to growth—all the sources of government revenues were insufficient to meet the appropriate provision of public goods, infrastructure, and services (e.g., education) that would complement private investment. One element of this constraint is that a narrow tax base significantly constrains government revenues. Therefore, the JCAP named “increase fiscal space by expanding the tax base” one of its goals, with the need to improve both revenue collection and fairness within the tax system. The PFG aimed to work toward this goal by:

1. expanding the tax base by supporting a specialized fiscal unit within the DOF; 2. supporting revenue enhancement measures through data sharing within and across pertinent GPH

agencies; 3. indexing excise taxes; 4. rationalizing fiscal incentives; 5. mitigating revenue-eroding measures; and 6. reviewing the Internal Revenue Code.

Following these objectives, USAID designed a project, FPI, which will run from 2013 to 2018. It is the only project in PFG to address this particular goal. The contract for FPI was signed on August 22, 2013, 21 months after PFG was initiated.

In the SOW for FPI, the following activities are related to Goal 15 (as well as other goals):

• Activity #1: Support a comprehensive review and analysis of the effectiveness of the tax regime to produce revenues.

• Activity #2: Assist in vetting viable fiscal policy reforms that supplement tax administration modernization.

• Activity #3: Assist in the integration of tax‐related database systems among pertinent government agencies.

• Activity #4: Assist in the design of institutional capacity building that facilitates enhanced revenue collection.

• Activity #5: Support a reinvigorated privatization program to bolster the government’s rationalization plan.

Mapping these activities to those mentioned in the JCAP seems like a highly inexact exercise; for example, privatization does not appear to be mentioned in the JCAP, but appears as a main activity in the SOW for the FPI project.

As well as having the SOW, the evaluation received for FPI the first year work plan, the as-yet-to-be-official M&E plan, and progress reports covering October through December 2013 and January through March 2014. The evaluation team gathered a list of relevant activities from the project work plan, which is listed in table 9.8 below. From the various progress reports and from the responses from USAID leadership, managers (both USAID and GPH), and implementers working on this goal, the evaluation team was able to ascertain whether the activities were ongoing or not, or whether there was no


information at all on the activities (in which case the presumption could possibly be that the activity has not yet started).

Table 9.8: Status on projects and activities under the tax effort strengthened goal for the FPI project

Activities under the Tax effort strengthened goal for the FPI project International tax benchmarking. In progress E-filing bottlenecks assessment. No information Mobile money as a mode of tax payments assessment. No information Core tax data integrity analysis (including taxpayer registration). In progress Tax gap analysis. In progress Comprehensive review of tax policy options. In progress Political economy of taxation analysis. No information Taxation of mineral resources recommendations. No information Tax incentives strategy paper. No information Capital markets taxation analysis and recommendations. No information

Activity toward meeting Goal 15 (increase fiscal space by expanding the tax base) is only in its very earliest phases. Table 9.3 (provided previously) reflects only 7 months of activities of FPI (approximately 28 months after PFG officially began and more than 2 years after the JCAP forecasted that projects would start). The list of project accomplishments in table 9.8 above includes a list of ongoing activities that would be normal for such an early stage in a project’s implementation. There are really two issues that arise when qualifying an answer to the question, “Are the performances of the selected PFG interventions on target?” The first is whether the start date of the relevant projects was on target given when PFG began. The second is whether or not the interventions envisaged in the project SOWs are on target given when the project started. The answer to the first question for Goal 15 (increase fiscal space by expanding the tax base) is that the start date of the relevant project seems to have been quite delayed relative to when PFG started and relative to the following statement in the JCAP: “Q2-2012: PFG projects [to be] rolled out.” The answer to the second question for Goal 15 is that there is no evidence indicating that the relevant interventions were delayed in implementation once the relevant project began official operation.

Finding 3: Most USG staff feel PFG is on target. Most GPH staff cite delays in procurement

As indicated previously in table 9.1 above, interviewee responses to whether PFG is on target varied by the USG and GPH. Overall, when asked whether the PFG interventions are on track at midterm, USG largely felt that they were—10 out of 13 members of leadership said “yes,” and three out of five program managers said “yes.” GPH respondents were more skeptical—five leaders said “yes,” none said “no,” and four did not respond. The USG, especially USAID, felt that the re-design of USG development interventions and the collaborative processes of the PFG were executed efficiently. They also felt strongly that after agreements were reached with GPH counterparts on project design, their procurement was much faster than is usual for the agency. The GPH leadership who largely felt that the PFG was not on target, explained that it was primarily due to projects not starting as quickly as they should have. They seemed largely supportive of the design and consultation phase of the PFG, but expressed some dissatisfaction


with project procurement and implementation. Some GPH respondents cited specific projects and implementers as having taken too long to provide consultants and key deliverables.

Finding 4: The annual PFG scorecards indicate macro-level progress in the Philippines that is less indicative of PFG progress PFG progress is also tracked by an annual scorecard, per the JCAP guidelines. This scorecard uses macro-level indicators to measure overall and constraint-specific progress. Nine out of the 10 scorecard indicators for the Philippines, including in each constraint, demonstrate progress since the PFG started in 2011. This number may seem to imply that the PFG is on track and having the anticipated results. However, improved Philippine economic performance can be dated to early in 2012, far before most PFG projects were online. Moreover, there was much discussion among respondents about whether or not these indicators measure inclusive growth. Many pointed to the persistent poverty level as evidence that other indicators need to be considered.

Overall Progress According to PFG Philippines Scorecard

The Philippines has experienced tremendous accelerated economic growth over the past few years, as indicated by the country’s GDP. However, there is concern that this growth has not been inclusive and has only benefited a few sectors. Other PFG indicators indicate varying progress: GDP outside of Manila has increased, while unemployment has not changed. Table 9.9: Overall progress according to the PFG Philippines scorecard

Overall Performance

Indicator Year Rationale/Definition Source 2011 2012 2013

GDP Growth (at constant 2000 prices, %)

3.6 6.8 7.2 Primary indicator of a country’s economic performance. Defined as the value of all goods and services produced domestically; the sum of gross value added of all resident institutional units engaged in production (plus any taxes, and minus any subsidies, on products not included in the values of their outputs).

Philippine Statistics Authority (PSA) National Statistical Coordination Board (NSCB) – National Accounts of the Philippines

GRDP Growth of Regions Outside Metro Manila (at constant 2000 prices, %)

3.1 7.3 - Inclusive growth indicator. GRDP is the aggregate of gross value added of all resident producer units in the region.

PSA NSCB – Regional Accounts

Unemployment Rate (%) – Jan/Apr/Jul/Oct Average

7.0 7.0 7.1

Inclusive growth indicator. Persons who are 15 years and over as of their last birthday and are reported as: (1) without work and currently available for work and seeking work, or (2) without work and currently available for work but not seeking work.

PSA NSCB – Labor and Employment

SOURCE: Philippines – U.S. Partnership for Growth Scorecard, April 14, 2004 (produced by the Philippines National Economic Development Authority [NEDA]).

http://www.nscb.gov.ph/sna/default.asp







http://www.nscb.gov.ph/grdp/2012/default.asp

http://www.nscb.gov.ph/grdp/2012/default.asp

http://www.nscb.gov.ph/secstat/d_labor.asp

http://www.nscb.gov.ph/secstat/d_labor.asp


Facilitate Trade and Investment

Progress on this goal is indicated by the growth of fixed capital formation, the value of exports of goods and services, and the ranking on the World Economic Forum Competitiveness Scores. All of these began improving in 2012, well before the TRADE project came online. Therefore, the improved economic performance cannot be attributed to the inception of the PFG. Table 9.10 provides scorecard data for Facilitate Trade and Investment.

Table 9.10: Facilitate trade and investment scorecard data FACILITATE TRADE AND INVESTMENT SCORECARD DATA


Growth of Fixed Capital Formation (at constant 2000 prices, %)

–2.0 10.4 11.7 Investment indicator. Fixed capital formation measures the value of acquisitions of new or existing fixed assets by the business sector, government and households less disposals of fixed assets.

PSA NSCB – National Accounts of the Philippines

Value of Exports of Goods and Services (U.S. $, millions)

56,134 64,884 51,330 (Q1–Q3)

Exports cover all goods in which ownership has been transferred from Philippine residents to non-residents through sales, grants, gifts, and donations. Services are the result of a production activity that changes the conditions of the consuming units, or facilitates the exchange of products or financial assets. This account covers 12 services transactions based on the Balance of Payments and International Investment Position Manual, 6th edition (BPM6).

Bangko Sentral ng Pilipinas – Balance of Payments BPM6 Format, New Concept

World Economic Forum Competitiveness Ranking – Percentile Rank

47 (75/142)

55 (65/144)

60 (59/148)

Indicator for competitiveness landscape of economies, providing insight into the drivers of their productivity and prosperity. The Global Competitiveness Index (GCI) is a weighted average of many different components that are grouped into three sub-indexes and 12 pillars of competitiveness: (A) Basic Requirements Sub-index - (1) institutions, (2) infrastructure, (3) macroeconomic environment, and (4) health and primary education; (B) Efficiency Enhancers Sub-index - (5) higher education and training, (6) goods market efficiency, (7) labor market efficiency, (8) financial market development, (9) technological readiness, and (10) market size; and (C) Innovation and Sophistication factors - (11)

WEForum Global Competitiveness Rankings





http://www.bsp.gov.ph/statistics/efs_bop2.asp






http://www.weforum.org/issues/global-competitiveness





FACILITATE TRADE AND INVESTMENT SCORECARD DATA Indicator Year Rationale/Definition Source

2011 2012 2013 business sophistication, and (12) innovation. The computation of the GCI is based on successive aggregations of scores from the indicator level (i.e., the most disaggregated level) up to the overall GCI score. Unlike the case for the lower levels of aggregation, the weight put on each of the three sub-indexes (basic requirements, efficiency enhancers, and innovation and sophistication factors) is not fixed; it depends on each country’s stage of development.

Weak governance constraint

Progress under the weak governance constraint is measured on the scorecard by the Rule of Law and Control of Corruption indices from the World Bank Worldwide Governance Indicators. Both indicators are macro, country-level composite indices from various surveys and analysis, such as the Freedom House “freedom in the World” indicator and World Justice Project Rule of Law Index. Little progress is shown under the Rule of Law Index with a slight percentile rank increase from 36.15 to 36.49 (out of 100). Control of corruption has seen better progress moving in 1 year from 26.07 to 33.49 percentile rank among all countries (see table 9.11 below).

Table 9.11: Status of weak governance constraint according to scorecard WEAK GOVERNANCE CONSTRAINT


Rule of Law index – Percentile Rank

36.15 36.49 –

An index of perceptions of the extent to which agents have confidence in and abide by the rules of society, including the quality of contract enforcement, property rights, the police, and the courts, and the likelihood of crime and violence.

World Bank Worldwide Governance Indicators

Control of Corruption Index – Percentile Rank

26.07 33.49 – An index of perceptions of the extent to which public power is exercised for private gain, including both petty and grand forms of corruption, as well as “capture” of the state by elites and private interests.

World Bank Worldwide Governance Indicators

http://info.worldbank.org/governance/wgi/index.aspx?fileName=wgidataset.xlsx









Fiscal space constraint

Progress on this constraint is measured in the scorecard by revenue effort (total revenue to GDP, percent) and by national government expenditures relative to GDP (percent). In the latest version of the scorecard there are no values for 2013 and the major project in this area came online in 2013. Therefore, the scorecard contains no information that is relevant to whether the inception of the PFG led to changed performance on this constraint.

Table 9.12: Scorecard data for fiscal space constraint FISCAL SPACE


Revenue Effort (Total Revenue to GDP, %)

14.0

14.5

– Revenue administration indicator (includes all tax and nontax revenues).

PSA NSCB – National Government Cash Budget; PSA NSCB – National Accounts of the Philippines

National Government Expenditures to GDP (%)

16.0

16.8

– Increased revenues are expected to increase public expenditures. This indicator uses national government cash expenditures.

PSA NSCB – National Government Cash Budget; PSA NSCB – National Accounts of the Philippines

The relevance of these indicators was questioned by many stakeholders, as discussed previously in Country-Specific Question 2, as well as by the evaluation team, for their effectiveness in measuring PFG impact and progress. High-level indicators such as those in the scorecard tend to capture too wide an array of variables to allow for attribution to PFG activities. Many felt the current scorecard indicators are not useful for managing PFG results. There is also some concern that more important and relevant indicators are not being used, including many of those identified in the JCAP (see below): Enhanced Domestic Investment Climate

• Share of regional foreign direct investment (FDI) increased. • Gross capital formation increased. • Global Competitiveness Index: increasing or on par with leading regional and emerging

market peers. • Index of Economic Freedom (overall score, business, investment): increasing or on par

with regional and emerging market peers. • Doing Business Ranking and Related Indicators: improving or on par with regional and

emerging market peers.

http://www.nscb.gov.ph/secstat/d_finance.asp

















• Business Start-ups: increased in key sectors (agribusiness, business process outsourcing, creative industries, infrastructure, manufacturing and logistics, mining, and tourism).

Governance

• Corruption Perception Index: improving or on par with regional and emerging market peers. • World Governance Indicators (Regulatory Quality, Rule of Law, Control of

Corruption): improving or on par with regional and emerging market peers. • World Justice Project’s Rule of Law Index (Accountable Government, Open Government

and Regulatory Enforcement, Access to Justice): improving or on par with regional and emerging market peers.

• Court Congestion: eased through docket cleansing, case track management, and capacity building of commercial, tax, and anti-graft courts.

Fiscal Space

• Tax effort improved. • Budget deficit at manageable levels. • Public spending on priority areas, particularly in infrastructure and education.36

Finding 5: PFG project implementation have not progressed enough to effectively measure progress Despite the overarching sentiment from USG staff that PFG is on target, there is no objective, results-based method of verifying this claim. Most projects were recently initiated. PFG has also not clearly identified an M&E framework or set of indicators beyond the macro-level PFG scorecard indicators that identify how progress will be measured. At the time of this evaluation, there are two, unconnected levels of measuring progress—project-specific indicators that individual implementers and partner GPH institutions are tracking for their own progress and macro-level, global, country-wide indicators compiled by the World Bank and other global institutions that are not closely matched with PFG interventions and provide little to no effective data on progress. Many respondents raised the question of whether it is too early to conduct a mid-term evaluation of PFG Philippines.


The PFG process takes time, particularly if projects are starting from scratch. It should be included in the expectations of any PFG initiative that the actual implementation of programs will be delayed by the necessary collaborative and consensus-building activities. In the Philippines, this delay was not taken into account and the JCAP indicated some idealistic timelines.37 However, the findings lead to the conclusion that the extra time and delay caused by the PFG process was worth the effort. Establishing consensus and commitments from each side on key reforms was a key

36 See JCAP, pg. 11 37 The PFG process could potentially be accelerated in countries where projects already exist or have been previously discussed and agreed upon with government counterpart and are retro-fitted into the PFG analysis.


byproduct of PFG analysis and negotiations. Establishing these components has led to more effective programming and more sustainable solutions. Given the time that implementation actually begun, projects and activities are generally on target. Finally, monitoring systems for the PFG Philippines initiative need to improve quickly in order to accurately measure progress toward PFG objectives. Output and other activity data have not yet been assembled and analyzed across PFG activities. Even if this data analysis was available, a theoretical framework does not exist that links outputs to outcomes for PFG, making it impossible to objectively measure whether or not initiatives are on target at midterm.


Based on the findings from the evaluation, the evaluation team recommends the following:

• The PFG Philippines management team should define “midterm” and “on target” under the PFG as these apply to each particular country and context, taking into consideration the analytical, consensus-building, design, and joint-implementation process. Given the PFG Philippines context and the timeline for implementation, “midterm” and “end line” should be revised to incorporate the 1-year inception period.

• PFG Philippines leadership should facilitate the rapid completion of a comprehensive M&E framework that links outputs to outcomes for all JCAP goals, and also appropriately links these goals to the constraints.

• Reporting and monitoring against this M&E framework needs to begin and occur regularly at the technical sub-committee level for each constraint, with regular reporting and reviews of all constraints by the steering committee.

• Technical sub-committees need to be re-organized as a council of peers where project results are discussed and areas of collaboration identified. Remove the “chair” denomination or have a rotating chair with responsibilities to convene and host.

• Conduct an annual strategic review of PFG progress across the USG and GPH with a pre-established date, both as a means to review progress and to incentivize implementation.


ANNEXES

Annex 1: Evaluation Statement of Work

STATEMENT OF WORK PARTNERSHIP FOR GROWTH (PFG)

MID-TERM EVALUATION: EL SALVADOR AND THE PHILIPPINES

I. BACKGROUND INFORMATION Partnership for Growth (PFG) is a set of bilateral partnerships between the United States and a select group of four countries (El Salvador, Ghana, the Philippines, and Tanzania) to accelerate and sustain broad-based economic growth by putting into practice the principles of President Obama’s September 2010 Presidential Policy Directive on Global Development. It involves rigorous joint analysis of constraints to growth, the development of joint action plans to address these constraints, and high-level mutual accountability for implementation. One of PFG’s signature objectives is to engage governments, the private sector and civil society with a broad range of tools to unlock new sources of investment, including domestic resources and foreign direct investment. By improving coordination, leveraging private investment, and focusing political commitment throughout both governments, the Partnership for Growth enables partners to achieve better development results. Core principles of the Partnership for Growth include:

• Country ownership and partnership; • High-level political leadership and commitment to development progress; • Rigorous, evidence-based joint analysis on constraints to growth conducted by integrated teams

of U.S. Government and PFG country officials; • Joint decision making on where to focus and prioritize resources; • Use of a broad range of tools, including catalytic policy change, institutional reform, aid,

diplomatic engagement, and other ‘non-assistance’ policy tools; • Leveraging the whole of the US government;38 • Transparency, mutual accountability and fact-based monitoring and evaluation.

The PFG process consists of several steps, including:

• Agreement to initiate PFG with selected partner countries; • Joint analysis on constraints to growth, followed by broad consultation, dialogue on the findings; • Development of joint country action plans (JCAPs) that outline potential tools, reforms, technical

assistance and resources that can be applied over the next five years to address priority constraints to growth;

38 Examples of how whole of government is expressed and applied include http://www.whitehouse.gov/blog/2013/01/07/whole-government-commitment-inclusive-entrepreneurial-growth and http://www.state.gov/documents/organization/153139.pdf.

http://www.whitehouse.gov/blog/2013/01/07/whole-government-commitment-inclusive-entrepreneurial-growth

http://www.whitehouse.gov/blog/2013/01/07/whole-government-commitment-inclusive-entrepreneurial-growth

http://www.state.gov/documents/organization/153139.pdf


• Implementation of priority initiatives by USG agencies and partner governments; • Regular monitoring and evaluation, which includes semi-annual scorecards • Transparency and consultation with private and public sectors.

Documentation on PFG design, goals, objectives and accomplishments can be found at http://www.state.gov/e/eb/ifd/odf/pfg/countries/index.htm and http://www.mcc.gov/pages/activities/activity-two/partnership-for-growth. II. CONTEXT An important aspect to bear in mind at all times is that PFG is a bilateral partnership at the country level. The evaluation will be conducted by an external evaluator agreed upon by the U.S. government (USG) and, for evaluations in their countries, the governments of El Salvador and the Philippines. II.a Timing Considerations This evaluation will only focus on El Salvador and the Philippines, as they are the first PFG countries due for a mid-term evaluation in 2013 and 2014. It will span the U.S. government and national (i.e. Salvadorian and Filipino) government leadership, implementing agencies, activities, strategies, stakeholders and audiences (both public and private). El Salvador is scheduled to hold national Presidential elections in February 2014; the evaluation team must therefore consider the time limitations on national government staff and personnel leading up to this date and efficiently and effectively plan its engagement with these stakeholders. The fourth PFG-El Salvador scorecard (see section II.d “Existing Documents and Data Sources” for scorecard details) is scheduled for release in late November 2013, which requires extensive consultations and negotiations between the two government teams. There may be overlap between the mid-term evaluation field visit and efforts to complete the scorecard. In addition, the country-specific results of the evaluation will serve as a tool for transitioning into the new administration in El Salvador. By June 30, 2013, the current Philippines administration would have completed the first half of its six-year term. It is presently reviewing priorities articulated in the Philippine Development Plan. The administration will likely focus on efforts that will better ensure inclusive growth, increased employment, and policy continuity into the successor administration. A PFG evaluation in spring 2014 will benefit from the GPH’s review and revalidation of its priorities. II.b Target Areas and Groups

No single criterion was used to identify target populations for PFG activities. Some PFG activities are national in scope, and others target specific sub-populations, regions and sectors. II.c Results Frameworks and Intended Results The El Salvador and the Philippines PFG efforts have tailored, unique results framework developed in response to the constraints to growth analysis. Following is the logical framework, reflected in detail in the Joint Country Action Plan (JCAP). The frameworks reflect only the constraints and the goals necessary to alleviate or address the constraints. More information on the agreed lines of action to achieve the goals can be found in each country’s JCAP.

http://www.state.gov/e/eb/ifd/odf/pfg/countries/index.htm

http://www.mcc.gov/pages/activities/activity-two/partnership-for-growth


Constraint = a binding constraint to growth, identified explicitly in the constraints to growth analysis39 Goal = a necessary objective to alleviate and address the constraint, identified in the Joint Country Action Plan Line of Action = a programmatic response, by one or both governments identified in the Joint Country Action Plan. A line of action may be a project or a policy change or any other discrete intervention at the implementation level. The government responsible for executing the line of action is clearly identified in the JCAP. For the Philippines, the lines of action are identified in “Section B: Summary Matrix of PFG Activities” starting on page 15 of the JCAP. For El Salvador, the lines of action are identified under the “What the GOES / USG intends to [do]” bulleted lists under each Goal description, starting on page 8 of the JCAP. LOA may in many cases be synonymous with project or activity.

39 A Constraints Analysis is a study based on the growth diagnostic approach originally developed by Haussmann, Rodrik, and Velasco (2005) and since elaborated by others, including the United States Millennium Challenge Corporation (MCC). Growth diagnostics seeks to identify, for a particular country at a particular point in time, the principal barriers – the “binding constraints” – to that country achieving and sustaining faster economic growth. It starts with the premise that those constraints affect growth by preventing private investment and entrepreneurship from reaching the levels they would attain in the absence of those constraints.

El Salvador Results Framework - adapted from the Joint Country Action Plan -

Increased Economic Growth and Investment

2 Binding Constraints

Crime/Insecurity Constraint

14 Goals

Low Productivity in

Tradables Constraint

6 Goals

Strengthen Justice Sector

Institutions 6 LOAs,

Improve Criminal Justice

Procedures

5 LOAs

Reduce Impact of Crime on Businesses

5 LOAs

Reduce Impact of Crime on

Public Transportation

H7 LOAs

Remove Assets f rom Criminal Organizations

3 LOAs

Strengthen El Salvador's

Civil Service

6 LOAs

Promote a National

Dialogue to Improve Security

3 LOAs

Growth Council

13 LOAs

Reduce Firms' Costs t o Improve Competit ive

ness 13 LOAs

Strengthen Labor Force

24 LOAs

Assist At-Risk Youth through

Economic Opportunit ies

4 LOAs

Strengthen the National Police

4 LOAs

Improve Education for Youth in High

Risk Municipalit ies

2 LOAs

Prevent Crime and Violence in

Key Municipalit ies

9 LOAs

Reduce Overcrowding

in Prisons 5 LOAs

Enhance Prison

Security

5 LOAs

Promote Use of Extradition to Combat Crime

5 LOAs

Raise Tax Revenues

10 LOAs

Strategy for Promoting

FDI 13

Surmount Low

Productivity in Tradables

11 LOAs

LOA = Line of Action

LOAs


PHILIPPINES Results Framework

Adapted from the Joint Country Action Plan

REGULATORY QUALITY IMPROVED

BROAD-BASED AND INCLUSIVE GROWTH ACCELERATED AND SUSTAINED

RULE OF LAW AND ANTI-CORRUPTION MEASURES

STRENGTHENED

Trade and Investment Facilitated

FISCAL PERFORMANCE IMPROVED

Competition Policy Improved

Philippine Participation in Regional and International

Trade Arrangements Enhanced

Human Capacity for Economic Growth Enhanced

Regulatory Bottlenecks, Entry Barriers and Discriminatory Provisions to Investments

Reduced

Efforts to Improve Judicial Efficiency Supported

Anti-Corruption Institutions Strengthened

Contract Enforcement Strengthened

Intellectual Property Rights Enforcement Strengthened

Integrity- and Confidence-Enhancing Measures Supported

Corporate Governance Strengthened

Opportunities for Corruption Reduced

Ethical Standards Enforced

Accountability Measures Strengthened

Fiscal Space by Expanding the Tax Base Increased

Fiscal Space by Minimizing Revenue Loss Increased

Fiscal Space by Improving Expenditures Management

Increased

105

II.d Existing Documents and Data Sources

A wide range of documentation is publicly available on PFG, including semi-annual “scorecards” of progress made on JCAP implementation. Scorecards: As described in the El Salvador M&E Addendum, once PFG implementation began the governments of El Salvador and the United States decided to publish semi‐annual scorecards to periodically report to the public on progress towards achievement of PFG goals. The two governments use the descriptors “Ahead of Schedule,” “On Track,” “Behind Schedule,” or “Completed”, to characterize progress on each goal. Scores reflect the consensus view of the two governments. The score for each goal is accompanied by a description that provides a justification for each score assigned. This justification includes the results of the goal indicators included in the M&E addendum along with other relevant supporting information and data (which may include results of monitoring and evaluation conducted independently by each government on individual LOAs). The goal indicators are designed to reflect the shared purposes of the governments of El Salvador and the United States. If data for goal indicators is not available in a particular reporting period, progress will be reported in the following period. The Philippines’ PFG performance is tracked through indicators that are already collected and which are mostly publicly available and verifiable, as follow:

• Overall: GDP Growth, Exports Growth, Positive Credit Ratings, Index of Economic Freedom, Employment Growth, Non-Metro Manila Regions Share in GDP

• Enhanced Domestic Investment Climate: Foreign direct investment (FDI) to GDP ratio, Capital

formation to GDP ratio, IMD Competitiveness Ranking, World Economic Forum Competitiveness Ranking, Global Enabling Trade Index Ranking, Doing Business Ranking, Government Effectiveness Indicator (World Governance indicators), Land Rights and Access, Regulatory Quality Indicator (World Governance Indicators)

• Rule of Law and Anti-Corruption: Corruption Perceptions Index, Control of Corruption Ranking,

Rule of Law index – Regulatory Enforcement, Rule of Law Index – Absence of Corruption, Court Congestion Indicator

• Fiscal Space: Tax Effort, Infrastructure Expenditures to GDP ratio, Education Expenditures to

GDP ratio, National Government Deficit to GDP ratio, Open Budget Ranking The above indicators will be supplemented by activity level output and outcome indicators for specific programs The following sites archive multiple reference documents for El Salvador and the Philippines: El Salvador: http://sansalvador.usembassy.gov/partnership-growth.html http://tecnica.presidencia.gob.sv/temas/iniciativa-asocio-para-el-crecimiento.html Additional baseline data El Salvador: Not as much crime and citizen security data is available as information on productivity and investment, so the USG and GOES are making efforts to obtain more information for PFG monitoring purposes as well as to inform public policy and donor interventions outside of PFG. A recent extensive baseline study on public perceptions on security and crime was completed by USAID/El Salvador, available publicly at: http://pdf.usaid.gov/pdf_docs/PNAEA859.pdf.

http://sansalvador.usembassy.gov/partnership-growth.html

http://tecnica.presidencia.gob.sv/temas/iniciativa-asocio-para-el-crecimiento.html

http://pdf.usaid.gov/pdf_docs/PNAEA859.pdf

106

This information should be used by the evaluation team both as background to understand the development context in the country, and to inform the responses to the evaluation questions. The scorecards are made public in English on the US Embassy in El Salvador website, and in Spanish on the Executive Secretary to the President of El Salvador website. To date, three semi-annual scorecards have been released. The Philippines: http://manila.usembassy.gov/partnership_for_growth.html http://www.iro.ph/index.php http://www.neda.gov.ph/PDP/rm/pdprm2011-2016.pdf http://222.127.10.196/national.html In addition, the Philippines PFG team prepares updates that can be made available to the evaluation team.

III. EVALUATION RATIONALE III.a Evaluation Purpose

The evaluation will serve two purposes. As a result, there are two sets of evaluation questions. Purpose 1: The first purpose is to evaluate whether the PFG process demonstrates improvements over pre-PFG assistance approaches. In particular, the evaluation will examine the extent to which the PFG’s whole-of-government and constraints analysis approach led to a change in the manner of USG delivery of development assistance and whether these changes demonstrated improvements in terms of operational efficiency, selection, coordination, design and management of development interventions, and ultimately increased the probability and effectiveness of assistance efforts in achieving verifiable results. The findings and conclusions of this part of the mid-term evaluation will help decision makers determine whether PFG indicates an improved model for providing assistance and whether it portends a higher probability of achieving desired development results. Furthermore, it will inform governments in their work with all donors. Purpose 2: The second purpose is to: 1) evaluate whether PFG efforts have been developed in such a way as to allow for the eventual determination of their impact on addressing the identified constraints and desired outcomes; and, 2) to evaluate the performance of certain initiatives to date to determine whether or not they are moving in the right direction, are considered necessary and sufficient to achieve PFG goals, and are contributing to national interests through the integration and coordination of work done by both governments. The findings and conclusions of the country-specific portion of the mid-term evaluation are of particular relevance and will provide tangible input to the national government and USG entities for identifying obstacles and optimizing PFG implementation in the field, allowing for country program course corrections where feasible and needed in order to enhance the likelihood of achieving sustainable, cost-effective and measurable results. In fulfilling this second purpose, the expectation is to conduct an assessment of the evaluability of the PFG JCAPs (i.e. in other words, assess the extent to which the current PFG programs, as designed and implemented, are evaluable and can or will demonstrate , in verifiable terms, the results they intend to deliver) and evaluations of performance to date. The contractor first will be asked to conduct a preliminary evaluability assessment of each country’s PFG JCAP (this assessment links to question “1” of the country-specific questions in Section IV.a). The

http://manila.usembassy.gov/partnership_for_growth.html

http://www.iro.ph/index.php

http://www.neda.gov.ph/PDP/rm/pdprm2011-2016.pdf

http://222.127.10.196/national.html

107

preliminary evaluability assessment should use the available program information to assess the following components of each goal and its corresponding LOAs (or LOA equivalent):

• problem diagnostic and baseline situation assessment; • causal logic of activities, objectives, and outcomes; • intended beneficiaries; and • data availability.

The preliminary evaluability assessment will be used to identify goals and LOAs that are ready or amenable for further in-depth “second-tier” review during the mid-term evaluation, taking into consideration Post and host country recommendations, when it is technically feasible and maintains the integrity of the evaluative methodology. At a minimum, at least two goals (one per constraint) that are amenable will then be reviewed to determine whether:

• The indicators selected to measure their progress cover the overall logic of the PFG interventions;

• There are any major gaps in data collection and analysis that could prevent the interagency partners and joint steering committees from:

o Adequately managing implementation towards expected results; o Evaluating the effectiveness of PFG.

This second tier evaluability assessment links to question “2” of the country-specific questions. (See section IV.a for details) The goals and LOAs selected for the second tier assessment also will form the “sample” of LOAs or projects that will be evaluated to determine performance at the mid-term. This performance evaluation links to question “3” of the country-specific questions. (See section IV.a for details) The current scope is only for the El Salvador and Philippines mid-term evaluations. It is expected that the other PFG countries will undertake mid-term PFG evaluations at a later date. Similarly, a final evaluation of PFG and PFG efforts in each country is anticipated. While not covered under this SOW, data captured may be employed in the eventual final evaluations and provide the foundation for making conclusions at that time. III.b Audience and Intended Uses

The mid-term evaluation will be made available on-line to the public. There are many audiences for the mid-term evaluation, including: Implementers

• The national government Minister of Foreign Relations (or the equivalent) and PFG Coordinating Committee in,

• The U.S. Ambassadors and Country Teams, • The White House and participating U.S. Agency PFG Coordinators and country desk officers in

Washington, DC, • Relevant agencies/organizations implementing JCAP activities;

Stakeholders

• Citizens of El Salvador and the Philippines, • Civil society representatives and organizations, in the U.S., El Salvador, the Philippines; • Private sector commercial companies and organizations, in the U.S., El Salvador and the

Philippines,

108

• Diaspora communities residing in the United States, and • The international donor community interested in lessons learned from applying the 2005 Paris

Declaration on Aid Effectiveness and the 2008 Accra Agenda for Action-assistance40

IV. EVALUATION DESIGN AND METHODOLOGY The Contractor should propose the most rigorous evaluation methodology feasible and cost-effective given the learning potential and scope of the study. To the greatest extent possible, the Contractor shall analyze and collect quantitative data. IV.a Evaluation Questions The evaluation questions address issues of common concern for all PFG countries (the cross-cutting questions), as well as country specific questions tailored to each country’s individual situation. Most of the cross-cutting evaluation questions will focus on organizational management structures which are common across all four countries. Country-specific evaluation questions are more appropriate to testing the theory of change at the technical level and will be used to make country-specific recommendations in the final Mid-Term Evaluation Report. Mid-Term Cross-Cutting Evaluation Questions:

1. What are the advantages and/or disadvantages of the PFG* whole of government approach to development assistance? The intent of this question is to assess the extent to which the PFG efforts intended changes in development assistance have or have not materialized. The whole of government approach is relevant to identifying areas for assistance, selecting interventions, and determining implementation coordination. The question is relevant both to national government agencies and institutions, and U.S. government agencies and institutions overseas and in Washington, DC. * Explanation of “Whole of Government”: In large bilateral efforts such as Partnership for Growth, many different governmental agencies and ministries are involved in both governments in different dimensions of the larger effort. Within the U.S. government, the term “whole of government” reflects efforts to align each agency’s activities to achieve a common objective. Footnote 1 provides resources for further explanation.

2. To what extent has Partnership for Growth affected the workload on national government and

U.S. government staff, as compared to the workload created by traditional forms of development assistance delivery?

3. What contribution has non-assistance41 made to the PFG process and how can it be utilized

moving forward?

40 For further information on the 2005 Paris Declaration on Aid Effectiveness and the 2008 Accra Agenda for Action-assistance please visit http://www.mcc.gov/pages/activities/activity-two/aid-effectiveness: http://pdf.usaid.gov/pdf_docs/pdacq942.pdf 41 PFG calls upon the US Government (USG) and partner countries to be more comprehensive and creative in our development work – to reach beyond aid to all the instruments that both governments can bring to bear to connect and amplify the impact of current investments and unlock growth potential. USG commitments under PFG are comprised of both assistance and non-assistance tools that, undertaken in close coordination with partner countries, will maximize our impact and success. In addition

http://www.mcc.gov/pages/activities/activity-two/aid-effectiveness

http://pdf.usaid.gov/pdf_docs/pdacq942.pdf

109

Mid-Term Country-Specific Evaluation Questions: Country-specific questions look directly at the efforts unique to a PFG country. In this portion of the evaluation, evaluators are expected to assess the evaluability of the PFG effort in each country and, to the extent possible, determine progress to date in a select amount of initiatives in PFG framework. El Salvador

1. The constraints analysis does not identify remedies to address the binding constraints to growth. For each of the constraints, are the goal-level commitments set forth in the JCAP alone capable of achieving the constraints-level objectives and outcomes? (See Section III.a “Evaluation Purpose” for details on expectations related to this question.)

2. The PFG model places an emphasis on evidence-based decision making and fact-based monitoring. Is quantitative and objectively verifiable information being used to manage JCAP implementation in order to achieve and measure results? (See Section III.a “Evaluation Purpose” for details on expectations related to this question.)

3. At the mid-term, are the performances of the selected PFG interventions on target and creating the necessary outputs to achieve the desired outcomes?

The Philippines

1. The constraints analysis does not identify remedies to address the binding constraints to growth. For each of the constraints, are the goal-level commitments set forth in the JCAP alone capable of achieving the constraints-level objectives and outcomes? (See Section III.a “Evaluation Purpose” for details on expectations related to this question.)

2. The PFG model places an emphasis on evidence-based decision making and fact-based monitoring. Is quantitative and objectively verifiable information being used to manage JCAP implementation in order to achieve and measure results? (See Section III.a “Evaluation Purpose” for details on expectations related to this question.)


IV.b Evaluation Design The evaluation will be a performance evaluation, but should highlight the results of any impact or other rigorous analyses done separately on PFG goals or lines of action (LOAs) at the country level. A performance evaluation should include descriptive questions. The mid-term evaluation will include but to those actions already identified by the interagency and partner countries, additional non-assistance activities should be considered over the life of PFG for a sustained and focused effort. Non-assistance options provide a venue for demonstrating United States support to partner countries and the Partnership for Growth. Options are intended to fully leverage the United States’ unique convening authority, NGOs, professional organizations and academic institutions, donor groups, regional banks, and diaspora communities, and policy options for development results.

110

not be limited to semi-structured interviews, focus groups of stakeholders, and documentation reviews. Where feasible and appropriate, efforts should be made to incorporate quantitative data collection or analysis to measure program performance. The evaluator is expected to incorporate input from a reasonable range of civil society and the private sector. Offerors are encouraged to propose cost effective approaches to the evaluation. Additionally, for addressing country-specific questions, the contract may propose various methodologies to create a representative sample of the larger effort (for example, selecting to analyze only certain LOAs or goals, based on the level of foreign assistance investment they’re receiving) to ensure the scope of the evaluation is manageable and cost-effective while retaining its ability to provide a general assessment of the PFG effort and provide actionable recommendations for the Steering Committees, partner governments and US interagency going forward. At a minimum, at least two goals (one per constraint) that are amenable to an in-depth second-tier review will be selected for this purpose. Cost-benefit and cost-effectiveness analysis will not be utilized in the mid-term evaluations. IV.c Evaluation Points of Contact The COR for this evaluation will be the primary POC for the cross-cutting and both country-specific evaluations. The COR will be located in Washington. He or she will have responsibility for representing the evaluation and its progress to the larger USG PFG group. Each country will establish a POC team, consisting of one USG POC in Washington, one USG POC in the field, and one national government POC. The POC teams for each country will be responsible for communications with the COR. The USG-POC in Washington, DC, will help the evaluation teams liaise with all relevant stakeholders within the US inter-agency community at headquarters. The USG-POC based in the partner country, either within the U.S. Embassy or in another U.S. Agency there, will help the evaluation team reach all relevant USG stakeholders in country. The national government POC will help the evaluation team reach all relevant stakeholders within the country. IV.d Planning for Data Collection

Within the first 6 months of PFG implementation, a USG Goal Lead was named in El Salvador for each of the twenty goals. Goal leads are responsible for coordinating and consolidating line of action monitoring information that feeds into the semi-annual PFG scorecard. In general, the technical focus of each Goal determines which US Agency will be selected to act as Goal Lead and shepherd information collection among all USG agencies with lines of action under that goal. Goal Leads have been named from the Department of State, USAID, Treasury, Department of Commerce, Department of Justice, and MCC. Likewise, the GOES has named Goal Leads on their side to consolidate information on GOES progress related to their corresponding lines of action and to coordinate with the corresponding USG Goal Leads. Finally, several US agencies that do not have permanent representation in the country are implementing lines of action under PFG in El Salvador. The US Labor Department and Inter-American Foundation are two examples. Twenty ‘mirror’ Washington USG Goal Leads have been named to assist with the

111

consolidation of information and field queries from Post that need input or guidance from Washington, including progress reports from the non-presence agencies. For the Philippines, the GPH and the USG have stood up a Steering Committee within the first six months of JCAP approval to set the policy directions on the areas for PFG support, approve the general plans of action of the PFG Technical Sub-Committees on Regulatory Quality, Rule of Law and Anti-Corruption, and Fiscal Space, and oversee overall progress, among other functions. The three sub-committees provide advisory technical support to implement program goals and objectives and coordinate with implementing agencies involved in program activities. These sub-committees also assess/conduct analysis of sector performance and overall performance of programs and how these contribute to PFG goals; review overall PFG progress for reporting to the Steering Committee; and conduct periodic technical discussions and exchange views on sector issues affecting progress. One week of field work in Washington, DC, is estimated in the present scope of work so the evaluation team can meet with the USG Washington-based Goal Leads and other Washington-based PFG stakeholders. The field work in Washington, DC, should take place before the field work in country. In addition to the monitoring data on program activities normally collected by U.S. government and national government agencies during the course of implementation, PFG’s emphasis on shared responsibility with the national government and public transparency has resulted in an additional layer of periodic monitoring data that will be available to the evaluation team, such as the scorecards (see description above) and other host country data systems. The three evaluation POCs identified in section IV.c will provide the evaluation team with access to all existing PFG program monitoring data. The format, frequency and type of monitoring data collected by the GOES and GPH may be significantly different from the formats and types used by the U.S. government. The evaluation team will process the information and identify information gaps and data quality concerns in an inception report, to guide additional data collection required as part of the evaluation. Once the gaps in monitoring information are identified, the evaluation team will fill out the “Pre-Field Visit Data Needs and Analytical Guide” below and discuss the recommended approach with the COR to negotiate a final guide to be used once the team is in country.

112

Template: Pre-Field Visit Data Needs and Analytical Guide Cross-Cutting Questions

Evaluation Questions Type of Answer Needed (e.g. descriptive, normative, cause-effect)

Data Collection Method(s)

Gender Disaggregation of Data, where Possible

Sampling or Selection Criteria

Data Analysis Method(s)

1. What are the advantages and/or disadvantages of the PFG whole-of-government approach to development assistance?

2. To what extent has Partnership for Growth affected the workload on national government and U.S. government staff, as compared to the workload created by traditional forms of development assistance delivery?

3. What contribution has non-assistance made to the PFG process and how can it be utilized moving forward?

113

El Salvador Country-Specific Questions Evaluation Questions Type of

Answer Needed (e.g. descriptive, normative, cause-effect)





1. The constraints analysis does not identify remedies to address the binding constraints to growth. For each of the constraints, are the goal-level commitments set forth in the JCAP alone capable of achieving the constraints-level objectives and outcomes?

2. The PFG model places an emphasis on evidence-based decision making and fact-based monitoring. Is quantitative and objectively verifiable information being used to manage JCAP implementation in order to achieve and measure results?


114

The Philippines Country-Specific Questions Evaluation Questions Type of

Answer Needed (e.g. descriptive, normative, cause-effect)








115

V. EVALUATION PRODUCTS The set of evaluation milestones/products required are detailed below: 1. [Written Document ] Work Plan –

Due to the COR within the first 5 business days after start of the evaluation. The work plan will detail the Evaluation Team’s schedule in weekly blocks of time for the various tasks and deliverables, including desk review, inception report development, evaluation design, interviews (in the U.S., El Salvador, and the Philippines), data collection, data analysis and preparation of initial evaluation results, report writing, briefings and presentations.

2. [Written Document] Inception Report and Preliminary Evaluability Assessment–

Due to the COR within 3 weeks after the start of the evaluation. The inception report (see section IV.d) is a desk review of all existing documentation and monitoring data relevant to the specific PFG evaluation in question. The PFG evaluation places added emphasis on the inception report to ensure that all available monitoring and program data has been received, read and analyzed by the evaluation team prior to approval of field work.

A useful template and guide for the inception report is provided by the UNODC at http://www.unodc.org/documents/evaluation/IEUwebsite/Chapter_4_C.pdf. The current scope adopts the UNDP’s definition of an inception report:

“Evaluation inception report—An inception report should be prepared by the evaluators before going into the full-fledged evaluation exercise. It should detail the evaluators’ understanding of what is being evaluated and why, showing how each evaluation question will be answered by way of: proposed methods; proposed sources of data; and data collection procedures. The inception report should include a proposed schedule of tasks, activities and deliverables, designating a team member with the lead responsibility for each task or product. The inception report provides the programme unit and the evaluators with an opportunity to verify that they share the same understanding about the evaluation and clarify any misunderstanding at the outset.” (source: http://web.undp.org/evaluation/handbook/Annex3.html)

The preliminary evaluability assessment should use the available program information to assess the ability of the JCAP projects to demonstrate in measurable terms the results they intend to deliver (See section III.a “Evaluation Purpose” for details regarding the expectations and scope of the preliminary evaluability assessment.) The contractor should propose a methodology for sampling LOA for review in order to ensure that the scope of the evaluation and field work is manageable and cost-effective while retaining its ability to provide a general assessment of the PFG effort and provide actionable recommendations for the Steering Committees, partner governments and US interagency going forward, and will take into account Post and host country recommendations.

3. Updated Methodology and Evaluation Plan.

Once a final sampling strategy has been decided, the detailed evaluation methodology should be updated based on the preliminary review of all available JCAP and PFG data. The updated methodology should include a Pre-Field Visit Data Needs and Analytical Guide Report, which

http://www.unodc.org/documents/evaluation/IEUwebsite/Chapter_4_C.pdf

http://web.undp.org/evaluation/handbook/Annex3.html

116

includes information on data gaps, sampling strategy, pre-tested interview questionnaires and data collection timeline.

See section IV.d for the basic template, which can be adapted to country-specific needs with COR agreement.

4. End of Field Visit Debrief

Debrief to national and U.S. governments in El Salvador and the Philippines, including Washington, DC, PFG staff via teleconference. This will be delivered prior to departing the country while there to conduct the field visit. This presentation will update the team on the status of evaluation progress, identify any outstanding data or information, and describe any preliminary evaluation findings to date.

5. Draft Evaluation Reports (See Deliverable Six for types and quantities of reports)

Draft reports will be provided for all final reports outlined in deliverable seven. Draft reports “a” and “b,” as described in deliverable six, are due to the COR within four weeks after the end of the field visit. Draft report “c” is due two weeks after the presentation (deliverable seven) of report “b.” The evaluating findings shall be treated as an independent assessment and opinion of the contractor. USAID, GOP and GOES stakeholders will review the draft evaluation report with the expressed and sole objective of reviewing the factual accuracy of any information contained therein and to indicate areas where further clarification are warranted. The contractor should address these concerns prior to submitting a final report.

The evaluation reports should include but may not be limited to the following elements:

1. Executive Summary 2. Objectives of the evaluation, including evaluation questions 3. Methodology used and limitations of study 4. Results of analysis, assessment of performance against evaluation questions, and to what

extent results can be attributed to the actual interventions 5. Key lessons learned, recommendations, and course corrections for PFG implementation.

6. Final Mid-Term Evaluation Reports

The contractor will provide three Mid-Term Evaluation Reports:

a. A mid-term evaluation report covering the country-specific and cross-cutting findings, lessons learned, recommendations, and course corrections for PFG implementation in El Salvador;

b. A mid-term evaluation report covering the country-specific and cross-cutting findings, lessons learned, recommendations, and course corrections for implementation in the Philippines;

c. And, a report that compares and contrasts the findings, conclusions and recommendations from the cross-cutting questions in El Salvador and the Philippines, to identify commonalities in the PFG process regardless of the location of its implementation.

117

Report “c,” in the list above will be issued after the completion of both the El Salvador and the Philippines fieldwork.

Reports will be due to the COR within 1 week of receiving COR written feedback on the draft evaluation report (see schedule below for total estimated time line). Reports must adhere to the evaluation report requirements outlined USAID’s ADS chapter 203.3.1.8.

The evaluator will provide a Spanish translation of the executive summary for report a. in the list above as well as submit a final report in Spanish.

7. Evaluation Report Presentations:

d. Due within 1 week after the delivery of the accepted final report “a” under deliverable six, the proposer will deliver an in-person presentation in Washington, DC, to Washington-based USG PFG staff and to national and U.S. government personnel in El Salvador, who will participate via video or teleconference.

e. Due within 1 week after the delivery of the accepted final report “b” under deliverable six, the proposer will deliver an in-person presentation in Washington, DC, to Washington-based USG PFG staff and to national and U.S. government personnel in the Philippines, who will participate via video or teleconference.

VI. TEAM COMPOSITION For the life of the contract, the team will contain two permanent staff members, the Evaluation Team Leader and the Data Methods Specialist. For each country, two additional evaluation specialists with sector-specific experience will be added for those portions of the contract. For example, the El Salvador evaluation will require an evaluation specialist with experience in citizen security and an evaluation specialist who has a background in economic development and trade. The Philippines will require an evaluation specialist with a background in economic development, trade and public finance, and another with a background in rule of law and anti-corruption. The Offeror is encouraged to consider the inclusion of country nationals or regional country nationals to the evaluation team. In addition to their core technical specialties, country nationals are instrumental in ground-truthing information analyzed during the evaluation and helping the rest of the team see the larger picture and put things in perspective. The Offeror must verify the availability of any personnel working on the evaluation for more than 60 days. Please include letters of availability for all applicable personnel when submitting the proposal. Submissions not including letters of availability will not be considered for the award. The permanent staff members, as well as the specialists required for the El Salvador portion of the contract, must be able to communicate in Spanish—allowing them to analyze documents in Spanish and to conduct interviews and hold conversations in Spanish.

Evaluation Team Leader a) An advanced degree in Economics, Business Administration, Statistics, Economic

Development, or a related field; b) At least 15 years professional experience in evaluation, including in overseas settings; c) Experience managing teams and working with USG and international governments;

118

d) Proven ability to design and implement quantitative and qualitative research instruments and methodologies;

e) Ability to communicate in Spanish and English.

Data Methods Specialist: a) An advanced degree in social science, statistics or mathematics; b) At least 7 years technical experience with qualitative and quantitative study design,

questionnaire development, data collection, quality control, coding and analysis; c) Ability to design, manage, and implement qualitative and quantitative field-based data

collection for evaluations; d) Proven competency in the use of data management software for evaluation; e) Ability to communicate in Spanish and English.

The following are suggested specialists required for each country: El Salvador Citizen Security Evaluation Specialist

a) An advanced degree in Economics, Business Administration, Statistics, Economic Development, Law, Criminology or a related field;

b) At least 7 years professional experience in evaluation, including in overseas settings; c) At least 5 years of experience in the fields of promoting citizen security, protecting at-risk

youth, and/or crime reduction; d) Proven ability to implement quantitative and qualitative evaluation instruments and

methodologies; e) Ability to communicate in Spanish.

Productivity and Tradables Evaluation Specialist

a) An advanced degree in Economics, Business Administration, Statistics, Economic Development, or a related field;

b) At least 7 years professional experience in evaluation, including in overseas settings; c) At least 5 years of experience in the fields of economic development, trade, and/or business

development; d) Proven ability to implement quantitative and qualitative evaluation instruments and

methodologies; e) Ability to communicate in Spanish.

The Philippines Productivity, Tradables and Public Finance Evaluation Specialist

a) An advanced degree in Economics, Business Administration, Statistics, Economic Development, or a related field;

b) At least 7 years professional experience in evaluation, including in overseas settings; c) At least 7 years of experience in the fields of economic development, trade, and public

finance; d) Proven ability to implement quantitative and qualitative evaluation instruments and

methodologies;

Rule of Law and Anti-Corruption Evaluation Specialist a) A law degree plus an advanced degree in economics, business administration or public

policy b) At least 7 years professional experience in evaluation, including in overseas settings;

119

c) At least 7 years of prior technical experience with a focus on rule of law, anti-corruption enforcement and corruption prevention e) Proven ability to implement quantitative and qualitative evaluation instruments and

methodologies; VII. EVALUATION MANAGEMENT

A) Logistics The various POCs listed above will provide logistical support in terms of providing the team with the necessary U.S. and host-country contacts, contact information and required background information. Other logistics required for the execution of the evaluations will be the responsibility of the contractor. See section IV.d for additional information.

B) Scheduling The contract is expected to begin in September 2013 and run until July 2014. The El Salvador portion of the evaluation will precede the Philippines portion, while the cross-cutting elements will spread across both. The El Salvador field work must take place prior to January 1, 2014. An evaluation schedule follows:

SEE ACCOMPANYING SPREADSHEET

C) Level of Effort The USG has anticipated that the evaluation will require 626 personnel days to complete. An USG estimate of the time LOE by personnel is provided below, but the level and type of staff and their days is at the discretion of the Proposer. TASKS (DAYS) Team Leader SME (1)

RS SME (2) GM

DMS HW

Work Plan –

3 1 8

2 16

3

2.Inception Report/ Preliminary Evaluability Assessment 10 5 40

10 80

12

3.Updated Methodology /Evaluation Plan.

5 2 16

4 32

3

4. Interviews in DC 3 2 16

4 32

3

5. Field Work 18 18 144

36 288

18

6. Field Visit Debrief 2 2 16

4 32

2

7. Draft Evaluation Report

20 15 120

30 240

20

8. Final mid term Evaluation Report

5 2 16

4 32

5

9. Cross Cutting report 5 2 16

4 32

3

10. Evaluation Report Presentation 2 4 32

4 32

2

120

Per Country Totals 73 53 102 71

Additional Days for the Philippines

2 2 4 2

Additional Cross-Country Report 10 0 0 10

TOTAL LOE 158 108 208 152

SME = Subject Matter Expert DMS = Data Methods Specialist

121

Annex 2: Work Plan

I. Mid-term Performance Evaluation of Partnership for Growth (PFG) Project II. PHILIPPINES

No. Deliverable Due Date No. of Days 1 Work Plan December 31, 2013 Completed PFG Review December 31 - January 2, 2014

LEAP Team revises with PFG comments January 2 - 6, 2014 PFG Review Final Version and Approves January 6 - 10, 2014 LEAP Team Submit Final January 15. 2014 2 Inception Report May 30, 2014 Completed

PFG Review June 2 - 5, 2014 4 days LEAP Team revises with PFG comments June 6 - 11, 2014 4 days PFG Reviews Final Version and Approves June 12 – 16, 2014 3 days LEAP Team Submits Final June 17 – June19, 2014 3 days 3 Evaluability Assessment June 23, 2014 Completed PFG Reviews June 24 - 26, 2014 3 days

LEAP Team revises with PFG comments June 27 – July 3, 2014 5 days

PFG Review and approval July 7 - 10 2014 4 days

4 Updated Methodology and Evaluation Plan June 13, 2014 Completed

Submit interview protocols June 13 - 20, 2014

PFG Review June 20 - 25, 2014 5 days PFG revises and submits report June 25 - 27, 2014 2 days

Field Visit June 30, 2014 Completed

Philippines Field Work June 30 – July 15, 2014 17 days In Brief with USG PFG Team July 1, 2014 5 End of Field Visit Debrief July 14, 2014 Completed

6 Evaluation Report

Completed PFG Review August 19 – September 22, 2014

LEAP team revises with PFG comments September 24 – October 14, 2014 14 days PFG reviews final version and approves October 15 – 24, 2014 8 days LEAP team submits final version October 27 – November 3, 2014 6 days

122

Annex 3: Advantages and Disadvantages for Data Collection Methods for this Evaluation

The tables below provide brief descriptions of the pros and cons of different data collection types (e.g. survey, interviews), tool structure (e.g. semi-structured) and method of delivery (e.g. email). Advantages Disadvantages Type

Survey • Standardization; • Easy to do with a large group; • Ease of administration; • Suitability to tabulation and statistical analysis; • Sample can be used to provide much information

about a population; • Can be sensitivity to subgroup differences; • Can be used to record behaviors as well as opinions,

attitudes, beliefs and attributes; • Can be combined with other methods, i.e.,

observation or case study.

• May requires a separate data-entry step; • May require a data cleaning step; • Can be more time-consuming compared with less

formal methods; • Respondents might misinterpret questions, depending

on how questions are designed and asked; • Tendency for scope of data to be limited (no follow-

up questions); • Samples must be carefully selected to ensure

statistical meaning.

Individual Interview

• Allows for clarification; • Able to gather in-depth information and to pursue

hunches; • Can tailor the line of discussion to the individual; • Easier to reach those who are considered

unreachable (the poor, homeless, high status, mobile, etc.);

• May be easier to reach specific individuals (i.e., community leaders, etc.);

• More personalized approach; • Easier to ask open-ended questions, use probes and

pick up on nonverbal cues; • Longer interviews are sometimes tolerated.

Particularly with in-home interviews that have been arranged in advance, people may be willing to talk longer face-to-face than to someone on the phone.

• Reactive effect: interviewer’s presence and characteristics may bias results;

• Cost more per interview than other research methods. This is particularly true of in-home interviews, where travel time is a major factor;

• Requires strong interviewing skills; • Slowest method of data collection and analysis; • Responses may be less honest and thoughtful; • Interviewer should go to location of respondent; • Respondents who prefer anonymity may be inhibited

by personal approach; • May reach only a smaller sample; • Difficult to analyze and quantify results.

Group Interview

• Can increase the sample size substantially; • Provide some quality controls on data collection can

focus on the most important topics and issues in the program;

• Can assess the extent to which there is relatively consistent, shared views among participants.

• Less questions can be asked than in an individual interview;

• Require considerable group process skill; • Conflicts may arise; • Status differences may become a factor; • More difficult to guarantee confidentiality.

Observation • Setting is natural; • Can generates relevant, quantifiable data; • Most useful for studying a “small unit” such as a

classroom, Extension Council, etc.

• Requires skilled data collector; • The evaluator has less control over the situation in a

natural environment; • Hawthorne effect (awareness of being observed

might affect behavior); • If observer chooses to be involved in the activity,

he/she may lose objectivity • Not realistic for use with large groups

Archival (data

already collected by

agency)

• Low cost; • Unobtrusive; • Can be highly accurate; • Often good to moderate validity;

• May be difficult to access local data; • Often out of date; • When rules for record-keeping are changed, makes

trend analysis difficult or invalid; • Need to learn how records were compiled to assess

123

Advantages Disadvantages • Usually allows for historical comparisons/trend

analysis; • Often allows for comparisons with larger

populations.

validity; • May not be data on knowledge, attitudes, and

opinions; • May not provide a complete picture.

Advantages Disadvantages Structure

Structured • Emphasizes reliability (how accurately different respondents’ answers can be compared);

• Can reach a large sample; • A representative sample is possible and results can

be used to make statements; • Questions are structured and asked in the same way

so that respondents’ answers can be more easily compared.

• Respondents are forced to choose between the alternatives provided by the interviewer;

• It can be difficult to obtain reliable data on opinions, attitudes and values;

• Interviewer has to stick to the agreed questions even though interesting lines of enquiry might emerge in an interview;

• More time-consuming than postal or online questionnaires.

Semi-structured

• Provides valuable information from context of participants (and stakeholder) experiences;

• Use of pre-determined questions provides uniformity.

• Can be time consuming to collect and analyze data; • Requires some level of training or practice in order to

prevent interviewer suggesting answers.

Unstructured • Respondents may be more likely to discuss sensitive and painful experiences if they feel the interviewer is sympathetic and understanding;

• If respondent feels at ease with the interviewer, they might open up and be very honest;

• They give respondents time and opportunities to develop their answers;

• Gives the interviewer more chance to pursue a topic, to explore with any further questions, and ask the respondent to qualify and develop their answers.

• Interviewer bias is unavoidable. To some extent the interviewer will affect the responses of the interviewee;

• Can develop in all sorts of directions. This makes comparison between data from different interviews difficult;

• Respondents are likely to present themselves in a favorable light, exaggerating aspects of their behavior which they see as socially desirable and minimizing aspects seen as undesirable;.

• Unstructured interviews can take up a great amount of time and cost for the interviews to take place.

Advantages Disadvantages Delivery

Email • Can gather responses within a short timeframe; • Economy of scale (large samples do not cost much

more than smaller ones, except for any cost associated with acquiring the sample);

• Files with pictures, videos and audio can be attached;

• It allows the respondent to answer at their leisure. For this reason, they are not considered intrusive;

• Records can be easily compiled into a database.

• List of emails must be already owned or purchased; • Respondents might pass questions along to someone

else to answer. Limited capacity to enforce recruitment criteria;

• Respondent might dislike unsolicited email even more than unsolicited regular mail;

• Email surveys may not be used to generalize findings to whole populations as people who have email are different from those who do not;

• Email surveys cannot automatically skip questions or randomize question or answer choice order or use other automatic techniques that can enhance surveys the way on-line web-based surveys can;

• Cannot assess body language or tone of voice; • May require a data cleaning step.

Internet-based

• Can gather responses within a short timeframe; • Economy of scale (large samples do not cost much

more than smaller ones, except for any cost associated with acquiring the sample);

• Current use of the Internet is growing but far from universal. Internet surveys may not reflect the population as a whole;

• Respondents may quit a long internet-based survey on the middle more easily than they would if talking to a

124

• Pictures, videos and audio files can be included; • Can use complex question skipping logic,

randomizations and other features not possible with paper questionnaires or most email surveys;

• Respondents might be more honest answering to questions about sensitive topics when giving answers to a computer instead of a person or writing them down on paper;

• Complete record of session instantly available.

good interviewer; • Respond may respond to an email invitation to take a

Web-based survey and then not answer the survey; • Respondents might pass questions along to someone

else to answer. Limited capacity to enforce recruitment criteria;

• Cannot assess body language or tone of voice.

Postal • Can include pictures; • Allows the respondent to answer at their leisure.

For this reason, they are not considered intrusive.

• Requires the name and address of the respondents; • Long duration of time. It might take several weeks

after mailing out of questionnaires before a minimum numbers of responses are returned.

• Respondents might pass questions along to someone else to answer. Limited capacity to enforce recruitment criteria;

• Cannot assess body language or tone of voice; • May requires a separate data-entry step; • May require a data cleaning step.

Telephone • Can be administered with respondent in hard to reach areas with limited internet access;

• Relative anonymity may result in frank discussion of sensitive issues.

• Cannot assess non-verbal reactions; • More difficult to get reactions to visuals; • Noise interference from callers’ environments might

distract respondents; • May need to train data collector • May requires a separate data-entry step; • May require a data cleaning step.

Face-to-face • Can include pictures, videos and audio; • Can assess body language; • Can use complex question skipping logic; • Have participants’ undivided attention.

• Responders lose anonymity; • Logistical challenges and travel expenses; • May need to train data collector; • May requires a separate data-entry step; • May require a data cleaning step.

125

Annex 4: Updated Methodology and Evaluation Plan Report

June 13, 2014

Updated Methodology and Evaluation Plan


MID-TERM EVALUATION

CROSS-CUTTING AND THE PHILIPPINES

Optimal Solutions Group, LLC

Christabel Dadzie

Peter Murrell

Tiernan Mennen

126

Table of Tables

Table 1: Constraint-level Indicators, Constraint 1 .................................................................................... 130 Table 2: Constraint-level Indicators, Constraint 2 .................................................................................... 136 Table 3: Constraint-level Indicators, Constraint 3 .................................................................................... 139 Table 4: Semi-structured Interview Guides by Research Question .......................................................... 144 Table 5: Online Survey Items by Research Question ............................................................................... 144 Table 6: Cross-cutting Questions .............................................................................................................. 148 Table 7: The Philippines Country-Specific Questions .............................................................................. 149 Table 8: Foreign direct investment, net inflows (% of GDP) ................................................................... 151 Table 9: Gross capital formation (% of GDP) .......................................................................................... 151 Table 10: Global Competitiveness Index, Rank ....................................................................................... 151 Table 11: Ease of Doing Business Rank ................................................................................................... 151 Table 12: Starting a Business, Rank ......................................................................................................... 152 Table 13: Government Effectiveness, Percentile Rank............................................................................. 152 Table 14: Rule of Law, Percentile Rank ................................................................................................... 152 Table 15: Control of Corruption, Percentile Rank .................................................................................... 153 Table 16: Regulatory Quality, Percentile Rank ........................................................................................ 153 Table 17: Tax revenue (% of GDP) .......................................................................................................... 153 Table 18: General government net lending/borrowing ............................................................................. 153 Table 19: Total investment ....................................................................................................................... 154

Table of Figures

Figure 1: Foreign direct investment, net inflows (% of GDP) .................................................................. 132 Figure 2: Gross capital formation (% of GDP) ......................................................................................... 132 Figure 3: Global Competitiveness Index, Rank ........................................................................................ 133 Figure 4: Ease of Doing Business Rank .................................................................................................... 134 Figure 5: Starting a Business, Rank .......................................................................................................... 135 Figure 6: Government Effectiveness, Percentile Rank ............................................................................. 136 Figure 7: Rule of Law, Percentile Rank .................................................................................................... 137 Figure 8: Control of Corruption, Percentile Rank ..................................................................................... 138 Figure 9: Regulatory Quality, Percentile Rank ......................................................................................... 139 Figure 10: Tax revenue (% of GDP) ......................................................................................................... 140 Figure 11: General government net lending/borrowing (% of GDP)........................................................ 141 Figure 12: Total investment (% of GDP) .................................................................................................. 141

127

List of Acronyms

CA Constraints Analysis

DOJ U.S. Department of Justice

GDP Gross Domestic Product

GPH Government of the Philippines

FDI Foreign Direct Investment

FBI Federal Bureau of Investigation

INL Bureau of International Narcotics and Law Enforcement Affairs

IUDOP Instituto Universitario de Opinión Pública at the Universidad Centroamericana José Simeón Cañas

JSSP Justice Sector Strengthening Project

JCAP Joint Country Action Plan

LAPOP Latin America Public Opinion Polls

LOAs Lines of Action

M&E Monitoring and Evaluation

MCC Millennium Challenge Corporation

MIJSP Ministerio de Justicia y Seguridad Pública

NGOs Non-Governmental organizations

PAS GOES Public Affairs

PFG Partnership for Growth

PNC National Police

SMEs Subject Matter Experts

SOW Statement of Work

USG United States Government

USAID United States Agency for International Development

WGA Whole-of-Government Approach

128

A. Project Purpose

i. Partnership for Growth

The Partnership for Growth (PFG) aims to achieve accelerated, sustained, broad-based economic growth in partner countries, including El Salvador and the Philippines, through bilateral agreements between the United States Government (USG) and the partnering countries’ national governments. Using principles set forth in President Obama’s September 2010 Presidential Policy Directive on Global Development, the PFG requires rigorous, joint analyses of countries’ individual constraints to growth in order to develop joint action plans to address the most pressing of these constraints and to establish high-level mutual accountability for the goals and activities selected to alleviate them.

ii. PFG Mid-Term Evaluation

The PFG mid-term evaluation seeks answers to two sets of questions; the first set includes cross-cutting questions whereas the second set addresses questions that are country specific. As summarized in the Statement of Work, the objective of the first set is:

[to assess] whether the PFG process demonstrates improvements over pre-PFG assistance approaches. In particular, the evaluation will examine the extent to which the PFG’s whole-of-government and constraints analysis approach led to a change in the manner of USG delivery of development assistance and whether these changes demonstrated improvements in terms of operational efficiency, selection, coordination, design and management of development interventions, and ultimately increased the probability for success and effectiveness of assistance efforts in achieving verifiable results. The findings and conclusions of this part of the mid-term evaluation will help decision makers determine whether PFG indicates an improved model for providing assistance and whether it portends a higher probability of achieving desired development results. Furthermore, it will inform governments in their work with all donors.42

Three specific questions are posed by the USG to gauge the merits of the PFG approach.

1. What are the advantages and/or disadvantages of the PFG whole of government approach to development assistance? The intent of this question is to assess the extent to which the PFG efforts intended changes in development assistance have or have not materialized. The whole of government approach is relevant to identifying areas for assistance, selecting interventions, and determining implementation coordination. The question is relevant both to national government agencies and institutions, and U.S. government agencies and institutions overseas and in Washington DC.

2. To what extent has Partnership for Growth affected the workload on national government and U.S. government staff, as compared to the workload created by traditional forms of development assistance delivery?

42 Statement of Work, p. 7.

129

3. What contributions has “non-assistance”43 made to the PFG process and how can it be utilized moving forward?

According to the SOW, the second set of questions is country specific and its objectives are to:

“[E]valuate whether PFG efforts have been developed in such a way as to allow for the eventual determination of their impact on addressing the identified constraints and desired outcomes; and, 2) to evaluate the performance of certain initiatives to date to determine whether or not they are moving in the right direction, are considered necessary and sufficient to achieve PFG goals, and are contributing to national interests through the integration and coordination of work done by both governments.”

1. For each of the constraints, are the goal-level commitments set forth in the Joint Country Action Plan (JCAP) capable of achieving the constraint-level objectives and outcomes?

2. Is quantitative and objectively verifiable information being used to manage JCAP implementation in order to achieve and measure results?


iii. The Updated Methodology and Evaluation Plan for PFG the Philippines

This updated methodology and evaluation plan (UMEP) first describes the baseline characteristics of Philippines’s most severe constraints to development. As part of the evaluation, the team examined the situation at baseline, which highlights why the Philippines requires development assistance, describes the scale of the Philippines’ issues, and underscores how challenging it can be to achieve macro-level, measurable results over the course of only two years. This initial exploration served as a situational analysis with the following purposes:

• The exploratory analysis guided the team in gaining an understanding of the Philippines PFG initiative at the constraints level.

• This study was useful, in combination with project documents received, in developing the data collection tools.

During the evaluation, with this information as a necessary first step, the team will delve deeper to analyze the goal–level and activity-level situation of the Philippines’ PFG initiative. In the final report, macro-level data will be updated44 and compared to similar data for other Southeast Asian countries, thus providing points of comparison. The reader should be cautioned, though, that due to the specificity of PFG projects in the Philippines, movements in macro variables over such a short period of time should be interpreted as impressionistic only.

43 Non-assistance tools include diplomatic engagement, convening authority, and other forms of non-monetized assistance to engage both governmental and non-governmental stakeholders in support of catalytic policy change and development priorities. 44 To the extent that updates are available.

130

The subsequent sections of the UMEP describe the data collection methodologies, namely the interview guides and online survey. Finally, a description of the data needs of the pre-field visit is provided.

As part of the mid-term evaluation, the SOW requested that goals be selected for an in-depth study. Selected goals were chosen to best represent the themes of the PFG, stressing constraint/sub-constraint subject matter and multi-agency cooperation. They were also chosen to reflect the diversity of partnering agencies and implementing partners and the extent to which their activities represent new initiatives formed within the PFG. Please refer to the Mid-Term Evaluation’s Evaluability Assessment and Goal Selection Report for details on how goals will be selected.

B. Constraints to the Philippines’s Development at Baseline

In the spring of 2011, a team of economists representing the Philippines and the United States completed a constraints analysis (CA) following the methods of Hausman, Rodrik and Velaso45. The analysis identified five candidate constraints to growth: namely, prioritization, fiscal space, infrastructure, human capital and governance. However, the CA did not explicitly set out a path toward slackening the constraints. Based on the CA, three inter-related themes of development intervention (sub-constraints) were selected—improving regulatory quality, strengthening the rule of law and anti-corruption measures, and improving fiscal performance—and associated goals for each were laid out in the countries’ JCAP, published in November of that year. For each of the three sub-constraints, the Amplified Monitoring and Evaluation Plan to the JCAP identified indicators that capture the major constraints to development. Whenever possible, third party data from 2010-2013 was obtained for each of these indicators. Given the status of the Philippines as a member of the Association of Southeast Asian Nations (ASEAN), the remaining countries from ASEAN—Brunei, Cambodia, Indonesia, Laos, Malaysia, Myanmar, Singapore, Thailand, Vietnam—were used for comparison.

i. Constraint 1 “Regulatory Quality”: Domestic Investment Climate

A total of nine anticipated outcomes were listed in the Amplified M&E Plan for Constraint 1. All these outcomes are related to enhancing the domestic investment climate in the Philippines. Table 1 below summarizes the indicators and data sources used to measure each of these outcomes. It should be noticed that Optimal has yet to receive data related to five of these outcomes (four of them from the MCC scorecards). Table 13: Constraint-level Indicators, Constraint 1 Anticipated Outcome (from "Amplified M&E Plan") Source Suggested Item Status

Share of foreign direct investment (FDI) to GDP increased

World Bank: World Development Indicators

Foreign direct investment, net inflows (% of GDP)

Received

Gross capital formation increased World Bank: Gross capital formation (% of GDP) Received

45 Hausmann, Ricardo, Rodrik, Gani and Velasco, Andres. Growth Diagnostics, in J. Stiglitz and N.Serra, eds., The Washington Consensus Reconsidered: Towards a New Global Governance, Oxford University Press, New York, 2008. (Originally circulated as a working paper in 2005).

131

Anticipated Outcome (from "Amplified M&E Plan") Source Suggested Item Status

World Development Indicators

Improvement in MCC Scorecard Indicator on Regulatory Quality: An index of surveys and expert assessments that rate countries on: the burden of regulations on business; price controls; the government’s role in the economy; and foreign investment regulation, among other areas. Source: Worldwide Governance Indicators (World Bank)

Multiple Note: index combining up to 14 different assessments and surveys, depending on availability, each of which receives a different weight, depending on its estimated precision and country coverage

Not yet received

Global Competitiveness Index: increasing or on par with leading regional and emerging market peers

World Economic Forum: Global Competitiveness

Global Competitiveness Index Received

Doing Business and Related Indicators: improving or on par with regional and emerging market peers

World Bank: Doing Business, and Worldwide Governance Indicators

Ease of Doing Business Rank (DoingBusiness) Starting a Business score (DoingBusiness) Government Effectiveness rank (WGI) Government Effectiveness estimate (WGI)

Received

Improvement in MCC Scorecard Indicator on Business Start-up: An index that rates countries on the time and cost of complying with all procedures officially required for an entrepreneur to start up and formally operate an industrial or commercial business. Source: International Finance Corporation

World Bank: Doing Business

Composite indicator calculated as the average of two indicators: Days to Start a Business, and Cost of Starting a Business

Not yet received

Business Start-ups: increased in key sectors (agribusiness, business process outsourcing, creative industries, infrastructure, manufacturing & logistics, mining, and tourism)

Host country or project M&E data

Unkown Not yet received

Improvement in MCC Scorecard Indicator on Government Effectiveness: An index of surveys and expert assessments that rate countries on: the quality of public service provision; civil servants’ competency and independence from political pressures; and the government’s ability to plan and implement sound policies, among other things. Source: Worldwide Governance Indicators (World Bank/Brookings)

Multiple Note: Index combining up to 15 different assessments and surveys, depending on availability, each of which receives a different weight, depending on its estimated precision and country coverage

Not yet received

Improvement in MCC Scorecard Indicator on Land Rights and Access: An index that rates countries on the extent to which the institutional, legal, and market framework provide secure land tenure and equitable access to land in rural areas and the time and cost of property registration in urban and peri-urban areas. Source: The International Fund for Agricultural Development and the International Finance Corporation

World Bank: Doing Business, and IFAD

Composite indicator calculated as the weighted average of three indicators: Access to Land (IFAD), Days to Register Property (DoingBusiness), Cost of Registering Property (DoingBusiness)

Not yet received

Among the ASEAN countries, the Philippines was the nation with the lowest net inflow of FDI as percentage of GDP, and this pattern was constant for the entire 2010-2013 period. Almost no change was detected over time, with net inflows to the Philippines ranging from 0.8 percent of the GDP in 2010 to 1.1 percent in 2012.

132

Figure 3: Foreign direct investment, net inflows (% of GDP)

Source: World Bank, World Development Indicators. Supporting Table 8 in Annex 1 The Philippines’ gross capital formation as percentage of GDP during the same period was only higher than two other ASEAN countries (Brunei and Cambodia). Over time, a small negative fluctuation was identified, with gross capital formation for the Philippines decreasing from 21 percent of GDP in 2010 to 18 percent in 2012. Figure 4: Gross capital formation (% of GDP)

Source: World Bank, World Development Indicators. Supporting Table 9 in Annex 1 Out of 148 countries ranked in terms of the competitive landscape of their economy, the Philippines improved from 85th position in 2010 to 59th in 2013. Furthermore, the Filipino economy was ranked as more competitive than the economy of the other four ASEAN countries (Cambodia, Laos, Myanmar and Vietnam).

0.8

5.1 7.0

1.9 3.9 4.4

23.1

2.9 6.9

0.8

7.4 7.0

2.3 3.6 5.2

21.1

2.6

5.5

1.1

5.0

11.1

2.2 3.1 3.2

20.5

2.9 5.4

0

5

10

15

20

25

2010 2011 2012

21 16 17

32

24 23 21

26

36

20

13 17

33

26 23 22

27 30

18 14

36 32

26 27 30

27

05

10152025303540

2010 2011 2012

133

Figure 5: Global Competitiveness Index, Rank

Source: World Economic Forum, Global Competitiveness. Supporting Table 10 in Annex 1 Out of 189 countries ranked in terms of the ease of doing business in the country, the Philippines improved from 79th position in 2012 to 59th in 2013. Furthermore, the Filipino economy was ranked as more competitive than the economy of other six ASEAN countries (Brunei, Cambodia, Indonesia, Malaysia, Myanmar and Vietnam).

85 75

65 59

28 28 28 26

109 97

85 88

44 46 50 38

81

26 21 25 24

139

3 2 2 2

38 39 38 37

59 65 75 70

0

20

40

60

80

100

120

140

2010 2011 2012 2013

Philippines Brunei DarussalamCambodia IndonesiaLao PDR MalaysiaMyanmar SingaporeThailand Vietnam

134

Figure 6: Ease of Doing Business Rank

Source: World Bank, Doing Business. Supporting Table 11 in Annex 1 However, no change was observed for the Philippines rank for ease of starting a business. Out of the 189 countries ranked, the Philippines fluctuated from 136th position in 2012 to 137th in 2013. Philippines’ rank was superior to that of four ASEAN countries (Brunei, Cambodia, Malaysia and Myanmar).

79

59

135 137

116 120

163 159

8 6

182 182

133

108

1 1

18 18

98 99

0

20

40

60

80

100

120

140

160

180

200

2012 2013Philippines Brunei DarussalamCambodia IndonesiaLao PDR MalaysiaMyanmar SingaporeThailand Vietnam

135

Figure 7: Starting a Business, Rank

Source: World Bank, Doing Business. Supporting Table 12 in Annex 1 The Philippines showed a slight improvement in the government effectiveness ranking in the same period, from outperforming 56 percent of the countries measured in 2010 to outperforming 58 percent of the countries measured in 2012. Among the ASEAN countries, it outperformed five countries (Cambodia, Indonesia, Laos, Myanmar and Vietnam).

136 137

181 184 171 175

82 85

19 16

189 189

166 170

3 3

86 91

107 109

0

20

40

60

80

100

120

140

160

180

200

2012 2013Philippines Brunei DarussalamCambodia IndonesiaLao PDR MalaysiaMyanmar SingaporeThailand Vietnam

136

Figure 8: Government Effectiveness, Percentile Rank

Source: World Bank, Worldwide Governance Indicators. Supporting Table 13 in Annex 1

ii. Constraint 2 “Rule of Law and Anti-Corruption Measures”: Governance

A total of four anticipated outcomes were listed in the Amplified M&E Plan for Constraint 2. All these outcomes are related to improving governance in the Philippines. Table 2 below summarizes the indicators and data sources used to measure each of these outcomes. It should be noticed that Optimal has yet to receive data related to three of these outcomes (two of them from the MCC scorecards). Table 14: Constraint-level Indicators, Constraint 2 Anticipated Outcome (from "Amplified M&E Plan") Source Suggested Item Status

Improvement in MCC Scorecard Indicator on Control of Corruption: An index of surveys and expert assessments that rate countries on: “grand corruption” in the political arena; the frequency of petty corruption; the effects of corruption on the business environment; and the tendency of elites to engage in “state capture”, among other things. Source: Worldwide Governance Indicators (World Bank/Brookings)

Multiple Note: Index combining up to 21 different assessments and surveys, depending on availability, each of which receives a different weight, depending on its estimated precision and country coverage

Not yet received

World Governance Indicators (Regulatory Quality, Rule of Law, Control of Corruption): improving or on par with regional and emerging market peers

World Bank: Worldwide Governance Indicators

Rule of Law rank Control of Corruption rank Regulatory Quality rank

Received

Improvement in MCC Scorecard Indicator on Rule of Law: An index of surveys and expert assessments that rate countries on: the extent to

Multiple Note: Index combining up to 22 different assessments and surveys, depending on

Not yet received

56 57 58

78 77 75

19 20 22

48 46

44

20 20 21

83 81 80

2 3 4

100 100 100

62 61 61

46 47

44

0

20

40

60

80

100

2010 2011 2012


137

Anticipated Outcome (from "Amplified M&E Plan") Source Suggested Item Status

which the public has confidence in and abides by the rules of society; the incidence and impact of violent and nonviolent crime; the effectiveness, independence, and predictability of the judiciary; the protection of property rights; and the enforceability of contracts, among other things. Source: Worldwide Governance Indicators (World Bank/Brookings)

availability, each of which receives a different weight, depending on its estimated precision and country coverage

Court Congestion: eased through docket cleansing, case track management, and capacity building of commercial, tax, and anti-graft courts

Host country or project M&E data

Unknown Not yet received

The Philippines showed a slight improvement in the rule of law ranking, from outperforming 34 percent of the countries measured in 2010 to outperforming 36 percent of those measured in 2012. Among the ASEAN countries, it outperformed four countries (Cambodia, Indonesia, Laos and Myanmar). Figure 9: Rule of Law, Percentile Rank

Source: World Bank, Worldwide Governance Indicators. Supporting Table 14 in Annex 1 Although a low performer overall, the Philippines showed a greater improvement in the control of corruption ranking, from outperforming 22 percent of the countries measured in 2010 to outperforming 33 percent of the countries measured in 2012. Among the ASEAN countries, it outperformed four countries (Cambodia, Indonesia, Laos and Myanmar).

34 36 36

74 75 73

13 15 17

32 32 34 19 18 23

66 65 66

3 4 6

93 93 96

49 49 50

35 39 38

0

20

40

60

80

100

2010 2011 2012


138

Figure 10: Control of Corruption, Percentile Rank

Source: World Bank, Worldwide Governance Indicators. Supporting Table 15 in Annex 1 A better performer in terms of regulatory quality, the Philippines also showed a greater improvement for this indicator, from outperforming 45 percent of the countries measured in 2010 to outperforming 52 percent of those measured in 2012. Among the ASEAN countries, it outperformed five countries (Cambodia, Indonesia, Laos, Myanmar and Vietnam).

22 26 33

80 78 72

7 7

14 25 27 29

8 8 15

63 60 66

0 0

11

99 97 97

48 50 47

31 31 35

0

20

40

60

80

100

2010 2011 2012


139

Figure 11: Regulatory Quality, Percentile Rank

Source: World Bank, Worldwide Governance Indicators. Supporting Table 16 in Annex 1

iii. Constraint 3 “Fiscal Performance”: Fiscal Space

A total of three anticipated outcomes were listed in the Amplified M&E Plan for Constraint 3. All these outcomes are related to ameliorating the fiscal space in the Philippines. Table 3 below summarizes the indicator and data source used to measure each of these outcomes. Because the MCC indicator for Fiscal Policy relies only on one variable and this variable can be found in the IMF’s World Economic Outlook database, Optimal was able to present information for this indicator. Table 15: Constraint-level Indicators, Constraint 3 Anticipated Outcome (from "Amplified M&E Plan") Source Suggested Item Status Tax effort improved World Bank: World

Development Indicators Tax revenue (% of GDP)

Received

Public spending on priority areas, particularly in infrastructure and education

International Monetary Fund, World Economic Outlook

Total investment as percent of GDP

Received

Improvement in MCC Scorecard Indicator on Fiscal Policy: The overall budget balance divided by GDP, averaged over a three-year period. The data for this measure come primarily from IMF country reports or, where public IMF data are outdated or unavailable, are provided directly by the recipient government with input from U.S. missions in host countries. All data are cross-checked with the IMF’s World Economic Outlook database to try to ensure consistency across countries and made publicly available. Source: International Monetary Fund Country Reports, National Governments, and the International Monetary Fund’s World Economic Outlook Database

International Monetary Fund, World Economic Outlook

General government net lending/borrowing (% of GDP)

Received

45 45 52

83 85 85

36 32 39 38 41 43

17 18 22

71 71 70

1 1 2

98 96 100

56 56 58

28 31 27

0

20

40

60

80

100

2010 2011 2012Philippines Brunei DarussalamCambodia IndonesiaLao PDR MalaysiaMyanmar SingaporeThailand Vietnam

140

The Philippines’ tax revenue as percentage of GDP was similar to that of other ASEAN countries, such as Cambodia and Singapore. During 2010-2012, the tax revenue as percentage of GDP fluctuated only slightly, from 12 percent of the GDP in 2010 to 13 percent in 2012.

Figure 12: Tax revenue (% of GDP)

Source: World Bank, World Development Indicators. Supporting Table 17 in Annex 1 Similar to most other ASEAN countries—with the exception of Brunei and Singapore—the Philippines faced a negative balance of general government lending/borrowing. However, it is interesting to note that this negative balance decreased from 2.4 percent of the GDP in 2010 to almost 0 percent in 2013.

12 10

13 14 13

16

12 10

14 15

14

18

13 12

15 16

14 17

0

4

8

12

16

20

2010 2011 2012

141

Figure 13: General government net lending/borrowing (% of GDP)

Source: International Monetary Fund, World Economic Outlook. Supporting Table 18 in Annex 1

The Philippines’ total domestic investment as percentage of GDP remained almost constant, fluctuating from 21 percent of the GDP in 2010 to 19 percent in 2013. Among the ASEAN countries for which 2013 data was available (Indonesia, Malaysia, Singapore, Thailand and Vietnam), the Philippines was the country with the lowest total investment as percentage of GDP. Figure 14: Total investment (% of GDP)

Source: International Monetary Fund, World Economic Outlook. Supporting Table 19 in Annex 1

-2.4

8.4

-2.8

-1.2

-4.7

-4.7

-5.4

7.3

-0.8

-2.8

-0.4

28.1

-4.1

-0.6

-3.0

-3.8

-4.6

9.3

-0.6

-1.1

-0.7

16.9

-3.8

-1.7

-1.4

-3.6

-3.8

8.7

-1.8

-4.8

-0.1

16.6

-2.1

-4.7

-0.2

-5.7

-10 -5 0 5 10 15 20 25 30

Philippines

Brunei Darussalam

Cambodia

Indonesia

Lao P.D.R.

Malaysia

Myanmar

Singapore

Thailand

Vietnam

2010 2011 2012 2013

21 16 17

32

23

16

23 26

36

20

13

22

33

23

15

24 27

30

18 14

24

35

26 28 30

27

19

34

26 26 29

27

05

10152025303540

2010 2011 2012 2013

142

iv. Analysis of Data Collected with Semi-structured Interviews

Semi-structured Interviews were designed to last approximately one hour, with the understanding that some respondents will not be able to dedicate that much time, while others might give a little more time. In the former instance, the interviews will be abbreviated and only central themes will be explored. For example, the PFG Leadership Guide focuses on four central themes on the WGA. These major themes will be explored even if time is limited, but “deep dives” into subjects such as how the PFG has changed the development of initiatives relative to previous assistance and development protocols will be omitted unless the respondent pursues such themes spontaneously. In all instances, but particularly for goal leads and implementers, the guides will be tailored to the experiences and subject matter knowledge of individual respondents.

Interviews with independent experts will be conducted over both periods, subject to their availability. Should any leadership or architect stakeholders not be available during the first phase; the team will prioritize interviewing them during the second round.

Semi-structured interview guides were developed for each of the six classes of respondents described below. Copies of the interview guides are contained in Annex 2. The guides have been pre-tested, and the final versions are included in this report, per the Statement of Work requirement.

1. (a) High-level PFG Leadership: Targeted respondents includes current and former GPH ministers, ambassadors, agency administrators and similarly positioned leadership who provide, or provided, the highest level of direction and oversight for the PFG. For the USG, this includes high-ranking officials in both the Philippines and the United States. This guide is similar to the PFG Leadership guide except it is designed to be shorter in recognition of the limited time these respondents can dedicate to interviews and the fact that they are less likely to be familiar with the details of the day-to-day administration of the PFG. (b) PFG Leadership: The current and former USG and GPH officials who have held or hold leadership positions within the PFG initiative, as well as PFG points of contacts. For the USG, this includes officials both in the Philippines and in the United States. The Leadership guides includes (i) cross-cutting questions about the PFG WGA, changes in operational efficiency and workload, and non-assistance (ii) the Philippines -specific questions about the remedial capacities of the JCAP, M&E issues, and the mid-term performance of selected goals, as related to the desired outcomes.

2. PFG Architects: The targeted respondents for this guide are individuals directly involved in the design and planning of the Philippines PFG or whose input was sought for these processes. Respondents will mainly include USG and GPH officials (current and former), members of the Growth Council, representatives from private sector and other civil society organizations, and independent experts.

143

The interview questions focus on the overall PFG objectives and the role that WGA, constraint analysis (CA), JCAP, non-assistance, M&E, and yearly performance reviews play in the PFG performance and were considerations in its design.

3. PFG Goal Leads: The targeted respondents for this guide are active and former GPH and USG goal leads who have directly worked on respective goal(s). Interviews will focus on goal-level achievement, and its contribution to reaching constraint level objectives, M&E, and whether interventions are on-target at mid-term. This guide includes cross-cutting questions to gauge changes in the operational efficiency, selection, coordination, design, and management of development interventions under the PFG strategy as compared to previous/other approaches. The guide also includes country-specific questions that probe the selected goals, corresponding projects, and performance at mid-term.

4. PFG Project Implementers: The targeted respondents for this guide are the implementers of all or selected activities under the selected goals. The interviews will be performed primarily with the chiefs of party, directors, coordinators or their representatives, and goal leads, in circumstances where the goal leads are also implementers. The guide focuses on questions regarding the performance of PFG, the monitoring of activities, evidence-based decision making, non-assistance, and beneficiaries. Given the detailed questioning in this interview, the chiefs of party will most likely be joined by team members or part of the interview will be completed with team members only. Subject to respondent availability, these interviews will take up to 1.5 hours each.

5. Independent Experts: Respondents for are independent experts. Experts include academics, subject-matter experts, journalists, and others who contribute to public debate on the PFG in general or specific areas of the PFG but who are not responsible for directing or implementing components of the PFG. The guide includes cross-cutting questions (to gauge changes in the operational efficiency, selection, coordination, design, and management of development interventions under the PFG strategy as compared to previous/other approaches) as well as goal- and project-level questions (the latter, in particular, will be contextualized by the SME interviewer for the specific area of expertise of the interviewee at hand).

All questions will be asked to the interviewee; however, the interviewer understands that not all questions will be applicable to the interviewee depending on his or her knowledge and length of time working on PFG; therefore, all core questions will be asked, but sub-questions may be skipped, if time is limited. Ensuring that all core questions are covered maximizes comparability across interviews.46

46 It also reduces the potential for the introduction of bias into core question responses due to unknown but observable differences between respondents who can quickly provide in-depth responses to earlier questions and those who cannot.

144

Table 16: Semi-structured Interview Guides by Research Question

Interview Guide CCQ1 CCQ2 CCQ3 CSQ1 CSQ2 CSQ3

High-level PFG Leadership

PFG Leadership

PFG Architects

PFG Goal Leads

PFG LOA Implementers

Independent Experts

v. Analysis of Interview Data

Upon completing the interviews, the evaluation team will document and code session transcripts by topic areas and themes to identify common trends and outliers within the research gathered. (Refer to the language section on how translations will be handled prior to analysis). Qualitative analysis will be conducted using NVivo. NVivo will provide easy access to relevant portions of the transcripts—identifying common issues, successes, and challenges—and access to associations between activity characteristics and their results.

vi. Analysis of Data Collected with Confidential Online Surveys

PFG is a new process for delivering development assistance. The first set of research questions for this evaluation—the questions that cut across governments and are germane to all PFG programs regardless of partner country—are being used to assess some of the costs and benefits, intended and unintended, of the PFG.

A short, confidential online survey provided to the staff of USG and GPH agencies/ministries responsible for administering and monitoring PFG initiatives will be used to collect data addressing the cross-cutting questions. Table 5 summarizes survey items by research question.

Annex 2 provides proposed table shells. Where feasible,47 chi-square tests for deviance from uniform response will be used, and when respondent characteristics are used in cross-tabulations, tests for independence will be conducted. Regardless of results, these tests must be considered descriptive only. Table 17: Online Survey Items by Research Question

Questions CCQ1 CCQ2 CCQ3 CSQ1 CSQ2 CSQ3 Q1. To the best of your recollection, when did you begin work on PFG? Q2. For your agency/ institution, did you have a role in the planning and development of PFG? Q3. For approximately how many weeks were you involved in the planning and development of PFG? Q4. During the PFG planning and development stages, approximately how many hours per week, on average, did you dedicate to these tasks? Q5. What is or was your specific PFG assignment? (Leadership)

47 For example, where cell sizes are large enough

145

Questions CCQ1 CCQ2 CCQ3 CSQ1 CSQ2 CSQ3 Q5. What is or was your specific PFG assignment? (Goal lead) Q5. What is or was your specific PFG assignment? (Project management) Q5. What is or was your specific PFG assignment? (Project implementation) Q5. What is or was your specific PFG assignment? (Monitoring and Evaluation) Q5. What is or was your specific PFG assignment? (Other) Q6. Do you have experience planning, implementing or monitoring development projects outside of PFG? Q7. As a result of your involvement with PFG, has/did your workload Q8. On average, about how much time per week do/did your PFG responsibilities require? Q9. As a result of your involvement with PFG, for each of the tasks in the table, has/did your workload (PFG task coordination with colleagues within my government) Q9. As a result of your involvement with PFG, for each of the tasks in the table, has/did your workload (PFG task coordination with colleagues in other (partner) governments) Q9. As a result of your involvement with PFG, for each of the tasks in the table, has/did your workload (Monitoring progress (indicators, site visits, milestones) of PFG tasks) Q9. As a result of your involvement with PFG, for each of the tasks in the table, has/did your workload (Communicating on PFG with my superiors and senior leadership in my government) Q9. As a result of your involvement with PFG, for each of the tasks in the table, has/did your workload (Managing PFG activities) Q9. As a result of your involvement with PFG, for each of the tasks in the table, has/did your workload (Designing and/or procuring PFG activities) Q9. As a result of your involvement with PFG, for each of the tasks in the table, has/did your workload (Other administrative tasks) Q10. On average, about how many hours per week are/were dedicated to the PFG tasks in the table? (PFG task coordination with colleagues within my government) Q10. On average, about how many hours per week are/were dedicated to the PFG tasks in the table? (PFG task coordination with colleagues in other (partner) governments) Q10. On average, about how many hours per week are/were dedicated to the PFG tasks in the table? (Monitoring progress (indicators, site visits, milestones) of PFG tasks) Q10. On average, about how many hours per week are/were dedicated to the PFG tasks in the table? (Communicating on PFG with my superiors and senior leadership in my government) Q10. On average, about how many hours per week are/were dedicated to the PFG tasks in the table? (Managing PFG activities) Q10. On average, about how many hours per week are/were dedicated to the PFG tasks in the table? (Designing and/or procuring PFG activities) Q10. On average, about how many hours per week are/were dedicated to the PFG tasks in the table? (Other administrative tasks) Q11. In your opinion, compared to other approaches to development assistance intended to affect economic growth, does the PFG represent… Q12. In your opinion, is PFG meeting its goal of advancing economic growth in the Philippines? Q13. One of PFG's goals is to employ 'non-assistance' development tools. 'Non-assistance' tools include diplomatic engagement, convening authority, and other forms of non-monetized assistance to engage both

146

Questions CCQ1 CCQ2 CCQ3 CSQ1 CSQ2 CSQ3 governmental and non-governmental stakeholders in support of catalytic policy change and development priorities. Have you seen non-assistance tools being used in the PFG activity you are or were involved with? Q14. Can you briefly describe the non-assistance tools you have seen, and how they were used? (Example no. 1) Q14. Can you briefly describe the non-assistance tools you have seen, and how they were used? (Example no. 2) Q14. Can you briefly describe the non-assistance tools you have seen, and how they were used? (Example no. 3) Q15. In your opinion, are the appropriate indicators being used to allow for transparent, accountable and fact-based monitoring of the PFG? Q16. Can you provide some examples of alternative indicators to allow for transparent, accountable, fact-based monitoring of the PFG? (Example no. 1) Q16. Can you provide some examples of alternative indicators to allow for transparent, accountable, fact-based monitoring of the PFG? (Example no. 2:) Q16. Can you provide some examples of alternative indicators to allow for transparent, accountable, fact-based monitoring of the PFG? (Example no. 3:) Q17. In your opinion, are the appropriate indicators being used to allow for transparent, accountable and fact-based monitoring of the PFG? (Please explain why you are not sure) Q18. In your opinion, what are the main strengths of PFG program? Q19. In your opinion, what are the main weaknesses of PFG program?

vii. Cross-cutting Questions

Respondent Characteristics

The online survey is designed to assure the anonymity of respondents, but certain questions are framed to put respondents’ answers in context with their experiences. The following items will be used as cross-tabulation variables.

• The length of time the respondent has been involved with PFG (Q.1) • Whether the respondent was involved in the initial planning and development of PFG (Q.2) • The respondents’ PFG role (leadership, goal lead, management, implementation, monitoring and

evaluation, other) (Q.5) • The respondents’ experience managing, implementing, or monitoring and evaluating

development projects other than PFG (Q.6)

For example, a respondent who was involved in the initial planning and development of PFG may have predispositions based upon those earliest PFG experiences.

Advantages and Disadvantages of PFG’s WGA, CCQ1

Lines of survey questioning that stand apart from those specifically targeting changes in workload include the following.

• The overall advantage/disadvantage of PFG relative to other development assistance methods (Q.11)

147

• Whether PFG is meeting its goal of advancing economic growth through investment relative to other approaches (Q.12)

• The main strengths of the PFG approach (Q.18) • The main weaknesses of the PFG approach (Q.19)

Changes in Workload of National and USG Personnel as a Result of PFG, CCQ2

For a number of reasons, predominantly the lack of baseline data, it is not possible to measure directly the change in workload due to PFG. Instead, impressionistic accounts will be gathered and reported. The questions that directly address CCQ2 include:

• Approximate number of weeks spent planning and developing PFG (Q.3) • Approximate number of hours spent per week on planning and developing PFG (Q.4) • Increase in workload as a result of involvement in PFG (Q.7) • Number of hours per week spent on PFG activities (Q.8) • Burden and time spent on management and coordination with other PFG partners (Q.9 and Q.10) • Burden and time spent on management and coordination within your own agency (Q.9 and Q.10) • Burden and time spent monitoring PFG progress and outcomes (Q.9 and Q. 10) • Burden and time spent communicating about PFG with superiors and senior leadership in my

government (Q.9 and Q.10) • Burden and time spent managing PFG activities (Q. 9 and Q. 10) • Burden and time spent designing and/or procuring PFG activities (Q.9 and Q.10) • Percentage of time spent on other PFG administrative tasks (Q.10)

Non-assistance (CCQ3)

Non-assistance tools include diplomatic engagement, convening authority, and other forms of non-monetized assistance to engage both governmental and non-governmental stakeholders in support of catalytic policy change and development priorities. Non-assistance may be considered the benefit of a broader social network due to the WGA. Respondents will be asked to identify and describe examples of non-assistance.

• Evidence of the use of non-assistance tools (Q.13) • Descriptions of the use of non-assistance tools (Q.14)

viii. Country-specific Questions

Quantitative information used to manage JCAP implementation (CSQ2)

Respondents will be asked to identify and describe examples of quantitative information used to monitor the PFG.

• Whether appropriate indicators are being used for transparent, accountable and fact-based monitoring of the PFG (Q. 15)

• Examples of alternative indicators that allow for transparent, accountable, fact-based monitoring of the PFG (Q.15)

148

ix. Pre-Field Visit Data Needs and Analytical Guide

Cross-cutting Questions

Table 18: Cross-cutting Questions Evaluation Questions

Type of Answer Needed (e.g. descriptive, normative, cause-effect)

Data Collection Method(s)*

Gender Disaggregation of Data, where Possible**

Selection Criteria Data Analysis Method(s)

1. What are the advantages and/or disadvantages of the PFG whole-of-government approach to development assistance?

• Normative • Descriptive

• Documents reviewed

• Initial online, confidential survey

• In-depth semi-structured interviews

• ***Gender disaggregation will be conducted whenever possible

• PFG POCs (USG, GOES, GPH)

• PFG Troika selected staff (MCC, State Department, USAID)

• PFG Goal Leads (USG, GOES, GPH)

• Government Ministers who lead PFG constraints ministries

• Private sector community (associated with Growth Council)

• Civil society (associated with PFG LOAs)

• Qualitative (typology, induction, matrix/logical analyses)

• Quantitative (descriptive statistics)

2. To what extent has the Partnership for Growth affected the workload on national government and U.S. government staff, as compared to the workload created by traditional forms of development assistance delivery?

• Descriptive • Initial online, confidential survey


• Gender disaggregation will be conducted whenever possible




• Qualitative (induction)

• Quantitative (descriptive and inferential statistics)

3. What contribution has non-assistance made to the PFG process and how can it be utilized moving forward?






• PFG Goal leads (USG, GOES, GPH)

• Government ministers who lead PFG constraints ministries


149

Philippines Country-Specific Questions

Table 19: The Philippines Country-Specific Questions Evaluation Questions





















• Program documents reviewed

• Performance measurement data

• ***Gender disaggregation will be conducted whenever possible








150

Evaluation Questions






• Normative • Descriptive • Cause-effect



• Publicly available, international datasets

• Program documents reviewed

• Performance measurement data

• **Gender disaggregation will be conducted whenever possible




• Central American nations


• Quantitative (descriptive)

• Comparative (quasi-experimental)

*To the greatest extent possible, information from all data sources will be used for cross-validation. Information from the desk review and interviews with key personnel and stakeholders, for example, can be cross-validated with objective performance measurement data where subject matter overlap exists. For the cross-cutting questions, the opportunities for cross-validation will be limited.

**Analysis of the data will be disaggregated by gender whenever the variable is included in existing administrative data, for example in M&E measures. However, the evaluation team will not collect gender specification through its surveys as to do so would threaten the anonymity of respondents. Interviews and site visits will take gender (sex) into account when observing and seeking responses.

Per USAID ADS Chapter 205 3.6.2 all interview teams will comprise both males and females to provide appropriate cultural context. It will not be possible, however, to sample interview respondents by gender as the sample is purposive. If USAID wishes, the gender of interviewees can be noted and reported.

*** Beyond acquiring the sex of respondents, the evaluation team will endeavor to identify whether the program implementation considered gender elements and/or gender mainstreaming. For instance, when trainings were conducted, did implementers make an effort to balance the gender of participants or inclusion of vulnerable groups such as youth, or did the planning phase address gender mainstreaming. The evaluation team will include these elements into questioning of CS 2 and 3 during interviews with the Goal Leads and the Project Implementers. Where possible, site visits will be conducted to further confirm gender inclusion within the PFG initiative.

151

Annex 1: Supporting Tables for Constraint-Level Indicators

The tables in this annex provide the data underlying the figures presented in section I, Constraints to the Philippines’s Development at Baseline.

Table 20: Foreign direct investment, net inflows (% of GDP) 2010 2011 2012 Philippines 0.8 0.8 1.1 Brunei Darussalam 5.1 7.4 5.0 Cambodia 7.0 7.0 11.1 Indonesia 1.9 2.3 2.2 Lao PDR 3.9 3.6 3.1 Malaysia 4.4 5.2 3.2 Myanmar Singapore 23.1 21.1 20.5 Thailand 2.9 2.6 2.9 Vietnam 6.9 5.5 5.4

Source: World Bank, World Development Indicators Table 21: Gross capital formation (% of GDP) 2010 2011 2012 Philippines 21 20 18 Brunei Darussalam 16 13 14 Cambodia 17 17 Indonesia 32 33 36 Lao PDR 24 26 32 Malaysia 23 23 26 Myanmar Singapore 21 22 27 Thailand 26 27 30 Vietnam 36 30 27

Source: World Bank, World Development Indicators Table 22: Global Competitiveness Index, Rank 2010 2011 2012 2013 Philippines 85 75 65 59 Brunei Darussalam 28 28 28 26 Cambodia 109 97 85 88 Indonesia 44 46 50 38 Lao PDR 81 Malaysia 26 21 25 24 Myanmar 139 Singapore 3 2 2 2 Thailand 38 39 38 37 Vietnam 59 65 75 70

Source: World Economic Forum, Global Competitiveness

152

Philippines 79 59 Brunei Darussalam 135 137 Cambodia 116 120 Indonesia 163 159 Lao PDR 8 6 Malaysia 182 182 Myanmar 133 108 Singapore 1 1 Thailand 18 18 Vietnam 98 99

Source: World Bank, Doing Business Table 24: Starting a Business, Rank 2012 2013 Philippines 136 137 Brunei Darussalam 181 184 Cambodia 171 175 Indonesia 82 85 Lao PDR 19 16 Malaysia 189 189 Myanmar 166 170 Singapore 3 3 Thailand 86 91 Vietnam 107 109

Source: World Bank, Doing Business Table 25: Government Effectiveness, Percentile Rank 2010 2011 2012 Philippines 56 57 58 Brunei Darussalam 78 77 75 Cambodia 19 20 22 Indonesia 48 46 44 Lao PDR 20 20 21 Malaysia 83 81 80 Myanmar 2 3 4 Singapore 100 100 100 Thailand 62 61 61 Vietnam 46 47 44

Source: World Bank, Worldwide Governance Indicators Table 26: Rule of Law, Percentile Rank 2010 2011 2012 Philippines 34 36 36 Brunei Darussalam 74 75 73 Cambodia 13 15 17 Indonesia 32 32 34 Lao PDR 19 18 23 Malaysia 66 65 66 Myanmar 3 4 6

Table 23: Ease of Doing Business Rank 2012 2013

153

Singapore 93 93 96 Thailand 49 49 50 Vietnam 35 39 38

Source: World Bank, Worldwide Governance Indicators Table 27: Control of Corruption, Percentile Rank 2010 2011 2012 Philippines 22 26 33 Brunei Darussalam 80 78 72 Cambodia 7 7 14 Indonesia 25 27 29 Lao PDR 8 8 15 Malaysia 63 60 66 Myanmar 0 0 11 Singapore 99 97 97 Thailand 48 50 47 Vietnam 31 31 35

Source: World Bank, Worldwide Governance Indicators Table 28: Regulatory Quality, Percentile Rank 2010 2011 2012 Philippines 45 45 52 Brunei Darussalam 83 85 85 Cambodia 36 32 39 Indonesia 38 41 43 Lao PDR 17 18 22 Malaysia 71 71 70 Myanmar 1 1 2 Singapore 98 96 100 Thailand 56 56 58 Vietnam 28 31 27

Source: World Bank, Worldwide Governance Indicators Table 29: Tax revenue (% of GDP) 2010 2011 2012 Philippines 12 12 13 Brunei Darussalam Cambodia 10 10 12 Indonesia Lao PDR 13 14 15 Malaysia 14 15 16 Myanmar Singapore 13 14 14 Thailand 16 18 17 Vietnam

Source: World Bank, World Development Indicators

154

Philippines -2.4 -0.4 -0.7 -0.1 Brunei Darussalam 8.4 28.1 16.9 16.6 Cambodia -2.8 -4.1 -3.8 Indonesia -1.2 -0.6 -1.7 -2.1 Lao P.D.R. -4.7 -3.0 -1.4 -4.7 Malaysia -4.7 -3.8 -3.6 Myanmar -5.4 -4.6 -3.8 Singapore 7.3 9.3 8.7 Thailand -0.8 -0.6 -1.8 -0.2 Vietnam -2.8 -1.1 -4.8 -5.7

Source: International Monetary Fund, World Economic Outlook

Table 31: Total investment 2010 2011 2012 2013 Philippines 21 20 18 19 Brunei Darussalam 16 13 14 Cambodia 17 22 24 Indonesia 32 33 35 34 Lao P.D.R. Malaysia 23 23 26 26 Myanmar 16 15 Singapore 23 24 28 26 Thailand 26 27 30 29 Vietnam 36 30 27 27

Source: International Monetary Fund, World Economic Outlook

Table 30: General government net lending/borrowing 2010 2011 2012 2013

155

Annex 2: Table Shells for Confidential Online Survey Results

CCQ1 - What are the advantages and disadvantages of the PFG WGA to development assistance?

Perception of PFG approach by length of time working on PFG (Q11 & Q1). Marginals (row and column statistics) provide the univariate statistics

One year or less

Between one and two years

Two years or more

Significant improvement Improvement No change Step backwards Significant step backwards Don't know

Perception of PFG approach by planning role (Q11 & Q2). Marginals (row and column statistics) provide the univariate statistics

Planning role No planning role


Perception of PFG approach by length of PFG role (Q11 & Q5). Marginals (row and column statistics) provide the univariate statistics

Leadership Goal Lead Management Implementation M&E

Significant improvement Improvement No change Step backwards Significant step backwards Don't know *Roles are not mutually exclusive so row marginal do not sum to totals

Perception of PFG approach by experience with other development protocols (Q11 & Q6). Marginals (row and column statistics) provide the univariate statistics

PFG Experience Only Non-PFG Experience


156

PFG meets its development goals by length of time working on PFG (Q12 & Q1). Marginals (row and column statistics) provide the univariate statistics

One year or less


Two years or more

Strongly agree Agree Neither agree nor disagree Disagree Strongly disagree Don't know

PFG meets its development goals by planning role (Q12 & Q1). Marginals (row and column statistics) provide the univariate statistics



PFG meets its development goals by length of PFG role (Q12 & Q5). Marginals (row and column statistics) provide the univariate statistics


Strongly agree Agree Neither agree nor disagree Disagree Strongly disagree Don't know *Roles are not mutually exclusive so row marginal do not sum to totals

PFG meets its development goals by experience with other development protocols (Q12 & Q6). Marginals (row and column statistics) provide the univariate statistics



Main strengths of the PFG program (Q18) – Coded and non-responses. Strengths Freq Percent

Main weaknesses of the PFG program (Q19) – Coded and non-responses. Weaknesses Freq Percent

157

CCQ2 - To what extent has PFG affected the workload on national government and USG staff, as compared to traditional forms of development assistance?

Increase in workload by length of time working on PFG (Q7 & Q1). Marginals (row and column statistics) provide the univariate statistics

One year or less


Two years or more

Workload increased significantly

Increased somewhat About the same Decreased somewhat Workload Decreased

significantly

Increase in workload by planning role (Q7 & Q2). Marginals (row and column statistics) provide the univariate statistics




significantly

Increase in workload by length of PFG role (Q7 & Q5). Marginals (row and column statistics) provide the univariate statistics




significantly

*Roles are not mutually exclusive so row marginal do not sum to 100% Increase in workload by experience with other development protocols (Q5 & Q6). Marginals (row and column statistics) provide the univariate statistics




significantly

158

Average PFG hours per week by length of time working on PFG (Q6 & Q1). Marginals (row and column statistics) provide the univariate statistics

One year or less


Two years or more

More than 20 hours 16 to 20 hours 11 to 15 hours 6 to 10 hours 1 to 5 hours Zero

Average PFG hours per week by planning role (Q6 & Q2). Marginals (row and column statistics) provide the univariate statistics



Average PFG hours per week by length of PFG role(Q8 & Q5). Marginals (row and column statistics) provide the univariate statistics


More than 20 hours 16 to 20 hours 11 to 15 hours 6 to 10 hours 1 to 5 hours Zero *Roles are not mutually exclusive so row marginals will not sum to 100%

Average PFG hours per week by experience with other development protocols (Q8 & Q6). Marginals (row and column statistics) provide the univariate statistics



159

Average PFG hours per week by workload increase by increase due to coordination with colleagues within my government (Q8 & Q9). Marginals (row and column statistics) provide the univariate statistics

Increased Significantly

Increased Somewhat

Stayed about the

Same Decreased Somewhat

Decreased Significantly


Average PFG hours per week workload increase by increase due to coordination within other governments (Q8 & Q9). Marginals (row and column statistics) provide the univariate statistics


Increased Somewhat

Stayed about the




Average PFG hours per week workload increase by increase due to monitoring progress (Q8 & Q9). Marginals (row and column statistics) provide the univariate statistics


Increased Somewhat

Stayed about the




Average PFG hours per week workload increase by increase due to communicating with superiors (Q8 & Q9). Marginals (row and column statistics) provide the univariate statistics


Increased Somewhat

Stayed about the




160

Average PFG hours per week workload increase by increase due to managing PFG activities (Q8 & Q9). Marginals (row and column statistics) provide the univariate statistics


Increased Somewhat

Stayed about the




Average PFG hours per week workload increase by increase due to designing and/or procuring PFG activities (Q8 & Q9). Marginals (row and column statistics) provide the univariate statistics


Increased Somewhat

Stayed about the




Average PFG hours per week workload increase by increase due to other administrative tasks (Q8 & Q9). Marginals (row and column statistics) provide the univariate statistics


Increased Somewhat

Stayed about the




For approximately how many weeks was the respondent involved in the planning and development of PFG? (Q3).

Mean SD

Weeks

During the PFG planning and development stages, approximately how many hours per week, on average, has the respondent dedicate to these tasks? (Q4).

Freq Percent


161

CCQ3 - What contributions has "non-assistance" made to the PFG process and how can it be utilized moving forward?

Has observed non-assistance by length of time working on PFG (Q13 & Q1). Marginals (row and column statistics) provide the univariate statistics

One year or less


Two years or more

Yes No Not sure

Has observed non-assistance by planning role (Q13 & Q2). Marginals (row and column statistics) provide the univariate statistics


Yes No Not sure

Has observed non-assistance by length of PFG role (Q13 & Q4). Marginals (row and column statistics) provide the univariate statistics


Yes No Not sure *Roles are not mutually exclusive so row marginal do not sum to 100%

Has observed non-assistance by experience with other development protocols (Q13 & Q5). Marginals (row and column statistics) provide the univariate statistics


Yes No Not sure

Description of non-assistance tools and how they were used (Q14) – Coded and non-responses. Tools Freq Percent

CSQ2 - Is quantitative and objectively verifiable information being used to manage JCAP implementation in order to achieve and measure results?

Use of best indicators by length of time working on PFG (Q15 & Q1). Marginals (row and column statistics) provide the univariate statistics

One year or less


Two years or more

Best indicators are used Best indicators are not used Not sure

162

Use of best indicators by planning role (Q15 & Q2). Marginals (row and column statistics) provide the univariate statistics



Use of best indicators by length of PFG role (Q15 & Q4). Marginals (row and column statistics) provide the univariate statistics


Best indicators are used Best indicators are not used Not sure *Roles are not mutually exclusive so row marginal do not sum to 100%

Use of best indicators by experience with other development protocols (Q15 & Q5). Marginals (row and column statistics) provide the univariate statistics



Examples of alternative indicators to allow for transparent, accountable, fact-based monitoring of the PFG (Q16) – Coded and non-responses. Indicators Freq Percent

163

Annex 3: Confidential Online Surveys

Partnership for Growth Government Agency Survey

Page 1 of 14

Dear respondent,

You are receiving this questionnaire because of your general knowledge and/or affiliation with the Partnership for Growth program (PFG) between the Governments of El Salvador and the United States of America, and the Philippines and the United States.

This is a confidential survey and your identity will be known only to the evaluation team and will not be shared. All survey responses are treated by Optimal Solutions Group, LLC in strict confidentiality. Individual responses will not be reported or made public, except to the extent required by law. This is to ensure that your responses can be as frank as possible, without concern for the possible sensitivities of any other parties. It is a brief questionnaire that should take less than 20 minutes to complete. Your participation is absolutely voluntary. If you wish not to answer a question, simply skip it and move to the next one. By participating in this survey you are giving your informed consent.

The confidential information you provide will be invaluable to the successful conduct of the PFG evaluation. Please complete the survey no later than May 12, 2014. If you have any questions or issues please contact Optimal at [email protected].

Thank you in advance for your cooperation.

164

Partnership for Growth Government Agency Survey

Page 2 of 14

The Partnership for Growth (PFG), was initiated in 2011 through bilateral agreements between the United States Government (USG) and partnering countries’ national governments (El Salvador and the Philippines) with the aim of achieving accelerated, sustained, broad-based economic growth in partner countries. The PFG requires the identification of countries’ constraints to growth in order to develop a joint plan to address the most pressing of these constraints. It also requires transparency, mutual accountability and fact-based monitoring and evaluation. The following questions request information on the PFG’s ability to meet these goals.

This portion of the survey asks questions concerning your assignment and workload on PFG.

Q1. To the best of your recollection, when did you begin work on PFG?

More than 2 years ago

Between 1 and 2 years ago

One year ago or less

Q2. For your agency/ institution, did you have a role in the planning and development of PFG?

Yes

No

165

Partnership for Growth Government Agency Survey Agency Survey

Page 3 of 14

Q3. For approximately how many weeks were you involved in the planning and development of PFG? (please enter a non-negative, numeric value only)

Q4. During the PFG planning and development stages, approximately how many hours

per week, on average, did you dedicate to these tasks?

Zero

1 to 5 hours

6 to 10 hours

11 to 15 hours

16 to 20 hours

More than 20 hours per week

166


Page 4 of 14

Q5. What is or was your specific PFG assignment? (Select All that Apply)

Leadership

Goal lead

Project management

Project implementation

Monitoring and Evaluation

Other

Q6. Do you have experience planning, implementing or monitoring development projects outside of PFG?

Yes

No

167


Page 5 of 14

Q7. As a result of your involvement with PFG, has/did your workload

Increased significantly

Increased somewhat

Stayed about the same

Decreased somewhat

Decreased significantly

Q8. On average, about how much time per week do/did your PFG responsibilities

require?

Zero

1 to 5 hours

6 to 10 hours

11 to 15 hours

16 to 20 hours

More than 20 hours per week

168


Page 6 of 14

Q9. As a result of your involvement with PFG, for each of the tasks in the table, has/did your workload…

Increase significantly

Increase somewhat

Stay about

the same

Decrease somewhat

Decrease significantly

PFG task coordination with colleagues within my government

PFG task coordination with colleagues in other (partner) governments

Monitoring progress (indicators, site visits, milestones) of PFG tasks

Communicating on PFG with my superiors and senior leadership in my government

Managing PFG activities Designing and/or procuring PFG activities

Other administrative tasks

169


Page 7 of 14

Q10. On average, about how many hours per week are/were dedicated to the PFG tasks in the table? (please enter non-negative, numeric values only)

Average Hours per Week

PFG task coordination with colleagues within my government

PFG task coordination with colleagues in other (partner) governments

Monitoring progress (indicators, site visits, milestones) of PFG tasks

Communicating on PFG with my superiors and senior leadership in my government

Managing PFG activities Designing and/or procuring PFG activities

Other administrative tasks

170


Page 8 of 14

We would now like to ask you a few brief questions about your perceptions of the PFG approach. Q11. In your opinion, compared to other approaches to development

assistance intended to affect economic growth, does the PFG represent

A significant improvement

An improvement

No change

A step backwards

A significant step backwards

Don’t know

Q12. In your opinion, is PFG meeting its goal of advancing economic growth in the

Philippines?

Strongly agree

Agree

Neither agree nor disagree

Disagree

Strongly disagree

Don’t know

171


Page 9 of 14

Q13. One of PFG’s goals is to employ “non-assistance” development tools. “Non-assistance” tools include diplomatic engagement, convening authority, and other forms of non-monetized assistance to engage both governmental and non-governmental stakeholders in support of catalytic policy change and development priorities. Have you seen non-assistance tools being used in the PFG activity you are or were involved with?

Yes

No

Not sure

172


Page 10 of 14

Q14. Can you briefly describe the non-assistance tools you have seen, and how they were used?

Example no. 1:

Example no. 2:

Example no. 3:

173


Page 11 of 14

Q15. In your opinion, are the appropriate indicators being used to allow for transparent, accountable and fact-based monitoring of the PFG?

The best available indicators are being used

Some of the best available indicators are being used

The best available indicators are not being used

Not sure

174


Page 12 of 14

Q16. Can you provide some examples of alternative indicators to allow for transparent, accountable, fact-based monitoring of the PFG?

Example no. 1:

Example no. 2:

Example no. 3:

175


Page 13 of 14

Q18. In your opinion, what are the main strengths of PFG program?

Q19. In your opinion, what are the main weaknesses of PFG program?

176


Page 14 of 14

Survey Completed

Thank you for your valuable contribution to the PFG evaluation!

177

Annex 4: Interview Guides

Stakeholder Types

Leadership – Interview Guide 1

Respondent Type: The targeted respondents for this semi-structured Interview Guide # 1 are current and former USG and GOES officials who have held or hold leadership positions within the PFG initiative, particularly POCs and others at their level who will be more informed of the implementation of PFG.

Architect – Interview Guide 2

Respondent Type: The targeted respondents for this semi-structured Interview Guide # 2 are stakeholders directly involved with the design and planning of the El Salvador PFG, or whose inputs were sought after for these processes. The stakeholders mainly include USG and GOES officials (current and former), members of the Growth Council, representatives from private sector and other civil society organizations, as well as independent experts.

Goal Lead – Interview Guide 3

Respondent Type: The targeted respondents for this semi-structured Interview Guide # 3 are active and former GOES and USG Goal Leads who have directly worked on respective goal(s).

LOA Implementer – Interview Guide 4

Respondent Type: The targeted respondents for this semi-structured Interview Guide # 4 are the implementers of all or selected lines of action (LOA) under the selected goals. The interview will be performed primarily with the chiefs of party, directors, and/or coordinators or their representatives.

Independent Expert – Interview Guide 5

Respondent Type: The targeted respondents for this semi-structured Interview Guide # 5 are independent experts. Experts include academics, subject matter experts, journalists and others who contribute to public debate on the PFG in general or specific areas of the PFG, but are not responsible for directing or implementing components of the PFG. The guide includes cross-cutting questions (to gauge changes in the operational efficiency, selection, coordination, design, and management of development interventions under the PFG strategy as compared to previous / other approaches) as well as goal and LOA level related questions (the latter in particular will be contextualized by the SME interviewer for the specific area of expertise of the interviewee at hand).

178

Semi-structured Interviews

Interview Guide 1 - PFG LEADERSHIP

Background Information: The Partnership for Growth (PFG) aims to achieve accelerated, sustained, broad-based economic growth in partner countries, including El Salvador and the Philippines, through bilateral agreements between the United States Government (USG) and the partnering countries’ national governments. Using principles set forth in President Obama’s September 2010 Presidential Policy Directive on Global Development, the PFG requires rigorous, joint analyses of countries’ individual constraints to growth in order to develop joint action plans to address the most pressing of these constraints and to establish high-level mutual accountability for the goals and activities selected to alleviate them. This interview guide was designed to collect information on cross-cutting questions about the program.

Respondent Type: The targeted respondents for this semi-structured Interview Guide # 1 are current and former USG and GPH officials who have held or hold leadership positions within the PFG initiative, particularly POCs and others at their level who will be more informed of the implementation of PFG.

Central Focus of Questions: This guide includes (i) cross-cutting questions on the PFG Whole of Government Approach (WGA), changes in operational efficiency and work load, as well as on non-assistance; and (ii) Philippines specific questions on the remedial capacities of the JCAP, on M&E issues, and on the mid-term performance of selected goals as related to the desired outcomes. The interviewer will note that the term Whole of Government Approach is not known to all parties, especially in the Philippines. Be prepared to probe with the terms inter-agency cooperation or inter-agency coordination.

Methodology. Semi-structured interview. Approximately 1 hour.

179

Respondent Background Information Date and Time of Interview: Name of Interviewer:

Interviewee Name:

Interviewee Stakeholder Type:

Interviewee Occupation:

Job Title:

Overall or “Goal Specific”:

Length of Involvement with PFG: Terminology Used

• Constraints – these refer (as referenced in the JCAP and SOW) to the two main constraints – weak governance and narrow fiscal space – that have been identified by the Philippines to be tackled in order to remove economic constraints

• Subconstraints – these refer to the three topic areas that have been identified within the two main constraints. These are also the expected outcomes. They are:

o (1) regulatory quality improved; o (2) rule of law and anti-corruption measures strengthened; and o (3) fiscal performance improved.

• Goals – given that the SOW requests a goal-selection process, the 17 indicators provided in the

JCAP and viewed as outputs for the Philippines PFG initiative will be referred to as goals.

• Activities and projects: Activities that are conducted to address these goals will be referred to as activities and projects.

180

INTERVIEW QUESTIONS A. CROSS-CUTTING Advantages and disadvantages of the PFG approach in general

1. Based on your role within the PFG initiative, has the PFG approach resulted in changes in the

way responsibilities and leaderships are shared or exercised within or among the USG agencies directly involved in the implementation of activities you manage? YES or NO

a. If Yes, how? What are the principal advantages and disadvantages of these changes?

2. Has the PFG resulted in changes in the implementation coordination process between USG and Philippines agencies? YES or NO.

a. If yes, how? What are the main advantages and disadvantages of these changes?

The following questions seek responses concerning the PFG process – in particular the Constraints Analysis (CA) and the Whole of Government Approach (WGA) – and whether these new approaches have demonstrated improvements over pre-PFG assistance approaches. Information obtained within this section will feed into CCQ1 (advantages/disadvantages of the PFG approach to development assistance), CCQ2 (PFG impact on workload, and CCQ3 on the role of “non-assistance”).

Regarding the WGA (to USG ONLY) The Whole of Government Approach (WGA) is relevant to identifying areas for assistance, selecting interventions, and determining implementation coordination. The approach reflects efforts to align each agency’s activities to achieve a common objective.

3. In your opinion, has the WGA led to change in the way the USG delivers development assistance in Philippines? YES or NO?

a. If yes, what kind of change? Please provide specific example(s).

4. In your opinion, compared to previous forms of development assistance, has the WGA approach in Philippines led to:

a. Change(s) in design of development initiatives? (Please provide examples) b. Change(s) in management and coordination of development initiatives? (Please provide

examples) c. Change(s) in operational efficiency? (Please explain and/or provide example(s))

181

Regarding the WGA (to GPH ONLY): The Whole of Government Approach (WGA) is relevant to identifying areas for assistance, selecting interventions, and determining implementation coordination. The approach reflects efforts to align each agency’s activities to achieve a common objective, hence promoting inter-agency coordination and collaboration

NOTE FOR THE INTERVIEWER: Interviewers should prompt with “interagency efforts/collaboration” since WGA is not a widely used term.

5. Is WGA being implemented within the Philippines Government? YES or NO.

If yes, how is the WGA being implemented within the Philippines Government?

6. In your opinion, compared to traditional forms of development assistance, has the WGA led to (if answer to the previous question is Yes):

a. Change in design of development initiatives? (Please provide examples) b. Change in coordination of development initiatives? (Please provide examples) c. Change in operational efficiency? (Please provide examples) d. Change in workload? (Please provide examples)

On non-assistance (Both USG and Philippines) “Non-assistance” tools include diplomatic engagement, convening authority, and other forms of non-monetized assistance to engage both governmental and non-governmental stakeholders in support of catalytic policy change and development priorities. Note to Interviewer: Consider the following terminology “non-aid assistance; “use of the Good Offices”…, if the interviewee is confused by the term.

7. What contribution has non-assistance made to the PFG process in Philippines? Please provide

specific examples.

8. How can non-assistance (within the context of Philippines) be best utilized moving forward?

PHILIPPINES - SPECIFIC QUESTIONS The following questions seek responses concerning whether the PFG has been developed in such a way to allow for an effective impact on the identified constraints and production of the desired outcomes. [Information obtained will feed into CSQ2 (if JCAP goal level commitments are capable of achieving the constraint level objectives and goals]. Joint Country Action Plan (JCAP) As you know, the Constraints Analysis (CA) was centered on identifying the central binding constraints to growth, but did not identify remedies to address these. To address these remedies, the JCAP was produced.

182

9. What indications do you have that the JCAP is performing its central task of guiding the PFG to perform and move in a direction that will solve the problems identified by the constraints analysis?

10. To what extent are the goal level (objective-level) commitments set forth in the JCAP capable of achieving the constraints-level objectives and outcomes?

11. Are the objectives and activities in the JCAP well defined remedies to overcome the constraints? YES or NO To the interviewer – objectives are goals, based on the terminology explanations above

12. Were there additional objectives (to the interviewer – these are goals, based on the terminology explanations above) and activities that you think should have been included in the PFG that do not already exist? Or were there objectives and activities that should not have been included? If yes, please list and explain.

Technical Sub-committees: The GPH PFG M&E Plan, to the JCAP requires Technical Sub-committees to be established to assess/conduct analysis of sector performance and overall performance of programs and how these contribute to PFG goals

13. To your knowledge, have these Sub-committees been fully established? YES or NO a. If yes, have they contributed to the implementation and monitoring of projects?

Evidence-Based Decision Making and Fact-Based Monitoring The following questions are in reference to PFG’s overarching goal of promoting evidence-based decision making and fact-based monitoring. [Information obtained will feed into CSQ2 (PFG emphasis on quantitative and objectively verifiable evidence feeding into decision making and fact-based monitoring].

14. The PFG model places specific emphasis on [1] evidence-based decision making and [2] fact-based monitoring.

a. Is quantitative and objectively verifiable information being used to manage JCAP implementation in order to achieve and measure results? YES or NO. Please explain why (whether Yes or No)

b. How is evidence-based decision making part of managing PFG? (Please illustrate and/or

provide an example).

c. How is “fact-based monitoring” designed and managed under PFG? (Please illustrate and/or provide an example).

The GPH M&E Plan to the JCAP requires the involvement of civil society and the private sector in the design and implementation of PFG. Particularly for: selection of benchmarks and indicators, as well as in

183

developing plans to collect M&E data.

15. Has civil society and/or the private sector been involved in the design and monitoring strategy of PFG? YES or NO?

a. Please explain how, and provide specific examples.

16. As stated in the GPH PFG M&E plan, monitoring of the JCAP occurs on two levels: (1) overall goals, and (2) sector theme-specific programs and activities

a. How has monitoring occurred at these two levels? b. What role do the scorecards play in monitoring, if any? c. Which people or committee(s) are tasked with conducting this monitoring?

17. From your point of view, are Scorecards used within the PFG initiative as a monitoring tool? YES or NO.

a. If Yes, how do you assess the role of the PFG scorecards in monitoring performance and making sure the necessary outputs are produced to achieve the desired PFG objectives?

b. If No, are you aware of why scorecards are not used?

18. Given that the PFG includes 17 objectives/goals and numerous activities and projects under each

objective/ goal, how do you identify under-performing activities, and what systems are used to assess their impact on outcomes?

19. GPH PFG M&E Plan also states that high level representatives of both governments will perform

a yearly “general review of JCAP implementation” (each November from 2012 to 2016). a. Have the yearly meetings occurred? YES or NO b. If yes, what format did the yearly November reviews of 2012 and 2013 take? What

information was reviewed? Who participated? c. If no, why did they not occur? d. What decisions were identified as a result of these meetings, and who were they shared

with?

20. Which indicators were reviewed to gauge progress towards successfully addressing the two

constraints? How were the activity level indicators taken into account for the 2012 and 2013 November reviews?

21. Did the conclusions of the review lead to specific actions (e.g. to overcome an obstacle identified during the review)? YES or NO.

a. If “yes”, what were these actions and how have they been enacted? Note to interviewer – Skip this question if not applicable to the interviewee

22. The PFG M&E Plan states that progress on the governance subconstraint would be gauged in particular through 4 sets of indicators (World Governance Indicators Control of Corruption Index, World Governance Indicators Rule of Law Index, World Governance Indicators Regulatory Quality Index, and court congestion statistics).

184

Was a written review of these indicators produced for the November 2012 and November 2013 reviews? YES or NO

a. If No, how was the review of these indicators performed? Note to interviewer – Skip this question if not applicable to the interviewee

23. The PFG Monitoring and Evaluation Plan states that progress on the regulatory quality subconstraint would be gauged in particular through 9 sets of indicators (Share of foreign direct investment (FDI) to GDP, Gross capital formation increased, World Governance Indicators Regulatory Quality index, Global Competitiveness Index, Doing Business Indicators, International Finance Corporation index of Business Start-ups, Business Start-ups in key sectors, World Governance Indicators Government Effectiveness index, The International Fund for Agricultural Development and the International Finance Corporation measures on Land Rights and Access.). Was a written review of these indicators produced for the November 2012 and November 2013 reviews? YES or NO

a. If no, how was the review of these indicators performed? Note to interviewer – Skip this question if not applicable to the interviewee

24. The PFG M&E Plan states that progress on the fiscal space subconstraint would be gauged in particular through 3 sets of indicators (Tax effort improved, Public spending on priority areas, and the overall budget balance divided by GDP, averaged over a three-year period.). Was a written review of these indicators produced for the November 2012 and November 2013 reviews? YES or NO

a. If no, how was the review of these indicators performed? Being on target, course-corrections, and moving forward [Information obtained will feed into CSQ3 (if selected interventions are on target and creating the necessary outputs to achieve the desired outcomes)].

25. Today, at the mid-term of implementation of the PFG approach, what evidence exists to demonstrate whether the overall Philippines PFG performance is on target and creating the necessary outputs to achieve the desired outcomes?

26. For each of the two constraints, and for each of the selected objectives (goals) [interviewer should name the objective/goal that is applicable to the respondent, if respondent does not work with a specific goal, ask question in general], are the various interventions GPH and USG committed to in the JCAP on target? YES or NO

a. If yes, provide examples. b. If No, can you share reasons why they are behind?

27. In practice, under each constraint, and for the selected objectives/ goals [interviewer should name

the goal that is applicable to the respondent, if respondent does not work with a specific goal, ask question in general], which M&E mechanisms are used to evaluate if interventions are on target or below target?

185

Interview Guide 2 – PFG ARCHITECTS


Respondent Type: The targeted respondents for this semi-structured Interview Guide # 2 are stakeholders directly involved with the design and planning of the Philippines PFG, or whose inputs were sought after for these processes. The stakeholders mainly include USG and GPH officials (current and former), members of the Growth Council, representatives from private sector and other civil society organizations, as well as independent experts.

Central Focus of Questions: The focus of the interview questions are on the PFG overall objectives, the role that Whole of Government Approach (WGA); Constraint Analysis (CA), Joint County Action Plan (JCAP), non-assistance, monitoring and evaluation (M&E), and yearly performance reviews, play in the PFG performance.

Methodology: Semi-structured interview. Approximately 1 hour.

186


Interviewee Name:


Interviewee Occupation/ Title:









187

INTERVIEW QUESTIONS On the WGA:

The Whole of Government Approach (WGA) is relevant to identifying areas for assistance, selecting interventions, and determining implementation coordination. The approach reflects efforts to align the activities of each agency in order to achieve a common objective” 1. Have any changes been realized with how the design of development assistance initiatives

(particularly in the Philippines) has been approached as a result of the initiation of the WGA approach? YES or NO

a. If yes, what are the changes? b. Please cite specific examples

2. Have there been distinctive differences between the PFG approach and other economic-growth development approaches? YES or NO

a. If yes, please cite examples On the JCAP

The Philippines CA was centered on identifying the central binding constraints to growth, but not on identifying remedies to address these. To address these remedies, the JCAP was produced. Is the JCAP fulfilling its role? 3. Are there any indication that the JCAP is leading towards the achievement of constraints-level

objectives and outcomes?

4. Is there any evidence that the goal-level commitments set forth in the JCAP have been effective in achieving the constraints-level objectives and outcomes?

On non-assistance “Non-assistance” tools include diplomatic engagement, convening authority, and other forms of non-monetized assistance to engage both governmental and non-governmental stakeholders in support of catalytic policy change and development priorities. Note to Interviewer: Consider the following terminology “non-aid assistance; “use of the Good Offices”…, if the interviewee is confused by the term

5. What contribution (if any) has non-assistance made to the PFG process, in relation to the Philippines?

6. How can non-assistance (within the context of Philippines) be best utilized moving forward?

On evidence-based decision making and fact-based monitoring The PFG places specific emphasis on evidence-based decision making, fact-based monitoring, and quantitative verifiable information.

7. With the initiation of the PFG, have changes been realized in terms of improving monitoring systems?

188

8. How was evidence-based decision making designed for the PFG initiative? What mechanisms were included in the design to inform its appropriate implementation? Please illustrate and/or provide an example.

9. How is “fact-based monitoring” designed and managed under PFG? What mechanisms were included

in the design to inform its appropriate implementation? (Please illustrate and/or provide an example).

The M&E Addendum also states that high level representatives of both governments will perform a yearly “general review of JCAP implementation” (each November from 2012 to 2016).

10. What was envisioned to be the outcome of these yearly meetings? Please provide specific examples.

On the PFG Main Constraints Progress on the governance subconstraint would be gauged in particular through 4 sets of indicators (World Governance Indicators Control of Corruption Index, World Governance Indicators Rule of Law Index, World Governance Indicators Regulatory Quality Index, and court congestion statistics). 11. What was the rationale for choosing these four indicators among others?

Progress on the regulatory quality subconstraint would be gauged in particular through 9 sets of indicators (Share of foreign direct investment (FDI) to GDP, Gross capital formation increased, World Governance Indicators Regulatory Quality index, Global Competitiveness Index, Doing Business Indicators, International Finance Corporation index of Business Start-ups, Business Start-ups in key sectors, World Governance Indicators Government Effectiveness index, The International Fund for Agricultural Development and the International Finance Corporation measures on Land Rights and Access.). 12. What was the rationale for choosing these nine indicators among others?

On the PFG at Mid-Term 13. Today, at mid-term, is there any evidence that the overall Philippines PFG performance is on target

and creating the necessary outputs to achieve the desired outcomes?

14. The Constraints Analysis identifies challenges including corruption. What risk does this pose for PFG performance, if any?

15. If there are risks, what mechanisms is the PFG using to diminish these risks?

189

Interview Guide 3 - PFG Program Managers/ CORs


Respondent Type: The targeted respondents for this semi-structured Interview Guide # 3 are active and former GPH and USG Program Managers who have directly worked on respective goal(s).

Central Focus of Questions: This guide includes cross-cutting questions to gauge changes in the operational efficiency, selection, coordination, design, and management of development interventions under the PFG strategy as compared to previous / other approaches. Methodology: Semi-structured interview. Approximately 1 hour.

190


Interviewee Name:


Interviewee Occupation/ Title:









191

INTERVIEW QUESTIONS Advantages and disadvantages of the PFG approach in general

1. Based on your role as a Program Manager (in charge of implementing mechanisms) within the PFG initiative, has the PFG approach resulted in changes in the way responsibilities are shared or exercised among the USG agencies directly involved in your goal? YES or NO.

a. If yes, how? b. What are the principal advantages and disadvantages of these changes?

2. Has the PFG resulted in changes in the implementation coordination process between USG and GPH agencies? YES or NO

a. If yes, how? b. What are the main advantages and disadvantages of these changes?

The following questions seek responses concerning how the Whole of Government Approach (WGA) and Joint Country Action Plans (JCAP) have demonstrated improvements (or not) over pre-PFG assistance approaches Regarding the WGA: The Whole of Government Approach (WGA) is relevant to identifying areas for assistance, selecting interventions, and determining implementation coordination. The approach reflects efforts to align each agency’s activities to achieve a common objective, hence promoting inter-agency coordination and collaboration [NOTE FOR THE INTERVIEWER: Interviewers should prompt with “interagency efforts/collaboration” since WGA is not a widely used term.]

3. Are you aware of the WGA as described?

4. In your role as a Program Manager or COR, have you experienced how the WGA is being implemented within the PFG initiative? YES or NO

a. If yes, please provide specific examples. [NOTE FOR INTERVIEWER: If the interviewee is not aware of the WGA under his/her goal: skip to question 6]

5. In your opinion, compared to traditional forms of development assistance, has the WGA led to: a. Change(s) in the design of development initiatives? (Please provide examples) b. Change(s) in the coordination of development initiatives? (Please provide examples) c. Change(s) in operational efficiency? (Please explain and/or provide example)

6. Has the WGA impacted the performance of the activities you are directing as a Program Manager or COR? YES or NO

a. If yes, please explain and provide examples.

192

Changes in development approach due the introduction of the Joint Country Action Plan (JCAP): As you know, while the Constraints Analysis identified the central binding constraints to growth, the Joint Country Action Plan (JCAP) defines the remedies to address these.

7. What indications do you have that the JCAP is performing its central task of guiding the PFG to perform and move in a direction that will solve the problems identified by the constraints analysis?

a. How are these tied to your goals and activities/ projects you oversee?

8. As a Program Manager, do you consider that the JCAP is performing its central role in guiding

the PFG to perform and move in the right direction?

9. For your goal, does the JCAP provide sufficient guidance on performance benchmarks for the

activities? On non-assistance as an inherent part of the PFG: “Non-assistance” tools include diplomatic engagement, convening authority, and other forms of non-monetized assistance to engage both governmental and non-governmental stakeholders in support of catalytic policy change and development priorities.

10. What role is non-assistance playing under your goal? Please provide an example.

11. Please provide examples of specific cases, e.g. of enhanced goodwill, access, receptivity, collaboration or additional or different resources (non-monetized ones, local level ones, etc.)

12. How can non-assistance be best utilized under your goal going forward? On evidence-based decision making and fact based-monitoring: As you know, the PFG model places specific emphasis on [1] evidence-based decision making and [2] fact-based monitoring.

13. For your goal, how do you use quantitative and objectively verifiable information to manage implementation in order to achieve and measure results?

14. For your goal, please provide examples of evidence-based decision making? What role (if any) does “quantitative verifiable information” play in this decision making?

15. Is there a specific M&E plan for your goal as a whole? Is there an M&E plan for each of the activities under your goal? When and how were these formulated?

193

16. Do you use a PFG-issued or a goal-specific M&E indicator system? How do your activities feed

into this? How does your system feed into the PFG system in general? The PFG produces semi-annual scorecards per goal.

17. For your goal, how is consensus reached on the scorecard, given that various agencies and two governments are involved? If there was lack of consensus, how was it overcome?

Note to interviewer – Skip this question if not applicable to the interviewee The PFG M&E Plan states that progress on the governance subconstraint would be gauged in particular through 4 sets of indicators (World Governance Indicators Control of Corruption Index, World Governance Indicators Rule of Law Index, World Governance Indicators Regulatory Quality Index, and court congestion statistics).

18. To what extent do these four indicators reflect performance under your goal?

19. If any, which other indicator would you like to see included or removed, as related to your own goal?

Note to interviewer – Skip this question if not applicable to the interviewee The PFG Monitoring and Evaluation Plan states that progress on the regulatory quality subconstraint would be gauged in particular through 9 sets of indicators (Share of foreign direct investment (FDI) to GDP, Gross capital formation increased, World Governance Indicators Regulatory Quality index, Global Competitiveness Index, Doing Business Indicators, International Finance Corporation index of Business Start-ups, Business Start-ups in key sectors, World Governance Indicators Government Effectiveness index, The International Fund for Agricultural Development and the International Finance Corporation measures on Land Rights and Access.).



Note to interviewer – Skip this question if not applicable to the interviewee The PFG M&E Plan states that progress on the fiscal space subconstraint would be gauged in particular through 3 sets of indicators (Tax effort improved, Public spending on priority areas, and the overall budget balance divided by GDP, averaged over a three-year period.).



194

Progress on the regulatory quality subconstraint would be gauged in particular through 9 sets of indicators (Share of foreign direct investment (FDI) to GDP, Gross capital formation increased, World Governance Indicators Regulatory Quality index, Global Competitiveness Index, Doing Business Indicators, International Finance Corporation index of Business Start-ups, Business Start-ups in key sectors, World Governance Indicators Government Effectiveness index, The International Fund for Agricultural Development and the International Finance Corporation measures on Land Rights and Access.). [This question is only for stakeholders involved in this constraint].

24. To what extent do these nine indicators reflect performance under your goal?

25. If any, which other indicator would you like to see included, as related to your own goal? On being on-target and creating the necessary outputs to achieve the desired outcomes:

26. At mid-term of PFG implementation, is there any evidence that the overall PH PFG performance is on target and creating the necessary outputs to achieve the desired outcomes? Please provide specific example(s).

27. Is your goal(s) on target (or behind target)? Which M&E mechanisms are used to evaluate if goal(s) are on target (or behind target), beyond the scorecards?

28. Please provide examples of successes made and challenges faced with implementing your goal(s).

29. In what way do you coordinate with implementing partners within your goal to ensure that the performance of your goal is on target?

195

Interview Guide No. 4 – PROGRAM/ ACTIVITY IMPLEMENTERS (only for selected objectives/ goals)

Background Information: The Partnership for Growth (PFG) aims to achieve accelerated, sustained, broad-based economic growth in partner countries, including El Salvador and the Philippines, through bilateral agreements between the United States Government (USG) and the partnering countries’ national governments. Using principles set forth in President Obama’s September 2010 Presidential Policy Directive on Global Development, the PFG requires rigorous, joint analyses of countries’ individual constraints to growth in order to develop joint action plans to address the most pressing of these constraints and to establish high-level mutual accountability for the goals and a selected to alleviate them. This interview guide was designed to collect information on cross-cutting questions about the program. Respondent Type: The targeted respondents for this semi-structured Interview Guide # 4 are the implementers of all or selected activities under the selected goals. The interview will be performed primarily with the chiefs of party, directors, and/or coordinators or their representatives. Central Focus of Questions: The guide includes questions regarding the performance of PFG, the monitoring of activities, evidence based decision making, non-assistance, and beneficiaries. Methodology: Semi-structured interview with COP / director / coordinator. Given the detailed questioning, the COP will most likely be joined by team members, or part of the interview will be realized with team members directly. About 1 hour (per team). Overall Note to Interviewer: Some of the Goal Leads are activity Implementers, therefore there will be the need to ensure that repetitive questions are not asked. The evaluation coordination team will ensure that the appropriate guide is provided to the interviewer.

196


Interviewee Name:


Interviewee Occupation:

Job Title:









197

INTERVIEW QUESTIONS The following questions seek responses concerning whether the PFG has been developed in such a way as to allow for the eventual determination of their impact on addressing the identified constraints and desired outcomes. On the JCAP

1. To what extent are the activities you implement guided by the goal-level commitments set forth

in the JCAP?

2. What indications do you have that the activities you and your team are implementing, contribute to the corresponding goal as established in the JCAP?

Technical Sub-committees:

3. The GPH PFG M&E Plan, to the JCAP requires Technical Sub-committees to be established to assess/conduct analysis of sector performance and overall performance of programs and how these contribute to PFG goals

a. To your knowledge, have these Sub-committees been fully established and have they contributed to the implementation and monitoring of projects?

The following questions seek responses concerning evidence based decision making and fact-based monitoring.

4. As you know, the PFG model places specific emphasis on [1] evidence based decision making and [2] fact-based monitoring.

a. Do your activities have a specific M&E plan? YES or NO? b. If yes, when and how were these formulated?

5. How is progress measured for your activities? Do you have set indicators for measuring progress?

Please explain and provide specific examples.

6. Have any changes been made to your activity targets, if yes, what are these and why were the changes made?

7. The PFG is producing semi-annual scorecards per goal. Activity-level indicators are meant to feed into these.

a. Are you aware about whether your activities are included in scorecard reporting? YES or NO?

b. How do your activities and M&E processes feed into the scorecards?

198

8. GPH PFG M&E Plan also states that high level representatives of both governments will perform a yearly “general review of JCAP implementation” (each November from 2012 to 2016).

a. Have the yearly meetings occurred? YES or NO? b. If yes, what format did the yearly November reviews of 2012 and 2013 take? What

information was reviewed? Who participated? c. If no, why did they not occur? d. What decisions were identified as a result of these meetings, and who were they shared

with? The following questions seek responses concerning beneficiaries:

12. How are the beneficiaries of your activity defined?

13. How do you monitor and evaluate the effectiveness of your activities towards targeted beneficiaries?

14. Do your activities include systematized monitoring strategies? YES or NO? a. If yes, is the monitoring strategy defined generally by a PFG methodology or does each

project have its own specific methodology tied to your clients M&E process (e.g. USAID policy)?

15. Is gender equality and/or gender mainstreaming among beneficiaries considered as a measure? If yes, how are you working towards attaining this measure? And how is gender equality measured for each activity?

199

Interview Guide No. 5 – INDEPENDENT EXPERTS

Background Information: The Partnership for Growth (PFG) aims to achieve accelerated, sustained, broad-based economic growth in partner countries, including El Salvador and the Philippines, through bilateral agreements between the United States Government (USG) and the partnering countries’ national governments. Using principles set forth in President Obama’s September 2010 Presidential Policy Directive on Global Development, the PFG requires rigorous, joint analyses of countries’ individual constraints to growth in order to develop joint action plans to address the most pressing of these constraints and to establish high-level mutual accountability for the goals and activities selected to alleviate them. This interview guide was designed to collect information on cross-cutting questions about the program. Respondent Type: The targeted respondents for this semi-structured Interview Guide # 5 are independent experts. Experts include academics, subject matter experts, journalists and others who contribute to public debate on the PFG in general or specific areas of the PFG, but are not responsible for directing or implementing components of the PFG. The guide includes cross-cutting questions (to gauge changes in the operational efficiency, selection, coordination, design, and management of development interventions under the PFG strategy as compared to previous / other approaches) as well as goal and activity level related questions (the latter in particular will be contextualized by the SME interviewer for the specific area of expertise of the interviewee at hand). Methodology: Semi-structured interview with independent experts, approximately 1 hour.

200

Respondent Background Information Date and Time of Interview: Interviewee Name:


Interviewee Occupation/ Job Title:









201

INTERVIEW QUESTIONS Advantages and disadvantages of the PFG WGA: The Whole of Government Approach (WGA) is relevant to identifying areas for assistance, selecting interventions, and determining implementation coordination. The approach reflects efforts to align each agency’s activities to achieve a common objective NOTES TO INTERVIEWER:

- it is likely that the independent experts will not know too much about WGA, so when interviewing be careful to denote if there is confusion with the response, for analysis purposes]

- Interviewers should prompt with “interagency efforts/collaboration” since WGA is not a widely used term.

1. Are you familiar with the PFG-WGA? YES or NO (if no, skip to question #4)

2. From your point of view, has the PFG WGA in Philippines led to change coordination between

the government of the Philippines (GPH) and the U.S. Government (USG) on selecting, planning and implementing growth-oriented development programs? YES or NO.

a. If yes, what changes in leadership, coordination and distribution of responsibilities have you observe? Please provide specific examples.

3. What are the principal advantages and disadvantages of the PFG Initiative and approaches to development? Please provide specific examples.

The role of “non-assistance” under the PFG: “Non-assistance” tools include diplomatic engagement, convening authority, and other forms of non-monetized assistance to engage both governmental and non-governmental stakeholders in support of catalytic policy change and development priorities. Note to Interviewer: Consider the following terminology “non-aid assistance; “use of the Good Offices”…, if the interviewee is confused by the term.

4. In your opinion (if you are aware of this concept), what contribution has non-assistance made to the PFG process in Philippines? Please provide specific examples. (If they do not know what non-assistance is then skip to question 7).

5. How do you think non-assistance has contributed to the PFG initiative in Philippines?

6. How do you think PFG can best measure “non-assistance” and its contribution to reaching its

overall objectives for the PFG initiative in Philippines?

202

Constraint selection and performance The PFG initiative in Philippines identified two binding constraints to growth – weak governance and narrow fiscal space. Within these constraints, 3 development intervention themes – Regulatory Quality, Rule of Law and Anticorruption, and Fiscal Performance, 18 goals and various activities have been created to address the constraints.

7. Based on your observations of the PFG initiative in Philippines, what is your opinion on the effectiveness of the program in responding to these constraints? Please provide specific examples.

On JCAP Goal-Level Commitments As you know, the Constraints Analysis (CA) was centered on identifying the central binding constraints to growth, but did not identify remedies to address these. To address these remedies, the Joint Country Action Plan (JCAP) was produced.

8. In your opinion, to what extent are the goal-level commitments set forth in the JCAP capable of achieving the constraints-level objectives and outcomes?

9. Were there additional goals and activities that you think should have been included in the Initiative that do not already exist? YES or NO.

a. If yes, please list and explain. The use of quantitative, objectively and verifiable information to achieve and measure results:

10. How do the PFG performance indicators and its M&E methodology compare to practices used in

pre-PFG approaches? 11. In your opinion or within your expertise, how relevant, objective and verifiable are the

quantitative indicators the PFG is using? Please provide specific examples if known. On being on target and creating the necessary outputs to achieve the desired outcomes:

12. Are you familiar with the PFG scorecards? YES or NO. (if No, skip this the next question)

13. From your point of view, how do you assess the role of the PFG scorecards in monitoring

performance and making sure the necessary outputs are produced to achieve the desired PFG objectives?

14. Which indicators or measuring instruments other than the scorecards (if any) would be best suited

to monitor the development activities, goal-level commitments and constraint level objectives and outcomes?

203

Annex 5: List of References

Document Type Title of Documentation Status Date Received Source

(USAID/ GPH)

Context Background Constraints Analysis Acquired America in 3D Received July3, 2014 --

Project Planning & Structure

Joint Country Action Plan (JCAP) 2012-2016 Acquired Official list of all PFG Partners and Agencies Received April 25, 2014 GPH USAID/Philippines Country Development Cooperation Strategy 2012-2016

USAID

Philippine Development Plan (PDP) Received December 16, 2013 Mid-Term Update of PDP and Revalidated Results Matrices

Received May 7, 2014

Project Implementation

USG and GPH Matrix of Activities (Received via Annex B of JCAP 2012-2016)

Received January 24, 2014

PFG Core Matrix of Activities as of February 2014 Received May 8, 2014 Examples of USAID’s PFG Accomplishments as of May 2014

Received May 8, 2014 USAID

Philippines PFG Accomplishments as of February 2014

Received May 8, 2014 USAID

Minutes of First Steering Committee Meeting, April 23, 2012

Received January 24, 2014 GPH

Minutes of Second Steering Committee Meeting, Oct 25, 2012


Meeting of Third Steering Committee Meeting, August 5, 2013


M&E

Scorecard as of April 14, 2014 Received May 7, 2014

Amplified Monitoring and Evaluation Plan Received January 24, 2014

Partnership for Growth Scorecard April 2014 with PFG Projects Notes

Received May 9, 2014

Philippines PFG Results Framework Received May 9, 2014

Joint Statement of Principles Acquired Regulatory Quality Relevant Government Agencies: Department of Trade and Industry, National Economic and Development Authority, Department of the Interior and Local Government, Department of Justice, Department of Agriculture, Department of Energy, Department of Tourism, Department of Public Works and Highways, Department of Transportation and Communication, Securities and Exchange Commission, Commission on Higher Education, Bangko Sentral ng Pilipinas, City Governrnents of Batangas, Iloilo and Cagayan de Oro

Activity 1:Trade and Investment Facilitated

INVEST - Matrix Received May 9, 2014 USAID

TRADE - Matrix Received May 9, 2014 USAID

TRADE - Revised Year 2 Workplan Oct13-Setp14 Received May 13, 2014 USAID

204


(USAID/ GPH)

TRADE - Quarterly Jan-Mar 2014 Received May 13, 2014 USAID

COMPETE - Matrix Received May 9, 2014 USAID

COMPETE – Year 2 Plan 11 13 13 Received May 13, 2014 USAID

TAPP - Matrix Received May 9, 2014 USAID

SIMM – Matrix Received May 9, 2014 USAID

COMPETE – M&E Plan Draft Received June 17, 2014 USAID

COMPETE – Quarterly Report Oct-Dec 2013 Received June 17, 2014 USAID

COMPETE – Year 1 Annual Report Received June 17, 2014 USAID

COMPETE – Year 2 Work Plan Received June 17, 2014 USAID


COMPETE – SOW Received June 5, 2014 USAID

COMPETE – Year 2 Plan 11 13 13 Final Received May 20, 2014 USAID

INVEST – Report Jan-Mar 2014 Received June 13, 2014 USAID

INVEST – M&E Plan Received June 13, 2014 USAID

INVEST – Revisions to Option Year Received June 13, 2014 USAID

INVEST – Short Summary Received June 5, 2014 USAID

INVEST – SOW Received June 27, 2014 USAID

SIMM – 1st Quarterly Progress Report Apr-Jun 2012 Final Received June 17, 2014 USAID

SIMM – 2nd Quarterly Progress Report Jul-Sep 2012 Final Received June 17, 2014 USAID

SIMM – 3rd Quarterly Progress Report Oct-Dec 2012 Final Received June 17, 2014 USAID

SIMM – 4th Quarterly Progress Report Jan-Mar 2014 Final Received June 17, 2014 USAID

SIMM – 5th Quarterly Progress Report 07312913 with PMP Results Table Received June 17, 2014 USAID

SIMM – 7th Quarterly Progress Report 01302014 Final Received June 17, 2014 USAID

SIMM – 8th Quarterly Progress Report 043014 Received June 17, 2014 USAID

SIMM – Annual Progress Report 09302013 Received June 17, 2014 USAID

SIMM – Project Performance Management Plan Received June 17, 2014 USAID

205


(USAID/ GPH)

Revised 12202012

SIMM – Work Plan Year 1 Revise 080312 Approved Received June 17, 2014 USAID

SIMM – Work Plan Year 2 Final for Approval 03252013 Received June 17, 2014 USAID

SIMM - SOW Received June 17, 2014 USAID

TAPP – Full Proposal 41712 Revised Received June 27, 2014 USAID

TAPP – Quarterly Report Apr-Jun 2013 Received June 27, 2014 USAID

TAPP – Quarterly Report Jan-Mar 2013 Received June 27, 2014 USAID

TAPP – Quarterly Report Jul-Sep 2013 Received June 27, 2014 USAID

TAPP – Quarterly Report Oct-Dec 2013 Received June 27, 2014 USAID

TAPP – Quarterly Report Sep-Dec 2012 Received June 27, 2014 USAID

TAPP – SOW (Cooperative Agreement) Received June 13, 2014 USAID

TAPP - SOW Received June 27, 2014 USAID

TRADE – Draft M&E Plan 2 Received June 27, 2014 USAID

TRADE – Draft M&E Plan Received June 27, 2014 USAID

TRADE – Report May-Sept2013 Received June 13, 2014 USAID

TRADE – Report Oct-Dec 2013 Received June 27, 2014 USAID

TRADE – PMP 2 (Draft) Received June 13, 2014 USAID

TRADE – Revised Year 2Work Plan Received June 27, 2014 USAID

TRADE – SOW Received June 27, 2014 USAID

TRADE – Report Jan-Mar 2014 Received June 13, 2014 USAID

TRADE – M&E Plan (Draft) Received June 13, 2014 USAID

TRADE – Quarterly Jan-Mar 2014 Received May 20, 2014 USAID

TRADE – Revised Year 2 Work Plan Oct 13-Sept 14 Endorsed by GPH Received May 20, 2014 USAID

Activity 2: Reduced Regulatory Bottlenecks, Entry Barriers and Discriminatory

Philippines National Statistical Coordination Board • GDP, 2006 – 3Q 2013

Acquired --

Philippines Central Bank • Foreign direct investment by sector; 2011 –

2012 Acquired --

COMPETE - Matrix Received May 9, 2014 USAID

206


(USAID/ GPH)

Provisions to Investments

COMPETE Y2 Plan 11 13 13 Received May 13, 2014 USAID TAPP - Matrix Received May 9, 2014 USAID INVEST - Matrix Received May 9, 2014 USAID COMPETE – M&E Plan Draft Received June 17, 2014 USAID COMPETE – Quarterly Report Oct-Dec 2013 Received June 17, 2014 USAID COMPETE – Year 1 Annual Report Received June 17, 2014 USAID COMPETE – Year 2 Work Plan Received June 17, 2014 USAID COMPETE – Year 1 Work Plan Received June 27, 2014 USAID COMPETE – SOW Received June 5, 2014 USAID COMPETE – Year 2 Plan 11 13 13 Final Received May 20, 2014 USAID TAPP – Full Proposal 41712 Revised Received June 27, 2014 USAID TAPP – Quarterly Report Apr-Jun 2013 Received June 27, 2014 USAID TAPP – Quarterly Report Jan-Mar 2013 Received June 27, 2014 USAID TAPP – Quarterly Report Jul-Sep 2013 Received June 27, 2014 USAID TAPP – Quarterly Report Oct-Dec 2013 Received June 27, 2014 USAID TAPP – Quarterly Report Sep-Dec 2012 Received June 27, 2014 USAID TAPP – SOW (Cooperative Agreement) Received June 13, 2014 USAID TAPP - SOW Received June 27, 2014 USAID INVEST – Report Jan-Mar 2014 Received June 13, 2014 USAID INVEST – M&E Plan Received June 13, 2014 USAID INVEST – Revisions to Option Year Received June 13, 2014 USAID INVEST – Short Summary Received June 5, 2014 USAID INVEST – SOW Received June 27, 2014 USAID

Activity 3: Improved Competition Policy

World Economic Forum, Global Competitiveness Report

• Global Competitiveness Index and related measures, 2010 – 2013

Acquired --

Doing Business: Measuring Business Regulations • Ease of doing business and related

indicators; 2013 (not available until 6/14) – 2006

Acquired --

Matrix – Federal Trade Commission Technical Assistance Program Received May 9, 2014 --

COMPETE Y2 Plan 11 13 13 Received May 13, 2014 USAID

TAPP - Matrix Received May 9, 2014 USAID

COMPETE – M&E Plan Draft Received June 17, 2014 USAID

COMPETE – Quarterly Report Oct-Dec 2013 Received June 17, 2014 USAID

COMPETE – Year 1 Annual Report Received June 17, 2014 USAID


207


(USAID/ GPH)


COMPETE – SOW Received June 5, 2014 USAID

COMPETE – Year 2 Plan 11 13 13 Final Received May 20, 2014 USAID

TAPP – Full Proposal 41712 Revised Received June 27, 2014 USAID

TAPP – Quarterly Report Apr-Jun 2013 Received June 27, 2014 USAID

TAPP – Quarterly Report Jan-Mar 2013 Received June 27, 2014 USAID

TAPP – Quarterly Report Jul-Sep 2013 Received June 27, 2014 USAID

TAPP – Quarterly Report Oct-Dec 2013 Received June 27, 2014 USAID

TAPP – Quarterly Report Sep-Dec 2012 Received June 27, 2014 USAID

TAPP – SOW (Cooperative Agreement) Received June 13, 2014 USAID

TAPP - SOW Received June 27, 2014 USAID

Activity 4: Enhanced Philippine Participation in Regional And International Trade Arrangements

Matrix – Trade-Related Assistance For Development Received May 9, 2014 USAID

TRADE Revised Year 2 Workplan Oct13-Setp14 Received May 13, 2014 USAID

TRADE Quarterly Jan-Mar 2014 Received May 13, 2014 USAID

TRADE – Draft M&E Plan 2 Received June 27, 2014 USAID

TRADE – Draft M&E Plan Received June 27, 2014 USAID

TRADE – Report May-Sept2013 Received June 13, 2014 USAID

TRADE – Report Oct-Dec 2013 Received June 27, 2014 USAID

TRADE – PMP 2 (Draft) Received June 13, 2014 USAID

TRADE – Revised Year 2Work Plan Received June 27, 2014 USAID

TRADE – SOW Received June 27, 2014 USAID

TRADE – Report Jan-Mar 2014 Received June 13, 2014 USAID

208


(USAID/ GPH)

TRADE – M&E Plan (Draft) Received June 13, 2014 USAID

TRADE – Quarterly Jan-Mar 2014 Received May 20, 2014 USAID

TRADE – Revised Year 2 Work Plan Oct 13-Sept 14 Endorsed by GPH Received May 20, 2014 USAID

Activity 5: Enhanced Human Capacity for Economic Growth

World Bank • World Development Indicators, Gross

Capital Formation, 1960 – 2012

Acquired --

Indices of Economic Freedom, Heritage Foundation • Overall score, 2010 – 2013 • Business freedom score, 2010 – 2013 • Investment freedom score, 2010 – 2013

Acquired --

Relevant Republic of the Philippines Ministries • # of new agribusiness ventures, 2010 – 2013 • # of new “business process outsourcing”

ventures, 2010 – 2013 • # of new creative industries ventures, 2010 –

2013 • # of new ventures in infrastructure

maintenance and improvement, 2010 – 2013 • # of new manufacturing and logistics

ventures, 2010 – 2013 • # of new mining ventures, 2010 – 2013 • # of new tourism ventures, 2010 – 2013

Pending --

STRIDE - Matrix Received May 9, 2014 USAID HEPP – Work Plan Received May 9, 2014 USAID STRIDE – Annual Work Plan Received May 9, 2014 USAID STRIDE – Quarterly Report April 2014 Received May 9, 2014 USAID

HEPP – Work Plan 2 Received May 9, 2014 USAID

Spending on Education (primary, secondary) 2013-2013

Received June 2, 2014 USAID

STRIDE – M&E Plan Draft Received June 17, 2014 USAID

STRIDE – 1st Quarterly and 1st Annual Report Oct 2013


STRIDE – Quarterly Report Jan 2014 Final Received June 18, 2014 USAID

STRIDE – SOW Received June 5, 2014 USAID

TOR Philippines Final Received June 17, 2014

US Treasury Work Plan Received June 2, 2014 --

Other Additional Child Labor Project – USDOL - M&E Plan Received June 12, 2014 --

209


(USAID/ GPH)

Documents Child Labor Project – USDOL – Progress Report April 2014

Received June 12, 2014 --

Child Labor Project – USDOL – Philippines Technical Narrative

Received June 12, 2014 --

Technical Subcommittee on Regulatory Quality – July 5 2012 Highlights

Received June 5, 2014 GPH

USAID Philippines Performance Management Plan 2012-2016 Redacted


Rule of Law and Anti-Corruption Measures Relevant GPH Agencies: Supreme Court, Courts of Quezon City, Department of Justice, Ombudsman, Civil Service Commission, Commission on Audit

Activity 1: Efforts to Improve Judicial Efficiency

UN Crime Trends Survey (Through 2010; Attempting to acquire 2011 and 2012)

• Police personnel • Professional judges • Adult prison staff • Juvenile prison staff • Adult prison capacity • Juvenile prison capacity

--

JUSTICE - Matrix Received May 9, 2014 USAID JUSTICE – Q2 FY2014 Quarterly Report Received May 9, 2014 USAID JUSTICE – Year 2 Work Plan Received May 9, 2014 USAID JUSTICE – Year 1 Work Plan Received June 19, 2014 USAID JUSTICE – 1st Quarter Progress Report FY2014 Received June 19, 2014 USAID I3 – Year 2 Work Plan Received May 21, 2014 USAID I3 – Program Report Jan-Mar 2014 Received June 17, 2014 USAID I3- Progress Report 2 Oct-Dec 2013 Received June 17, 2014 USAID I3 - Progress Report Oct-Dec 2013 Received June 17, 2014 USAID I3 – Quarterly Progress Report Jan-Mar 2014 Received May 21, 2014 USAID I3 – SOW Received June 17, 2014 USAID I3 – Work Plan Year 1 Matrix Received June 17, 2014 USAID I3 – Work Plan Year 1 Received June 17, 2014 USAID JUSTICE – 2nd Quarter Progress Report FY 2013 Received June 19, 2014 USAID JUSTICE – Annual Report FY 2013 Received June 19, 2014 USAID JUSTICE – M&E Plan Draft Received June 17, 2014 USAID JUSTICE – SOW Received June 19, 2014 USAID I3 – Matrix Received June 21, 2014 USAID

I3 – Contract Performance Assessment Report Received September 24, 2014

I3 – M&E Plan Received September 24, 2014

PHIL-AM – Matrix Received May 21, 2014 USAID PHIL-AM – Quarterly Progress Report Jan-Mar 2014 Received May 21, 2014 USAID PHIL-AM – M&E Plan Draft Received June 19, 2014 USAID

210


(USAID/ GPH)

PHIL-AM – Annual Work Plan Received May 21, 2014 USAID UN CTS 2013 Criminal Justice Resources Acquired May 27, 2014 USAID

Activity 2: Strengthen Anti-Corruption Institutions

I3 - Matrix Received May 9, 2014 USAID

I3 – Year 2 Work Plan Received May 9, 2014 USAID

I3 - Quarterly Progress Report Received May 9, 2014 USAID

PHIL-AM - Matrix Received May 9, 2014 USAID

PHIL-AM - Annual Work Plan Received May 12, 2014 USAID

PHIL-AM – Quarterly Progress Report Jan-Mar 2014 Received May 12, 2014 USAID

I3- Progress Report 2 Oct-Dec 2013 Received June 17, 2014 USAID

I3 - Progress Report Oct-Dec 2013 Received June 17, 2014 USAID

I3 – Quarterly Progress Report Jan-Mar 2014 Received May 21, 2014 USAID

I3 – SOW Received June 17, 2014 USAID

I3 – Work Plan Year 1 Matrix Received June 17, 2014 USAID

I3 – Work Plan Year 1 Received June 17, 2014 USAID

PHIL-AM – M&E Plan Draft Received June 19, 2014 USAID



Activity 3: Strengthen Contract Enforcement

I3 – Matrix Received May 9, 2014 USAID I3 - Year 2 Work Plan Received May 9, 2014 USAID I3 – Quarterly Progress Report Jan-Mar 2014 Received May 9, 2014 USAID JUSTICE – Matrix Received May 9, 2014 USAID JUSTICE – Q2 FY2014 Quarterly Report Received May 9, 2014 USAID JUSTICE – Year 2 Work Plan Received May 9, 2014 USAID

PHIL-AM - Matrix Received May 9, 2014 USAID



JUSTICE – 2nd Quarter Progress Report FY 2013 Received June 19, 2014 USAID

JUSTICE – Annual Report FY 2013 Received June 19, 2014 USAID

JUSTICE – M&E Plan Draft Received June 17, 2014 USAID

JUSTICE – SOW Received June 19, 2014 USAID

PHIL-AM – Matrix Received May 21, 2014 USAID



211


(USAID/ GPH)

PHIL-AM – Annual Work Plan Received May 21, 2014 USAID


I3 – Program Report Jan-Mar 2014 Received June 17, 2014 USAID









Activity 4: Strengthen Intellectual Property Rights Enforcement

I3 – Matrix Received May 9, 2014 USAID

JUSTICE – Matrix Received May 9, 2014 USAID

JUSTICE – Q2 FY2014 Quarterly Report Received May 9, 2014 USAID

JUSTICE – Year 2 Work Plan Received May 9, 2014 USAID









JUSTICE – 2nd Quarter Progress Report FY 2013 Received June 19, 2014 USAID

JUSTICE – Annual Report FY 2013 Received June 19, 2014 USAID

JUSTICE – M&E Plan Draft Received June 17, 2014 USAID

JUSTICE – SOW Received June 19, 2014 USAID



Activity 5: Support Integrity- and Confidence-



212


(USAID/ GPH)

Enhancing Measures I3 - Quarterly Progress Report Received May 9, 2014 USAID


















Activity 6: Strengthen Corporate Governance





I3 – Program Report Jan-Mar 2014 Received May 12, 2014 USAID

I3- Progress Report 2 Oct-Dec 2013 Received May 12, 2014 USAID

I3 - Progress Report Oct-Dec 2013 Received May 21, 2014 USAID

I3 – Quarterly Progress Report Jan-Mar 2014 Received June 17, 2014 USAID



I3 – Work Plan Year 1 Received May 21, 2014 USAID

PHIL-AM – Matrix Received June 17, 2014 USAID

PHIL-AM – Quarterly Progress Report Jan-Mar 2014 Received June 17, 2014 USAID


213


(USAID/ GPH)



PHIL-AM - Annual Work Plan Received June 19, 2014 USAID




Activity 7: Reduce Opportunities for Corruption

World Bank: Worldwide Governance Indicators, 2010 – 2013

• Voice & Accountability • Political Stability and Lack of Violence • Government Effectiveness • Regulatory Quality • Rule of Law • Control of Corruption

Acquired --

Transparency International: Global Coalition Against Corruption, 2010 – 2013

• Ranks how corrupt the public sector is perceived to be

Acquired --

World Justice Project: Rule of Law, 2010 – 2013 • Limited Government Powers • Absence of Corruption • Order and Security • Fundamental Rights • Open Government • Regulatory Enforcement • Civil Justice • Criminal Justice

Acquired --

I3 - Matrix Received May 9, 2014 USAID I3 – Year 2 Work Plan Received May 9, 2014 USAID I3 - Quarterly Progress Report Received May 9, 2014 USAID I3 – Year 2 Work Plan Received May 9, 2014 USAID I3 – Program Report Jan-Mar 2014 Received May 12, 2014 USAID I3- Progress Report 2 Oct-Dec 2013 Received May 12, 2014 USAID I3 - Progress Report Oct-Dec 2013 Received May 21, 2014 USAID I3 – Quarterly Progress Report Jan-Mar 2014 Received June 17, 2014 USAID I3 – SOW Received June 17, 2014 USAID I3 – Work Plan Year 1 Matrix Received June 17, 2014 USAID I3 – Work Plan Year 1 Received May 21, 2014 USAID



214


(USAID/ GPH)

Activity 8: Enforce Ethical Standards














Activity 9: Strengthen Accountability Measures














Additional Documents

ROL AC Sub-Com 25 Jul 2013 Meeting Received June 5, 2013 GPH

ROL AC Sub-Com 5 Jul 2013 Meeting Received June 5, 2013 GPH Fiscal Performance

Activity 1: Increased Philippines Treasury Department • Tax revenues, 2008 – 3Q 2013

Acquired GPH

215


(USAID/ GPH)

Fiscal Space by Expanding the Tax Base

FPI – Matrix Received May 9, 2014 USAID FPI – Annual Work Plan Year 1 2014 Received May 12, 2014 USAID FPI – Progress Report Jan-Mar 2014 Received May 12, 2014 USAID FPI – Report Aug-Dec 2013 Received June 17, 2014 USAID FPI –M&E Plan Draft Received June 17, 2014 USAID FPI – SOW Received June 17, 2014 USAID GPH – National Gvt Cash Ops Report Received May 27, 2014 GPH

Activity 2: Increased Fiscal Space by Minimizing Revenue Loss

Philippines Treasury Department (Budget Surplus/Deficit)

• Public debt, 2008 – 3Q 2013 Acquired GPH

FPI – Matrix Received May 9, 2014 USAID FPI – Annual Work Plan Year 1 2014 Received May 12, 2014 USAID FPI – Progress Report Jan-Mar 2014 Received May 12, 2014 USAID FPI – Report Aug-Dec 2013 Received June 17, 2014 USAID FPI –M&E Plan Draft Received June 17, 2014 USAID FPI – SOW Received June 17, 2014 USAID

Activity 3: Increase Fiscal Space by Improving Expenditure Management Other Priority Public Works

World Bank • World Development Indicators, 2005 – 2012

(Infrastructure Improvement) Acquired --

Philippines Department of Education • Spending on education (primary, secondary

and postsecondary); 2010 – 2013 http://www.dbm.gov.ph/wp-content/uploads/GAA/GAA2010//DEPED/DEPED.pdf http://www.dbm.gov.ph/wp-content/uploads/GAA/GAA2011/DEPED/DEPED.pdf http://www.dbm.gov.ph/wp-content/uploads/GAA/GAA2012/DEPED/DEPED-A.pdf http://www.dbm.gov.ph/wp-content/uploads/GAA/GAA2013/DEPED/1DEPED.pdf Department of Budget and Management (post-secondary) (http://www.dbm.gov.ph/)


World Bank • World Development Indicators, Education

Expenditures ($US), 1970 - 2011 Acquired --

Relevant Republic of the Philippines Ministries • Department of Finance, Department of

Budget and Management, Bureau of Internal Revenue, Bureau of Customs 2010 - 2013

GPH

U.S. Department of Treasury: Terms of Reference, --

http://www.dbm.gov.ph/wp-content/uploads/GAA/GAA2010/DEPED/DEPED.pdf






http://www.dbm.gov.ph/wp-content/uploads/GAA/GAA2012/DEPED/DEPED-A.pdf



http://www.dbm.gov.ph/wp-content/uploads/GAA/GAA2013/DEPED/1DEPED.pdf



http://www.dbm.gov.ph/

216


(USAID/ GPH)

Original Work Plan and latest monthly report Philippines Compact Summary by Millenium Challenge Corporation (MCC) - http://www.mcc.gov/pages/countries/program/philippines-compact (Overview sub-tab)

Received May 14, 2014 --

Quarterly Status Report by MCC – December 2013 Received May 14, 2014 -- Philippines Compact Monitoring and Evaluation Plan and Table of Key Performance Indicators http://www.mcc.gov/pages/countries/evaluation/philippines-compact (Monitoring and Evaluation sub-tab)


Compact and implementation plan: Compact activities outline http://www.mcc.gov/documents/agreements/compact-philippines.pdf


Bureau of Customs – 2010 Acquired May 27, 2014 --

Bureau of Customs - 2011 Acquired May 27, 2014 --



Bureau of Internal Revenue - 2010 Acquired May 27, 2014 --




Dept. of Budget and Management - 2010 Acquired May 27, 2014 --




Dept. of Finance - 2010 Acquired May 27, 2014 --




FPI - Matrix Received May 9, 2014 USAID

FPI – Annual Work Plan Year 1 2014 Received May 12, 2014 USAID

http://www.mcc.gov/pages/countries/program/philippines-compact

http://www.mcc.gov/pages/countries/program/philippines-compact

http://www.mcc.gov/pages/countries/evaluation/philippines-compact

http://www.mcc.gov/pages/countries/evaluation/philippines-compact

http://www.mcc.gov/documents/agreements/compact-philippines.pdf

http://www.mcc.gov/documents/agreements/compact-philippines.pdf

217


(USAID/ GPH)

FPI – Progress Report Jan-Mar 2014 Received May 12, 2014 USAID

FPI – Report Aug-Dec 2013 Received June 17, 2014 USAID

FPI –M&E Plan Draft Received June 17, 2014 USAID

FPI – SOW Received June 17, 2014 USAID

TOR Philippines Final Received June 2, 2015

WorldBank – Infrastructure Indicators – GPH and Regional Countries Acquired May 27, 2014

Tropical Forest Conservation Act

TFCA of 1998 Debt Conversion – Briefer Received June 19, 2014

TFCA Debt Conversion – Signed 2nd DRA US Text 2013-07-18 Received June 19, 2014

Additional Document June 21, 2012 Highlights – Technical Subcommittee on Fiscal Space Received June 5, 2014 GPH