Evaluating the Governance of Global and Regional Partnership Programs (GRPPs)

Anna Aghumian and Chris GerrardIEG World BankNovember 13, 2009

Main Messages

► Assessing legitimacy and effectiveness of governance and management is essential since shared governance is a key characteristic of all GRPPs

► As international public sector organizations, GRPPs should be expected to comply with generally accepted principles of public sector governance

► Understanding how governance is actually practiced requires more than just a cursory examination of a program’s charter, organizational chart and TORs

2

Prevailing Governance Models among the 60 Programs Reviewed

 

3

Shareholder

Model

Stakeholder

Model

Prominent

Individuals

Hybrid

Global Partnership programs

9 35 4 2

Regional Partnership Programs

3 7 -- --

Special Features of GRPPs in Relation to Governance & Management► Often have complex governance and

management structures► Need to establish their legitimacy on a

basis other than shareholder rights► Often have a long chain of accountability

— from global to local► Have a global community clientele,

making transparency in planning and implementation particularly important

► Often housed in existing international organizations

4

5

Assessing Governance & Management – Suggested Criteria► Legitimacy – in the exercise of authority in

relation to those with a legitimate interest in the program

► Efficiency – governance & management structures facilitate efficient allocation and use of resources

► Accountability – up and down the internal chain of command and control

► Responsibility – to stakeholders outside the internal chain of command and control

► Transparency – in relation to decision-making, reporting, and evaluation

► Fairness – equal opportunity for partners and participants, similarly situated, to influence and benefit from the program

Governance & Management: Suggested Approach► Start from a clear understanding of the G& M

arrangements and processes: • The extent to which these are well articulated and

working well to bring about legitimate and effective governance and management of the program

• Host arrangements, if any

► Primary focus on governance. Focus on those aspects of management that most directly affect program performance

► Build upon and add to the assessments of relevance, efficacy, and efficiency of the Program

6

Evaluating Governance and Management of GRPPs: Tools and Instruments ► Desk review of key founding documents

► Interviews with key partners and other stakeholders. Good practice evaluations use interview protocols, semi- structured surveys (e.g. GAVI’s interview guide for Board members; GDN’s Board Survey questionnaire)

► Surveys of members of the governing bodies, wider circle of stakeholders, beneficiaries

► Other : review of meeting minutes of the governing, executive, and advisory bodies; Board meeting attendance rate

► Good practice evaluations use a mix of evaluation instruments

7

Extent to Which 60 Evaluations Assessed Different Dimensions of G &M

8

Good Practice Examples: Legitimacy Global Fund to Fight AIDS, Tuberculosis and Malaria►Evaluation question : To what extent the Board is representative and to what extent its decision making is in accord with the GF’s founding principles►Methodology: Board interviews and stakeholders survey, Board meeting minutes reviews, assessment of Board and committee meeting attendance rates►Findings:

• Board is formally representative. Yet, some constituencies are not participating effectively and do not have equal voice

• Poor communication with beneficiaries, linguistic barriers, lack of adequate financial resources for beneficiary representation

►Recommendation: The Board should improve the quality of representation by enhancing communication with all constituencies and by favorably considering proposals for assistance from constituencies with limited resources

9

Efficiency

Association for the Development of Education in Africa (ADEA)

► Evaluation question: Are the ADEA structures adequate and functioning properly in light of their assigned tasks and available resources?

► Methodology: Organizational assessment based on interviews with stakeholders and staff, on-line survey, desk review of documents and field visits

► Findings: • The Steering Committee is overburdened with the management of the

program, and no time is left for considering strategic issues.• “Decision-by-consensus” model, while provides equal access to decision

making to all members of the SC, slows down its responsiveness• Host arrangements with IIEP reduces the efficiency of G& M of ADEA

► Recommendations: The SC should revise organizational structure of ADEA, improve division of the roles and responsibilities between the SC and the Secretariat, conduct a cost-benefit analysis of the host arrangements with IIEP

10

Accountability

Global Alliance for Vaccines and Immunization (GAVI)

► Evaluation questions: How well has the GAVI Fund governance structure worked? Is there clarity of role/responsibilities between various entities?

► Methodology: Interviews with key informants, desk review of documents, prior assessments of GAVI’s governance

► Findings: • Lack of accountability due to separation of programmatic and

fiduciary responsibilities between GAVI Alliance and GAVI Fund• Unclear and weak accountability chain within each of the governing

bodies (vertical accountability) • Unclear view among partners regarding their respective roles and

responsibilities

► Recommendation: Drastic changes in governance structure

11

Responsibility

Medicines for Malaria Venture► Evaluation question: The extent to which the program

accepts and exercises responsibility to stakeholders who are not directly involved in governance

► Methodology: Desk review of documents, interviews► Findings:

• MMV has increased engagement of researchers and research institutions in endemic countries

• Has held key meetings in countries where malaria is widespread and MMV-sponsored research is underway

• Has included a majority of beneficiary country members on its Access and Delivery Advisory Committee

► Recommendation: To engage more advocacy NGOs in the design and execution of MMV’s access and delivery work program.

12

Transparency

Partnership for Maternal, Newborn and Child Health

► Evaluation question: Is there adequate transparency in governance?

► Methodology: Systematic review of the program’s website

► Findings: • Board meeting minutes are accessible on the website, but

other important background material is not provided • Financial information is not fully available• Work plans are not available on the website, leading to

potential duplication of activities by partners

► Recommendations: Develop a Board-approved disclosure policy

13

Fairness

Cities Alliance ► Evaluation question: The extent to which

participants similarly situated have equal opportunity to receive benefits from the program

► Methodology: Desk review of documents, interviews

► Findings: • All grant applications have to be sponsored by a board

member and about 90% of grant applications are approved• Some kind of screening or pre-selection process was

taking place which was not transparent and potentially unfair to potential recipients

► Recommendation: To make the grant management criteria and process more transparent

14

Host Arrangements

International Land Coalition► Evaluation question: The extent to which the relationship

between ILC and IFAD, as host and international focal point, is mutually beneficial?

► Methodology: Desk review, interviews, survey of partners ► Findings:

• There are gaps and conflicting clauses in the legal and administrative agreements regulating IFAD-ILC relations

• Dominant role of IFAD reduces the independence of ILC and the incentives of other partners to participate effectively in the program

• Host arrangements contribute to ILC efficiency in the short term, but hinder its financial sustainability in the long term.

► Recommendation: Coalition Council should prepare a strategy for transition from an IFAD-hosted institution to an independent legal entity with international status.

15

Conclusions

► Most evaluations assess some aspects of G&M, such as accountability, but few assess all aspects

► Using a consistent approach that focuses on compliance with generally excepted principles of public sector governance facilitates comparison across programs

► Evaluators are developing innovative ways to apply this approach and find out how governance is actually being practiced in individual GRPPs

16