EMBEDDING IMPACT ASSESSMENT IN ADVOCACY PROJECTSUnderstanding the value of monitoring, evaluation and learning

Transparency International is the world’s leading non-governmental anti-corruption organisation. With more than 100 chapters worldwide, Transparency International has extensive global expertise and understanding of corruption.

Transparency International UK (TI-UK) is the UK chapter of Transparency International. We raise awareness about corruption; advocate legal and regulatory reform at national and international levels; design practical tools for institutions, individuals and companies wishing to combat corruption; and act as a leading centre of anti-corruption expertise in the UK.

Author: Roberta Falvo

Editor: Lucia Cirimello

Images: The photos in this report were taken during the MEL training conducted in Nairobi, Kenya in November 2019 and are property of Transparency International UK (TI-UK).

We would like to thank Jenny Ross and Brendan O’Donnell for conducting the Monitoring, Evaluation and Learning (MEL) workshop referred to in this report. We would also like to thank Dilara Mehrab Arif for sharing perspectives from the Transparency International Secretariat for this report.

We also thank the following chapters for taking part in the MEL workshop: Integrity Watch Afghanistan, TI Sri Lanka, TI Ghana, Corruption Watch Afghanistan and particularly, those that contributed to the development of the impact stories showcased in this report: Trasparencia Mexicana, TI Kenya and CISLAC Nigeria.

Reproduction in whole or in parts is permitted, providing that full credit is given to TI-UK and provided that such reproduction, in whole or in parts, is not sold or incorporated in works that are sold. Written permission must be sought from TI-UK if any such reproduction would adapt or modify the original content.

Published April 2020.

Every effort has been made to verify the accuracy of the information contained in this report. All information was believed to be correct as of April 2020. Nevertheless, TI-UK cannot accept responsibility for the consequences of its use for other purposes or in other contexts.

This project is funded with UK aid from the UK government.

Transparency International UK’s registered charity number is 1112842.

EMBEDDING IMPACT ASSESSMENT IN ADVOCACY PROJECTSUnderstanding the value of monitoring, evaluation and learning

D Embedding Impact Assessment in Advocacy Projects

Transparency International UK E

CONTENTS

Introduction 1

Common Challenges in Measuring Advocacy Impact 4

Lessons Learned 9

Recommendations: Implementing a culture of Monitoring, Evaluation and Learning

12

Bibliography 14

Impact Stories

Impact Story 1: Working in Coalition for Public Procurement Transparency in Nigeria 3

Impact Story 2: Pledge tracking Helps to Monitor Government’s Conduct in Mexico 7

Impact Story 3: Advocating for Beneficial Ownership Transparency in Kenya 10

1 Embedding Impact Assessment in Advocacy Projects

INTRODUCTION

Background

The Promise to Practice project was born following the 2016 Anti-Corruption Summit hosted in London, from the desire to ensure that the promises of the governments participating in the Summit actually turn into tangible actions and efforts to advance their contribution to the fight against corruption.

Since 2017, the Promise to Practice project has been tracking roughly 170 anti-corruption pledges from over 20 countries around the world through the use of a digital tracking tool, The Global Anti-Corruption Pledge Tracker, and eight national pledge trackers. These were developed to support national advocacy efforts of Mexico, United Kingdom, Nigeria, Kenya, Ghana, Jordan, Afghanistan and Sri Lanka.

In November 2019, three years into the project, Transparency International UK (TI-UK) held a learning workshop with the project’s key partners to assess the impact of the project and draw out key lessons learned in order to ensure the effort’s sustainability and resonance in the future work of the Transparency International (TI) movement as a whole.

Context

The Monitoring, Evaluation and Learning (MEL) Workshop was hosted in Nairobi, Kenya, and two representatives from the nine TI country chapters and partners were invited to participate. The countries that joined the workshop were Mexico, Ghana, Nigeria, South Africa,

Kenya, Jordan, Sri Lanka, Afghanistan and the United Kingdom, as all of them were supported through the resources of the Promise to Practice project. The workshop was facilitated by external consultants, and a representative of the MEL team from the TI-Secretariat (TI-S) was also invited to contribute and lead a session on their tools.

Approach

The workshop had a participatory and interactive format. The content was flexible and easily adjustable to the needs of the participants, as well as to their daily feedback, with meeting the expected outcomes always in mind.

The sessions included hands-on training on monitoring, evaluation and learning tools, as well as strategy building and planning exercises. The workshop also dived deeper into logical frameworks and timelines for impact assessment.

Finally, impact stories were presented as an effective way not only to communicate activities, but also to tie them together to best showcase how they are leading to project objectives, and ultimately, change in society as a whole. The participants put these tools into practice, and openly discussed challenges with peers during the three days. At the end of each day, the group shared lessons learned and objectives for the next day. This allowed contributors to absorb MEL practices step by step and enabled the conversation to evolve based on the ongoing and arising needs of the participants.

Transparency International UK 2

Objectives

Many have long been concerned to better measure the impact of an organisation’s advocacy activities. The MEL Workshop was seen as an opportunity to address this issue. The impact assessment workshop was carried out following an external evaluation. It has, in fact, long been advised that these tools be used together to bring about the best outcome1. Both these processes are really “seen as subsets to a larger, more adaptive policy process”2. In fact, ”impact assessment plays a fundamental role in knowledge creation”, so it is necessary to carry it out in the last year of a project to ensure that the project feeds into the wider arena3.

Specific objectives included:

• To build monitoring, evaluation and learning capacity across TI chapters by presenting and practicing in groups impact and learning tools, developing an advocacy strategy and project planning;

• To share successes and creative ways of addressing challenges by learning to articulate stories of progress and impact and understand how to discern one’s own contribution through the effective use of evidence;

• To improve support from the project centre by discussing challenges in bilateral meetings between TI-UK and all the chapters present.

1 Thomas Diez and Alicia Pfund, ‘An Impact Identification Method for Development Projects Evaluation’ [1986] Review of Policy Research 143.

2 Ibid 2, 144.

3 Sue Phillips and Richard Edwards ‘Development, Impact Assessment and the Praise Culture’ [2000] SAGE 45-65

3 Embedding Impact Assessment in Advocacy Projects

CISLAC (Transparency International’s national chapter in formation in Nigeria) started advocating for open data on public procurement as a result of the grand corruption cases involving the acquisition of Malabu’s Oil Prospecting License 245. The biggest concerns that guided CISLAC’s advocacy strategy were related to allegations concerning revenue losses, conflict of interest and the abuse of power by government officials in the oil procurement sector4.

Nigeria underwent a general election in February last year, and tackling issues of anti-corruption were a key topic during the election campaign. CISLAC worked hard to make this central to the electoral discussion, especially stressing the prevalence of corruption in government in Nigeria and how the corruption scandals within the extractive sector required attention and action. As the current President of Nigeria is also the Federal Commissioner for Petroleum and Natural Resources, CISLAC advocated for the President to sign the OGP, and specifically commit to using open data and an open procurement process. To reach this advocacy goal, CISLAC effectively collaborated with the Open Alliance of the Open Government Partnership (OGP), which is a coalition they pioneered together with other actors. In fact, it was during an event that CISLAC helped organise where various stakeholders agreed to create the Open Alliance in the first place and developed a first draft of the Anti-Corruption Strategy, which subsequently fed back into the actual National Anti-Corruption Strategy adopted by the Nigerian Government. Working in coalition was fundamental for CISLAC to be able to enact an open procurement process in the country. A contributing factor was the good relationship that CISLAC developed with the Nigerian government. Thanks to this trust, CISLAC was the advising body that drafted the President’s presentation for the London Anti-Corruption Summit, hence continuing to support the government Anti-Corruption Strategy from several angles.

CISLAC’s leadership, network and collaboration with other civil society organisations working on anti-corruption was key to build the capacity to advocate for the government to sign the OGP and in committing to set up a central public procurement portal (NOCOPO). The portal has been established and it is ready for public use. However, as it is only in its initial stages, at the moment is only being used for the Health and Education sectors. There is an annual public procurement presentation to the press, and a quarterly reporting mechanism in which CISLAC contributes through monitoring the utilisation of the portal by the different

4 The Guardian, Nigeria, 20 January, 2020 Malabu oil deal: FG files fresh charges against Adoke, Shell, others

sectors and its use by the general public. Next steps for CISLAC’s advocacy strategy will focus on the reviews of the operability of this platform. In this process, CISLAC has successfully incorporated lessons learned and leveraged national interests keeping into account the general public’s requests, as well as the intense and complex external political landscape to obtain the desired result. As a consequence of these, advocates in Nigeria have started working more effectively to create political will by leveraging the understanding that the government officials will be more keen on listening and collaborating when they see political gain.

While it is premature to assess the change that the NOCOPO has produced and further efforts need to be made in keeping the website operational, public procurement data is evidently now more accessible to the public than ever before. All the entities involved in the procurement value chain are being incorporated into this process, hoping for reduced conflicts of interest and abuse of power by government officials, which were the guiding concerns to this advocacy strategy and execution.

CISLAC’s contribution to this process can be clearly assessed when checking the OGP progress reports along the advancements made subsequently to the 2016 Anti-Corruption Summit on their digital tracking platform www.tracnigeria.ng. It is worth noting that CISLAC’s contribution is specifically stated within the OGP annual plan and the annual progress reports. CISLAC has worked closely with various CSOs as well as government bodies, namely the Public Private Development Centre (PPDC), the Nigeria Extractive Industries Transparency Initiative (NEITI), OGP secretariat and various actors within the Nigerian government and will continue to work with them for the sustainability and constant improvement of the central public procurement portal.

WORKING IN COALITION FOR PUBLIC PROCUREMENT TRANSPARENCY IN NIGERIA

Impact Stories

Transparency International UK 4

COMMON CHALLENGES IN MEASURING ADVOCACY IMPACT

Monitoring, Evaluation and Learning in Advocacy ProjectsAdvocacy does not work in linear ways; change happens in complex and unexpected patterns. A change of government, a referendum, an outbreak of violence or even an epidemic may massively impact the work of an advocate on the ground. External factors, contextual elements, relationships and power structures can determine how effective activists, advocates and civil society organisations (CSOs) are in influencing change, because “advocacy, is, in essence, about getting others to think or behave differently”5.

The contingent nature of advocacy is especially problematic when trying to measure and evaluate impact. Taking the time to assess impact is fundamental for advocates to understand whether what they are doing is making a difference, as well as to learn ”what works”6.

There are, in fact, lessons to be learned on the replicable approaches despite the external context. More efforts should be invested in forming creative, strategic and resilient advocates who are capable of adjusting to the changing external environment. This can be achieved through better monitoring, evaluation and learning, celebrating success and collaboration.

5 Jim Coe and Rhonda Schlangen, ‘No Royal Road: Finding and following the natural pathways in advocacy evaluation’ [2019] The Centre for Evaluation and Innovation 5.

6 Jenny Ross, ‘Supporting Learning? Exploring the relationship between grantee learning and grant making practice in the transparency and accountability sector’ [2015] INTRAC.

7 The examples used in this report reveal the challenges specific to the accountability and transparency sector.

8 Peter Oakley, ‘The Evaluation of Social Development’ [1990] Oxfam 27-36.

Common Challenges

However, reinforcing organisational learning is never easy. It is a long-term process that requires gathering information through a variety of sources, this is exactly because change is going to take extended timeframes and involve multiple actors. There are barriers to assessing impact and learning in advocacy projects, including some challenges specific to the accountability and transparency sector7.

Nonetheless, if identified, these obstacles can be mitigated. This section will focus on how the difficulties encountered by the contributors of the Promise to Practice Project fit into wider narratives and how they have been addressed through a variety of recommendations and creative ideas to instigate a sustainable culture of change-making through learning.

1. The battle between objectivity and subjectivity

One point of criticism towards impact assessment “revolves around the twin issue of attribution and causality”8. There is a commonly accepted and acknowledged difficulty in attributing outcomes to project’s activities.

5 Embedding Impact Assessment in Advocacy Projects

Early approaches to measuring impact employed social cost-benefit analysis, environmental and social impact assessment, tools often built from outside, grounded on assumptions and claims of objectivity. This, in turn, seemed to favour a bias towards positive outcome, ultimately proving to be limited in evaluating “non-tangible and non-material broader development objectives”9. It is important to keep in mind that there is judgement employed in this process: the so-called battle between objectivity and subjectivity10.

Specifically, in the context of transparency and accountability advocacy, it has been found that successes are the result of short, medium and long-term strategies operating at multiple levels. Therefore, when assessing a project, equal attention and importance should be given to all the dimensions of change. Ultimately, impact measuring should be qualitative, dynamic, participatory, continuous and sustained11.

“In any case, not being able to state contribution isn’t necessarily a sign of a weakly delivered campaign. Indeed, this will sometimes be a positive sign. Given what we know about how change happens, there are many cases (particularly more intractable issues) where it’s not possible — and not desirable — to see a single organization’s specific fingerprints on an outcome because ideally that organization will have been successful in bringing in other actors. Rarely would one actor be identifiable/singled out in a dynamic movement”12.

9 Ibid 6, 29.

10 Chris Roche, ‘Impact Assessment: seeing the wood and the trees’ [2001] Debating Development 365.

11 Ibid 5.

12 Jim Coe and Rhonda Schlangen, ‘No Royal Road: Finding and following the natural pathways in advocacy evaluation’ [2019] The Centre for Evaluation and Innovation 27

13 Ibid 5.

14 Ibid 4.

15 ibid 4, 10. The table curated by Jim Coe and Rhonda Schlangen includes: · Raw data and information: These are various details about activities of advocates that can be verified illustrating content and magnitude of activities but does not in itself provide evidence

of influence. · Filtered information: This is typically the type of information that a campaign might track or report to funders. It provides highly valuable insight as it comes from advocates who are close to

the work and are necessarily well-informed by deep and rich experience. Yet, this evidence is limited because it is filtered by the perspective and biases of the author.· Informed opinion: Views, based on observation or participation, about what happened, the influences involved, and the significance of developments vary according to the actors. They

should be aggregated to provide a helpful composite picture.

16 Ibid 4.

Newer qualitative approaches to assess impact and learning should be favoured over outdated quantitative ones that fail to understand the multifaceted nature of change. Impact assessment has since been defined as “the systemic analysis of the lasting or significant changes – positive or negative, intended or not – in people’s lives, brought about by a given action or a series of actions”, thereby shifting the focus towards the collaborative change-making processes and their impact in people’s lives, whether positive or negative13. In this process, an evaluation should remember to incorporate and respect doubt by “being much more explicit in recognizing and factoring in the barriers to certainty and embracing uncertainty as a result”14.

The workshop enabled the project’s participants to be reminded that the contingent nature of advocacy should be appreciated in the project design of each activity and outcome. As a result, our partners have been employing a flexible approach in how they design their activities and adjust them and learn from mistakes along the way.

The table below15 explores the different sources that will contribute to shaping information for evaluation. Instead of claiming impartiality and full knowledge, one should factor filters, such as opinions and judgment, which will make the MEL approach “credible, reliable and instructive”16.

Transparency International UK 6

The table named “Sources to draw on in an evaluation” is curated by Jim Coe and Rhonda Schlangen and is included in the publication “No Royal Road: finding and following the natural pathways in advocacy evaluation”.

2. Moving from attribution to contribution

Participatory approaches and the involvement of partners and stakeholders in gathering the evidence and triangulating information is key to capture the full story of change. Bringing all the project’s partners together shifts the lens from the amount of contribution a single actor makes to an understanding of the typologies of the different actors contributing to change17. The workshop was effective in stimulating participants to understand the existing will to move from attribution to contribution. This represents in itself a movement towards the use of narration to grapple with the various contributions. This way of assessing impact is fundamental ‘to building strategic alliances between NGOs and the state to produce significant change in people’s lives’18. It is, in fact, the ‘insiders’ and ‘outsiders’ that together create a story of impact. Far from serving exclusively the purpose of ‘upward’ accountability, this activity supported both “tactical and strategic learning”19.

To visualise and communicate progress, it is not only important to understand what progress looks like but also fundamental to have a good strategy in place. MEL can be referred to as “Impact and Effectiveness”, as this allows for a change in the perception that exists towards MEL being a cumbersome procedure required by donors. In many organisations the Monitoring & Evaluating officers sit in a different department from the advocacy teams. Traditional MEL tools make the collection of some data burdensome for advocacy teams, as this exercise is often not readily useful to them. Failure to show results can undermine both internal and external relationships of trust among colleagues and between grantees and grant-makers. Conversely, if done properly, monitoring, learning

17 Ibid 4, 29.

18 Ibid 5, 373.

19 Jim Coe and Juliette Majot, ‘Monitoring, Evaluation and Learning in NGO Advocacy: Findings from Comparative Policy Advocacy MEL’ [2013] ODI and Oxfam.

20 Ibid 5.

and evaluation can be extremely useful to build effective advocacy strategies: assessing impact serves the purpose of guiding future planning and forecasting, building confidence in all the actors involved. MEL has the potential to equip experts with the tools to face the constant challenges posed by the volatile nature of politics.

3. Communicating complex issues

The challenge of effectively communicating about the technical issues related to corruption both to the government and the general public is a commonly encountered barrier to conveying change and impact.

Specifically, communicating to the public in the accountability and transparency sector can be difficult due to the lack of understanding and involvement in the fight against corruption by citizens. This in turn is a major obstacle to making the matter a priority on the government’s agenda.

When it comes to communication between partners, and grantees, there is an over-reliance on written forms of communication (proposals, reporting) which are not sufficient to develop relationships of trust and openness, which support the sharing of learning20. Peer learning and external relationships can shift the perspective and fuel communication in new creative ways, helping grantees to analyse shifts in their context.

Comparing and contrasting practical experiences from various contexts and backgrounds has enabled the project’s partners to think of alternative ways of communicating their issues to different audiences with varying degrees of knowledge or roles in society.

7 Embedding Impact Assessment in Advocacy Projects

The government of Mexico promised to implement fifteen pledges to combat corruption at the 2016 Anti-Corruption Summit. The Mexican chapter of Transparency International, Transparencia Mexicana, participated in the global anti-corruption pledge tracker and also developed a separate online tracker to monitor and raise awareness of the process, which additionally facilitated the chapter’s national advocacy work. Transparencia Mexicana has, in fact, been involved in the monitoring of international anticorruption commitments since 2009. Tracking commitments has allowed the organisation to establish communication with the Mexican government in order to advocate for their implementation. Advocates on the ground have contacts with different federal agencies from the Mexican government, particularly, the Minister of Public Administration (SFP for its Spanish acronym), the Minister of Foreign Affairs (SRE), the Executive Secretariat of the National Anticorruption System (SESNA) and the Ministry of Finance. The advocates’ strategy includes the elaboration of shadow reports of different international conventions and commitments21. 

Since 2016, the anti-corruption pledge tracker was an important tool in order to make the government aware that their commitments were being monitored internationally along those of many other countries. Transparencia Mexicana coordinated regular meetings with the government agencies and related actors in order to understand the status of each commitment and the role of other actors involved. The change of government at the end of 2018 reconfigured the political context of the country, and with that, the priorities of the new administration. The new government focused particularly on domestic affairs. Therefore, the fight against corruption became a priority. Transparencia Mexicana contacted the new members of the administration in order to raise awareness of the importance of Mexico’s international anti-corruption commitments, including the Summit Commitments. As a result, the anti-corruption pledge tracker also allowed the team of Transparencia Mexicana to broaden the communication with the new government to exchange information about the commitments made in 2016 and their status.

21 Mexico has signed different anti-corruption instruments, including: The United Nations Convention against Corruption (UNCAC), the Inter-American Convention against Corruption (IACC), the OECD Convention on Combating Bribery of Foreign Public Officials in International Business Transactions and different international commitments.

22 For more information about the National Anticorruption System access: https://sna.org.mx/en/about-us/

23 With the creation of the National Anticorruption System, and the reform of the Constitution, other seven secondary laws were reformed (https://www.tm.org.mx/reformasna/). One of these included the General Law of Administrative Responsibilities, commonly known as Ley #3de3. This citizen initiative created by the broad coalition of citizens and CSOs, including TM, defined different types of conducts that should be deemed acts of corruption and established firm punishments for these. These range from removal from public office to compensatory and punitive damages, and permanent restriction from holding any public sector position. Amongst other things, Ley #3de3 also demanded that all members of Congress and Government officials, including the President himself, make three declarations public: Assets, potential conflicts of interest and taxes. The National Digital Platform will be the repository of these declarations.

The project and the tracker have acted as entry points to start a conversation about other international conventions and commitments with the new government in Mexico and, three years since the summit, commitments continue to make progress. For example, the new government announced in August 2019 that they will develop a New Digital Platform for Public Procurement. The government stated that the platform is being developed to generate better transparency in different topics. It will also publish information under Open Data Contracting Standards, which was implemented already in the previous government with the participation of civil society. This Platform will ensure the reliability of information, open competition and reduce barriers to the public procurement.

Also, after the anti-corruption reforms of 2014 implemented by a broad coalition integrated by the government, civil society, academia and the private sector, the National Anti-Corruption System (NACS)22 was created. With this, the SESNA was formed and it is in the process of developing a National Digital Platform which is an interoperability tool that will work with data provided by different governmental agencies at the federal and local levels, and by autonomous entities in order to crosscheck strategic information and generate intelligence to fight corruption. This Platform includes the first ever asset declaration23 open data standard, which will be applied in all asset and conflict of interest declarations made by public officials. This standard will allow anti-corruption authorities to use and crosscheck open data form asset declarations for the first time in history. The National Digital Platform will soon publish a new version of the standard, as well as different technological tools and subsystems.

Regarding beneficial ownership, the 4th Action Plan of the Open Government Partnership (OGP) includes a new commitment that was co-created between the government and civil society. Transparencia Mexicana has participated in creation and implementation of this commitment, which pledges to create a beneficial ownership register of mining, oil & gas concessions.

PLEDGE TRACKING HELPS TO MONITOR GOVERNMENT’S CONDUCT IN MEXICO

Impact Stories

Transparency International UK 8

In 2019, out of the 15 commitments that Mexico pledged during the London Summit, nine of them are ongoing, four of them are classified as having “no data”, one is complete and one is inactive. Progress has been heavily influenced by the tracking process, as Transparencia Mexicana ensured better coordination and communication across different federal agencies. Additionally, the 2016 Summit commitments are only some of the commitments that the Mexican government has pledged internationally in relation to anti-corruption. Transparencia Mexicana is using the pledge tracker to give the government an overview of their commitments and to share information across government agencies. The Mexican TI chapter has identified an area in which such digital tools, if improved, enlarged in scope and used by local advocacy experts, would offer a great service to the government, CSOs, citizens and society as a whole.

9 Embedding Impact Assessment in Advocacy Projects

LESSONS LEARNED

1. Embedding MEL in work routines

The workshop was a safe space for all the participants to freely share difficulties without feeling that relationships may be jeopardised. On the contrary, TI chapters played the role of ‘‘critical friend’’ for one another, which led to an amplified understanding of how learning can be strengthened by building critical and adaptive capacities within the movement.

The workshop provided an opportunity to understand how to measure impact. It was aimed at engaging advocates to become experts in observing and analysing change whilst influencing the change-making process. The most effective way to do this is to embed MEL practices within work routines. Data should be gathered exclusively based on its utility to the project’s activities, and MEL processes and tools should be well understood and accessible to everyone. This process features building of the institutional memory of advocacy, for instance the use of narrative stories to explain impact. A curated compilation of impact stories from a number of countries has been included in this report.

2. Assessing transformational advocacy

Evaluating advocacy, and especially transformative advocacy can be a tricky process because it works on a spectrum24. At one end, there are clearly structured actions that are undertaken to lead to policy change, such as navigating the political institutions in place in order to propose or change a law. At the other end of the spectrum there are subtler processes which, while slower, tackle a more deeply rooted problem. An example of the increasing complexity of change in advocacy work is the case of a TI chapter that is working to educate young people to change the embedded culture of corruption in order to build a movement for change. Similarly, in many of the contexts of the project’s partners, the political space of influence and the institutionalised political structures are

24 Jenny Ross, ‘Supporting Learning? Exploring the relationship between grantee learning and grant making practice in the transparency and accountability sector’ [2015] INTRAC.

25 Ibid 16.

26 Ibid 16

difficult to navigate and the CSO space is shrinking. This is evident in fragile states and conflict-affected countries, as well as in those that lack a well-established democratic system.

Advocates should not be discouraged from addressing the more deeply rooted problems. On the contrary, they should think strategically and creatively to build longer-term, visionary advocacy strategies that involve a number of actors contributing to more transformational outputs and objectives. Building advocates’ resilience and creative thinking should be in the interest of all donors. By developing long lasting relationships of trust with its partners, TI-UK has been able to rebalance these relationships and receive honest feedback that has enabled the team to provide useful support to the partners on the ground. Strategies for transformational advocacy engage a variety of actors and actions in an extended period of time, and while harder to build, they are the type of strategies that should yield valuable spillover effects for wider anti-corruption efforts.

3. Taking account of uncertainty and planning for unpredictable change

Depending on the work to be assessed, there are tools and small changes that a team, or an organisation as a whole, can incorporate into their daily activities to ensure this information is gathered effortlessly. The workshop presented MEL systems that could be built-in routines and ways of working that are already in place hence “operating in rhythm with existing organizational processes”25. In the impact assessment workshop, TI-UK employed this approach to include ”invisible informal processes [to ensure that] important tacit knowledge and innovation” is acknowledged and preserved26. The impact assessment approaches should be a mix of tools and systems that look into other dimensions of change, better consider the uncertainty of the context, and plan for unpredictability.

Transparency International UK 10

Kenya was one of the five Sub-Saharan countries that made commitments at the 2016 Anti-Corruption Summit. The government of Kenya made 8 commitments, four of which are on beneficial ownership. In 2018, five of the commitments were ongoing and three were inactive. Transparency International Kenya (TI Kenya) played a pivotal role in the advancement of the commitments since the Summit, especially on beneficial ownership, through the use of a digital tool, their online national pledge tracker.

Right after the Summit, TI Kenya noticed a lot of international enthusiasm around the topic of beneficial ownership transparency. Kenya’s corruption scandals about the government doing business with alleged shell companies resonated in national and international newspapers. The scandal that saw the arrest of the head of the National Youth Service, for the theft of 8bn KES (£49 m and $78 m) 27 pushed citizens, civil society, journalists and parliamentarians to reform. TI decided to carry out an assessment of the beneficial ownership legal framework in Kenya. This assessment, completed in 2017 by an external legal firm, JMiles, resulted in a series of recommendations.

In 2016 and 2017, TI Kenya was one of the few civil society organisations that was actively advocating in favour of beneficial ownership transparency (BOT). After a network mapping exercise, TI Kenya organised a series of consultations with the identified actors. The first one was at a regional meeting held in Accra in 2016 alongside Ghana and Nigeria to discuss countries’ commitments on beneficial ownership pledged at the 2016 Anti-Corruption Summit. Following this encounter, a second meeting took place in April in 2017 in Naivasha, just a few hours north of Nairobi, and it was mainly aimed at raising awareness of beneficial ownership transparency among key players from the government, civil society organisations and technical groups.28 Finally, the third forum took place in August 2019, to communicate and validate the findings of the assessment of the current legal frameworks on this topic in Kenya. This strategy was fundamental in powering the advancements

27 BBC news: “Head of Kenya agency arrested in $78 million scandal” - https://www.bbc.co.uk/news/world-africa-44280453

28 There were representatives from the International Commission of Jurists in Kenya, Hivos, the Office of the Attorney General and the Department of Justice-Business Registration Services (BRS), the Kenya National Human Rights Commission, the Commission on Administrative Justice, the Federation of Women Lawyers-Kenya, the Kenya Revenue Authority, Infonet, the Public Procurement Regulatory Authority, the National Police Service with the Criminal Investigation Division and Article 19 East Africa, Financial Reporting Center and GIZ, among others.

29 “beneficial owner” means the natural person who ultimately owns or controls a legal person or arrangements or the natural person on whose behalf a transaction is conducted, and includes those persons who exercise ultimate effective control over a legal person or arrangement. Find this definition on page 571 of the Kenya Gazette Supplement No. 121 (Acts No 28), 2017

30 3. Acquiring Beneficial Ownership Information:• Fast track the removal of section 104 (1) of the Companies Act which states that “A company shall not accept, and shall not enter in its register of members, notice of any trust,

expressed, implied or constructive” and sub-section 2 which makes it an offence to convene sub-section 1• Make explicit the obligation of shareholders to declare information on beneficial ownership and sanction the failure or delay of providing it.

31 Referenced from the table on page 15 within the Kenya OGP NAP III, more information on pages 17 and 18.

32 4. Access to Beneficial Ownership: Regulate the access to register of members of private companies to make it open for inspection by any person, as in the case of public companies.

that Kenya underwent in relation to this subject. As a consequence of these meetings, the Business Registration Service (BRS) became a “BOT champion” and followed up with the commitments ever since. Specifically, the BRS pursued the 2017 amendment to the 2015 Companies Act. The Companies (Amendment) Act, 2017, now included a definition of a beneficial owner29 and required that every company keeps a register of its members, with information relating to the beneficial owners of the company. This had been part of TI Kenya’s third30 recommendation in their assessment. The new amended legislation was a step in the right direction, as now there is much less ambiguity in the definition of beneficial owners. Even though regulations to guide implementation of the act are yet to be approved by the National Assembly, TI Kenya continues to support whenever required.

Furthermore, TI Kenya is listed as one of the main actors involved in the Kenya OGP Action Plan 2018-2020. It was TI Kenya, in fact, who championed the inclusion of the beneficial ownership commitment in the 2nd and 3rd OGP National Action Plans (NAP). Its staff participated in the collaborative process that led to the creation of the 3rd OGP NAP and followed up on the activities in the 2nd NAP that still needed more work. TI Kenya ensured that incomplete activities would be included in the 3rd NAP, which created an important element of continuity of the process. In the 3rd OGP NAP, the beneficial ownership commitment by the Kenyan Government states: “we will publish a central register of company beneficial ownership information operating in the Republic of Kenya”31. Additionally, many of the beneficial ownership milestones included in the OGP national action plan, such as “conducting a risk assessment,” are taken from the 2017 TI Kenya assessment, especially the idea of creating the central register of both national and foreign companies easily accessible to any person32.

Without the 2016 Anti-Corruption Summit, funding dedicated to follow up, or the TI movement’s global reach, this work would have not been pursued with such dedication and

ADVOCATING FOR BENEFICIAL OWNERSHIP TRANSPARENCY IN KENYA

Impact Stories

11 Embedding Impact Assessment in Advocacy Projects

expertise. TI Kenya’s advocates leveraged the OGP’s attention to the assessment, which resulted in many of its recommendations being included in the most recent OGP National Action Plan. The 2016 Anti-Corruption Summit, the National Pledge Tracker, and the Assessment were able to generate political will in a diplomatic manner.

For instance, in June 2018, the Attorney General issued a series of 17 regulations to bring forward Beneficial Ownership Transparency. Along the 17 regulations, the latest and most positive development in relation to Beneficial Ownership is strictly tied to public procurement. In June 2018, the Presidency issued an executive order (2/2018) on Procurement of Public Goods, Works and Services by Public Entities. It requires public procurement agencies to provide details of companies they are doing business with, including beneficial ownership information. While a public procurement portal (PPIP) is in place, the information on beneficial owners is not yet published on it as the BRS has not yet started collecting this information.

It is important to keep these developments in mind because even though more work clearly needs to be done in Kenya, the 2016 Summit, the TI movement as a whole and the OGP have facilitated the creation of a strong support system to achieve this in the foreseeable future. TI Kenya continues to remain in close contact with BRS and engages with them through the OGP and other forums to ensure the regulations are adopted.

Transparency International UK 12

RECOMMENDATIONSImplementing a culture of Monitoring, Evaluation and Learning

The following series of tools and techniques that were tested and explored during the workshop, are designed to address the challenges outlined in this paper, and incorporate the lessons learned. We have found them to be a practical aid in seeking to embed impact assessment in our advocacy work.

We recommend the following advocacy tools, from which to select based on your needs:

Forcefield Analysis

Describe a context and choose categories of factors (political, social, technological, environmental, economic etc.) that may obstruct or facilitate change.

• Similar to political economy analyses and timeline building exercises;• Reflects on the factors that are blocking change from happening and those enabling the change;• Helps advocates to ensure they have a full detailed picture and are targeting/tackling the right

factors.

System Mapping

Graphically build a web of stakeholders representing the various layer of influence to reach the actor that is fundamental to enabling change.

• Example: In one the chapters the anti-bribery commission was not collaborating, so the advocacy team tried to better understand the layers of power. Through a mapping exercise they were able to understand and build relationships with the “second in command”.

• Example: Our partner in Nigeria was great at understanding and influencing networks to achieve open procurement. They collaborated with all concerned through the ‘Open Alliance’.

Iceberg Tool

Start by mapping the present, or the tip of the iceberg, to then dig deeper to understand the underlying structures of a problem. Subsequently, move from the bottom up to come up with strategies to tackle all the layers of a problem.

• Helps advocates to go under the surface of an issue;• Useful to think creatively about possible futures;• Allows advocates to understand what negative behaviours are deeply embedded and are

obstructing change.

13 Embedding Impact Assessment in Advocacy Projects

Progress and Impact Log

Before/after team meetings introduce the good practice of sharing impact, knowledge and learning and keep a shared document including this information.

• When change happens, this helps advocates to understand how they were involved in it. Looking back at external and internal meetings and conversations will allow changemakers to understand how who they met and what they said has had an impact.

• Advocates should carry out this exercise not in relation to external contexts but only to the factors they can control or influence.

Contribution scale

Build a scale of contribution to single out your input to change. This could be from 0 to 3, where 0 = no contribution, 1 = little contribution; 2 = medium contribution, 3 = significant contribution

• There are various layers of contribution and this scale would help advocates measure their level of confidence in having influenced change.

• This is especially useful when a plethora of work is done in coalitions of several actors to identify the contribution of an organisation to the coalition’s win.

Pre-mortem tool

Fast forward to a point in the future of your strategy and imagine that you have completely failed to achieve the goals you set out.

• Useful to understand what may lead to failure and hence troubleshoot and think strategically. • Advocates should carry out this exercise not in relation to external contexts but only on the factors

they control or can influence.

Impact Stories

Impact stories are an effective way of explaining how change happens. Stories enable the advocate to describe the richness and complexity of change, demonstrate impact, define contribution and identify effective approaches and challenges.

• Use impact stories to harness and share knowledge - narrating impact can be an illuminating exercise to reassess assumptions and identify gaps in feedback and evidence, readjusting the plan as a consequence.

• Story telling should take a considerable amount of time building trust and gathering information through personalised, interactive means of communication, such as sets of interviews with a variety of stakeholders involved in the change. This technique is called triangulation.

Transparency International UK 14

Fill in the missing middle: Demonstrating impact requires advocates to fill in the ‘missing middle’ – to link their activities and contribution to the overall change they hope to achieve through the identification of signs of progress. There is often no clear, simple causal link between activities and the change. Explaining and providing evidence of contribution is the most effective way to understand the relevance of project activities to a successful outcome.

This can be done by reconstructing the story of change from the beginning: start with the initial goal, break down the subsequent chronological chain of activities of the organisation, link them up, add other actors involved, triangulate your findings and ask questions probing the small, incremental progress each of these activities had until the team understands how these created the change.

Remember that small changes matter: In order to capture the activities’ contribution to change, advocates should be reminded that small changes matter and need to be valued as part of advocacy and campaign strategies. While change can often only be understood retrospectively, forecasting and imagining change helps to have a clear sense of direction to guide strategies. As a result, a strategy should always identify short, medium and long-term changes, which would not only help advocates measure progress but also indicate that small incremental progress is valuable.

For example, convening a meeting with stakeholders could be the starting point for a particular reform, hence it is always useful to keep track of these, to be able to account for a wider change years later.

Customise and adapt the tools to your context: The tools presented here, and many more, are adaptable and customisable to contexts and organisations’ structures. Implementing some or all of these tools should demonstrate that an organisation and its donors are truly committed to monitoring impact and effectiveness for learning.

For instance, the Forcefield analysis tool is key for considering both positive and negative factors in planning, and what it reveals is that there is something to learn both from successes and failures.

Keep in mind that monitoring does not mean measuring: It is indeed important to remember that monitoring doesn’t mean measuring. Measuring impact requires monitoring in a qualitative, dynamic, participatory, continuous and sustained manner. The approach should be heuristic and holistic, in the sense that the evaluation ‘evolves’ continuously with the evaluator’s knowledge of the project and it gives equal attention to all the dimensions of the change.

Change the organisation’s perception of MEL: MEL can be referred to as “Impact and Effectiveness”, allowing for a change in perception. It should not be viewed as a cumbersome procedure required by evaluators because, if done properly, it can be extremely useful to build effective advocacy strategies by assessing impact to instruct future planning and forecasting. Monitoring impact and effectiveness should be about drawing out the clear implications of the learning process in order to subsequently act on them and bring more value to future advocacy.

For instance, a more collaborative process where colleague continually share the way in which impact reporting has been used for wider purposes (communication tools, donor reports, funding proposals) could be beneficial for advocates to feel engaged and motivated to play an active part in these activities.

Make it a collaborative and participatory exercise: Strategy building, prior and subsequent to a MEL exercise, should be done in a clear, collective and participatory manner, and the MEL tools should be embedded in the strategy. Far from creating more space for MEL out of busy schedules, the best ways could be simple and straightforward, such as adding an agenda item to existing meetings or email chains to introduce quick processes of reflection.

15 Embedding Impact Assessment in Advocacy Projects

BIBLIOGRAPHYChris Roche, ‘Impact Assessment: seeing the wood and the trees’ [2001] Debating Development 359-373

David Hulme, ‘Impact Assessment Methodologies for Microfinance: Theory, Experience and Better Practice’ [2000] World Development 79-95

Jenny Ross, ‘Supporting Learning? Exploring the relationship between grantee learning and grant making practice in the transparency and accountability sector’ [2015] INTRAC

Jim Coe and Juliette Majot, ‘Monitoring, Evaluation and Learning in NGO Advocacy: Findings from Comparative Policy Advocacy MEL’ [2013] ODI and Oxfam

Jim Coe and Rhonda Schlangen, ‘No Royal Road: Finding and following the natural pathways in advocacy evaluation’ [2019] The Centre for Evaluation and Innovation

Peter Oakley, ‘The Evaluation of Social Development’ [1990] Oxfam 27-36

Sue Phillips and Richard Edwards ‘Development, Impact Assessment and the Praise Culture’ [2000] SAGE 45-65

Thomas Diez and Alicia Pfund, ‘An Impact Identification Method for Development Projects Evaluation’ [1986] Review of Policy Research 137-145

Transparency International UK10 Queen Street Place, London, EC4R 1BE

www.transparency.org.uktwitter.com/transparencyuk