Climate Risk Management 12 (2016) 1–16

Contents lists available at ScienceDirect

Climate Risk Management

journal homepage: www.elsevier .com/ locate/crm

Priming adaptation pathways through adaptiveco-management: Design and evaluation for developingcountries

http://dx.doi.org/10.1016/j.crm.2016.01.0012212-0963/� 2016 The Authors. Published by Elsevier B.V.This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

⇑ Corresponding author. Tel.: +61 2 6776 1358; fax: +61 2 6776 1333.E-mail address: james.butler@csiro.au (J.R.A. Butler).

J.R.A. Butler a,⇑, W. Suadnya b, Y. Yanuartati b, S. Meharg c, R.M. Wise c, Y. Sutaryono d,K. Duggan e

aCSIRO Land and Water Flagship, EcoSciences Precinct, GPO Box 2583, Brisbane, QLD 4001, Australiab Faculty of Agriculture, University of Mataram, Jl. Majapahit 62, Mataram 83127, Nusa Tenggara Barat Province, IndonesiacCSIRO Land and Water Flagship, Black Mountain, Canberra, ACT 2911, Australiad Faculty of Animal Sciences, University of Mataram, Jl. Majapahit 62, Mataram 83127, Nusa Tenggara Barat Province, IndonesiaeGriffin NRM, Suite 15 Baileys Corner, Arcade 143, London Circuit, Canberra, ACT, Australia

a r t i c l e i n f o

Article history:Received 25 April 2015Revised 19 December 2015Accepted 11 January 2016Available online 18 January 2016

Keywords:Climate compatible developmentGovernanceIndonesiaResearch for developmentSocial learningTheory of Change

a b s t r a c t

Mainstreaming climate change and future uncertainty into rural development planning indeveloping countries is a pressing challenge. By taking a complex systems approach todecision-making, the adaptation pathways construct provides useful principles.However, there are no examples of how to operationalise adaptation pathways in develop-ing countries, or how to evaluate the process. This paper describes a 4 year governanceexperiment in Nusa Tenggara Barat Province, Indonesia, which applied adaptive co-management (ACM) as a governance approach to ‘prime’ a transformation to adaptationpathways-based development planning. The project’s Theory of Change (ToC) consistedof three causally-linked phases which mirrored the evolutionary stages of ACM: primingstakeholders, enabling policies and programs, and implementing adaptation. The firstphase established a trans-disciplinary research team to act as facilitators and brokers, amulti-stakeholder planning process demonstrating adaptation pathways practice, and tri-alling of ‘no regrets’ adaptation strategies in case study sub-districts. A participatory eval-uation method was designed to test the ToC’s assumptions and measure ACM outcomes.Stakeholder interviews at the project’s closure indicated that through ACM, stakeholdershad been successfully primed: leaders emerged, trust, cross-scale social networks andknowledge integration grew, communities were empowered, and innovative adaptationstrategies were developed and tested. However, there was limited evidence of institutionalchange to existing planning processes. This was attributed to the absence of policy win-dows due to ineffective and insufficient time for political engagement, and the fluid insti-tutional environment caused by a national decentralisation policy. To enhance the primingof adaptation pathways into development planning under these conditions, three recom-mendations are made: (1) provide long term support for emergent leaders and brokersto become ‘policy entrepreneurs’ who can capitalise on policy windows when they appear,(2) establish and support local livelihood innovation niches as ‘bridgeheads’ for ACM, and(3) maintain participatory evaluation amongst primary stakeholders to re-kindle ACM.� 2016 The Authors. Published by Elsevier B.V. This is an open access article under the CC

BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

2 J.R.A. Butler et al. / Climate Risk Management 12 (2016) 1–16

Introduction

Mainstreaming climate change into decision-making to achieve ‘climate compatible development’ for vulnerable ruralcommunities in the developing world is a pressing challenge (Mitchell and Maxwell, 2010). The process is complicated bythe uncertainties in climate change projections and impacts (Ranger and Garbett-Shiels, 2012), and the multi-stakeholderand cross-sectoral nature of adaptation (Conway and Mustelin, 2014). Population growth, market volatility and modernisa-tion driven by globalisation also interact with climate change to generate non-linear and unexpected outcomes and shocksfor livelihoods (Scoones et al., 2007; Leach, 2008). Ramalingam (2013), p. 361 recently argued that as a consequence, devel-opment planning and evaluation requires a more ‘‘systemic, adaptive, networked, dynamic approach” based on complex sys-tems thinking and governance.

By taking a complex systems approach, the construct of adaptation pathways provides a useful framework for integratingclimate adaptation into decision-making under future uncertainty (Wise et al., 2014), and is potentially applicable to ruraldevelopment planning in developing countries (Butler et al., 2014, 2016a). There are three central principles. First, climatechange impacts and responses cannot be considered in isolation, but instead are components of dynamic, multi-scalesocial–ecological systems. Second, planning should design incremental adaptation strategies to address the proximatesymptoms of communities’ vulnerability, plus transformative strategies to tackle the underlying systemic causes, whichin developing countries are often institutional and political (Lemos et al., 2007; Pelling, 2011; Rodima-Taylor et al., 2012).Third, to avoid mal-adaptation (i.e. actions that impact adversely on or increase the vulnerability of other systems, sectorsor social groups; Barnett and O’Neill, 2010), strategies should yield benefits under any future conditions of change, andtherefore be ‘no regrets’ (Hallegatte, 2009).

The process of implementing these principles requires multi-stakeholder engagement and adaptive governance (Butleret al., 2014), whereby stakeholders voluntarily coordinate action through self-organisation (Folke et al., 2005). Adaptiveco-management (ACM) is a novel manifestation of adaptive governance tailored to the stewardship of complex social-ecological systems, where multi-stakeholder collaboration is required to match the ecological scale and dynamics of a sys-tem (Armitage et al., 2009). By integrating the iterative learning, knowledge generation and problem solving of adaptivemanagement with the power-sharing and negotiated decision-making of co-management (Olsson et al., 2004a; Armitageet al., 2007; Plummer et al., 2012; Fabricius and Currie, 2015), ACM fosters stakeholders’ adaptive capacity (Plummer,2013), defined here as ‘‘the potential for actors within a system to respond to changes, and to create changes in that system”(Chapin et al., 2006, p. 16641).

ACM is potentially applicable to climate change adaptation because of the necessity to engage private and public actorsacross societal levels, to build their adaptive capacity, and to develop and implement policy and collective action throughcollaboration (Plummer, 2013). ACM’s characteristic focus on learning is especially relevant because both individual andsocial learning are pre-requisites for adaptation (Pelling, 2011). Within this, double-loop (re-visiting of assumptions aboutcause and effect) and triple-loop learning (re-assessing underlying values and beliefs, potentially resulting in changes toinstitutional norms; Pahl-Wostl, 2009; Reed et al., 2010) are particularly important because they can identify and challengethe systemic causes of communities’ vulnerability to climate change (Pelling, 2011; Rodima-Taylor et al., 2012). In addition,interventions which apply ACM principles to link actors across sectors and scales can ‘prime’ them to implement adaptationprocesses (Baird et al., 2014).

Hence ACM may facilitate the implementation of adaptation pathways, particularly for rural communities in developingcountries which are dependent on ecosystems and often excluded from government planning processes (Butler et al., 2014).Nonetheless, this context also presents several challenges for successful ACM. Stakeholder capacity is low at all societallevels, constraining collaboration, and without long term resourcing and facilitation, self-organisation dissipates (Cundilland Fabricius, 2010). Participatory planning aiming to empower communities is often dominated by more powerful govern-ment and expert stakeholders’ agendas (Sherman and Ford, 2013), requiring approaches that can account for power asym-metries (Armitage, 2008). Also, rapid economic and cultural change continually re-shapes the political and institutionalenvironment, undermining trust and cooperation between stakeholders (Wollenberg et al., 2007).

In this paper we present a governance experiment which applied ACM principles as a means to prime stakeholders indeveloping countries to integrate adaptation pathways principles into rural development planning. Using the example ofa 4 year project in Nusa Tenggara Barat Province (NTB), Indonesia, we demonstrate the project’s design, and a participatoryevaluation methodology which tracked project outcomes in terms of ACM. The results assess the applicability of ACM as amechanism for establishing climate compatible development in developing countries. We also discuss the implications ofour results for the design of future adaptation initiatives in similar social and political contexts.

Attributes and evolution of ACM

The evolution and maintenance of ACM depends upon a combination of endogenous and exogenous system characteris-tics (Plummer, 2009; Plummer et al., 2012), summarised from the literature in Table 1. Endogenous factors are the combinedattributes of individuals, organisations, social networks and governance processes which together yield ACM outcomes.However, power asymmetries often marginalise communities and individuals in negotiated decision-making and knowledgeexchange with more powerful actors (Nadasdy, 2007; Armitage et al., 2008; Plummer et al., 2012), and in a development

Table 1Attributes and outcomes of successful ACM.

Attributes Outcomes References

Stakeholders engagedthroughout thesystem

Holistic and integrated systems understanding; multi-scale stakeholdersengaged in co-management

Armitage (2008), Armitage et al. (2009)

Power dynamicsaddressed

Barriers to stakeholder representation understood and rectified Armitage (2008)

Marginalisedcommunitymembers included

Resource users empowered in co-management; traditional ecologicalknowledge engaged in management

Armitage et al. (2009), Berkes (2009)

Trust generated Conflict resolution; bridging of stakeholders within and across scales Armitage (2008), Armitage et al. (2009)Cross-scale social

networks establishedSocial learning; double- and triple-loop learning; self-organisation;resources mobilised for problem-solving and innovation

Plummer and Armitage (2007), Cundill andFabricius (2010), Armitage et al. (2009)

Leadership generated Leaders emerge as ‘policy entrepreneurs’, providing an alternative vision,acting as agents of change, and brokering amongst stakeholders

Olsson et al. (2004a, b), Armitage et al.(2009)

Political engagement ‘Windows of opportunity’ for policy change exploited by leaders Olsson et al. (2004a, b)Monitoring and

evaluationFeedback enables reflexivity to complex system effects Armitage et al. (2009), Cundill and Fabricius

(2009)

J.R.A. Butler et al. / Climate Risk Management 12 (2016) 1–16 3

context this issue is a systemic cause of vulnerability (Lemos et al., 2007; Rodima-Taylor et al., 2012). Consequently, to besuccessful ACM must also address power dynamics and the inclusion of marginalised communities.

Exogenous factors are also necessary to trigger ACM. These can be a resource crisis or shock, a change in the political con-text (Plummer, 2009; Butler et al., 2015a), or an intervention intentionally engineered to induce ACM through collaborativeactivities (e.g. Cundill and Fabricius, 2010; Smedstad and Gosnell, 2013). Following an exogenous influence, ACM evolvesthrough stages, enabled by the endogenous characteristics of actors and processes (Olsson et al., 2004a; Berkes et al.,2007; Plummer, 2009). Olsson et al. (2004b) identified three. First, in ‘preparing the system for change’, leadership emergesamongst local resource stewards. These ‘policy entrepreneurs’ build knowledge of the problem, develop social networkswithin and across scales, and provide a vision and goal for an alternative approach. Second, in ‘the window of opportunity’,policy entrepreneurs exploit or precipitate policy windows at higher political levels to enact the alternative approach agreedin the first stage. Third, in ‘building resilience of the desired state’, a ‘policy community’ is formed. This consists of expandingcross-scale social networks and partnerships between stakeholders with common objectives that arise during the window ofopportunity, who act to build resilience of the social–ecological system’s desired state.

Project design

Study area

Nusa Tenggara Barat (Fig. 1) is one of the poorest provinces in Indonesia, and most poverty occurs in rural areas. Systemiccauses of poverty and community vulnerability are population growth, corruption, weak leadership and inadequate commu-nity participation in development planning. These underpin proximate causes including a lack of health and educationservices, and declines in land, food and water availability and mutual assistance practices (Butler et al., 2014, 2016b). In2010 the NTB provincial government established a Climate Change Task Force (CCTF) to integrate adaptation and develop-ment within the province, the first of its kind in Indonesia (Ministry of Environment, 2010). However, due to poor coordi-nation between government, donor and non-government organisations (NGOs), exacerbated by stakeholders’ lack ofawareness of future impacts of climate change and other drivers, the potential for mal-adaptive decision-making remainshigh (Butler et al., 2014).

As part of a national decentralisation policy, in 2004 the Indonesian government introduced an annual cycle of inte-grated top-down and bottom-up development planning (‘musrenbang’). Through multi-stakeholder consultations at thevillage, sub-district and district government levels, the process formulates community development plans which arelinked to provincial and national public expenditure planning. While communities are represented at all consultationstages, the process is often captured by political elites and government officials, resulting in communities’ needs not beingmet. Information on which to base decisions is often unavailable to communities, women and poorer households are mar-ginalised by the lack of procedural justice, and non-government stakeholders are seldom included (Purba, 2011; Aswadet al., 2012). There is also considerable confusion amongst participants about their roles (Aswad et al., 2012), exacerbatedby the regular subdivision of districts and sub-districts to create new administrative units and bureaucracies (Hunter,2004).

Governance experiment

The establishment of the CCTF and the institutional flux caused by decentralisation presented an opportunity to establishan alternative approach to development planning in NTB. Consequently, in 2010–2014 the Australian government’s aid

Fig. 1. Nusa Tenggara Barat Province (NTB), Indonesia, showing the five rural sub-district case studies.

4 J.R.A. Butler et al. / Climate Risk Management 12 (2016) 1–16

donor, AusAID, funded a project with the goal to ‘integrate adaptation pathways practice into development planning to buildstakeholders’ adaptive capacity, and reduce the vulnerability of rural communities to adverse future change’. This would beachieved by running a planning process mimicking musrenbang in five rural sub-district case studies (Fig. 1). Throughengagement and learning, stakeholders would be primed to transform musrenbang to include adaptation pathways practiceand diverse stakeholders’ knowledge in an iterative, self-organising and equitable decision-making process reflecting ACM.

The project was designed as a governance experiment, whereby actors in a system are engaged to purposefully inducechange in that system (Loorbach, 2010). Creating a ‘learning architecture’ is fundamental (Burns, 2014), and is achievedby linking stakeholders and their knowledge through planning, learning-by-doing and reflection (Brown, 2008; Loorbach,2010). Studies from socio-technical transition (e.g. Bos and Brown, 2012; Bos et al., 2013) and action research (e.g.Ballard, 2005; Brown, 2008; Burns, 2014) have recommended design principles for governance experiments that createopportunities for individual, social, double- and triple-loop learning, and thus systemic change. Bos et al. (2013) collatedthese into three themes: pre-conditions, project governance and process design. When combined with similar principlesfrom action research, plus Burns (2014) suggestion that running parallel processes can mitigate power asymmetries amongststakeholders, 10 design dimensions are apparent across the themes (Table 2).

The intent of these dimensions to generate multi-stakeholder engagement, empowerment and learning clearly mirror theattributes and outcomes of ACM (Table 1). Hence, by applying these principles in the design of our governance experiment,we expected to prime an ACM process. This was to be achieved by linking multiple actors across geographical scales andjurisdictional levels through four activities, facilitated by a research team (the ‘Tim Kolaboratif’) and a Steering Committee(Fig. 2). The facets of the design are described below and summarised in Table 2.

Tim KolaboratifThe project was initiated as a formal partnership between researchers from the Australian Commonwealth Scientific and

Industrial Research Organisation (CSIRO), the University of Mataram (UNRAM), and Indonesia’s National Agricultural Tech-nology Assessment Agency and the Bureau of Meteorology and Geosciences (BMKG). The first year was spent establishingthe Tim Kolaboratif, consisting of 15 biophysical and social scientists (five CSIRO and 10 Indonesian). An experiencedbi-lingual Indonesian facilitator was also employed to join the Tim Kolaboratif, who through the project activities coachedteam members in facilitation techniques. The Tim Kolaboratif was the project’s fulcrum, brokering amongst stakeholders,and organising and facilitating all activities (Fig. 2).

Steering CommitteeTo provide the partnership with strategic direction, a Steering Committee was established. This consisted of the directors

of the two provincial government departments responsible for the CCTF, plus a representative of the project donor, AusAID.The World Food Program (WFP), which was running extensive climate change and food security projects in NTB, was alsorepresented by its provincial director. The Tim Kolaboratif was represented by the senior CSIRO and UNRAM researchers.Meetings were held approximately quarterly, enabling an agile response to changes in the policy environment. The Steering

Table 2Design themes, dimensions and definitions for governance experiments based on Ballard (2005), Brown (2008), Bos and Brown (2012), Bos et al. (2013) andBurns (2014), and equivalent features of the project in NTB.

Theme Dimension Definition Project equivalent

(a) Pre-conditions – Legitimacy Political support to endorsealternative and experimentalprocess

Political support secured through CCTF membership ofSteering Committee, enabling the Tim Kolaboratif toengage with musrenbang in Activity 4

– Resources Dedicated resources to develop andimplement innovative policy andtools

AusAID committed 4 years of funding

– Time Time to develop and adapt learningprocesses and trust amongst actors

1 year to establish Tim Kolaboratif through Activity 1;2 years for Activity 2; 18 months for Activity 3

(b) Governance – Multi-stakeholderrepresentation

Actors engaged within and acrosslevels of the system

Steering Committee represented international,national and provincial levels; Activity 2 and 3engaged stakeholders from village to internationallevels

– Reflexive, adaptive Monitoring and evaluationdesigned to respond to shiftingpolitical context and learningprocess

Monitoring and evaluation sub-team established inTim Kolaborotif; embedded in activities and quarterlySteering Committee meetings to encourage learningand capacity building within the project

(c) Process design – Shared learning agendawith resonance

Issues of mutual concern to beexplored

Project focussed on community adaptation andpoverty alleviation

– Focus projects and learning-by-doing

Multiple inquiries connectedwithin and across levels to analysesystems as a complex problem, andto test innovations

Activity 2 workshops analysed climate change as acomponent of social–ecological systems, and linkedstakeholders’ perspectives and solutions across levels;Activity 3 tested adaptation strategies as innovations

– Multi-organisational peergroups

Form groups to promoteknowledge exchange, socialnetworks, double and triple-looplearning, leadership and trust

Stakeholder groups formed by Activity 2 workshops,which also promoted social learning and double- andtriple-loop learning; Activity 3 created farmer groupsand collaborations with Tim Kolaboratif and othermulti-level stakeholders

– Parallel processes to managepower

Separate stakeholders to managepower, knowledge and competinggoals

Activity 2’s stage 1 and 2 workshops segregated higherfrom local level stakeholders prior to integration inStage 3

– Distributed facilitation Diverse roles and responsibilitiesto broker knowledge and bridgescales

Tim Kolaboratif coached as facilitators and acted asbrokers to link stakeholders; skilled facilitatoremployed; Steering Committee provided politicalleadership

J.R.A. Butler et al. / Climate Risk Management 12 (2016) 1–16 5

Committee was intended to gain political legitimacy for the project by including the CCTF’s directors and their linkages tointernational, national and provincial policy processes (Table 2, Fig. 2).

ActivitiesActivity 1 Adaptation pathways tools: In this activity the Tim Kolaboratif prepared the methods, data and tools required to

implement adaptation pathways practice in musrenbang. Undertaken in the first year, it fostered the trans-disciplinaritynecessary to analyse social–ecological systems by invoking four principles: the joint development of an analytical frameworkthat integrated and equally recognised social and biophysical components (Strang, 2007), allowing time for idea generation(Pennington, 2008), and encouraging self-reflection and learning (Angelstam et al., 2013) mediated by an independent facil-itator (Harris and Lyon, 2013). Tools, data and their application have been presented in other papers in this special issue, andin summary include:

� Downscaled climate change projections (McGregor et al., 2016) and potential impacts for the agriculture and watersectors (Kirono et al., 2016).

� Population projections (Fachry et al., 2011).� A typology of sub-districts based on ecosystem services (Rochester et al., 2016).� A participatory model for evaluating future scenarios’ impacts on ecosystem services and human well-being (Skeweset al., 2016).

� A scenario planning framework which analysed rural communities as components of social–ecological systems, capturingmultiple drivers of change for livelihoods including climate change. Planning workshops formed consecutive, structuredlearning cycles (Butler et al., 2015b), and steps were designed to trigger double- and triple-loop learning and identifytransformative strategies to tackle systemic issues. Back-casting from future scenarios screened strategies to ensure thatthey were no regrets under all potential development trajectories (Butler et al., 2016b).

� Multi-criteria analysis which prioritised adaptation strategies according to their ‘climate compatibility’ (Wise et al.,2016).

Fig. 2. The project’s design and four activities, relative to stakeholder groups and their jurisdictional levels. Acronyms are CSIRO (Commonwealth Scientificand Industrial Research Organisation), UNRAM (University of Mataram), BPTP (Agricultural Technology Assessment Agency), BMKG (Bureau of Meteorologyand Geosciences), NTB Climate Change Task Force (CCTF) and the World Food Program (WFP).

6 J.R.A. Butler et al. / Climate Risk Management 12 (2016) 1–16

Activity 2 Planning workshops: Planning workshops aimed to engage stakeholders from the village to international levels(Fig. 2) in the five case studies. The process covered the project’s second 2 years, and involved three stages: Stage 1 provincialscenario planning, Stage 2 sub-district scenario planning, and Stage 3 sub-district integration (see Appendix A). The Stage 1and 2 workshops analysed future livelihoods from the perspectives of stakeholders from higher and local levels, respectively,which were shown to differ (Bohensky et al., 2016). This parallel process aimed to mitigate potential power asymmetriesamongst stakeholders prior to their integration in Stage 3 (Butler et al., 2015b).

The workshops applied the Activity 1 tools to formulate priority no regrets strategies and a development plan for eachcase study (Fig. 3a). Prior to each workshop a stakeholder analysis was carried out to identify and prioritise 30–40 individ-uals with responsibility for and knowledge of community development and natural resource management at the levels con-cerned, and included musrenbang participants. To ensure representation of women, female stakeholders were prioritised.

Activity 3 Adaptation strategy trials: Twelve prioritised adaptation strategies were tested by the Tim Kolaboratif over18 months through participatory action research with farmers and fishermen in the case studies. Examples included an inno-vative seaweed growing technique that minimised storm damage (Fig. 3b), and food diversification and processing bywomen’s groups to reduce dependence on drought-prone rice (Fig. 3c; Liu et al., 2016). Trials aimed to integrate knowledge,empower communities and women, and generate cross-scale social networks and partnerships between local communitiesand other stakeholders (Fig. 2).

Activity 4 Musrenbang engagement: Following completion of the planning workshops and strategy trials, the Tim Kolabo-ratif engaged with the 2013–2014 musrenbang cycle of meetings in the case studies. Team members presented the devel-opment plans, trialled adaptation strategies, adaptation pathways tools, and the Activity 2 planning process to musrenbangparticipants, consisting largely of male village representatives, sub-district and district government officials (Fig. 2). It wasexpected that this engagement would be expedited by the Steering Committee’s political influence.

Monitoring and evaluationMonitoring and evaluation was embedded within the project by a sub-team of the Tim Kolaboratif. The sub-team

carried out evaluation surveys amongst workshop participants (Butler et al., 2015b; Bohensky et al., 2016), and ‘wash-up’ sessions with the Tim Kolaboratif after events to review lessons learned and consider improvements to the process,which also fostered trans-disciplinarity (see in Section ‘‘Activities”). The adaptation strategy trials were reviewed every6 months. Feedback on activities’ progress was reported to the Steering Committee (Fig. 2). Together, these methodsintended to promote reflexive project management and learning amongst the Tim Kolaboratif and Steering Committee(Table 2), exemplifying ACM within the project’s governance. This was augmented by the ex-post evaluation presentedin the following section.

(a)

(b)

(c)

Fig. 3. (a) Workshop participants applying scenario planning, and adaptation strategy trials for (b) seaweed production and (c) food processing by awomen’s group.

J.R.A. Butler et al. / Climate Risk Management 12 (2016) 1–16 7

ACM evaluation

Theory of Change

To frame the anticipated evolution of ACM, a project Theory of Change (ToC) was developed by the Tim Kolaboratif. A ToCarticulates a vision of social change, and describes the assumptions about the sequential relationships between interventionsand change (Vogel, 2012; Bours et al., 2014). The ToC assumed that ACM would be triggered by the project design, primingstakeholders to implement adaptation pathways practice in the case studies. As a consequence, policy windows would beprecipitated, enabling the transformation of musrenbang in the case studies and throughout NTB, ultimately achievingthe goal of building stakeholders’ adaptive capacity and reducing the vulnerability of rural communities to adverse futurechange.

The ToC followed three causally-linked phases, which we equated with Olsson et al. (2004b) stages of ACM evolution(Fig. 4). Phase 1 priming stakeholders reflected ‘preparing the system for change’; Phase 2 enabling policies and programsreflected the ‘window of opportunity’, and Phase 3 implementing adaptation reflected ‘building resilience of the desiredstate’. Through each stage it was assumed that the numbers of stakeholders engaged would grow, cumulatively buildingadaptive capacity at all levels in NTB.

As the primary brokers and facilitators, it was planned and expected that Tim Kolaboratif members would become leadersand policy entrepreneurs in all phases (Fig. 4). The Steering Committee’s role peaked in Phase 2 when they would provide

Tim Kolaboratif

Goal

Adap

tive

capa

city

2009

1. Adaptation pathways tools Development of tools creates a trans-disciplinary Tim Kolaboratifwith skills in systems analysis, stakeholder engagement, knowledge brokering, and monitoring and evaluation

2. Planning workshopsMulti-stakeholder workshops generate leadership and trust, empowerment of communities and women, a vision, cross-scale social networks, enhanced knowledge, double-and triple-loop learning identifying systemic issues and innovative solutions

/////

4. MusrenbangengagementDevelopment plans, trialled adaptation strategies, adaptation pathways tools and ACM principles adopted by musrenbang in case studies

2010 2011 2012 20132010 2011 2012 2013 2014 2015

Sphere of control

Sphere of influence

Sphere of concern

Stak

ehol

ders

Through leadership, new partnerships and networks, policy windows enable musrenbang to be transformed in case studies and throughout NTB to include adaptation pathways and ACM principles, and resources committed for support

Key

/////

Adaptation pathways and ACM implemented in all musrenbang in NTB, growing social networks, partnerships, innovation, community self-organisation and stakeholders’ adaptive capacity

Phase 2Enabling policies and programs

Phase 3plementing adaptation

Phase 1Priming stakeholders

Process

Window of opportunity Building resiliencePreparing the system for change

Women, men, government, NGOs, businesses, donors

Steering Committee

3. Adaptation strategy trialsAction research integrates stakeholder knowledge and empowers communities and women, generating cross-scale social networks

Fig. 4. The project Theory of Change, time frame, phases, activities and assumed outcomes. Darker shading of stakeholders indicates a stronger role. O son et al. (2004b) equivalent stages of ACM are also shown(top). The project goal was to ‘integrate adaptation pathways practice into development planning to build stakeholders’ adaptive capacity, and ‘red e the vulnerability of rural communities to adverse futurechange’.

8J.R

.A.Butler

etal./Clim

ateRisk

Managem

ent12

(2016)1–

16

Im

lsuc

J.R.A. Butler et al. / Climate Risk Management 12 (2016) 1–16 9

political influence to transform musrenbang, and linkages to broader policy processes. This would recede in Phase 3 oncepolicy windows had been generated. Other multi-level stakeholders were engaged cumulatively starting in Activity 2, andtheir roles were assumed to strengthen in Phases 2 and 3 as leadership, new partnerships and networks emerged andexpanded, involving more communities. The project’s sphere of control covered the four activities. This was assumed torecede after project closure in June 2014, but the leadership and partnerships developed amongst the stakeholders wereexpected to continue through Phases 2 and 3.

Participatory evaluation

ACM indicatorsThere is no standardised approach to evaluating ACM due to the plasticity and context-specific nature of the process

(Plummer, 2009). Hence we adapted three complementary perspectives. Plummer and Armitage (2007) proposed a genericframework to evaluate ACM interventions based on 20 tangible and intangible outcome parameters. Plummer (2009) syn-thesised key characteristics necessary for the emergence of ACM, and Armitage et al. (2009) suggested 10 preconditions forthe evolution and maintenance of effective ACM. These perspectives highlight the dual objectives necessary for evaluatingcollaborative processes: measuring the attainment of intended outcomes, plus assessing whether the outcomes have createdpreconditions for the process to continue (Innes and Booher, 1999).

We selected indicators from these approaches to match the ToC’s assumptions (Fig. 4). Nine outcome indicators wereused to evaluate Phase 1, five of which were preconditions for ACM to continue (Table 3). Six outcome indicators wererelevant to Phases 2 (Table 4) and 3 (Table 5), of which four were also preconditions for ACM to continue. Two outcomeindicators in Phase 3 reflected the project’s goal. Community empowerment was included as a precondition and outcomeindicator for Phase 1 due to its importance for ACM in developing country contexts. As recommended by Plummer andArmitage (2007), indicators within each phase were intended to overlap to provide triangulation.

Stakeholder interviewsAt the project’s closure in June 2014, the monitoring and evaluation sub-team carried out structured interviews with

project participants. Interviewees were selected following a three stage filtering process. First, to focus on participantswho had the broadest and most contextualised perspectives of the process, only individuals that had attended at leasttwo planning workshops were selected, including Indonesian members of the Tim Kolaboratif. Second, their availabilitywas determined, and third, if available their willingness and consent to be interviewed was elicited.

Indicators were transposed into questions (Appendix B) and translated into Bahasa Indonesia. Interviews were carried outindividually and took 1 to 1 ½ h. To counter ‘memory distortion’ (Wiek et al., 2014), the project’s ToC was revisited at thestart of each interview. Responses to questions were discussed for up to 5 min. Each question was then presented as a propo-sition, and the interviewee was asked to give a score on a 5-point Likert scale from ‘strongly agree’ (score = 2), ‘agree’ (1),‘maybe’ (0), ‘disagree’ (�1) and ‘strongly disagree’ (�2). Interviewees’ responses were digitally recorded and later translatedinto English for analysis. The scores given for each indicator were averaged to standardise responses (van Vaerenbergh andThomas, 2013). All scores in each phase were also averaged, providing a more robust composite measure of outcomes (deVaus, 2002). ‘Don’t know’ responses were recorded but omitted from the calculation of means.

Table 3Indicators for ACM preconditions and outcomes to evaluate the Theory of Change assumptions in Phase 1 priming stakeholders (see Fig. 4).

Indicator Precondition (P)or outcome (O)

Theory of Change assumptions

1. Emergence of leaders prepared to champion the process P and O Leaders emerge as policy entrepreneurs and engage politically totake and create opportunities for action

2. Trust created amongst key stakeholders P and O Trust generated amongst stakeholders3. Empowerment of communities P and O Marginalised community members empowered, particularly women4. Vision and goal for an alternative pathway O Leaders champion an alternative vision for development planning5. Cross-scale social networks P and O Social networks established to enable self-organisation, bridge scales

to mobilise resources and knowledge for problem-solving, andengage politically

6. Enhanced knowledge of the problem P and O Stakeholders engaged to enhance a systems understanding ofpoverty and community adaptation

7. Different knowledge types successfully integrated andaccepted

O Stakeholders’ knowledge of poverty and adaptation successfullyintegrated and accepted

8. Questioning of values, norms, routines and governanceunderlying the problem, and awareness of itscomplexity

O Double- and triple-loop learning identifies systemic causes ofcommunity vulnerability, and systems understanding

9. Creative solutions and innovations O Tim Kolaboratif develops adaptation pathways and systems analysisskills and tools; innovative adaptation strategies developed andtrialled

Table 4Indicators for ACM preconditions and outcomes to evaluate Theory of Change assumptions in Phase 2 enabling policies and programs (see Fig. 4).

Indicator Precondition (P)or outcome (O)

Theory of Change assumptions

1. Management plans and/or agreements O Development plans produced for case studies; policies and programsintroduced to enable adaptation throughout NTB

2. Enabling changes to/or new institutionalarrangements

O Adaptation pathways and ACM adopted by case studies’ musrenbang;policy changes create windows of opportunity to transformmusrenbang throughout NTB

3. Cross-scale social networks expanding to mobiliseknowledge and resources

P and O Expanding social networks between participants, new partners and TimKolaboratif

4. Resources made available for implementation P and O Adaptation strategies funded in case studies’ musrenbang; resourcescommitted to support musrenbang transformation throughout NTB

5. New partnerships and cooperative initiatives O New partnerships between Tim Kolaboratif and stakeholders createwindows of opportunity

6. Outgrowth from the initial arrangements to addressadditional issues within or beyond the problemdomain

O Policies and programs introduced to enable adaptation more broadly inNTB

Table 5Indicators for ACM preconditions and outcomes to evaluate Theory of Change assumptions in Phase 3 implementing adaptation (see Fig. 4).

Indicator Precondition (P)or outcome (O)

Theory of Change assumptions

1. Implementation of innovations in arenas that trial,monitor and learn

P and O Adaptation strategies implemented in case studies and other sub-districts within a monitoring and learning framework

2. Enabling changes to/or new institutionalarrangements

O Musrenbang transformed throughout NTB to adopt adaptationpathways and ACM

3. Cross-scale social networks expanding to mobiliseknowledge and resources

P and O Social networks expanding between original and new partners,communities and Tim Kolaboratif

4. New partnerships and cooperative initiatives O New partnerships providing resources for implementation ofadaptation pathways, ACM and adaptation strategies

5. Enhanced community self-organisation that matchesscales and anticipates external drivers of change

O Project goal achieved: enhanced stakeholder adaptive capacity andreduced community vulnerability to future adverse change

6. Enhanced community capacity to live with uncertaintyand change

O Project goal achieved: reduced community vulnerability to futureadverse change

10 J.R.A. Butler et al. / Climate Risk Management 12 (2016) 1–16

Results

Indicator scores

Seventeen stakeholders were interviewed, comprising nine Tim Kolaboratif members, four provincial government, twoNGO and two village level participants. The highest mean score was for Phase 1, with the average response lying mid-way between ‘agree’ and ‘strongly agree’ (Fig. 5). The highest-scoring indicator was ‘trust created amongst key stakeholders’,followed by ‘emergence of leaders prepared to champion the process’. Both were ACM preconditions and outcomes. Thelowest-scoring indicator, although still positive, was ‘questioning values, norms, routines and governance’. One respondentdisagreed that ‘creative solutions and innovations’ had occurred, and one recorded a ‘don’t know’ score.

Phases 2 and 3 had similar mean scores, lying slightly above ‘agree’ (Fig. 5). For Phase 2 the highest-scoring indicator was‘cross-scale social networks’, followed by ‘management plans and/or agreements’. The former was an ACM precondition andoutcome. The lowest-scoring indicator, although still positive, was ‘enabling changes to/or new institutional arrangements’.Six respondents disagreed on four indicators, and there were five ‘don’t know’ scores.

For Phase 3 the highest-scoring indicator was ‘implementation of innovations in arenas that can trial, monitor and learn’,followed by ‘enhanced community capacity to live with uncertainty and change’, which was one of the project goal indica-tors (Table 5). The former was an ACM precondition and outcome. As for Phase 2, the lowest-scoring indicator, although stillpositive, was ‘enabling changes to/or new institutional arrangements’. One respondent disagreed on two indicators, andthere were no ‘don’t know’ scores.

Interviewees’ responses

Summaries of interviewees’ responses are collated below, and explain several of the highest-scoring plus the lowest-scoring indicators for each phase.

1. Emerg

ence

of le

aders

*

2. Trus

t crea

ted*

3. Empo

wermen

t of c

ommun

ities*

4. Visio

n for

altern

ative

pathw

ay

5. Cros

s-sca

le so

cial n

etwork

s*

6. Enh

ance

d kno

wledge

of pr

oblem

*

7. Kno

wledge

succ

essfu

lly int

egrat

ed

8. Que

stion

ing va

lues,

norm

s and

gove

rnanc

e

9. Crea

tive s

olutio

ns an

d inn

ovati

ons

1. Man

agem

ent p

lans a

nd/or

agree

ments

2. Ena

bling

chan

ges t

o/or n

ew in

stitut

ions

3. Cros

s-sca

le so

cial n

etwork

s*

4. Res

ource

s for

imple

mentat

ion*

5. New

partn

ership

s and

coop

erativ

e init

iative

s

6. Add

ressin

g add

itiona

l issu

es

1. Im

plemen

tation

of in

nova

tions

*

2. Ena

bling

chan

ges t

o/or n

ew in

stitut

ions

3. Cros

s-sca

le so

cial n

etwork

s*

4. New

partn

ership

s and

coop

erativ

e init

iative

s

5. Com

munity

self-o

rganis

ation

6. Com

munity

capa

city t

o live

with

chan

ge

Strongly agree

Agree

Maybe

Disagree

Strongly disagree

Phase 1Priming stakeholders

Phase 2Enabling policies

and programs

Phase 3Implementing

adaptation

Phase meanResponse range

Fig. 5. Mean scores for each indicator given by the interviewees (n = 17), ranges for their responses, and mean scores for all indicators in each phase.⁄ denotes indicators that are ACM preconditions and outcomes.

J.R.A. Butler et al. / Climate Risk Management 12 (2016) 1–16 11

Phase 1 Priming stakeholdersTrust created amongst key stakeholders (highest-scoring): Interviewees highlighted the increased trust that emerged

amongst the Tim Kolaboratif, especially from different organisations. One commented: ‘‘this project opened an opportunityto know other people or experts from other study backgrounds, and has initiated ideas that can be followed up and realisedin other projects. This has opened the team’s mind about working across disciplinary expertise”. Trust was also evident incollaborations forged between the Tim Kolaboratif and the WFP, and with farmers involved in the adaptation strategy trials.

Emergence of leaders to champion the process (second-highest scoring): Eight Indonesian members of the Tim Kolaboratifwere identified as champions for the project’s process. Two research assistants also emerged as leaders, and they wererecruited by an international NGO and a national eco-tourism business in 2013 to establish new climate adaptation pro-grams for these organisations. The directors of the NTB offices of BMKG and the WFP were also identified as new leaders.Several village stakeholders involved in the adaptation strategy trials were also identified as leaders, including one woman.One was described as follows: ‘‘before the project he was just an ordinary participant, but after he became a motivator, asource of information, and a facilitator for other people”.

Enhanced knowledge of the problem (third-highest scoring): Tim Kolaboratif interviewees unanimously agreed that theirunderstanding of climate change adaptation and social–ecological systems had increased. One stated: ‘‘climate change ismuchmore complex than just climate – it needs to consider all aspects of livelihoods and their drivers”. Through the strategytrials the researchers also learned collaboratively with community members about the realities of local livelihood challengesand the relevance of the strategies. Knowledge exchange and integration about climate and population projections, plus theidentification of locally-appropriate strategies was also strongly evident amongst the Tim Kolaboratif and other stakeholdersas a result of the planning workshops and trials.

Empowerment of communities (fourth-highest scoring): Communities were empowered through two activities. First, theStage 2 and 3 planning workshops enabled communities to be equitably represented, and to promote their priority strategiesto government, NGO and business stakeholders. One interviewee commented: ‘‘with the workshops at the sub-district levelthe community was empowered to cope with their problems. Some of them brought their strategies to the 2013–2014 mus-renbang”. Second, through the trials farmers and fishermen were trained, and in two trials women’s groups were empoweredthrough collaborations with the Tim Kolaboratif.

Questioning of values, norms, routines and governance underlying the problem (lowest-scoring): There had been someidentification of systemic issues by workshop participants, and realisation of the complexity of adaptation and poverty

12 J.R.A. Butler et al. / Climate Risk Management 12 (2016) 1–16

alleviation. However, this did not extend to political issues such as corruption. In terms of current development planning,one NGO interviewee claimed that ‘‘adaptation strategy issues have not been the government’s priority; infrastructure isthe government’s priority”. A community representative considered that as a result of the workshops the government‘‘should now consider what we need, because before they have thought only about what they want”.

Phase 2 enabling policies and programsCross-scale social networks (highest-scoring): There was widespread evidence of expanding social networks between stake-

holders, the Tim Kolaboratif and new partners. For example, linkages emerged between farmer groups involved in strategytrials and business sector buyers of produce, facilitated by the Tim Kolaboratif. The WFP also established new networksbetween two district governments and with UNRAM.

Management plans and/or agreements (second highest-scoring): Development plans were produced by the project for allcase studies, and trialled strategies were included in the 2013–2014 musrenbang in four case studies. In one case study,UNRAM, BMKG, WFP, the district government and an eco-tourism company signed an agreement to implement adaptationstrategies. Facilitated by leaders from theWFP, the adaptation pathways tools were applied to highlight priority sub-districtsfor the NTB government’s 2012 action plans on food security and climate change. However, the NTB 5 year Development Planproduced in 2014 did not target resources towards transformingmusrenbang in the case studies or elsewhere, or support the2012 action plans.

Enabling changes to/new institutional arrangements (lowest-scoring): Other than the inclusion of some trialled strategies infour case studies’ musrenbang, there was no evidence of adaptation pathways practice or ACM being adopted in the casestudies. Nor did policy changes occur at the national or provincial government level to create a window of opportunityfor transforming musrenbang more broadly. Despite their engagement in project activities, a major impediment was the lackof political will amongst the provincial and district government officers to modify the process because the ongoing sub-division of districts and sub-districts was creating new administrative roles and structures. Several interviewees noted thatthese officers were unwilling to relinquish control of musrenbang. Others mentioned that the Steering Committee had noteffectively engaged with national and provincial politics to generate windows of opportunity for the project to exploit. Thiswas partly attributable to the re-deployment of the CCTF representatives to new positions in the NTB government in 2013,following provincial elections. It was also highlighted that there had been insufficient recognition by the Tim Kolaboratif andSteering Committee of the political importance of district level leaders and officials, who had become increasingly powerfulduring the project through ongoing decentralisation.

Phase 3 Implementing adaptationImplementation of innovations in arenas that trial, monitor and learn (highest-scoring): All of the strategy trials had been

implemented in the case studies, and 6-monthly evaluations were carried out by the Tim Kolaboratif to provide on-goingreflection and learning. The national Ministry for Regional Development also signed agreements with UNRAM to scale-outtwo trialled adaptation strategies to other sub-districts. However, none of the other priority strategies included in each casestudy’s development plan had been included in their 2013–2014 musrenbang, or implemented.

Enhanced community capacity to live with uncertainty and change (second-highest): This was the highest-scoring indicatorreflecting the project’s goal. All interviewees stated that through the planning workshops community participants hadincreased their awareness of climate change and other future issues, and that the trials had equipped them with new skillsand opportunities for livelihood diversification. However, these impacts only related to the case studies, and not to othercommunities.

Enabling changes to/new institutional arrangements (lowest-scoring): Through the trials some local informal institutions hadbeen altered, such as the organisation of cassava growing and processing by women, and co-management of a river catch-ment. Interviewees agreed that musrenbang had not yet been modified in NTB.

Discussion

Mainstreaming climate change and future uncertainty into rural development planning in the developing world presentsnumerous challenges. Stakeholders must first understand adaptation pathways concepts, and then existing planning pro-cesses must be modified to adopt these concepts within an adaptive governance framework which can account for limitedcapacity, power asymmetries and institutional flux caused by rapid change. We conducted a 4 year governance experimentthat applied ACM principles as a priming mechanism to modify the existing community development planning process(musrenbang) to include adaptation pathways practice. A ‘learning architecture’ was created to engage multi-level and mus-renbang stakeholders in activities designed to generate community empowerment, collaboration, leadership, trust, cross-scale social networks, knowledge integration and innovation. The project’s ToC assumed that as a result of these activities,the trialled adaptation strategies, development plans and adaptation pathways tools would be adopted in the case studies.Through leaders and ‘policy entrepreneurs’ cultivated by the process, policy windows of opportunity would be precipitatedto transform musrenbang in the case studies and throughout NTB.

Our evaluation results suggest that within the 4 year life of the project the experiment was unsuccessful. Although someof the trialled adaptation strategies were implemented by musrenbang in four case studies, the prioritised strategies that

J.R.A. Butler et al. / Climate Risk Management 12 (2016) 1–16 13

formed the development plans were not. The adaptation pathways tools and ACM principles were not adopted in the casestudies, or more broadly in NTB. For the case studies, a simple explanation is that once the adaptation pathways tools andplanning process had been completed, the project only had time to engage with one musrenbang cycle in 2013–2014. Also,the anticipated policy windows did not materialise within the case studies or at higher jurisdictional levels. Several inter-viewees commented that the Steering Committee did not link effectively with political processes to precipitate policy win-dows. This was exacerbated by the re-deployment of the CCTF representatives in 2013 following a provincial election,curtailing their potentially influential role. The importance of engaging district government politicians and officials was alsonot recognised until the ex-post evaluation interviews.

The unpredictability of policy windows is not unusual in ACM. In the Kristianstad wetlands of Sweden, the second stage ofACM only progressed once four political issues had converged over a period of 1–2 years (Olsson et al., 2004b). For the ACMof conflict between seal conservation and salmon fishery interests in Scotland, the policy window emerged 5 years after theinitial exogenous shock (Butler et al., 2015a). Kingdon (1995) emphasised that policy windows occur irregularly becausethey are both problem-driven (i.e. a pressing problem that needs a solution) and politically-driven (i.e. problems that justifyattention because they meet the incumbent government’s political agenda). While the establishment of the CCTF in 2010reflected the former, sub-optimal political engagement by the project failed to create the latter. Also, interviewees suggestedthat the lack of a political response was exacerbated by the institutional flux caused by decentralisation, and the desire bygovernment officials to maintain control of musrenbang.

This is corroborated byWollenberg et al. (2007), who found in another Indonesian province that during a similar period ofpolitical upheaval the chaotic institutional environment undermined trust and the legitimacy of decision-making. In this sit-uation, Wollenberg et al. (2007) resorted to establishing ‘spontaneous cooperation’ on the periphery of formal governanceprocesses. In NTB, rather than expecting whole-sale changes to musrenbang, it may have been more realistic to focus onestablishing informal ‘livelihood innovation niches’ which create safe arenas for innovation in governance and technology,allowing transformative practice to develop amongst more fluid and emergent rules, and without penalty for failure (Butleret al., 2014). These could act as ‘bridgeheads’ for ACM to expand into formal planning processes when conditions allowed. Tosome extent this was achieved by the adaptation strategy trials, which created nascent ACM around natural resourcemanagement in the case studies, including the emergence of community leaders, cross-scale social networks, knowledgeintegration, empowerment, learning-by-doing and changes to local institutions.

Despite the lack of political progress, the evaluation did reveal many positive ACM outcomes, indicating that stakeholdershad been successfully primed to transform musrenbang. The mean score for Phase 1 priming stakeholders lay mid-waybetween ‘agree’ and ‘strongly agree’. All individual indicators’ mean scores were positive, with strong evidence of trust, lead-ership, enhanced knowledge of the problem and community empowerment. Together with a fifth, the establishment ofcross-scale social networks, these were also preconditions for the ACM process to continue (Fig. 5). Interviewees’ responsesidentified a broad range of evidence for these preconditions. In particular, leadership emerged within the Tim Kolaboratif, anational government agency, the WFP and amongst communities, suggesting that policy entrepreneurs had been establishedat multiple jurisdictional levels. Hence the capacity to precipitate or exploit policy windows may have been created atseveral levels in the system, increasing the likelihood of future progress.

Furthermore, the positive mean scores for Phase 2 enabling policies and programs and Phase 3 implementation adapta-tion indicated that the ACM process had evolved further than expected, particularly within the case studies. However, thelower mean phase scores and higher frequency of disagreements and ‘don’t knows’ relative to Phase 1 indicated that out-comes were nascent and more variable. Importantly, there was evidence of three ACM preconditions being established:cross-scale social networks (Phases 2 and 3), resources for implementation of adaptation strategies (Phase 2, see Fig. 5),and the implementation of innovations in the case studies within a monitoring and learning framework (Phase 3). As a result,the project’s goal had been partially achieved by increasing case study communities’ self-organisation and capacity to livewith uncertainty and change. The unexpected progress of Phases 2 and 3, in spite of the absence of policy windows, reflectsthe unpredictable and emergent nature of ACM (Plummer, 2009). This also validated the project’s design, which deliberatelyengaged stakeholders at multiple levels of the system, sowing the seeds for outcomes to emerge through several impactpathways (Stone-Jovicich et al., 2015).

Other than changes to institutions, the weakest outcome was the ‘questioning of values, norms and governance’ inPhase 1. While several interviewees indicated that government stakeholders had reconsidered their focus on infrastruc-ture, there was little evidence of double- and triple-loop learning that challenged the underlying systemic causes ofcommunity vulnerability. This was corroborated by the scenario planning workshops, where despite the identificationof numerous systemic drivers by participants, few transformative strategies were designed to tackle them (Butler et al.,2016b). Similarly, in the subsequent sub-district integration workshops, the vast majority of prioritised strategies wereincremental rather than transformative (Wise et al., 2016). It is well established that social learning (Cundill, 2010;Armitage and Plummer, 2010; Burns, 2012) and resulting transformative action (Feola, 2015) evolves slowly, and requiresnumerous action-learning cycles (Brown, 2008; Plummer et al., 2012). We have concluded that our series of planningworkshops was too brief to fully achieve this (Butler et al., 2016b; Wise et al., 2016). In this sense, our 4 year timeframedid not meet the governance experiment design principle of allowing adequate time to develop and adapt learningprocesses.

Many of the successful outcomes were founded on the establishment of the Tim Kolaboratif as the fulcrum for the pro-cess. Both the ACM and action research literature acknowledge that researchers can be important catalysts for systemic

14 J.R.A. Butler et al. / Climate Risk Management 12 (2016) 1–16

change (Ballard, 2005; Wollenberg et al., 2007; Cundill and Fabricius, 2010; Baird et al., 2014). Also, scientists are them-selves actors within a system, who have specialised knowledge that should be integrated with other stakeholders’ perspec-tives (Brown, 2008). This philosophy is reflected in agricultural ‘research for development’, which acknowledges thatinnovation is an emergent property of scientists working in an equal partnership with other stakeholders, and the sociallearning that occurs (Hawkins et al., 2009; Mbabu and Hall, 2012; Stone-Jovicich et al., 2015). The evaluation interviewsclearly demonstrated that the first year’s focus on building the trans-disciplinary skills of the Tim Kolaboratif paid divi-dends by creating trust, leadership, collaborations and new adaptation pathways and systems analysis skills amongstthe team, which subsequently emanated throughout the other project activities. However, Cundill and Fabricius (2010)have demonstrated in South Africa that if resourcing of such brokers and facilitators is not maintained, nascent ACMquickly dissipates.

By creating a learning and feedback mechanism amongst stakeholders, participatory evaluation can re-kindle the adap-tive and reflexive component of ACM (Cundill and Fabricius, 2009, 2010; Berkeley, 2013), and if repeated over time can gal-vanise leaders to tackle flagging progress (Butler et al., 2015a). Our ex-post participatory evaluation may have achieved this,but its influence on the process is not considered here. If repeated, the method could be improved in several respects. A dis-advantage was the risk of bias due to interviewees’ ‘acquiescent response styles’ (Dolnicar and Grun, 2007), resulting in anexaggeration of affirmative scores. This possibly occurred in our evaluation because nine of the interviewees were TimKolaboraif members, and they may have had a vested interest in giving positive scores. To mitigate this, if time and resourceshad allowed, a wider range of stakeholders from different levels should have been interviewed to capture a broader perspec-tive of ACM outcomes throughout the system. Also, at the community level more direct measures of impact will be neces-sary, such as Liu et al.’s (2016) economic modelling of household benefits derived from the trialled adaptation strategies.Nonetheless, the process of encouraging participants to reflect on the project may be more important than the accuracyof the indicator scores, which were only intended to illustrate trends in ACM outcomes against the ToC, and highlight issuesto be redressed.

Conclusions

Our study demonstrates the challenge of achieving climate compatible development in the developing world. This 4 yeargovernance experiment applied ACM principles to prime stakeholders in NTB to transform the musrenbang planning processby exemplifying adaptation pathways tools and ACM principles. Although our evaluation showed evidence of stakeholdersbeing primed, policy windows did not materialise to generate systemic change to musrenbang. This suggests that the designof future governance experiments in similar contexts should focus on effective political engagement at relevant jurisdic-tional levels, and allow more time for double- and triple-loop learning and policy windows to emerge. This has implicationsfor the funding and timeframes of similar adaptation interventions.

Our experience provides useful lessons about the utility of ACM as a governance approach for adaptation in developingcountries. First, while ACM can generate collective action, innovation and adaptive capacity, it is not a panacea. This isacknowledged in the literature (e.g. Armitage et al., 2009; Plummer et al., 2012). For climate adaptation, Plummer (2013)recommends that because ACM is most effective when applied at the local scale, ACM processes must ‘cascade upward’to hybridise with more centralised forms of governance. In NTB this linkage should involve coordination between the CCTF,the 5 year provincial development plan and musrenbang. Second, opportunities for ACM can be constrained in situations ofinstitutional and political flux, and this was the case in NTB. As a consequence, the clearest manifestations of ACM were gen-erated by the adaptation strategy trials. This may have been expected, since Armitage et al. (2009) suggest that successfulACM usually occurs within well-defined resource systems, with small scale resource use, an identifiable set of social entitieswith shared interests, and clear property rights to the resource.

Third, most examples of engineered ACM have proven unsustainable (e.g. Cundill and Fabricius, 2010; Smedstad andGosnell, 2013). This may be because the policy windows required to change institutions are inherently unpredictable, aswas evident in NTB. Instead, in conditions of limited capacity and rapid political and social change, we recommend thatto enhance priming processes informal livelihood innovation niches should be established as bridgeheads for ACM throughactivities such as adaptation strategy trials. However, this requires the long term support of trans-disciplinary teams akin tothe Tim Kolaboratif, who can act as leaders, brokers, facilitators and policy entrepreneurs to exploit policy windows whenthey eventuate. Finally, participatory monitoring and evaluation may be a useful tool for re-igniting ACM amongst stake-holders, particularly if timed to follow exogenous shocks or crises occurring after project completion, when conditions forsystemic change may be temporarily improved, As demonstrated by our approach, it is important that such exercises mea-sure ACM outcomes which are also preconditions for the longer term achievement of tangible adaptation impacts (Brookset al., 2011; Bours et al., 2013).

Acknowledgements

We acknowledge funding support from the Australian Government’s Department of Foreign Affairs and Trade-CSIROResearch for Development Alliance. We also thank the Tim Kolaboratif members and project participants who contributedto the evaluation. Adam Harper (CSIRO) provided the photographs in Fig. 3.

J.R.A. Butler et al. / Climate Risk Management 12 (2016) 1–16 15

Appendix A. Supplementary data

Supplementary data associated with this article can be found, in the online version, at http://dx.doi.org/10.1016/j.crm.2016.01.001.

References

Angelstam, P., Andersson, K., Annerstedt, M., Axelsson, R., Elbakidze, M., Garrido, P., Grahn, P., Ingemar Jonsson, K., Pedersen, S., Schlyter, P., Skarback, E.,Smith, M., Stjernquist, I., 2013. Solving problems in social-ecological systems: definition, practice and barriers of transdisciplinary research. Ambio 42,254–265.

Armitage, D., 2008. Governance and the commons in a multi-level world. Int. J. Commons 2, 7–32.Armitage, D., Berkes, F., Doubleday, N. (Eds.), 2007. Adaptive Co-Management: Collaboration, Learning, and Multi-Level Governance. UBC Press, Vancouver.Armitage, D., Marschke, M., Plummer, R., 2008. Adaptive co-management and the paradox of learning. Global Environ. Change 18, 86–98.Armitage, D., Plummer, R., Berkes, F., Arthur, R., Charles, A.T., Davidson-Hunt, I., Diduck, A.P., Doubleday, N.C., Johnson, D.S., Marschke, M., McCooney, P.,

Pinkerton, E.W., Wollenberg, E.K., 2009. Adaptive co-management for social–ecological complexity. Front. Ecol. Environ. 7, 95–102.Armitage, D., Plummer, R. (Eds.), 2010. Adaptive Capacity and Environmental Governance. Springer-Verlag, Berlin.Aswad, S., Heywood, P., Susilawati, C., 2012. The roles of procedural justice and social learning in improving self organizing capabilities of local communities

for sustainable development in decentralized Indonesia. OIDA Int. J. Sustainable Dev. 03 (10), 73–90.Baird, J., Plummer, R., Pickering, K., 2014. Priming the governance system for climate change adaptation: the application of a social–ecological inventory to

engage actors in Niagara, Canada. Ecol. Soc. 19 (1), 3.Ballard, D., 2005. Using learning processes to promote change for sustainable development. Action Res. 3, 135–156.Barnett, J., O’Neill, S., 2010. Maladaptation. Global Environ. Change 20, 211–213.Berkes, F., 2009. Evolution of co-management: role of knowledge generation, bridging organisations and social learning. J. Environ. Manage. 90, 1692–1702.Berkes, F., Armitage, D., Doubleday, N., 2007. Synthesis: adapting, innovating, evolving. In: Armitage, D., Berkes, F., Doubleday, N. (Eds.), Adaptive Co-

Management: Collaboration, Learning, and Multi-Level Governance. UBC Press, Vancouver, pp. 308–327.Berkeley, J., 2013. Opportunities for collaborative adaptive management progress: integrating stakeholder assessments into progress measurement. Ecol.

Soc. 18 (4), 69.Bohensky, E.L., Kirono, D.G.C., Butler, J.R.A., Rochester, W., Habibi, P., Handayani, T., Yanuartati, Y., 2016. Climate knowledge cultures: stakeholder

perspectives on change and adaptation in Nusa Tenggara Barat, Indonesia. Clim. Risk Manage. 12, 17–31.Bos, J.J., Brown, R.R., 2012. Governance experimentation and factors of success in socio-technical transitions in the urban water sector. Technol. Forecast.

Soc. Change 79, 1340–1353.Bos, J.J., Brown, R.R., Farrelly, M.A., 2013. A design framework for creating social learning situations. Global Environ. Change 23, 398–412.Bours, D., McGinn, C., Pringle, P., 2013. Monitoring and Evaluation for Climate Change Adaptation: A Synthesis of Tools, Frameworks and Approaches. SEA

Change CoP, Phnom Penh and UKCIP, Oxford.Bours, D., McGinn, C., Pringle, P., 2014. Guidance Note 1: Twelve Reasons Why Climate Change Adaptation M&E is Challenging. SEA Change CoP, Phnom

Penh and UKCIP, Oxford.Brooks, N., Anderson, S., Ayers, J., Burton, I., Tellam, I., 2011. Tracking Adaptation and Measuring Development. IIED Climate Change Working Paper No. 1,

November 2011. International Institute for Environment and Development, London, UK.Brown, V.A., 2008. Leonardo’s Vision: A Guide to Collective Thinking and Action. Sense Publishers, Rotterdam.Burns, D., 2012. Participatory systemic inquiry. IDS Bull. 43, 88–100.Burns, D., 2014. Systemic action research: changing system dynamics to support sustainable change. Action Res. 12, 3–18.Butler, J.R.A., Suadnya, W., Puspadi, K., Sutaryono, Y., Wise, R.M., Skewes, T.D., Kirono, D., Bohensky, E.L., Handayani, T., Habibi, P., Kisman, M., Suharto, I.,

Hanartani, , Supartarningsih, S., Ripaldi, A., Fachry, A., Yanuartati, Y., Abbas, G., Duggan, K., Ash, A., 2014. Framing the application of adaptation pathwaysfor rural livelihoods and global change in Eastern Indonesian islands. Global Environ. Change 28, 368–382.

Butler, J.R.A., Young, J.C., McMyn, I., Leyshon, B., Graham, I.M., Walker, I., Baxter, J., Warburton, C., 2015a. Evaluating adaptive co-management asconservation conflict resolution: learning from seals and salmon. J. Environ. Manage. 160, 212–225.

Butler, J.R.A., Wise, R.M., Skewes, T.D., Bohensky, E.L., Peterson, N., Suadnya, W., Yanuartati, Y., Handayani, T., Habibi, P., Puspadi, K., Bou, N., Vaghelo, D.,Rochester, W., 2015b. Integrating top-down and bottom-up adaptation planning to build adaptive capacity: a structured learning approach. CoastalManage. 43, 346–364.

Butler, J.R.A., Bohensky, E.L., Darbas, T. Kirono, D., Wise, R.M., Sutaryono, Y., 2016a. Building capacity for adaptation pathways in eastern Indonesian islands:synthesis and lessons learned. Clim. Risk Manage. 12, A1–A10.

Butler, J.R.A., Bohensky, E.L., Suadnya, W., Yanuartati, Y., Handayani, T., Habibi, P., Puspadi, K., Skewes, T.D., Wise, R.M., Suharto, I., Park, S.E., Sutaryono, Y.,2016. Scenario planning to leap-frog the Sustainable Development Goals: an adaptation pathways approach. Clim. Risk Manage. 12, 83–99.

Chapin, F.S., Lovecraft, A.L., Zavaleta, E.S., Nelson, J., Robards, M.D., Kofinas, G.P., Trainor, S.F., Peterson, G.D., Huntingdon, H.P., Naylor, R.L., 2006. Policystrategies to address sustainability of Alaskan boreal forests in response to a directionally changing climate. Proc. Natl. Acad. Sci. 103, 16637–16643.

Conway, D., Mustelin, J., 2014. Strategies for improving adaptation practice in developing countries. Nat. Clim. Change 4, 339–342.Cundill, G., 2010. Monitoring social learning processes in adaptive comanagement: three case studies from South Africa. Ecol. Soc. 15 (3), 28.Cundill, G., Fabricius, C., 2009. Monitoring in adaptive co-management: toward a learning based approach. J. Environ. Manage. 90, 3205–3211.Cundill, G., Fabricius, C., 2010. Monitoring the governance dimension of natural resource co-management. Ecol. Soc. 15 (1), 15.De Vaus, D.A. (Ed.), 2002. Surveys in Social Research. Allen and Unwin.Dolnicar, S., Grun, B., 2007. Cross-cultural differences in survey response patterns. Int. Market. Rev. 24 (2), 127–143.Fabricius, C., Currie, B., 2015. Adaptive co-management. In: Allen, C.R., Garmestani, A.S. (Eds.), Adaptive Management of Social–Ecological Systems. Springer

Science+Business Media, Dordrecht, pp. 147–179.Fachry, A., Hanartani, Supartiningsih, S., Butler, J.R.A., 2011. Social, cultural and economic trends in NTB and their drivers of change. CSIRO Climate

Adaptation Flagship, Brisbane, and University of Mataram, Lombok. Available at: <http://ccap-unram.org/>.Feola, G., 2015. Societal transformation in response to global environmental change: a review of emerging concepts. Ambio 44, 376–390.Folke, C., Hahn, T., Olsson, P., Norberg, J., 2005. Adaptive governance of social–ecological systems. Annu. Rev. Environ. Res. 30, 441–473.Hallegatte, S., 2009. Strategies to adapt to an uncertain climate change. Global Environ. Change 19, 240–247.Harris, F., Lyon, F., 2013. Transdisciplinary environmental research: building trust across professional cultures. Environ. Sci. Policy 31, 109–119.Hawkins, R., Heemskerk, W., Booth, R., Daane, J., Maatman, A., Adekunle, A.A., 2009. Integrated agricultural research for development (IAR4D). A Concept

Paper for the Forum for Agricultural Research in Africa (FARA) Sub-Saharan Africa Challenge Programme (SSA CP). FARA, Accra, Ghana.Hunter, C.L., 2004. Local issues and changes: the post-new order situation in rural Lombok. Sojourn 19, 100–122.Innes, J.E., Booher, D.E., 1999. Consensus building and complex adaptive systems: a framework for evaluating collaborative planning. J. Am. Planning Assoc.

65, 412–423.Kingdon, J.W., 1995. Agendas, Alternatives, and Public Policies. Harper Collins College Publishers, New York, NY, USA.Kirono, D.G.C., Butler, J.R.A., McGregor, J., Ripaldi, A., Katzfey, J., Nguyen, K., 2016. Historical and future seasonal rainfall variability in Nusa Tenggara Barat

Province, Indonesia: implications for the agriculture and water sectors. Clim. Risk Manage. 12, 45–58.

16 J.R.A. Butler et al. / Climate Risk Management 12 (2016) 1–16

Leach, M., (Ed.), 2008. Re-framing resilience: a symposium report. STEPS Working Paper 13. Social, Technological and Environmental Pathways toSustainability (STEPS) Centre, Brighton, UK. <http://steps-centre.org/wp-content/uploads/Resilience.pdf>.

Lemos, M.C., Boyd, E., Tompkins, E.L., Osbahr, H., Liverman, D., 2007. Developing adaptation and adapting development. Ecol. Soc. 12 (2), 26.Liu, S., Connor, J., Butler, J.R.A., Jaya, I.K.D., Nikmatullah, A., 2016. Evaluating economic costs and benefits of climate resilient livelihood strategies. Clim. Risk

Manage. 12, 115–129.Loorbach, D., 2010. Transition management for sustainable development: a prescriptive, complexity-based governance framework. Governance 23, 161–

183.Mbabu, A., Hall, A. (Eds.), 2012. Capacity building for agricultural research for development: lessons from practice in Papua New Guinea. Maastricht, United

Nations University-Maastricht Economic and Social Research Institute on Innovation and Technology (UNUMERIT).McGregor, J.L., Nguyen, K.C., Kirono, D.G.C., Katzfey, J.J., 2016. High-resolution climate projections for the islands of Lombok and Sumbawa, Nusa Tenggara

Barat Province, Indonesia: challenges and implications. Clim. Risk Manage. 12, 32–44.Ministry of Environment, 2010. National action plan addressing climate change. Indonesia Ministry of the Environment (Kementerian Lingkungan Hidup),

Jakarta, Indonesia.Mitchell, T., Maxwell, S., 2010. Defining climate compatible development. Climate and Development Knowledge Network Policy Brief, November 2010.Nadasdy, P., 2007. Adaptive co-management and the gospel of resilience. In: Armitage, D., Berkes, F., Doubleday, N. (Eds.), Adaptive Co-Management:

Collaboration, Learning, and Multi-level Governance. UBC Press, Vancouver, pp. 208–227.Olsson, P., Folke, C., Berkes, F., 2004a. Adaptive co-management for building resilience in social–ecological systems. Environ. Manage. 34, 75–90.Olsson, P., Folke, C., Hahn, T., 2004b. Social-ecological transformation for ecosystem management: the development of adaptive co-management of a

wetland landscape in southern Sweden. Ecol. Soc. 9 (4), 2.Pahl-Wostl, C., 2009. A conceptual framework for analysing adaptive capacity and multi-level learning processes in resource governance regimes. Global

Environ. Change 19, 354–365.Pelling, M., 2011. Adaptation to Climate Change: From Resilience to Transformation. Routledge, London.Pennington, D.D., 2008. Cross-disciplinary collaboration and learning. Ecol. Soc. 13 (2), 8.Plummer, R., 2009. The adaptive co-management process: an initial synthesis of representative models and influential variables. Ecol. Soc. 14 (2), 24.Plummer, R., 2013. Can adaptive comanagement help to address the challenges of climate change adaptation? Ecol. Soc. 18 (4), 2.Plummer, R., Crona, B., Armitage, D.R., Olsson, P., Tengö, M., Yudina, O., 2012. Adaptive comanagement: a systematic review and analysis. Ecol. Soc. 17 (3),

11.Plummer, R., Armitage, D.R., 2007. A resilience-based framework for evaluating adaptive co-management: linking ecology, economics and society in a

complex world. Ecol. Econ. 61, 62–74.Purba, R.E., 2011. Public participation in development planning: a case study of Indonesian Musrenbang. Int. J. Int. Soc. Sci. 5 (12), 265–277.Ramalingam, B., 2013. Aid on the Edge of Chaos. Oxford University Press.Ranger, N., Garbett-Shiels, S.-L., 2012. Accounting for a changing and uncertain climate in planning and policymaking today: lessons for developing

countries. Clim. Dev. 4, 288–300.Reed, M.S., Evely, A.C., Cundill, G., Fazey, I., Glass, J., Laing, A., Newig, J., Parrish, B., Prell, C., Raymond, C., Stringer, L.C., 2010. What is social learning? Ecol.

Soc. 15, 4.Rochester, W.A., Skewes, T.D., Suadnya, W., Butler, J.R.A., Lyne, V.D., Handayani, T., Habibi, P., Karnan, Cokrowati, N., 2016. A typology of natural resource use

for livelihood impact assessments in Nusa Tenggara Barat Province, Indonesia. Clim. Risk Manage. 12, 59–68.Rodima-Taylor, D., Olwig, M.F., Chhetri, N., 2012. Adaptation as innovation, innovation as adaptation: an institutional approach to climate change. Appl.

Geog. 33, 107–111.Scoones, I., Leach, M., Smith, A., Stagl, S., Stirling, A., Thompson, J., 2007. Dynamic systems and the challenge of sustainability. Social, Technological and

Environmental Pathways to Sustainability (STEPS) Centre Working Paper 1, Brighton, UK. <http://opendocs.ids.ac.uk/opendocs/handle/123456789/2470>.

Sherman, M., Ford, J., 2013. Stakeholder engagement in adaptation interventions: an evaluation of projects in developing nations. Climate Policy, http://doi.org/r45.

Skewes, T.D., Hunter, C.M., Butler, J.R.A., Lyne, V.D., Suadnya, W., Wise, R.M., 2016. The asset drivers well-being interaction matrix (ADWIM): a participatorytool for estimating future impacts on ecosystem services and livelihoods. Clim. Risk Manage. 12, 69–82.

Smedstad, J.A., Gosnell, H., 2013. Do adaptive comanagement processes lead to adaptive comanagement outcomes? A multi-case study of long-termoutcomes associated with the national riparian service team’s place-based riparian assistance. Ecol. Soc. 18 (4), 8.

Stone-Jovicich, S., Butler, J.R.A., McMillan, L., Williams, L.J., Roth, C., 2015. Agricultural Research for Development in CSIRO: A Review of Principles andPractice for Impact. CSIRO Agriculture, Brisbane, pp. 43.

Strang, V., 2007. Integrating the social and natural sciences in environmental research: a discussion paper. Environ. Dev. Sustainability. http://dx.doi.org/10.1007/s10668-007-9095-2.

Van Vaerenbergh, Y., Thomas, T.D., 2013. Response styles in survey research: a literature review of antecedents, consequences, and remedies. Int. J. Publ.Opin. Res. 25 (2), 195–217.

Vogel, I., 2012. Review of the use of ‘Theory of Change’ in international development. Report for the UK Department for International Development (DFID).Wiek, A., Talwar, S., O’Shea, M., Robinson, J., 2014. Toward a methodological scheme for capturing societal effects of participatory sustainability research.

Res. Eval. 23, 117–132.Wise, R.M., Fazey, I., Stafford Smith, M., Park, S.E., Eakin, H.C., Archer van Garderen, E.R.M., Campbell, B., 2014. Reconceptualising adaptation to climate

change as part of pathways of change and response. Global Environ. Change 28, 325–336.Wise, R.M., Butler, J.R.A., Suadnya, W., Puspadi, K., Suharto, I., Skewes, T.D., 2016. How climate compatible are livelihood adaptation strategies and

development programs in rural in Indonesia? Clim. Risk Manage. 12, 100–114.Wollenberg, E., Iwan, R., Limberg, G., Moeliono, M., Rhee, S., Sudana, M., 2007. Facilitating cooperation during times of chaos: spontaneous orders and

muddling through in Malinau District, Indonesia. Ecol. Soc. 12, 3.