Evidence that a national REDD+ program reduces treecover loss and carbon emissions in a high forestcover, low deforestation countryAnand Roopsinda,1, Brent Sohngenb, and Jodi Brandtc

aDepartment of Biological Sciences, Boise State University, Boise, ID 83725; bDepartment of Agricultural, Environmental and Development Economics, TheOhio State University, Columbus, OH 43210; and cHuman–Environment Systems, Boise State University, Boise, ID 83725

Edited by Anthony J. Bebbington, Clark University, Worcester, MA, and approved October 10, 2019 (received for review March 7, 2019)

Reducing emissions from deforestation and forest degradation(REDD+) is a climate change mitigation policy in which rich coun-tries provide payments to developing countries for protectingtheir forests. In 2009, the countries of Norway and Guyana enteredinto one of the first bilateral REDD+ programs, with Norway of-fering to pay US$250 million to Guyana if annual deforestationrates remained below 0.056% from 2010 to 2015. To quantifythe impact of this national REDD+ program, we construct a coun-terfactual times-series trajectory of annual tree cover loss usingsynthetic matching. This analytical approach allows us to quantifytree cover loss that would have occurred in the absence of theNorway-Guyana REDD+ program. We found that the Norway-Guyana REDD+ program reduced tree cover loss by 35% duringthe implementation period (2010 to 2015), equivalent to 12.8 milliontons of avoided CO2 emissions. Our analysis indicates that nationalREDD+ payments attenuated the effect of increases in gold prices,an internationally traded commodity that is the primary deforesta-tion driver in Guyana. Overall, we found strong evidence that theprogram met the additionality criteria of REDD+. However, wefound that tree cover loss increased after the payments ended,and therefore, our results suggest that without continued pay-ments, forest protection is not guaranteed. On the issue of leakage,which is complex and difficult to quantify, a multinational REDD+program for a region could address leakage that results from dif-ferences in forest policies between neighboring countries.

climate mitigation | climate policy | deforestation | impact evaluation |tropical forests

In 2007, under the Bali Action Plan, the global communityadopted a policy mechanism known as reducing emissions from

deforestation and degradation (REDD+) as a climate mitigationstrategy (1). REDD+ at its genesis was intended to incentivizeand compensate developing countries (non-Annex I) for verifiedemissions reduction through payments from developed countries(Annex I) via a compliance carbon market (1). These financialtransfers for the protection, sustainable management, and en-hancement of vulnerable forest carbon stocks was developed tooffset the opportunity costs associated with agricultural andother commodity production (e.g., palm oil, cattle, gold) thatdrive deforestation (2, 3). REDD+ was thus seen as a cost-effective means to help limit global warming to 2 °C and recti-fied a deficiency in its predecessor, the Kyoto protocol andsubsequent United Nations Framework Convention on ClimateChange (UNFCCC) Conference of the Parties meetings, thatfailed to tackle greenhouse gas emissions from deforestation.In the years after the Bali Action Plan, REDD+ attracted

global interest, garnering more than US $9.8 billion in financingcommitments from 2006 to 2014, primarily through bilateral andmultilateral funding (4). As of 2018, there are an estimated 467REDD+ projects and programs located in 57 countries, of which359 are classified as active interventions, 67 have been completed,and 42 either have not been initiated or have been discontinued(5). However, almost 2 decades after the conception of REDD+,

less than one-half of all REDD+ finance (42%) has been allo-cated for ex-post results-based and verified emissions reductionpayments (5). Furthermore, there is a paucity of REDD+ inter-ventions implemented at the national jurisdictional level thatuse results-based outcomes for evaluation (6). The lack of im-pact evaluation of REDD+ initiatives at the project and thenational levels has led to questions on its effectiveness, and hashandicapped our ability to learn from REDD+ climate finance(6–8). Understanding the outcomes of REDD+ is pivotal toinform current initiatives as well as the design of future itera-tions of results-based payments for climate mitigation (9).In this article, we assess the impact of a REDD+ program

implemented at the national jurisdictional level through a bilateralagreement between the Kingdom of Norway, the largest donor ofglobal REDD+ climate finance, and Guyana, a high forest cover,low deforestation (HFLD*) country (10–12). We apply an emerg-ing policy evaluation approach called the synthetic control method,or synthetic matching, to quantify the impact of the REDD+program on Guyana’s tree cover loss. Synthetic matching is anempirical approach that constructs a counterfactual time-seriesscenario, in our case “what tree cover loss would have occurredwithout the REDD+ program?” to evaluate the causal impact ofpolicy interventions. As is the case in this paper, the method is

Significance

REDD+ is the main international policy to reduce CO2 emissionsfrom deforestation in tropical countries. However, there are noempirical studies on the impact of REDD+ implemented at thecountry level. Here, we evaluated a nationwide REDD+ pro-gram implemented in Guyana. We apply synthetic matching toestimate tree cover loss that would have occurred in the ab-sence of the national REDD+ program (the counterfactual sce-nario). We found evidence that the program reduced tree coverloss by 35%, equivalent to 12.8 million tons of avoided carbonemissions. We also found evidence of accelerated tree cover lossat the end of the program. A multinational REDD+ approachimplemented in a region that includes continuous forest pro-tection payments will improve national REDD+ outcomes.

Author contributions: A.R. designed research; A.R. performed research; A.R. analyzeddata; and A.R., B.S., and J.B. wrote the paper.

The authors declare no competing interest.

This article is a PNAS Direct Submission.

Published under the PNAS license.

Data deposition: All datasets reported in this paper are open access and described inMethods.1To whom correspondence may be addressed. Email: anandroopsind@boisestate.edu.

This article contains supporting information online at www.pnas.org/lookup/suppl/doi:10.1073/pnas.1904027116/-/DCSupplemental.

*HFLD countries are classified as countries with more than 50% of historical forest coverremaining with deforestation rates less than the global average of 0.22% during thereference period of 1990–2000.

www.pnas.org/cgi/doi/10.1073/pnas.1904027116 PNAS Latest Articles | 1 of 8

SUST

AINABILITY

SCIENCE

Dow

nloa

ded

by g

uest

on

July

25,

202

1

particularly relevant when assessing outcomes of a policyimplemented over large aggregate jurisdictions, such as a singlecountry or state, where there are not identifiable comparisons forcausal impact inference (13–17).Synthetic matching builds a counterfactual country based on

the outcome of interest (in our case, the rate of tree cover loss),and observable covariates, or factors that drive that outcome (inour case, deforestation drivers such as agricultural expansion,and mining) collected for countries that can serve as appropriatecomparisons to the treated country (Fig. 1). To reduce, andhopefully eliminate selection bias, the comparison countries insynthetic matching are weighted based on the similarity of theircovariates to the treated country. In addition to matching on thecovariates, the average outcome of interest before the interventionis matched based on different linear combinations to control forunobserved factors that influence the outcome and whose effectvary over time (14). Thus, synthetic matching explicitly accountsfor unobserved drivers of the outcome (i.e., factors that influencethe outcome that we do not know of or do not have empirical datafor). This matching on outcomes is an improvement over otherquasiexperimental impact evaluation methods, such as propensityscore matching, that only account for observed and measuredcovariates to build the counterfactual scenario (18–20).

REDD+ Case StudyGuyana is a critical test case for national REDD+ programs† asit has one of the most intact tracts of old-growth tropical rain-forest globally, with an estimated forest cover of 85% (18.4 millionha) and an annual deforestation rate of 0.01% for the period 1990to 2000 (21). The Norway–Guyana REDD+ program is unique inthat it is the first REDD+ ex-post results-based payment foremissions reduction to a HFLD country. This REDD+ initiative isimplemented at the national jurisdictional level and employs acombined incentive-crediting baseline for emissions reductions (SIAppendix, 1). We construct a theory of change (ToC) of theNorway–Guyana REDD+ program based on the intended impactpathway for forest protection described in the agreement in orderto evaluate this national REDD+ program (SI Appendix, 2).We can quantify the impact of this national REDD+ program

because the outcome of interest, annual tree cover loss, is avail-able due to advances in remote sensing and computational powerthat has resulted in annual global forest cover change datasets at30-m spatial resolution (22). We use these Global Forest Changedata for several years prior (2000 to 2009), during (2010 to 2015),and after the Norway–Guyana REDD+ program (2016 to 2017),to construct a time-series counterfactual scenario of tree cover lossfor Guyana applying the synthetic control method. Our method-ological approach, which builds a time-series counterfactual tra-jectory for Guyana’s tree cover loss, directly quantifies theadditionality criteria of national REDD+, i.e., did this REDD+intervention reduce forest loss during the intervention period? Wealso use the tree cover loss time-series data and analytical outputsfrom the synthetic matching to explore key attributes that influ-ence the effectiveness of REDD+: leakage, i.e., the displacementof deforestation to neighboring countries, and permanence, i.e.,how long forests protected under the national REDD+ interven-tion remain intact after the program ended. Additionality, absenceof leakage, and permanence are considered central to the successof REDD+ as a climate mitigation strategy (Fig. 2) (23–25).Guyana is also a unique national REDD+ case study because of

its development status as a pretransition country characterized byhigh forest cover and low deforestation rates based on the foresttransition theory (SI Appendix, 3) (12, 26). The central focus of our

study is to determine whether the Norway–Guyana REDD+ pro-gram reduced forest loss. Results from our analysis will address amajor concern at the conception of national REDD+, that is,whether monetary payments to countries to protect forests withhistorically high forest cover and low deforestation rates wouldprovide a sufficient incentive to limit their deforestation (23, 27).

ResultsTree Cover Loss in Guyana and Other HFLD Countries. In 2000, forestcover in Guyana was 18.9 million hectares, covering ∼88% of thecountry. The average annual tree cover loss in Guyana during thepreimplementation period (2001 to 2009) was 0.036% y−1 (6,787ha·y−1) and increased to 0.056% y−1 (10,652 ha·y−1) during theNorway–Guyana REDD+ program (2010 to 2015; Fig. 3A). Dur-ing the implementation of the Norway–Guyana REDD+ program(2010 to 2015), annual tree cover loss did not increase above the0.1% threshold at which REDD+ payments would have ceased.There were 3 y where deforestation was higher than the 0.056%baseline deforestation rate, against which payment deductionsoccurred (Fig. 3B) (28). In the 2 y after the Norway–GuyanaREDD+ program ended (2016 to 2017), tree cover loss morethan doubled to 0.122% y−1 (22,985 ha·y−1; Fig. 3A).For the HFLD comparison countries used to build the syn-

thetic counterfactual for Guyana, all had forest cover >60% oftheir land area in 2000. Average tree cover loss across thesecountries was 0.136% y−1 (136,721 ha·y−1) in 2001 to 2009, 0.214% y−1

(216,441 ha·y−1) in 2010 to 2015, and 0.362% y−1 (365,190 ha·y−1) in2016 to 2017 (Fig. 3A). Gabon and Suriname had the most similartree cover loss rates and trajectory to that of Guyana in the pre-treatment period (2000 to 2009). Tree cover loss rates in Gabonand Suriname, respectively, were 0.065% y−1 (16,096 ha·y−1)and 0.034% y−1 (4,758 ha·y−1) from 2001 to 2009; 0.112% y−1

(27,485 ha·y−1) and 0.082% y−1 (11, 507 ha·y−1) from 2010 to2015; and 0.145% y−1 (35,402 ha·y−1) and 0.126% y−1 (17,458 ha·y−1)from 2016 to 2017. Notably, Guyana’s tree cover loss was mostsimilar to that of Suriname, before and after the Norway–GuyanaREDD+ program, but Guyana’s tree cover loss was 32% lowerthan Suriname’s during the program implementation period (2010to 2015).

Additionality: Effect of the Norway–Guyana REDD+ Program. Ourestimate of the effect of the Norway–Guyana REDD+ programon tree cover loss in Guyana is given by the difference in de-forestation rates between Guyana and its synthetic counterfac-tual for the 2010 to 2015 period (SI Appendix, 4). Estimatedannual tree cover loss in the synthetic counterfactual for Guyanawas higher (0.087% y−1) than observed tree cover loss in Guyana(0.056% y−1). The difference between the counterfactual andobserved tree cover loss is equivalent to a 0.031% y−1 reductionin the tree cover loss rate for Guyana over the 5 y, with the largestreduction occurring in 2012 to 2014 (Fig. 3C). Note that becausethere are only 4 countries with valid synthetic counterfactuals(Guyana, Gabon, Republic of Congo, and Suriname; SI Appendix,5), we would expect the probability of observing a difference aslarge as that for Guyana and its synthetic counterfactual for the2010 to 2015 period to be 60% by chance alone. The annual treat-ment effects show that the Norway–Guyana REDD+ program hadthe strongest effect in 2014 and 2015 (Fig. 3C), with a 25% proba-bility of observing a reduction in tree cover loss as large as the oneobserved for Guyana by chance alone compared to the other3 in-space placebo comparison countries. The bootstrapped un-certainty estimates (95% confidence intervals), which includes allcountries in the donor pool, also indicate a significant decline inGuyana’s tree cover loss during the Norway–Guyana REDD+program relative to the counterfactual scenarios (SI Appendix, 5).

Leakage: Spillover Effects of the REDD+ Program. To test for leak-age, we conducted a geospatial analysis that employed the

†Throughout the text, we refer to “national REDD+ program” as efforts to incentivizegovernments to change future trends in deforestation through implementation ofcountry-wide programs. In contrast, “project-level REDD+” refers to efforts to changedeforestation in specific projects that are undertaken in part of a country.

2 of 8 | www.pnas.org/cgi/doi/10.1073/pnas.1904027116 Roopsind et al.

Dow

nloa

ded

by g

uest

on

July

25,

202

1

assumption that transboundary leakage of deforestation (neg-ative spillover effects) would occur in forests closer to Guyanadue to proximity (SI Appendix, 6). If there is elevated tree coverloss in the border region of Suriname, relative to its interiorduring the Norway–Guyana REDD+ program (2010 to 2015)compared to the period prior to the Norway–Guyana REDD+program (2001 to 2009), this could be interpreted as evidenceof deforestation leakage from Guyana. In our analysis, wefound that tree cover loss along the border region of Surinamebefore the Norway–Guyana REDD+ program (2001 to 2009)

was 0.020% y−1 (SE: 0.015) compared to 0.003% y−1 (SE:0.001) in the interior of Suriname. Tree cover loss along theborder region and the interior of Suriname during the Norway–Guyana REDD+ agreement (2010 to 2015) was 0.032% (SE:0.023) y−1 and 0.011% (SE: 0.003) y−1, respectively (SI Ap-pendix, 6). We found that tree cover loss along the border re-gion with Guyana and the interior region of Suriname didincrease during the Norway–Guyana REDD+ program, but co-incided with a 250% increase in the price for an ounce of gold(SI Appendix, Fig. S9).

ConditionalPayments

+ImprovedGovernance

REDD+

LeakageEmissions are not spatially displaced

PermanenceEmissions are not displaced into the

future

AdditionalityQuantified

emissions reduction achieved

Fig. 2. Conceptual diagram of the key attributes of REDD+ as an international climate mitigation strategy. Additionality is the reduction in de-forestation and emissions achieved because of the REDD+ intervention that is site specific. The effectiveness of the REDD+ intervention is, however,mediated by displaced deforestation outside of the REDD+ jurisdiction (leakage), or increased deforestation after the REDD+ intervention has ended(permanence).

Suriname

Gabon

DRCCongo

Colombia

Peru

Guyana

Fig. 1. Map with high forest cover, low deforestation (HFLD) countries that are in the pretransition phase based on the forest transition theory. Base map(green) is forest cover estimated in year 2000 applying a 30% forest cover threshold; red is measured tree cover loss, and blue is measured tree cover gainbetween 2001 and 2017 derived from the Global Forest Change dataset (22).

Roopsind et al. PNAS Latest Articles | 3 of 8

SUST

AINABILITY

SCIENCE

Dow

nloa

ded

by g

uest

on

July

25,

202

1

Permanence: Tree Cover Loss after the Norway–Guyana REDD+Program Ended. At the end of the Norway–Guyana REDD+program in 2015, observed tree cover loss for Guyana increased

by 200% from 0.069 to 0.140% in 2016. This increase in treecover loss for Guyana is equivalent to the level of its syntheticcounterfactual after the Norway–Guyana REDD+ program ended

0.000

0.025

0.050

0.075

0.100

0.125

0.150

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017

)%(

ssolrevoceertlaunna

ssorG

Synthetic Guyana Guyana

Norway-Guyana REDD+ Project

Payments begin to decline above 0.056%

Payments stop at 0.100%

-0.06

-0.03

0.00

0.03

0.06

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017

Norway-Guyana REDD+ Project

niecnereffi

D)

%(ssolrevoc

eert

Colombia

DRC

Gabon

Guyana

Peru

Congo

Suriname

Tree cover loss in HFLD countries from 1990-2000 (0.22% Yr-1)

0.2

0.0

0.4

0.6

0.8

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017

)%(

ssolrevoceertlaunna

ssorG

A

B

C

Fig. 3. Annual tree cover loss rates (percentage) extracted from the Global Forest Change dataset for 2001 to 2017 and synthetic matching outputs. (A) Treecover loss rates for Guyana and comparison countries classified as HFLD countries. (B) Observed trends in gross tree cover loss for Guyana compared to thesynthetic counterfactual for Guyana. (C) The difference between the observed rate of tree cover loss for Guyana and its synthetic counterfactual. TheNorway–Guyana REDD+ program was implemented from 2010 to 2015 (blue panel). The agreement stipulated that REDD+ payments would decline abovedeforestation rates of 0.056%, and no payments would be made if deforestation reaches 0.10%.

4 of 8 | www.pnas.org/cgi/doi/10.1073/pnas.1904027116 Roopsind et al.

Dow

nloa

ded

by g

uest

on

July

25,

202

1

(Fig. 3B). In the 2 y after the REDD+ program ended (2016 and2017), tree cover loss in Guyana also increased and remainedabove the 0.1% deforestation benchmark that would have trig-gered no REDD+ payments under the Norway–Guyana REDD+agreement (SI Appendix, 1).

DiscussionNorway–Guyana REDD+ Additionality. We found that tree coverloss increased in our focal country, Guyana, and all of the othercomparison countries, classified as pretransition forest econo-mies from 2001 to 2017 (Fig. 3A). However, we identified a 35%reduction in tree cover loss in Guyana (5,800 ha·y−1) from2010 to 2015 during the Norway–Guyana REDD+ program (Fig.3B). This reduction in forest loss is equivalent to 12.8 milliontons of CO2 not emitted to the atmosphere over the programperiod, achieving the intended impact of national REDD+.The sliding scale crediting baseline that penalized Guyana for

increasing deforestation was effective at keeping Guyana’s treecover loss below the 0.10% threshold, at which point Guyanawould receive no REDD+ payments. Specifically, the results-based payments under the Norway–Guyana REDD+ programseemed to have attenuated the deforestation pressure associatedwith the international price of gold. This finding is particularlyimportant to the Guiana Shield, where the production of goldfrom artisanal mining, which manifests in small-scale (<5-ha)deforestation patches, is the dominant driver of forest loss acrossthe region (29–31).In the year 2012 when gold prices peaked at US $1,669/oz, tree

cover loss increased by 71% in Suriname, a country that sharessimilar biophysical characteristics and deforestation drivers asGuyana, but only increased by 36% in Guyana (SI Appendix, Fig.S9). The Government of Guyana, as the owner of all subsurfacemineral rights, directly controls the rate of deforestation byleases and incentives for commercial forest use activities such asmining. REDD+ financing, at a US$5 carbon price, surpasses theGovernment yields from gold, and as such, with REDD+ fi-nancing, the government is incentivized to reduce forest lossto capture more revenue for the national budget (32). We discussthe motivation of the Guyanese government to engage inREDD+ programs in more detail in SI Appendix, 2: REDD+Theory of Change as a Climate Mitigation Strategy.Aside from results-based payments, the Norway–Guyana

REDD+ program also emphasized performance-enabling REDD+activities (33). Specific emphasis and allocation of Norway–GuyanaREDD+ payments were made to support institutional and tech-nical capacity to achieve the Norway–Guyana REDD+ outcomeof reduced emissions. There is evidence that this emphasis onthese Norway–Guyana REDD+ enabling activities, classified asoutputs in the REDD+ ToC (SI Appendix, 2), resulted in bothregulatory and technological additionality that improved overallforest governance. For example, prior to 2010, Guyana was rankedthe lowest for its national forest monitoring capabilities, but by2015, Guyana was reporting at tier 3 and higher for carbonemissions assessment under the Intergovernmental Panel on Cli-mate Change guidelines and had strong in-country remote sensingcapacity for monitoring forest cover change (34). Concerning thelegal forestry framework, Guyana was highly ranked for key policyfeatures that are considered necessary for national REDD+ out-comes (35). Clearly defined tenure and use rights were included inGuyana’s forest legislation, policy, and governance framework.Guyana also had forest management regulations in place andenforced, relative to the other donor pool countries (SI Appendix,Table S2). The advancements in both regulatory forest gover-nance framework and technological capacity to measure, monitor,and verify emissions reductions seem to have been important el-ements achieved under the Norway–Guyana REDD+ program.Improvements in forest governance quality such as those quan-tified for Guyana are seen as pivotal for the reduction in forest

loss, biodiversity protection, and prevention of environmentaldegradation (36).These results show that a national REDD+ program reduces

forest loss. Such a rigorous national-level counterfactual analysis,to our knowledge, has not yet been conducted due to method-ological challenges of building the counterfactual scenario (6).We acknowledge that the REDD+ policy arena at the interna-tional and country level is highly dynamic, with the REDD+concept itself evolving since its inception (9). The availability ofnational REDD+ finance to countries in the donor pool couldhave influenced our comparative analysis and impact evaluationinference. To account for country-level dynamics around na-tional REDD+ implementation, we report on forest governanceindicators and funding support for national REDD+ programs,which draws off the ToC (SI Appendixes, 2 and 3). In this qual-itative comparison, we note that Suriname and Gabon, whichconstituted the majority of the weights in the synthetic matchingto build the counterfactual Guyana, ranked poorly across allmetrics for national REDD+ enabling activities, including na-tional REDD+ finance. The more recent national and project-level REDD+ activities in Gabon and Suriname indicate thatthese 2 control countries were less affected by national REDD+interventions during the Norway–Guyana REDD+ program.

Norway–Guyana REDD+ Leakage. Due to its proximity, similarecological and geological features, and reliance on gold mining,Suriname would have served as the best match for Guyana fora simple qualitative comparison to evaluate the impact of theNorway–Guyana REDD+ program. The synthetic matching vali-dates Suriname’s position as an appropriate comparison countryfor Guyana, with the largest weight assigned to Suriname to con-struct Guyana’s counterfactual scenario of deforestation rates (SIAppendix, 4). The synthetic counterfactual for Guyana was, how-ever, extremely sensitive to the exclusion of Suriname, with nocombination of the other HFLD comparison countries able torecover a tree cover loss time-series trajectory that matched thetree cover loss observed for Guyana in the pretreatment period (SIAppendix, 5). A concern in causal impact inference and the syntheticcontrol method is the potential existence of leakage or spillover ef-fects from the treated country to the countries used in the com-parison (14, 25). The issue of leakage is especially a valid concern asGuyana shares a particularly porous border with Suriname, and thecountries have close socioeconomic links that include beingmembers of an economic trading block (i.e., Caricom).Our geospatial approach to explore leakage is deficient in that

it does not address the specific drivers of deforestation, whichare difficult to assess due to both their complexity and con-founding factors (25, 37, 38). Further quantification of cross-border leakage associated with the Norway–Guyana REDD+program would require more in-depth economic analysis thatfocuses on transboundary investment flows, labor, and marketeffects, especially those related to the demand and supply of goldand policies implemented by other countries (see SI Appendix, 6for additional details on leakage). Such an approach is war-ranted, considering that we found increases in tree cover loss inboth the border and interior region in Suriname, but which co-incided with a significant increase in gold prices. In sum, ac-counting for both negative spillovers (e.g., displacement ofdeforestation) and positive spillovers (e.g., other countries beginto implement stricter forest policies) is needed to ensure there isno overestimation or underestimation of the REDD+ policyeffectiveness.

Norway–Guyana REDD+ Permanence.The Norway–Guyana REDD+program did not explicitly account for permanence, in that there isno requirement after the program for the areas that were notdeforested to remain forested after the agreement ended (33). Weobserve that Guyana’s deforestation rate increased above the

Roopsind et al. PNAS Latest Articles | 5 of 8

SUST

AINABILITY

SCIENCE

Dow

nloa

ded

by g

uest

on

July

25,

202

1

0.10% threshold immediately after the REDD+ program endedin 2016 (Fig. 3B). Under the performance clauses in the contract,a deforestation rate of 0.10% would have stopped paymentsduring the program, but because it occurred after the programtimeline, payments were not altered. This outcome suggests thatpolicymakers need stronger permanence clauses in nationalREDD+ contracts, especially to guard against socioeconomicshocks such as higher prices of commodities that drive defor-estation. We illustrate this point by calculating the effectivecosts of the carbon benefit over the 2010 to 2015 period. Basedon assumed payments of US$250 million, the cost of the12.8 million tons of avoided CO2 emissions is US$19.53 per tonof CO2. If these emission reductions are permanent, then this(US $19.53) is the effective cost per ton of CO2. However, ifthis emission is only avoided for 5 y, then the rental rate at a 5%discount rate is US $4.50 per ton of CO2 per year, and theeffective cost of carbon is around US$90 per ton of CO2.Moving forward, we will have additional years of the forest-loss

time-series data from the Global Forest Change dataset, whichwill enable us to quantify permanence better. A major advance-ment provided by our application of the synthetic matching is theconstruction of a counterfactual time-series trajectory, which iswell suited to the task of assessing permanence outcomes fromforest protection initiatives. Such long-term data will also provideinformation on how shocks, such as those related to commodityvalues, and other sociopolitical events, such as government elec-tions, impact forest loss rates.

Synthetic Matching and Building a Time-Series Trajectory. There is atremendous challenge to quantify the impact of policies imple-mented at an aggregate level with a single treated unit, particularlywhen that treated unit is a country (14). Synthetic matching offersa systematic and transparent way to overcome this challenge be-cause it builds credible estimates of counterfactuals in order toestimate the treatment effect and critically evaluates those resultsin a quantitative manner (13, 17). Synthetic matching removes thesubjectivity of the analyst in selecting the best-suited comparisonunit, and it enables us to overcome the limitations of othermatching methods (18, 19). Through matching on the outcome ofinterest, it explicitly accounts for unobserved or unknown factorsthat might influence the outcome.The use of annual time-series data in the synthetic matching

process also improves upon before-after-control-impact analysismethods that rarely account for interannual variability. Thecounterfactual trajectory produced by synthetic matching enablesus to assess thoroughly the quality of the counterfactual com-parison at multiple points in time before, during, and after theintervention. To further improve transparency, synthetic matchingprovides explicit weights for both the comparison countries andcovariates, thus allowing for a critical assessment of the counter-factual and the overall quality of the comparison based on expertknowledge.Irrespective of these advances, finding appropriate counter-

factuals for programs implemented at large aggregate scales likethe country level is still a challenge, as seen by the limited numberof donor pool countries, and the synthetic matching outputs (SIAppendixes, 3 and 4). The low number of appropriate comparativeunits affects the strength of inferences that can be made, as isevident by the 3 in-space placebo comparison countries in ouranalysis (Gabon, Republic of Congo, and Suriname), and thedecreasing counterfactual fit with the exclusion of Suriname (SIAppendix, 5). To overcome this statistical sample size challenge,we employed a parametric bootstrap procedure that resamples theresiduals of the counterfactuals to obtain the uncertainty estimatesof the policy effect (SI Appendix, 5).

ConclusionsTargeted efforts at reducing deforestation in developing countrieshave received substantial support as evident by unprecedentedfunding commitments (4), country-level efforts to characterizedeforestation drivers (12), and the large number of national andproject-level REDD+ pilots being implemented (5). These ini-tiatives are important because tropical forest loss has increasedglobally with the most significant loss in tropical forests recordedin 2016 and 2017, even in countries with historically low defores-tation rates with the majority of intact forest landscapes (39, 40).Results such as these based off of rigorous impact evaluation ofclimate mitigation programs will help countries refine their climateaction plans post-2020 (6).Overall, we find strong evidence in our analysis that the

Norway–Guyana REDD+ payments were effective in reducingtree cover loss in a country with historically low deforestationrates. This result is proof of emissions reduction that would nothave occurred without the Norway–Guyana REDD+ programand thus satisfies the additionality pillar of REDD+. Our analysisalso provides important insights into how carbon crediting base-lines for national REDD+ can be effective, the establishment ofwhich has stymied the progress of many other national REDD+programs (23). The application of a sliding scale deforestationreference baseline, with increasing penalties as deforestation risesseems to have empowered both the buyer and provider of carboncredits. The use of a flexible reward scheme potentially bodes wellfor a forest carbon market that can respond to demand and supplyrelative to other commodities (32, 41).Leakage and permanence are more challenging to quantify in

the context of national REDD+ as an international climatemitigation strategy. The difficulty of accounting for leakage is aresult of its complexity and finding direct causal links, especiallyas the drivers of deforestation are global in scope and associatedwith global commodity trade and investment flows (24, 38, 42).The fragmentary implementation of national and project-levelREDD+ across forested countries leaves room for the dis-placement of deforestation from early adopters like Guyana thathave developed rigorous forest carbon regulatory and gover-nance systems to countries that have not engaged in REDD+and that have weaker regulatory policies (43). A multinationalREDD+ approach across all of the Guiana Shield biome coun-tries that includes political cooperation and harmonizes forestgovernance and deforestation regulations would address the is-sue of leakage (Fig. 2) (24, 44, 45). The fact that the observed treecover loss in Guyana climbs above the 0.10 threshold after theNorway–Guyana REDD+ program ended indicates that contin-ued forest protection is dependent on sustained conditional na-tional REDD+ payments, and stronger permanence clauses.

Materials and MethodsDeforestation Data. We utilized the Global Forest Change dataset spanning2000 to 2017, which measures percent tree cover in 2000, tree cover loss, andtree cover gain at 30-m spatial resolution based on earth observation data,primarily Landsat imagery (22). The percent tree cover in the 2000 layerrepresents the canopy cover of each 30-m pixel in the baseline year of 2000.Each pixel is classified from 0 to 100, with 0 being no canopy cover and100 being 100% canopy cover. For this analysis, we considered a pixel ≥30%canopy cover as forested, which is consistent with Guyana’s and other na-tional submissions to the UNFCCC REDD+ program (46). Tree cover loss isdefined as a stand-replacing disturbance or a change from a forest tononforest and is measured annually from 2001 to 2017. The Global ForestChange dataset provides us with a time-series trajectory of annual tree coverloss that includes 9 y of data preintervention (2001 to 2010), 5 y duringprogram implementation (2010 to 2015), and 2 y after the end of the pro-gram (2015 to 2017).

The tree cover loss (percentage) was calculated as�

forest   losstforest   covert

�* 100, where

forest   losst is the tree cover loss in year t for a country, and forest covert isthe country level forest area at the beginning of year t. The base year forest

6 of 8 | www.pnas.org/cgi/doi/10.1073/pnas.1904027116 Roopsind et al.

Dow

nloa

ded

by g

uest

on

July

25,

202

1

cover map is set as year 2000, from which subsequent annual forest area iscalculated by subtracting forest loss estimated for the preceding year. It isimportant to note that, with this calculation method, even with a constantamount of forest loss, the tree cover loss rate will increase, as the total forestarea decreases. We exclude forest gain from our analysis, as the specificpolicy outcome under evaluation is the deforestation indicator under theNorway–Guyana REDD+ program. We calculate the annual rate of forest lossin Google Earth Engine, a cloud-based platform for the analysis of earthobservation data that combines a public data catalog on satellite imagerythat includes the Global Forest Change dataset, with a large-scale compu-tational facility optimized for parallel processing of geospatial data (47). Wecross-checked the annual rate of forest loss derived from the 30-m GlobalForest Change dataset with deforestation rates reported and independentlyvalidated under Guyana’s measurement, reporting, and verification (MRV)system for REDD+ that is based on 5-m high-resolution RapidEye imagery(21). We found no systematic underestimation or overestimation betweenthe annual rate of forest loss extracted from the Global Forest Changedataset and Guyana’s independently verified deforestation rates under itsMRV system (ref. 49 and SI Appendix, Table S6).

Construction of the Counterfactual Time Series Using the Synthetic ControlMethod. Evaluation of policy outcomes where a single aggregate unit likea country is exposed to an intervention and the policy target is determined forthe country as a whole is challenging, in part because it is difficult to decidewhat the comparative units are (i.e., counterfactuals; what would have oc-curred in the absence of the intervention). In many cases where projects toreduce deforestation are examined such as the establishment of protectedareas, the control group is easier to determine, particularly if it is derived fromwithin the same country (17, 49, 50). In our case, the policy target is the rateof forest loss for an entire country, Guyana, and we choose as the controlgroup other countries that are similar to Guyana based on the underlyingstructural processes related to forest loss and forest cover (SI Appendix, 2).We adopt the synthetic control method (hereafter “synthetic matching”)from the field of economics and political science for comparative case studies(13, 14, 17). Synthetic matching provides a systematic and quantitative meansto construct counterfactuals and removes the subjectivity of identifying ap-propriate comparison units. As there may never be a perfect match to thetreated unit, synthetic matching builds the counterfactual scenario based on aweighted combination of potential comparison units that approximates thecharacteristics of the treated unit. We apply the synthetic control method toreproduce the tree cover loss that would have been observed for Guyana inthe absence of the Norway–Guyana REDD+ program.

In the selection of our donor pool countries (i.e., comparison countriesused to build the counterfactual of Guyana without the national REDD+intervention), it was important to use only those countries that share thesame structural processes that influence forest loss and not subject tostructural shocks during the program intervention (13). Thus, we restrictedthe donor pool to pretransition forest economies with HFLD countriesidentified by Hosonuma et al. (12) (see SI Appendix, 3: Identification ofDonor Pool Countries). Based on these criteria, our donor pool includes6 countries: Colombia, Republic of Congo (Congo), Democratic Republic ofCongo, Gabon, Peru, and Suriname (10, 11). Pretransition economic status isbased on the forest transition model, which posits that countries at differentstages of economic development are subject to different drivers of de-forestation (12, 26). All of our comparison countries had also submittedReadiness Preparation Proposals (R-PP), a preliminary requirement for theestablishment of a national REDD+ financing mechanism under the WorldBank’s Forest Carbon Partnership Facility. To account for the country level

REDD+ dynamics in the donor pool countries, such as access to nationalREDD+ funding, we report on metrics related to the ToC for national REDD+(SI Appendixes, 2 and 3).

As covariates in the synthetic matching, we include annual time series (2001 to2017) of proximate drivers of forest loss for pretransition economies identified byHosonuma et al. (12) based on the country level R-PPs as well as other national-level socioeconomic pressures that relate to forest loss identified by other studies(29, 36, 41, 51). These include mining (mineral rents), forest utilization (forestrents), agricultural expansion, population growth, gross domestic product (GDP),governance quality, and land area under some form of protection (SI Appendix,4). Annual data on agricultural expansion (percentage of land area), populationgrowth (percentage), GDP growth (percentage), forest rents (percentage ofGDP), and mineral rents (percentage of GDP) were extracted from the WorldBank Statistical Database (52). Land area under some form of protection wasextracted from the World Database on Protected Areas (53) and governanceeffectiveness from the World Governance dataset (54). In cases where therewere gaps in the annual time series of the covariates, we impute these valuesusing a structural modeling approach fitted by maximum likelihood (55).

To construct the synthetic counterfactual, synthetic matching assigns weightsto 1) each comparison country and 2) each covariate, based on the similaritybetweenboth theoutcomeof interest and the covariates for the country thatwasexposed to the policy intervention and the comparison countries that were notexposed to the policy intervention (14). The synthetic counterfactual is achievedby minimizing the mean-squared prediction error between the outcome for thetreated country and the synthetic counterfactual summed over all of the years inthe pretreatment period (SI Appendix, 4). The counterfactual should thus matchthe observed pretreatment deforestation time-series trajectory as the treatedunit. The effect of the policy intervention is then measured as the differencebetween the observed outcome from the treated unit and the outcome fromthe synthetic counterfactual for the period the project was implemented. Weimplement the synthetic control method with the Synth package in the openstatistical software R (16). We interpret the synthetic counterfactual for theNorway–Guyana REDD+ in terms of additionality, leakage, and permanence. Wereport our results from the synthetic matching process (weights assigned tocountries) as well as the placebo tests and sensitivity analysis conducted to testthe quality of the synthetic matching process in SI Appendix, 5.

ToC. As the Norway–Guyana REDD+ program did not have an explicit ToC toguide the impact evaluation, we adapt a generic ToC based on the in-ternational policy discourse for REDD+ as a climate mitigation strategy (56).We modify this generic REDD+ ToC based on the Norway–Guyana REDD+agreement to understand why an intervention such as the Norway–GuyanaREDD+ program is expected to contribute to climate mitigation using con-ditional financial payments for forest protection (57) (SI Appendix, 2). TheToC maps the path to achieve the outcome of emissions reductions underthe national REDD+ program (e.g., capacity and institutional strengtheningneeded for a national REDD+ forest governance framework). We quantifyprogress related to these activities and associated outputs related to the ToCfor Guyana and all other donor pool countries in SI Appendix, Table S2.

Data Availability. All data discussed in the paper are publicly available toreaders; please see refs. 23, 53, 54, and 55.

ACKNOWLEDGMENTS.We express our gratitude to the researchers and dataanalysts who have compiled the multiple global datasets and statistics thatwe employed in our analysis. We also thank Vitor Possebom for providingcomments on preliminary drafts of the manuscript and University of FloridaREDD+ Working Group for comments on the REDD+ conceptual diagram.

1. UNFCCC, Decision 1/CP.13 (United Framework Convention on Climate Change,2007). https://unfccc.int/resource/docs/2007/cop13/eng/06a01.pdf. Accessed 27 July2018.

2. N. Stern, Stern Review on the Economics of Climate Change (Cambridge UniversityPress, 2007).

3. B. Strassburg, R. K. Turner, B. Fisher, R. Schaeffer, A. Lovett, Reducing emissions fromdeforestation—The “combined incentives” mechanism and empirical simulations. Glob.Environ. Change 19, 265–278 (2009).

4. M. Norman, S. Nakhooda, The state of REDD+ finance. SSRN Electron. J. (2015) https://doi.org/10.2139/ssrn.2622743 (27 July 2018).

5. G. Simonet, et al., ID-RECCO, international database on REDD+ projects and pro-grams, linking economic, carbon and communities data (Version 3.0, 2018). http://www.reddprojectsdatabase.org. Accessed 29 June 2019.

6. A. E. Duchelle, G. Simonet, W. D. Sunderlin, S. Wunder, What is REDD+ achieving onthe ground? Curr. Opin. Environ. Sustain. 32, 134–140 (2018).

7. R. Fletcher, W. Dressler, B. Büscher, Z. R. Anderson, Questioning REDD+ and the futureof market-based conservation. Conserv. Biol. 30, 673–675 (2016).

8. K. H. Redford, C. Padoch, T. Sunderland, Fads, funding, and forgetting in three decades

of conservation. Conserv. Biol. 27, 437–438 (2013).9. A. Angelsen et al., Learning from REDD+: A response to Fletcher et al. Conserv. Biol.

31, 718–720 (2017).10. G. A. da Fonseca et al., No forest left behind. PLoS Biol. 5, e216 (2007).11. B. Griscom, D. Shoch, B. Stanley, R. Cortez, N. Virgilio, Sensitivity of amounts and

distribution of tropical forest carbon credits depending on baseline rules. Environ. Sci.

Policy 12, 897–911 (2009).12. N. Hosonuma et al., An assessment of deforestation and forest degradation drivers in

developing countries. Environ. Res. Lett. 7, 044009 (2012).13. A. Abadie, A. Diamond, J. Hainmueller, Comparative politics and the synthetic control

method. Am. J. Pol. Sci. 59, 495–510 (2015).14. A. Abadie, A. Diamond, J. Hainmueller, Synthetic control methods for comparative

case studies: Estimating the effect of California’s tobacco control program. J. Am.

Stat. Assoc. 105, 493–505 (2010).15. A. Abadie, J. Gardeazabal, The economic costs of conflict: A case study of the Basque

Country. Am. Econ. Rev. 93, 113–132 (2003).

Roopsind et al. PNAS Latest Articles | 7 of 8

SUST

AINABILITY

SCIENCE

Dow

nloa

ded

by g

uest

on

July

25,

202

1

16. J. Hainmueller, A. Diamond, M. J. Hainmueller, Synth Package (2014). http://www.stanford.edu/∼jhain//synthpage.html. Accessed 14 February 2018.

17. E. O. Sills et al., Estimating the impacts of local policy innovation: The syntheticcontrol method applied to tropical deforestation. PLoS One 10, e0132590 (2015).

18. K. H. Brodersen, F. Gallusser, J. Koehler, N. Remy, S. L. Scott, Inferring causal impactusing Bayesian structural time-series models. Ann. Appl. Stat. 9, 247–274 (2015).

19. S. Guo, M. W. Fraser, Propensity Score Analysis: Statistical Methods and Applications(Sage, ed. 2, 2015).

20. Y. Xu, Generalized synthetic control method: Causal inference with interactive Fixedeffects models. Polit. Anal. 25, 57–76 (2017).

21. Guyana Forestry Commission, Guyana REDD+ Monitoring Reporting and VerificationSystem (MRVS). Year 6 Interim Measures Report (2017). https://forestry.gov.gy/wp-content/uploads/2018/05/MRVS-Interim-Measures-Report-Year-6-Version-3.pdf. Accessed15 May 2018.

22. M. C. Hansen et al., High-resolution global maps of 21st-century forest cover change.Science 342, 850–853 (2013).

23. A. Angelsen, Ed., Moving Ahead with REDD: Issues, Options and Implications (Centerfor International Forestry Research, 2008).

24. B. C. Murray, Leakage from an Avoided Deforestation Compensation Policy: Concepts,Empirical Evidence, and Corrective Policy Options (Nicholas Institute for EnvironmentalPolicy Solutions, Duke University, 2008).

25. P. van Oosterzee, J. Blignaut, C. J. A. Bradshaw, iREDD hedges against avoided de-forestation’s unholy trinity of leakage, permanence and additionality. Conserv. Lett.5, 266–273 (2012).

26. A. S. Mather, The forest transition. Area 24, 367–379 (1992).27. L. Miles, V. Kapos, Reducing greenhouse gas emissions from deforestation and forest

degradation: Global land-use implications. Science 320, 1454–1455 (2008).28. Ministry of the Environment, Norway’s REDD+ Disbursement. Government.no (2018).

https://www.regjeringen.no/en/topics/climate-and-environment/climate/climate-and-forest-initiative/kos-innsikt/how-are-the-funds-being-spent/id734170/. Accessed 27June 2019.

29. C. Dezécache et al., Gold-rush in a forested El Dorado: Deforestation leakages and theneed for regional cooperation. Environ. Res. Lett. 12, 034013 (2017).

30. N. L. Alvarez-Berríos, T. Mitchell Aide, Global demand for gold is another threat fortropical forests. Environ. Res. Lett. 10, 014006 (2015).

31. M. Kalamandeen et al., Pervasive rise of small-scale deforestation in Amazonia. Sci.Rep. 8, 1600 (2018).

32. H. Overman, A. R. Cummings, J. B. Luzar, J. M. V. Fragoso, National REDD+ out-competes gold and logging: The potential of cleaning profit chains. World Dev. 118,16–26 (2019).

33. Joint Concept Note on REDD+ cooperation between Guyana and Norway (2011).https://www.regjeringen.no/globalassets/upload/md/2011/vedlegg/klima/klima_skogprosjektet/guyana/jointconceptnote_31mars2011.pdf. Accessed 16 May2018.

34. E. Romijn et al., Assessing change in national forest monitoring capacities of99 tropical countries. For. Ecol. Manage. 352, 109–123 (2015).

35. M. Brockhaus et al., REDD+, transformational change and the promise ofperformance-based payments: A qualitative comparative analysis. Clim. Policy 17, 708–730 (2017).

36. R. J. Smith, R. D. J. Muir, M. J. Walpole, A. Balmford, N. Leader-Williams, Governanceand the loss of biodiversity. Nature 426, 67–70 (2003).

37. L. Aukland, P. M. Costa, S. Brown, A conceptual framework and its application foraddressing leakage: The case of avoided deforestation. Clim. Policy 3, 123–136 (2003).

38. S. Henders, M. Ostwald, Accounting methods for international land-related leakageand distant deforestation drivers. Ecol. Econ. 99, 21–28 (2014).

39. M. Weisse, L. Goldman, 2017 was the second-worst year on record for tropical treecover loss. Global Forest Watch (2018). https://blog.globalforestwatch.org/data-and-research/2017-was-the-second-worst-year-on-record-for-tropical-tree-cover-loss. Accessed3 January 2019.

40. P. Potapov et al., The last frontiers of wilderness: Tracking loss of intact forest land-scapes from 2000 to 2013. Sci. Adv. 3, e1600821 (2017).

41. J. Busch, K. Ferretti-Gallon, What drives deforestation and what stops it? A meta-analysis. Rev. Environ. Econ. Policy 11, 3–23 (2017).

42. P. Meyfroidt, E. F. Lambin, K.-H. Erb, T. W. Hertel, Globalization of land use: Distantdrivers of land change and geographic displacement of land use. Curr. Opin. Environ.Sustain. 5, 438–444 (2013).

43. M. L. Ingalls, P. Meyfroidt, P. X. To, M. Kenney-Lazar, M. Epprecht, The transboundarydisplacement of deforestation under REDD+: Problematic intersections between thetrade of forest-risk commodities and land grabbing in the Mekong region. Glob.Environ. Change 50, 255–267 (2018).

44. Y. le Polain de Waroux, R. D. Garrett, R. Heilmayr, E. F. Lambin, Land-use policies andcorporate investments in agriculture in the Gran Chaco and Chiquitano. Proc. Natl.Acad. Sci. U.S.A. 113, 4021–4026 (2016).

45. J. Gan, B. A. McCarl, Measuring transnational leakage of forest conservation. Ecol.Econ. 64, 423–432 (2007).

46. Forest Carbon Partnership Facility, REDD+ Countries (Forest Carbon Partnership Facility,2018). https://www.forestcarbonpartnership.org/redd-countries. Accessed 27 July 2018.

47. N. Gorelick et al., Google Earth engine: Planetary-scale geospatial analysis for ev-eryone. Remote Sens. Environ. 202, 18–27 (2017).

48. N. Harris, C. Davis, E. D. Goldman, R. Petersen, S. Gibbes, Comparing Global andNational Approaches to Estimating Deforestation Rates in REDD+ Countries (WorldResources Institute, 2018). https://www.wri.org/publication/comparing-global-national-approaches. Accessed May 24 2019.

49. A. Blackman, L. Goff, M. Rivera-Planter, Does eco-certification stem tropical defor-estation? Forest stewardship council certification in Mexico. J. Environ. Econ. Manage.89, 306–333 (2018).

50. L. Fortmann, B. Sohngen, D. Southgate, Assessing the role of group heterogeneity incommunity forest concessions in Guatemala’s Maya biosphere reserve. Land Econ. 93,503–526 (2017).

51. A. Waldron et al., Reductions in global biodiversity loss predicted from conservationspending. Nature 551, 364–367 (2017).

52. The World Bank, DataBank: The World Bank. https://databank.worldbank.org/home.aspx. Accessed 13 December 2018.

53. UNEP-WCMC, World Database on Protected Areas. IUCN (2016). https://www.iucn.org/theme/protected-areas/our-work/world-database-protected-areas. Accessed 6 December2018.

54. D. Kaufmann, A. Kraay, M. Mastruzzi, The worldwide governance indicators: Meth-odology and analytical issues. https://openknowledge.worldbank.org/handle/10986/3913. Accessed 13 December 2018.

55. S. Moritz, T. Bartz-Beielstein, imputeTS: Time series missing value imputation in R. R J9, 207–218 (2017).

56. A. Martius et al., “Pathway to Impact: Is REDD+ a Viable Theory of Change?” inTransforming REDD+: Lessons and New Directions, A. Angelsen et al., Eds. (Center forInternational Forestry Research, 2018), pp. 17–28.

57. P. Rogers, Theory of Change: Methodological Briefs–Impact Evaluation No. 2., UNICEF-IRC (UNICEF Office of Research–Innocenti, 2014). https://www.unicef-irc.org/publications/747-theory-of-change-methodological-briefs-impact-evaluation-no-2.html. Accessed 17June 2019.

8 of 8 | www.pnas.org/cgi/doi/10.1073/pnas.1904027116 Roopsind et al.

Dow

nloa

ded

by g

uest

on

July

25,

202

1