Comparative and Alternative Modeling Approaches Subgroup

Draft Final Recommendations

October 14-15, 2010

Who We Are• Michael O’Hare• Bob Larson• Wally Tyner• Jay Noel• Seth Myer• Steve Kaffka• Jeremy Martin• Michelle Manion• Blake Simmons• Uwe Fritsche• John Sheehan• Manisha Singh• John Courtis• Peter Ward• Jim McKinney

Why We Are Here

• Subgroup focused on CARB’s choice to use GTAP to assess ILUC of biofuels

• What other economic modeling approaches are available for assessing land use change? GE and PE

• Strengths / weaknesses compared to GTAP

• Limitations of economic modeling• Recommendations

First Overarching Issue

• CARB’s Legal Constraint– Required to be fully transparent which has

been interpreted to require the use of models that are publically available and thus can be independently run to verify results

– Can CARB use proprietary data or information and results which from models that are not in the public domain?

Second Overarching Issue

• We were asked to assess the capability of models to “score” various biofuels on their lifecycle GHG impact– Not evaluating whether California’s LCFS as a policy reduces

GHG emissions• In choosing to “score” biofuels, CARB has assumed a

baseline and an increment of biofuel production and use– This raises the concern that non-linearities may affect the results– Removing the policy assessment from the modeling regime

means that, for example, impacts on commodity costs or food prices is not part of the evaluation

Should GTAP Be Replaced with Another Model?

• No single model or modeling system has been identified that necessarily– Gives the “correct” LCA result – Or even provides the ‘best” result at this time

• Models have their strengths and weaknesses– How can CARB take advantage of strengths

of various models to improve CARB’s assessment?

Comparison to Other Models

• Very difficult to numerically compare results from various models– Recent effort by JRC on behalf of EU demonstrates

that inconsistencies in definitions, scope, assumed elasticities, data sources, etc. can significantly affect results

– Comparing results of models does not necessarily indicate how good the models are at estimating ILUC• E.g. Even widely disparate results may be due to differences

in assumptions used in the model, thus making comparison’s of model capabilities very difficult

CARB Should Continue to Use GTAP but with Improvements

• Found no model or modeling system that is necessarily better than GTAP

• However, GTAP needs to improve– As with similar models, need to continue updating to

reflect new information and data• Need improved data on global agricultural and forest land

supplies and use; more regionally-specific estimates of land substitution elasticities

• “Elasticities” subgroup noted the high sensitivity of GTAP to assume elasticities

• Projected changes in agricultural practices such as use of genetically modified seeds, improvements in fertilizer application and trends toward no-till farming need to be established and reflected in GTAP predictions

CARB Should Continue to Use GTAP but with Improvements

• CARB should work with USDA, FAO and other widely recognized sources of data to determine where sufficiently detailed and up to date data is lacking and encourage its development with funding support– National estimates may not reflect California’s unique

farming systems and markets• Need to consider impacts on base cropping

patterns due to changes in population, wealth and diet and due to changes in land use policy and practices that influence land use outside of economic factors

CARB Should Continue to Use GTAP but with Improvements

• Recommendation: CARB should work with Purdue and other GTAP model developers to sponsor and independent periodic assessment of GTAP

GTAP Comparison to Other Models and Model Results

• While GTAP is a reasonable model to use in scoring the GHG impact of biofuel production, it is not the model of choice by other jurisdictions

• A developing effort is focusing on assessing the capabilities of other models, including attempts to compare model results– US EPA’s use of FASOM/FAPRI plus satellite data– JRC efforts to compare models

• CARB should monitor this work as part of an ongoing reassessment of biofuel land use impacts

GTAP Comparison to Other Models and Model Results

• Recommendation: CARB should set up an external advisory group (perhaps in cooperation with other regulatory bodies) to systematically review and compare ILUC GHG estimates from diverse models and complementary assessment approaches used elsewhere– At a minimum, this could highlight areas where GTAP is

inconsistent with other information, suggesting the need for focused assessment

– This group should engage additional experts on LUC, especially from regions/countries of greatest concern for iLUC

– Expertise should include experts in land use change, tropical ecology, legal, governance, and cultural institutions that affect LUC as well as economists

Use of Other Model Results and Information to Improve GTAP

• While GTAP is a good model, it needs to be verified and updated with the latest information

• Particularly for the US but likely also in other major agricultural areas, increasingly robust data is available on actual recent performance

• We recognize the importance of accessing annual (or as frequently as possible) data to determine how well GTAP is tracking performance and periodically updating GTAP

• Recommendation: CARB should fund a study of existing land use data and relevant market and economic data to determine how well GTAP (and other models) predicts land use change due to biofuels

Limitations of Economic Models• Validation of model results (from GTAP and other

models) is exceptionally difficult and resource-intensive: – Validation efforts require substantial improvements in our current

information base (e.g., own- and cross-price elasticities)– Land use and related markets are highly dynamic in nature, and

information bases will require consistent updating

• As noted earlier, improving our understanding of the role of biofuel demand alone on LUC requires better understanding of other empirical factors that drive land use change

Limitations of Economic Models• Some of these other factors are economic, but not well-

reflected in global markets and/or highly aggregated models:– Supply and demand forces at the local or regional level– Value of lands providing non-market services (e.g., biodiversity)

• Other factors are non-economic in nature, thus are entirely outside the scope of economic models:– Cultural influences (e.g., values and belief structures of

indigenous peoples)– Ecological factors (e.g., role of fire in forest carbon

sequestration)

Incorporating Other Assessments and Data Directly

• In addition to using other model results and data sources to verify GTAP or highlight areas for future improvement, these other results and data sources can be used directly

• In particular, regional assessments may provide results that are verifiable but differ from the predictions of the current GTAP model

• Rather than wait for GTAP to be updated to reflect this alternative information, GTAP results can be replaced or at least used to supplement GTAP results

Incorporating Other Assessments and Data Directly

• Recommendation: CARB should develop the research capacity to verify GTAP results with other sources of regionally-specific credible information and develop a capability to analytically integrate this alternative information into its GTAP-based assessment– A coordinated and cooperative effort with US EPA and

other governmental groups including US multistate groups and other governments should be considered

Future Model Development and Use

• Assessing land use impacts (GHG and other impacts) due to changes such as policy mandates supporting biofuels is an evolving discipline

• CARB should support cooperative research with other regulators in the US and abroad to establish a common framework (but not necessarily a common model) which would facilitate collection and incorporation of new data and analytical methods into existing regulations– External advisory group noted earlier could assist – Broadening the base of expertise to include other disciplines and

perspectives and experts from regions/countries of greatest concern