1. LEARN. CONNECT. EXPERIENCE.www.aesp.org

2. HowtoDesignaGasProgramImpact H t D i G P I t Evaluation JonathanB.Maxwell,Energy&ResourceSolutions(ERS) KathrynParlin,WestHillEnergy&Computing January19,2011aesp.org 3. Agenda Examine results for 13 gas evaluations Realization rates Variation of realization rates Net-to-gross Non-gas impact Ramifications on evaluation designsaesp.org 4. Key Points Gas evaluation less mature than electric Gas realization rates somewhat lower than electric Evaluated project savings varies widely from reportedeven in large custom programs Account for interaction with other fuels Account for non-energy benefits & costsaesp.org 5. Gas Programs Examined Program or Portfolio or Targeted Measure TypeNo. ProgramsSample SizeResidential single family new construction125Multifamily retrofit16C/I new construction & retrofit t ti t fit248C/I performance contracting16g Commercial retrocommissioning434Industrial129Agriculture130Specialized ( i i S i li d (pipe insulation, bid program) l ti )273TOTAL13251aesp.org 6. Program & Evaluation Types Applicants typically estimated site-specific savings site specific Expectation of good estimates Evaluators estimated site-specific savings for all site specific Majority enhanced level of evaluation engineering rigor Many p g y program types & administrators; many yp ; y evaluation engineering firms Actor bias unlikelyaesp.org 7. Savings Realization Rates 0.68 median portfolio realization rate .08 .72.08 .91.21 .92.33 .93.53 .98.64 1.07.68 Lower RR than for similar electric portfolios 5 of 13 portfolios had both electric & gas RRs 0.70 median elec RR for the 5 0.53 median gas RR for the 5aesp.org 8. Error Ratio - Definition Measures variation of realization rates (Stratified ratio estimation) ti ti ti ) Lower is better Higher requires larger sample to get high precision Unrelated to magnitude of realization rate 0 4 to 1 0 typical in EE evaluation 0.4 1.0 0.4 to 0.6 typical for electric EE programs with site specific site-specific analysisaesp.org 9. Error Ratio - IllustrationFrom California Evaluation Framework Ch 13 June 2004 Framework, 13, 2004.aesp.org 10. Error Ratio - Results Median 1.04 Shown with 6 outlier projects removedaesp.org 11. Error Ratio - Observations Why are error ratios so poor? Less mature programs Difficult for applicants (& evaluators) to measure Baseline less clear (11% projects with 0 RR) ( p j ) Fuel switching Inherently difficult-to-predict measures (RCx) What to do? Intensively study gas measures Do not increase sample sizes at expense of rigor per siteaesp.org 12. Net to Gross Net-to-Gross Similar methods used as with electric Mostly enhanced for these portfolios 0 85 median NTG factor 0.85 0.31 to 1.09 range for NTG factor More consistent than realization rateaesp.org 13. Gas Measure Impact on Electricity, Steam, Oil Results from 3 portfolio evaluations Electric Impacts ( (kWh/ MMBtugas)Other Energy Impacts (MMbtu/ MMBtugas) (Commercial New Construction1.80.00C/I Retrofit15.10.02Manufacturing--Ag-Food Man fact ring Ag Food18.5 18 5naPortfolio Type yp Infrequent, but significant when it occurs Customers add ~22% in utility bill savings 22% Should include in benefit-cost calculationsaesp.org 14. Impact on Non-Energy Costs Non Energy Results from 4 program evaluations GroupDelivery ProgramC/INew Construction Existing Facilities gNon Energy Non-Energy Impact (/ MMBtugas) $0.00Loan Fund 1-4 Res. Multifamily $ $9.46 $1.22ENERGY STAR Homes$0.91Multifamily Building (Existing)$0.06 $0 06Infrequent but significant when it occurs Labor L b & water most common savings t t i Customer save up to $0.75 for every $1 gas saved Should include in benefit-cost calculations aesp.org 15. Summary - Evaluation Planning Gas programs are less mature than electric Expect large error ratios--evaluated savings varies widely from reported Dont sacrifice digging deeper for more sites Invest in enhanced M&V until programs mature Allow time for vigorous feedback with program staff A Account for interaction with other fuels tf i t ti ith th f l Account for non-energy benefits & costsaesp.org 16. Thank You Thank you to clients & prime contractors (CA) NYSERDA KEMA CPUC Itron SBWJon Maxwell, ERS Maxwell jmaxwell@ers-inc.comKathryn Parlin, West Hill Energy & Computing kathryn@westhillenergy.comaesp.org 17. 22nd National Conference&Expo February610,2012 HiltonSanDiegoBayFront Hilton San Diego Bay Front