Overview of theRegional Technical Forum

Guidelines

January 22, 2013

After A Decade of Making Decisions the RTF Recently “Codified” Its

Decision-Making Process

Why Guidelines For Estimating Savings?

• RTF Mission– Reliable estimates of savings– Transparent methods for estimating savings

• RTF operated for 10 years transparently but its “rule book” evolved with experience

• RTF needed to capture its current best practices to ensure more consistent decision-making and operations

RESEARCH QUANTIFY DELIVER VERIFY

Brief History and Coming Events

• Guidelines Development– RTF identified need to codify guidance on how it does its

work– Developed and tested guidelines for savings estimation

methods. – Second version adopted 12-11-2013– Also, developed guidance on how to estimate measure

cost and lifetime

• Next Steps– Developing an overall Roadmap and integrating all

guidelines and supporting tools

Guidelines Scope and Intent

• Describe how the RTF selects, develops and maintains approved methods for estimating savings from the delivery of energy efficiency measures

• Four savings estimation methods:– Unit Energy Savings (UES) – RTF Decides on

– Standard Protocols – RTF Decides on

– Custom Measure Protocols– RTF provides guidance on

– Program Impact Evaluations – RTF provides guidance on

• Each method is intended to produce savings estimates of “comparable reliability” at the lowest reasonable cost

Key Concepts – Measure and Savings

• Measure– …changes in system configuration, equipment specifications or

operating practices – …reduces electric power consumption as a result of increases in the

efficiency of energy use, production, or distribution

• Savings– …difference in energy use between the baseline and post periods– … caused by the delivery of a measure– …terms “net” or “gross” are intentionally not used

Savings Reliability and Diligent Review

• Savings Reliability– …sampling, data collection and other errors are sufficiently small and

unbiased. – Sufficiency is determined by the collective opinion of the RTF

• Diligent Review– RTF uses estimates of parameters, e.g., average length of a residential

shower or heat/cool interaction factors, from studies performed by other agencies

– Diligent review required before use

Diligent Review

• Diligent review includes, but is not limited to understanding– characteristics of the sample studied– study’s data collection methods and analysis methods– variability of parameter estimates– feasibility of normalizing results to this region

Best Practice Savings Estimate

• Best Practice Savings Estimate– …relies on the best practical and reliable data collection and

estimation methods– …can be carried out with proven techniques and resources deemed

reasonable by the RTF– …may rely on parameter values from studies performed by other

agencies, if found to be sufficiently reliable based on diligent review.

Two Possible Measure Baselines

• Current Practice Baseline– …used if the measure affects systems, equipment or practices that are

at the end of their useful life– …includes measures delivering new equipment or practices– …baseline is defined by the recent typical choices of eligible end users

in purchasing new equipment and services

• Pre-Conditions Baseline– …used when the measure-affected equipment or practice still has

remaining useful life– …defined by typical existing conditions found among eligible end users – For custom measures choices of individual end users define typical

current practice conditions

Transparency and Accessibility

• All methods have documentation standards• Sources must be cited• Data and documents must be accessible• Must show your work

– Estimation algorithms– Derivation of input parameters– Software must be either inherently transparent, such as a Microsoft

Excel® workbook or fully documented

Method 1 - Unit Energy Savings (UES)

• RTF approves measure methods and savings values• Stable, unitized savings that can be reliably forecast

through the period defined by the measure’s sunset date• Reduce program delivery costs by simplifying the data

that must be collected– Verified count of delivered units is all that is required– Statistical or meta-statistical analysis used to reliably estimate

savings• Examples: CFL Res. Insulation

Getting to Unitized Savings (UES)*

• Statistical or meta-statistical data– Quality judged by relative error of mean savings estimate– Avoid when savings significantly interact with other measures due to

large sample needs

• Calibrated engineering models– Adjusted to individual cases or to the average characteristics and

consumption of groups• Ex. SEEM heating loads calibrated to billing data from representative

sample of SF homes– Savings expected to be regionally applicable– Significant interactions need to be dealt with

• Planning/Provisional provides a path

*The savings formerly known as “deemed”

Method 2 - Standard Protocols

• RTF approves saving estimation method• Intended for measures with widely varying savings, but

where data collection and analysis can be standardized• Site-specific data collection and analysis is the minimum

required for a reliable savings estimation– Known as simplest reliable savings estimation method, +/- 20% error

that is unbiased compared to best practice

• Data collection standardized and savings calculator developed for each protocol

• Examples:

T8 Lighting VFD

Method 3 - Custom Protocols

• RTF provides guidance on appropriate methods and documentation standard

• Appropriate for measures that require site-specific savings estimation planning, data collection and analysis – Likely numerous components affecting operation,

maintenance, and energy use of many separate systems or pieces of equipment

• Savings report required• Example:

Energy Management System (EMS)

Method 4 – Program Impact Evaluation

• RTF provides guidance on appropriate methods and documentation standard

• Based on reliable sample of program participants (and possibly non-participants) to determine the savings achieved during a program delivery period– UES claim & delivery; Standard Protocol faithfully applied

• Rely on the other three methods, plus guidance provided for “Other UES” measures

• Examples: O-POWER

Measure Category/Quality Standards

• Statistical or calibrated engineering data are available and reliable to characterize both the baseline and efficient-case energy consumption for measure-affected end uses

Proven

• RTF approves with special conditions requiring the collection of data from all or a sample of specific measure applications to improve savings estimation

Provisional

• Peer reviewed by RTF for measures with regional applicability and reasonable expectation that data will be collected to bring measure through an RTF path in the future

Planning

• RTF approves based on sound engineering analysis and applicability to the region and because cost to obtain quality data outweighs expected regional savings potential

Small Saver

Guidelines Measure StatusAc

tive

• Measure meets all requirements set forth in Guidelines

Und

er R

evie

w

• Errors need to be corrected

• Calculation updates with new identified data sources

• Calculation updates with data to be developed or sought O

ut o

f Com

plia

nce

• Lack of data sources identified that can bring the savings estimations into guideline compliance

• 1 year to bring research plan to RTF for review

De-

activ

ated

• Inadequate data exists to bring into compliance with Guidelines

• Federal/State codes require target efficiency level

Sunset Date

• As part of approving a UES or standard protocol measure, the RTF will establish a sunset date for the measure

• Beyond this date, the RTF may decide that the measure’s savings estimation method is no longer RTF-approved.

• …RTF records factors that justify each sunset date…– revisions to energy codes or federal standards– completion of RTF approved research plans– shifts in current practices of consumers

Provisional Category and Research Planning

• Defined for UES and Standard Protocol• RTF approves plans for research needed to

advance provisional measure to the proven category

• Provisional UES measures need data on efficient-case energy use

• Provisional standard protocol measures need data to prove simplest reliable method

Development of Application of RTF-Savings Estimation Methods

Prioritize and Classify

UES

Collect provisional data

Standard

Custom

Standard Protocols

Protocols for custom measures

Data available to prove savings

reliability

No

Yes

Program Impact Evaluation

New measures from regional R&D / program

plannning

Measure Specifications

Guidelines for program impact

evaluations

Research design and RTF peer review

Maintain and update according to sunset date

Legacy UES measures approved

before 6/1/2011

Approve measure

categories

Provisional

Proven

Planning

Small Saver

Maintain and update

according to sunset date

Program portfolios

including other- UES measures

Research to collect baseline and efficient

case UES measure data

UES savings values

Prove simplest reliable method

Cost and Lifetime Guidelines

• RTF also approved guidelines for measure cost and lifetime estimation in 2011

• Capital and O&M costs along with a measure’s expected useful lifetime are used by the RTF in evaluating regional cost-effectiveness using the ProCost model

• Program operators may use these values in performing their own cost-effectiveness analyses