centerpoint energy arkansas energy efficiency program ... · overall energy savings of its...

CenterPoint Energy Arkansas Energy Efficiency Program Portfolio Annual Report 2016 Program Year

Filed: May 1, 2017

APSC FILED Time: 5/1/2017 12:07:02 PM: Recvd 5/1/2017 11:42:32 AM: Docket 07-081-TF-Doc. 442

2016 ANNUAL REPORT

CENTERPOINT ARKANSAS, DOCKET NO. 07-081-TF

12

Portfolio. The HEAL program is no longer implemented in CenterPoint Arkansas’s

2017-2019 CIP Portfolio.

Planned or Proposed Changes to Program & Budget.

Funds are no longer budgeted for the HEAL program in the 2017-2019 CIP Portfolio and

weatherization projects will be implemented through the Company’s Saving Homes

Weatherization Program.

2.1.4 Residential Home Energy Reports Program

Program Description.

The Residential Home Energy Reports (HER) program provides customers with energy

usage information, including energy savings tips and personalized energy usage

comparisons, to encourage and motivate recipients to lower their energy usage.

CenterPoint Energy’s HER program is administered by Oracle, and combines technology,

direct marketing and behavioral science to deliver its Home Energy Reporting System.

The Home Energy Reporting System is a unique software platform that combines energy

usage data with customer demographics, housing and GIS data to develop specific,

targeted recommendations that educate and motivate consumers to reduce their energy

consumption.

Energy savings for the HER program are quantified by taking the difference in energy

usage between a control group that receives no program information and a statistically

identical group of customers that receive the home energy reports.

Program Highlights.

The HER program was approved by the APSC on June 30, 2011, and implementation of

the program began immediately. The Program continues to be a key driver of the overall

energy savings delivered through the CIP Portfolio.

Oracle analyzed customer data and established a control group and participant

group, and program participants received four home energy reports throughout the

heating season.

In 2016, approximately 96,900 customers were actively enrolled in the program.

In 2016, the program provided annual savings of 828,576 therms.


2016 ANNUAL REPORT


13

Program Budget, Savings and Participation.

Program Events and Training.

In order to preserve the scientific integrity of calculating energy savings on the

differences in usage between a control group and participant group, customers cannot opt

into the program if they are not randomly selected into the participant group. For this

reason, the program is not widely promoted to non-participants, and no mass marketing

of the program is conducted.

Internal training regarding responses and support for customer requests is provided to

CenterPoint Energy representatives. A select group of highly trained customer service

representatives and energy efficiency program staff were trained on customer service

tools provided by Oracle.

Savings.

Oracle calculates the energy savings from the program by comparing the program

participants against a similar size control group. The difference in energy usage will

show the effect the program had on participating Arkansas customers.

The savings reported by the program are net savings, and there are no free riders because

the program does not have an open enrollment process. In 2012, Protocol J of the TRM

2.0 was proposed by the IEM and Parties Working Collaboratively and was adopted by

the Commission. Protocol J sets guidelines and standards for behavior based programs.

Savings for the program conform to this guideline. The HER program yielded the

following residential energy savings:

Program Budget Actual % Plan Evaluated % Plan Evaluated % Plan Actual %

Program Year 2014 879,688$ 871,505$ 99% 1,020,310 1,102,598 108% 0 16,648 - 100,000 82,086 82%

Program Year 2015 879,688$ 848,589$ 96% 1,000,000 917,376 92% 0 13,852 - 100,000 103,695 104%

Program Year 2016 861,766$ 861,766$ 100% 1,000,000 828,576 83% 0 12,346 - 100,000 96,967 97%

Home Energy ReportsCost Energy Savings (Therms) ParticipantsDemand Savings (Therms)


2016 ANNUAL REPORT


14

Description of Participants.

Participants in the HER program are CenterPoint Arkansas customers who receive

personalized energy reports.

Challenges & Opportunities.

The initial HER program delivered to new participants often results in some customer

calls (substantially less than 1% of participants). Customer inquiries provide an

opportunity to discuss energy saving strategies and confirm or correct any incorrect data

that may have been associated with their report (e.g., square footage of the residence). In

2017-2019, customer communication via email and enhanced online resources provide a

major opportunity to improve the effectiveness of the program at driving behavioral

change.

Outlook for Continuation, Expansion, Reduction or Termination.

Per Order No. 85 in Docket No. 07-081-TF, the APSC has approved delivery of the HER

program in the Company’s 2017-2019 CIP Portfolio, and the Company expects that this

program will continue to deliver significant energy savings. CenterPoint Arkansas has

also put emphasis on increasing the cost-effectiveness of this program.

Planned or Proposed Changes to Program & Budget.

In 2017-2019, CenterPoint Arkansas will continue to provide customer reports via direct

mail, but the Company also plans to deliver electronic reports to customers through

email. While CenterPoint Arkansas projects the HER to be a major contributor to the

overall energy savings of its 2017-2019 CIP Porfolio, the company has significantly

reduced the program delivery budget to increase cost-effectiveness. The table below

compares the program’s 2016 budget to the budgets approved for years 2017-2019:

Annual Net Energy Savings (Therms) 828,576

Lifetime Energy Savings (Therms) 828,576

Demand Savings (Therms) 12,346

Home Energy Reports


Evaluation of 2016 DSM Portfolio

Submitted to:

CenterPoint Energy Arkansas

April 2017

Final

ADM Associates, Inc. VuPoint Research


Corporate Headquarters: 3239 Ramos Circle Sacramento, CA 95827 Tel: (916) 363-8383

ADM Associates Inc. Energy Research & Evaluation

39650 Liberty St. Suite 425 Fremont, CA 94539 Tel: (510) 371-0763

Prepared by:

Adam Thomas Daniel Chapman, P.E.

Jeremey Offenstein, Ph.D. Jennifer Shen

Lily Forest Don Dohrmann, Ph.D.


Executive Summary 1-1

1. Executive Summary

This report is to provide a summary of the evaluation effort of the 2016 Demand Side

Management (DSM) portfolio by the CenterPoint Energy Arkansas (CenterPoint). This

evaluation was conducted by ADM Associates and VuPoint Research (the Evaluators).

This report provides verified gross and net savings estimates for evaluated programs.

1.1 Summary of CenterPoint Demand Side Management Programs

In 2016, the CenterPoint DSM portfolio contained the following programs:

Heating Equipment CIP;

Water Heating CIP;

Commercial Boiler CIP;

C&I Solutions;

Commercial Food Service CIP;

Home Energy Reports;

Low Flow Showerhead & Faucet Aerator CIP; and

HEAL Partnership.

1.2 Evaluation Objectives

The goals of the 2016 EM&V effort are as follows:

For prescriptive measures, verify that savings are being calculated according to

appropriate TRM guidelines. For most measures, this constitutes applying TRM

V6.0 methodologies.

For custom measures, this effort comprises the calculation of savings according

to accepted protocols (such as IPMVP). This is to ensure that custom measures

are cost-effective and providing reliable savings.

Conduct process evaluation of all CenterPoint programs and of the portfolio

overall. This is to provide a comprehensive review of program operations,

marketing and outreach, quality control procedures, and program successes

relative to goals. From this, the Evaluators are to provide program and portfolio-

level recommendations for CenterPoint. Process evaluation activities include

interviews of key program actors, surveys of participants and non-participants,

literature reviews and best-practices assessments, and documentation of

program activities, successes, and shortcomings.

Conduct net-to-gross assessments. The Evaluators developed net-to-gross ratios

specific to each program.


2016 CenterPoint DSM Portfolio Final Evaluation Report


1.3 Summary of Findings

1.3.1 Impact Findings

Table 1-1 and 1-2 present the gross and net impact by program.

Table 1-1 Gross Impact Summary

Program

Annual Energy

Savings (Therms)

Lifetime Energy Savings

(Therms) Peak Therms

Gross

Realization

Rate Ex Ante Ex Post Ex Ante Ex Post Ex Ante Ex Post

Space Heating CIP 401,978 398,373 6,488,655 6,416,625 100,201.56 103,064.96 99.10%

Water Heating CIP 91,054 87,377 1,817,909 1,743,332 242.66 232.24 95.96%

Commercial Boiler CIP 84,070 84,070 1,681,400 1,681,400 1,339.29 1,339.29 100.00%

C&I Solutions 1,202,048 1,239,463 19,369,318 19,972,208 3,704.89 3,170.60 103.11%

Commercial Food Service CIP 62,047 86,184 744,564 1,034,209 169.99 236.12 138.90%

Home Energy Reports 857,467 828,576 857,467 828,576 12,862.34 12,346.00 96.63%

Low Flow Showerhead & Faucet Aerator CIP

148,643 148,643 1,486,430 1,486,430 445.93 445.93 100.00%

HEAL Partnership 0 0 0 0 0 0 N/A

Saving Homes Program 149,597 150,253 2,497,202 2,485,909 409.85 411.65 100.44%

Total 2,996,904 3,022,939 34,942,945 35,648,689 119,376.51 121,246.79 100.87%

Table 1-2 Net Impact Summary

Program

Annual Energy Savings

(Therms)

Lifetime Energy Savings

(Therms) Peak Therms

NTGR

Net

Realization

Rate Ex Ante Ex Post Ex Ante Ex Post Ex Ante Ex Post

Space Heating CIP 350,360 347,001 5,646,527 5,580,713 87,317.67 89,771.04 87.10% 99.04%

Water Heating CIP 75,774 72,391 1,512,812 1,444,208 204.16 194.47 82.85% 95.54%

Commercial Boiler CIP 67,491 67,491 1,349,828 1,349,828 1,075.18 1,075.18 80.28% 100.00%

C&I Solutions 1,200,000 1,273,739 18,976,010 20,142,070 3,083.22 3,272.68 102.77% 106.14%

Commercial Food Service CIP 47,900 66,534 574,803 798,409 131.23 182.29 77.20% 138.90%

Home Energy Reports 857,467 828,576 857,467 828,576 12,862.34 12,346.00 100.00% 96.63%

Low Flow Showerhead & Faucet Aerator CIP

147,948 147,948 1,479,479 1,479,479 429.43 429.43 99.53% 100.00%

HEAL Partnership 0 0 0 0 0 0 NA NA

Saving Homes Program 145,408 142,741 2,405,738 2,361,613 398.38 391.07 95.00% 98.17%

Total 2,892,348 2,946,421 32,802,664 33,984,896 105,501.61 107,662.16 97.47% 101.87%

The contribution to portfolio savings by program is summarized in Figure 1-1.




Figure 1-1 Contribution to Portfolio Net Savings by Program

Figure 1-2 and Figure 1-3 summarize the share of savings by measure category for

residential and non-residential segments, respectively.

Figure 1-2 Residential Portfolio Savings Share by Measure

2.1% 2.3% 2.5% 4.9% 5.1%

11.4%

28.3%

43.5%

0.0%5.0%

10.0%15.0%20.0%25.0%30.0%35.0%40.0%45.0%50.0%




Figure 1-3 C&I Portfolio Savings Share by Measure

Further, the Evaluators put the net savings into the context of CenterPoint’s 2016 goal.

Table 1-3 summarizes the performance against goals of programs evaluated in this

report.

Table 1-3 CenterPoint 2016 DSM Portfolio Performance against Goals

Program 2016 Verified Net

Therms 2016 Net Therms

Goal % of Goal Attained

Space Heating CIP 347,001 340,660 101.86%

Water Heating CIP 72,391 86,040 84.14%

Commercial Boiler CIP 67,491 92,160 73.23%

C&I Solutions 1,273,739 1,320,150 96.47%

Commercial Food service CIP 66,534 60,210 110.50%

Home Energy Reports 828,576 1,000,000 82.86%

Low Flow Showerhead & Faucet Aerator CIP 147,948 147,440 100.34%

HEAL Partnership 0 26,860 0.00%

Saving Homes Program 142,741 87,820 162.54%

Total 2,946,421 3,161,340 93.20%

The CenterPoint portfolio reached 92.2% of the filed savings goal. Percent of goal

attainted and budget spent by program is summarized in Figure 1-4.

20.5%

12.2% 11.2% 10.4%

7.7% 6.3% 6.1% 6.0%

3.8% 3.7% 3.7% 2.5% 2.4%

1.3% 1.1% 0.8% 0.1% 0.0%

5.0%

10.0%

15.0%

20.0%

25.0%




Figure 1-4 Summary of Goal Attainment & Budget Expenditure by Program

1.3.2 Process Findings

Following a review of present program offerings and interviews with utility and third

party implementation staff, the Evaluators found that:

1.3.2.1 Portfolio Findings

The programs are adequately staffed. CenterPoint has allocated sufficient

resources to successfully promote and implement their program offerings. The

staff is knowledgeable regarding energy efficiency technologies and the market

opportunities in their service territory. For some programs, they have brought in

personnel with past industry ties, allowing for improved marketing and outreach.

Many recommendations from prior program years were addressed in

CenterPoint’s 2017-2019 triennial filing that had not been addressed during the

bridge years in between program cycles.

CenterPoint QA/QC procedures were adequate for residential programs, in

accordance with industry best practices.

The portfolio formerly a gap in residential building envelope offerings, in that they

were available only through the AWP and the HEAL Partnership. The Saving

Homes Weatherization Program has improved CenterPoint’s building envelope

offerings markedly, and successfully applies this Consistent Weatherization

Approach.




CenterPoint reallocated budget from the C&I Solutions Program to allow for

increased participation in the Saving Homes Weatherization Program. This was

due to high customer demand in its first year of operation. The reallocation was

within CenterPoint’s allowed 10% cap. .

1.3.2.2 Space Heating CIP

Much of the success of the Space Heating CIP was driven by customers needing

to replace their air conditioner. HVAC contractors hired for this purpose

successfully sold high efficiency furnaces using the program incentive, but in

many instances the customer installed a standard efficiency air conditioner.

There was significant early replacement in 2016. This is an extension of the

activities of HVAC contractors in tacking on the sale of a high efficiency furnace

during the replacement of a failed central air conditioner.

CenterPoint has added an incentive for smart thermostats beginning in 2017, and

a package incentive for installing a furnace and a tankless water heater.

1.3.2.3 Water Heating CIP

The program has had a lack of commercial storage tank participation, and this is

being addressed in 2017 with the addition of a scaled incentive based on input

BTU for large storage tank water heaters. This will address a participation gap for

facilities that often do not install tankless systems (such as hospitals).

The Evaluators found significant differences in the customer decision-making

process and customer experience for residential storage tank and tankless

participants. Tankless participants are more affluent, more likely to have planned

their replacement, make greater use of the CenterPoint website, and are more

likely to have learned of the program from a contractor. Storage tank participants

typically complete emergency replacements, and learn of the program from

retailers or from bill inserts.

Tankless participants indicated some dissatisfaction when they needed to speak

with program staff about their application.

1.3.2.4 Commercial Boiler CIP

CenterPoint has removed boiler controls from the program beginning in 2017 due

to their being no participation in this channel since program inception.

1.3.2.5 Commercial Food Service CIP

CenterPoint has incorporated Evaluator recommendations from past years in its

2017 program plan. This includes the development of scaling incentives (with

examples including paying per-rack in a rack oven and having three size




categories for combi ovens based on number of pans) and increasing the

incentive for fryers.

The FSTC has produced savings values for broilers which should be considered

for the TRM in the next update.

1.3.2.6 C&I Solutions

The program fell slightly short of goal in 2016, but this is due in large part to a

significant funds reallocation to expand the Saving Homes Weatherization

Program. When adjusting for its reduced budget, the program successfully met

goal.

The program continues to leverage long term relationships with large industrial

customers. In the custom channel, 32.6% of net savings were from customers

that have participated in the program in prior years. The program has had

significant success in developing multiple projects and long-term energy planning

at the large industrial customers that have not opted out of the rider.

1.3.2.7 Home Energy Reports

The Home Energy Reports Program is still provides a significant (but declining)

portion of residential savings for CenterPoint. It’s share of the residential portfolio

declines from 64% to 58% from 2015-2016.

The program has had over 35% attrition in Wave 1, which is constituted of high-

using customers and produces the most savings per customer. This decline may

warrant new strategies on how CenterPoint approaches behavioral programs

going forward.

1.3.2.8 Low Flow Showerhead & Faucet Aerator CIP

The program has continued the success seen in all prior program years and is

providing consistent and reliable savings for CenterPoint.

CenterPoint has added effective screening for electric water heating, reducing

the rate of electric water heating found among program participants.

1.3.2.9 HEAL Partnership

Similar to 2015, the program was defunded for 2016 and is no longer part of

CenterPoint’s portfolio in 2017.

1.3.2.10 Saving Homes Weatherization Program

The program successfully applied the Consistent Weatherization Approach and

produced significant savings in its first year of operation for CenterPoint. This

program will be expanded in the 2017-2019 cycle.




The program got off to a quick start by using contractors already engaged in

Entergy Arkansas’ Home Performance with Energy Star Program.

1.3.3 Recommendation Summary

In 2015, 15 program or portfolio level recommendations were provide to CenterPoint as

part of the EM&V of their portfolio. The Evaluators reviewed CenterPoint’s response to

recommendations from the 2015 EM&V report and categorized them as follows:

1) Adopted. This applied to recommendations that pertained to the correction of an

issue (such as using an incorrect baseline methodology) or modifications in

program outreach that do not require a filing.

2) Under consideration. This applies most typically to larger recommendations

that would require APSC approval. This included the development of multiple-

measure bonus incentives and adding vendor incentives for boiler controls.

3) Rejected. This applies to recommendations which are reviewed by CenterPoint

and rejected. This included recommendations such as dataset consolidation for

the boiler and food service programs which were not needed due to low

participation.

4) Not applicable. This would apply to recommendations which are no longer

applicable to the CenterPoint portfolio.

5) Incomplete. This applies to recommendations which were included in the 2013

EM&V report but have either not yet been adopted or have been explicitly

rejected by CenterPoint. This included the addition of points of contact for

commercial customers for the Space Heating and Water Heating CIPs.

The responses recommendations are summarized in Figure 1-5




Figure 1-5 Summary of Status of 2015 Recommendations

1.4 Report Organization

This report is organized with one chapter providing the full impact and process summary

of a specified program. The report is organized as follows:

Chapter 3 provides portfolio-level and cross-cutting findings;

Chapter 4 provides results for the Space Heating Equipment CIP;

Chapter 5 provides results for the Water Heating CIP;

Chapter 6 provides results for the Commercial Boiler CIP;

Chapter 7 provides results for the C&I Solutions Program;

Chapter 8 provides results for the Commercial Food Service CIP;

Chapter 9 provides results for the Home Energy Reports Program;

Chapter 10 provides results for the Low Flow Showerheads & Faucet Aerators

CIP;

Chapter 11 provides the results for the HEAL Partnership;

Chapter 12 provides the results for the Saving Homes Program;

Chapter 13 provides a summary of recommendations for TRM updates; and

Appendix A provides the site-level custom reports for the C&I Solutions Program.


General Methodology 2-1

2. General Methodology

This section details general impact evaluation methodologies by program-type as well

as data collection methods applied. This section will present full descriptions of:

Gross Savings Estimation;

Sampling Methodologies;

Free-Ridership determination;

Process Evaluation Methodologies; and

Data Collection Procedures.

2.1 Glossary of Terminology

A first step to detailing the evaluation methodologies, the Evaluators provide a glossary

of terms to follow1:

Ex Ante – Forecasted savings used for program and portfolio planning purposes

(from the Latin for “beforehand”

Ex Post – Savings estimates reported by an evaluator after the energy impact

evaluation has been completed (From the Latin for “From something done

afterward”)

Deemed Savings – An estimate of an energy savings or demand savings

outcome (gross savings) for a single unit of an installed energy efficiency

measure. This estimate (a) has been developed from data sources and analytical

methods that are widely accepted for the measure and purpose and (b) are

applicable to the situation being evaluated. (e.g., assuming 17.36 Therms

savings for a low-flow showerhead)

Gross Savings – The change in energy consumption and/or demand that results

directly from program-related actions taken by participants in an efficiency

program, regardless of why they participated

Gross Realization Rate – Ratio of Ex Post Savings / Ex Ante Savings (e.g. If

ADM verifies 15 Therms per showerhead, Gross Realization Rate = 15/17.36 =

86%)

Free-Rider – A program participant who would have implemented the program

measure or practice in the absence of the program. Free riders can be total,

partial, or deferred.

1 Arkansas TRM V6.0, Volume 1, Pg. 86-92




Spillover – Reductions in energy consumption and/or demand caused by the

presence of the energy efficiency program that exceed the program-related gross

savings of the participants. There can be participant and/or non-participant

spillover rates depending on the rate at which participants (and non-participants)

adopt energy efficiency measures or take other types of efficiency actions on

their own (i.e., without an incentive being offered).

Net Savings – The total change in load that is attributable to an energy efficiency

program. This change in load may include, implicitly or explicitly, the effects of

free drivers, free riders, energy efficiency standards, changes in the level of

energy service, and other causes of changes in energy consumption or demand.

(e.g., if Free-Ridership for low-flow showerheads = 50%, net savings = 15

Therms x 50% = 7.5 Therms)

Net-to-Gross-Ratio (NTGR) = (1 – Free-Ridership % + Spillover %), also defined

as Net Savings / Gross Savings

Ex Ante Net Savings = Ex Ante Gross Savings x Ex Ante Free-Ridership Rate

Ex Post Net Savings = Ex Post Gross Savings x Ex Post Free-Ridership Rate

Net Realization Rate = Ex Post Net Savings / Ex Ante Net Savings

Effective Useful Life (EUL) – An estimate of the median number of years that the

efficiency measures installed under a program are still in place and operable.

Gross Lifetime Therms = Ex Post Gross Savings x EUL

2.2 Overview of Methodology

The proposed methodology for the evaluation of the 2016 CenterPoint DSM Portfolio is

intended to provide:

Net impact results at the 90% confidence and +/-10% precision level; and

Program feedback and recommendations via process evaluation; and

In doing so, this evaluation will provide the verified net savings results, provide the

recommendations for program improvement, and ensure cost-effective use of ratepayer

funds. By leveraging experience and lessons learned from prior evaluations, the 2016

evaluation is streamlined to focus on areas in needed of research and improvement.

2.3 Sampling

Sampling is necessary to evaluate savings for the CenterPoint DSM portfolio insomuch

as verification of a census of program participants is typically cost-prohibitive. As per

evaluation requirements set forth by the Independent Evaluation Monitor (IEM), samples

are drawn in order to ensure 90% confidence at the +/- 10% precision level. Programs

are evaluated on one of three bases:




Census of all participants

Simple Random Sample

Stratified Random Sample

2.3.1 Census of Participants

A census of participant data was used for select programs where such review is

feasible. For example, the Home Energy Reports program’s savings estimates are

based on a regression model that incorporates billing data for a census of program

recipients. Programs that received analysis of a census of participants include:

Home Energy Reports;

Commercial & Industrial Solutions – Custom Component

2.3.2 Simple Random Sampling

For programs with relatively homogenous measures (largely in the residential portfolio),

ADM conducted a simple random sample of participants. The sample size for

verification surveys is calculated to meet 90% confidence and 10% precision (90/10).

The sample size to meet 90/10 requirements is calculated based on the coefficient of

variation of savings for program participants. Coefficient of Variation (CV) is defined as:

𝐶𝑉(𝑥) = 𝑆𝑡𝑎𝑛𝑑𝑎𝑟𝑑 𝐷𝑒𝑣𝑖𝑎𝑡𝑖𝑜𝑛 (𝑥)

𝑀𝑒𝑎𝑛(𝑥)

Where x is the average Therms savings per participant. Without data to use as a

basis for a higher value, it is typical to apply a CV of .5 in residential program

evaluations. The resulting sample size is estimated at:

𝑛0 = (1.645 ∗ 𝐶𝑉

𝑅𝑃)

2

Where,

1.645 = Z Score for 90% confidence interval in a normal distribution

CV = Coefficient of Variation

RP = Required Precision, 10% in this evaluation

With 10% required precision (RP), this calls for a sample of 68 for programs with a

sufficiently large population. However, in some instances, programs did not have

sufficient participation to make a sample of this size cost-effective. In instances of low

participation, ADM then applied a finite population correction factor, defined as:

𝑛 =𝑛0

1 +𝑛0

𝑁⁄

Where




n0 = Sample Required for Large Population

N = Size of Population

n = Corrected Sample

For example, if a program were to have only 100 participants, the finite population

correction would result in a final required sample size of 41. The Evaluators applied

finite population correction factors in instances of low participation in determining

samples required for surveying or onsite verification. Programs subject to Simple

Random Sampling include:

Space Heating CIP – Residential;

Water Heating CIP – Residential;

Low Flow Showerhead & Faucet Aerator CIP; and

2.3.3 Stratified Random Sampling

For the CenterPoint Commercial & Industrial programs, Simple Random Sampling is not

an effective sampling methodology as the CV values observed in business programs

are typically very high because the distributions of savings are generally positively

skewed. Often, a relatively small number of projects account for a high percentage of

the estimated savings for the program.

To address this situation, we use a sample design for selecting projects for the M&V

sample that takes such skewness into account. With this approach, we select a number

of sites with large savings for the sample with certainty and take a random sample of

the remaining sites. To further improve the precision, non-certainty sites are selected for

the sample through systematic random sampling. That is, a random sample of sites

remaining after the certainty sites have been selected is selected by ordering them

according to the magnitude of their savings and using systematic random sampling.

Sampling systematically from a list that is ordered according to the magnitude of

savings ensures that any sample selected will have some units with high savings, some

with moderate savings, and some with low savings. Samples cannot result that have

concentrations of sites with atypically high savings or atypically low savings. As a result

of this methodology, the required sample for the C&I Industrial Solutions Program was

reduced to 24 with one certainty stratum and five sample strata. Programs that were

evaluated using stratified random sampling include:

Space Heating CIP – Non-Residential;

Water Heating CIP – Non-Residential;

Commercial Boiler CIP;

Commercial Food Service CIP; and




Commercial & Industrial Solutions – Direct Install Component.

2.3.4 Free-Ridership

In determining ex post net savings for the CenterPoint DSM portfolio, the Evaluators

provide estimates of free-ridership for individual programs. Free-riders are program

participants that would have implemented the same energy efficiency measures at

nearly the same time absent the program. As per TRM guidelines, free-riders are

defined as:

“…program participants who received an incentive but would have installed the same

efficiency measure on their own had the program not been offered. This includes partial

free riders, defined as customers who, at some point, would have installed the measure

anyway, but the program persuaded them to install it sooner or customers who would

have installed the measure anyway but the program persuaded them to install more

efficient equipment and/or more equipment. For the purposes of EM&V activities,

participants who would have installed the equipment within one year will be considered

full free riders; whereas participants who would have installed the equipment later than

one year will not be considered to be free riders (thus no partial free riders will be

allowed).”2

Given this definition, participants are defined as free-riders through a binary scoring

mechanism, in being either 0% or 100% free-riders.

2.3.4.1 Prescriptive Free-Ridership

The general methodology for evaluating free-ridership among prescriptive program

participants involved examination of four factors:

(1) Demonstrated financial ability to purchase high efficiency equipment absent the

rebate

(2) Importance of the rebate in the decision-making process

(3) Prior planning to purchase high efficiency equipment

(4) Importance of the contractor in influencing the decision-making process

In this methodology, Part (1) is essentially a gateway value, in that if a participant does

not have the financial ability to purchase energy efficient equipment absent a rebate, the

other components of free-ridership become moot. As such, if they could not have

afforded the high efficiency equipment absent the rebate, free-ridership is scored at 0%.

If they did have the financial capability, the Evaluators then examine the other three

components. The respondent is determined to be a free-rider based upon a

preponderance of evidence of these three factors; that is, if the respondent’s answers

indicate free-ridership in two or more of these three components, they are considered

2 Arkansas TRM V6.0, Pg. 49.




free-riders. Specific questions and modifications to this general methodology are

presented in the appropriate program chapters.

For residential programs, free-ridership is calculated as the average score determined

for the sample of participants surveyed. This value is then applied to the program-level

savings to discount savings attributable to free-ridership.

2.3.4.2 Custom Free-Ridership

For custom projects from the C&I Solutions program, free-ridership is assessed on a

case-study basis, through which the Evaluators conduct an in-depth interview that

includes a battery of questions addressing:

The timing of learning of the program relative to the timing of the planning of the

retrofit;

The impact the program incentive has on measure payback relative to the stated

payback requirements by the respondent;

Whether the respondent learned of the energy efficiency measure from a

program-funded audit; and

Whether any influence the program had in modifying the project affected savings

by greater than 50%.

In the C&I Solutions chapter, the free-rider “case studies” are provided for every custom

project.

2.4 Process Evaluation

2.4.1 General Approach

The Evaluators’ general approach to process evaluation begins with a review of the

tests for timing and appropriateness of process evaluation as defined in Protocol C of

the TRM V6.0. In this review, the Evaluators determine what aspects of the program

warrant a process evaluation (due to issues identified in the 2016 evaluations). Most

CenterPoint programs over-performed, and as such most of the 2016 process

evaluation activity was focused around identifying CenterPoint and implementer

response to 2016 recommendations.

The 2016 process overviews began with interviews of program staff. These interviews,

along with guidance from IEM protocols, inform the establishment of goals for the

process evaluation, provide background history of programs, and give an introduction to

portfolio-level issues. From this, the Evaluators then develop a list of data collection

activities. The data collection procedures for process evaluations typically included:

Participant Surveying. The Evaluators surveyed statistically significant samples

of participants in each program in order to provide feedback for the program and

provide an assessment of participant satisfaction.




In-Depth Interviews. The Evaluators conducted in-depth interviews with high-

level program actors, including CenterPoint program staff, third-party

implementation staff, and program Trade Allies. These interviews are semi-

structured, in having general topics to be covered, without fully prescribed

question and answer frameworks.

Review of Marketing Materials. The Evaluators reviewed marketing materials for

each program, providing feedback as to the appropriateness of the message in

reaching its target audience, the breadth of the audience that the effort is

attempting to reach, and identifying possible cross-promotional opportunities.


Portfolio-Level Findings 3-1

3. Portfolio-Level Findings

This chapter provides a summary of the portfolio-level findings and any cross-cutting

evaluation activities that occurred over the course of the 2016 EM&V Effort. Specifically,

this chapter includes:

A summary of program and portfolio performance in 2016;

A summary of EM&V activities and expenditures in 2016;

High-level findings that cut across programs.

3.1 Summary of EM&V Effort

Table 3-1 summarizes the EM&V expenditures by the Evaluators, total EM&V

expenditures by all parties, and total program budgets.

Table 3-1 CenterPoint DSM Portfolio 2016 EM&V Expenditures

Total EM&V

Expenditures

Evaluators’

EM&V

Expenditures

2016 Program

Expenditures

Evaluators’

EM&V as % of

Budget

576,863.60 $183,749.93 $6,357,547 2.32%

All programs received a full process evaluation in 2012 and limited process reviews in

2013 and 2014. Table 3-2 summarizes the data collection efforts for the 2016 EM&V

effort. “Interviews” should be distinguished from “Surveys” in that “Interviews” reflect

semi-structured, in-depth discussions with high-level program actors (such as utility staff

and third-party implementation staff) whereas surveys are fully-structured and typically

conducted with program participants.

Table 3-2 Summary of Data Collection Efforts

Program # Site Visits # Surveys # Interviews

Space Heating CIP 26 87 2

Water Heating CIP 0 82 6

C&I Boiler CIP 0 0 2

C&I Solutions 25 33 4

Commercial Food Service CIP 4 0 2

Home Energy Reports 0 800 1

Low Flow Showerhead & Faucet Aerator CIP 0 0 1

HEAL Partnership 0 0 1

Total 55 1,002 19

3.2 Tests of Portfolio Comprehensiveness

The Arkansas Public Service Commission has in place a set of criteria in order to

determine whether a DSM portfolio qualifies as “Comprehensive”. These criteria are:




Factor 1: Whether the programs and/or portfolio provide, either directly or

through identification and coordination, the education, training, marketing, or

outreach needed to address market barriers to the adoption of cost-effective

energy efficiency measures;

Factor 2: Whether the programs and/or portfolio, have adequate budgetary,

management, and program delivery resources to plan, design, implement,

oversee and evaluate energy efficiency programs;

Factor 3: Whether the programs and/or portfolio, reasonably address all major

end-uses of electricity or natural gas, or electricity and natural gas, as

appropriate;

Factor 4: Whether the programs and/or portfolio, to the maximum extent

reasonable, comprehensively address the needs of customers at one time, in

order to avoid cream-skimming and lost opportunities

Factor 5: Whether such programs take advantage of opportunities to address

the comprehensive needs of targeted customer sectors (for example, schools,

large retail stores, agricultural users, or restaurants) or to leverage non-utility

program resources (for example, state or federal tax incentive, rebate, or lending

programs)

Factor 6: Whether the programs and/or portfolio enables the delivery of all

achievable, cost-effective energy efficiency within a reasonable period of time

and maximizes net benefits to customers and to the utility system;

Factor 7: Whether the programs and/or portfolio, have evaluation, measurement,

and verification ("EM&V") procedures adequate to support program

management and improvement, calculation of energy, demand and revenue

impacts, and resource planning decisions.

The Evaluators reviewed the CenterPoint programs and portfolio in order to assess

whether it was in compliance with the APSC Comprehensiveness Goals. In assessing

these metrics, the Evaluators score them on numerous subcomponents. The scoring

methodology is as follows:

: Meets all requirements and is in full compliance with this performance indicator

: Meets some requirements and is in partial compliance with this performance indicator

: Is not in compliance with this performance indicator.

NA: Performance indicator is not applicable to this program.

The HEAL Partnership has been changed to “N/A” for all comprehensiveness factors,

due to CCI ceasing implementation in 2015.




3.2.1 Factor 1: Education, Training, Marketing, and Outreach

3.2.1.1 Assessment of Education

The Evaluators assessed the educational components of the CenterPoint programs, in

order to identify whether the programs were providing potential participants with the

needed information to guide their decision-making, and whether the channels used to

reach the target markets are appropriate. The Evaluators found that:

CenterPoint’s programs used a range of channels to provide educational

materials to their programs’ target markets. The educational materials included

brochures, case studies, and presentations to trade & industry groups.

CenterPoint program staff conducts outreach and education through a wide

range of potential program partners, including contractors, retailers, home

builders, and local governments.

The breadth of educational materials by program is summarized in Table 3-3.

Table 3-3 Assessment of Customer Education by Program

Program Provides

Educational Materials

Outreach Through Multiple Channels

Education Targeted to

Specific Market Barriers

Coordination of Education by Multiple

Entities

Space Heating CIP

Water Heating CIP

Commercial Boiler CIP

Commercial Food Service CIP

C&I Solutions

Home Energy Reports NA NA

Low Flow Showerhead & Faucet Aerator CIP NA

HEAL Partnership NA NA NA NA

Saving Homes Program

3.2.1.2 Assessment of Training

The Evaluators reviewed each CenterPoint program to assess whether:

1) Whether the program is trade ally-driven

2) If not, is it a program that could or should be trade ally-driven

3) The program provides training classes to support their program offerings

4) Whether the programs need trade ally certification




Table 3-4 Assessment of Trade Ally Training by Program

Program Trade Ally Training Offered

Training Requirements

Adhere to Best Practices

Trade Allies Participate in Training

Space Heating CIP

Water Heating CIP


Commercial Food Service CIP NA

C&I Solutions

Home Energy Reports NA NA NA Low Flow Showerhead & Faucet Aerator CIP NA NA NA HEAL Partnership NA NA NA


The Commercial Food Service CIP has several categories marked as “NA” in that it is

driven by equipment vendors, but that their training only constituted being informed on

identifying qualifying equipment and instruction on the application process. Technical

training was not provided (and was not needed).

CenterPoint does not require trade ally registration to participate, except for in the

Saving Homes Program. Their approach has been to allow all licensed dealers or

contractors to apply for the appropriate equipment rebates. The Evaluators have

concluded that this has not to-date affected the quality assurance of the programs.

3.2.1.3 Marketing & Outreach

The Evaluators reviewed the marketing and outreach strategies associated with each of

the CenterPoint programs. These strategies were reviewed to assess whether they

adequately addressed the relevant participant barriers, the extent to which trade allies

were actively marketing the program (where appropriate), and whether the materials

were correctly targeted in marketing a comprehensive approach to energy efficiency.

A summary of the Evaluators’ assessment of CenterPoint’s marketing and outreach is

presented in Table 3-5.

Table 3-5 Assessment of Marketing & Outreach by Program

Program Marketing Trade Marketing Marketing




Addresses Specific Barriers

Allies Promote Program

Support Provided to Trade

Allies

Performed Through Diverse

Channels

Space Heating CIP

Water Heating CIP



C&I Solutions

Home Energy Reports NA NA NA

Low Flow Showerhead & Aerator CIP NA NA

HEAL Partnership NA NA NA NA


After reviewing the marketing and outreach materials, the Evaluators concluded that:

Most programs have marketing materials that address specific barriers

associated with the targeted segments or technologies.

CenterPoint has initiated sector-specific marketing, including fact sheets for

restaurants and food processing plants.

The CenterPoint programs are marketed through a diverse range of channels,

including mass-media advertising, online advertising, meetings and training

sessions with professional organizations and trade groups, and partnered

marketing with municipal governments.

The Saving Homes Program intentionally had limited marketing in 2016 due to its

low first-year budget and concerns that CenterPoint had about too much program

interest at the outset. This will be resolved when the program’s budget is

expanded.

3.2.2 Factor 2: Budgetary, Management, and Program Delivery Resources

Several performance indicators were assessed in reviewing the adequacy of budgetary,

management, and program delivery resources. This included:

Self-reports from program management staff

Cost per Therm saved

Review of trade ally resources dedicated to program promotion.




Table 3-6 Assessment of Budgetary, Management, and Program Delivery Resources by Program

Program

Budget is Sufficient to

Support Program

Goals

Cost per-Therm

Aligns with Program

Plan

Program Has

Sufficient Staffing

Program Has

Sufficient Trade Ally Support

Space Heating CIP Water Heating CIP Commercial Boiler CIP


C&I Solutions

Home Energy Reports NA NA

Low Flow Showerhead & Aerator CIP NA NA HEAL Partnership NA NA NA NA


From this review, the Evaluators concluded that the CenterPoint portfolio overall has the

adequate budget and staff allocations. Programs were credited with full compliance if

acquisition costs exceeded plan values by no more than 10%. Programs were credited

with partial compliance if acquisition costs exceeded program plan values by no more

than 20%. Figure 3-1 summarizes the planned and actual first-year savings acquisition

costs.

Figure 3-1 Planned vs. Actual Acquisition Costs

The portfolio overall had acquisition costs that were 3.0% higher than the

program plan.




Three programs had costs that were at least 10% higher than planned: Water

Heating CIP, Commercial Boiler CIP, and Home Energy Reports.

Some programs had significantly lower acquisition costs than planned; the

Commercial Food Service CIP acquisition costs were 36% lower than planned,

and the new Saving Homes Program had costs that were 27% lower than

planned.

C&I Solutions has historically been among the lowest-cost programs offered by

CenterPoint. In 2016, acquisition costs for the program reached an all-time low of

$1.56/Therm.

3.2.3 Factor 3: Addressing Major End-Uses

The Evaluators identified the end-uses served by each of the CenterPoint programs.

Most CenterPoint programs are designed around a specific technology or end-use.

Table 3-7 summarizes the end-uses addressed by each program.

Table 3-7 End-Uses Addressed by Program

Program HVAC Hot

Water Food

Service Building Envelope

Industrial Process

Behavioral

Space Heating CIP Water Heating CIP Commercial Boiler CIP


C&I Solutions

Home Energy Reports

Low Flow Showerhead & Aerator CIP

HEAL Partnership NA NA NA NA NA NA


Measure targeted Measure offered Measure not offered

Presently, the CenterPoint portfolio covers all end-uses. In past Evaluations, it was

found that CenterPoint was not adequately capturing the emergency replacement

market for water heaters and was not providing an adequate offering for building

envelope improvements. Both of these issues were resolved in 2016 and we have

revised their comprehensiveness score to reflect this.

3.2.4 Factor 4: Comprehensively Addressing Customer Needs

To assess Factor 4, the Evaluators reviewed CenterPoint programs to discern the

extent of:

Program-provided technical assistance;

Incentives of comprehensive projects/measure suites; and

Tiered incentives for higher efficiency levels.




The CenterPoint portfolio has no specific requirements for installation of multiple

measures. Customers can participate to an extent of their choice. This is a program

best-practice in enabling customers to engage in energy efficiency in a manner in

accordance with their budget constraints.

Table 3-8 summarizes the comprehensiveness of offerings for each program.

Table 3-8 Assessment of Project Comprehensiveness by Program

Program

Technical Assistance

and/or Audits

Information Provided

Comprehensive for Efficiency

Bundled Incentives

for Multiple

Measures

Tiered Incentives

for Premium Efficiency

Trade Ally Incentives

for Premium Efficiency

Space Heating CIP Water Heating CIP



C&I Solutions

Home Energy Reports NA NA NA

Low Flow Showerhead & Aerator CIP NA NA NA

HEAL Partnership NA NA NA NA NA

Saving Home Program

Findings from the assessment of this factor included:

Most CenterPoint prescriptive programs offer incentives to trade allies for

installation of top-tier efficiency measures. This has included incentives for

condensing furnaces, tankless water heaters, and high efficiency food service

equipment. Trade ally incentives are not offered for the Commercial Boiler CIP,

but could be potentially viable for boiler controls.

The CenterPoint portfolio offers tiered incentives for premium efficiency across all

of their rebate programs. This includes:

- The incentives for the Space Heating CIP increase from $400 to $600 for

units with 95% AFUE or greater.

- Incentives in the Water Heating CIP range from $75 for storage tank water

heaters to $500 for tankless water heaters

- High efficiency boiler incentives are $1,400/MMBtuh for units < 92%

efficient and $2,000/MMBtuh for units with 92% efficiency or greater.

- The Commercial Food Service CIP does not differentiate between

efficiency levels. For food service, the range of efficiency levels is not as

wide as for other equipment types. However, this program could

potentially benefit from tiered incentives for different equipment sizes

(such as single vs. double-sized ovens).




- The C&I Solutions program pays an incentive per verified Therm, and as a

result projects with higher savings are by design paid a higher incentive.

The CenterPoint portfolio has programs that bundle on-site technical assistance

with direct installation.

The range of technical assistance varies by program. The Space Heating, Water

Heating, and Commercial Boiler CIPs offer technical assistance through program

trade allies. The level of on-site technical assistance is lower for the Commercial

Food Service CIP in that the market is driven by in-store contact with vendors

rather than by on-site assessment. C&I Solutions provides on-site technical

assistance that is directly funded by the program.

The programs have procedures for following up with customers after their

participation, which includes thank-you calls or emails and verification inspection.

Marketing materials typically make attempts at cross-promotion of programs.

3.2.5 Factor 5: Targeting Market Sectors & Leveraging Opportunities

The Evaluators reviewed whether the CenterPoint portfolio offered a comprehensive

range of energy efficiency opportunities to all major customer sectors. Table 3-9

summarizes the market sectors and what programs target or allow each sector.

Table 3-9 Assessment of Targeted Customer Sectors by Program

Program

Re

sid

en

tial

Mu

ltif

amily

Mo

bile

Ho

me

Smal

l Co

mm

erc

ial

Larg

e C

om

me

rcia

l

Ind

ust

rial

Agr

icu

ltu

ral

Pu

blic

Se

cto

r

Space Heating CIP

Water Heating CIP


Commercial Food Service CIP C&I Solutions Home Energy Reports Low Flow Showerhead & Aerator CIP HEAL Partnership NA NA NA NA NA NA NA NA Saving Homes Program

Program targets this sector Sector is eligible for this program Sector is ineligible for this program

Each sector has several programs for which they are eligible, and at least one program

that targets them. Segments with fewer targeted outreach avenues include:




Mobile/manufactured housing. This is often not targeted as there is a much

higher prevalence of electric space and water heating. Further, this segment

receives outreach from the Arkansas Weatherization Program.

Agriculture and Industrial sectors are not specifically targeted by the Space

Heating and Water Heating CIPs as the equipment used by these facilities

generally requires custom calculations.

Public Sector facilities are targeted with a wide range of programs. This has

included residential programs that reach out to public housing authorities.

In addition, the Evaluators reviewed the extent of collaboration and leveraging of

available partnership opportunities by CenterPoint.

Examples of cross-utility coordination included:

Through a joint contract, the Evaluators provide EM&V to CenterPoint, Black Hills

Energy, and Arkansas Oklahoma Gas. This allows for sharing of fixed EM&V

costs (such as development of data collection instruments) and more seamless

comparison of program offerings and lessons learned across the natural gas

energy efficiency portfolio. This has reduced the overall cost of EM&V across all

three natural gas utilities.

CenterPoint has brought on a third-party implementer (CLEAResult) for their C&I

Solutions Program. This implementer uses the same program design and

incentive levels for Black Hills Energy and AOG. This has allowed for reduced

program costs for C&I Solutions, which is the largest program in each of the

three gas utility portfolios.

CenterPoint engages in several joint-marketing efforts with the other gas utilities

as well as with Entergy Arkansas, Inc. (EAI). This has included joint-

implementation of education and promotional opportunities when interests with

the other gas utilities or EAI align.

Examples of coordination with non-utility partners included:

CenterPoint’s programs are marketed through industry partners included

professional organizations, trade groups, universities, and homeowners

associations.

CenterPoint works with a local technical college to help provide training

opportunities to trade allies and students interested in careers related to energy

efficiency.

3.2.6 Factor 6: Cost-Effectiveness of Energy Efficiency

To assess this factor, the Evaluators reviewed whether:




Programs met net savings goals;

Whether the NTG ratios were in line with industry norms; and

Whether programs passed cost-effectiveness (TRC) testing.

Table 3-10 Assessment of Cost-Effectiveness

Program NTGR NTGR Within

Industry Norms

Met Net Savings Goal

Program TRC

Space Heating CIP 87.08% Yes No 2.14

Water Heating CIP 82.83% Yes Yes 1.30

Commercial Boiler CIP 80.28% Yes No 2.27

C&I Solutions 102.77% Yes No 4.91

Commercial Food Service CIP 77.20% Yes Yes 1.51

Home Energy Reports 100.00% Yes No .81

Low Flow Showerhead & Aerator CIP 99.53% Yes Yes 13.15

HEAL Partnership N/A NA NA .00

Saving Homes Program 95.00% Yes Yes 6.04

The Home Energy Reports Program had a TRC less than 1.00. The Evaluators attribute

this to the aggregate 35% attrition in Wave 1; Wave 1 of the program was developed in

2011 and is comprised of CenterPoint’s highest users, therefore producing the most

savings per-recipient. Subsequent waves in the program have yielded significantly lower

savings per-participant.

3.2.7 Factor 7: Adequacy of EM&V Procedures

The Evaluators conducted a review of EM&V procedures by program as implemented

by several parties:

QA/QC and EM&V procedures by CenterPoint program staff;

QA/QC and EM&V procedures by third-party implementation staff (where

applicable)

QA/QC and EM&V procedures by the Evaluators.

The EM&V of the CenterPoint programs incorporated industry best practices and was

conducted in an iterative process that incorporated feedback from CenterPoint and

implementation contractors as well as the Independent Evaluation Monitor (IEM). The

Evaluators developed EM&V plans that corresponded to protocols set out in the

Arkansas TRM V6.0. However, over the course of the EM&V process, some activities

deviated from the EM&V plans:

Evaluation activities for the Space Heating CIP (including vendor interviewing)

needed to be curtailed in order to allow for a furnace metering study.




Survey samples for the Commercial Boiler and Commercial Food Service CIPs

were scaled down based on lower participation levels.

Finally, the Evaluators reviewed the quality of program tracking data in order to assess

whether the data allowed for complete evaluation. Further, the Evaluators reviewed the

extent to which individual savings calculations were performed using facility-specific

inputs into the TRM V6.0 algorithms versus the use of simplifying assumptions3. The

results of the review are summarized in Table 3-11.

Table 3-11 Assessment of Data & QA/QC Procedures by Program

Program

Tracking Contains

Necessary Fields

Savings Calculations Performed

and Reported

Savings Calculations

Based on Facility Data

QA/QC Inspections by Program Staff

Space Heating CIP Water Heating CIP Commercial Boiler CIP Commercial Food Service CIP C&I Solutions Home Energy Reports NA

Low Flow Showerhead & Aerator CIP HEAL Partnership NA NA NA NA


Findings of this review included:

The Space Heating and Water Heating CIPs lacked contact names for most

commercial projects. This added a significant amount of difficulty to the EM&V

process.

Home Energy Reports has savings calculations performed at the end of the

program year. This is not tracked mid-year, though that might not be necessary

given the program’s existing verified performance.

C&I Solutions tracking data contained all needed fields for evaluation and

recreation of energy savings calculations.

QA/QC inspections are in place for all programs other than Home Energy

Reports (where it is not needed) and the Low Flow Showerhead & Faucet

Aerator CIP. For the Low Flow Showerhead & Faucet Aerator CIP, post-

inspection of participant residences is not likely to add value, and savings

calculations by CenterPoint already incorporate expected in-service rates.

QA/QC is performed by the Evaluators via telephone survey.

3 Examples of this could include assuming average facility square footage for commercial water heating and using that as an input to the savings calculation, as opposed to collecting facility-specific square footage.



Home Energy Reports 9-1

9. Home Energy Reports

The Home Energy Reports Program is an educational program run by Oracle35, a third

party implementer for CenterPoint. The program provides educational materials to a

sample of CenterPoint’s residential customers, in which their usage is compared against

similar households. The program is designed to encourage behavioral change and

program participation on the part of the recipients of the Home Energy Report.

9.1 Program Overview

The Home Energy Reports Program had $879,668 in budget allocated in 2016. Table

9-1 summarizes the Home Energy Reports historical performance against goals.

Table 9-1 Home Energy Reports Program Historical Performance against Goals

Program

Year

# Participants Budget Net Therms

Actual Goal Spent Allocated Achieved Goal

2011 50,000 50,000 $225,417 $277,364 76,655 108,800

2012 99,846 50,000 $524,839 $524,839 529,715 506,000

2013 99,846 100,000 $860,810 $379,688 1,112,462 887,160

2014 113,856 100,000 $871,505 $879,688 1,102,598 1,000,000

2015 103,995 100,000 $848,588 $879,668 917,376 1,000,000

2016 96,967 100,000 $861,766 $879,668 823,045 1,000,000

9.2 Participation Summary

The Home Energy Reports Program began in September 2011. The program is

designed to generate quantifiable behavioral savings that cannot be feasibly attained

through standard DSM efforts. The comparison against their neighbors is intended to

have a jarring effect; when informed that their usage is above average, the program

theory would assert that they are then driven to engage in conservation behaviors. In

2011, a sample of 50,000 recipients and 50,000 non-recipients was developed. In

November 2012, this was expanded, with a second wave of recipients brought in. In

November 2012, the program added:

52,498 recipients; and

26,248 non-recipients.

In November 2014, CenterPoint added a third wave to the program. This wave includes:

24,732 recipients; and

10,493 non-recipients.

35

Formerly known as Opower, who were acquired by Oracle.




Over time, the population of recipients faces attrition. This occurs mostly due to

members of the recipient group moving to a new residence. Table 9-2 summarizes the

attrition that has occurred in each wave36.

Table 9-2 Home Energy Reports Recipient Attrition

Wave 1 Wave 2 Wave 3

Inception 50,000 52,498 24,732

Current 34,278 38,697 21,088

Attrition % 31.4% 26.3% 14.7%

9.2.1 Savings Calculation Methodologies

The Evaluators utilized a post-only model with pre-usage controls. Other model

specifications were tested (including fixed effects), but the post-only model was found to

provide the highest precision level in results. The model specification applied uses one

year of pre-treatment data to construct control variables which capture the primary

drivers of a household’s energy use.

The model specification is as follows:

𝑈𝑠𝑎𝑔𝑒𝑖𝑡 = 𝛼0 + 𝛽 ∗ 𝑡𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡𝑖

+𝛼1 ∗ 𝑃𝑟𝑒𝑈𝑠𝑎𝑔𝑒𝑖

+𝛼2 ∗ 𝑃𝑟𝑒𝑆𝑢𝑚𝑚𝑒𝑟𝑖

+𝛼3 ∗ 𝑃𝑟𝑒𝑊𝑖𝑛𝑡𝑒𝑟𝑖

+𝛾 ∗ 𝑚𝑚𝑡

+𝛿1 ∗ 𝑚𝑚𝑡 ∗ 𝑃𝑟𝑒𝑈𝑠𝑎𝑔𝑒𝑖

+𝛿2 ∗ 𝑚𝑚𝑡 ∗ 𝑃𝑟𝑒𝑆𝑢𝑚𝑚𝑒𝑟𝑖

+𝛿3 ∗ 𝑚𝑚𝑡 ∗ 𝑃𝑟𝑒𝑊𝑖𝑛𝑡𝑒𝑟𝑖

+휀𝑖𝑡

Where

i denotes the ith customer

t denotes the first, second, third, etc. month of the post-treatment period

Usageit is the average daily use for read t for household i during the post-

treatment period

36

The participant total in Table 9-1 is based on average monthly participants. The values in Table 9-2 are based on the end-of-year values for 2016.




PreUsagei is the average daily usage across households i’s available pre-

treatment billing reads.

PreWinteri is the average daily usage over the months of December January,

February, and March over household i’s available pre-treatment meter reads.

PreSummeri is the average daily usage over the months of June, July, August,

and September over household i’s available pre-treatment meter reads.

mmt is a vector of month-year dummies

And parameter definitions are:

𝛼0 is an intercept term

𝛼1, 𝛼2 , 𝛼3 are effects of control variables PreUsagei , PreWinteri , PreSummeri on

Usageit in the reference month.

𝛿1, 𝛿2, 𝛿3 are the effect of the control variables in each month-year (mmt) of the

post period.

휀𝑖𝑡 is an error term.

In this specification, savings are calculated by:

Savings = Treatment_Coeff * #Recipients * 365

Where,

Treatment_Coeff = Coefficient for treatment parameter (daily use is the

dependent variable, a negative value for treatment reflects the difference in

Therms/day used by the recipient group after report delivery)

#Recipients = Total recipients in the Wave, after accounting for attrition

365 = days/year

Home Energy Report Peak Savings

To estimate peak Therms, the Evaluators split savings into two categories:

Weather Sensitive; and

Non-Weather Sensitive

From this Evaluators used the ratio of peak to annual Therms for residential furnaces

and water heaters for weather-sensitive and non-weather sensitive (respectively) from

the TRM V6.0. These multipliers are defined in Table 9-3.




Table 9-3 Home Energy Reports Peak-to-Annual Multipliers

Savings Type Zone 6 Zone 7 Zone 8 Zone 9

Weather- Sensitive37

.015 .016 .015 .019

Non-Weather-Sensitive38

.0024 .0024 .0024 .0024

Home Energy Report Net Savings

The HER program uses a randomized control trial, comparing recipients to non-

recipients. As a result, the savings estimates from the model are net savings estimates,

and no further deduction of free-ridership is taken.

Model Output Results

9.2.1.1 Wave 1

Table 9-4 provides the model coefficients for the regression of customer billing data in

the analysis of Wave 1.

Table 9-4 Regression Coefficients & Model Details – Wave 1

Variable Description Regression Coefficient

Standard Error T-Stat PR > |T|

INTERCEPT 0.60633 0.01504 40.32 <0.0001

TREATMENT -0.03694 0.00250 -14.79 <0.0001

AVG_PREUSAGE 0.57440 0.01782 32.24 <0.0001

AVG_PREUSAGE_WINTER -0.17444 0.01487 -11.73 <0.0001

AVG_PREUSAGE_SUMMER 0.55062 0.00568 96.94 <0.0001

FEB -0.12478 0.02118 -5.89 <0.0001

MAR -0.29312 0.02120 -13.83 <0.0001

APR -0.49048 0.02123 -23.10 <0.0001

MAY -0.56783 0.02126 -26.71 <0.0001

JUN -0.56355 0.02129 -26.47 <0.0001

JUL -0.54844 0.02132 -25.73 <0.0001

AUG -0.56591 0.02135 -26.51 <0.0001

SEP -0.61131 0.02138 -28.59 <0.0001

OCT -0.62288 0.02141 -29.09 <0.0001

NOV -0.61133 0.02143 -28.52 <0.0001

DEC -0.29334 0.02145 -13.67 <0.0001

AVG_PREUSAGE:FEB 0.26921 0.02522 10.68 <0.0001

AVG_PREUSAGE:MAR 0.55483 0.02525 21.98 <0.0001

AVG_PREUSAGE:APR 0.32559 0.02528 12.88 <0.0001

AVG_PREUSAGE:MAY -0.19107 0.02533 -7.54 <0.0001

AVG_PREUSAGE:JUN -0.46567 0.02539 -18.34 <0.0001

AVG_PREUSAGE:JUL -0.54815 0.02544 -21.55 <0.0001

37

Multipliers from Arkansas TRM V6.0 Volume 2, Page 59. Table 43: Gas Furnace Peak Heating Ratio

38 Multipliers from Arkansas TRM V6.0 Volume 2, Page 138. Equation 85: Peak Day Therm Savings – Gas Storage Tank Water Heater Replacement




AVG_PREUSAGE:AUG -0.53736 0.02547 -21.10 <0.0001

AVG_PREUSAGE:SEP -0.46290 0.02550 -18.15 <0.0001

AVG_PREUSAGE:OCT -0.06951 0.02555 -2.72 0.007

AVG_PREUSAGE:NOV 0.33937 0.02558 13.27 <0.0001

AVG_PREUSAGE:DEC 0.14919 0.02559 5.83 <0.0001

AVG_PREUSAGE_SUMMER:FEB -0.12209 0.02105 -5.80 <0.0001

AVG_PREUSAGE_SUMMER:MAR -0.18410 0.02107 -8.74 <0.0001

AVG_PREUSAGE_SUMMER:APR 0.19540 0.02111 9.26 <0.0001

AVG_PREUSAGE_SUMMER:MAY 0.71922 0.02114 34.02 <0.0001

AVG_PREUSAGE_SUMMER:JUN 0.84916 0.02119 40.08 <0.0001

AVG_PREUSAGE_SUMMER:JUL 0.81297 0.02121 38.34 <0.0001

AVG_PREUSAGE_SUMMER:AUG 0.83312 0.02123 39.24 <0.0001

AVG_PREUSAGE_SUMMER:SEP 0.84702 0.02126 39.85 <0.0001

AVG_PREUSAGE_SUMMER:OCT 0.55096 0.02129 25.88 <0.0001

AVG_PREUSAGE_SUMMER:NOV 0.04559 0.02131 2.14 0.032

AVG_PREUSAGE_SUMMER:DEC -0.08250 0.02133 -3.87 0.000

AVG_PREUSAGE_WINTER:FEB -0.23834 0.00804 -29.64 <0.0001

AVG_PREUSAGE_WINTER:MAR -0.61537 0.00805 -76.44 <0.0001

AVG_PREUSAGE_WINTER:APR -0.70449 0.00806 -87.41 <0.0001

AVG_PREUSAGE_WINTER:MAY -0.63296 0.00808 -78.37 <0.0001

AVG_PREUSAGE_WINTER:JUN -0.56881 0.00810 -70.26 <0.0001

AVG_PREUSAGE_WINTER:JUL -0.54729 0.00811 -67.48 <0.0001

AVG_PREUSAGE_WINTER:AUG -0.55065 0.00812 -67.81 <0.0001

AVG_PREUSAGE_WINTER:SEP -0.56853 0.00813 -69.92 <0.0001

AVG_PREUSAGE_WINTER:OCT -0.65451 0.00815 -80.34 <0.0001

AVG_PREUSAGE_WINTER:NOV -0.50462 0.00815 -61.89 <0.0001

AVG_PREUSAGE_WINTER:DEC -0.11154 0.00816 -13.67 <0.0001

Adjusted R-Square: 0.8309

The resulting annual savings are:

Annual Savings = .03694 * 35,684 * 365 = 481,127 Therms

95% Confidence Interval: +/- 63,023 (13.3%)

9.2.1.2 Wave 2






INTERCEPT 0.73669 0.00760 96.93 <0.0001

TREATMENT -0.01839 0.00130 -14.10 <0.0001

AVG_PREUSAGE 0.36161 0.02332 15.51 <0.0001



FEB -0.12443 0.01069 -11.64 <0.0001




MAR -0.47776 0.01070 -44.64 <0.0001

APR -0.61824 0.01072 -57.69 <0.0001

MAY -0.64777 0.01073 -60.36 <0.0001

JUN -0.64818 0.01075 -60.27 <0.0001

JUL -0.64389 0.01078 -59.73 <0.0001

AUG -0.64223 0.01080 -59.44 <0.0001

SEP -0.66942 0.01082 -61.87 <0.0001

OCT -0.68950 0.01083 -63.65 <0.0001

NOV -0.53631 0.01085 -49.45 <0.0001

DEC -0.05518 0.01086 -5.08 <0.0001

AVG_PREUSAGE:FEB 0.57253 0.03301 17.34 <0.0001

AVG_PREUSAGE:MAR 1.12200 0.03306 33.94 <0.0001

AVG_PREUSAGE:APR 0.85135 0.03311 25.71 <0.0001

AVG_PREUSAGE:MAY 0.34814 0.03317 10.49 <0.0001

AVG_PREUSAGE:JUN -0.07089 0.03323 -2.13 0.033

AVG_PREUSAGE:JUL -0.17933 0.03328 -5.39 <0.0001

AVG_PREUSAGE:AUG -0.15763 0.03333 -4.73 <0.0001

AVG_PREUSAGE:SEP -0.02779 0.03337 -0.83 0.405

AVG_PREUSAGE:OCT 0.34280 0.03341 10.26 <0.0001

AVG_PREUSAGE:NOV 0.92344 0.03345 27.61 <0.0001

AVG_PREUSAGE:DEC 0.36289 0.03347 10.84 <0.0001

AVG_PREUSAGE_SUMMER:FEB -0.22501 0.02539 -8.86 <0.0001

AVG_PREUSAGE_SUMMER:MAR -0.30932 0.02543 -12.16 <0.0001

AVG_PREUSAGE_SUMMER:APR 0.08125 0.02547 3.19 0.001







AVG_PREUSAGE_SUMMER:NOV -0.31945 0.02572 -12.42 <0.0001

AVG_PREUSAGE_SUMMER:DEC -0.28591 0.02574 -11.11 <0.0001











AVG_PREUSAGE_WINTER:DEC -0.23752 0.01088 -21.83 <0.0001








9.2.1.3 Wave 3






INTERCEPT 0.47559 0.01323 35.94 <0.0001

TREATMENT -0.01002 0.00217 -4.62 <0.0001

AVG_PREUSAGE 1.60728 0.01968 81.68 <0.0001



FEB -0.07819 0.01860 -4.20 <0.0001

MAR -0.27961 0.01864 -15.00 <0.0001

APR -0.42205 0.01867 -22.60 <0.0001

MAY -0.45706 0.01870 -24.45 <0.0001

JUN -0.41590 0.01872 -22.21 <0.0001

JUL -0.39849 0.01882 -21.17 <0.0001

AUG -0.42805 0.01889 -22.66 <0.0001

SEP -0.47964 0.01893 -25.34 <0.0001

OCT -0.50084 0.01897 -26.40 <0.0001

NOV -0.31303 0.01902 -16.45 <0.0001

DEC -0.04050 0.01900 -2.13 0.033

AVG_PREUSAGE:FEB -0.35067 0.02786 -12.59 <0.0001

AVG_PREUSAGE:MAR -0.39831 0.02789 -14.28 <0.0001

AVG_PREUSAGE:APR -0.74682 0.02791 -26.75 <0.0001

AVG_PREUSAGE:MAY -1.26865 0.02797 -45.36 <0.0001

AVG_PREUSAGE:JUN -1.52024 0.02802 -54.26 <0.0001

AVG_PREUSAGE:JUL -1.55046 0.02807 -55.23 <0.0001

AVG_PREUSAGE:AUG -1.53553 0.02815 -54.54 <0.0001

AVG_PREUSAGE:SEP -1.47302 0.02820 -52.23 <0.0001

AVG_PREUSAGE:OCT -1.08751 0.02827 -38.47 <0.0001

AVG_PREUSAGE:NOV -0.86201 0.02830 -30.46 <0.0001

AVG_PREUSAGE:DEC -0.42428 0.02835 -14.97 <0.0001

AVG_PREUSAGE_SUMMER:FEB 0.33941 0.02287 14.84 <0.0001

AVG_PREUSAGE_SUMMER:MAR 0.52873 0.02289 23.10 <0.0001

AVG_PREUSAGE_SUMMER:APR 0.95221 0.02292 41.55 <0.0001







AVG_PREUSAGE_SUMMER:NOV 0.79473 0.02327 34.15 <0.0001

AVG_PREUSAGE_SUMMER:DEC 0.29402 0.02330 12.62 <0.0001














AVG_PREUSAGE_WINTER:DEC 0.02453 0.00878 2.79 0.005





Group Comparison

The difference in consumption between the two groups is observable when presented

graphically. Figure 9-1 presents the monthly differences in consumption between the

two groups. Reports were first delivered in October of 2011, and at that point the

magnitude of difference in consumption increases. Further, the difference in use

between the recipient and control group are shown to increase significantly in 2014

compared to 2012 and 2013.

Similar representations for Wave 2 and Wave 3 are presented in Figure 9-2 and Figure

9-3, respectively. The impact of the reports on Wave 2 and Wave 3 is significantly lower

than Wave 1.




Figure 9-1 Difference in Daily Consumption between Recipient & Control Group –

Wave 1

Figure 9-2 Difference in Daily Consumption between Recipient & Control Group –

Wave 2




Figure 9-3 Difference in Daily Consumption between Recipient & Control Group – Wave 3

9.2.2 Per-Customer Performance

The change in annual savings per-recipient is summarized in Figure 9-4. Wave 1 shows

a downward trend in per participant savings starting in 2014, while Wave 2 and 3 show

an upward trend, although the differences from year to year are not significant at the

95% level.

Figure 9-4 Savings per Recipient by Year




9.2.3 Double Counting Analysis

Protocol J in TRM V6.0 specifies double counting as the difference in per-participant

other-program savings

When comparing all of the other-program-participation, the Evaluators found:

1.012 Therms per participant for the recipient group; and

1.015 Therms per participant for the control group.

The difference was less than .01%, and as a result the Evaluators concluded no double

county penalty was applicable.

9.2.4 Verified Savings

With the model output results and double count analysis, the Home Energy Reports

Program has:

828,576 annual Therms savings; and

12,346 peak Therms.

When aggregating the savings and confidence intervals, the Evaluators found that the

overall 95% confidence interval was ±16.28% of program savings.

9.3 Conclusions & Recommendations

9.3.1 Conclusions

1. The Home Energy Reports Program is providing statistically significant

behavioral change. The evaluators identified quantifiable savings at the 95%

confidence level for the recipients of Home Energy Reports. When asked to

name specific energy conservation behaviors, report recipients displayed

statistically significant differences incidence of identifying thermostat.

2. The program has not generated statistically significant increase in rebate

program participation. The evaluators found the savings to be behavioral-

based. The double counting analysis for this program found little to no increase

in other-program-participation when compared against the control group.

9.3.2 Recommendations

Based on survey data findings, the Evaluators recommend the following:

1. Consider opt-in mechanisms for new higher-use customers. Wave 1

provides 58% of program savings despite being just 36.4% of the total recipients.

Wave 1 is constituted of the highest-using customers in CenterPoint’s service

area. With this wave having reached 31.4% attrition, program staff should

research alternative behavioral mechanisms to target high-use customers that

are not currently involved with the program.


centerpoint energy arkansas energy efficiency program ... · overall energy savings of its...

Documents