Experience you can trust.

NJCEP Energy Impact Evaluation

RFP No 07-X-38683Miriam Goldberg and Ryan Barry, KEMA.

David Carroll and Jackie Berger, APPRISE.New Jersey Board of Public Utilities

Newark, New Jersey

October 6, 2009

2

Overview of Presentation

Introduction Evaluation Objectives Programs Reviewed Allocation of Evaluation Resources Analysis Methods & Data Sources Evaluation Deliverables High Level Findings & Recommendations Questions

3

KEMA at a glance

Established in 1927, Arnhem, the Netherlands 1,900 professionals in 20 counties, 600 in North America

– Broad, deep bench of industry SMEs Annual revenues of €227 million

Independent experts to the global energy industry Independent experts to the global energy industry

4

KEMA’s global consulting practice Our mission:

– Addressing the complex multi-dimensional nature of energy industry challenges.

Bridging the gap between strategists and implementers; between engineers and accountants.

Business Process

Technology Systems

ProgramManagement

Advancing processes & technology to do more for the industryAdvancing processes & technology to do more for the industry

Sustainable Market Strategies– Energy Efficiency,

Renewable Energy, Demand Response

– Research, Planning & Evaluation

– ~110 US professional staff

5

Evaluation Teammates

Princeton based APPRISE is a nonprofit research institute dedicated to collecting and analyzing data and information to assess and improve public programs.

Princeton based Braun Research Marketing and Public Opinion Research firm

NJCEP Evaluation Roles

– Residential New Construction Program Evaluation

– Data transfer from utilities for all program areas

NJCEP Evaluation Roles

– Administered four participant questionnaires

Residential HVAC

ENERGY STAR Products

SmartStart Buildings

CORE

6

Project TeamOffice of Clean Energy

Miriam GoldbergPrincipal-in-Charge

Ryan BarryProject Manager

ENERGY STAR Products - CFL

Residential HVAC

Kathleen GaffneyJeremiah Robinson, PE

SmartStart Buildings

Combined Heat & Power

Rick FioravantiJessica Harrison

Tammy Kuiken, PEJeremiah Robinson, PE

Ken AgnewBrian Dunn, PE

Residential New Construction

(APPRISE)

David CarrollJackie Berger

CORE - PV

Brian Dunn, PEDoug Kneale

7

Programs Reviewed

1. WARM Advantage and COOL Advantage2. Residential New Construction3. Residential Lighting – CFL4. SmartStart Buildings5. Combined Heat and Power6. Customer On-site Renewable Energy – PV

8

Evaluation Objectives

Prospective Assessment– To revise the savings calculation Protocols so that

going forward the calculations using these Protocols provide more accurate statements of savings accomplishments. Protocols to Measure Resource Savings 2007

9

Evaluation Objectives – cont.

Retrospective Assessment– To provide a retrospective assessment of program

accomplishment, as part of a due diligence review of past utility program effectiveness on behalf of ratepayers.

10

Evaluation Timeline

2001 2002 2003 2004 2005 2006 2007 2008 2009

Period Under Evaluation

Evaluation Kick-Off Meeting

NJCEP Work Plan Approval

CORE Work Plan Approval

Final Presentation

Transition to Market Managers

OCE assumes role of NJCEP Administrator

RFPs to hire Market Managers

11

Statewide ProgramNatural Gas UtilitiesElectric Utilities

12

Evaluation Methods Selection for Each Program Estimated program savings magnitude relative to

the total for the portfolio included in the evaluation Program expenditures relative to the total for the

portfolio included in the evaluation Nature of uncertainties in the protocol savings

estimates and their potential effect on net savings Quality and applicability of existing studies Total evaluation budget

13

Allocation of Evaluation Resources

Electric Savings

Gas Savings Expenditures

Residential HVAC 6% 39% 18% 12%Residential New Construction 2% 36% 19% 12%ENERGY STAR Products/Lighting 17% 0% 5% 10%SmartStart 71% 25% 30% 30%Combined Heat & Power 1% 0% 0% 5%Customer On-Site Renewable Energy 4% 0% 26% 30%

Program Area

Fraction of Evaluation Portfolio Fraction of Evaluation

Budget

Evaluation Budget: $857,680

~0.1% of program budget

14

Program & Utility Provided Data

OCE

6 Program Areas

6 Program Years

(2001-2006)

• Program Tracking Databases (electronic & hard-copy)

• Customer Usage Data

Electric & GasUtilities

Market Managers

Contractors

15

Analysis Methods & Data Sources

Data requirements for the selected methods were significant.

Analaysis Methods Data/Input SourceResidential

HVAC

Residential New

Construction

ENERGY STAR

Products/Lighting

SmartStart Buildings

Combined Heat & Power

Customer On-Site

Renewable Energy

Engineering Analysis On-site data P P PEngineering Review of Energy Savings Calculations

Program records and secondary research

P P P PAnalysis of Prior Work Past NJ studies and data P P P P P PDatabase Review Tracking system databases P P P P P

KEMA's Proprietary ModelsTracking system databases, on-site data, survey research and consumption data

P

Billing/Consumption AnalysisConsumption data, weather data and survey research

P P

Survey AnalysisParticipant & nonparticipant surveys

P P P P P PInterview analysis Market actor interviews P PRatio Estimation

Tracking databases and participant surveys

P P

16

Evaluation Deliverables

7 Individual Program Reports– 2 SmartStart reports

Summary Report Summary of Recommendations Report Non-confidential Data Today’s Presentation

17

Summary of High LevelFindings & Recommendations NJCEP Overall and Each Program Area

– Prospective Results– Retrospective Results– Recommendations

18

NJCEP Overall

Continue with periodic independent reviews, such as this evaluation

Improve and expand the Protocol documentation Use consistent energy savings terminology across

measures OCE should consider conducting an impact

evaluation covering first three years of program under market manager model.

In general, the Protocols provide the Program with consistent and reasonable methods for estimating gross energy savings.

19

NJCEP Overall – cont.

Revise the current assumption that free ridership and spillover cancel each other out.

The program should develop a standard policy for the assignment of baseline efficiency levels for the purpose of calculating energy savings.

Improved program tracking is critical. KEMA understands the program has developed a statewide database following the period covered by this evaluation (2001-2006). The statewide tracking database should be reviewed by an independent party as soon as possible.

20

Residential HVAC

Source for Hour (EFLH) Estimate

Post-Program Cooling Usage (kWh)

Effective Full Load Hours (EFLH)

EFLH Confidence Interval (+/-, 90%)

Impact of Efficiency Improvement (kWh)

Protocols 1,500 600 409

17 341Impact Evaluation 1,252 501

Source for Hour (EFLH) Estimate

Post-Program Usage

(Therms)Equivalent Full Load

Hours (EFLH)

EFLH Confidence Interval

(+/-, 90%, Hours) Baseline CapacityImpact Relative to Standard (Therms)

Protocols 860 965 235Impact Evaluation 648 727 13 177

91,000

Gross saving estimates for both programs are lower due to decrease

Equivalent Full Load Hours (EFLH)

CoolAdvantage EFLH and savings drop 17%.

WarmAdvantage EFLH and savings drop 25%.

21

Residential HVAC – cont.

Lower EFLH estimate, if adopted in Protocols, will result in lower saving estimates.

CoolAdvantage Protocol savings for optimal installation and proper sizing are not supported by secondary sources.

Installation (QIV) and sizing savings are an important focus of future studies.

Peak period efficiency-related savings require additional study.

Recommendations made to update Protocol equations and input values.

22

Residential New Construction

ENERGY STAR Homes are meeting the electric and gas usage projections established by the REM/RateTM model.

It appears that nonparticipants had caught up with ENERGY STAR minimum by 2006.

It appears that higher rated homes are performing better than projected.

23

Residential New Construction – cont.

The program underwent a substantial upgrade in 2006/07.

The 2005/2006 program was using an older REM/Rate model.

Older REM/Rate models do not effectively model the impacts of energy efficient products and lighting on baseload.

Revise the Protocols with the understanding that nonparticipating homes perform at least at the 2006 ENERGY STAR minimum standards.

24

Residential Lighting - CFL

Fix conversion error in the kW savings algorithm (W to kW)

Estimated Free ridership 15%. Consistent with similar programs during the 2003-2005 timeframe.

Updated Protocol Algorithm Values

Variable Type Value Source

∆W Fixed 48.5 2009 New Jersey CFL Study

CFLhours Fixed 2.8 2009 New England Study

ISRCFL Fixed 83.4% 2009 New Jersey CFL Study

CF Fixed 9.9% 2007 New England Metering Study

25

Residential Lighting (CFL) – cont.

Peak Summer Usage CF Values from New England Study

Adjusted for New England peak period (1-5 PM) to the New Jersey peak period (1-8 PM) results in our recommended CF value of 9.9%.

26

SmartStart Buildings Program

Protocol Recommendations – Standard terminology and definitions– Team of 9 engineers reviewed:

Performance Lighting, Prescriptive Lighting, Lighting Controls, Motors, Electric HVAC Systems, Electric Chillers, Variable Frequency Drives, Air Compressors with

Variable Frequency Drives,

Gas Chillers (Absorption

Chillers), Gas Fired Desiccants, Gas Booster Water Heaters, Gas Water Heaters, Furnaces and Boilers, Compressed Air System

Optimization, and Custom Projects

27

SmartStart Buildings Program – cont.

Gross Savings Adjustment Factors

– Gross kW lower than kWh due to

error in Program’s calculation tool

– Gross therms lower than kWh

and kW due to decrease in one

large project.

KEMA recommends further NTG

research covering first 3 years of

implementation under the market

manager model.

Gross and Net Savings Adjustment Factors

0%

20%

40%

60%

80%

100%

120%

kWh kW Therms

Energy Unit

Per

cen

tag

e

Gross Net

28

Combined Heat & Power 2 of the installations differed from those specified

in the applications.

Case 1:Nursing Home

Facility

Case 2:Large

Industrial Company

Case 3:Commercial

Food Processor

Case 4:Recreational/Athle

tic Center

Equipment Type

MicroturbineBackpressure Steam Turbine

MicroturbineReciprocating Gas

Engine

Equipment # 2 1 5 2

Per Unit Capacity

70kW 509kW 60kW 75kW

Equipment Cost

$357,000 $654,701 $750,000 $515,500

Incentive Amount

$107,000 $196,410 $225,000 $150,000

Displaced Thermal Loads

- Heating- Service HW

NA- Chiller

- Service HW- Process HW

- Service HW- Pool

OperationRoughly full-time Roughly full-time Roughly full-time

Engine 1: 100% all yr;Engine 2: 100% for ½

yr

Notes: “NA” = Not applicable, “Roughly full-time” means operators were not baseloading operation in the past, and “HW” = hot water applications

Application: 60 kW

Application:

Gas Engine

1

260 kW

29

Combined Heat & Power – cont.

Protocol savings estimates should continue to be performed on an individual basis. – Information collected during the post-installation visit

should be used rather than the application data. KEMA supports two post-evaluation period program

updates:– Post-installation inspections with all CHP installations;

and– Participants are required to provide the program with key

information about the system design and operation after installation.

30

Customer On-site Renewable Energy

Gross and Net Adjustment Factors

Customer Segment

Gross Net

kWh kW kWh kW

All 96% 100% 71% 70%

Res REIP Eligible 89% 100% 94% 89%

Res REIP Ineligible 89% 100% 90% 84%

Nonres REIP Eligible 84% 93% 96% 91%

Nonres REIP Ineligible 106% 101% 46% 47%

31

CORE – cont. Recommend continued use of PVWatts to calculate

energy production and discontinue its deemed value method for purposes of reporting energy production to the BPU.

Recommend two additions to the PVWatts calculation methodology:– separate PVWatts calculations for arrays at same site with

different tilt angles, orientations, or shading levels; and– incorporate a shading factor.

Several recommendations were made with regards to the calculation of peak demand (kW).

Experience you can trust.

Thank you for your attention.

33

Residential HVAC – appendixSummary of Protocol Recommendations

Component Value or Equation Component Value or Equation

EFLHC(Cooling Equivalent Full Load Hours)

600 Hours EFLHC 501 Hours

PSF (Proper Sizing Factor)5% of post program usage or

demand

QIF (Quality Installation Factor)15% of post program usage or

demand

Cooling Energy Savings for Proper Sizing (Eq. 5)

Cooling Energy Savings for Quality Installation Verification (Eq. 7)

Cooling Energy Savings for Proper Sizing (Eq. 6)

Cooling Energy Savings for Quality Installation Verification (Eq. 8)

EFLHH (Heating Equivalent Full Load Hours)

2250 Hours (Heat pump),965 Hours (Furnace or boiler) EFLHH 727 Hours

CAPYT (Output capacity of typical heating unit, in Btu/hr) 91,000 Btu/hr CAPYT CAPYQ

Heating Gas Savings (Eq. 22)Heating Gas Savings with

baseline typical capacity equal to new capacity.

BWHU (Annual Baseline Water Heater Usage, in therms) 212 therms BWHU 180 therms

ESF (Energy savings factor) DSF (Demand savings factor)

Cooling Energy Savings for Proper Sizing and QIV.

Cooling Demand Savings for Proper Sizing and QIV.

2007 Protocol Recommended

9.2% of post program usage or demand

PSFkWhkWh QP *

QIFPSFkWhkWh QV *1* ESFkWhkWh QP *

PSFkWkW QP *

QIFPSFkWkW QV *1* DSFkWkW QP *

H

Q

Q

B

T

S

EFLHBtu

therm

AFUE

CAPY

AFUE

CAPY

TH

*000,100

1

*

H

Q

Q

B

Q

S

EFLHBtu

therm

AFUE

CAPY

AFUE

CAPY

TH

*000,100

1

*