final annual report to the pennsylvania public utility ...final annual report to the pennsylvania...

©2019 Guidehouse Inc.

Final Annual Report to the Pennsylvania Public Utility Commission Phase III of Act 129

Program Year 10

(June 1, 2018 - May 31, 2019)

For Pennsylvania Act 129 of 2008

Energy Efficiency and Conservation Plan

Prepared for:

Prepared by: Navigant, A Guidehouse Company 1375 Walnut Street Suite 100 Boulder, Colorado 80302

303.728.2500 guidehouse.com

November 15, 2019

Final Annual Report to the Pennsylvania Public Utility Commission: Phase III of Act 129

©2019 Guidehouse Inc. Page i

TABLE OF CONTENTS

1. Introduction ............................................................................................................... 1

2. Summary of Achievements ...................................................................................... 2

2.1 The Carryover Savings from Phase II of Act 129 .......................................................................... 2 2.2 Phase III Energy Efficiency Achievements to Date ....................................................................... 3 2.3 Phase III DR Achievements to Date .............................................................................................. 6 2.4 Phase III Performance by Customer Segment.............................................................................. 7 2.5 Summary of Participation by Program .......................................................................................... 9 2.6 Summary of Impact Evaluation Results ...................................................................................... 10 2.7 Summary of Energy Impacts by Program ................................................................................... 12

2.7.1 Incremental Annual Energy Savings by Program .......................................................... 13 2.7.2 Lifetime Energy Savings by Program ............................................................................. 16

2.8 Summary of Demand Impacts by Program ................................................................................. 17 2.8.1 Energy Efficiency ............................................................................................................ 17 2.8.2 Demand Response ......................................................................................................... 20

2.9 Summary of Fuel Switching Impacts ........................................................................................... 21 2.10 Summary of Cost-Effectiveness Results ................................................................................... 22 2.11 Comparison of Performance to Approved EE&C Plan .............................................................. 26 2.12 Findings and Recommendations ............................................................................................... 29

3. Evaluation Results by Program ............................................................................. 31

3.1 Residential EE Program .............................................................................................................. 33 3.1.1 Participation and Reported Savings by Customer Segment .......................................... 33 3.1.2 Gross Impact Evaluation ................................................................................................ 34 3.1.3 Net Impact Evaluation .................................................................................................... 39 3.1.4 Process Evaluation ......................................................................................................... 43 3.1.5 Cost-Effectiveness Reporting ......................................................................................... 46 3.1.6 Status of Recommendations .......................................................................................... 49

3.2 Residential Low-Income EE Program ......................................................................................... 54 3.2.1 Participation and Reported Savings by Customer Segment .......................................... 54 3.2.2 Gross Impact Evaluation ................................................................................................ 54 3.2.3 Net Impact Evaluation .................................................................................................... 58 3.2.4 Process Evaluation ......................................................................................................... 58 3.2.5 Cost-Effectiveness Reporting ......................................................................................... 58 3.2.6 Status of Recommendations .......................................................................................... 60

3.3 Small C&I EE Program ................................................................................................................ 61 3.3.1 Participation and Reported Savings by Customer Segment .......................................... 61 3.3.2 Gross Impact Evaluation ................................................................................................ 62 3.3.3 Net Impact Evaluation .................................................................................................... 66 3.3.4 Process Evaluation ......................................................................................................... 69 3.3.5 Cost-Effectiveness Reporting ......................................................................................... 73 3.3.6 Status of Recommendations .......................................................................................... 76

3.4 Large C&I EE Program ................................................................................................................ 81 3.4.1 Participation and Reported Savings by Customer Segment .......................................... 81 3.4.2 Gross Impact Evaluation ................................................................................................ 81 3.4.3 Net Impact Evaluation .................................................................................................... 86 3.4.4 Process Evaluation ......................................................................................................... 88 3.4.5 Cost-Effectiveness Reporting ......................................................................................... 89


©2019 Guidehouse Inc. Page ii

3.4.6 Status of Recommendations .......................................................................................... 91 3.5 Combined Heat and Power Program .......................................................................................... 96

3.5.1 Participation and Reported Savings by Customer Segment .......................................... 96 3.5.2 Gross Impact Evaluation ................................................................................................ 96 3.5.3 Net Impact Evaluation .................................................................................................... 98 3.5.4 Process Evaluation ......................................................................................................... 98 3.5.5 Cost-Effectiveness Reporting ......................................................................................... 99 3.5.6 Status of Recommendations ........................................................................................ 101

3.6 Demand Response Programs ................................................................................................... 103 3.6.1 Participation and Reported Savings by Customer Segment ........................................ 103 3.6.2 Gross Impact Evaluation .............................................................................................. 104 3.6.3 Process Evaluation ....................................................................................................... 105 3.6.4 Cost-Effectiveness Reporting ....................................................................................... 105 3.6.5 Status of Recommendations ........................................................................................ 112

4. Summary of Finances ........................................................................................... 114

4.1 Program Financials ................................................................................................................... 114 4.2 Cost Recovery ........................................................................................................................... 115

Appendix A. Upstream Lighting Cross-Sector Sales ............................................. A-1

Appendix B. Site Inspection Summary .................................................................... B-1

Appendix C. HER Impact Evaluation Detail ............................................................ C-1

Appendix D. Participation Counts ........................................................................... D-1

FIGURES Figure 2-1. Carryover Savings from Phase II of Act 129 ............................................................................. 2 Figure 2-2. Customer Segment-Specific Carryover from Phase II .............................................................. 3 Figure 2-3. EE&C Plan Performance toward Phase III Portfolio Compliance Target .................................. 4 Figure 2-4. EE&C Plan Performance toward Phase III Low-Income Compliance Target ........................... 5 Figure 2-5. EE&C Plan Performance against Phase III G/E/NP Compliance Target .................................. 5 Figure 2-6. Event Performance Compared to 85% Per-Event Target ......................................................... 7 Figure 2-7. PYTD Energy Savings by Program ......................................................................................... 13 Figure 2-8. P3TD Energy Savings by Program.......................................................................................... 14 Figure 2-9. PYTD Demand Savings by EE Program ................................................................................. 18 Figure 2-10. P3TD Demand Savings by EE Program ............................................................................... 18 Figure 3-1. Overall Tenant Satisfaction with Multifamily Targeted Market Segment, n=61 ....................... 45 Figure 3-2. Sources of Tenant Multifamily Targeted Market Segment Awareness, n=56 ......................... 46 Figure 3-3. Overall Landlord Satisfaction with Multifamily Targeted Market Segment, n=8 ...................... 71 Figure 3-4. Sources of Landlord Multifamily Targeted Market Segment Awareness, n=8 ........................ 72 Figure 3-5. Influence of Sources Multifamily Targeted Market Segment Awareness, n=6 ........................ 72 Figure 3-6. Sources of Additional Multifamily Targeted Market Segment Information, n=8 ...................... 73 Figure 3-7. Event Performance Compared to 85% Per-Event Target ..................................................... 105


©2019 Guidehouse Inc. Page iii

TABLES Table 2-1. Phase III to Date DR Performance by Event .............................................................................. 6 Table 2-2. PY10 Summary Statistics by Customer Segment ...................................................................... 8 Table 2-3. Phase III Summary Statistics by Customer Segment ................................................................. 8 Table 2-4. PY10 Summary Statistics by Carveout ....................................................................................... 9 Table 2-5. Phase III Summary Statistics by Carveout ................................................................................. 9 Table 2-6. EE&C Portfolio Participation by Program and Solution .............................................................. 9 Table 2-7. Impact Evaluation Results Summary ........................................................................................ 11 Table 2-8. HIM NTG Summary .................................................................................................................. 12 Table 2-9. Summary of Incremental Annual Energy Savings by EE Program .......................................... 14 Table 2-10. Summary of Changes to VTD Savings from PY9 ................................................................... 16 Table 2-11. Summary of Lifetime Energy Savings by EE Program ........................................................... 16 Table 2-12. Summary of Demand Savings by EE Program ...................................................................... 19 Table 2-13. Summary of Changes to VTD Demand Savings from PY9 .................................................... 20 Table 2-14. Summary of Demand Savings for DR Programs by Customer Segment and Event ............. 21 Table 2-15. List of Fuel Switching Measures ............................................................................................. 22 Table 2-16. Summary of Fuel Switching Measure Portfolio Impacts ......................................................... 22 Table 2-17. Summary of Portfolio Finances – Gross Verified .................................................................... 22 Table 2-18. Summary PY10 Gross TRC Results by Program ($1,000) [1] ................................................. 24 Table 2-19. Summary PY10 Net TRC Results by Program ($1,000) [1] ..................................................... 24 Table 2-20. Summary P3TD Gross TRC Results by Program ($1,000) [1] ................................................. 24 Table 2-21. Summary P3TD Net TRC Results by Program ($1,000) [1] ..................................................... 25 Table 2-22. Comparison of PY10 Expenditures to Phase III EE&C Plan ($1,000).................................... 26 Table 2-23. Comparison of Expenditures to Phase III EE&C Plan by Program ($1,000) .......................... 26 Table 2-24. Comparison of PY10 Actual Program Savings to EE&C Plan Projections for PY10 .............. 27 Table 2-25. Comparison of Energy Savings to Phase III EE&C Plan by Program .................................... 27 Table 2-26. Summary of Evaluation Recommendations ........................................................................... 29 Table 3-1. Evaluation Activity Matrix .......................................................................................................... 31 Table 3-2. Residential EE Program Summary by Customer Segment ...................................................... 33 Table 3-3. Residential EE Program Gross Impact Sample Design for PY10 ............................................ 35 Table 3-4. Residential EE Program Gross Results for Energy .................................................................. 37 Table 3-5. Residential EE Program Gross Results for Demand ................................................................ 38 Table 3-6. Residential EE Program Net Impact and Process Survey Sample Design for PY10 ............... 40 Table 3-7. Residential EE Program Net Energy Savings Impact Evaluation Results for PY10 ................ 41 Table 3-8. Residential EE Program HIM NTG Summary ........................................................................... 43 Table 3-9. Summary of Residential EE Program Finances – Gross Verified ............................................ 46 Table 3-10. Summary of Residential EE Program Finances – Net Verified .............................................. 48 Table 3-11. Summary of Findings and Recommendations for the Residential EE Program ..................... 50 Table 3-12. Low-Income EE Program Summary by Customer Segment .................................................. 54 Table 3-13. Low-Income EE Program Gross Impact Sample Design for PY10 ........................................ 55 Table 3-14. Low-Income EE Program Gross Results for Energy .............................................................. 56 Table 3-15. Low-Income EE Program Gross Results for Demand ............................................................ 57 Table 3-16. Summary of Low-Income EE Program Finances – Gross Verified ........................................ 58 Table 3-17. Summary of Low-Income EE Program Finances – Net Verified ............................................ 59 Table 3-18. Summary of Findings and Recommendations for the Residential Low-Income EE Program 61 Table 3-19. Small C&I EE Program Summary by Customer Segment ...................................................... 62 Table 3-20. Small C&I EE Program Gross Impact Sample Design for PY10 ............................................ 63 Table 3-21. Small C&I EE Program Gross Results for Energy .................................................................. 64


©2019 Guidehouse Inc. Page iv

Table 3-22. Small C&I EE Program Gross Results for Demand................................................................ 65 Table 3-23. Small and Large C&I EE Programs Net Impact Sample Design for PY10 ............................. 67 Table 3-24. Small C&I EE Programs Net Energy Savings Impact Evaluation Results for PY10 .............. 68 Table 3-25. Small and Large C&I EE Program HIM NTG Summary ......................................................... 69 Table 3-26. Summary of Small C&I EE Program Finances – Gross Verified ............................................ 74 Table 3-27. Summary of Small C&I EE Program Finances – Net Verified ................................................ 75 Table 3-28. Summary of Findings and Recommendations for the Small C&I EE Program ...................... 77 Table 3-29. Large C&I EE Program Summary by Customer Segment ..................................................... 81 Table 3-30. Large C&I EE Program Gross Impact Sample Design for PY10 ............................................ 82 Table 3-31. Large C&I EE Program Gross Results for Energy .................................................................. 84 Table 3-32. Large C&I EE Program Gross Results for Demand ............................................................... 85 Table 3-33. Large C&I EE Program Net Energy Savings Impact Evaluation Results for PY10 ................ 87 Table 3-34. Summary of Large C&I EE Program Finances – Gross Verified ............................................ 89 Table 3-35. Summary of Large C&I EE Program Finances – Net Verified ................................................ 90 Table 3-36. Summary of Findings and Recommendations for the Large C&I EE Program ...................... 92 Table 3-37. CHP Program Summary by Customer Segment .................................................................... 96 Table 3-38. CHP Program Gross Results for Energy ................................................................................ 97 Table 3-39. CHP Program Gross Results for Demand .............................................................................. 97 Table 3-40. CHP Program Net Energy Savings Impact Evaluation Results for PY10............................... 98 Table 3-41. Summary of CHP Program Finances – Gross Verified .......................................................... 99 Table 3-42. Summary of CHP Program Finances – Net Verified ............................................................ 100 Table 3-43. Summary of Findings and Recommendations for the CHP Program ................................... 102 Table 3-44. PY10 DR Program Summary by Customer Segment........................................................... 104 Table 3-45. PY10 DR PYVTD Performance by Event ............................................................................. 104 Table 3-46. Summary of Residential DR Finances – Gross Verified ....................................................... 106 Table 3-47. Summary of Residential DR Finances – Net Verified ........................................................... 107 Table 3-48. Summary of Small C&I DR Finances – Gross Verified......................................................... 108 Table 3-49. Summary of Small C&I DR Finances – Net Verified............................................................. 109 Table 3-50. Summary of Large C&I DR Finances – Gross Verified ........................................................ 110 Table 3-51. Summary of Large C&I DR Finances – Net Verified ............................................................ 111 Table 3-52. Summary of Findings and Recommendations for the DR Programs ................................... 113 Table 4-1. PYTD Financials ..................................................................................................................... 114 Table 4-2. Phase III to Date Financials .................................................................................................... 115 Table 4-3. EE&C Plan Expenditures by Cost Recovery Category........................................................... 116 Table B-1. PY10 Site Inspection Summary .............................................................................................. B-1 Table D-1. Overview of Participation Definitions...................................................................................... D-1


©2019 Guidehouse Inc. Page v

ACRONYMS

AC Air Conditioner BMS Building Management System CAC Central Air Conditioner C&I Commercial and Industrial CAP Customer Assistance Program CDO Commercial Date of Operation CF Coincidence Factor CFL Compact Fluorescent Lamp CHP Combined Heat and Power CI Confidence Interval CRAC Computer Room Air Conditioning CSP Conservation Service Provider, Curtailment Service Provider CV Coefficient of Variation DLC Direct Load Control DR Demand Response DRA Demand Response Aggregator EDC Electric Distribution Company EDT Eastern Daylight Time EE Energy Efficiency EE&C Energy Efficiency and Conservation EEMF Energy Efficiency Marketing Firm EFLH Effective Full Load Hour EM&V Evaluation, Measurement, and Verification EPA US Environmental Protection Agency EUL Effective Useful Life FPL Federal Poverty Level G/E/NP Government/Education/Non-Profit HER Home Energy Report HERS Home Energy Rating System HIM High Impact Measure HOU Hours of Use HSPF Heating Seasonal Performance Factor HVAC Heating, Ventilating, and Air Conditioning ICSP Implementation Conservation Service Provider IMP Interim Measure Protocols kW Kilowatt kWh Kilowatt-Hour LAH Lighting, Appliances & HVAC


©2019 Guidehouse Inc. Page vi

LDV Lagged Dependent Variable LED Light-Emitting Diode LIURP Low-Income Usage Reduction Program M&V Measurement and Verification MSRP Manufacturer Suggested Retail Price MW Megawatt MWh Megawatt-Hour MWh/yr Megawatt-Hour per Year NPV Net Present Value NTG Net-to-Gross O&M Operations and Maintenance P3TD Phase III to Date PA PUC Pennsylvania Public Utility Commission PILD PECO Instant Lighting Discounts PSA Phase III to Date Preliminary Savings Achieved; equal to VTD + PYRTD PSA+CO PSA Savings plus Carryover from Phase II PSD Performance Systems Development PUF Part-Use Factor PY Program Year—e.g., PY8, from June 1, 2016, to May 31, 2017 PYRTD Program Year Reported to Date PYVTD Program Year Verified to Date QC Quality Control RCT Randomized Control Trial RTD Phase III to Date Reported Gross Savings RTO Regional Transmission Organization RUL Remaining Useful Lifetime SEER Seasonal Energy Efficiency Ratio SF Single-Family SIDS Smart Ideas Data System SKU Stock Keeping Unit SSMVP Site-Specific Measurement and Verification Plan SWE Statewide Evaluator T&D Transmission and Distribution TRC Total Resource Cost TRM Technical Reference Manual UEC Unit Energy Consumption UPS Uninterruptible Power Supply VFD Variable Frequency Drive VTD Phase III to Date Verified Gross Savings VTD + CO Phase III to Date Verified Gross Savings plus Carryover from Phase II


©2019 Guidehouse Inc. Page vii

GROSS IMPACT STRATUM STRUCTURE

Program Solution Stratum Name Abbreviated Stratum Name

Residential EE

Lighting, Appliances & HVAC

Appliances Appliances HVAC HVAC Standard LED Standard LED Specialty LED Specialty LED Marketplace Marketplace

Appliance Recycling Refrigerators Refrigerators Freezers Freezers Room Air Conditioners (AC) Room ACs

Whole Home

Large Projects (>2,081 kWh) Large Projects Medium Projects (1,340-2,080 kWh) Medium Projects Small Projects (370-1,399 kWh) Small Projects Very Small Projects (<369 kWh) Very Small Projects

New Construction Large (>3,939 kWh) Large Medium (2,226-3,939 kWh) Medium Small (<2,226 kWh) Small

Behavioral Solution Total Solution Total

Multifamily Targeted

Large – Residential: Buildings in residential market segment with a single decision maker for all projects in the building.

Large Residential

Small – Residential: Projects in residential market segment with individual decision makers

Small Residential

Multisector – C&I and Res: Buildings with common areas in the C&I segments and units in residential segment. (Residential portion of savings)

Multisector


©2019 Guidehouse Inc. Page viii


Residential Low-Income EE

Low-Income Whole Home

Large SF (>1,946 kWh) Large SF Medium SF (1,135-1,945 kWh) Medium SF Small SF (336-1,134 kWh) Small SF Very Small SF (<335 kWh) Very Small SF Multifamily (all buildings) Multifamily Energy Coordinating Agency ECA Giveaways Giveaways Refrigerator Recycling Refrigerator Recycling Freezers Recycling Freezers Recycling Room AC Recycling Room AC Recycling

Small C&I EE

Equipment and Systems

Very high impact projects (≥1,000 MWh) Very High Impact

High impact and/or high uncertainty projects (157-999 MWh) High Impact

Medium impact and/or medium uncertainty projects (53 MWh-156 MWh)

Medium Impact

Low impact projects (above the bottom 2% of solution savings-52 MWh per project)

Low Impact

Very low impact projects (bottom 2% of solution savings) Very Low Impact

Midstream high impact measures (>25 MWh) Midstream High Impact

Midstream low impact measures (<25 MWh) Midstream Low Impact

New Construction


High impact and/or high uncertainty projects (100-999 MWh) High Impact


Low/Medium Impact


Whole Building Medium impact/uncertainty measures Medium Impact/Uncertainty

Low impact/uncertainty measures Low Impact/Uncertainty Data Centers All Projects All Projects

Small C&I Multifamily Targeted

Small – C&I: Buildings in small C&I segment. Small

Multisector – C&I and Res: Buildings with common areas in the C&I segments and units in residential segment. (Small C&I portion of savings)

Multisector


©2019 Guidehouse Inc. Page ix


Large C&I EE



High impact and/or high uncertainty projects (978-2,999 MWh) High Impact

Medium impact and/or medium uncertainty projects (240 MWh-977 MWh)

Medium Impact


Low Impact

Very low impact measures (bottom 2% solution savings) Very Low Impact

Midstream high impact measures (>23 MWh) Midstream High Impact

Midstream low impact measures (<23 MWh) Midstream Low Impact

New Construction


High impact and/or high uncertainty projects (300-1,999 MWh) High Impact


Low Impact


Data Centers All Projects All Projects

Multifamily Targeted

Large – C&I: Buildings in large C&I market segment. Large

Multisector – C&I and Res: Buildings with common areas in the C&I segments and units in residential segment (Large C&I portion of savings)

Multisector

Combined Heat and Power

CHP All Projects All Projects


©2019 Guidehouse Inc. Page x

NET IMPACT STRATUM STRUCTURE

Program Solution Stratum Name Stratum Description

Residential EE Multifamily Targeted

Small Residential Projects in residential market segment with individual decision makers

Large Residential Projects in residential segment with a single decision maker for all projects in the building

Multisector – C&I and Res

Buildings with common areas in the C&I segments and units in residential segment with NTG results informing the Residential EE Program

Small C&I EE Multifamily Targeted Landlords Small C&I segment landlords

Large C&I EE Multifamily Targeted Landlords Large C&I segment landlords

Combined Heat and Power

CHP All Projects All Projects

TYPES OF SAVINGS Gross Savings: The change in energy consumption and/or peak demand that results directly from program-related actions taken by participants in an Energy Efficiency and Conservation (EE&C) program, regardless of why they participated. Net Savings: The total change in energy consumption and/or peak demand that is attributable to an EE&C program. Depending on the program delivery model and evaluation methodology, the net savings estimates may differ from the gross savings estimate due to adjustments for the effects of free riders, changes in codes and standards, market effects, participant and nonparticipant spillover, and other causes of change in energy consumption or demand not directly attributable to the EE&C program. Reported Gross: Also referred to as ex ante (Latin for “beforehand”) savings. The energy and peak demand savings values calculated by the electric distribution company (EDC) or its program implementation conservation service providers (ICSPs) and stored in the program tracking system. Unverified Reported Gross: The Phase III Evaluation Framework allows EDCs and the evaluation contractors the flexibility to not evaluate each program every year. If an EE&C program is being evaluated over a multiyear cycle, the reported savings for a program year where evaluated results are not available are characterized as unverified reported gross until the impact evaluation is completed and verified savings can be calculated and reported. Verified Gross: Also referred to as ex post (Latin for “from something done afterward”) gross savings. The energy and peak demand savings estimates reported by the independent evaluation contractor after the gross impact evaluation and associated measurement and verification (M&V) efforts have been completed.


©2019 Guidehouse Inc. Page xi

Verified Net: Also referred to as ex post net savings. The energy and peak demand savings estimates reported by the independent evaluation contractor after applying the results of the net impact evaluation. Typically calculated by multiplying the verified gross savings by a net-to-gross (NTG) ratio. Annual Savings: Energy and demand savings expressed on an annual basis, or the amount of energy and/or peak demand an EE&C measure or program can be expected to save over the course of a typical year. Annualized savings are noted as MWh/yr or MW/yr. The Pennsylvania Technical Reference Manual (TRM) provides algorithms and assumptions to calculate annual savings, and Act 129 compliance targets for consumption reduction are based on the sum of the annual savings estimates of installed measures or behavior change. Lifetime Savings: Energy and demand savings expressed in terms of the total expected savings over the useful life of the measure. Typically calculated by multiplying the annual savings of a measure by its effective useful life. The total resource cost (TRC) test uses savings from the full lifetime of a measure to calculate the cost-effectiveness of EE&C programs. Program Year Reported to Date (PYRTD): The reported gross energy and peak demand savings achieved by an EE&C program or portfolio within the current program year. PYTD values for energy efficiency will always be reported gross savings in a semiannual or preliminary annual report. Program Year Verified to Date (PYVTD): The verified gross energy and peak demand savings achieved by an EE&C program or portfolio within the current program year as determined by the impact evaluation findings of the independent evaluation contractor. Phase III to Date (P3TD): The energy and peak demand savings achieved by an EE&C program or portfolio within Phase III of Act 129. Reported in several permutations described below.

1. Phase III to Date Reported (RTD): The sum of the reported gross savings recorded to date in Phase III of Act 129 for an EE&C program or portfolio.

2. Phase III to Date Verified (VTD): The sum of the verified gross savings recorded to date in Phase III of Act 129 for an EE&C program or portfolio, as determined by the impact evaluation finding of the independent evaluation contractor.

3. Phase III to Date Preliminary Savings Achieved (PSA): The sum of the verified gross savings (VTD) from previous program years in Phase III where the impact evaluation is complete plus the reported gross savings from the current program year (PYTD).

4. Phase III to Date Preliminary Savings Achieved + Carryover (PSA+CO): The sum of the verified gross savings from previous program years in Phase III plus the reported gross savings from the current program year plus any verified gross carryover savings from Phase II of Act 129. This is the best estimate of an EDC’s progress toward the Phase III compliance targets.

5. Phase III to Date Verified + Carryover (VTD + CO): The sum of the verified gross savings recorded to date in Phase III plus any verified gross carryover savings from Phase II of Act 129.

Per guidance from the Pennsylvania Statewide Evaluator (SWE), all demand savings that were achieved from energy efficiency measures are shown in this report without line losses (i.e., at the meter). All demand savings that were achieved from demand response (DR) measures are shown in this report with line losses (i.e., at the generator).

Note that all values in the report are summed prior to rounding. Therefore, table totals may not equal the sum of all rows.


©2019 Guidehouse Inc. Page 1

1. INTRODUCTION Pennsylvania Act 129 of 2008, signed on October 15, 2008, mandated energy savings and demand reduction goals for the largest electric distribution companies (EDCs) in Pennsylvania for Phase I (2008-2013). Phase II of Act 129 began in 2013 and concluded in 2016. In late 2015, each EDC filed a new Energy Efficiency and Conservation (EE&C) Plan with the Pennsylvania Public Utilities Commission (PA PUC) detailing the proposed design of its portfolio for Phase III. These plans were updated based on stakeholder input and subsequently approved by the PUC in 2016. Implementation of Phase III of the Act 129 programs began on June 1, 2016. This report documents the progress and effectiveness of the Phase III EE&C accomplishments for PECO in Program Year 10 (PY10), as well as the cumulative accomplishments of the Phase III programs since inception. This report also documents the energy savings carried over from Phase II. The Phase II carryover savings count toward EDC savings compliance targets for Phase III. This report details the participation, spending, reported gross, verified gross, and verified net impacts of the energy efficiency (EE) programs in PY10. Compliance with Act 129 savings goals are ultimately based on verified gross savings. This report also includes estimates of cost-effectiveness according to the total resource cost (TRC) test.1 PECO has retained Navigant Consulting, Inc., n/k/a Guidehouse Inc. (Navigant),2 as an independent evaluation contractor for Phase III of Act 129. Navigant is responsible for the measurement and verification (M&V) of the savings and the calculation of gross verified and net verified savings. For select program solution offerings (solutions), Navigant also performed targeted process evaluation activities to examine specific research areas. This report presents relevant key findings and recommendations identified by the process evaluation and documents any changes to EE&C program delivery to be considered based on the recommendations. Phase III of Act 129 includes a demand response (DR) goal for PECO. DR events are limited to the months of June through September, which are the first 4 months of the Act 129 program year. Because the DR season is completed early in the program year, it is possible to complete the independent evaluation of verified gross savings for DR sooner than for the EE programs. PECO reported the verified gross DR impacts for PY10 as well as the cumulative DR performance of the EE&C program to date for Phase III of Act 129 in the Semiannual Report to the Pennsylvania Public Utility Commission3 filed July 15, 2019. Section 3.6 of this report includes PECO’s previously reported DR performance results for PY10. Navigant worked with the Statewide Evaluator (SWE) throughout the report’s development to address questions related to compliance as they arose and appreciates the SWE’s collaboration to ensure this final report is accurate and agreeable to relevant parties.

1 The Pennsylvania TRC test for Phase I was adopted by PUC order at Docket No. M-2009-2108601 on June 23, 2009 (2009 PA TRC Test Order). The TRC Test Order for Phase I was later refined in the same docket on August 2, 2011 (2011 PA TRC Test Order). The 2013 TRC Order for Phase II of Act 129 was issued on August 30, 2012. The 2016 TRC Test Order for Phase III of Act 129 was adopted by PUC order at Docket No. M-2015-2468992 on June 11, 2015. 2 On October 11, 2019, Guidehouse LLP completed its previously announced acquisition of Navigant Consulting Inc. In the months ahead, we will be working to integrate the Guidehouse and Navigant businesses. In furtherance of that effort, we recently renamed Navigant Consulting Inc. as Guidehouse Inc. 3 PECO. Semiannual Report to the Pennsylvania Public Utility Commission, http://www.puc.pa.gov/pcdocs/1628145.pdf.

http://www.puc.pa.gov/pcdocs/1628145.pdf



2. SUMMARY OF ACHIEVEMENTS

2.1 The Carryover Savings from Phase II of Act 129

PECO reported 0 MWh/yr of portfolio-level carryover savings from Phase II to Phase III. The Commission’s Phase III Implementation Order4 allowed EDCs to carry over savings achieved within Phase II that were in excess of the Phase II portfolio savings target. Phase I carryover savings cannot be counted in the calculation of Phase II carryover savings. Figure 2-1 compares PECO’s Phase II verified gross savings total to the Phase II compliance target to illustrate the carryover calculation. Because PECO’s Phase II verified gross savings did not exceed PECO’s Phase II target, it was not eligible to carry over savings from Phase II toward its Phase III overall compliance target.5

Figure 2-1. Carryover Savings from Phase II of Act 129

Sources: Smart Ideas Data System (SIDS) database, conservation service provider (CSP) tracking data

The Commission’s Phase III Implementation Order6 also allowed EDCs to carry over savings in excess of the Phase II government, educational, and non-profit (G/E/NP) savings goal and excess savings from the low-income customer segment.7 PECO carried over 0 MWh/yr of G/E/NP and 0 MWh/yr of low-income

4 Pennsylvania Public Utility Commission, Energy Efficiency and Conservation Program Implementation Order, at Docket No. M-2014-2424864, (Phase III Implementation Order), entered June 11, 2015. 5 Pennsylvania Public Utility Commission, Energy Efficiency and Conservation Program Compliance Order, at Docket No. M-2012-2289411, (Phase II Compliance Determination Order), entered August 3, 2017. 6 Pennsylvania Public Utility Commission, Energy Efficiency and Conservation Program Implementation Order, at Docket No. M-2014-2424864, (Phase III Implementation Order), entered June 11, 2015. 7 Proportionate to those savings achieved by dedicated low-income programs in Phase III.



customer segment savings.8 Figure 2-2 shows the calculation of carryover savings for the low-income and G/E/NP targets.9

Figure 2-2. Customer Segment-Specific Carryover from Phase II

Sources: Navigant analysis

2.2 Phase III Energy Efficiency Achievements to Date

In PY10, starting on June 1, 2018 and ending on May 31, 2019, PECO has claimed the following savings:

• 439,399.2 MWh/yr of reported gross electric energy savings (PYRTD)

• 48.86 MW of reported gross peak demand savings (PYRTD) from EE programs

• 427,771.4 MWh/yr of verified gross electric energy savings (PYVTD)

• 57.84 MW of verified gross peak demand savings (PYVTD) from EE programs

Since the beginning of Phase III of Act 129 on June 1, 2016, PECO has achieved the following savings:

• 1,049,688.2 MWh/yr of reported gross electric energy savings (RTD)

• 111.77 MW of reported gross peak demand savings (RTD) from EE programs

• 1,028,611.0 MWh/yr of verified gross electric energy savings (VTD)

• 136.51 of verified gross peak demand savings (VTD) from EE programs

8 Pennsylvania Public Utility Commission, Energy Efficiency and Conservation Program Compliance Order, at Docket No. M-2012-2289411, (Phase II Compliance Determination Order), entered August 3, 2017. 9 Pennsylvania Public Utility Commission, Energy Efficiency and Conservation Program Compliance Order.



Including carryover savings from Phase II, PECO has achieved:

• 1,028,611.0 MWh/yr of VTD plus portfolio-level carryover (CO) energy savings

o This represents 52.4% of the May 31, 2021 energy savings compliance target of 1,962,659 MWh/yr

Figure 2-3 summarizes PECO’s progress toward the Phase III portfolio compliance target.

Figure 2-3. EE&C Plan Performance toward Phase III Portfolio Compliance Target

Source: Navigant analysis

The Phase III Implementation Order directed EDCs to offer conservation measures to the low-income customer segment based on the proportion of electric sales attributable to low-income households. The proportionate number of measures targeted for PECO is 8.8%. PECO offers 269 EE&C measures to its residential and non-residential customer classes. There are 117 measures available to the low-income customer segment at no cost to the customer. This represents 43.5% of the total measures offered in the EE&C Plan and exceeds the proportionate number of measures targeted. The PA PUC also established a low-income energy savings target of 5.5% of the portfolio savings goal. The verified gross low-income savings target for PECO is 107,946 MWh/yr. Figure 2-4 compares the VTD performance for the low-income customer segment to the Phase III savings target. Based on the latest available information, PECO has achieved 56.3% of the Phase III low-income energy savings target.



Figure 2-4. EE&C Plan Performance toward Phase III Low-Income Compliance Target


The Phase III Implementation Order established a G/E/NP energy savings target of 3.5% of the portfolio savings goal. The verified gross G/E/NP savings target for PECO is 68,693 MWh/yr. Figure 2-5 compares the VTD performance for the G/E/NP customer segment to the Phase III savings target. Based on the latest available information, PECO has achieved 137.2% of the Phase III G/E/NP energy savings target.

Figure 2-5. EE&C Plan Performance against Phase III G/E/NP Compliance Target




2.3 Phase III DR Achievements to Date

The Phase III DR performance target for PECO is 161 MW. Compliance targets for DR programs are based on average performance across events and are established at the system level, which means the load reductions measured at the customer meter must be escalated to reflect transmission and distribution (T&D) losses. Act 129 DR events are triggered by PJM’s day-ahead load forecast. When the day-ahead forecast is above 96% of the peak load forecast for the year, a DR event is initiated for the following day. In PY10, PECO called six DR events: July 2, July 3, August 6, August 28, September 4, and September 5. The average performance for these six events is presented in Table 2-1. The full methodology and results are available in the standalone PY10 DR report, submitted to the SWE on January 15, 2019.10 Table 2-1 shows a summary of the DR performance to date.

Table 2-1. Phase III to Date DR Performance by Event

PY Event Date Residential DR (MW)

Small C&I DR (MW)

Large C&I DR (MW)

Portfolio (MW)

Relative Precision at

90% Confidence

PY9 June 13, 2017 39.53 0.00 118.21 157.74 8.8% PY9 July 20, 2017 33.48 0.00 107.88 141.36 9.6% PY9 July 21, 2017 23.34 0.00 125.82 149.16 8.9% PY10 July 2, 2018 38.93 0.00 155.98 194.92 10.0% PY10 July 3, 2018 33.84 0.00 146.76 180.60 10.8% PY10 August 6, 2018 25.07 1.15 180.12 206.34 10.4% PY10 August 28, 2018 30.69 0.92 160.76 192.36 11.3% PY10 September 4, 2018 29.99 0.77 142.69 173.45 11.1% PY10 September 5, 2018 29.52 0.84 131.75 162.12 11.8%

PYVTD - Average PY10 DR Event Performance 31.34 0.61 153.01 184.96 10.9%

P3TD - Average Phase III DR Event Performance 31.60 0.41 141.11 173.12 10.3%


10 PECO. Annual Report to the Pennsylvania Public Utility Commission Demand Response Performance Report Only. January 15, 2019. http://www.puc.pa.gov//pcdocs/1602629.pdf




The Commission’s Phase III Implementation Order also established a requirement that EDCs achieve at least 85% of the Phase III compliance reduction target in each DR event. For PECO, this translates to a 136.9 MW minimum for each DR event. Figure 2-6 compares the performance of each of the DR events in PY10 to the event-specific minimum and average targets.

Figure 2-6. Event Performance Compared to 85% Per-Event Target


2.4 Phase III Performance by Customer Segment

Table 2-2 through Table 2-5 present the participation, savings, and spending results by customer sector for PY10 and Phase III. The residential, small commercial and industrial (C&I), and large C&I sectors (also referred to as customer segments or rate classes) are defined by PECO tariff. The residential low-income and G/E/NP customer segments (Table 2-4 and Table 2-5) were defined by statute (66 Pa. C.S. § 2806.1). The residential low-income segment is primarily a subset of the residential customer class; however, it also includes low-income-qualified residents in master-metered buildings in the small C&I and large C&I sectors. The G/E/NP segment is a subset of the small C&I and large C&I sectors.



Table 2-2 provides the PY10 participation counts and spending totals for PECO’s programs for the three sectors (residential, small C&I, and large C&I) inclusive of all low-income and G/E/NP segments.

Table 2-2. PY10 Summary Statistics by Customer Segment

Parameter Residential Small C&I Large C&I Total

No. of Participants 1,679,099 3,583 1,377 1,684,059

PY10 Energy Realization Rate 0.98 1.00 0.93 0.97

PYVTD MWh/yr 260,592 58,483 108,696 427,771 PY10 Demand Realization Rate 1.32 1.02 1.05 1.18 PYVTD MW (EE) 33.16 9.37 15.31 57.84 PYVTD MW (DR) 32.12 0.00 117.31 149.43 Incentives ($1,000) $11,540 $3,126 $6,139 $20,804


Table 2-3 provides the Phase III to-date participation counts and spending totals for PECO’s programs for the three sectors (residential, small C&I, and large C&I) inclusive of all low-income and G/E/NP segments.

Table 2-3. Phase III Summary Statistics by Customer Segment

Parameter Residential Small C&I Large C&I Total

No. of Participants 4,350,511 6,440 2,034 4,358,985 P3TD Energy Realization Rate 0.98 1.03 0.96 0.98 VTD MWh/yr 680,954 130,036 217,621 1,028,611 P3TD Demand Realization Rate 1.38 1.03 1.01 1.22 VTD MW (EE) 86.45 18.98 31.08 136.51 VTD MW (DR) 32.12 0.00 117.31 149.43 Incentives ($1,000) $30,769 $6,292 $11,023 $48,084


Table 2-4 summarizes the savings, spending, and participation values for the low-income and G/E/NP customer segment carveouts only. PECO tracks activities for two low-income segments that contribute to the low-income carveout:

• PECO customers at 50% or below the federal poverty level (FPL)

• PECO customers at 51% to 150% of the FPL

The low-income totals correspond to achievements shown in Figure 2-4, and the G/E/NP totals correspond to achievements shown in Figure 2-5.



Table 2-4. PY10 Summary Statistics by Carveout

Parameter Low-Income

(0%-50% FPL) Low-Income

(51%-150% FPL) Low-Income Total G/E/NP Total

No. of Participants 3,853 15,549 19,402 1,052 PY10 Energy Realization Rate 0.87 0.88 0.88 0.97

PYVTD MWh/yr 3,702 17,708 21,409 40,006 Incentives ($1,000) $0 $130 $130 $2,736 Program (Non-Incentive) Costs ($1,000) $855 $3,502 $4,357 $66


Table 2-5 summarizes the carveout performance since the beginning of Phase III.

Table 2-5. Phase III Summary Statistics by Carveout

Parameter Low-Income

(0%-50% FPL) Low-Income

(51%-150% FPL) Low-Income Total G/E/NP Total

No. of Participants 8,216 197,373 205,589 1,490 P3TD Energy Realization Rate 0.85 0.82 0.83 0.96 VTD MWh/yr 9,449 51,288 60,737 94,260 Incentives ($1,000) $1 $1,034 $1,035 $5,827 Program (Non-Incentive) Costs ($1,000) $3,079 $15,638 $18,716 $196


2.5 Summary of Participation by Program

Participation is defined differently for each program and solution depending on the program delivery channel and data tracking practices. Appendix D includes an overview of the different participation definitions for each solution, and Table 2-6 provides the current participation totals by program and solution for PY10 and for Phase III to date.

Table 2-6. EE&C Portfolio Participation by Program and Solution

Program and Solution PYTD Participation P3TD Participation

Lighting, Appliances & HVAC 1,186,451 2,802,576

Appliance Recycling 17,134 41,746

Whole Home 6,204 13,928

New Construction 706 1,577

Behavioral1 383,747 1,204,107

Multifamily Targeted Market Segment 9,817 20,444

Residential EE Total 1,604,059 4,084,378



Program and Solution PYTD Participation P3TD Participation

Lighting 0 167,058

Whole Home 19,402 38,531

Low-Income EE Total 19,402 205,589 Equipment and Systems 1,418 2,907

New Construction 59 123

Whole Building 290 725

Data Centers Targeted Market Segment 2 2

Multifamily Targeted Market Segment 117 353

Small C&I EE Total 1,886 4,110 Equipment and Systems 814 1,304

New Construction 41 101

Data Centers Targeted Market Segment 1 4

Multifamily Targeted Market Segment 48 120

Large C&I EE Total 904 1,529 Combined Heat and Power (CHP) 3 5 Residential DR 56,030 61,4402 Small C&I DR 1,427 1,5862 Large C&I DR 348 3482

Portfolio Total 1,684,059 4,358,985 1 Behavioral participation was updated from 363,076 in the PECO PY10 Semiannual Report11 to 383,747 in this report. The Behavioral participation in the PY10 Semiannual Report was based on 11 months of PY10 program delivery, whereas the updated number is based on the full 12 months of PY10 of program delivery. 2 DR participation is not additive like other programs because the same participants tend to remain in the program with only small attrition. Therefore, total participation in the DR programs for Phase III is equal to the highest program year participation count for each of the three programs. Source: Navigant analysis

2.6 Summary of Impact Evaluation Results

During PY10, Navigant completed impact evaluations for all active EE programs and solutions in the PECO portfolio. Table 2-7 summarizes the realization rates and net-to-gross (NTG) ratios by program and solution.

11 PECO. Semiannual Report to the Pennsylvania Public Utility Commission. July 15, 2019. http://www.puc.state.pa.us//pcdocs/1628145.pdf

http://www.puc.state.pa.us/pcdocs/1628145.pdf



Table 2-7. Impact Evaluation Results Summary

Program and Solution Energy Realization Rate

Demand Realization Rate NTG Ratio

Lighting, Appliances & HVAC 1.02 1.03 0.49

Appliance Recycling 1.07 1.03 0.37

Whole Home 0.87 0.90 0.88

New Construction 1.04 1.02 0.87

Behavioral 0.94 -* 1.00

Multifamily Targeted Market Segment 0.99 0.96 0.92

Residential EE Total 0.99 1.38 0.65 Whole Home 0.88 0.88 1.00

Low-Income EE Total 0.88 0.88 1.00 Equipment and Systems 1.01 1.13 0.75


Whole Building 0.99 0.68 0.99

Data Centers 0.42 0.33 0.92


Small C&I EE Total 1.01 1.02 0.76 Equipment and Systems 1.00 1.05 0.77


Data Centers 0.60 0.61 0.67


Large C&I EE Total 0.99 1.08 0.74 CHP 0.71 0.87 0.87

Portfolio Total 0.97 1.18 0.71 * For the Residential Behavioral Solution, the implementer does not report demand savings; however, the SWE requires PECO to verify demand savings. As a result, there is no demand realization rate for the Behavioral Solution. The verified demand savings do get added to the residential program savings. As a result, the demand realization rate for the residential program is greater than the demand realization rate for each individual solution in the program. Source: Navigant analysis

Navigant conducted NTG research, including analysis of high impact measures (HIMs), in PY10 for the Residential, Small C&I, and Large C&I Multifamily Targeted Market Segments and the Combined Heat and Power (CHP) Program. The team applied the PY9 NTG results to the other solutions. Findings from NTG research are not used to adjust compliance savings in Pennsylvania. Instead, NTG research provides directional information for program planning purposes. Table 2-8 presents NTG findings for the HIMs studied in PY10.12 12 The NTG estimates provided for HIMs are based on findings from surveys conducted with tenants and landlords participating in the Multifamily Targeted Market Segment aligning with guidance provided in Section 3.4.1.4 of the Phase III Evaluation Framework indicating that HIM research should focus on measures in downstream programs only.



Navigant initially planned HIM research for LEDs, low flow faucet aerators, and low flow showerheads in the Residential and Small C&I Multifamily Targeted Market Segments, and LEDs, water pumps, and lighting controls for the Large C&I Multifamily Targeted Market Segment. However, due to the limited landlord population, the evaluation team only gathered enough data to report HIM results for LEDs for the C&I sector in PY10.13

Table 2-8. HIM NTG Summary

HIM Free Ridership Spillover NTG Ratio Associated Program and Solution

ENERGY STAR LED 0.08 0.00 0.92 Residential EE – Multifamily Targeted

Low Flow Faucet Aerator 0.03 0.00 0.97 Residential EE – Multifamily Targeted

Low Flow Showerhead 0.06 0.00 0.94 Residential EE – Multifamily Targeted

ENERGY STAR LED 0.19 0.00 0.81 Small and Large C&I EE – Multifamily Targeted


2.7 Summary of Energy Impacts by Program

Act 129 compliance targets are based on annualized savings estimates (MWh/yr). Each program year the annual savings achieved by EE&C program activity are recorded as incremental annual—or first-year—savings and are added to EDCs’ progress toward compliance. Incremental annual savings estimates are presented in Section 2.7.1. Lifetime energy savings incorporate the effective useful life (EUL) of installed measures and estimate the total energy savings associated with EE&C program activity. Lifetime savings are used in the TRC test by program participants when assessing the economics of upgrades and by the SWE when calculating the emissions benefits of Act 129 programs. Section 2.7.2 presents the lifetime energy savings by program.

13 LEDs represented the largest portion of solution savings: 85% of Residential, 99% of Small C&I, and 97% of Large C&I Multifamily Targeted solutions.



2.7.1 Incremental Annual Energy Savings by Program

Figure 2-7 presents a summary of the PYTD energy savings by program for PY10. The energy impacts in this report are presented at the meter level and do not reflect adjustments for T&D losses. The verified gross savings are adjusted by the energy realization rate, and the verified net savings are adjusted by both the realization rate and the NTG ratio.

Figure 2-7. PYTD Energy Savings by Program




Figure 2-8 presents a summary of the energy savings by program for Phase III of Act 129.

Figure 2-8. P3TD Energy Savings by Program


A summary of energy impacts by program through PY10 is presented in Table 2-9.

Table 2-9. Summary of Incremental Annual Energy Savings by EE Program

Program and Solution

PYRTD (MWh/yr)

PYVTD Gross

(MWh/yr)

PYVTD Net (MWh/yr)

RTD (MWh/yr)

VTD Gross (MWh/yr)

VTD Net (MWh/yr)


137,639 140,330 69,104 335,106 341,313 166,895

Appliance Recycling 16,939 18,171 6,657 41,415 40,651 14,902

Whole Home 7,616 6,663 5,853 17,353 16,335 14,428

New Construction 1,872 1,946 1,686 4,062 4,127 3,309

Behavioral 73,716* 69,127 69,127 218,074 210,907 210,907

Multifamily Targeted Market Segment

4,384 4,333 3,986 9,862 9,649 8,573

Residential EE Total 242,167 240,570 156,414 625,872 622,982 419,014




PYRTD (MWh/yr)

PYVTD Gross

(MWh/yr)

PYVTD Net (MWh/yr)

RTD (MWh/yr)

VTD Gross (MWh/yr)

VTD Net (MWh/yr)

Lighting 0 0 0 9,086 9,081 9,081

Whole Home 24,339 21,409 21,409 64,221 58,049 58,049

Low-Income EE Total 24,339 21,409 21,409 73,307 67,129 67,129

Equipment and Systems 40,550 41,003 30,947 89,780 88,846 67,477

New Construction 3,379 3,608 965 7,816 8,171 2,264

Whole Building 8,471 8,419 8,337 17,345 17,267 16,854

Data Centers Targeted Market Segment

119 50 46 119 50 46


4,219 4,145 3,350 8,033 6,880 5,117

Small C&I EE Total 56,738 57,224 43,644 123,093 121,214 91,759

Equipment and Systems 83,053 82,853 63,428 176,925 173,813 133,090

New Construction 6,661 5,840 2,370 15,735 14,945 6,521


36 22 14 546 529 418


3,957 3,937 3,181 8,507 8,374 6,049

Large C&I EE Total 93,707 92,652 68,993 201,713 197,662 146,079

CHP 22,449 15,916 13,899 25,703 19,624 17,199

Portfolio Total 439,399 427,771 304,359 1,049,688 1,028,611 741,180 *Behavioral reported savings in preliminary report was 78,154 MWh, which was based on an estimate provided by the implementer because the full data for the program year was not available yet at that point. The reported savings in this annual report are updated to reflect the full data for PY10. Source: Navigant analysis

For the following programs, the previously reported VTD savings from prior years have changed since Navigant submitted the PY9 final annual report. SWE audit activities recommended an adjustment to increase savings by 2,132 MWh/yr for the PY9 verified gross savings because of discrepancies found in their review (described in Table 2-10). These adjustments, in sum, represent a 0.55% increase of the PY9 gross PYVTD energy savings.



Table 2-10. Summary of Changes to VTD Savings from PY9

Program Solution VTD

Adjustment (MWh/yr)

Low-Income Carveout

Adjustment (MWh/yr)

Explanation

Residential Lighting, Appliances & HVAC -248 -

Adjusted lighting baseline assumption for five models; adjusted commercial hours of use (HOU) and coincidence factor (CF) assumption for one data point (related to lighting cross-sector sales); corrected calculations related to rebated refrigerators

Residential Whole Home -157 - Corrected calculations related to sample verification

Residential Multifamily Targeted Market Segment 533 - Corrected a HOU calculation

Low-Income Lighting -3 -1 Adjusted lighting baseline assumption for five models

Low-Income Whole Home 1,527 1,527 Corrected a HOU calculation

Small C&I Multifamily Targeted Market Segment 224 - Corrected a HOU calculation

Large C&I Multifamily Targeted Market Segment 256 - Corrected a HOU calculation

Total 2,132 1,526 Source: SWE Audit

2.7.2 Lifetime Energy Savings by Program

Table 2-11 presents the PYVTD and VTD lifetime energy savings by program. Lifetime energy savings are calculated by multiplying the annual energy savings by the EUL. Per the PA 2016 TRC Order, the measure EUL does not exceed 15 years for any measure in the portfolio. Additionally, early replacement measures are subject to a dual baseline calculation, leading to modified lifetime savings. For these measures, savings relative to the in-place baseline equipment are used for the remaining useful lifetime (RUL) of the base equipment. After the RUL, savings relative to code equipment are used for the remainder of the efficient measure’s EUL.

Table 2-11. Summary of Lifetime Energy Savings by EE Program

Program PYVTD Gross

Lifetime Energy (MWh/yr)

PYVTD Net Lifetime Energy

(MWh/yr) VTD Gross Lifetime

Energy (MWh/yr) VTD Net Lifetime Energy (MWh/yr)

Residential EE 1,630,444 870,395 4,201,812 2,237,205 Low-Income EE 159,515 159,515 498,047 498,047 Small C&I EE 597,809 456,007 1,264,447 956,577 Large C&I EE 1,078,739 796,029 2,343,000 1,719,086 CHP 238,744 208,490 294,356 257,984

Portfolio Total 3,705,250 2,490,436 8,601,663 5,668,899 Source: Navigant analysis



For the following programs, the previously reported VTD lifetime savings from prior years have changed since Navigant submitted the PY9 final annual report. These VTD lifetime savings adjustments are directly related to the details provided in Table 2-10:

• Residential EE: PYVTD Gross Lifetime Energy decreased by 75 MWh/yr and PYVTD Net Lifetime Energy increased by 809 MWh/yr

• Low-Income EE: PYVTD Gross Lifetime Energy increased by 11,626 MWh/yr and PYVTD Net Lifetime Energy increased by 11,626 MWh/yr

• Small C&I EE: PYVTD Gross Lifetime Energy increased by 1,760 MWh/yr and PYVTD Net Lifetime Energy increased by 1,137 MWh/yr

• Large C&I EE: PYVTD Gross Lifetime Energy increased by 2,359 MWh/yr and PYVTD Net Lifetime Energy increased by 1,525 MWh/yr

2.8 Summary of Demand Impacts by Program

PECO’s Phase III EE&C programs achieve peak demand reductions in two primary ways. The first is through coincident reductions from EE measures, and the second is through dedicated DR offerings that exclusively target temporary demand reductions on peak days. EE reductions coincident with system peak hours are reported and used to calculate benefits in the TRC test but do not contribute to Phase III peak demand reduction compliance goals. Phase III peak demand reduction targets are exclusive to DR programs. The two types of peak demand reduction savings are also treated differently for reporting purposes. Peak demand reductions from EE are generally additive across program years, meaning that the P3TD savings reflect the sum of the first-year savings in each program year. Conversely, DR goals are based on average portfolio impacts across all events, so cumulative DR performance is expressed as the average performance of each of the DR events called in Phase III to date. Because of these differences, demand impacts from EE and DR are reported separately in the following subsections.

2.8.1 Energy Efficiency

Act 129 defines peak demand savings from EE as the average expected reduction in electric demand from 2:00 p.m. to 6:00 p.m. EDT on non-holiday weekdays from June through August. The peak demand impacts from EE in this report are presented at the meter level and do not reflect adjustments for T&D losses. Figure 2-9 presents a summary of the PYTD demand savings by EE program for PY10.



Figure 2-9. PYTD Demand Savings by EE Program


Figure 2-10 presents a summary of the P3TD demand savings by EE program for Phase III of Act 129.

Figure 2-10. P3TD Demand Savings by EE Program




A summary of the peak demand impacts by EE program through the current reporting period are presented in Table 2-12.

Table 2-12. Summary of Demand Savings by EE Program


PYRTD (MW)

PYVTD Gross (MW)

PYVTD Net (MW) RTD (MW) VTD Gross

(MW) VTD Net

(MW)

Lighting, Appliances & HVAC 17.88 18.45 9.08 43.95 46.19 22.58

Appliance Recycling 2.49 2.57 0.94 6.11 5.89 2.16

Whole Home 0.92 0.83 0.73 2.01 1.88 1.66

New Construction 0.54 0.55 0.48 1.22 1.17 0.95

Behavioral 0.00 7.89 7.89 0.00 24.08 24.08

Multifamily Targeted Market Segment 0.56 0.54 0.50 1.26 1.18 1.05

Residential EE Total 22.39 30.83 19.62 54.55 80.40 52.47 Lighting 0.00 0.00 0.00 1.07 1.07 1.07

Whole Home 2.93 2.57 2.57 7.58 6.81 6.81

Low-Income EE Total 2.93 2.57 2.57 8.65 7.88 7.88

Equipment and Systems 6.22 7.018 5.30 12.26 12.65 9.61


Whole Building 1.82 1.23 1.22 3.53 2.48 2.42


0.02 0.006 0.01 0.02 0.01 0.01


Small C&I EE Total 8.95 9.14 6.96 17.94 17.19 12.92 Equipment and Systems 11.02 11.522 8.82 24.84 25.04 19.14



0.00 0.003 0.00 0.04 0.04 0.03


Large C&I EE Total 12.40 13.40 9.78 27.96 28.67 21.02 CHP 2.18 1.89 1.65 2.67 2.36 2.07

Portfolio Total 48.86 57.84 40.59 111.77 136.51 96.36 Source: Navigant Analysis

For the following programs, the previously reported VTD demand savings from prior years have changed since Navigant submitted the PY9 final annual report. SWE audit activities recommended an adjustment



to decrease savings by 0.19 MW for the PY9 verified gross demand savings because of discrepancies found in their review (described in Table 2-13, note that explanations are identical to Table 2-10). These adjustments, in sum, represent a 0.37% reduction of the PY9 gross PYVTD demand savings. These adjustments did not affect any of the carveouts.

Table 2-13. Summary of Changes to VTD Demand Savings from PY9

Program Solution VTD

Adjustment (MW/yr)

Explanation

Residential Lighting, Appliances & HVAC -0.03

Adjusted lighting baseline assumption for five models; adjusted commercial HOU and CF assumption for one data point (related to lighting cross-sector sales)

Residential Whole Home 0.02 Corrected a HOU calculation

Residential Multifamily Targeted Market Segment -0.05 Corrected a HOU calculation

Small C&I Multifamily Targeted Market Segment -0.11 Corrected a HOU calculation

Large C&I Multifamily Targeted Market Segment -0.02 Corrected a HOU calculation

Total -0.19 Source: SWE Audit

2.8.2 Demand Response

Act 129 defines peak demand savings from DR as the average reduction in electric demand during the hours when a DR event is initiated. Phase III DR events are initiated according to the following guidelines:14

• Curtailment events shall be limited to the months of June through September.

• Curtailment events shall be called for the first 6 days of each program year (starting in PY9) in which the peak hour of PJM’s day-ahead forecast for the PJM regional transmission organization (RTO) is greater than 96% of the PJM RTO summer peak demand forecast for the months of June through September.

• Each curtailment event shall last 4 hours.

• Each curtailment event shall be called such that it will occur during the day’s forecasted peak hour(s) above 96% of the PJM RTO summer peak demand forecast.

• Once six curtailment events have been called in a program year, the peak demand reduction program shall be suspended for that program year.

Phase III DR programs began operating in PY9; therefore, no DR program savings were reported for PY8. Starting in PY9, the peak demand impacts from DR are presented at the system level and reflect

14 Pennsylvania Public Utility Commission, Energy Efficiency and Conservation Program Implementation Order, at Docket No. M-2014-2424864, (Phase III Implementation Order), entered June 11, 2015.



adjustments to account for T&D losses. PECO uses the following line loss percentages/multipliers by sector:15

• Residential = 107.99% or 1.0799

• Small C&I = 107.99% or 1.0799

• Large C&I = 107.99% or 1.0799 Table 2-14 (which is also shown as Table 2-1) summarizes the demand reductions for each of the DR programs in PECO’s EE&C Plan and for the DR portfolio. Verified gross demand savings are the average performance across all Phase III DR events independent of how many events occurred in a given program year. The P3TD – Average Phase III DR Event Performance row is calculated as an average of all events to date, so years with more or fewer events will not be weighted disproportionately.

Table 2-14. Summary of Demand Savings for DR Programs by Customer Segment and Event


Small C&I DR (MW)

Large C&I DR (MW)

Portfolio (MW)


90% Confidence



P3TD - Average Phase III DR Event Performance 31.60 0.41 141.11 173.12 10.3%

*P3TD impacts are based on an average of all events and not an average of program years. Source: Navigant analysis

2.9 Summary of Fuel Switching Impacts

Act 129 allows EDCs to achieve electric savings by converting electric equipment to non-electric equipment. Table 2-15 lists the fuel switching measures offered in Phase III, while Table 2-16 provides the key fuel switching metrics to date.

15 Pennsylvania Public Utility Commission, “Section 1.14 Transmission and Distribution System Losses,” Technical Reference Manual; State of Pennsylvania Act 129 Energy Efficiency and Conservation Program & Act 213 Alternative Energy Portfolio Standards, dated June 2016, errata update February 2017.



Table 2-15. List of Fuel Switching Measures

Fuel Switching Measures Implemented in PY9

Electric clothes dryer to ENERGY STAR gas clothes dryer

Electric range to gas range

Electric baseboard to ENERGY STAR fossil fuel furnace

Electric furnace to ENERGY STAR fossil fuel furnace

Electric water heater to ENERGY STAR gas water heater

CHP Source: Navigant analysis

Table 2-16. Summary of Fuel Switching Measure Portfolio Impacts

Parameter PYVTD VTD

Total No. of Units Implemented 196 573

Gross Energy Savings via Fuel Switching (MWh/yr) 16,292 21,212

Fossil Fuel Consumption Change (MMBtu/yr) 156,302 176,653

P3TD Incentive Spending ($1,000) $939 $1,272 Source: Navigant analysis

2.10 Summary of Cost-Effectiveness Results

A detailed breakdown of portfolio finances and cost-effectiveness is presented in Table 2-17. TRC benefits in Table 2-17 were calculated using gross verified impacts. Net present value (NPV) PY10 costs and benefits are expressed in 2018 dollars. NPV costs and benefits for P3TD financials are expressed in 2016 dollars.

Table 2-17. Summary of Portfolio Finances – Gross Verified

Category Parameter PYTD ($1,000) P3TD ($1,000)

NPV of Incremental Measure Costs ($1,000)

EDC Incentives to Participants [1] $28,224 $64,744 EDC Incentives to Trade Allies $0 $0

Participant Costs (Net of Incentives/Rebates Paid by Utilities) $109,356 $164,932

Cost Subtotal $137,580 $229,676

NPV of Program Overhead Costs ($1,000)

Design and Development (EDC Costs) [2] $0 $0 Design and Development (CSP Costs) [2] $0 $0

Administration, Management, and Technical Assistance (EDC Costs) [3] $809 $2,300

Administration, Management, and Technical Assistance (CSP Costs) [3] $0 $0

Marketing (EDC Costs) [4] $8,551 $22,882 Marketing (CSP Costs) [4] $0 $0




Program Delivery (EDC Costs) [5] $9,616 $26,774 Program Delivery (CSP Costs) [5] $26,584 $63,941 EDC Evaluation Costs $0 $0 SWE Audit Costs $0 $0 Cost Subtotal $45,561 $115,897

NPV of Fossil Fuel Impacts from Fuel Switching ($1,000)

Increased Fossil Fuel Consumption $7,208 $7,078

Cost Subtotal $7,208 $7,078

Total NPV of Costs [6] ($1,000) Cost Total $190,350 $352,651

Total NPV of Benefits [7] ($1,000)

Lifetime Electric Energy Benefits $107,640 $226,449 Lifetime Electric Capacity Benefits $52,650 $103,157

Lifetime Non-Electric Benefits (Fossil Fuel, Water, Operations & Maintenance [O&M]) $84,483 $104,602

Benefits Total $244,772 $434,209

TRC Benefit-Cost Ratio [8] Benefits Total/Costs Total 1.29 1.23

[1] Includes direct install equipment costs and costs for EE&C kits. [2] Includes direct costs attributable to plan and advance the programs. [3] Includes rebate processing, tracking system, general administration, program management, general management, legal, and technical assistance. [4] Includes the marketing CSP and marketing costs by program CSPs. EDC marketing costs broken out as a percentage of sector lifetime savings. This is an adjustment from the Preliminary Annual Report. [5] Direct program implementation costs. Labor, fuel, and vehicle operation costs for appliance recycling and direct install programs. EDC program delivery costs include crosscutting portfolio costs. [6] Total TRC Costs includes Total EDC Costs and Participant Costs. [7] Total TRC Benefits equals the sum of Total Lifetime Electric and Non-Electric Benefits. Benefits include avoided supply costs, including the reduction in costs of electric energy, generation, transmission, and distribution capacity, and natural gas valued at marginal cost for periods when there is a load reduction. NOTE: Savings carried over from Phase II are not to be included as a part of Total TRC Benefits for Phase III. [8] TRC Ratio equals Total NPV TRC Benefits divided by Total NPV TRC Costs.


TRC benefit-cost ratios are calculated by comparing total NPV TRC benefits and total NPV TRC costs. TRC costs are materially different from the EDC spending and cost recovery tables presented in Section 4. TRC costs include estimates of the full cost incurred by program participants to install efficient equipment—not just the portion covered by the EDC rebate. Table 2-18 through Table 2-21 show the gross and net TRC ratios by program and for the portfolio. The Navigant team calculated the benefits using gross and net verified impacts, where appropriate. Costs and benefits for PY10 results are expressed in 2018 dollars, while Phase III to date values are expressed as an NPV in 2016 dollars using a discount rate of 7.6%.



Table 2-18. Summary PY10 Gross TRC Results by Program ($1,000) [1]

Program TRC NPV Benefits

TRC NPV Costs TRC Ratio TRC Net Benefits

(Benefits – Costs)

Residential EE $145,402 $89,877 1.62 $55,525

Low-Income EE $10,376 $7,736 1.34 $2,640

Small C&I EE $23,795 $22,920 1.04 $875

Large C&I EE $40,747 $42,116 0.97 -$1,368

CHP $8,356 $9,640 0.87 -$1,283

Residential DR $2,905 $3,363 0.86 -$458

Small C&I DR $57 $122 0.47 -$65

Large C&I DR $13,133 $4,960 2.65 $8,173

Portfolio Total [2] $244,772 $190,350 1.29 $54,422 [1] Costs and benefits are expressed as follows: PY8 = 2016, PY9 = 2017, PY10 = 2018, PY11 = 2019, PY12 = 2020. [2] The portfolio total benefits include crosscutting costs. Source: Navigant analysis

Table 2-19. Summary PY10 Net TRC Results by Program ($1,000) [1]




Residential EE $75,270 $53,867 1.40 $21,403

Low-Income EE $10,376 $7,736 1.34 $2,640

Small C&I EE $18,092 $17,900 1.01 $191

Large C&I EE $29,923 $31,915 0.94 -$1,993

CHP $7,297 $8,424 0.87 -$1,126

Residential DR $2,905 $3,363 0.86 -$458

Small C&I DR $57 $122 0.47 -$65

Large C&I DR $13,133 $4,960 2.65 $8,173


Table 2-20. Summary P3TD Gross TRC Results by Program ($1,000) [1]




Residential EE $246,580 $138,700 1.78 $107,880

Low-Income EE $25,617 $22,898 1.12 $2,719

Small C&I EE $46,092 $48,579 0.95 -$2,487

Large C&I EE $80,848 $83,111 0.97 -$2,263

CHP $9,029 $15,701 0.58 -$6,672

Residential DR $5,280 $9,072 0.58 -$3,792






Small C&I DR $49 $324 0.15 -$275

Large C&I DR $20,715 $7,493 2.76 $13,222


Table 2-21. Summary P3TD Net TRC Results by Program ($1,000) [1]




Residential EE $129,436 $93,282 1.39 $36,155

Low-Income EE $25,617 $22,898 1.12 $2,719

Small C&I EE $34,689 $38,617 0.90 -$3,929

Large C&I EE $59,056 $64,498 0.92 -$5,442

CHP $7,915 $13,844 0.57 -$5,929

Residential DR $5,280 $9,072 0.58 -$3,792

Small C&I DR $49 $324 0.15 -$275

Large C&I DR $20,715 $7,493 2.76 $13,222


The previously reported TRCs from PY9 have changed since Navigant submitted the PY9 final annual report. These changes to TRCs relate directly to the SWE’s recommended adjustments for verified energy and demand impacts, previously detailed in Table 2-10 and Table 2-13, respectively. That is, an adjustment to energy and demand impacts results in a change to the TRC’s lifetime benefits. Additionally, Navigant identified several discrepancies with incremental costs in the Residential EE and Large C&I EE Programs and an overarching issue with the application of line losses. Correcting these issues resulted in minor updates to program TRC ratios for PY9:

• Residential EE: Gross TRCs decreased by 0.03 and net TRCs decreased by 0.01.

• Low-Income EE: Gross TRCs increased by 0.04 and net TRCs increased by 0.04.

• Small C&I EE: Gross TRCs increased by 0.01 and net TRCs did not change.

• Large C&I EE: Gross TRCs decreased by 0.04 and net TRCs decreased by 0.02.

For PY8, the adjustment made to line loss application resulted in TRC decreases of 0.01 or less for all programs.



2.11 Comparison of Performance to Approved EE&C Plan

Table 2-22 presents PY10 expenditures, by program, compared to the budget estimates set forth in the EE&C Plan for PY10. All dollar values in Table 2-22 are presented in 2018 dollars.

Table 2-22. Comparison of PY10 Expenditures to Phase III EE&C Plan ($1,000)

Program PY10 Budget from

EE&C Plan PY10 Actual Expenditures

Ratio (Actual/Plan)

Residential EE $19,790 $25,943 1.31

Low-Income EE $7,003 $7,877 1.12

Small C&I EE $9,031 $9,411 1.04

Large C&I EE $10,932 $10,897 1.00

CHP $5,829 $922 0.16

Residential DR $2,799 $4,030 1.44

Small C&I DR $188 $149 0.79

Large C&I DR $6,752 $4,961 0.73

Portfolio Total $62,315 $64,169 1.03 Source: Navigant Analysis

Table 2-23 presents program year and P3TD expenditures by program compared to the budget estimates set forth in the EE&C Plan. All values shown are the sum of nominal dollars.

Table 2-23. Comparison of Expenditures to Phase III EE&C Plan by Program ($1,000)

Program Phase III Budget from EE&C Plan through PY10

P3TD Actual Expenditures

Ratio (Actual/Plan)

Residential EE $59,278 $71,762 1.21

Low-Income EE $20,663 $24,131 1.15

Small C&I EE $27,054 $23,136 0.86

Large C&I EE $32,348 $25,539 0.79

CHP $16,729 $1,176 0.07

Residential DR $7,844 $11,884 1.52

Small C&I DR $561 $437 0.78

Large C&I DR $13,689 $8,740 0.64

Portfolio Total $178,466 $166,805 0.93 Source: Navigant analysis

Table 2-24 compares PY10 verified gross program savings compared to the energy savings projections set forth in the EE&C Plan.



Table 2-24. Comparison of PY10 Actual Program Savings to EE&C Plan Projections for PY10

Program EE&C Plan Projections for PY10 (MWh/yr)

PY10 VTD Gross Savings (MWh/yr)

Ratio (Actual/Plan)

Residential EE 147,191.0 240,570.1 1.63

Low-Income EE 22,600.0 21,409.1 0.95

Small C&I EE 84,818.0 57,224.0 0.67

Large C&I EE 95,678.0 92,651.9 0.97

CHP 85,057.0 15,916.2 0.19

Portfolio Total 435,346.0 427,771.4 0.98 Source: Navigant analysis

Table 2-25 compares Phase III verified gross program savings to the energy savings projections filed in the EE&C Plan.

Table 2-25. Comparison of Energy Savings to Phase III EE&C Plan by Program

Program EE&C Plan through PY10 (MWh/yr)

VTD Gross Savings (MWh/yr)

Ratio (Actual/Plan)

Residential EE 417,603 622,982 1.49

Low-Income EE 67,926 67,129 0.99

Small C&I EE 238,651 121,214 0.51

Large C&I EE 286,334 197,662 0.69

CHP 245,573 19,624 0.08

Portfolio Total 1,256,087 1,028,611 0.82 Source: Navigant analysis

The list below briefly discusses key reasons why programs exceeded or fell short of projected gross energy savings in PY10.

• For the third year of Phase III, the Residential EE Program exceeded its annual projections specified in the PECO EE&C Plan. Similar to previous years, the Lighting component of the Lighting, Appliances & HVAC Solution, including lighting sold through the PECO Marketplace, and the Behavior Solution were the significant contributors to the portfolio. The remaining Residential EE Program solutions represent approximately 17% of PY10 gross verified energy savings achievements. PECO continues to explore ways to increase residential participation levels, such as promoting the PECO Marketplace for online sales. This program is discussed in Section 3.1 of this report.

• The Low-Income EE Program attained most of its planned savings in PY10 and to-date for the phase; the program is nearly meeting EE&C Plan targets. PECO discontinued the Low-Income Lighting offering prior to PY9, so the Low-Income Whole Home Solution is the sole contributing solution to the Low-Income EE Program. Since that change, PECO has adjusted implementation tactics to increase Whole Home Solution savings, including expanding partnerships with other utility and income-eligible programs to identify and comprehensively serve households. PECO also expanded the lighting giveaway component of the Low-Income Whole Home Solution. This program is discussed in Section 3.2 of this report.



• The Small and Large C&I EE Programs each fell short of planned savings in PY10. However, from PY8 through PY10, savings achievements have increased each year as the programs continue to ramp up activities. For example, the Large C&I EE Program nearly met its PY10 savings goals specified in the EE&C Plan. The Equipment and Systems Solution, across the Small and Large C&I EE Programs, also saw significant savings in PY10 and represents roughly 29% of the portfolio’s overall achievements. These programs are discussed in Sections 3.3 and 3.4 of this report.

• The CHP Program continued to ramp up savings through the third year of Phase III; the program reported savings for three projects in PY10. To-date, five projects have contributed savings to Phase III. Navigant conducted a participant-focused process evaluation for CHP in PY10, and findings from that effort are informing recommendations and ongoing program improvement planning. This program is discussed in Section 3.5 of this report.

• PECO’s average DR performance to-date is 173.12 MW, which exceeds the Phase III compliance reduction target of 161 MW by 8% (108% of target achieved to-date). All six DR events in PY10 exceeded the 161 MW curtailment target. This was due to higher performance in the Large C&I DR Program compared to PY9. PECO’s DR programs are discussed in Section 3.6 of this report.

Navigant and PECO are working together to conduct targeted process evaluation activities on an ongoing basis to identify potential changes to the Phase III programs and to support Phase IV portfolio design. There are no official significant program changes to report at this time; however, Navigant has made program-specific recommendations, which are discussed in subsequent sections of this report.



2.12 Findings and Recommendations

The PY10 impact and process evaluation activities Navigant completed led to a variety of recommendations for program improvement. Table 2-26 shows the overarching recommendations that affect more than one program, the evaluation activity(s) that uncovered the finding, and Navigant’s recommendation(s) to PECO to address the finding.

Table 2-26. Summary of Evaluation Recommendations

Evaluation Activity Finding Recommendation

Progress Toward Goals

PECO has achieved 52.4% of the energy savings compliance target to date. The PY10 verified gross savings of 427,771 MWh are 10% greater than PY9 savings, and PY10 savings are more than double PY8 savings. The Residential Program contributed 56% of the savings in PY10, the Low-Income Program 5%, the Small C&I Program 13%, the Large C&I Program 22%, and the CHP Program 4%.

Program managers and CSPs have made several implementation changes to try to increase program participation. For example and particularly for C&I customers, these changes include expanding the number of distributors participating in the PECO Instant Lighting Discounts (PILD) offering in PY10, increasing incentives for select lighting and motor and drive measures, hiring additional program outreach staff, creating an express application process that allows participants to submit certain documentation after project completion, creating a user-friendly Excel form to help participants complete Appendix C from the Pennsylvania Technical Reference Manual16 (PA TRM or TRM), providing bonus incentives to trade allies for installing qualified projects through limited-time offers, and extending the window to submit an application for a qualified project to 180 days following project completion.

PECO should continue to track program performance against the Phase III compliance target. Navigant will assess the impact of these described changes and others on customer participation and the customer experience as part of the PY11 activities.

16 Pennsylvania Public Utility Commission, Technical Reference Manual; State of Pennsylvania Act 129 Energy Efficiency and Conservation Program & Act 213 Alternative Energy Portfolio Standards, dated June 2016, errata update February 2017.



Evaluation Activity Finding Recommendation

Program Tracking Database

Navigant identified some inconsistencies and limitations to the eTrack database. Most of the inconsistencies pertain to data accuracy, discrepancies, and completeness. For example, Navigant identified instances where the same customer was identified as a multifamily customer for one measure and a single-family customer for another measure.

Similarly, certain data is excluded from the eTrack database that would simplify evaluation activities. For example, there is no unique identifier in eTrack data that can help identify all the projects in a single multifamily complex that have the same decision maker.

Based on these findings, PECO held a session with its program managers to highlight fields that may include data inconsistencies and require additional checks. In addition, PECO started working with ANB Systems Inc. to add more data validation checks on the data before it is accepted into eTrack.

PECO should continue to work with its data management CSP to add more data validation checks into eTrack to only accept data that is complete and does not include contradictory information.




3. EVALUATION RESULTS BY PROGRAM This section documents the gross impact, net impact, and process evaluation activities conducted in PY10 along with the outcomes of those activities. Not every program receives an evaluation every program year. Table 3-1 shows a breakdown of the evaluation activity plan, with a check indicating the type of evaluation Navigant will conduct for each program over each year.

Table 3-1. Evaluation Activity Matrix

Program Solution PY8 PY9 PY10 PY11 PY12

Gross Net Process Gross Net Process Gross Net Process Gross Net Process Gross Net Process

Residential EE

Lighting, Appliances & HVAC √ √ √ √ √ √ √ √ √ √ √ √

Appliance Recycling √ √ √ √ √ √ √ √ √ √ √

Whole Home √ √ √ √ √ √ √ √

New Construction √ √ √ √ √ √ √ √

Multifamily Targeted Market Segment √ √ √ √ √ √ √ √ √ √

Behavioral √ √ √ √ √ √ √

Residential Low-Income EE

Whole Home √ √ √ √ √ √

Lighting √ √

Small C&I EE

Equipment and Systems √ √ √ √ √ √ √ √ √


Whole Building √ √ √ √ √ √

Behavioral √ √ √

Data Centers Targeted Market Segment √ √ √ √ √ √ √ √ √


Large C&I EE Equipment and Systems √ √ √ √ √ √ √ √ √




Program Solution PY8 PY9 PY10 PY11 PY12

Gross Net Process Gross Net Process Gross Net Process Gross Net Process Gross Net Process

Data Centers Targeted Market Segment √ √ √ √ √ √ √ √ √


CHP Combined Heat and Power √ √ √ √ √ √ √ √ √ √ √ √

DR

Residential DR √ √ √ √ √ √ √ √ √ √ √

Small C&I DR √ √ √ √ √ √ √ √ √ √ √

Large C&I DR √ √ √ √ √ √ √ √ √




3.1 Residential EE Program

The PECO Residential EE Program is designed to offer residential customers opportunities to save energy across all their electric end uses and to market those opportunities in ways that minimize lost savings opportunities. The program encompasses a comprehensive series of solutions designed to influence customer behavior and purchasing decisions. The Residential EE Program represents more than half of PECO’s PY10 portfolio reported energy savings and consists of six solutions, or initiatives, that contribute to those savings. Savings are achieved through a range of delivery mechanisms and methods including upstream incentives (e.g., manufacturer buy-downs), downstream incentives (e.g., mail-in rebates), appliance removal and recycling, in-home audits, direct install measures, efficient building construction, and changes in household behaviors. PECO relies on five CSPs to deliver the program, listed here with its corresponding solution:

• Lighting, Appliances & HVAC Solution – CLEAResult

• Appliance Recycling Solution – ARCA

• Whole Home Solution – CLEAResult

• New Construction Solution – Performance Systems Development (PSD)

• Behavioral Solution – Oracle

• Multifamily Targeted Market Segment – Franklin Marketing for the six solutions in the Residential EE Program is handled through a separate energy efficiency marketing firm (EEMF), ICF. ICF markets PECO’s range of Residential EE Program offerings delivered through the six solutions with consistent approaches and messaging. Marketing from a crosscutting perspective is intended to promote all savings opportunities available to residential customers.

3.1.1 Participation and Reported Savings by Customer Segment

This section provides the total Residential EE Program results for PY10, including participation, energy and demand savings, and incentive costs. Table 3-2 presents the participation counts and incentive payments for the Residential EE Program in PY10 by customer segment.

Table 3-2. Residential EE Program Summary by Customer Segment

Parameter Residential Small C&I Large C&I

PYTD No. of Participants 1,603,720 220 119

PYRTD MWh/yr* 241,816.8 238.5 111.4

PYRTD MW 22.34 0.04 0.01

PY10 Incentives ($1,000) $8,750 $23 $9 * The residential sector PYRTD was updated from the PY10 semi-annual report as a result of a change in reported behavioral savings. In addition, the residential program small C&I and large C&I changed slightly, because the small C&I and large C&I lighting, appliance and HVAC solution savings were flipped. Source: Navigant analysis



3.1.2 Gross Impact Evaluation

The Residential EE Program’s gross impact evaluation activities involved different approaches tailored to each solution’s characteristics to verify the reported gross savings values for PY10. First, Navigant reviewed each solution’s program tracking data to verify proper application of the PA TRM17 algorithms in reported savings values. The evaluation team completed these reviews for the full population of PY10’s implemented Residential EE Program measures. Next, the team identified appropriate evaluation activities for each solution depending on the nature of the participants, implementation, and the level of information accompanying the reported savings. A goal of the evaluation activities was to verify a given measure was implemented. Additionally, for partially deemed measures, the evaluation verified certain measure characteristics that inform gross energy and demand impact estimations—in particular, those characteristics where the PA TRM does not provide default or deemed values for EDCs to use for savings estimations. The evaluation activities varied for each solution and for specific strata within some solutions. Activities included engineering file reviews of program applications and invoices, participant phone verifications, onsite field verifications, billing and regression analyses, or a combination of these activities. Onsite verification field activities occurred for the Whole Home Solution. Master-metered multifamily projects and individually metered residential projects within the Multifamily Targeted Market Segment also received onsite visits for verification. Navigant drew samples from each solution for gross impact evaluation activities according to the sampling plans. The team developed and sought approval for representative samples that complied with the Phase III Evaluation Framework, the PA TRM, and industry standards, as well as those that helped PECO meet SWE and Commission requirements. The following provides a summary of the activities conducted for each solution and for specific components or sampled strata within a given solution. Appendix E contains additional detail on the gross impact evaluation approaches used for the Residential EE Program’s individual solutions.

• Lighting, Appliances & HVAC Solution o Lighting

Invoice reviews and record-level savings calculation Verification of ENERGY STAR certification

o Appliances and HVAC Engineering file reviews and phone verification (for HVAC measures) Phone verification (for Appliance measures)

• Appliance Recycling Solution o Phone verification o Regression analysis

• Whole Home Solution o Engineering file reviews and onsite verification (large and medium projects) o Engineering file reviews and phone verification (small projects)

• New Construction Solution o Engineering file review and building simulation modeling




• Behavioral Solution o Billing analysis

For home energy report (HER) participants in the test and control groups, used a lagged dependent variable (LDV) model

For AC Saver cohort recipients included in the Behavioral Solution, used a regression with pre-program matching method to estimate savings related to HERs

o Peak demand impacts Accounting for average peak demand impacts resulting from HERs18

o Double counting (dual participation) analysis Accounting for Behavioral Solution participant activities within other PECO EE

solutions • Multifamily Targeted Market Segment

o Engineering file reviews and onsite verification (residential projects that contribute to the Residential EE Program)

Table 3-3 provides the sampling frame for the gross impact evaluation of the Residential EE Program in PY10.

Table 3-3. Residential EE Program Gross Impact Sample Design for PY10

Solution Stratum Name

Population Size

Targeted Sample

Size

Achieved Sample

Size Verification Method


Appliances 13,628 28 28 Engineering file review and phone verification

HVAC 13,272 25 25 Engineering file review and phone verification

Standard LED 676,963 676,963 676,963 PY8 analysis and census of tracking system review

Specialty LED 472,355 472,355 472,355 PY8 analysis and census of tracking system review

Marketplace 10,216 10,216 10,216 PY8 analysis and census of tracking system review

Solution Total 1,186,434 1,159,587 1,159,587

Appliance Recycling

Refrigerators 15,013 150 170 File review and phone verification survey

Freezers 2,563 30 39 File review and phone verification survey

Room Air Conditioners (ACs)

2,223 20 43 File review and phone verification survey

Solution Total 19,799 200 252

18The Behavioral Solution implementer, Oracle, does not report demand savings. Navigant completes this analysis as part of its annual reporting, as required by the SWE. A realization rate cannot be calculated due to the lack of reported demand savings.




Population Size

Targeted Sample

Size

Achieved Sample


Whole Home

Large Projects 866 7 7 Engineering file review with onsite verification

Medium Projects 1,348 7 7 Engineering file review with onsite

verification

Small Projects 3,133 6 6 Engineering file review with phone verification

Very Small Projects 1,131 0 0 Applied small projects’ realization

rate Solution Total 6,478 20 20

New Construction

Large 123 4 4 File review and building energy simulation

Medium 217 4 4 File review and building energy simulation

Small 332 4 4 File review and building energy simulation

Solution Total 672 12 12

Behavioral Solution Total 0 N/A N/A


Large Residential 51 5 4 File review and onsite verification

Small Residential 3,327 6 6 File review and onsite verification

Multisector 19 6 5 File review and onsite verification


Total Program All 1,216,780 1,159,836 1,159,886


Navigant shows the PY10 verified energy and demand savings in Table 3-4 and Table 3-5, respectively. These tables also include the realization rates on a stratum and solution level, which is calculated by dividing the total verified savings by the reported savings. Table 3-4 summarizes the reported and verified energy savings results along with the coefficient of variation (CV) and relative precision for each stratum sampled for the Residential EE Program in PY10.



Table 3-4. Residential EE Program Gross Results for Energy


Reported Gross Energy Savings (MWh/yr)

Verified Gross Energy Savings (MWh/yr)

Energy Realization

Rate

Achieved Sample CV or Error Ratio


85% Confidence

Interval


Appliances 1,588 1,742 1.10 0.00 0.0% HVAC 5,314 5,332 1.00 0.11 1.4% Standard LED 69,227 69,771 1.01 0.00 0.0% Specialty LED 58,112 60,097 1.03 0.00 0.0% Marketplace 3,398 3,389 1.00 0.00 0.0% Solution Total 137,639 140,330 1.02 0.39 0.1%

Appliance Recycling

Refrigerators 14,373 15,712 1.09 0.28 3.1% Freezers 2,212 2,106 0.95 0.30 7.1% Room ACs 353 353 1.00 0.00 0.0% Solution Total 16,939 18,171 1.07 0.31 2.8%

Whole Home*

Large Projects 2,670 2,391 0.90 0.13 7.8% Medium Projects 2,174 2,034 0.94 0.07 4.1% Small Projects 2,548 2,057 0.81 0.47 32.9% Very Small Projects 224 181 0.81 N/A 100.0% Solution Total 7,616 6,663 0.87 0.29 9.8%

New Construction

Large 627 661 1.05 0.12 11.4% Medium 623 703 1.13 0.25 23.6% Small 622 582 0.93 0.17 16.1% Solution Total 1,872 1,946 1.04 0.19 8.5%

Behavioral Solution Total 73,716 69,127 0.94 0.00 0.0%


Large Residential 2,033 1,906 0.94 1.18 113.8% Small Residential 1,785 1,866 1.04 0.13 8.8% Multisector 566 561 0.99 0.06 4.9% Solution Total 4,384 4,333 0.99 1.01 39.8%

Total Program All 242,167 240,570 0.99 5.68 0.87% [90% CI]




Table 3-5 summarizes the reported and verified demand savings results along with the CV and relative precision for each stratum sampled for the Residential EE Program in PY10.

Table 3-5. Residential EE Program Gross Results for Demand


Reported Gross

Demand Savings

(MW)

Verified Gross

Demand Savings

(MW)

Demand Realization

Rate



85% Confidence

Interval


Appliances 0.31 0.32 1.06 0.00 0.0% HVAC 2.38 2.33 0.98 0.31 4.1% Standard LED 8.16 8.30 1.02 0.00 0.0% Specialty LED 6.85 7.31 1.07 0.00 0.0% Marketplace 0.19 0.19 0.99 0.00 0.0% Solution Total 17.88 18.45 1.03 3.76 0.5%

Appliance Recycling

Refrigerators 1.66 1.76 1.06 0.28 3.1% Freezers 0.25 0.24 0.94 0.30 7.1% Room ACs 0.58 0.58 1.00 0.00 0.0% Solution Total 2.49 2.57 1.03 0.25 2.2%

Whole Home

Large Projects 0.26 0.23 0.92 0.12 7.5% Medium Projects 0.32 0.32 0.98 0.05 3.2% Small Projects 0.30 0.24 0.81 0.49 33.7% Very Small Projects 0.05 0.04 0.81 N/A 100.0% Solution Total 0.92 0.83 0.90 0.30 10.1%

New Construction

Large 0.17 0.18 1.03 0.09 8.7% Medium 0.20 0.20 1.04 0.11 10.4% Small 0.18 0.17 0.99 0.12 11.2% Solution Total 0.54 0.55 1.02 0.11 4.7%

Behavior Solution Total 0.00 7.89 -* 0.00 0.0%


Large Residential 0.26 0.24 0.93 1.05 101.4% Small Residential 0.22 0.22 1.00 0.09 6.2% Multisector 0.08 0.07 0.95 0.06 4.6% Solution Total 0.56 0.54 0.96 0.92 36.3%

Total Program All 22.39 30.83 1.38 5.59 0.9% * For the Residential Behavioral Solution, the implementer does not report demand savings; however, the SWE requires PECO to verify demand savings. As a result, there is no demand realization rate for the Behavioral Solution. However, the verified demand savings are added to the Residential EE Program savings. As a result, the demand realization rate for the Residential EE Program is greater than the demand realization rate for each individual solution in the program. Source: Navigant analysis

Factors leading to variations between the reported and verified savings and the observed realization rates for the Residential EE Program are detailed in Appendix E for each solution. Overall, the Lighting, Appliances & HVAC Solution and the Behavioral Solution are the most significant drivers of the program-level results and realization rates, as these two solutions represent almost 90% of the program’s savings.



3.1.3 Net Impact Evaluation

The Residential EE Program’s net impact evaluation uses several methods to estimate free ridership, spillover, market effects, and NTG ratios. Navigant relied on consistent, crosscutting approaches as well as ones tailored to certain solutions’ characteristics. The primary objective of the net savings analysis is to determine the program's net effect on customer electricity usage. The evaluation team derives net program impacts by estimating an NTG ratio that quantifies the percentage of the gross program impacts that can reliably be attributed to the program. The team conducted primary NTG research through surveys of residential tenants participating in the Multifamily Targeted Market Segment in PY10. Free ridership is defined as those participants who would have implemented a measure or purchased equipment anyway, without program support or a rebate. The questions determining free ridership focus on the influence of key program interventions and the customer’s perception of what they would most likely have done in the absence of the program. Interventions vary by solution but can include discounted prices, program information regarding efficient products, and placement of program-discounted products in stores. Spillover is defined as those participants who were influenced by the program to purchase and install additional energy efficient equipment that saves electricity without a rebate or other program support. Navigant analyzed participant responses to a battery of spillover questions. The intent of these questions was to identify what types and amounts of equipment customers purchased and installed on their own to inform a quantitative estimate of program spillover within the overall NTG calculation. Market effects represent a change in the structure of a market or the behavior of participants in a market that is reflective of an increase in the adoption of EE products, services, or practices and is causally related to market intervention(s). Navigant surveyed PECO program participants over the phone to gather information about free ridership and spillover. The evaluation team developed survey instruments consistent with the Phase III Evaluation Framework’s guidance on net impact evaluation techniques19 and guidance from the Uniform Methods Project on estimating net savings.20 Survey instruments also captured feedback about customer experiences from participants to inform the process evaluation.21 Table 3-6 provides the sampling frame for the net impact evaluation of the Residential EE Program in PY10, where sampling occurred. Navigant experienced low response rates from the Multisector population in PY10. To improve response, the team conducted additional Large – Residential surveys because the ownership structure of the buildings in those two strata are similar and most of the properties are rental units.

19 Pennsylvania PUC. Phase III Evaluation Framework. Section 3.4. http://www.puc.pa.gov/Electric/pdf/Act129/SWE_PhaseIII-Evaluation_Framework102616.pdf 20 The Uniform Methods Project. Estimating Net Savings: Common Practices. NREL. https://www.nrel.gov/docs/fy14osti/62678.pdf 21 Navigant screened survey participants and only collected NTG data from tenants who reported having a choice to participate in the Multifamily Targeted Market Segment. Of the 65 survey participants, 15 provided NTG results.

http://www.puc.pa.gov/Electric/pdf/Act129/SWE_PhaseIII-Evaluation_Framework102616.pdf


https://www.nrel.gov/docs/fy14osti/62678.pdf



Table 3-6. Residential EE Program Net Impact and Process Survey Sample Design for PY10

Solution Stratum Name and Boundaries

Population Size

Target Sample Size

Achieved Sample

Size*

Response Rate

Verification Method

Multifamily Targeted Market Segment (tenants)

Large – Residential 4,262 18 33 (36) 0.8% Phone survey

Small – Residential 2,931 30 22 (24) 0.8% Phone survey

Multisector – Res and C&I 1,341 26 5 0.4% Phone survey

Solution Total 8,534 74 60 (65) 0.7% *The achieved sample targets listed are for participants who answered NTG related questions; the sample targets achieved in parentheses are for the number of participants who answered process evaluation questions. Source: Navigant analysis.

Table 3-7 summarizes the reported and verified energy savings results, the calculated NTG results, and the CV and relative precision for each stratum sampled for the Residential EE Program in PY10.



Table 3-7. Residential EE Program Net Energy Savings Impact Evaluation Results for PY10

Solution Stratum Name Verified Gross

Energy Savings (MWh/yr)

Verified Net Energy Savings (MWh/yr)

Free Ridership

Rate Spillover

Rate NTG

Ratio* Achieved

Sample CV or Error Ratio


85% Confidence

Interval


Appliances 1,742 1,143 0.55 0.21 0.66 1.72 28.9% HVAC 5,332 3,006 0.46 0.03 0.56 0.28 4.7% Standard LED 69,771 35,578 0.53 0.04 0.51 0.45 3.4% Specialty LED 60,097 27,595 0.58 0.04 0.46 0.40 2.6% Marketplace 3,389 1,783 0.51 0.03 0.53 N/A 24.9% Solution Total 140,330 69,104 0.54 0.04 0.49 0.44 2.0%

Appliance Recycling

Refrigerators 15,712 5,449 0.65 0.00 0.35 1.41 15.9% Freezers 2,106 1,045 0.50 0.00 0.50 1.00 21.3% Room ACs 353 164 0.54 0.00 0.46 1.08 53.5% Solution Total 18,171 6,657 0.63 0.00 0.37 1.37 13.3%

Whole Home

Large Projects 2,391 2,146 0.12 0.01 0.90 0.12 5.3% Medium Projects 2,034 1,725 0.16 0.01 0.85 0.19 5.6% Small Projects 2,057 1,822 0.19 0.08 0.89 0.36 6.3% Very Small Projects 181 160 0.19 0.08 0.89 N/A 100.0% Solution Total 6,663 5,853 0.16 0.04 0.88 0.31 4.3%

New Construction

Large 661 573 0.13 0.00 0.87 N/A N/A Medium 703 609 0.13 0.00 0.87 N/A N/A Small 582 504 0.13 0.00 0.87 N/A N/A Solution Total 1,946 1,686 0.13 0.00 0.87 0.48 14.0%

Behavioral Solution Total 69,127 69,127 0.00 0.00 1.00 N/A N/A



Solution Stratum Name Verified Gross



Free Ridership

Rate Spillover

Rate NTG

Ratio* Achieved

Sample CV or Error Ratio


85% Confidence

Interval


Large Residential 1,906 1,775 0.07 0.00 0.93 0.18 4.6% Small Residential 1,866 1,722 0.08 0.00 0.92 0.23 7.2% Multisector 561 490 0.13 0.00 0.87 0.20 15.7% Solution Total 4,333 3,986 0.08 0.00 0.92 0.21 3.9%

Total Program All 240,570 156,414 0.37 0.02 0.65 0.27 1.2%

[90% CI] * Navigant conducted NTG research in PY10 for the Residential Multifamily Targeted Market Segment. For the other residential solutions, Navigant applied stratum-level NTG results from PY9. Source: Navigant analysis



Navigant compared the PY10 NTG results for the Residential Multifamily Targeted Market Segment to the results calculated in PY8. Each sector strata showed a decrease in free ridership, likely caused by the increased offering of prescriptive measures through the program. Tracking data shows an increase in prescriptive lighting fixture upgrades in the Large – Residential and C&I sectors, which likely would not occur without program assistance.

3.1.3.1 High Impact Measure Research

HIMs represent measure categories or technologies of high importance in the PECO portfolio. In Phase III, the SWE suggested EDCs oversample HIMs to help program planners make decisions concerning those measures for downstream programs only.22 EDCs were to identify three to five measures for study within each program year based on energy impact, level of uncertainty, prospective value, funding, or other parameters. Navigant reviewed program- and solution-level savings, energy impact, and overall value to PECO to identify the HIMs for PY10. The results indicated that ENERGY STAR-qualified LED bulbs, low flow faucet aerators, and low flow showerheads held the highest impact for the Multifamily Targeted Market Segment. Table 3-8 shows the results of the HIM analysis in PY10.

Table 3-8. Residential EE Program HIM NTG Summary

Solution HIM Free

Ridership Rate

Spillover Rate NTG Ratio


ENERGY STAR LED 0.08 0.00 0.92

Low Flow Faucet Aerator 0.03 0.00 0.97

Low Flow Showerhead 0.06 0.00 0.94 Source: Navigant analysis

3.1.4 Process Evaluation

As described in the Phase III Evaluation Plan23 updated for PY10 and approved by the SWE, Navigant did not complete any in-depth process evaluation activities for the Lighting, Appliances & HVAC, Appliance Recycling, Whole Home, New Construction, and Behavioral Solutions. Instead, the team interviewed key PECO and CSP staff and reviewed program databases and tracking systems across all solutions. The team carried out in-depth process evaluations in PY9 and plans to conduct additional activities in PY11. The PY10 process evaluation included participant surveys for the Residential Multifamily Targeted Market Segment. Participants in this segment received telephone surveys to collect their feedback on a series of questions designed to gauge customer satisfaction, program channeling efforts, and to inform the NTG analysis. This section summarizes the evaluation methods, data collection techniques, sample design, and key results related to these PY10 activities. 22 Pennsylvania PUC. Phase III Evaluation Framework. Section 3.4.1.4. http://www.puc.pa.gov/Electric/pdf/Act129/SWE_PhaseIII-Evaluation_Framework102616.pdf 23 PECO. Phase III Evaluation Plan, Energy Efficiency and Conservation Portfolio. Revised February 15, 2019.





Navigant developed all survey instruments according to SWE requirements and had the SWE review and approve each in advance of fielding.24 In general, the evaluation team defined the survey population for each solution’s participants based on the program tracking databases provided by PECO. In some cases, demographic and geographic information, data on installed measures, installation dates, and estimated savings were used for sample design and the subsequent analysis of results. The team developed a sample sufficient to provide 85/15 confidence/precision for the survey results. The following provides a summary of the process evaluation activities conducted for each Residential EE solution.

• Lighting, Appliances & HVAC Solution o Lighting

PECO and CSP staff interviews Program tracking data review

o Appliances and HVAC components PECO and CSP staff interviews Program tracking data review

• Appliance Recycling Solution o PECO and CSP staff interviews o Program tracking data review

• Whole Home Solution o PECO and CSP staff interviews o Program tracking data review

• New Construction Solution o PECO and CSP staff interviews o Program tracking data review

• Behavioral Solution o PECO and CSP staff interviews o Program tracking data review

• Multifamily Targeted Market Segment (tenant focus) o PECO and CSP staff interviews o Program tracking data review o Phone survey: Navigant used phone surveys to assess how condo owners and building

tenants in residentially metered apartment units heard about the Multifamily Targeted Market Segment, their satisfaction with the program, and ways to improve the program overall. The survey sample was stratified by project size (large, multisector, and small).

See Table 3-6 in Section 3.1.3 for customer survey sample details for the Multifamily Targeted Market Segment.

24 The survey instruments included more questions than are presented in this compliance report. Navigant and PECO are analyzing findings from these additional questions to inform continuous process improvements to the programs, solutions, and customer experience and will include relevant results in subsequent reports as findings and recommendations are formalized.



3.1.4.1 Key Findings from Process Evaluation

For PY10, Navigant surveyed Residential Multifamily Targeted Market Segment participants to measure satisfaction and the customer experience. This section provides results for the Residential Multifamily Target Segment metrics including overall satisfaction and sources of awareness. Appendix H provides additional findings from the Multifamily Targeted Market Segment tenant survey. Multifamily Targeted Market Segment tenants were asked about their overall satisfaction with the solution. Average satisfaction was 4.0 on a 5-point scale, with 1 being Extremely Dissatisfied and 5 being Extremely Satisfied. As shown in Figure 3-1, 51% of tenants reported extreme satisfaction with the program. Participants were pleased with their interactions with the energy advisor and the ability to receive free energy efficiency products through the program. Some Multifamily Targeted Market Segment tenants (11%) reported dissatisfaction with the solution. Similar to PY8, reasons provided for dissatisfaction included the quality of measures installed in tenant units and the inconvenience of the direct installation of those measures in their homes. Two respondents stated needing to replace burnt out LED bulbs too soon, and one was dissatisfied by the reduced water flow from a faucet aerator.

Figure 3-1. Overall Tenant Satisfaction with Multifamily Targeted Market Segment, n=61

Question: “Using a scale of 1 to 5, with 5 meaning extremely satisfied and 1 meaning extremely dissatisfied, how would you rate your OVERALL satisfaction with the [Solution] program?” Note: Refused and Do Not Know responses have been excluded. Source: Navigant analysis

The evaluation team’s examination into key sources of information reveals that communication from landlords or property managers, emails from PECO, and PECO bill inserts play an important role in participants learning about the Multifamily Targeted Market Segment. Figure 3-2 shows that half (50%) of tenants learned about the solution from their landlord or property manager. Multifamily Targeted Market Segment participants stated that emails (20%) and PECO bill inserts received in the mail (17%) were also sources of information in learning more about the program. The influence of word of mouth through family and friends increased slightly over PY8 results (7% compared to 4%) suggesting that tenants may be talking about the program to others more than in prior years.



Figure 3-2. Sources of Tenant Multifamily Targeted Market Segment Awareness, n=56

*PECO Employee includes a PECO account representative and a customer service representative. Question: “How did you learn about the [SOLUTION] program?” Multiple responses allowed; sum of percentages will not add up to 100%. Other includes notification through email or at the time of installation. Source: Navigant analysis

3.1.5 Cost-Effectiveness Reporting

A detailed breakdown of program finances and cost-effectiveness is presented in Table 3-9. Navigant calculated TRC benefits using gross verified impacts. Costs and benefits for PYTD results are expressed in 2018 dollars, while Phase III to date values are expressed as an NPV in 2016 dollars using a discount rate of 7.6%.

Table 3-9. Summary of Residential EE Program Finances – Gross Verified



EDC Incentives to Participants [1] $10,103 $22,234

EDC Incentives to Trade Allies $0 $0


Cost Subtotal $73,895 $94,788 Design and Development (EDC Costs) [2] $0 $0

Design and Development (CSP Costs) [2] $0 $0







Marketing (EDC Costs) [4] $5,294 $14,052

Marketing (CSP Costs) [4] $0 $0

Program Delivery (EDC Costs) [5] $0 $0

Program Delivery (CSP Costs) [5] $10,110 $28,302

EDC Evaluation Costs $0 $0

SWE Audit Costs $0 $0

Cost Subtotal $15,840 $43,524 NPV of Fossil Fuel Impacts from Fuel Switching ($1,000)

Increased Fossil Fuel Consumption $142 $387

Cost Subtotal $142 $387



Lifetime Electric Energy Benefits $47,600 $111,629

Lifetime Electric Capacity Benefits $15,695 $37,782

Lifetime Non-Electric Benefits (Fossil Fuel, Water, O&M) $82,107 $97,168

Benefits Total $145,402 $246,580 TRC Benefit-Cost Ratio [8] Benefits Total/Costs Total 1.62 1.78

[1] Includes direct install equipment costs and costs for EE&C kits. [2] Includes direct costs attributable to plan and advance the programs. [3] Includes rebate processing, tracking system, general administration, program management, general management, legal, and technical assistance. [4] Includes the marketing CSP and marketing costs by program CSPs. EDC marketing costs broken out as a percentage of sector lifetime savings. This is an adjustment from the Preliminary Annual Report. [5] Direct program implementation costs. Labor, fuel, and vehicle operation costs for appliance recycling and direct install programs. [6] Total TRC Costs includes Total EDC Costs and Participant Costs. [7] Total TRC Benefits equals the sum of Total Lifetime Electric and Non-Electric Benefits. Benefits include avoided supply costs, including the reduction in costs of electric energy, generation, transmission, and distribution capacity, and natural gas valued at marginal cost for periods when there is a load reduction. NOTE: Savings carried over from Phase II are not to be included as a part of Total TRC Benefits for Phase III. [8] TRC Ratio equals Total NPV TRC Benefits divided by Total NPV TRC Costs. Source: Navigant analysis



Table 3-10 presents program financials and cost-effectiveness on a net savings basis.

Table 3-10. Summary of Residential EE Program Finances – Net Verified










Marketing (EDC Costs) [4] $5,294 $14,052 Marketing (CSP Costs) [4] $0 $0 Program Delivery (EDC Costs) [5] $0 $0 Program Delivery (CSP Costs) [5] $10,110 $28,302 EDC Evaluation Costs $0 $0 SWE Audit Costs $0 $0 Cost Subtotal $15,840 $43,524



Cost Subtotal $122 $335 Total NPV of Costs [6] ($1,000) Cost Total $53,867 $93,282







[1] Includes direct install equipment costs and costs for EE&C kits. [2] Includes direct costs attributable to plan and advance the programs. [3] Includes rebate processing, tracking system, general administration, program management, general management and legal, and technical assistance. [4] Includes the marketing CSP and marketing costs by program CSPs. EDC marketing costs broken out as a percentage of sector lifetime savings. This is an adjustment from the Preliminary Annual Report. [5] Direct program implementation costs. Labor, fuel, and vehicle operation costs for appliance recycling and direct install programs. [6] Total TRC Costs includes Total EDC Costs and Participant Costs. [7] Total TRC Benefits equals the sum of Total Lifetime Electric and Non-Electric Benefits. Benefits include avoided supply costs,



including the reduction in costs of electric energy, generation, transmission, and distribution capacity, and natural gas valued at marginal cost for periods when there is a load reduction. NOTE: Savings carried over from Phase II are not to be included as a part of Total TRC Benefits for Phase III. [8] TRC Ratio equals Total NPV TRC Benefits divided by Total NPV TRC Costs. Source: Navigant analysis

3.1.6 Status of Recommendations

The impact and process evaluation activities in PY10 led to several findings and recommendations from Navigant to PECO. Table 3-11 presents those solution-level findings and recommendations along with a summary of how PECO plans to address the recommendations in program delivery. Additional details on the solution-level analysis activities that led to these findings and recommendations can be found in Appendix E.



Table 3-11. Summary of Findings and Recommendations for the Residential EE Program

Solution Finding Recommendation EDC Status

Lighting, Appliances & HVAC Solution (Lighting Component)

There is an opportunity to improve the process for verifying program bulb ENERGY STAR certification by incorporating additional bulb information into the tracking system, which does not include the ENERGY Star unique ID. Navigant had a similar finding in PY9.

PECO could consider adding ENERGY STAR unique ID as a field in the lighting data in eTrack. The SWE should also consider updating the data collection guidance to include this field (referring to the Act 129 Quarterly Data Requests for PY8 memo dated October 17, 2016 (or later issued memo)).

Under consideration. PECO will explore adding ENERGY STAR unique ID as an additional field in the lighting data in eTrack. The SWE may also consider updating the data collection guidance to include this field (referring to the Act 129 Quarterly Data Requests for PY8 memo dated October 17, 2016 (or later issued memo)).

Lighting, Appliances & HVAC Solution (Lighting Component)

Navigant found some instances of discrepancies in records. Those discrepancies include the following: 1. Incorrect baseline wattages for some products: Navigant adjusted baseline wattages for several bulbs (105 out of 1,433). 2. A small quantity (39) of records listed pack quantity as bulb quantity in the tracking data. 3.) Eleven records (6 SKUs) reported different baseline wattages than what was used to calculate reported energy and demand savings. The number of discrepancies is lower than Navigant found in PY9.

Navigant can continue to work with PECO and the CSP to refine and improve the accuracy of tracking data, including processes for classifying bulbs per the TRM (e.g., exempt vs. non-exempt), checks to ensure that quantity is equal to SKU sales multiplied by pack size for all records, and reviews to confirm that the baseline wattage indicated in the tracking data matches the value used to calculate energy and demand savings.

Will be implemented. Navigant will continue to work with PECO and the CSP to refine and improve the accuracy of tracking data, including processes for classifying bulbs per the TRM (e.g. "exempt" vs "non-exempt"), checks to ensure that quantity is equal to SKU sales multiplied by pack size for all records, and reviews to confirm that the baseline wattage indicated in the tracking data matches the value used to calculated reported energy and demand savings.

Appliance Recycling

Navigant found a discrepancy in the total number of customers counted as program participants by PECO. The customer flag for multifamily (MF) is being used as a consideration when calculating total participation, which is incorrect. For example, a customer who recycles two appliances in one order may have "MF No" flagged for one appliance and "MF Yes" flagged for another, leading PECO to double count this participant. The MF designation should not be taken into account when calculating total participation.

PECO, with its data management CSP, should update eTrack calculations for counting ARP participants.

Being implemented. PECO is investigating the best approach to develop a solution for tracking the MF participants in its database.




Appliance Recycling

Tracking database savings uses a pre-1990 assumed equation variable in place of EDC-gathered data. The TRM states that the EDC-gathered data is the required source for this variable. Navigant had a similar finding in PY9.

Update eTrack algorithms to calculate refrigerator and freezer savings using EDC-gathered data for this variable.

Being implemented. PECO is working with the data management CSP to utilize the EDC-gathered data for the variable and make an adjustment after the program year as recommended by Navigant.

Whole Home

Savings calculations for five measures were not consistent with TRM algorithms, resulting in discrepancies between reported and verified savings. (Measures: residential air sealing, crawl space insulation, floor insulation, knee wall insulation, ductless mini-split heat pumps.)

Adjust data collection or savings calculation methodologies to align with TRM algorithms.

Being implemented. PECO is in progress of working with EM&V to determine the proper algorithms needed by the CSP to ensure compliance.

New Construction

Navigant identified data discrepancies in the lighting forms. For example, a few Home Energy Rating System (HERS) raters incorrectly documented the count of LED bulbs in the table for CFL bulb counts. The error was corrected based on notation in the REM/Rate model specifying 100% LEDs and confirmed by the CSP’s onsite inspections. Moreover, the lighting form does not capture a bulb count for specialty LED bulbs.

Navigant is working with PECO to propose updates to the lighting forms that will streamline bulb wattage ranges, address specialty LED bulbs, and clearly differentiate tables for CFL and LED bulb counts. Navigant can work with PECO and the CSP to conduct trainings or webinars to educate HERS raters about the importance of accurate data collection, especially in the lighting forms. Navigant will continue to review the information provided by HERS raters in the lighting forms for accuracy and reasonableness.

Implemented.

New Construction Navigant identified several instances where builders installed either a combination of LEDs and CFLs or all CFLs in new construction homes.

PECO can work with the CSP to encourage builders to shift lighting installation away from CFLs to entirely energy efficient LEDs. Installing LEDs will increase energy savings and customer satisfaction.

Being implemented. PECO is in progress of working with the CSP and builders. In preliminary discussions, it appears many of the larger builders have had CFL stock that they are working through.




New Construction

PECO does not collect measure details for new construction appliances including refrigerators, clothes dryers, and clothes washers; this information is necessary for non-weather-sensitive savings calculations. Therefore, Navigant assumed average per-unit savings from PECO’s Lighting, Appliances & HVAC Solution for these appliances.

Navigant is working with PECO and the CSP to propose and develop a new and streamlined appliance form to collect appliance data for new construction projects including appliance quantity, manufacturer, and model number. This data will aid in PY11 and PY12 evaluation activities. Requiring HERS raters to provide only the appliance model number and not the detailed specifications of appliances will save HERS rater's time and cost.

Implemented.

Behavioral Verified savings are almost 10,000 MWh lower in PY10 compared to PY9.

Monitor the newly launched HER wave, using Oracle's monthly savings reports, to ensure savings begin to ramp up by the end of PY11.

Being implemented. PECO will monitor the newly launched HER wave, using Oracle's monthly savings reports, to ensure savings begin to ramp up by the end of PY11.


There is an opportunity to improve the accuracy of HOU and CF assumptions by taking into account the quantity of lamps installed in each unique space type for each lighting project.

Navigant can work with PECO and the CSP to identify an optimal process to track the quantity of lamps installed in each unique space type for each project; this information can be used along with corresponding HOU and CF values deemed in the TRM to develop weighted average assumptions for each project.

Being Implemented. The CSP is currently capturing this information. Next steps are to coordinate the mapping of this data appropriately to enable the CSP to upload it to PECO's tracking system.


The CSP is using the incorrect watts base for BR30 lamps to calculate the energy and demand savings, leading to reduced reported savings.

The CSP should use a watts base of 65 W instead of the 40 W value currently being used. This will make the reported savings more accurate and aligned with the TRM.

Will be implemented. PECO will work with the CSP to correct the watts base.


The eTrack extract excludes the baseline bulb type information. This makes estimating the baseline wattage value challenging. This information is being collected in the project files but is excluded from eTrack databases.

Provide the baseline bulb type (e.g., incandescent, CFL, etc.) information with the eTrack extract. This will help make the savings estimate more accurate.

Being Implemented. The CSP is currently capturing this information. Next steps are to coordinate the mapping of this data appropriately to enable the CSP to upload it to PECO's tracking system.





Introducing a broader mix of prescriptive lighting measures contributed to an increase in the participation rates for the program.

Based on the findings from the PECO program manager interviews, Navigant recommends adding more custom measures to the measure mix; they might help increase the program participation further by making the program a one-stop-shop for the landlord's energy needs.

Under Consideration. PECO will work with the CSP to review the measure mix based on market demand and potential.


Free ridership is lower in PY10 compared to PY8, likely due to the increase in prescriptive measure offerings. Prescriptive measures such as lighting fixture upgrades in the Large – Residential and C&I sectors are more costly than screw-in LEDs and, therefore, more likely to require program assistance.

PECO and the CSP should continue to monitor participation levels and adapt offerings accordingly as customers respond to offerings. PECO should continue to support and promote the range of measures (including lighting fixtures and non-lighting measures) to increase program engagement and improve awareness of program benefits.

Under Consideration. PECO will work with the CSP to review the measure mix based on market demand and potential.




3.2 Residential Low-Income EE Program

The Residential Low-Income EE Program offers a comprehensive array of participant solutions and activities to achieve PECO’s goal of helping income-eligible customers save energy. The Low-Income EE Program targets PECO residential electric customers with a household income of less than or equal to 150% of the FPL to meet the 5.5% low-income energy saving carveout requirement. Additionally, PECO targets Customer Assistance Program (CAP) customers with high usage and incomes of 0%-50% of the FPL per the February 17, 2016 Joint Petition for Settlement agreement. In PY10, the Low-Income EE Program consisted of the Whole Home Solution. Savings are achieved through a range of delivery mechanisms and methods including product giveaways, in-home audits, and direct install measures. The Low-Income EE Program discontinued a Lighting Solution in PY9 that offered upstream incentives (retailer buy-downs). PECO relies on three CSPs to deliver the program savings: CMC Energy, ARCA Inc., and Energy Coordinating Agency (ECA). PECO’s Low-Income EE Program refers eligible customers to the Residential EE Program’s Appliance Recycling Solution when appropriate. Appliance Recycling Solution savings from referred customers who are on the CAP rate and at or below 150% of the FPL are applied toward the Low-Income carveout and reported through the Whole Home Solution. PECO’s income-qualified customers are also eligible to benefit from the other solutions offered by the Residential EE Program. However, low-income participation in those solutions and the associated savings are reported through the Residential EE Program; they are not applied toward the Low-Income carveout. Rather, only savings from the programs and solutions in this section specifically targeting income-eligible customers count toward the carveout.


This section provides the Low-Income EE Program results for PY10, including participation, energy and demand savings, and incentive costs. Table 3-12 presents the participation counts and incentive payments for the Low-Income EE Program in PY10 by customer segment.

Table 3-12. Low-Income EE Program Summary by Customer Segment


PYTD No. of Participants 19,349 50 3

PYRTD MWh/yr 23,058.2 1,273.8 6.8

PYRTD MW 2.69 0.24 0.00

PY10 Incentives ($1,000) $121 $8 $1 Source: Navigant analysis


The Low-Income EE Program’s gross impact evaluation activities started with a review of program tracking data to verify proper application of TRM algorithms in reported savings values. The Navigant team completed these reviews for the full population of implemented PY10 Low-Income EE Program



measures. Next, the team identified appropriate evaluation activities for each solution depending on the nature of the participants, implementation strategies, and the level of information accompanying the reported savings. The primary goal of the evaluation activities was to verify the implementation of a given measure occurred. Navigant then drew samples for these gross impact evaluation activities. The team developed and sought approval for representative samples that complied with the Phase III Evaluation Framework, the PA TRM, and industry standards and that helped PECO meet the SWE and Commission requirements. The Whole Home Solution was verified through engineering file reviews of program applications and invoices and phone verifications for projects with direct installation measures only. Table 3-13 provides the sampling frame for the gross impact evaluation of the Low-Income EE Program in PY10.

Table 3-13. Low-Income EE Program Gross Impact Sample Design for PY10

Solution Stratum Name Population Size

Targeted Sample Size

Achieved Sample


Whole Home

Large SF 1,114 8 8 Engineering file review with phone verification

Medium SF 2,516 8 8 Engineering file review with phone verification

Small SF 4,390 8 8 Engineering file review with phone verification

Very Small SF 1,631 0 0 N/A

Multifamily 110 9 9 Engineering file review with phone verification

ECA 25 N/A N/A N/A

Giveaways 4,439 N/A N/A N/A

Refrigerator Recycling 1,401 24 24 File review and phone

verification survey

Freezers Recycling 186 8 8 File review and phone verification survey

Room AC Recycling 181 14 14 File review and phone

verification survey


Total Program All 15,993 79 79 SF: Single-family Source: Navigant analysis

Table 3-14 summarizes the reported and verified energy (MWh/yr) savings results, along with the CV and relative precision for each stratum sampled for the Low-Income EE Program in PY10.



Table 3-14. Low-Income EE Program Gross Results for Energy

Solution Stratum Name Reported Gross Energy Savings

(MWh/yr)

Verified Gross Energy Savings

(MWh/yr)

Energy Realization Rate

Achieved Sample CV or

Error Ratio

Relative Precision at 85% Confidence

Interval

Whole Home

Large SF 3,506 3,055 0.87 0.05 2.9%

Medium SF 5,110 4,779 0.94 0.00 0.0%

Small SF 6,015 4,547 0.76 0.57 32.3%

Very Small SF 536 405 0.76 N/A 100%

Multifamily 1,866 1,439 0.77 0.05 2.5%

ECA 66 66 1.00 N/A 50%

Giveaways 5,708 5,708 1.00 N/A 50%

Refrigerator Recycling 1,341 1,246 0.93 0.44 13.4%

Freezers Recycling 161 134 0.84 0.36 20.8%

Room AC Recycling 29 29 1.00 0.00 0.0%

Solution Total 24,339 21,409 0.88 0.88 14.3%

Total Program All 24,339 21,409 0.88 0.88 16.4%

[90% CI] SF: Single-family Source: Navigant analysis



Table 3-15 summarizes the reported and verified demand (MW) savings results, along with the CV and relative precision for each stratum sampled for the Low-Income EE Program in PY10.

Table 3-15. Low-Income EE Program Gross Results for Demand

Solution Stratum Name Reported Gross Demand Savings

(MW)

Verified Gross Demand Savings

(MW)

Demand Realization Rate



Interval

Whole Home

Large SF 0.38 0.31 0.82 0.06 3.2%

Medium SF 0.54 0.51 0.93 0.00 0.0%

Small SF 0.64 0.49 0.76 0.54 31.0%

Very Small SF 0.05 0.04 0.76 N/A 100%

Multifamily 0.29 0.23 0.77 0.03 1.7%

ECA 0.01 0.01 1.00 N/A 50%

Giveaways 0.80 0.80 1.00 N/A 50%

Refrigerator Recycling 0.15 0.14 0.90 0.44 13.4%

Freezers Recycling 0.02 0.02 0.82 0.36 20.8%

Room AC Recycling 0.05 0.05 1.00 0.00 0.0%

Solution Total 2.93 2.57 0.88 0.99 16.2%

Total Program All 2.93 2.57 0.88 0.99 18.5% [90% CI]

SF: Single-family Source: Navigant analysis



The overall evaluation resulted in a reduction to reported savings.


Net impacts were not assessed for the Low-Income EE Program in PY10.


HIM measures were not assessed for the Low-Income EE Program in PY10.


As described in the Phase III Evaluation Plan25 updated for PY10 and approved by the SWE, Navigant did not complete any in-depth process evaluation activities for the Low-Income EE Program Whole Home solution. Instead, the team interviewed the PECO program manager to identify significant implementation changes to inform the impact evaluation activities. No significant changes were found. The team carried out in-depth process evaluations in PY8 and plans to conduct additional activities in PY11.


No significant process findings were identified for the Low-Income EE Program in PY10.



Table 3-16. Summary of Low-Income EE Program Finances – Gross Verified





Participant Costs (Net of Incentives/Rebates Paid by Utilities) -$121 -$154



Design and Development (EDC Costs) [2] $0 $0


Administration, Management, and Technical Assistance (EDC Costs) [3] $71 $205


25 PECO. Phase III Evaluation Plan, Energy Efficiency and Conservation Portfolio. Revised February 15, 2019.




Marketing (EDC Costs) [4] $420 $1,538








Cost Subtotal $0 $0

Total NPV of Costs [6]

($1,000) Cost Total $7,736 $22,898







[1] Includes direct install equipment costs and costs for EE&C kits. [2] Includes direct costs attributable to plan and advance the programs. [3] Includes rebate processing, tracking system, general administration, program management, general management and legal, and technical assistance. [4] Includes the marketing CSP and marketing costs by program CSPs. EDC marketing costs broken out as a percentage of sector lifetime savings. This is an adjustment from the Preliminary Annual Report. [5] Direct program implementation costs. Labor, fuel, and vehicle operation costs for appliance recycling and direct install programs. [6] Total TRC Costs includes Total EDC Costs and Participant Costs. [7] Total TRC Benefits equals the sum of Total Lifetime Electric and Non-Electric Benefits. Benefits include avoided supply costs, including the reduction in costs of electric energy, generation, transmission, and distribution capacity, and natural gas valued at marginal cost for periods when there is a load reduction. NOTE: Savings carried over from Phase II are not to be included as a part of Total TRC Benefits for Phase III. [8] TRC Ratio equals Total NPV TRC Benefits divided by Total NPV TRC Costs. Source: Navigant analysis


Table 3-17. Summary of Low-Income EE Program Finances – Net Verified























Cost Subtotal $0 $0


($1,000) Cost Total $7,736 $22,898







[1] Includes direct install equipment costs and costs for EE&C kits. [2] Includes direct costs attributable to plan and advance the programs. [3] Includes rebate processing, tracking system, general administration, program management, general management and legal, and technical assistance. [4] Includes the marketing CSP and marketing costs by program CSPs. EDC marketing costs broken out as a percentage of sector lifetime savings. This is an adjustment from the Preliminary Annual Report. [5] Direct program implementation costs. Labor, fuel, and vehicle operation costs for appliance recycling and direct install programs. [6] Total TRC Costs includes Total EDC Costs and Participant Costs. [7] Total TRC Benefits equals the sum of Total Lifetime Electric and Non-Electric Benefits. Benefits include avoided supply costs, including the reduction in costs of electric energy, generation, transmission, and distribution capacity, and natural gas valued at marginal cost for periods when there is a load reduction. NOTE: Savings carried over from Phase II are not to be included as a part of Total TRC Benefits for Phase III. [8] TRC Ratio equals Total NPV TRC Benefits divided by Total NPV TRC Costs. Source: Navigant analysis


The impact and process evaluation activities in PY10 led to several findings and recommendations from Navigant to PECO. Table 3-18 presents the solution-level finding and recommendation for the Low-Income EE Program, along with a summary of how PECO plans to address the recommendation in



program delivery. Additional details on the solution-level analysis activities that led to this finding and recommendation can be found in Appendix F.

Table 3-18. Summary of Findings and Recommendations for the Residential Low-Income EE Program


Whole Home

Savings calculations for six measures were not consistent with TRM algorithms, resulting in discrepancies between reported and verified savings. (Measures: low flow aerators, freezer replacement, refrigerator replacement, ceiling/attic and wall insulation, floor insulation, commercial LED exit signs.)

Adjust data collection or savings calculation methodologies to align with TRM algorithms.

Being implemented. PECO is working with Navigant to understand the inconsistencies with the reported savings for the measures and will work to implement with the data management and implementation CSPs.


3.3 Small C&I EE Program

The Small C&I EE Program offers a comprehensive and crosscutting array of opportunities to assist small C&I customers in reducing their energy consumption and costs. The program encompasses a variety of energy solutions and measures to achieve this goal. The Small C&I EE Program is made up of four solutions and two targeted market segments, listed with the implementers below:

• Equipment and Systems Solution – ICF

• New Construction Solution – ICF

• Whole Building Solution – SmartWatt

• Behavioral Solution – Not implemented in PY10

• Data Centers Targeted Market Segment – ICF

• Multifamily Targeted Market Segment – Franklin The Behavioral Solution was not implemented in PY10 and had no corresponding evaluation activities. Common measures within the Small C&I EE Program include efficient lighting equipment, lighting controls, HVAC equipment, variable frequency drives (VFDs), refrigeration, and building automation systems, among others. Several solutions cut across multiple programs (i.e., Small C&I EE and Large C&I EE), and participation rules vary according to program rules.


This section provides the Small C&I EE Program results for PY10, including participation, energy and demand savings, and incentive costs. Table 3-19 presents the participation counts and incentive payments for the Small C&I EE Program in PY10 by customer segment.



Table 3-19. Small C&I EE Program Summary by Customer Segment


PYTD No. of Participants 0 1,886 0

PYRTD MWh/yr 0.0 56,738.4 0.0

PYRTD MW 0.00 8.95 0.00

PY10 Incentives ($1,000) $0 $2,989 $0 Source: Navigant analysis


In PY10, the Small C&I EE Program gross impact evaluation consisted of desk reviews, phone verifications, onsite verifications, and onsite metering for a sample of projects. Summaries of verification activities for each solution and targeted market segment follow:

• Equipment and Systems: The evaluation team conducted ex post verification activities for a sample of 45 projects in the Small C&I Equipment and Systems Solution in PY10, which is three more than the number of projects specified in Navigant’s sampling design memo.26

• New Construction: The evaluation team conducted ex post verification activities for a sample of 14 projects in the Small C&I New Construction Solution in PY10, which aligns with the number of projects specified in the sampling design memo.

• Whole Building: As described in the evaluation plan, the evaluation team did not conduct ex post verification for the Whole Building Solution in PY10. Instead, PY9 realization rates were applied to PY10 reported savings. The evaluation team is planning to update realization rates during the PY11 evaluation.

• Data Centers: The evaluation team conducted ex post verification activities for two projects in the Small C&I Data Centers Targeted Market Segment. This represents a census evaluation for PY10.

• Multifamily Targeted Market Segment: The evaluation team conducted ex post verification activities for a sample of 29 multifamily buildings making up 134 projects in the Multifamily Targeted Market Segment PY10 evaluation sample. Of these 29 buildings, seven fell into the Small C&I EE Program,27 which is one more than the number of projects specified in the sampling design memo.

Table 3-20 provides the sampling frame for the gross impact evaluation of the Small C&I EE Program in PY10.

26 PECO. “PY10 Small and Large C&I EE Impact Sampling Design.” Dated April 16, 2019. 27 The remaining verified projects for the Multifamily Targeted Market Segment are included in the Large C&I EE Program and the Residential EE Program.



Table 3-20. Small C&I EE Program Gross Impact Sample Design for PY10


Targeted Sample

Size

Achieved Sample



Very High Impact 1 1 1 Onsite survey with metering or onsite verification only

High Impact 40 9 9 Onsite survey with metering or onsite verification only

Medium Impact/Uncertainty 138 9 9 Onsite survey with metering

or onsite verification only

Low Impact 576 7 7 Onsite survey with metering or onsite verification only

Very Low Impact 242 0 0 Onsite survey with metering or onsite verification only

Midstream High Impact 31 8 12 Onsite or phone verification only

Midstream Low Impact 390 8 7 Onsite or phone verification only


New Construction

Very High Impact 0 2 0 N/A


Medium/Low Impact 36 2 7 Onsite survey with metering or onsite verification only

Very Low Impact 13 0 0 Onsite or phone verification only


Whole Building

Medium Impact/Uncertainty 96 15 15 Engineering file review with

onsite verification

Low Impact/Uncertainty 194 18 19 Engineering file review with telephone verification





Small 69 8 8 File review and onsite verification

Multisector 19 6 5 File review and onsite verification


Total Program All 1857 97 108




Table 3-21 summarizes the reported and verified energy savings results, along with the CV and relative precision for each stratum sampled for the Small C&I EE Program in PY10.

Table 3-21. Small C&I EE Program Gross Results for Energy




Energy Realization

Rate

Achieved Sample CV

or Error Ratio

Relative Precision

at 85% Confidence

Interval


Very High Impact 1,948 1,948 1.00 0.00 0.0%

High Impact 11,384 10,283 0.90 0.05 2.7%

Medium Impact 11,449 11,400 1.00 0.28 14.1%

Low Impact 11,468 12,718 1.11 0.19 11.9%

Very Low Impact 736 786 1.07 N/A 100.0%

Midstream High Impact 1,960 1,694 0.86 0.48 21.4%

Midstream Low Impact 1,605 2,174 1.35 0.72 44.8%

Solution Total 40,550 41,003 1.01 0.26 5.8%

New Construction

Very High Impact 0 0 N/A 0.00 0.0%

High Impact 1,740 2,019 1.16 0.13 8.3%

Low/Medium Impact 1,570 1,523 0.97 0.12 7.5%

Very Low Impact 68 66 0.97 N/A 100.0%

Solution Total 3,379 3,608 1.07 0.14 5.6%

Whole Building

Medium Impact/Uncertainty 6,024 5,840 0.97 0.12 4.9%

Low Impact/Uncertainty 2,447 2,578 1.05 0.14 5.0%

Solution Total 8,471 8,419 0.99 0.14 3.6% Data Centers Targeted Market Segment

Solution Total 119 50 0.42 0.00 0.0%


Small 3,830 3,773 0.99 0.10 5.6%

Multisector 390 372 0.95 0.15 11.9%

Solution Total 4,219 4,145 0.98 0.12 4.9%


[90% CI] Source: Navigant analysis



Table 3-22 summarizes the reported and verified demand savings results, along with the CV and relative precision for each stratum sampled for the Small C&I EE Program in PY10.

Table 3-22. Small C&I EE Program Gross Results for Demand


Reported Gross

Demand Savings

(MW)

Verified Gross

Demand Savings

(MW)

Demand Realization

Rate



85% Confidence

Interval


Very High Impact 0.00 0.00 N/A 0.00 0.0%

High Impact 1.66 1.32 0.79 0.20 11.2%

Medium Impact 1.81 2.10 1.16 0.49 24.3%

Low Impact 2.08 2.64 1.27 0.67 41.8%

Very Low Impact 0.11 0.14 1.22 0.00 100.0%

Midstream High Impact 0.27 0.35 1.29 0.28 12.4%

Midstream Low Impact 0.28 0.47 1.70 0.28 17.2%

Solution Total 6.22 7.02 1.13 0.72 15.8%

New Construction

Very High Impact 0.00 0.00 N/A 0.00 0.0%

High Impact 0.26 0.27 1.06 0.25 15.6%

Low/Medium Impact 0.25 0.23 0.92 0.25 15.4%

Very Low Impact 0.01 0.01 0.92 N/A 100.0%

Solution Total 0.52 0.52 0.99 0.25 10.3%

Whole Building

Medium Impact/Uncertainty 1.28 0.91 0.71 0.29 11.5%

Low Impact/Uncertainty 0.54 0.33 0.60 0.47 16.1%

Solution Total 1.82 1.23 0.68 0.36 9.2% Data Centers Targeted Market Segment

Solution Total 0.02 0.01 0.33 0.00 0.0%


Small 0.32 0.32 0.99 0.08 4.8% Multisector 0.05 0.05 1.12 0.16 13.1%

Solution Total 0.37 0.37 1.00 0.10 4.2%

Total Program All 8.95 9.14 1.02 0.86 13.8%





The Small C&I EE Program net impact evaluation used several methods to estimate free ridership, spillover, market effects, and NTG ratios for each solution. Navigant relied on consistent, crosscutting approaches as well as ones tailored to certain solutions’ characteristics. The primary objective of the net savings analysis was to determine the program's net effect on customer electricity usage. The evaluation team derived net program impacts by estimating an NTG ratio that quantifies the percentage of the gross program impacts that can reliably be attributed to the program. Free ridership is defined as those participants who would have implemented a measure or purchased equipment anyway, without program support or a rebate. The questions determining free ridership focus on the influence of key program interventions and the customer’s perception of what they would most likely have done in the absence of the program. Interventions vary by solution but can include discounted prices, program information regarding efficient products, and placement of program-discounted products in stores. Spillover is defined as those participants who were influenced by the program to purchase and install additional energy efficient equipment that saves electricity without a rebate or other program support. The evaluation team analyzed participant responses to a battery of spillover questions. The intent of these questions was to identify what types and amounts of equipment customers purchased and installed on their own to inform a quantitative estimate of program spillover within the overall NTG calculation. Market effects represent a change in the structure of a market or the behavior of participants in a market that is reflective of an increase in the adoption of EE products, services, or practices and is causally related to market intervention(s). As described in the Phase III Evaluation Plan28, Navigant applied the PY9 NTG values to the Equipment and Systems, New Construction, and Whole Building Solutions as well as the Data Centers Targeted Market Segment. For the Multifamily Targeted Market Segment, Navigant surveyed PECO participants through telephone surveys to gather information about free ridership and spillover. The evaluation team developed survey instruments consistent with the Phase III Evaluation Framework’s guidance on net impact evaluation techniques29 and guidance from the Uniform Methods Project on estimating net savings.30 Survey instruments also captured feedback about customer experiences from participants to inform the process evaluation. Navigant attempted to survey a census of the limited population of landlords participating in the C&I Multifamily Targeted Market Segment. Navigant also combined the Small C&I and Large C&I Program participants due to the tendency of landlords to oversee projects in both Small C&I rate class facilities and Large C&I rate class facilities. Navigant applied the resulting NTG ratio to both the Small C&I and Large C&I Programs. Table 3-23 provides the sampling frame for the net impact evaluation of the Small and Large C&I EE Programs in PY10.

28 PECO. Phase III Evaluation Plan, Energy Efficiency and Conservation Portfolio. Revised February 15, 2019. 29 Pennsylvania PUC. Phase III Evaluation Framework. Section 3.4. http://www.puc.pa.gov/Electric/pdf/Act129/SWE_PhaseIII-Evaluation_Framework102616.pdf 30 The Uniform Methods Project. Estimating Net Savings: Common Practices. NREL. https://www.nrel.gov/docs/fy14osti/62678.pdf






Table 3-23. Small and Large C&I EE Programs Net Impact Sample Design for PY10

Solution Stratum Name Population Size*

Targeted Sample Size

Achieved Sample Size

Response Rate

Verification Method

Multifamily Targeted Market Segment (landlords)

C&I 108 51 8 7.4% Phone survey

Solution Total 108 51 8 7.4%

*Target sample size for Multifamily Targeted Market Segment includes the Small C&I and Large C&I populations combined using unique contact information provided in the tracking data. Source: Navigant analysis

Table 3-24 summarizes the reported and verified energy savings results, the calculated NTG results, and the CV and relative precision for each stratum sampled for the Small C&I EE Program in PY10.



Table 3-24. Small C&I EE Programs Net Energy Savings Impact Evaluation Results for PY10

Solution Name Stratum Name



Free Ridership

Rate

Spillover Rate

NTG Ratio*

Achieved Sample CV or

Error Ratio

Relative Precision at 85%

Confidence Interval

Equipment and Systems**

Very High Impact 1,948 1,049 0.46 0.00 0.54 0.09 7.4% High Impact/Uncertainty 10,283 5,535 0.46 0.00 0.54 0.09 7.4% Medium Impact/Uncertainty 11,400 10,909 0.04 0.00 0.96 0.05 4.0% Low Impact 12,718 9,756 0.23 0.00 0.77 0.16 6.2% Very Low Impact 786 603 0.23 0.00 0.77 N/A 100% Midstream High Impact 1,694 1,428 0.16 0.00 0.84 N/A 100% Midstream Low Impact 2,174 1,668 0.23 0.00 0.77 N/A 100% Solution Total 41,003 30,947 0.25 0.00 0.75 0.28 7.5%

New Construction

Very High Impact 0 0 0.73 0.00 0.27 N/A N/A High Impact 2,019 540 0.73 0.00 0.27 N/A N/A Low/Medium Impact 1,523 407 0.73 0.00 0.27 N/A N/A Very Low Impact 66 18 0.73 0.00 0.27 N/A N/A Solution Total 3,608 965 0.73 0.00 0.27 0.10 7.9%

Whole Building Medium Impact/Uncertainty 5,840 5,850 0.08 0.08 1.00 0.13 6.2% Low Impact/Uncertainty 2,578 2,487 0.10 0.06 0.96 0.30 17.2% Solution Total 8,419 8,337 0.08 0.07 0.99 0.20 4.5%

Data Centers** Solution Total 50 46 0.08 0.00 0.92 N/A 100%


Small 3,773 3,049 0.19 0.00 0.81 0.00 0.0% Multisector 372 300 0.19 0.00 0.81 0.30 17.2% Solution Total 4,145 3,350 0.19 0.00 0.81 0.30 17.2%

Total Program All 57,224 43,644 0.25 0.01 0.76 0.35 6.2% [90% CI]

* Navigant conducted NTG research in PY10 for the Small C&I Multifamily Targeted Market Segment. For the other Small C&I solutions, Navigant applied stratum-level NTG results from PY9. ** For Equipment and Systems Midstream Low Impact and Midstream High Impact and the Data Centers Targeted Market Segment, Navigant applied the NTG ratio for each project by assigning strata based on the reported energy savings from the Equipment and Systems Low Impact, Medium Impact, or High Impact Strata. Source: Navigant analysis




HIMs represent measure categories or technologies of high importance in the PECO portfolio. In Phase III, the SWE suggested EDCs oversample HIMs to help program planners make decisions concerning those measures for downstream programs only.31 EDCs were to identify three to five measures for study within each program year based on energy impact, level of uncertainty, prospective value, funding, or other parameters. Navigant identified HIMs through several steps involving careful review of program- and solution-level savings, energy impact, and value to PECO. In PY10, Navigant conducted NTG research and HIM analysis for the Multifamily Targeted Market Segment in the non-residential sector. The evaluation team identified HIMs aligned with PECO’s Phase III planning document, using both the measure category32 and end-use subcategory for these solutions, including ENERGY STAR-qualified LEDs, low flow faucet aerators, and low flow showerheads for the Small C&I Program, and ENERGY STAR-qualified LEDs, water pumps, and lighting controls for the Large C&I Program. Due to the lack of survey responses from the multifamily landlord population (n=8), Navigant only gathered enough data to report NTG results for ENERGY STAR-qualified LEDs for PY10. Table 3-25 shows the results of the NTG research conducted for HIMs in PY10.33

Table 3-25. Small and Large C&I EE Program HIM NTG Summary

Solution HIM Free

Ridership Rate

Spillover Rate NTG Ratio

Multifamily Targeted Market Segment ENERGY STAR LED 0.19 0.00 0.81



Navigant conducted a detailed review of program materials including program databases, tracking systems, and other documents across all Small C&I EE Program solutions. PECO and CSP staff also provided essential information about the program design and how the program experience on the ground—particularly in PY10—compares with the EE&C Plan. The evaluation team conducted in-depth interviews at the beginning of the PY10 evaluation and communicated with staff on an ongoing basis as needed. The team developed interview instruments to include questions of interest to the evaluation and to allow for free-flowing conversations to obtain candid feedback from the interviewees. In addition to conducting interviews with PECO and CSP staff, Navigant also deployed landlord experience surveys to Small and Large C&I EE Program participants in the Multifamily Targeted Market Segment. Participants in this segment received telephone surveys to collect their feedback on a series of questions designed to gauge customer satisfaction, program channeling efforts, and to inform the NTG

31 Pennsylvania PUC. Phase III Evaluation Framework. Section 3.4.1.4. http://www.puc.pa.gov/Electric/pdf/Act129/SWE_PhaseIII-Evaluation_Framework102616.pdf 32 PECO Phase III data refers to measure categories as the Measure Name. 33 Due to the limited achieved sample size, Navigant was only able to gather sufficient data to inform NTG research for ENERGY STAR-qualified LEDs for HIM analysis in PY10. The other measures in the HIM analysis have a limited impact on program design, making up approximately 5% of the remaining Small C&I EE and Large C&I EE gross program savings combined.





analysis. Navigant developed a sample sufficient to provide 85/15 confidence/precision for the survey results. Responses among the Small C&I EE Program participants were markedly similar to those from Large C&I EE Program participants. As such, the general findings across both Large and Small C&I EE Programs will be detailed here. Separate Large C&I EE Program insights are provided in Section 3.4.4, as applicable. The following summarizes the process evaluation activities conducted for each C&I EE Program solution.

• Equipment and Systems Solution o PECO and CSP staff interviews o Program tracking data review


• Data Centers Targeted Market Segment o PECO and CSP staff interviews o Program tracking data review

• Whole Building Solution o PECO and CSP staff interviews o Program tracking data review

• Multifamily Targeted Market Segment (landlord focus) o PECO and CSP staff interviews o Program tracking data review o Phone survey: Navigant used phone surveys to assess how landlords heard about the

Multifamily Targeted Market Segment; their satisfaction with the program, solution, and PECO overall; and awareness of other PECO solutions. The survey sample was stratified by project size (C&I and Residential segment landlords).

See Table 3-23 in Section 3.3.3 for landlord experience survey sample details for the Multifamily Targeted Market Segment. Navigant experienced low response rates for the landlord survey despite making concerted efforts to achieve the target sample size without burdening participants. Partway through the survey effort, the team began offering a $100 incentive for a completed interview. Landlords were contacted an average of seven times.


For all solutions, the team interviewed the PECO program manager and CSP staff to identify significant implementation changes to inform the impact evaluation activities. Based on these interviews, the evaluation team did not document any significant changes to the Small C&I Program overall in PY10. Details on changes to specific solutions are detailed in Appendix G. For the Multifamily Targeted Market Segment, Navigant conducted customer experience surveys to assess metrics including satisfaction, marketing effectiveness, and barriers to participation. This section of the report includes findings that apply to both the Small and Large C&I EE Programs. Multifamily



Targeted Market Segment landlords were asked about their overall satisfaction with the program. Average satisfaction was 4.1 on a 5-point scale, with 1 representing Extremely Dissatisfied and 5 representing Extremely Satisfied (n=8). As shown in Figure 3-3, the majority of landlords were satisfied with their participation in Multifamily Targeted Market Segment, with two landlords reporting extreme satisfaction with their involvement with the program. No participant expressed dissatisfaction with their involvement in the program. High satisfaction ratings among landlords were attributed to the energy advisor’s communication prior to the visit, the advisor’s professionalism and friendliness during the installation, and the energy efficiency products installed on the property during the audit. One respondent indicated the audit and direct install took a long time, and it appeared that there was a lack of staff resources available to accommodate the visit.

Figure 3-3. Overall Landlord Satisfaction with Multifamily Targeted Market Segment, n=8

Question: “Using a scale of 1 to 5, with 5 meaning Extremely Satisfied and 1 meaning Extremely Dissatisfied, how would you rate your overall satisfaction with [Solution]?” Source: Navigant analysis The evaluation team also examined sources of awareness in Multifamily Targeted Market Segment to understand effective channels for reaching this segment. The leading source of awareness for landlords was direct interactions with PECO employees or representatives (three out of six respondents). Other sources of awareness included learning about the Multifamily Targeted Market Segment from a PECO bill insert and a community organization (Figure 3-4).



Figure 3-4. Sources of Landlord Multifamily Targeted Market Segment Awareness, n=8

*Includes a PECO employee, account representative, customer service representative Other includes calling PECO and learning about the program through internal staff. Question: “How did you learn about the [Solution] program?” Source: Navigant analysis

Navigant also asked participants which source of awareness had the greatest influence on their decision to participate in the Multifamily Targeted Market Segment. The top most influential source of awareness for landlords involved person-to-person interaction with a PECO representative (Figure 3-5). As seen in Figure 3-6, seven out of eight landlords also stated they would contact PECO directly for additional information about ways to save energy. Other responses included conducting a search on the internet (four out of eight) or contacting an energy equipment vendor (one out of eight).

Figure 3-5. Influence of Sources Multifamily Targeted Market Segment Awareness, n=6

Question: “Thinking of the ways you heard about the [Solution], which one was most influential in your decision to participate in the program?” Source: Navigant analysis



Figure 3-6. Sources of Additional Multifamily Targeted Market Segment Information, n=8

Question: “If you wanted information about ways to save energy, where would you typically look for this information?” Multiple responses allowed. Other includes contacting the corporate office. Source: Navigant analysis Respondents were also asked if they were aware of any other PECO solutions. As in PY8, none of the respondents were aware of any other PECO programs. The Multifamily Targeted Market Segment is designed to be a one stop shop for energy efficiency upgrades; yet, few custom projects are completed through the solution. PECO and the CSP should continue to focus on ensuring that participants are aware of all the energy savings opportunities in their facility that could be addressed through the Multifamily Targeted Market Segment. Further details on many of the specific questions and customer responses asked in the survey can be found in Appendix H.





Table 3-26. Summary of Small C&I EE Program Finances – Gross Verified













Cost Subtotal $0 $0


($1,000) Cost Total $22,920 $48,579



Lifetime Non-Electric Benefits (Fossil Fuel, Water, O&M) -$569 $739



[1] Includes direct install equipment costs. [2] Includes direct costs attributable to plan and advance the programs. [3] Includes rebate processing, tracking system, general administration, program management, general management and legal, and technical assistance. [4] Includes the marketing CSP and marketing costs by program CSPs. EDC marketing costs broken out as a percentage of sector lifetime savings. This is an adjustment from the Preliminary Annual Report. [5] Direct program implementation costs. Labor, fuel, and vehicle operation costs for direct install programs. [6] Total TRC Costs includes Total EDC Costs and Participant Costs. [7] Total TRC Benefits equals the sum of Total Lifetime Electric and Non-Electric Benefits. Benefits include avoided supply costs, including the reduction in costs of electric energy, generation, transmission, and distribution capacity, and natural gas valued at marginal cost for periods when there is a load reduction. NOTE: Savings carried over from Phase II are not to be included as a part of Total TRC Benefits for Phase III. [8] TRC Ratio equals Total NPV TRC Benefits divided by Total NPV TRC Costs. Source: Navigant analysis




Table 3-27. Summary of Small C&I EE Program Finances – Net Verified













Cost Subtotal $0 $0


($1,000) Cost Total $17,900 $38,617



Lifetime Non-Electric Benefits (Fossil Fuel, Water, O&M) -$493 $455



[1] Includes direct install equipment costs. [2] Includes direct costs attributable to plan and advance the programs. [3] Includes rebate processing, tracking system, general administration, program management, general management and legal, and technical assistance. [4] Includes the marketing CSP and marketing costs by program CSPs. EDC marketing costs broken out as a percentage of sector lifetime savings. This is an adjustment from the Preliminary Annual Report. [5] Direct program implementation costs. Labor, fuel, and vehicle operation costs for direct install programs. [6] Total TRC Costs includes Total EDC Costs and Participant Costs. [7] Total TRC Benefits equals the sum of Total Lifetime Electric and Non-Electric Benefits. Benefits include avoided supply costs, including the reduction in costs of electric energy, generation, transmission, and distribution capacity, and natural gas valued at marginal cost for periods when there is a load reduction. NOTE: Savings carried over from Phase II are not to be included as a part of Total TRC Benefits for Phase III.



[8] TRC Ratio equals Total NPV TRC Benefits divided by Total NPV TRC Costs. Source: Navigant analysis


The impact and process evaluation activities in PY10 led to several findings and recommendations from Navigant to PECO. Table 3-28 presents those solution-level findings and recommendations, along with a summary of how PECO plans to address the recommendations in program delivery. Additional details on the solution-level analysis activities that led to these findings and recommendations can be found in Appendix G.



Table 3-28. Summary of Findings and Recommendations for the Small C&I EE Program


Equipment and Systems New Construction

The CSP typically uses the system's nominal capacity to calculate savings for HVAC measures instead of its rated capacity. Table 3-23 of the TRM requires that the capacities used in savings calculations be the rated cooling and heating capacities.

The CSP should use rated capacity from manufacturer's specs instead of nominal capacity.

Implemented.


For HVAC measures in the residential units of multifamily buildings, the CSP and applicants are using effective full load hours (EFLH) for commercial hotel/lodging in the energy savings calculation.

Use residential HOU in residential units, including multifamily residential. Hotel/lodging should be reserved for lodging areas of hotels, motels, and similar rental areas.

Implemented.


In most cases, savings are accurately translated from project files to the tracking database. Only 4% of projects in the evaluation sample had minor errors due to rounding or digit truncation. An additional 4% of projects had mismatches between savings calculated in the project files and the savings tracked in the tracking database.

Continue to refine and improve eTrack data procedures to prevent data entry errors.

Recommendation noted. PECO has implemented a process with the CSP to refine this process.





The midstream contact information does not consistently lead evaluators to the right person for verification work. For the contacts included in the verification sample, Navigant's engineers found that many of the designated contacts in the database did not know about the project.

Identifying and collecting contact information for the person that would be most helpful for the evaluation is inherently difficult for midstream programs. Getting better contact information would reduce evaluation costs and improve the customer experience, but it must be balanced against the barrier to participation it might impose. The CSP should consider strengthening their efforts to inform program participants that they will be receiving follow-up inquiries about their participation and that these inquiries might come months afterward. The CSP should also consider asking for the contact information—especially email addresses—of the person most knowledgeable of the lighting operation. This extra upfront effort from the CSP will help streamline the midstream evaluation and improve the customer experience through fewer contacts for verification.

Under Review. The CSP will work with midstream market actors to determine additional steps to improve the upfront data collection process.





In 36 out of 57 projects across Large and Small Equipment and Systems Solutions, the verified lighting HOU differed from those reported by the CSP. In some cases, Navigant found that building type was misapplied. In other cases, the CSP used whole facility operating hours, while the evaluation team used operating hours of the installed lighting. The CSP also used deemed operating hours in some cases where the evaluation team used customer-reported hours. Navigant had a similar finding in PY9.

The CSP should consider refining its questions to program participants to ask for the schedule of lighting installed as part of the project rather than whole facility hours. In alignment with TRM and evaluation protocols, if the customer-reported HOU are more than 10% different then the deemed value for the building type, the CSP should use customer-reported values. Otherwise, the CSP should carefully select the appropriate building type for the project. Given that the solution-level realization rate has been near 100% for all of Phase III, the CSP should carefully balance the potential improvements to ex ante savings estimates against the barrier to participation it might impose. Navigant made a similar recommendation in PY9.

Implemented. Discussions with the CSP has led to correcting this process.


The evaluation discovered a few projects where motor power factor was applied incorrectly. In one case, the ex ante calculations did not account for the VFD power factor correction. In another project, the ex ante calculations assumed a power factor higher than appropriate, and the evaluation team adjusted it to the nameplate value.

The CSP should carefully consider the appropriate power factor to use in motors and VFD projects. VFDs are typically built with power factor correction, leading to a higher power factor.

Implemented. Discussions with the CSP have corrected this process.

New Construction

The energy savings models submitted to PECO for program participation are typically built with the initial planned construction specifications, and they are not updated to represent actual construction. This results in models providing inaccurate ex ante savings.

Models should represent as-built criteria, including specifications that represent installed equipment. This may require modifying models after construction.





New Construction Peak demand savings for whole building simulation projects are typically calculated as the annual energy savings divided by 8,760 hours/yr.

All viable software packages include the ability to output 8,760 hourly demand. The CSP should use this output to calculate peak demand savings as the average demand savings during peak hours on peak days. For PY10, this difference in methodology frequently led to discrepancies in the ex ante and ex post demand savings.


New Construction

Building efficiency codes require occupancy sensor lighting controls for some space types, including restrooms, conference rooms, and private offices. Some facilities choose to remove or disable these occupancy sensors after construction for a variety of reasons. These and other changes in lighting controls can significantly alter HOU, control savings factors, lighting savings, and realization rates.

The CSP should confirm with the customer via phone that the planned lighting controls are installed and used as designed after new buildings become occupied and operational.



Data centers are typically constructed with redundancy built into their computer room air conditioning units. The evaluation team found one project where all units were incentivized and assigned savings even though only one unit operates—and thus generates savings—at a time.

The CSP should ask participants if any of the units will be redundant/backup. Being implemented.


The Interim Measure Protocols (IMP) for uninterruptible power supplies (UPSs) requires the CSP to collect direct power measurements to determine ex ante savings. The CSP requires the customer to collect this data and submit it with their application. One customer reported that the cost required to collect this data (via outside electrician) was more than the incentive that PECO offered.

The CSP should directly collect the power data needed to meet the requirements of the IMP rather than relying on the customer to provide the data.





3.4 Large C&I EE Program

The Large C&I EE Program offers a comprehensive and crosscutting array of opportunities to assist large C&I customers in reducing their energy consumption and costs. The program encompasses a variety of energy solutions and measures to achieve this goal. The Large C&I EE Program is made up of two solutions and two targeted market segments, listed with the implementers below:

• Equipment and Systems Solution – ICF

• New Construction Solution – ICF

• Data Centers Targeted Market Segment – ICF

• Multifamily Targeted Market Segment – Franklin Common measures within the Large C&I EE Program include efficient lighting equipment, lighting controls, HVAC equipment, VFDs, refrigeration, and building automation systems, among others. Several solutions cut across multiple programs (i.e., Small C&I EE and Large C&I EE), and participation rules vary according to program rules.


This section provides the Large C&I EE Program results for PY10, including participation, energy and demand savings, and incentive costs. Table 3-29 presents the participation counts and incentive payments for the Large C&I EE Program in PY10 by customer segment.

Table 3-29. Large C&I EE Program Summary by Customer Segment


PYTD No. of Participants 0 0 904

PYRTD MWh/yr 0.0 0.0 93,706.8

PYRTD MW 0.00 0.00 12.40

PY10 Incentives ($1,000) $0 $0 $5,250 Source: Navigant analysis


In PY10, the Large C&I EE Program gross impact evaluation consisted of desk reviews, phone verifications, onsite verifications, and onsite metering for a sample of projects. Summaries of verification activities for each solution and targeted market segment follow:

• Equipment and Systems: The evaluation team conducted ex post verification activities for a sample of 44 projects in the Large C&I Equipment and Systems Solution in PY10, which aligns with the number of projects specified in the sampling design memo.34

• New Construction: The evaluation team conducted ex post verification activities for a sample of 10 projects in the Large C&I New Construction Solution in PY10, which aligns with the number of projects specified in the sampling design memo.

34 PECO. “PY10 Small and Large C&I EE Impact Sampling Design.” Dated April 16, 2019.



• Data Centers Targeted Market Segment: The evaluation team conducted ex post verification activities for the one project in the Large C&I Data Centers Targeted Market Segment PY10 evaluation sample. This represents a census evaluation in PY10.

• Multifamily Targeted Market Segment: The evaluation team conducted ex post verification activities for 29 multifamily buildings making up 134 verified projects in the Multifamily Targeted Market Segment. Of these 29 buildings, six fell into the Large C&I EE Program,35 which aligns with the number of projects specified in the sampling design memo.

Table 3-30 provides the sampling frame for the gross impact evaluation of the Large C&I EE Program in PY10.

Table 3-30. Large C&I EE Program Gross Impact Sample Design for PY10


Targeted Sample

Size

Achieved Sample



Very High Impact 1 1 1 Onsite survey with metering


Medium Impact 61 10 11 Onsite survey with metering or onsite verification only


Very Low Impact 139 0 0 N/A

Midstream High Impact 20 8 8 Onsite or phone

verification only

Midstream Low Impact 328 8 8 Onsite or phone

verification only


New Construction

Very High Impact 1 6 1 Onsite survey with metering or onsite verification only



Very Low Impact 12 0 0 Onsite or phone verification only


35 The remaining verified projects for the Multifamily Targeted Market Segment are included in the Small C&I EE Program and the Residential EE Program.




Targeted Sample

Size

Achieved Sample



Solution Total 1 3 1 Onsite survey with metering


Large 39 6 6 File review and onsite verification

Multisector 0 0 0 N/A


Total Program All 894 72 61




Table 3-31 summarizes the reported and verified energy savings results, along with the CV and relative precision for each stratum sampled for the Large C&I EE Program in PY10.

Table 3-31. Large C&I EE Program Gross Results for Energy

Solution Stratum Name Reported Gross Energy Savings

(MWh/yr)

Verified Gross Energy Savings

(MWh/yr) Energy

Realization Rate Achieved Sample CV or Error Ratio


Confidence Interval


Very High Impact 3,758 3,758 1.00 0.00 0.0% High Impact 24,239 16,707 0.69 0.00 12.9% Medium Impact 26,068 25,965 1.00 0.23 9.9% Low Impact 24,724 31,771 1.29 0.21 19.7% Very Low Impact 1,591 1,888 1.19 0.34 19.7% Midstream High Impact 1,416 1,262 0.89 0.34 8.7% Midstream Low Impact 1,256 1,502 1.20 0.15 19.4% Solution Total 83,053 82,853 1.00 0.37 8.1%

New Construction

Very High Impact 2,022 2,160 1.07 0.00 0.0% High Impact 2,529 2,096 0.83 0.23 22.3% Low Impact 1,990 1,495 0.75 0.43 34.5% Very Low Impact 119 90 0.75 N/A 100.0% Solution Total 6,661 5,840 0.88 0.21 10.3%

Data Centers Solution Total 36 22 0.60 0.00 0.0%


Large 3,957 3,937 0.99 0.08 5.5% Multisector 0 0 N/A N/A N/A Solution Total 3,957 3,937 0.99 0.08 5.5%

Total Program All 93,707 92,652 0.99 0.39 0.9% [90% CI]




Table 3-32 summarizes the reported and verified demand savings results, along with the CV and relative precision for each stratum sampled for the Large C&I EE Program in PY10.

Table 3-32. Large C&I EE Program Gross Results for Demand

Solution Stratum Name Reported Gross Demand Savings

(MW)

Verified Gross Demand Savings

(MW)

Demand Realization Rate



Confidence Interval


Very High Impact 0.49 0.49 1.00 0.00 0.0%

High Impact 2.95 2.06 0.70 0.96 55.0%

Medium Impact 3.63 3.12 0.86 0.25 11.7%

Low Impact 3.25 4.94 1.52 0.43 24.8%

Very Low Impact 0.29 0.40 1.38 0.43 24.8%

Midstream High Impact 0.21 0.27 1.29 0.36 20.8%

Midstream Low Impact 0.21 0.26 1.24 0.28 16.1%

Solution Total 11.02 11.52 1.05 0.63 13.9%

New Construction

Very High Impact 0.23 0.66 2.88 0.00 0.0%

High Impact 0.33 0.43 1.31 0.47 45.4%

Low Impact 0.28 0.27 0.98 0.11 8.8%

Very Low Impact 0.02 0.02 0.98 N/A 100%

Solution Total 0.86 1.39 1.61 0.23 11.7% Data Centers Solution Total 0.00 0.00 0.61 0.00 0.0%


Large 0.52 0.49 0.95 0.08 5.7%

Multisector 0.00 0.00 N/A N/A N/A

Solution Total 0.52 0.49 0.95 0.08 5.7%

Total Program All 12.40 13.40 1.08 0.64 13.7% [90% CI]





As described in the Phase III Evaluation Plan36, Navigant applied the PY9 NTG values to the Equipment and Systems and New Construction Solutions and the Data Centers Targeted Market Segment; the evaluation team only conducted NTG analysis for the Multifamily Targeted Market Segment in PY10 under the Large C&I EE Program. As described in the Small C&I EE Program section (Section 3.3.3), Navigant combined the Small C&I EE Program and Large C&I EE Program NTG analysis due to the limited landlord population and the overlap of landlords overseeing both Large C&I rate class projects and Small C&I rate class projects. See Section 3.3.3 under the Small C&I EE Program for a detailed account of the C&I NTG analysis for PY10. Table 3-33 summarizes the reported and verified energy savings results, the calculated NTG results, and the CV and relative precision for each stratum sampled for the Large C&I EE Program in PY10.




Table 3-33. Large C&I EE Program Net Energy Savings Impact Evaluation Results for PY10

Solution Name Stratum Name



Free Ridership

Rate

Spillover Rate

NTG Ratio*



Interval

Equipment and Systems**

Very High Impact 3,758 3,758 0.00 0.00 1.00 0.00 0.0%

High Impact 16,707 11,551 0.38 0.07 0.69 0.09 6.4%

Medium Impact 25,965 23,697 0.09 0.00 0.91 0.05 5.9%

Low Impact 31,771 21,215 0.34 0.01 0.67 0.16 12.4%

Very Low Impact 1,888 1,261 0.34 0.01 0.67 N/A 100%

Midstream High Impact 1,262 942 0.26 0.01 0.75 N/A 100%

Midstream Low Impact 1,502 1,003 0.34 0.01 0.67 N/A 100%

Solution Total 82,853 63,428 0.25 0.02 0.77 0.18 4.9%

New Construction

Very High Impact 2,160 876 0.59 0.00 0.41 N/A N/A High Impact 2,096 850 0.59 0.00 0.41 N/A N/A Low Impact 1,495 607 0.59 0.00 0.41 N/A N/A Very Low Impact 90 36 0.59 0.00 0.41 N/A N/A Solution Total 5,840 2,370 0.59 0.00 0.41 0.53 42.1%

Data Centers Targeted Market Segment **

Solution Total 22 14 0.34 0.01 0.67 N/A 100%


Large 3,937 3,181 0.19 0.00 0.81 0.30 17.2%

Multisector 0 0 0.19 0.00 0.81 N/A N/A

Solution Total 3,937 3,181 0.19 0.00 0.81 0.30 17.2% Total Program All 92,652 68,993 0.27 0.02 0.74 0.21 5.4%

[90% CI] * Navigant conducted NTG research in PY10 for the Large C&I Multifamily Targeted Market Segment. For the other Large C&I solutions, Navigant applied stratum-level NTG results from PY9. ** For Equipment and Systems Midstream Low Impact and Midstream High Impact and the Data Centers Targeted Market Segment, Navigant applied the NTG ratio for each project by assigning strata based on the reported energy savings from the Equipment and Systems Low Impact, Medium Impact, or High Impact strata. Source: Navigant analysis




See Section 3.3.3.1 under the Small C&I EE Program for a detailed account of the HIM analysis for the C&I sector in PY10.


As in the Small C&I EE Program, Navigant conducted a detailed review of program materials including program databases, tracking systems, and other documents across all Large C&I EE Program solutions. PECO and CSP staff also provided essential information about the program design and how the program experience on the ground—particularly in PY10—compares with the EE&C Plan. The evaluation team conducted in-depth interviews at the beginning of the PY10 evaluation and communicated with staff on an ongoing basis as needed. The team developed interview instruments to include questions of interest to the evaluation and to allow for free-flowing conversations to obtain candid feedback from the interviewees. In addition to conducting interviews with PECO and CSP staff, Navigant also deployed landlord experience surveys to Small and Large C&I EE Program participants in the Multifamily Targeted Market Segment. Participants in this segment received telephone surveys to collect their feedback on a series of questions designed to gauge customer satisfaction, program channeling efforts, and to inform the NTG analysis. Navigant developed a sample sufficient to provide 85/15 confidence/precision for the survey results. Responses among the Small C&I EE Program participants were markedly similar to those from Large C&I EE Program participants. As such, the general findings across both Large and Small C&I EE Programs are detailed in Section 3.3.4. The following summarizes the process evaluation activities conducted for each C&I EE Program solution.

• Equipment and Systems Solution o PECO and CSP staff interviews o Program tracking data review


• Data Centers Targeted Market Segment o PECO and CSP staff interviews o Program tracking data review

• Multifamily Targeted Market Segment o PECO and CSP staff interviews o Program tracking data review o Phone survey: Navigant used phone surveys to assess how landlords heard about the

Multifamily Targeted Market Segment; their satisfaction with the program, solution, and PECO overall; and awareness of other PECO solutions. The survey sample was stratified by project size (C&I and Residential segment landlords).



See Table 3-23 in Section 3.3.3 for landlord experience survey sample details for the Multifamily Targeted Market Segment.


For all solutions, the team interviewed the PECO program manager and CSP staff to identify significant implementation changes to inform the impact evaluation activities. Based on PECO Staff and CSP interviews, the evaluation team did not document any significant changes to the Large C&I EE Program overall in PY10. Details on changes to specific solutions are detailed in Appendix G. For the Multifamily Targeted Market Segment, customer experience surveys were conducted to assess metrics including satisfaction, marketing effectiveness, and barriers to participation. See Section 3.3.4.1 of the report which includes findings that apply to both the Large and Small C&I EE Programs.



Table 3-34. Summary of Large C&I EE Program Finances – Gross Verified










Marketing (EDC Costs) [4] $804 $2,955 Marketing (CSP Costs) [4] $0 $0 Program Delivery (EDC Costs) [5] $0 $0 Program Delivery (CSP Costs) [5] $4,501 $11,701 EDC Evaluation Costs $0 $0 SWE Audit Costs $0 $0 Cost Subtotal $5,397 $14,931



Cost Subtotal $0 $0





($1,000) Cost Total $42,116 $83,111



Lifetime Non-Electric Benefits (Fossil Fuel, Water, O&M) -$1,026 -$775


TRC Benefit-Cost Ratio [8] Benefits Total/Costs Total 0.97 0.97 [1] Includes direct install equipment costs. [2] Includes direct costs attributable to plan and advance the programs. [3] Includes rebate processing, tracking system, general administration, program management, general management and legal, and technical assistance. [4] Includes the marketing CSP and marketing costs by program CSPs. EDC marketing costs broken out as a percentage of sector lifetime savings. This is an adjustment from the Preliminary Annual Report. [5] Direct program implementation costs. Labor, fuel, and vehicle operation costs for direct install programs. [6] Total TRC Costs includes Total EDC Costs and Participant Costs. [7] Total TRC Benefits equals the sum of Total Lifetime Electric and Non-Electric Benefits. Benefits include avoided supply costs, including the reduction in costs of electric energy, generation, transmission, and distribution capacity, and natural gas valued at marginal cost for periods when there is a load reduction. NOTE: Savings carried over from Phase II are not to be included as a part of Total TRC Benefits for Phase III. [8] TRC Ratio equals Total NPV TRC Benefits divided by Total NPV TRC Costs. Source: Navigant analysis


Table 3-35. Summary of Large C&I EE Program Finances – Net Verified























Cost Subtotal $0 $0


($1,000) Cost Total $31,915 $64,498




Lifetime Non-Electric Benefits (Fossil Fuel, Water, O&M) -$780 -$705



[1] Includes direct install equipment costs. [2] Includes direct costs attributable to plan and advance the programs. [3] Includes rebate processing, tracking system, general administration, program management, general management and legal, and technical assistance. [4] Includes the marketing CSP and marketing costs by program CSPs. EDC marketing costs broken out as a percentage of sector lifetime savings. This is an adjustment from the Preliminary Annual Report. [5] Direct program implementation costs. Labor, fuel, and vehicle operation costs for direct install programs. [6] Total TRC Costs includes Total EDC Costs and Participant Costs. [7] Total TRC Benefits equals the sum of Total Lifetime Electric and Non-Electric Benefits. Benefits include avoided supply costs, including the reduction in costs of electric energy, generation, transmission, and distribution capacity, and natural gas valued at marginal cost for periods when there is a load reduction. NOTE: Savings carried over from Phase II are not to be included as a part of Total TRC Benefits for Phase III. [8] TRC Ratio equals Total NPV TRC Benefits divided by Total NPV TRC Costs. Source: Navigant analysis


The impact and process evaluation activities in PY10 led to several findings and recommendations from Navigant to PECO. Table 3-36 presents those solution-level findings and recommendations, along with a summary of how PECO plans to address the recommendations in program delivery. Additional details on the solution-level analysis activities that led to these findings and recommendations can be found in Appendix G.



Table 3-36. Summary of Findings and Recommendations for the Large C&I EE Program



The CSP used trend data for some projects to determine reported savings. The analysis relied on second order polynomial regression curves to fit the data. This approach yields a good fit in the middle ranges of the data, but it distorts the results at the extreme ends, resulting in inaccurate ex ante energy savings that drove the realization rate down for PY10. Navigant had a similar finding in PY9.

The CSP should apply piecewise linear regressions to its analysis to prevent the extreme temperature ranges from distorting the annual savings. Typically, the CSP will have one regression for heating, one for cooling, and one that connects the two.

Being implemented.


The CSP typically uses the system's nominal capacity to calculate savings for HVAC measures instead of its rated capacity. Table 3-23 of the TRM requires that the capacities used in savings calculations be the rated cooling and heating capacities.

The CSP should use rated capacity from manufacturer's specs instead of nominal capacity.

Implemented.


For HVAC measures in the residential units of multifamily buildings, the CSP and applicants are using EFLH for commercial hotel/lodging in the energy savings calculation.

Use residential HOU in residential units, including multifamily residential. Hotel/lodging should be reserved for lodging areas of hotels, motels, and similar rental areas.

Implemented.


In most cases, savings are accurately translated from project files to the tracking database. Only 4% of projects in the evaluation sample had minor errors due to rounding or digit truncation. An additional 4% of projects had mismatches between savings calculated in the project files and the savings tracked in the tracking database.

Continue to refine and improve eTrack data procedures to prevent data entry errors.

Recommendation noted. PECO has implemented a process with the CSP to refine this process.





The midstream contact information does not consistently lead evaluators to the right person for verification work. For the contacts included in the verification sample, Navigant's engineers found that many of the designated contacts in the database did not know about the project.

Identifying and collecting contact information for the person that would be most helpful for the evaluation is inherently difficult for midstream programs. Getting better contact information would reduce evaluation costs and improve the customer experience, but it must be balanced against the barrier to participation it might impose. The CSP should consider strengthening their efforts to inform program participants that they will be receiving follow-up inquiries about their participation and that these inquiries might come months afterward. The CSP should also consider asking for the contact information—especially email addresses—of the person most knowledgeable of the lighting operation. This extra upfront effort from the CSP will help streamline the midstream evaluation and improve the customer experience through fewer contacts for verification.

Under Review. The CSP will work with midstream market actors to determine additional steps to improve the upfront data collection process.





In 36 out of 57 projects across Large and Small Equipment and Systems Solutions, the verified lighting HOU differed from those reported by the CSP. In some cases, Navigant found that building type was misapplied. In other cases, the CSP used whole facility operating hours, while the evaluation team used operating hours of the installed lighting. The CSP also used deemed operating hours in some cases where the evaluation team used customer-reported hours. Navigant had a similar finding in PY9.

The CSP should consider refining its questions to program participants to ask for the schedule of lighting installed as part of the project rather than whole facility hours. In alignment with TRM and evaluation protocols, if the customer-reported HOU are more than 10% different then the deemed value for the building type, the CSP should use customer-reported values. Otherwise, the CSP should carefully select the appropriate building type for the project. Given that the solution-level realization rate has been near 100% for all of Phase III, the CSP should carefully balance the potential improvements to ex ante savings estimates against the barrier to participation it might impose. Navigant made a similar recommendation in PY9.

Implemented. Discussions with the CSP has led to correcting this process.


The evaluation discovered a few projects where motor power factor was applied incorrectly. In one case, the ex ante calculations did not account for the VFD power factor correction. In another project, the ex ante calculations assumed a power factor higher than appropriate, and the evaluation team adjusted it to the nameplate value.

The CSP should carefully consider the appropriate power factor to use in motors and VFD projects. VFDs are typically built with power factor correction, leading to a higher power factor.


New Construction

The energy savings models submitted to PECO for program participation are typically built with the initial planned construction specifications, and they are not updated to represent actual construction. This results in models providing inaccurate ex ante savings.

Models should represent as-built criteria, including specifications that represent installed equipment. This may require modifying models after construction.





New Construction Peak demand savings for whole building simulation projects are typically calculated as the annual energy savings divided by 8,760 hours/yr.

All viable software packages include the ability to output 8,760 hourly demand. The CSP should use this output to calculate peak demand savings as the average demand savings during peak hours on peak days. For PY10, this difference in methodology frequently led to discrepancies in the ex ante and ex post demand savings.


New Construction

Building efficiency codes require occupancy sensor lighting controls for some space types, including restrooms, conference rooms, and private offices. Some facilities choose to remove or disable these occupancy sensors after construction for a variety of reasons. These and other changes in lighting controls can significantly alter HOU, control savings factors, lighting savings, and realization rates.

The CSP should confirm with the customer via phone that the planned lighting controls are installed and used as designed after new buildings become occupied and operational.



Data centers are typically constructed with redundancy built into their computer room air conditioning units. The evaluation team found one project where all units were incentivized and assigned savings even though only one unit operates—and thus generates savings—at a time.

The CSP should ask participants if any of the units will be redundant/backup. Being implemented.


The IMP for UPSs requires the CSP to collect direct power measurements to determine ex ante savings. The CSP requires the customer to collect this data and submit it with their application. One customer reported that the cost required to collect this data (via outside electrician) was more than the incentive that PECO offered.

The CSP should directly collect the power data needed to meet the requirements of the IMP rather than relying on the customer to provide the data.





3.5 Combined Heat and Power Program

The PECO CHP Program is designed to influence customer behavior and purchasing decisions. CHP technologies generate electric and thermal energy from a single fuel source. Customers with steady baseload electricity usage coupled with steady thermal demand can accomplish significant efficiencies and savings by incorporating CHP (sometimes referred to as cogeneration) in their facilities. The best economics are realized for CHP systems sized to match the minimum electric and thermal loads. PECO designed the CHP Program to ensure participating customers install CHP projects that maximize operational savings and minimize operations and maintenance (O&M) costs. The CHP Program has three types of incentives that are distributed at key milestones in the design, construction, and operation phases:

• Design: Incentives based on proposed system capacity.

• Capacity: Incentives based on a declining tiered incentive rate by installed capacity. Each tier has a fixed incentive per kilowatt paid toward the incremental capacity within each tier.

• Performance: Incentives are based on a fixed per kilowatt-hour basis based on actual energy production. The kilowatt-hour production is determined during a monitoring period that begins after the commercial date of operation (CDO) and is designed to capture the typical system operational performance. Savings for all projects are claimed upon implementation and can be adjusted based on the performance monitoring results.

PECO delivers the program directly through a rolling enrollment process. Projects that are deemed with a high probability of starting normal operation prior to the close of Phase III are enrolled in the program and provided a reservation letter for the anticipated incentive amount.


This section provides the total CHP Program results for PY10, including participation, energy and demand savings, and incentive costs. Table 3-37 presents the participation counts and incentive payments for the CHP Program in PY10 by customer segment.

Table 3-37. CHP Program Summary by Customer Segment


PYTD No. of Participants 0 0 3

PYRTD MWh/yr 0.0 0.0 22,448.6

PYRTD MW 0.00 0.00 2.18

PY10 Incentives ($1,000) $0 $0 $878 Source: Navigant analysis


The CHP Program gross impact evaluation involved onsite verification, telephone interviews with program participants, interviews with other CHP project developers who either have active CHP projects or may have such projects in the future, and interviews with the PECO program manager. Navigant conducted a



census of three participants and calculated gross impacts according to the CHP chapter of the Uniform Methods Project.37 Participants in the CHP Program are required to log the parameters necessary to calculate electricity generation net of parasitic loads (such as pumps necessary to operate the heat recovery systems) and thermal energy recovery. The evaluation team uses this data to estimate the system capacity and annual generation on which PECO bases its capacity and performance incentives. Table 3-38 summarizes the reported and verified energy savings results, along with the CV and relative precision for the CHP Program in PY10.

Table 3-38. CHP Program Gross Results for Energy




Energy Realization

Rate



85% Confidence

Interval

CHP Census 22,449 15,916 0.71 0.00 0.0%



Table 3-39 summarizes the reported and verified demand savings results, along with the CV and relative precision for the CHP Program in PY10.

Table 3-39. CHP Program Gross Results for Demand


Reported Gross

Demand Savings

(MW)

Verified Gross

Demand Savings

(MW)

Demand Realization

Rate



85% Confidence

Interval

CHP Census 2.18 1.89 0.87 0.00 0.0%

Total Program All 2.18 1.89 0.87 0.00 0.0%


Overall, the variations between the reported and verified savings and the observed realization rates for the CHP Program are reasonable given the complexity of these systems and uncertainty in their early operational periods. The low energy realization rate is attributable to the largest project operating with fewer than expected hours and low capacity factors on two of its three generators.

37 Simons, G.; Barsun, S. “Chapter 23: Combined Heat and Power Evaluation Protocol,” The Uniform Methods Project: Methods for Determining Energy-Efficiency Savings for Specific Measures. 2017. National Renewable Energy Laboratory. http://www.nrel.gov/docs/fy17osti/68579.pdf

http://www.nrel.gov/docs/fy17osti/68579.pdf




Navigant surveyed PECO program participants via telephone to gather information about free ridership and spillover. The evaluation team developed survey instruments consistent with the Phase III Evaluation Framework’s guidance on net impact evaluation techniques38 and guidance from the Energy Trust of Oregon NTG methodology.39 Table 3-40 summarizes the reported and verified energy savings results, the calculated NTG results, and the CV and relative precision for the CHP Program in PY10.

Table 3-40. CHP Program Net Energy Savings Impact Evaluation Results for PY10

Solution Name

Stratum Name


Verified Net


Free Ridership

Rate

Spillover Rate

NTG Ratio



85% Confidence

Interval

CHP Census 15,916 13,899 0.13 0.00 0.87 0.00 0.0%

Total Program All 15,916 13,899 0.13 0.00 0.87 0.00

0.0% [90% CI]



There were no HIM measures included in the CHP Program evaluation.


In PY10, per the Phase III Evaluation Plan40, Navigant conducted process evaluation consisting of in-depth interviews with a census of participants and the PECO CHP program manager. The interviews gathered information about the participation process and participant attitudes about and suggestions for the program. One of three participants interviewed indicated they were satisfied with the program (scoring the program a 4 on a 1-5 scale, with 5 meaning Extremely Satisfied and 1 meaning Extremely Dissatisfied). Two of three participants gave the program a satisfaction score of 2 or lower. Both respondents also rated the incentive amount a 3, with one participant saying the incentive is too small and the other participant saying that better incentives are available in other states. In PY9, Navigant compared PECO incentives to other states and found that PECO’s incentives were comparable to other programs in the region. The evaluation team also noted that incentive adjustments are unlikely to affect program participation in the current phase due to long project cycles for CHP projects. Both participants that rated their satisfaction as two or lower indicated that guidance on the participation process for the CHP program could be improved, with one respondent saying that the process was not as

38 Pennsylvania PUC. Phase III Evaluation Framework. Section 3.4. http://www.puc.pa.gov/Electric/pdf/Act129/SWE_PhaseIII-Evaluation_Framework102616.pdf 39 Phil Degens and Sarah Castor, “Energy Trust Free Ridership Methodology,” Energy Trust of Oregon. August 7, 2013. 40 PECO. Phase III Evaluation Plan, Energy Efficiency and Conservation Portfolio. Revised February 15, 2019.





easy as in other states. All respondents also noted the interconnection process was a barrier to completing their project, with one respondent derating their capacity to undergo a less rigorous, and therefore quicker, interconnection process.



Table 3-41. Summary of CHP Program Finances – Gross Verified



EDC Incentives to Participants [1] $878 $955









Marketing (EDC Costs) [4] $0 $0



Program Delivery (CSP Costs) [5] $44 $79



Cost Subtotal $44 $79 NPV of Fossil Fuel Impacts from Fuel Switching ($1,000)




($1,000) Cost Total $9,640 $15,701

Total NPV of Benefits [7]

($1,000)



Lifetime Non-Electric Benefits (Fossil Fuel, Water, O&M) $0 $0






[1] Includes direct install equipment costs. [2] Includes direct costs attributable to plan and advance the programs. [3] Includes rebate processing, tracking system, general administration, program management, general management, legal, and technical assistance. [4] Includes the marketing CSP and marketing costs by program CSPs. EDC marketing costs broken out as a percentage of sector lifetime savings. This is an adjustment from the Preliminary Annual Report. [5] Direct program implementation costs. Labor, fuel, and vehicle operation costs for appliance recycling and direct install programs. [6] Total TRC Costs includes Total EDC Costs and Participant Costs. [7] Total TRC Benefits equals the sum of Total Lifetime Electric and Non-Electric Benefits. Benefits include avoided supply costs, including the reduction in costs of electric energy, generation, transmission, and distribution capacity, and natural gas valued at marginal cost for periods when there is a load reduction. NOTE: Savings carried over from Phase II are not to be included as a part of Total TRC Benefits for Phase III. [8] TRC Ratio equals Total NPV TRC Benefits divided by Total NPV TRC Costs. Source: Navigant analysis Table 3-42 presents program financials and cost-effectiveness on a net savings basis.

Table 3-42. Summary of CHP Program Finances – Net Verified

























($1,000) Cost Total $8,424 $13,844







[1] Includes direct install equipment cost. [2] Includes direct costs attributable to plan and advance the programs. [3] Includes rebate processing, tracking system, general administration, program management, general management and legal, and technical assistance. [4] Includes the marketing CSP and marketing costs by program CSPs. EDC marketing costs broken out as a percentage of sector lifetime savings. This is an adjustment from the Preliminary Annual Report. [5] Direct program implementation costs. Labor, fuel, and vehicle operation costs for appliance recycling and direct install programs. [6] Total TRC Costs includes Total EDC Costs and Participant Costs. [7] Total TRC Benefits equals the sum of Total Lifetime Electric and Non-Electric Benefits. Benefits include avoided supply costs, including the reduction in costs of electric energy, generation, transmission, and distribution capacity, and natural gas valued at marginal cost for periods when there is a load reduction. NOTE: Savings carried over from Phase II are not to be included as a part of Total TRC Benefits for Phase III. [8] TRC Ratio equals Total NPV TRC Benefits divided by Total NPV TRC Costs. Source: Navigant analysis


The impact and process evaluation activities in PY10 led to several findings and recommendations from Navigant to PECO. Table 3-43 presents those findings and recommendations, along with a summary of how PECO plans to address the recommendations in program delivery.



Table 3-43. Summary of Findings and Recommendations for the CHP Program


CHP

Realization rates are depressed by ex ante savings analyses using limited amounts of performance data to estimate energy and demand impacts.

PECO should align the ex ante analysis schedule with the program evaluation schedule to increase the amount of performance data available.

Under consideration. PECO will review the existing project schedules against the evaluation cycle to ensure maximum performance data, but other factors often weigh on the final timing and decision to certify a project.

CHP Projects with generation components not designed to operate baseload do not deliver reliable savings.

PECO should prioritize incentivizing projects with base loaded operational schemes over projects with intermittent generation components. On projects with intermittent generation, PECO should review facility load and operation plans prior to issuing incentives.

Under consideration. Typically, all submitted projects are baseload by their inherent design. PECO relies on the project design criteria provided the customer, who themselves rely on their CHP designer. What we are seeing is that in some circumstances, systems do not operate as designed, and or are not sized properly. While problematic, this level of design scrutiny is not an inherent portion of the CHP incentive application criteria, and therefore is difficult to proactively manage.

CHP

Projects continue to have difficulty with the PECO interconnection process. The issues have resulted in diminished savings and project completion delays.

PECO should consider creating an interconnection liaison for the CHP Program. The liaison would work the PECO interconnection team to ensure that the project completion is not delayed due to interconnection processing times.

Being Implemented. The primary challenge is that the interconnection process spans two separate business units. Efforts are underway to ensure future CHP projects are not encumbered by the current and past interconnection permitting process. PECO will consider working with a liaison to ensure project completion and timely processing.

CHP Project IDs do not have a consistent naming convention.

PECO should implement a consistent naming convention for project IDs.

Being implemented. PECO will work with its data management CSP and project team to create uniform project IDs.




3.6 Demand Response Programs

PECO’s DR Programs include Residential DR, Small C&I DR, and Large C&I DR. These three programs encompass opportunities designed to engage customers across all sectors to reduce demand. The Residential DR Program’s eligible population and target markets are all PECO residential electric customers. The program encompasses three solutions:

• Residential DLC

• Smart Thermostat for DR Savings

• Behavioral DR Savings Only the Residential DLC Solution is currently active. The Residential DLC Program is implemented by Itron (formerly Comverge). It was designed to shift participant loads from peak to off-peak hours by cycling their central air conditioner during DR events by 50%. The PY10 summer DR events had over 55,000 residential participants. In PY10 and for the remainder of Phase III, participants receive an incentive of $40 per direct load control (DLC) unit per year. PECO designed its Small C&I DR Program to engage customers to reduce demand through DLC of major electrical end-use equipment during designated peak load hours. The eligible population and target markets for the Small C&I DR Program are all PECO small C&I customers; this includes customers in the G/E/NP sector. The Small C&I DLC Solution is implemented by Itron (formerly Comverge). The program shifts load to off-peak hours by cycling participant air conditioners by 50% during DR event days. The PY10 summer DR events had over 1,400 Small C&I participants. In PY10 and for the remainder of Phase III, participants receive an incentive of $40 per DLC unit per year. PECO designed the Large C&I DR Program to engage customers in demand reduction through DR aggregation across multiple customers. The eligible population and target markets for the PECO Large C&I DR Program are all PECO Large C&I electric customers, including those in the G/E/NP sector. The program encompasses a single solution, the Demand Response Aggregator (DRA) Solution, and is implemented by two CSPs: Enel X (formerly EnerNOC) and CPower. For Phase III, event days for all programs are called when the PJM day-ahead peak load forecast reaches 96%. Based on the day-ahead forecasts, PECO called six events during the summer of 2018: July 2, July 3, August 6, August 28, September 4, and September 5. Compliance targets for DR programs were established at the system level, which means the load reductions measured at the customer meter must be escalated to reflect T&D losses. The peak demand impacts presented in this section have been adjusted for line losses.


Table 3-44 presents the participation counts, reported peak demand savings, and EDC expenditures for the three DR program in PY10 by customer segment.



Table 3-44. PY10 DR Program Summary by Customer Segment

Parameter Residential

(Residential DR Program)

Small C&I (Small C&I DR

Program)

Large C&I (Large C&I DR

Program)

PYTD No. of Participants 56,030 1,427 348

PYRTD MWh/yr 0 0 0

PYRTD MW 0.00 0.00 0.00

PY10 Incentives ($1,000) $2,669.00 $106.24 $0.13 Source: Navigant analysis


The standalone DR report,41 submitted to the Commission on January 15, 2019, details the impact evaluation methodology and results. Table 3-45 lists the days that DR events were called along with the verified gross demand reductions achieved by each event. It also lists the average DR performance for PY10 and for Phase III to date. PECO’s average DR performance to date is 173.12 MW, which exceeds the Phase III compliance reduction target of 161 MW by 8% (108% of target achieved to date).

Table 3-45. PY10 DR PYVTD Performance by Event


Small C&I DR (MW)

Large C&I DR (MW)

Portfolio (MW)


90% Confidence



Phase TD - Average Phase III DR Event Performance 31.60 0.41 141.11 173.12 10.3%


The Commission’s Phase III Implementation Order also established a requirement that EDCs achieve at least 85% of the Phase III compliance reduction target in each DR event. For PECO, this translates to a

41 PECO. Annual Report to the Pennsylvania Public Utility Commission Demand Response Performance Report Only. January 15, 2019. http://www.puc.pa.gov//pcdocs/1602629.pdf




137 MW minimum for each DR event. Figure 3-7 shows PY10 event performance relative to the compliance target.

Figure 3-7. Event Performance Compared to 85% Per-Event Target



Navigant conducted a full process evaluation in PY9. No process evaluation activities were conducted in PY10.


A detailed breakdown of program finances and cost-effectiveness is presented in Table 3-46 for Residential DR. Navigant calculated TRC benefits using gross verified impacts. Costs and benefits for PYTD results are expressed in 2018 dollars, while Phase III to date values are expressed as an NPV in 2016 dollars using a discount rate of 7.6%.



Table 3-46. Summary of Residential DR Finances – Gross Verified





Participant Costs (Net of Incentives/Rebates Paid by Utilities) -$667 -$1,987















Cost Subtotal $0 $0



Lifetime Electric Energy Benefits $0 $0





[1] Includes direct install equipment costs. [2] Includes direct costs attributable to plan and advance the programs. [3] Includes rebate processing, tracking system, general administration, program management, general management and legal, and technical assistance. [4] Includes the marketing CSP and marketing costs by program CSPs. EDC marketing costs broken out as a percentage of sector lifetime savings. This is an adjustment from the Preliminary Annual Report. [5] Direct program implementation costs. Labor, fuel, and vehicle operation costs for appliance recycling and direct install programs. [6] Total TRC Costs includes Total EDC Costs and Participant Costs. [7] Total TRC Benefits equals the sum of Total Lifetime Electric and Non-Electric Benefits. Benefits include avoided supply costs, including the reduction in costs of electric energy, generation, transmission, and distribution capacity, and natural gas valued at marginal cost for periods when there is a load reduction. NOTE: Savings carried over from Phase II are not to be included as a part of Total TRC Benefits for Phase III.



[8] TRC Ratio equals Total NPV TRC Benefits divided by Total NPV TRC Costs. Source: Navigant analysis

Table 3-47 presents program financials and cost-effectiveness on a net savings basis for Residential DR.

Table 3-47. Summary of Residential DR Finances – Net Verified





Participant Costs (Net of Incentives/Rebates Paid by Utilities) -$667 -$1,987















Cost Subtotal $0 $0


($1,000) Cost Total $3,363 $9,072


($1,000)





TRC Benefit-Cost Ratio [8] Benefits Total/Costs Total 0.86 0.58 [1] Includes direct install equipment costs. [2] Includes direct costs attributable to plan and advance the programs. [3] Includes rebate processing, tracking system, general administration, program management, general management and legal, and technical assistance. [4] Includes the marketing CSP and marketing costs by program CSPs. EDC marketing costs broken out as a percentage of sector lifetime savings. This is an adjustment from the Preliminary Annual Report. [5] Direct program implementation costs. Labor, fuel, and vehicle operation costs for appliance recycling and direct install programs.



[6] Total TRC Costs includes Total EDC Costs and Participant Costs. [7] Total TRC Benefits equals the sum of Total Lifetime Electric and Non-Electric Benefits. Benefits include avoided supply costs, including the reduction in costs of electric energy, generation, transmission, and distribution capacity, and natural gas valued at marginal cost for periods when there is a load reduction. NOTE: Savings carried over from Phase II are not to be included as a part of Total TRC Benefits for Phase III. [8] TRC Ratio equals Total NPV TRC Benefits divided by Total NPV TRC Costs. Source: Navigant analysis

A detailed breakdown of program finances and cost-effectiveness is presented in Table 3-48 for Small C&I DR. Navigant calculated TRC benefits using gross verified impacts. Costs and benefits for PYTD results are expressed in 2018 dollars, while Phase III to date values are expressed as an NPV in 2016 dollars using a discount rate of 7.6%.

Table 3-48. Summary of Small C&I DR Finances – Gross Verified




















Cost Subtotal $0 $0


($1,000) Cost Total $122 $324



Lifetime Electric Capacity Benefits $57 $49


Benefits Total $57 $49





[1] Includes direct install equipment cost. [2] Includes direct costs attributable to plan and advance the programs. [3] Includes rebate processing, tracking system, general administration, program management, general management and legal, and technical assistance. [4] Includes the marketing CSP and marketing costs by program CSPs. EDC marketing costs broken out as a percentage of sector lifetime savings. This is an adjustment from the Preliminary Annual Report. [5] Direct program implementation costs. Labor, fuel, and vehicle operation costs for appliance recycling and direct install programs. [6] Total TRC Costs includes Total EDC Costs and Participant Costs. [7] Total TRC Benefits equals the sum of Total Lifetime Electric and Non-Electric Benefits. Benefits include avoided supply costs, including the reduction in costs of electric energy, generation, transmission, and distribution capacity, and natural gas valued at marginal cost for periods when there is a load reduction. NOTE: Savings carried over from Phase II are not to be included as a part of Total TRC Benefits for Phase III. [8] TRC Ratio equals Total NPV TRC Benefits divided by Total NPV TRC Costs. Source: Navigant analysis

Table 3-49 presents program financials and cost-effectiveness on a net savings basis for Small C&I DR.

Table 3-49. Summary of Small C&I DR Finances – Net Verified




















Cost Subtotal $0 $0


($1,000) Cost Total $122 $324






Lifetime Electric Capacity Benefits $57 $49


Benefits Total $57 $49


[1] Includes direct install equipment costs and costs. [2] Includes direct costs attributable to plan and advance the programs. [3] Includes rebate processing, tracking system, general administration, program management, general management and legal, and technical assistance. [4] Includes the marketing CSP and marketing costs by program CSPs. EDC marketing costs broken out as a percentage of sector lifetime savings. This is an adjustment from the Preliminary Annual Report. [5] Direct program implementation costs. Labor, fuel, and vehicle operation costs for appliance recycling and direct install programs. [6] Total TRC Costs includes Total EDC Costs and Participant Costs. [7] Total TRC Benefits equals the sum of Total Lifetime Electric and Non-Electric Benefits. Benefits include avoided supply costs, including the reduction in costs of electric energy, generation, transmission, and distribution capacity, and natural gas valued at marginal cost for periods when there is a load reduction. NOTE: Savings carried over from Phase II are not to be included as a part of Total TRC Benefits for Phase III. [8] TRC Ratio equals Total NPV TRC Benefits divided by Total NPV TRC Costs. Source: Navigant analysis

A detailed breakdown of program finances and cost-effectiveness is presented in Table 3-50 for Large C&I DR. Navigant calculated TRC benefits using gross verified impacts. Costs and benefits for PYTD results are expressed in 2018 dollars, while Phase III to date values are expressed as an NPV in 2016 dollars using a discount rate of 7.6%.

Table 3-50. Summary of Large C&I DR Finances – Gross Verified



EDC Incentives to Participants [1] $0 $1,707


Participant Costs (Net of Incentives/Rebates Paid by Utilities) $0 -$427

Cost Subtotal $0 $1,280

















Cost Subtotal $0 $0








[1] Large C&I DR incentives were not clearly tracked by the CSP during PY10. Navigant conservatively assumes that all Large C&I DR program spending be considered administrative costs based on the 2016 TRC Guidance document. Therefore, program spending break outs for the purpose of TRC calculations matches PECO’s PY10 Preliminary Annual Report. [2] Includes direct costs attributable to plan and advance the programs. [3] Includes rebate processing, tracking system, general administration, program management, general management and legal, and technical assistance. [4] Includes the marketing CSP and marketing costs by program CSPs. EDC marketing costs broken out as a percentage of sector lifetime savings. This is an adjustment from the Preliminary Annual Report. [5] Direct program implementation costs. Labor, fuel, and vehicle operation costs for appliance recycling and direct install programs. [6] Total TRC Costs includes Total EDC Costs and Participant Costs. [7] Total TRC Benefits equals the sum of Total Lifetime Electric and Non-Electric Benefits. Benefits include avoided supply costs, including the reduction in costs of electric energy, generation, transmission, and distribution capacity, and natural gas valued at marginal cost for periods when there is a load reduction. NOTE: Savings carried over from Phase II are not to be included as a part of Total TRC Benefits for Phase III. [8] TRC Ratio equals Total NPV TRC Benefits divided by Total NPV TRC Costs. Source: Navigant analysis

Table 3-51 presents program financials and cost-effectiveness on a net savings basis for Large C&I DR.

Table 3-51. Summary of Large C&I DR Finances – Net Verified



EDC Incentives to Participants [1] $0 $1,707


Participant Costs (Net of Incentives/Rebates Paid by Utilities) $0 -$427

Cost Subtotal $0 $1,280

















Cost Subtotal $0 $0


($1,000) Cost Total $4,960 $7,493


($1,000)






[1] Large C&I DR incentives were not clearly tracked by the CSP during PY10. Navigant conservatively assumes that all Large C&I DR program spending be considered administrative costs based on the 2016 TRC Guidance document. Therefore, program spending break outs for the purpose of TRC calculations matches PECO’s PY10 Preliminary Annual Report. [2] Includes direct costs attributable to plan and advance the programs. [3] Includes rebate processing, tracking system, general administration, program management, general management and legal, and technical assistance. [4] Includes the marketing CSP and marketing costs by program CSPs. EDC marketing costs broken out as a percentage of sector lifetime savings. This is an adjustment from the Preliminary Annual Report. [5] Direct program implementation costs. Labor, fuel, and vehicle operation costs for appliance recycling and direct install programs. [6] Total TRC Costs includes Total EDC Costs and Participant Costs. [7] Total TRC Benefits equals the sum of Total Lifetime Electric and Non-Electric Benefits. Benefits include avoided supply costs, including the reduction in costs of electric energy, generation, transmission, and distribution capacity, and natural gas valued at marginal cost for periods when there is a load reduction. NOTE: Savings carried over from Phase II are not to be included as a part of Total TRC Benefits for Phase III. [8] TRC Ratio equals Total NPV TRC Benefits divided by Total NPV TRC Costs. Source: Navigant analysis


The impact evaluation activities in PY10 led to several findings and recommendations from Navigant to PECO. Table 3-52 presents those findings and recommendations along with a summary of how PECO plans to address the recommendations in program delivery.



Table 3-52. Summary of Findings and Recommendations for the DR Programs

Program Finding Recommendation EDC Status

Residential DR

A large percentage of the advanced metering infrastructure meter data contained integers.

Prioritize the conversion for the AC Saver watt-hour population.

Complete

Small C&I DR

Technical issues within the CSP software prevented Events 1 and 2 from being executed successfully.

Work with Small C&I EE Program CSP to identify the cause of the technical issues and establish a plan to prevent the issue in the future.

In process

Large C&I DR

Meter data was unavailable for two sites, limiting the ability to evaluate impacts for those sites.

Investigate issues with onsite metering equipment for those sites in advance of the PY11 DR season.

Complete




4. SUMMARY OF FINANCES Section 4 provides an overview of the expenditures associated with PECO’s portfolio and the recovery of those costs from ratepayers.

4.1 Program Financials

Program-specific and portfolio total finances for PY10 are shown in Table 4-1. The columns in Table 4-1 and Table 4-2 are adapted from the Direct Program Cost categories in the Commission’s EE&C Plan template42 for Phase III. EDC Materials, Labor, and Administration includes costs associated with an EDC’s own employees. Implementation Conservation Service Provider (ICSP) Materials, Labor, and Administration includes both the program implementation contractor and the costs of any other outside vendors EDCs employ to support program delivery.

Table 4-1. PYTD Financials

Program Incentives to

Participants and Trade Allies

($1,000)

EDC Materials, Labor, and

Administration ($1,000)

ICSP Materials, Labor, and


Total Cost ($1,000)

Residential EE $8,782 $5,730 $11,432 $25,943 Low-Income EE $130 $490 $7,237 $7,857 Small C&I EE $2,989 $2,242 $4,180 $9,411 Large C&I EE $5,250 $896 $4,750 $10,897 CHP $878 $0 $44 $922 Residential DR $2,669 $1 $1,360 $4,030 Small C&I DR $106 $0 $42 $149 Large C&I DR $0 $2 $4,958 $4,961 Common Portfolio Costs1 N/A N/A N/A $9,616 Portfolio Total $20,804 $9,361 $34,004 $73,785 SWE Costs2 N/A N/A N/A $0 Total $20,804 $9,361 $34,004 $73,785

1 Includes the administrative CSP, tracking system, general administration, and clerical costs; EDC program management; CSP program management; general management; oversight of major accounts; and technical assistance. 2 SWE costs are outside of the 2% spending cap. Sources: PECO’s eTrack database, CSP tracking data

42 EE&C Plan Template. Section 10: http://www.puc.pa.gov/pcdocs/1372426.doc

http://www.puc.pa.gov/pcdocs/1372426.doc



Table 4-2 shows program-specific and portfolio total finances since the inception of Phase III.

Table 4-2. Phase III to Date Financials

Program Incentives to

Participants and Trade Allies

($1,000)

EDC Materials, Labor, and


ICSP Materials, Labor, and


Total Cost ($1,000)

Residential EE $21,298 $17,062 $33,401 $71,762 Low-Income EE $1,035 $1,210 $21,887 $24,131 Small C&I EE $5,893 $6,569 $10,673 $23,136 Large C&I EE $9,912 $2,209 $13,417 $25,539 CHP $1,089 $0 $87 $1,176 Residential DR $8,513 $33 $3,338 $11,884 Small C&I DR $343 $2 $92 $437 Large C&I DR $0 $65 $8,675 $8,740 Common Portfolio Costs1 N/A N/A N/A $28,807 Portfolio Total $48,085 $27,151 $91,570 $195,612 SWE Costs2 N/A N/A N/A $700

Total $48,085 $27,151 $91,570 $196,312 1 Includes the administrative CSP, tracking system, general administration, and clerical costs; EDC program management; CSP program management; general management; oversight of major accounts; and technical assistance. 2 SWE costs are outside of the 2% spending cap. Sources: PECO’s eTrack database, CSP tracking data

4.2 Cost Recovery

Act 129 allows Pennsylvania EDCs to recover EE&C Plan costs through a cost recovery mechanism. PECO’s cost recovery charges are organized separately by four customer sectors to ensure that the electric rate classes that finance the programs are the rate classes that receive the direct energy and conservation benefits. Cost recovery is governed by a tariffed rate class, so it is necessarily tied to the way customers are metered and charged for electric service. Readers should be mindful of the differences between Table 4-3 and Section 2.4. For example, the low-income customer segment is a subset of PECO’s residential tariff(s) and, therefore, is not listed in Table 4-3.



Table 4-3. EE&C Plan Expenditures by Cost Recovery Category43

Parameter Cost Recovery Sector

Rate Classes Included

Program Year

PY8 PY9 PY10 PY11 PY12 Phase III to Date 1

EE&C Plan Expenditures ($1,000 Nominal)

Residential R, RH, and CAP $35,450 $43,217 $41,675 - - $120,342

Small C&I GS $7,0352 $11,105 $12,024 - - $30,164

Large C&I PD, HT, and EP $9,713 $15,250 $20,056 - - $45,019

Municipal SLE, AL, and TLCL $28 $31 $30 - - $89

Portfolio Total All $52,225 $69,602 $73,785 - - $195,612 1 Phase III to date values expressed as the sum of nominal dollars. 2 As noted in the PY9 Preliminary Annual Report, Navigant determined that the rounded financial expenditure for the Small C&I EE Program was $882 above actual expenditures. The correction of this value, reflected here, results in a decrease in reported PY8 portfolio expenditures from 52,226 to 52,225 (in $1,000 units). Source: PECO

43 SWE costs not included.


©2019 Guidehouse Inc. Page A-1

APPENDIX A. UPSTREAM LIGHTING CROSS-SECTOR SALES

Navigant completed its analysis of the upstream lighting cross-sector sales estimation as part of the PY8 evaluation for the Lighting, Appliances & HVAC Solution. Navigant applied the PY8 cross-sector sales values to PY10. Details about the evaluation, including the cross-sector sales assumptions for the solution, can be found in Appendix E.1.


©2019 Guidehouse Inc. Page B-1

APPENDIX B. SITE INSPECTION SUMMARY

Table B-1 presents the site inspection summary and common discrepancies found during the evaluation.

Table B-1. PY10 Site Inspection Summary

Program/Solution Inspection Firm Number of

Inspections Conducted

Number of Sites with Discrepancies

from Reported Values

Summary of Common Discrepancies

Residential Whole Home Navigant 14 9

Pre/post quantities, mislabeled bulb locations, mislabeled bulb type

Small C&I/ Equipment and Systems1

Navigant/INCA 14 10 HOU and CFs

Small C&I/ New Construction1 Navigant/INCA 4 4 HOU, fixture quantities,

interactive factors

Small C&I/ Data Centers1 Navigant/INCA 1 1 Equipment quantity

Large C&I/ Equipment and Systems1

Navigant/INCA 19 12

Ex ante calculation methodology, HOU, CFs, fixture quantities, fixture wattages

Large C&I/ New Construction1 Navigant/INCA 5 4

HVAC equipment capacity, quantity, and HOU

Large C&I/ Data Centers1 Navigant/INCA 1 1 Ex ante calculation

methodology

Multifamily Targeted Market Segment2

Navigant/Mondre Energy 29 14 Pre/post fixture quantities,

space-specific HOU

Combined Heat and Power Navigant/INCA 3 3 Engineering assumptions

Total 90 55 1 For C&I projects, many projects had small discrepancies from the reported values, with few projects showing 100% realization rates for both energy and demand savings. The numbers listed here include projects whose energy or demand savings were more than 20% different from the ex ante results. 2 For the Multifamily Targeted Market Segment, the evaluation team conducted site inspections for projects under all three programs: Residential EE, Large C&I EE, and Small C&I EE. The buildings were sub-sampled at the project level. If field technicians observed a large discrepancy at a single sub-sampled project, the team reported the building as having a discrepancy from reported values Source: Navigant Analysis.


©2019 Guidehouse Inc. Page C-1

APPENDIX C. HER IMPACT EVALUATION DETAIL

Navigant completed an analysis of the Behavioral Solution (i.e., the HER impact evaluation) as part of its overall solution-level evaluation. Details about the evaluation, including the regression analysis results, can be found in Appendix E.5.


©2019 Guidehouse Inc. Page D-1

APPENDIX D. PARTICIPATION COUNTS

Across PECO’s portfolio, there are differences in how participation is calculated across solutions and CSPs, ranging from:

• Number of measures sold (see Lighting, Appliances & HVAC – Lighting)

• Number of rebates issued (see Lighting, Appliances & HVAC – Appliances and HVAC)

• Number of unique premises served (see Whole Home)

• Number of orders on distinct days (see Appliance Recycling)

• Number of participants (see Residential New Construction and Behavioral)

• Number of utility accounts served (see Multifamily Targeted Market Segment)

• Number of projects (see Small and Large C&I EE Solutions) Table D-1 provides an overview of the different participation definitions by program and solution.

Table D-1. Overview of Participation Definitions

Program Solution Conservation Service Provider Participation Definition

Residential

LAH (Lighting) CLEAResult Sum number of total lamp packs sold

LAH (Appliances and HVAC) CLEAResult Count of rebates issued

Whole Home CLEAResult Count of unique premise ID

Appliance Recycling ARCA Count of all orders on distinct days

New Construction PSD Sum No. of participants

Behavioral Oracle Sum No. of participants


Franklin Distinct count of utility account ID by program, solution, and invoice number

Low-Income

Whole Home CMC Count of unique premise numbers for component 1 and 2

Whole Home ARCA Count of all orders on distinct days

Whole Home ECA Count unique premise numbers



Program Solution Conservation Service Provider Participation Definition

Small C&I

Equipment and Systems ICF Count of unique project number

New Construction ICF Count of unique Project number

Whole Building SmartWatt Count of unadjusted projects



Large C&I

Equipment and Systems ICF Count of unique project number

New Construction ICF Count of unique project number


ICF Count of unique project number



Source: Navigant Analysis

Five solutions and one targeted market segment make up the Residential EE Program: Lighting, Appliances & HVAC Solution, Appliance Recycling Solution, Whole Home Solution, New Construction Solution, Behavioral Solution, and the Multifamily Targeted Market Segment. PECO defined participation counts in each solution as follows:

• For Lighting, Appliances & HVAC, upstream lighting participation is defined as the sum of the SKU sales. A SKU describes a sold lighting product, which can be a single bulb or a multi-pack of bulbs. For the Appliances and HVAC participants, participation is defined as the total number of non-adjusted records in PECO’s tracking data with an associated bill account number. A record may represent one or more rebated items (e.g., a single participant purchasing multiple thermostats during the same purchase event).

• For Appliance Recycling, a participant is a customer who schedules a pickup for one or more units. If the same customer initiates multiple pickup orders during the year, each order is counted as an individual participant. However, if a customer initiates more than one order in the same day it counts as a single participant.

• For Residential Whole Home, a participant is considered a unique project number for non-adjusted records with a project type that does not include other installations or central air conditioner (CAC) other installations.

• For Residential New Construction, a participant is a new home.

• For Behavioral, a participant is a utility account included in the program’s treatment group.

• For the Multifamily Targeted Market Segment, a participant is a unique combination of utility account ID and invoice number.

Two solutions make up the Low-Income EE Program: Lighting and Whole Home. Low-income participants are those participants with incomes at or below 150% of the FPL. PECO has defined participation counts in each solution as follows:



• The Low-Income Lighting Program was discontinued in PY9. For Phase III, participation is defined as a package of one or more light bulbs identified by a unique SKU number. As in the Residential EE Program, a SKU describes a sold lighting product that can be a single bulb or a multi-pack of bulbs.

• For Low-Income Whole Home, a participant is considered:

o Free Home Energy Checkups and Low-Income Usage Reduction Program (LIURP): A unique premise number (for both multifamily and single-family audits).

o Appliance Recycling: A low-income Appliance Recycling customer who schedules pickup for one or more units. If the same customer initiates multiple pickup orders during the year, each order is counted as an individual participant. However, if a customer initiates more than one order in the same day it counts as a single participant.

Product giveaways are also part of the Whole Home Solution but are not included in the participant count.

Four solutions and two targeted market segments make up the Small C&I EE Program: Equipment and Systems Solution, Whole Building Solution, Behavioral Solution, New Construction Solution, Data Centers Targeted Market Segment, and Multifamily Targeted Market Segment. The Behavioral Solution is not currently active. PECO has defined participation counts in each active solution as follows:

• For Small C&I Equipment and Systems, participation is defined as an activity with a unique project number. More than one measure per participant is permitted, with the impact sample defined on the project level.

• For Small C&I Whole Building, participation is defined as an activity with a unique project number. More than one measure per participant is permitted, with the impact sample defined on the project level.

• For Small C&I New Construction, participation is defined as an activity with a unique project number. More than one measure per participant is permitted, with the impact sample defined on the project level.

• For the Data Centers Targeted Market Segment, participation is defined as an activity with a unique project number. More than one measure per participant is permitted, with the impact sample defined on the project level.

• For the Multifamily Targeted Market Segment, participation is defined as an activity with a unique combination of utility account ID and invoice number. More than one measure per participant is permitted. A building may consist of multiple participants with measures installed in the dwellings and common areas of master-metered multifamily buildings.

Two solutions and two targeted market segments make up the Large C&I EE Program: Equipment and Systems Solution, New Construction Solution, Data Centers Targeted Market Segment, and Multifamily Targeted Market Segment. PECO has defined participation counts in each solution as follows:

• For Large C&I Equipment and Systems, participation is defined as an activity with a unique project number. More than one measure per participant is permitted, with the impact sample defined on the project level.

• For Large C&I New Construction, participation is defined as an activity with a unique project number. More than one measure per participant is permitted, with the impact sample defined on the project level.



• For the Data Centers Targeted Market Segment, participation is defined as an activity with a unique project number. More than one measure per participant is permitted, with the impact sample defined on the project level.

• For the Multifamily Targeted Market Segment, participation is defined as an activity with a unique combination of utility account ID and invoice number. More than one measure per participant is permitted. A building may consist of multiple participants with measures installed in the dwellings and common areas of master-metered multifamily buildings.

The CHP Program consists of the CHP Solution only. PECO has defined participation counts in the solution as follows:

• For CHP, participation is defined as an activity with a unique project number. Projects that consist of multiple prime movers at a single facility are classified as a single project.

Three solutions make up the Residential DR Program; however, only the DLC Solution is currently active. PECO has defined participation counts in the solution as follows:

• For Residential DLC, a participant is defined as a unique account number where device status is recorded in the PECO database as installed or swapped and the measure code is CACS (central air conditioner switch). One participant may have more than one DLC device installed at the home. Customers whose accounts are disconnected, have opted out of the program, or for whom the DLC device was removed are not counted as participants.

The Small C&I DR Program consists of the Small C&I DLC Solution. PECO has defined participation counts in the solution as follows:

• For Small C&I DLC, a participant is defined as a unique account number where device status is recorded in the PECO database as installed or swapped and the measure code is PCT (programmable communicating thermostat). One participant may have more than one DLC device installed on the premise. Customers whose accounts are disconnected, have opted out of the program, or for whom the DLC device was removed are not counted as participants.

The Large C&I DR Program consists of the DRA Solution. PECO has defined participation counts in the solution as follows:

• For DRA, a participant is defined as a large C&I customer (defined by PECO account number) enrolled with a DR program CSP for at least 1 hour of at least one event occurring in any given program year.

©2019 Guidehouse Inc.

Final Annual Report to the Pennsylvania Public Utility Commission Phase III of Act 129 Appendix E: Residential Energy Efficiency Program Detail Appendix F: Residential Low-Income EE Program Detail Appendix G: Small and Large C&I EE Programs Detail Appendix H: Multifamily Targeted Market Segment Detail

Program Year 10

(June 1, 2018 - May 31, 2019)

For Pennsylvania Act 129 of 2008

Energy Efficiency and Conservation Plan

Prepared for:

Prepared by: Navigant, A Guidehouse Company 1375 Walnut Street Suite 100 Boulder, Colorado 80302

303.728.2500 guidehouse.com

November 15, 2019

Final Annual Report to the Pennsylvania Public Utility Commission: Phase III of Act 129 – Appendix E-H

©2019 Guidehouse Inc. Page i

TABLE OF CONTENTS

Appendix E. Residential Energy Efficiency Program Detail ...................................E-1

E.1 Lighting, Appliances & HVAC Solution ..................................................................................... E-1 E.1.1 Lighting ..........................................................................................................................E-2 E.1.2 Appliances & HVAC ......................................................................................................E-6

E.2 Appliance Recycling Solution ................................................................................................... E-7 E.2.1 Gross Impact Evaluation ...............................................................................................E-8 E.2.2 Net Impact Evaluation ...................................................................................................E-9 E.2.3 Process Evaluation .......................................................................................................E-9

E.3 Whole Home Solution ............................................................................................................... E-9 E.3.1 Gross Impact Evaluation .............................................................................................E-10 E.3.2 Net Impact Evaluation .................................................................................................E-12 E.3.3 Process Evaluation .....................................................................................................E-12

E.4 New Construction Solution...................................................................................................... E-13 E.4.1 Gross Impact Evaluation .............................................................................................E-13 E.4.2 Net Impact Evaluation Methodology and Results .......................................................E-15 E.4.3 Process Evaluation .....................................................................................................E-15

E.5 Behavioral Solution ................................................................................................................. E-15 E.5.1 Impact Evaluation Methodology ..................................................................................E-17 E.5.2 Summary Statistics and Results .................................................................................E-17 E.5.3 Dual Participation Analysis..........................................................................................E-20 E.5.4 Behavioral Program Impacts .......................................................................................E-21

Appendix F. Residential Low-Income EE Program ................................................. F-1

F.1 Whole Home Solution ............................................................................................................... F-1 F.1.1 Gross Impact Evaluation ............................................................................................... F-1 F.1.2 Process Evaluation ........................................................................................................ F-4

Appendix G. Small and Large C&I EE Programs .................................................... G-1

G.1 Equipment and Systems Solution ............................................................................................ G-1 G.1.1 Gross Impact Evaluation .............................................................................................. G-2 G.1.2 Net Impact Evaluation .................................................................................................. G-5 G.1.3 Process Evaluation ...................................................................................................... G-5

G.2 New Construction Solution ....................................................................................................... G-5 G.2.1 Gross Impact Evaluation .............................................................................................. G-6 G.2.2 Net Impact Evaluation .................................................................................................. G-8 G.2.3 Process Evaluation ...................................................................................................... G-8

G.3 Whole Building Solution ............................................................................................................ G-9 G.3.1 Gross Impact Evaluation .............................................................................................. G-9 G.3.2 Net Impact Evaluation ................................................................................................ G-10 G.3.3 Process Evaluation .................................................................................................... G-10

G.4 Data Centers Targeted Market Segment ............................................................................... G-11 G.4.1 Gross Impact Evaluation ............................................................................................ G-11 G.4.2 Net Impact Evaluation ................................................................................................ G-11 G.4.3 Process Evaluation .................................................................................................... G-11

Appendix H. Multifamily Targeted Market Segment ............................................... H-1

H.1 Gross Impact Evaluation ........................................................................................................... H-1


©2019 Guidehouse Inc. Page ii

H.1.1 Methodology ................................................................................................................ H-1 H.1.2 Sampling ...................................................................................................................... H-2 H.1.3 Findings........................................................................................................................ H-3

H.2 Net Impact Evaluation ............................................................................................................... H-3 H.2.1 Methodology ................................................................................................................ H-3 H.2.2 Sampling ...................................................................................................................... H-4 H.2.3 Findings........................................................................................................................ H-4

H.3 Process Evaluation ................................................................................................................... H-5 H.3.1 Methodology ................................................................................................................ H-5 H.3.2 Sampling Stratification ................................................................................................. H-6 H.3.3 Findings........................................................................................................................ H-7

FIGURES Figure H-1. Tenant Multifamily Targeted Market Segment Participant Satisfaction with Program Components, n=65 ................................................................................................................................... H-8 Figure H-2. Landlord Multifamily Targeted Market Segment Participant Satisfaction with Program Components, n=8 ..................................................................................................................................... H-9 Figure H-3. Tenant Multifamily Targeted Market Segment Participant Satisfaction with PECO, n=65 ... H-9 Figure H-4. Landlord Multifamily Targeted Market Segment Participant Satisfaction with PECO, n=8 H-10 Figure H-5. Tenant Likelihood of Recommending the Multifamily Targeted Market Segment to Others, n=65 ....................................................................................................................................................... H-10 Figure H-6. Landlord Likelihood of Recommending the Multifamily Targeted Market Segment to Others, n=8 ......................................................................................................................................................... H-11

TABLES Table E-1. Cross-Sector Sales of Standard and Specialty LEDs ............................................................ E-3 Table E-2. Residential Lighting Summary of Desk Review Activities and Findings ................................ E-5 Table E-3. Residential Non-Lighting Summary of Desk Review Activities .............................................. E-6 Table E-4. Residential EE Program Whole Home Solution Tracking Database Review ....................... E-10 Table E-5. Behavioral Solution Treatment Group Counts by Cohort ..................................................... E-16 Table E-6. Behavioral Solution Cohort Regression Details – Waves 1-4 .............................................. E-17 Table E-7. Behavioral Solution Cohort Regression Details – Waves 5 and AC Saver ......................... E-18 Table E-8. Behavioral Solution Cohort Percent Savings – Waves 1-4 .................................................. E-18 Table E-9. Behavioral Solution Cohort Percent Savings – Waves 5 and AC Saver ............................. E-19 Table E-10. Behavioral Solution Monthly Verified Modeled Savings ..................................................... E-19 Table E-11. Default Upstream Adjustment Factors ............................................................................... E-20 Table E-12. Downstream and Upstream Savings Adjustments ............................................................. E-21 Table E-13. Behavioral Solution Net Impacts ........................................................................................ E-21 Table F-1. Low-Income EE Program Whole Home Solution Tracking Database Review ....................... F-2 Table G-1. Small C&I Whole Building Solution Summary of Desk Review Activities ............................ G-10 Table H-1. Multifamily Targeted Market Segment Impact Evaluation Strata ........................................... H-2 Table H-2. Multifamily Targeted Market Segment Updated Sample Size ............................................... H-3 Table H-3. Net-to-Gross Results for the Multifamily Targeted Market Segment ..................................... H-5 Table H-4. Multifamily Targeted Market Segment Tenant Customer Experience Strata ........................ H-6 Table H-5. Multifamily Targeted Market Segment Landlord Customer Experience Strata ...................... H-6


©2019 Guidehouse Inc. Page E-1

APPENDIX E. RESIDENTIAL ENERGY EFFICIENCY PROGRAM DETAIL This appendix provides additional detail on the evaluation sample design, methods, and activities deployed in program year 10 (PY10) for select Residential Energy Efficiency (EE) Program solutions (listed below). The reader should refer to Section 3.1 in the main body report for key evaluation findings, results, and conclusions.

• Lighting, Appliances & HVAC Solution

• Appliance Recycling Solution

• Whole Home Solution

• New Construction Solution

• Behavioral Solution The Multifamily Targeted Market Segment also contributes to the Residential EE Program; however, given its cross-program nature, evaluation details for that solution are included in Appendix H.

E.1 Lighting, Appliances & HVAC Solution

The Lighting, Appliances & HVAC (LAH) Solution offers customers energy savings opportunities by assisting customers in purchasing the most efficient technology when they are shopping for new products. The solution provides both upstream and downstream incentives:

• Upstream: Point of purchase discounts to increase the market share of ENERGY STAR-qualified LEDs

• Downstream: Rebates for appliances sold through retail and HVAC installer sales channels Appliances and HVAC equipment rebated by the program include central air conditioners (ACs), central heat pumps, ENERGY STAR refrigerators, and high efficiency furnace fans. The solution also distributes educational materials that increase customer awareness and acceptance. In PY10, PECO launched the PECO Marketplace as a way for customers to shop for energy efficient technologies online. Simple Energy, a conservation service provider (CSP), operates the PECO Marketplace and offers instant discounts on ENERGY STAR-qualified LED bulbs, advanced power strips, and program-eligible programmable thermostats. For purposes of this evaluation, participation in the upstream lighting component of LAH is defined as the sum of stock keeping unit (SKU) sales. A SKU describes a sold lighting product that can be a single bulb or a multi-pack of bulbs.1 For Appliances and HVAC participants, including power strips and thermostats sold through the PECO Marketplace, participation is defined as the total number of non-adjusted records in PECO’s tracking data. A record may represent one or more rebated items (e.g., a single participant purchasing multiple thermostats during the same purchase event).

1 This definition also holds true for LED bulbs sold through the PECO Marketplace.



E.1.1 Lighting

The following subsection presents the evaluation details and findings for the lighting component of the Lighting, Appliances & HVAC Solution. Navigant Consulting, Inc., n/k/a Guidehouse Inc. (Navigant)2, conducted the following activities to verify the gross impacts and ensure CSP reporting accuracy.

• Database reviews

• Record-level Technical Reference Manual (TRM) review

• ENERGY STAR certification verification

• Invoice review

• Incentive analysis Overall, these analysis activities verified the solution’s reported savings through a bulb-level, bottom-up recalculation of energy and demand impacts for all program bulbs incented by PECO during PY10.

E.1.1.1 Record-Level Technical Reference Manual (TRM) Review

Navigant applied energy and demand savings algorithms to verified input parameters as outlined in the Pennsylvania TRM (PA TRM) to calculate impacts. The evaluation team used commercial and industrial (C&I) facility lighting usage assumptions described in the PA TRM to calculate savings for the portion of bulbs purchased by non-residential customers as estimated by PY8 in-store intercept efforts. The team adjusted baseline wattages for some bulbs based on bulb characteristics and the TRM methodology for assigning baseline watts. Navigant applied all other TRM parameters consistent with the methodology outlined in the TRM. The evaluation team used the following methodology to verify and update baseline wattages:

1. Navigant generated a list of unique bulb model/description/lumens/watts/type from the tracking data, resulting in 1,433 unique models.

2. Using the bulb type and lumens, the team applied baseline wattages based on the TRM assumptions.

3. The evaluation team reviewed the bulb classification and reclassified products as necessary to determine an appropriate baseline wattage. When the lumen values fell out of the ranges specified in the TRM, the team applied a baseline wattage equal to the advertised replacement wattage on the product. Adjustments were made for the following products:

a. A-Line Omni, PAR30, BR40, PAR38 (four models): Navigant found four models that appeared to have typos in the baseline wattage in the reported savings. All values were off by a single watt.

b. BR20 reflectors (five models): Navigant classified BR20 reflectors as exempt products because the diameter is not less than 2.5”.

c. Candelabra base (four models): Navigant found specialty decorative products with candelabra bases that should be classified as exempt products.

2 On October 11, 2019, Guidehouse LLP completed its previously announced acquisition of Navigant Consulting Inc. In the months ahead, we will be working to integrate the Guidehouse and Navigant businesses. In furtherance of that effort, we recently renamed Navigant Consulting Inc. as Guidehouse Inc.



d. Globe (three models): Navigant classified globe products less than 5” and greater than 750 lumen as non-exempt. These models were originally classified as exempt.

e. Globe (27 models): Navigant found several globe products less than 750 lumens that should be classified as exempt. These models were originally classified as non-exempt.

f. High lumens (three models): Navigant found reflectors with an output higher than 1,300 lumens and applied replacement wattages.

g. Medium-sized reflectors (seven models): Navigant classified R30, PAR38, and R40 bulbs as generic reflectors and applied TRM baseline wattages.

h. Downlight fixtures (23 models): Navigant classified downlight fixtures as BR30-equivalent based on common replacements in the market.

i. Small reflectors (29 models): For reflectors smaller than 2.5 inches, Navigant applied replacement wattages.

These adjustments had a minor (<1%) effect on the lighting verified program savings. As a final step to calculating verified savings, Navigant incorporated cross-sector sales. Per the Phase III Evaluation Plan3, the evaluation team borrowed these values from the PY8 evaluation because the team did not perform primary research into these values in PY9 or PY10. Navigant applied the cross-sector sales values on a per-retailer and per-bulb type basis, as detailed in Table E-1. For lighting sales through the PECO Marketplace, Navigant assumed zero cross-sector sales.

Table E-1. Cross-Sector Sales of Standard and Specialty LEDs

Stratum Retailers Cross-Sector Sales – Standard LED

Cross-Sector Sales – Specialty LED

Ba

Ace Hardware, BJ’s Wholesale Club, Costco, Do it Best, The Home Depot, Lowe’s Home Improvement, Sam’s Club, True Value Hardware

0.007 0.02

Bb Batteries Plus Bulbs, Dollar General, Dollar Tree, Ollie’s Bargain Outlet, Target, Walgreens, Walmart

0.002 0.011

Bc Goodwill, Grocery Outlet, hard-to-reach retailers, independent hardware, Simple Energy (Marketplace)

0 0


For the C&I savings contributions, Navigant used hours of use (HOU) and coincident factor (CF) values that were slightly different than those used in PY9; the Statewide Evaluator (SWE) requested one of the building types from the PY8 in-store intercept surveys be reclassified. This reclassification resulted in slightly lower HOU (2,607 vs. 2,711) and CF (0.45 vs. 0.47) compared to PY9.




E.1.1.2 ENERGY STAR Certification Verification

The Phase III TRM stipulates that only ENERGY STAR-certified products be incented through the Lighting Solution; therefore, Navigant independently reviewed the unique SKUs for all PY10 program bulbs to ensure they met this requirement. The evaluation team performed several automated and manual checks to verify ENERGY STAR certification:

1. Navigant generated a list of unique SKUs and compared it against the current (as of September 2019) ENERGY STAR bulb and fixture lists for direct matches based on model number.

2. The evaluation team then compared the list against archived (December 2017 through December 2019) ENERGY STAR bulb and fixture lists for additional direct matches based on model number.

3. As a final automated check, Navigant performed a fuzzy match algorithm to find instances where a model number is present in one of the ENERGY STAR database fields representing additional model information.

4. The team manually reviewed any remaining unmatched SKUs (12 total) using web searches based on product information and by reaching out to program and CSP staff at PECO for confirmation. Of these products, Navigant found that all are advertised as ENERGY STAR-certified or could be matched to ENERGY STAR-certified products based on similarities between model numbers and detailed bulb information.

Navigant confirmed that all PY10 program bulbs are ENERGY STAR-certified models. The majority (1,421 out of 1,433 unique SKUs) mapped directly or indirectly based on model number to the ENERGY STAR bulb and fixture lists, with manual searches confirming the remainder.

E.1.1.3 Invoice Review

Navigant verified program database-reported incentive spend against PECO-provided retailer invoices. The evaluation team did not find any discrepancies based on incentives paid. Because only incentives were provided in the invoice documentation, Navigant could not directly verify quantities; thus, incentives are deemed as an appropriate surrogate for quantity.

E.1.1.4 Incentive Analysis

Navigant analyzed the minimum, maximum, and average incentive for each retailer and bulb type combination to ensure that incentives align with PECO’s Phase III Energy Efficiency and Conservation (EE&C) Plan. The evaluation team found no cases where the incentives fell outside of the plan’s guidelines. The team also conducted a pricing and incentive review to compare the per-bulb incentives with the manufacturer suggested retail price (MSRP) and the expected retail price.4 Navigant expected that the MSRP less the per-package incentive would equal the expected retail price. However, this equation does not hold true for 14,098 (7.7% of total) records. Furthermore, there were 1,308 cases where the expected retail price was higher than the MSRP (0.7% of records). The evaluation team found similar issues in PY8 and PY9 and learned there could be a manufacturer rebate to bring down the expected retail price more 4 The expected retail price is the actual price that the customer pays after incentives (including the PECO incentive and any other incentive, such as a manufacturer rebate).



than the PECO incentive. Through discussions with the PECO program manager, Navigant identified opportunities for PECO to continue working with the CSP and manufacturers to encourage maintenance of accurate inputs, communication of other manufacturer rebates, and updates to prices as often as possible to ensure these differences are not erroneous. The team also discussed with PECO opportunities for the CSP to institute a quality control (QC) check during data processing that would indicate an error when the MSRP is lower than the expected retail price, which should never be the case.

E.1.1.5 Results Summary

Table E-2 summarizes the results of these desk review activities.

Table E-2. Residential Lighting Summary of Desk Review Activities and Findings

Activity Finding Description

Record-Level Savings Review

Navigant adjusted baseline wattages for several bulbs (105 out of 1,433).

Navigant adjusted baseline wattages for a variety of lamp types. The effects of this adjustment were minor (<1%) and demonstrate that the program is accurately characterizing impacts.

A small quantity (39) of records listed pack quantity as bulb quantity in the tracking data.

Navigant confirmed with PECO program staff that there was a quantity error in one batch, affecting 39 records. The team updated the quantity to reflect the bulb quantity rather than the pack quantity and used these updated values in the final verified savings calculation.

Eleven records (six SKUs) were reported with different baseline wattages than what was used to calculate reported energy and demand savings.

Navigant could not directly replicate claimed energy and demand savings for 11 records (out of 181,268) due to inconsistencies in reported baseline wattage, affecting less than 0.01% of savings.

ENERGY STAR Certification Verification

No issues; difficulty identifying some products based on model number.

The team confirmed all program bulbs in PY10 were ENERGY STAR-certified models. The majority (1,421 out of 1,433 unique SKUs) mapped directly or indirectly based on model number to ENERGY STAR bulb and fixture lists, with manual searches confirming the remainder.

Invoice Review No issues.

Navigant reviewed the incentives to confirm that the invoiced amount from manufacturers equals the invoiced amount in the CSP data. The evaluation team found no discrepancies.

Incentive Analysis

No issues; opportunity to improve accuracy of MSRP and expected retail price after incentives in databases.

Navigant compiled the minimum, maximum, and average incentive for each retailer/bulb category/bulb type combination to ensure that incentives align with PECO’s Phase III EE&C Plan. The team found no cases where the incentives fell outside of the plan’s guidelines.




E.1.2 Appliances & HVAC

The following subsections present the evaluation details and findings for the non-lighting, downstream rebate components of the Lighting, Appliances & HVAC Solution (i.e., the Appliances and HVAC components).

E.1.2.1 Gross Impacts

Navigant conducted the following activities to verify the gross impacts and to review the CSP database for reporting accuracy:

• Engineering desk reviews for all strata

o Record-level TRM review

o Invoice review

• Engineering file reviews and phone verification for the HVAC component

o HVAC participant project file review

o HVAC participant phone verification

• Phone survey verification for the Appliances component

o Verification survey to collect installation rates for Appliances participants Engineering Desk Reviews Navigant’s desk review process included quarterly verification of program-reported savings in the program tracking database, eTrack. The evaluation team used the measure-specific variables listed in the eTrack database to populate the energy and demand savings algorithms as detailed in the Phase III TRM and SWE-approved Interim Measure Protocols (IMPs) to verify the reported savings estimates. The team completed desk reviews for a census of non-lighting measures for the Appliances and HVAC components, as well as the PECO Marketplace non-lighting measures. Table E-3 summarizes the results of these desk review activities.

Table E-3. Residential Non-Lighting Summary of Desk Review Activities

Impact Activity Finding Description


The CSP used default savings values as defined in the EE&C Plan documentation for five measures instead of using the algorithms defined in the SWE-approved IMPs.

The CSP used default savings values from the EE&C Plan for ENERGY STAR air purifiers, ENERGY STAR clothes washers-residential, ENERGY STAR clothes dryers, and heat pump clothes dryers instead of the savings algorithms defined in the SWE-approved IMPs for these measures.

Invoice Review No issues.

Navigant reviewed the incentives to confirm that the invoice amounts in the PECO tracking database matched those from the CSP data. The team found no discrepancies.




Engineering File Reviews and Phone Verification This activity included a detailed engineering review of project files for LAH HVAC participants, as well as phone calls to participants to verify equipment installation. The evaluation team requested a sample of 120 project files from the CSP to complete the necessary 25 reviews to achieve 15% relative precision at the 85% confidence level for the non-lighting HVAC measures, as detailed in the PY10 sample design memo5. Navigant used the project files to verify the measure-specific variables listed in the eTrack database, such as capacity (tons), seasonal energy efficiency ratio (SEER), and heating seasonal performance factor (HSPF). The team then applied these verified variables to the TRM and IMP algorithms to calculate verified energy and demand savings for the sample. Phone Survey Verification Navigant conducted phone surveys with a sample of 28 Appliances participants and 25 HVAC participants to verify the installation of their rebated energy efficient equipment. The team verified that all appliances were installed and operational.

E.1.2.2 Process Evaluation

As discussed in Section 3.1 in the main body report, Navigant performed limited process evaluation activities for the Residential EE Program and its solutions during PY10. For the Appliances and HVAC component, these activities included PECO and CSP staff interviews. PECO and CSP Staff Interviews Interviews with PECO and CSP staff covered improvements to program processes as the new CSP, CLEAResult, gained familiarity with the solution. Improvements included an increase to customer satisfaction as CLEAResult improved call center response time.

E.2 Appliance Recycling Solution

The Appliance Recycling Solution helps customers recycle energy-wasting appliances by removing and recycling operating, inefficient refrigerators, freezers, and room ACs (RACs) from residential customer sites at no cost to participants. ARCA is the CSP for this solution. A participant is a customer who schedules a pickup for one or more units. If the same customer initiates multiple pickup orders during the year, each order is counted as an individual participant. However, if a customer initiates more than one order in the same day, it counts as a single participant. Customers may be referred to the Appliance Recycling Solution through other solution activities. For example, Low-Income EE Program Whole Home Solution auditors may identify a qualifying appliance and recommend the Appliance Recycling Solution to the customer. A customer who implements Whole Home measures and recycles an appliance would be considered a participant in both solutions. Savings attributable to income-eligible customers are reported through the Low-Income EE Program Whole Home Solution.

5 PECO: PY10 Lighting, Appliances, and HVAC Sampling Adjustments. April 15, 2019.



E.2.1 Gross Impact Evaluation

Navigant checked participation counts and then conducted three primary evaluation tasks to verify gross impacts for the Appliance Recycling Solution in PY10.

• Conducted an algorithm review using the default coefficients and independent variable values specified in Table 2-78 of the Pennsylvania TRM.

• Used regression analysis to refine the deemed gross verified savings to account for the program’s specific appliance stock characteristics—average age, size (cubic feet), and configuration, among others.

• Conducted a phone survey to verify the appliance characteristics recorded in the tracking data and to gather additional data as inputs to the part-use factor (PUF) of the TRM’s algorithm.

The regression outputs, coupled with the PUF, formed the basis for the gross verified savings related to compliance. This product yields a gross savings per unit that, when summed, yields the solution’s verified savings. This section summarizes the findings of these activities.

E.2.1.1 Participant Counts

Similar to PY9, Navigant found a discrepancy in participant counts, specifically in the total number of customers counted as program participants by PECO. The CSP is using the customer flag for multifamily (MF) as a consideration when calculating total participation, which is incorrect. For example, a customer who recycles two appliances in one order may have “MF No” flagged for one appliance and “MF Yes” flagged for another, leading PECO to double count this participant. The MF designation should not be taken into account when calculating total participation. There were also instances where different customers were assigned the same customer ID. While Navigant was able to distinguish these customers from one another using additional data points, for the sake of clarity and data quality, the CSP should refine processes so that unique customer IDs are assigned to each customer in the program.

E.2.1.2 Gross Verification Findings

Navigant conducted an algorithm review using the default coefficients and independent variable values specified in Table 2-78 of the Pennsylvania TRM. The evaluation team performed this review on a census of recycled units in the program tracking data to determine whether the deemed values were properly applied when calculating program savings in the tracking data. The team calculated a lower annual unit energy consumption (UEC) value for freezers (1,079 kWh/yr) and a higher UEC for refrigerators (828 kWh/yr) than what PECO recorded in the tracking data. This discrepancy is the result of incorrect inputs for the pre-1990 independent variable, which the TRM specifies should be calculated using electric distribution company (EDC)-gathered data and not a prescribed deemed value. The RAC annual UEC, 159 kWh/yr, is specified in the TRM and was correctly applied in the tracking data. However, the three deemed UEC values are only a check and do not factor in to the final verified savings. The evaluation team refined the gross verified savings to account for the program’s specific appliance stock characteristics—average age, size (cubic feet), and configuration, among others—as recorded in the program tracking data. In this case, the regression analysis used the coefficients detailed in the PA TRM and the measure stock characteristics for the algorithm’s independent variables. Navigant



conducted this analysis on a census of recycled units and did not require sampling. The regression analysis of the recycled stock calculated a higher UEC for refrigerators than the deemed value: 1,176 kWh/yr for Residential EE and 1,121 kWh/yr for Low-Income EE. It also calculated a higher UEC for freezers: 911 kWh/yr for Residential EE and 841 kWh/yr for Low-Income EE. As specified by the TRM, Navigant used the deemed value of 159 kWh/yr for the RAC UEC. The team also conducted a phone survey to verify the appliance characteristics recorded in the tracking data and to gather additional data as input to the PUF. Survey respondents reported a lower PUF than the default values: 94.1% for Residential EE refrigerators, 91.7% for Low-Income EE refrigerators, and 83.1% for Residential EE freezers. Respondents reported a PUF of 100% for Low-Income EE freezers, higher than the default TRM value. The PUF for refrigerators and freezers is the primary driving factor for the solution’s realization rate differing from 1.00. There is no RAC PUF. Refer to Table 3-4 in the main body report for all energy, demand, and realization rate values. This includes refrigerator, freezer, and RAC verified savings values.

E.2.2 Net Impact Evaluation

As described in the Phase III Evaluation Plan6, Navigant did not complete any net-to-gross (NTG) research for the Appliance Recycling Solution in PY10. Instead, the team used the NTG ratios developed in PY8 and applied them to the PY10 gross verified impact results to arrive at PY10 net verified impact results. Navigant plans to update the NTG ratios during the PY11 evaluation.

E.2.3 Process Evaluation

As discussed in Section 3.1.4, Navigant performed limited process evaluation for the Residential EE Program and its solutions during PY10. For Appliance Recycling, this work included PECO and ARCA staff interviews. The PECO program manager and ARCA interviews revealed that one of the main solution updates for PY10 was getting the new recycling center running at full capacity. The new recycling center allows the solution to meet and exceed the monthly unit goals. The new center has also increased call center customer satisfaction by decreasing call back time, decreasing pickup time, and increasing available pickup days. The PECO program manager and ARCA interviews also confirmed that PECO tested the solution in retail stores. The test was successful, and the solution was able to engage new customers who would otherwise not participate in the solution. ARCA views retail stores as a future solution opportunity to gain more participants.

E.3 Whole Home Solution

PECO’s comprehensive Whole Home Solution is for customers who want to understand how to improve the energy performance of their entire home. This solution offers a general walkthrough assessment available to all PECO residential customers and a more comprehensive audit, including blower door and combustion safety tests, to PECO residential electric heat customers or customers with central AC.




Participating customers are sorted into one of these two categories based on the outcome of an initial screening call with CSP staff. During the assessment or audit site visits, customers receive direct installation of efficient products (such as lighting, power strips, and pipe insulation for electric domestic hot water tanks). Additionally, customers with electric heat may be eligible for additional thermal envelope improvements (insulation and air sealing). Customers may also participate through a retail pathway available for larger HVAC measures:

• Fuel switching – electric heat to gas, propane, or oil heat

• Fuel switching – electric hot water heater to fossil fuel water heater

• Heat pump water heater

• Variable speed pool pump Customers may be directed to the retail pathway through the walkthrough assessment or audit or through direct referral from customer service or a contractor. Customers may participate in one or the other or both pathways. Two CSPs implemented these solutions for PY8 and through one-half of PY9, when a single CSP was contracted to deliver the Whole Home Solution. Historically, customers participating in both pathways were not identified as such in the tracking databases.


The impact evaluation focused on verifying reported savings and determining the degree to which reported and verified savings were consistent with planned savings. PY10 impact evaluation activities included tracking database analysis, onsite and phone verifications, and desk reviews of a sample of projects. The team evaluated reported savings through a preliminary database review, comparing reported savings with TRM assumptions and algorithms. This analysis was conducted on a census of reported measures, resulting in an adjustment to reported savings. The database review identified five discrepancies between reported savings and verified savings, as illustrated in Table E-4.

Table E-4. Residential EE Program Whole Home Solution Tracking Database Review

Measure Discrepancy Observations Measure kWh

Realization Rate

Measure kW Realization

Rate

Residential Air Sealing

Fifteen Air Sealing with ASHP measures applied a default value of 675 kWh and 0.0023 kW, while others calculated savings based on TRM algorithms. Three Air Sealing with CAC and Fossil Fuel Heat and Air Sealing with CAC and Elec Resistance Heat measures used a default value of 43 kWh and 0.0029 kW, while others calculated savings based on TRM algorithms.

Navigant was unable to determine the source of the default values used and recommends all measures be calculated per TRM algorithms for accuracy and consistency.

0.98 1.21



Measure Discrepancy Observations Measure kWh

Realization Rate

Measure kW Realization

Rate

Crawl Space Insulation

The average R-value for the thermal resistance of the Earth at the height of insulated crawlspace wall below grade (Rearth) appeared to be reporting height of insulated crawlspace wall below grade (Hbg) instead of actual Rearth values based on Table 2-119 in the TRM.

Navigant recommends reporting actual Rearth values to ensure energy savings are correctly captured.

0.64 1.00

Floor Insulation

Missing key inputs to complete TRM calculation. Measure was reported as a custom savings measure and applied default values of 0.34 kWh and 0.00008 kW.

Navigant recommends calculating savings based on the Ceiling / Attic and Wall Insulation measure. Key inputs such as heating system type, heating system efficiency, cooling system type, cooling system efficiency, roof area, roof baseline, wall area, and wall R-value need to be captured to calculate savings.

1.00 1.00

Knee Wall Insulation



1.00 1.00

Ductless Mini-Split Heat Pumps

EFLHheating and EFLHcooling default values were reversed, causing savings discrepancies

Navigant recommends updating both EFLHheating and EFLHcooling so savings can be correctly calculated, ensuring savings accuracy.

1.46 0.59


The evaluation team used a combination of onsite and phone verification to confirm measure installation. Navigant used a random sample of projects from the population of program participants in the PY10 tracking database for its sampling strategy. The team selected sampled projects based on project size to ensure the sample reflected the participant population. Large projects (defined as reported savings >2,081 kWh) and medium projects (defined as reported savings between 1,340 kWh and 2,080 kWh) were sampled for onsite verifications. Small projects (370 kWh-1,399 kWh) were sampled for phone verification. The team did not sample very small projects (<369 kWh); instead, small project strata realization rates were applied to the very small project strata.



Navigant calculated verified savings for medium and large projects with onsite verification. The analysis used installed product data collected onsite (including equipment size, wattage, flow rate, location, etc.) applied to TRM algorithms. The team used phone verifications to confirm product installation. In cases where a customer could not remember the quantity of products installed, Navigant relied on the reported savings values. In cases where a customer provided definitive quantity values, the evaluation team used the customer’s reported values and adjusted the verified savings accordingly. Differences between reported and verified savings were due to the following reasons:

• Small strata phone verification: Navigant verified installations at six small strata homes. Navigant verified 71 out of 88 ENERGY STAR LEDs (0.81), 0 of 1 low flow faucet aerators (0.00), and 12 feet of 15 feet of electric water heater pipe insulation (0.80).

• Medium and large project site verification: Navigant verified installations at seven medium strata homes and seven large strata homes. ENERGY STAR LEDs had a measure-level realization rate of 0.96 due to the following factors:

o Six bulbs could not be located.

o Six bulbs were reported in the wrong room type (e.g., reported as living room, verified in hallway). Navigant recalculated savings for those measures based on the verified room type HOU.

o One bulb reported as decorative was verified as general service. Navigant recalculated savings for this measure based on the verified bulb type.

Navigant conducted file reviews for a sample of project files. The team did not identify any discrepancies between the sampled project files and reported savings. The evaluation team calculated the final program realization rate by applying realization rates determined through phone verification to the adjusted reported savings (reported savings adjusted by the TRM Tracking Database Review).

E.3.2 Net Impact Evaluation

As described in the Phase III Evaluation Plan7, Navigant did not complete any NTG research for the Whole Home Solution in PY10. Instead, the team used the NTG ratios developed in PY9 and applied them to the PY10 gross verified impact results to arrive at PY10 net verified impact results. Navigant plans to update the NTG ratios during the PY11 evaluation.


As described in the Phase III Evaluation Plan8, Navigant did not complete in-depth process evaluation activities for the Whole Home Solution. Instead, the team interviewed the PECO program manager to identify significant implementation changes to inform the impact evaluation activities. No significant

7 PECO. Phase III Evaluation Plan, Energy Efficiency and Conservation Portfolio. Revised February 15, 2019. 8 PECO. Phase III Evaluation Plan, Energy Efficiency and Conservation Portfolio. Revised February 15, 2019.



changes were found. The team carried out in-depth process evaluations in PY9 and plans to conduct additional activities in PY11.

E.4 New Construction Solution

PECO’s Residential New Home Rebates Program intends to accelerate the adoption of EE in the design, construction, and operation of new single-family and multifamily homes by using the US Environmental Protection Agency’s (EPA’s) ENERGY STAR Homes certification. Builders who are building new single-family or multifamily homes can take advantage of PECO’s New Home Rebate Program to incorporate EE. The program also offers incentives for Code Plus homes. A Code Plus home achieves savings of 30% above a code-level home and provides builders an additional incentive tier below the ENERGY STAR-certified home level. Performance Systems Development (PSD) is the CSP for this solution. A participant is a new home.


In the PY10 Residential New Construction solution impact evaluation for weather-sensitive measures, Navigant reviewed project REM/Rate files and conducted building simulation modeling in REM/Rate. For non-weather-sensitive measures, the evaluation team calculated verified impacts based on TRM algorithms and inputs and data gathered from Residential New Construction files, REM/Rate files, and PECO’s LAH Solution.

E.4.1.1 Engineering File Reviews

The evaluation team reviewed an extract of all project REM/Rate models and all prescriptive measure calculations (lighting, appliances, and hot water measures) for compliance with the appropriate PA TRM sections. The team reviewed project tracking data, ex ante measure savings calculations, and REM/Rate model files submitted by raters for compliance with program requirements.9

E.4.1.2 Energy and Demand Saving Calculation – Weather-Sensitive Measures Analysis

For weather-sensitive measure savings (heating and cooling), Navigant independently recalculated heating and cooling savings by running the REM/Rate building simulation models. The team conducted building simulation modeling for a sample of projects completed in PY10. Each REM/Rate file was run using a batch process against the PECO reference home10 based on TRM specifications. The evaluation team calculated the annual heating, cooling, and demand savings of heating and cooling associated with the program homes as the difference between the baseline (PECO reference home) and as-built simulation results.

E.4.1.3 Energy and Demand Saving Calculation – Non-Weather-Sensitive Measures Analysis

For non-weather-sensitive measures (lighting, appliances, and hot water measures), Navigant used a combination of data and assumptions from the TRM and PECO’s LAH Solution. For dishwashers and

9 The Phase III CSP relies on non-proprietary software, REM/Rate, for energy savings estimation. 10 The CSP shared the specifications for the Pennsylvania 2016 Savings Reference home for use in REM/Rate model.



water heaters, Navigant calculated savings using TRM assumptions. For lighting, refrigerators, clothes dryers, and clothes washers, Navigant obtained average per-unit savings from the LAH Solution, which is calculated using TRM algorithms. In its PY9 Annual Report11, the SWE recommended that Navigant remove savings for non-weather-sensitive measures (lighting, appliances, and hot water measures) from the REM/Rate model and calculate them separately using the TRM algorithms. This approach required granular data for these non-weather-sensitive measures that is not being collected by the CSP. Therefore, in cases where project-level CSP data was not available, the evaluation team used data and assumptions from PECO’s LAH Solution, as approved by the SWE. In general, the team considered the LAH Solution measure data as an appropriate proxy for the Residential New Construction measure data because the appliance configurations and sales were weighted toward customers preferences in the PECO service territory. Per the Phase III Evaluation Plan12, Navigant did not conduct onsite or phone verification for this solution due to the solution’s overall size of the solution’s contribution to PECO’s portfolio.

E.4.1.4 Realization Rate Calculation

The evaluation team obtained the total verified energy and demand savings13 by summing the savings from non-weather-sensitive measures and weather-sensitive measures. The team then compared the resulting total verified energy and demand savings to the reported savings to determine the realization rates.

E.4.1.5 Sampling

Navigant used a simple random sample to select 12 files for review to achieve 15% relative precision at the 85% confidence level, assuming a coefficient of variation (CV) of 0.3, which is a conservative estimate informed by PY9’s CV of 0.001. The evaluation team initially intended to complete a census review of projects, but data acquisition hurdles prevented that approach.

E.4.1.6 Findings

The realization rates for energy and demand savings are 1.04 and 1.02, respectively. The slightly higher realization rate can be attributed to the difference in the methodology and assumptions used by the REM/Rate software for non-weather-sensitive measures, which differs from the TRM. Navigant experienced the following challenges during the impact evaluation:

• The REM/Rate files shared by the CSP did not include REM/Rate versions for each file. Navigant requested REM/Rate versions from the CSP for each sample file and used the corresponding version of REM/Rate software to run the building energy simulation.

• Navigant identified several instances of builders installing either a combination of LEDs and CFLs or all CFLs in new construction homes.

11 SWE Annual Report Act 129 Program Year 9. January 15, 2019. 12 PECO. Phase III Evaluation Plan, Energy Efficiency and Conservation Portfolio. Revised February 15, 2019. 13 REM/Rate demand savings has a built-in 0.9 coincident demand factor applied, and the PECO TRM prescribed that 0.647 should be used as the coincident demand factor for heating and cooling. These two factors are considered in the savings calculation.



• Navigant identified data discrepancies in the lighting forms. Navigant found that a few raters incorrectly documented lighting counts and inadvertently recorded some CFLs as LEDs. The error was corrected based on notation in the REM/Rate model specifying 100% LEDs and confirmed by the CSP’s onsite inspections. Moreover, the lighting form does not currently capture a bulb count for specialty LED bulbs.

• The evaluation team used per-unit savings for LEDs from the LAH Solution that were calculated using TRM algorithms and multiplied them by the number of bulbs documented by HERS raters in the lighting forms. For projects with CFLs installed, the team multiplied per-unit LED savings obtained from the LAH Solution with a 0.84 factor (because LAH contains no CFLs to reference). Navigant calculated this factor based on its review of a sample of LED and CFL bulb savings. The team found that a few HERS raters incorrectly documented lighting counts and inadvertently recorded some CFLs as LEDs.

• Among the 12 sampled projects, two projects installed an electric heat pump. Because the TRM does not provide savings algorithms for new electric heat pumps, the team used REM/Rate savings for water heaters for these two projects.

• The CSP does not collect specifications and other parameters for appliances installed in new construction homes, including refrigerators, clothes dryers, and clothes washers. Therefore, Navigant used weighted average per-unit savings from PECO’s LAH Solution for these measures.

E.4.2 Net Impact Evaluation Methodology and Results

As described in the Phase III Evaluation Plan14 updated for PY10 and approved by the SWE, Navigant did not complete any NTG research for the Residential New Construction Solution in PY10. Instead, the team used the NTG ratios developed in PY9 and applied them to the PY10 gross verified impact results to arrive at PY10 net verified impact results. Navigant plans to update the NTG ratios during the PY11 evaluation.


As described in the Phase III Evaluation Plan15, Navigant did not complete any in-depth process evaluation activities for the Residential New Construction Solution in PY10. Instead, the team interviewed the PECO program manager to identify significant implementation changes to inform the impact evaluation activities. No significant changes were found. The team carried out in-depth process evaluations in PY9 and plans to conduct additional activities in PY11.

E.5 Behavioral Solution

A major objective of the Residential EE Program is to provide education, awareness, and motivation to customers that want easy entry into the EE market and want to benefit from energy efficient products. To achieve these ends, the Behavioral Solution partners with Oracle (the solution’s CSP) to implement a randomized control trial (RCT) that provides a select set of residential customers with home energy reports (HERs). The reports provide participants with helpful information about the ways they use energy.




HERs use social norms to compare a customer’s energy use to the average energy use of other households like theirs, so customers have a better sense of whether their energy use patterns fall above or below the norm. These reports also provide targeted recommendations or tips to customers that suggest actions that customers can take to reduce consumption. The combination of HER content serves to enhance a customer’s understanding of their energy use, encourage them to reduce their consumption using targeted tips and social norms, and enhance customer engagement and satisfaction. The reports are sent to a targeted subset of customers on an opt-out basis. As of September 2019, the reports are being provided to roughly 380,000 PECO customers. In addition to the RCT, the Behavioral Solution provides HERs to households enrolled in PECO’s AC Saver Program. The AC Saver Program is a residential demand response (DR) initiative that primarily seeks to reduce the peak demands of participants. The HERs sent to these participants are intended to maintain customer satisfaction while enhancing customer education and awareness related to EE benefits. One participant is counted as a utility account included in the solution’s treatment group, including those accounts associated with the AC Saver Program. Table E-5 summarizes the numbers of participating treatment group homes by cohort and month for those households included within the PY10 scope of evaluation activities.

Table E-5. Behavioral Solution Treatment Group Counts by Cohort

Month Wave 1 Wave 2 Wave 3 Wave 4 Wave 5 – Electric

Wave 5 – Dual Fuel

AC Saver Total

Jun 2018 26,565 33,324 57,442 186,845 27,204 16,127 36,240 383,747

Jul 2018 26,360 33,118 57,101 185,435 26,813 15,983 36,130 380,940

Aug 2018 26,142 32,907 56,702 183,930 26,358 15,792 35,873 377,704

Sep 2018 25,953 32,741 56,410 182,645 26,062 15,654 35,670 375,135

Oct 2018 25,804 32,619 56,158 181,769 25,836 15,544 35,484 373,214

Nov 2018 25,658 32,486 55,923 180,813 25,594 15,443 35,316 371,233

Dec 2018 25,526 32,355 55,690 179,924 25,380 15,362 35,171 369,408

Jan 2019 25,405 32,257 55,515 179,193 25,207 15,308 35,013 367,898

Feb 2019 25,292 32,154 55,353 178,513 25,032 15,248 34,877 366,469

Mar 2019 25,207 32,073 55,217 177,949 24,879 15,186 34,807 365,318

Apr 2019 25,096 31,952 55,040 177,202 24,696 15,110 34,691 363,787

May 2019 24,953 31,830 54,777 176,293 24,500 15,021 34,561 361,935 Source: Navigant analysis



E.5.1 Impact Evaluation Methodology

Navigant followed the impact evaluation methodology outlined in Section 6.1.1 of the Phase III Evaluation Framework16. The evaluation team estimated savings using a monthly lagged dependent variable (LDV) model. For details on this model, refer to Section 6.1.1.5 of the Phase III Evaluation Framework.

E.5.2 Summary Statistics and Results

Table E-6 through Table E-9 summarize the regression outputs and summary statistics by cohort within both the RCT and AC Saver groups. The tables also include the absolute precision results for each wave. The Phase III Evaluation Framework17 document (at Section 6.1.1.1.1) requires the solution-level verification achieve an absolute precision of ±0.5% at the 95% confidence level (two-tailed); individual waves may have a wider margin of error. Given that the Behavioral Solution analysis examines the solution’s entire population (a census evaluation), the precisions reported in Table E-6 reflect the error of the regression analysis estimate rather than a sampling uncertainty. The reader should also note that this uncertainty is reflected within the Behavioral Solution analysis only. That is, the regression analysis estimation error is not reflected within the Residential EE Program or the PY10 portfolio total savings uncertainty. Those rolled up uncertainties only reflect sampling uncertainties that may be associated with other solutions.

Table E-6. Behavioral Solution Cohort Regression Details – Waves 1-4

Month

Wave 1 Wave 2 Wave 3 Wave 4

Treatment Coefficient

Cluster Robust

Standard Error


Cluster Robust

Standard Error


Cluster Robust

Standard Error


Cluster Robust

Standard Error

Jun 2018 -0.58 0.11 -1.47 0.21 -0.94 0.13 -0.54 0.09

Jul 2018 -0.72 0.13 -1.57 0.23 -1.18 0.14 -0.66 0.10

Aug 2018 -0.63 0.13 -1.41 0.22 -1.14 0.14 -0.61 0.10

Sep 2018 -0.50 0.11 -1.19 0.19 -1.00 0.12 -0.56 0.08

Oct 2018 -0.45 0.10 -1.07 0.16 -0.80 0.10 -0.38 0.07

Nov 2018 -0.73 0.16 -1.08 0.21 -0.85 0.12 -0.29 0.09

Dec 2018 -0.96 0.19 -1.16 0.25 -0.89 0.14 -0.26 0.10

Jan 2019 -1.06 0.22 -1.34 0.28 -0.90 0.16 -0.29 0.11

Feb 2019 -1.13 0.22 -1.32 0.27 -0.88 0.16 -0.18 0.11

Mar 2019 -0.90 0.17 -1.15 0.22 -0.78 0.13 -0.18 0.09

Apr 2019 -0.50 0.11 -1.01 0.16 -0.74 0.10 -0.25 0.06

May 2019 -0.47 0.10 -1.02 0.18 -0.83 0.10 -0.34 0.07

16 SWE. Evaluation Framework for Pennsylvania Act 129 Phase III Energy Efficiency and Conservation Programs. October 21, 2016. http://www.puc.pa.gov/Electric/pdf/Act129/SWE_PhaseIII-Evaluation_Framework102616.pdf 17 SWE. Evaluation Framework for Pennsylvania Act 129 Phase III Energy Efficiency and Conservation Programs. October 21, 2016. http://www.puc.pa.gov/Electric/pdf/Act129/SWE_PhaseIII-Evaluation_Framework102616.pdf






Table E-7. Behavioral Solution Cohort Regression Details – Waves 5 and AC Saver

Month Wave 5 – Electric Wave 5 – Dual Fuel AC Saver


Cluster Robust Standard Error





Jun 2018 -0.56 0.13 -0.54 0.16 -0.41 0.07

Jul 2018 -0.63 0.15 -0.74 0.18 0.07 0.08

Aug 2018 -0.57 0.14 -0.83 0.17 -0.33 0.08

Sep 2018 -0.61 0.12 -0.54 0.15 -0.05 0.06

Oct 2018 -0.43 0.11 -0.35 0.12 -0.34 0.05

Nov 2018 -0.37 0.16 -0.23 0.14 -0.33 0.06

Dec 2018 -0.53 0.18 -0.43 0.15 -0.28 0.07

Jan 2019 -0.50 0.21 -0.36 0.16 -0.49 0.07

Feb 2019 -0.36 0.21 -0.22 0.15 -0.67 0.07

Mar 2019 -0.30 0.16 -0.27 0.13 -0.54 0.06

Apr 2019 -0.31 0.10 -0.17 0.10 -0.51 0.05

May 2019 -0.41 0.11 -0.33 0.12 -0.54 0.06 Source: Navigant analysis

Table E-8. Behavioral Solution Cohort Percent Savings – Waves 1-4

Month

Wave 1 Wave 2 Wave 3 Wave 4


Cluster Robust

Standard Error


Cluster Robust

Standard Error


Cluster Robust

Standard Error


Cluster Robust

Standard Error

Jun 2018 1.66% 0.64% 2.15% 0.60% 1.83% 0.48% 1.46% 0.45%

Jul 2018 1.83% 0.63% 2.02% 0.57% 1.95% 0.47% 1.47% 0.44%

Aug 2018 1.62% 0.66% 1.87% 0.58% 1.91% 0.47% 1.38% 0.44%

Sep 2018 1.49% 0.66% 1.89% 0.60% 2.07% 0.48% 1.60% 0.45%

Oct 2018 1.32% 0.60% 1.98% 0.58% 2.10% 0.50% 1.42% 0.49%

Nov 2018 1.40% 0.60% 1.60% 0.61% 2.04% 0.59% 1.00% 0.62%

Dec 2018 1.53% 0.59% 1.47% 0.61% 1.90% 0.60% 0.83% 0.63%

Jan 2019 1.43% 0.59% 1.56% 0.63% 1.84% 0.63% 0.88% 0.65%

Feb 2019 1.57% 0.61% 1.59% 0.64% 1.89% 0.66% 0.59% 0.68%

Mar 2019 1.65% 0.62% 1.71% 0.64% 1.94% 0.63% 0.67% 0.65%

Apr 2019 1.51% 0.64% 2.06% 0.64% 2.23% 0.56% 1.09% 0.54%

May 2019 1.60% 0.69% 1.97% 0.69% 2.25% 0.55% 1.34% 0.52% Source: Navigant analysis



Table E-9. Behavioral Solution Cohort Percent Savings – Waves 5 and AC Saver

Month

Wave 5 – Electric Wave 5 – Dual Fuel AC Saver







Jun 2018 1.54% 0.69% 1.43% 0.82% 1.42% 0.47%

Jul 2018 1.47% 0.66% 1.61% 0.75% -0.20% 0.46%

Aug 2018 1.34% 0.67% 1.84% 0.75% 0.95% 0.45%

Sep 2018 1.76% 0.69% 1.51% 0.81% 0.17% 0.47%

Oct 2018 1.48% 0.75% 1.36% 0.89% 1.73% 0.49%

Nov 2018 1.05% 0.92% 0.91% 1.04% 1.58% 0.56%

Dec 2018 1.34% 0.89% 1.53% 1.03% 1.24% 0.57%

Jan 2019 1.14% 0.93% 1.25% 1.08% 2.13% 0.59%

Feb 2019 0.85% 0.97% 0.81% 1.12% 3.05% 0.61%

Mar 2019 0.87% 0.90% 1.12% 1.09% 2.78% 0.62%

Apr 2019 1.22% 0.80% 0.78% 0.95% 3.03% 0.58%

May 2019 1.55% 0.78% 1.32% 0.95% 2.76% 0.56% Source: Navigant analysis

Table E-10 summarizes the monthly gross savings for the Behavioral Solution waves informed by the regression analysis activities. These results reflect the impacts before any consideration of the overlap analysis, which is described in the next section.

Table E-10. Behavioral Solution Monthly Verified Modeled Savings

Month Wave 1 Wave 2 Wave 3 Wave 4 Wave 5 – Electric

Wave 5 – Dual Fuel AC Saver Total

Jun 2018 459 1,466 1,621 3,049 453 262 443 7,754

Jul 2018 587 1,607 2,090 3,815 527 368 -78 8,915

Aug 2018 507 1,442 1,999 3,471 464 408 367 8,658

Sep 2018 386 1,169 1,697 3,095 476 252 49 7,124

Oct 2018 362 1,087 1,396 2,149 342 168 379 5,883

Nov 2018 563 1,055 1,431 1,553 282 109 347 5,341

Dec 2018 762 1,162 1,536 1,473 420 206 309 5,868

Jan 2019 836 1,345 1,540 1,603 389 170 536 6,418

Feb 2019 800 1,188 1,357 919 252 93 657 5,267

Mar 2019 704 1,142 1,332 1,000 232 126 586 5,123

Apr 2019 379 969 1,219 1,327 232 75 534 4,735

May 2019 364 1,011 1,412 1,883 312 156 581 5,718 Source: Navigant analysis



E.5.3 Dual Participation Analysis

To the extent that the Behavioral Solution increases participation in other solutions, some savings from the regression analysis could be double counted if appropriate adjustments are not made. Double counting can be avoided for solutions that track participation at the customer level by generating estimates of the increase in participation in the solution among Behavioral Solution participants. This is also known as dual participation savings. To generate estimates of dual participation, Navigant followed the Phase III Evaluation Framework18 guidance on completing dual participation analyses. The Phase III Evaluation Framework conveys that exposure to the Behavioral Solution messaging often motivates participants to take advantage of other solution offerings that may be promoted through Behavioral Solution materials. This exposure creates a situation where households in the treatment groups tend to participate in other solutions at a higher rate than households in the control groups.19 The framework methodology calls for program-specific uplift calculations, and the SWE requests those values be reported. Given PECO’s reorganization of Phase I and Phase II programs into solutions for Phase III, Navigant estimated aggregate uplift across residential programs. Navigant’s dual participation analysis also accounts for upstream EE solutions. The calculation of double counted savings from upstream solutions is complicated by participation not being tracked at the customer level; therefore, the approaches described previously for specific homes are infeasible. Per Section 6.1.1.8.2 of the Phase III Evaluation Framework, the evaluation team used an assumed upstream reduction factor subtracted from the estimate of energy savings for each wave of Behavioral Solution participants after downstream double counted savings had been removed. The specific reduction factors used for the waves are shown in Table E-11.

Table E-11. Default Upstream Adjustment Factors

Years Since Cohort Inception

Default Upstream Reduction Factor Behavior Waves

1 0.75% -

2 1.50% Wave 5

3 2.25% Wave 4, AC Saver

4 and beyond 3.00% Wave 1, Wave 2, Wave 3 Source: Phase III Evaluation Framework for Pennsylvania Act 129 Phase III Energy Efficiency and Conservation Programs, Prepared by The Statewide Evaluation Team: NMR Group, Inc., EcoMetric Consulting, LLC, and Demand Side Analytics, LLC. Contracted Under the Pennsylvania Public Utility Commission‘s RFP 2015-3 for the Statewide Evaluator, October 21, 2016

Table E-12 summarizes the overlap or uplift savings associated with downstream and upstream EE solutions found for each of the Behavioral Solution waves. These savings are subtracted from the total savings previously shown in Table E-10.

18 SWE. Evaluation Framework for Pennsylvania Act 129 Phase III Energy Efficiency and Conservation Programs. October 21, 2016. http://www.puc.pa.gov/Electric/pdf/Act129/SWE_PhaseIII-Evaluation_Framework102616.pdf 19 Pennsylvania PUC. “Section 6.1.1.8. Dual Participation Analysis.” Phase III Evaluation Framework. http://www.puc.pa.gov/Electric/pdf/Act129/SWE_PhaseIII-Evaluation_Framework102616.pdf





Table E-12. Downstream and Upstream Savings Adjustments

Behavior Waves Downstream Dual

Participation Savings (MWh/yr)

Upstream Dual Participation Savings

(MWh/yr)

Total Dual Participation (MWh/yr)

Wave 1 668 181 849 Wave 2 1,358 399 1,757 Wave 3 926 531 1,457 Wave 4 815 552 1,366 Wave 5 – Electric 62 65 127 Wave 5 – Dual Fuel 73 35 108 AC Saver 1,951 62 2,013


E.5.4 Behavioral Program Impacts

By combining the results of the regression analysis and the overlap analysis, Navigant created a final set of estimates of the PY10 Behavioral Solution impacts, detailed in Table E-13. These energy savings reflect the net impacts for each of the five waves of RCT participants. The evaluation team used an NTG ratio of 1.00.

Table E-13. Behavioral Solution Net Impacts

Behavior Waves Gross

Savings (MWh/yr)

Downstream Dual

Participation Savings (MWh/yr)

Upstream Dual Participation

Savings (MWh/yr)

Net Savings (MWh/yr)

Demand Savings (MW)20

Wave 1 6,709 668 181 5,860 0.7

Wave 2 14,643 1,358 399 12,887 1.5

Wave 3 18,629 926 531 17,172 2.0

Wave 4 25,339 815 552 23,972 2.7

Wave 5 – Electric 4,381 62 65 4,255 0.5

Wave 5 – Dual Fuel 2,393 73 35 2,284 0.3

AC Saver 4,709 1,951 62 2,697 0.3

Total 76,804 5,853 1,824 69,127 7.9 Source: Navigant analysis

Total verified savings are 69,127 MWh/yr. Solution-reported savings by PECO are 73,716 MWh/yr, resulting in an energy realization rate of 0.94.

20 PECO claims the verified demand savings, but the implementer (Oracle) does not evaluate these savings as part of its standard reporting. Therefore, there is no realization rate for demand savings.


©2019 Guidehouse Inc. Page F-1

APPENDIX F. RESIDENTIAL LOW-INCOME EE PROGRAM This appendix provides additional detail on the evaluation sample design, methods, and activities deployed in PY10 for the Residential Low-Income EE’s Whole Home Solution. (PECO discontinued the Lighting Solution in PY9.) The reader should refer to Section 3.2 in the main body report for evaluation findings, results, and conclusions. The Multifamily Targeted Market Segment does not contribute to the Low-Income EE Program. Income-eligible multifamily buildings are served through the Low-Income EE Program Whole Home Solution.

F.1 Whole Home Solution

PECO’s Low-Income Whole Home Solution offers income-eligible customers multiple pathways to engage with PECO to improve the energy performance of their entire home. These pathways include the following:

• Free home energy checkups, providing site visits, education, and direct installation of energy efficient products.

• Collaboration with property owners to deliver services to income-eligible customers living in multifamily buildings, consistent with the home energy checkup. This effort includes large private property owners and the city’s public housing authority.

• Collaboration with complementary income-eligible programs (such as the Philadelphia Gas Works and Weatherization Agencies) to identify income-eligible customers and serve them comprehensively with free home energy checkups through a single outreach effort.

• Workshops delivered to income-eligible multifamily buildings providing energy education and energy kits.

• Collaboration with the Low-Income Usage Reduction Program (LIURP), providing complementary efficient products to increase the LIURP service offering’s comprehensiveness.

• LED lighting giveaways through community events in collaboration with community partner organizations. In PY9, PECO expanded distribution to include food banks serving income-eligible customers.

• Direct customer referrals to the Appliance Recycling Solution.

F.1.1 Gross Impact Evaluation

The PY10 impact evaluation focused on verifying reported savings; activities conducted included tracking database analysis, phone verifications, and desk reviews of a sample of projects. The evaluation team conducted phone verifications on direct installation measures as part of the core home energy checkup pathway. The Low-Income EE Program Whole Home Solution referred eligible customers to the Residential EE Program Appliance Recycling Solution. Subsequent appliance recycling projects were implemented and evaluated consistent with the Appliance Recycling Solution’s procedures. Reported and verified



Appliance Recycling savings attributable to income-eligible customers are included in the Low-Income EE Program Whole Home Solution. The team evaluated reported savings through a preliminary database review, comparing reported savings with TRM assumptions and algorithms. The team analyzed a census of reported measures, resulting in an adjustment to reported savings. The database review identified six discrepancies between reported savings and verified savings, as illustrated in Table F-1.

Table F-1. Low-Income EE Program Whole Home Solution Tracking Database Review

Measure Discrepancy Observations kWh

Realization Rate

kW Realization

Rate

Low Flow Aerators

A mix of TRM-calculated savings and default values were applied for low flow aerators.

Navigant recommends calculating savings using TRM algorithms for accuracy and consistency.

0.73 0.72

Freezer Replacement

Navigant calculated savings of 571 kWh and 0.064 kW using the IMP average of replacement UEC for all freezer types. Reported savings are defaulting to 655 kWh and 0.0733 kW.

Navigant recommends capturing replacement UEC to calculate savings based on the IMP.

0.87 0.87

Refrigerator Replacement

Navigant calculated savings of 551 kWh and 0.0617 kW using the IMP average of replacement UEC for all refrigerator types except compact sizes. Reported savings are defaulting to 548 kWh and 0.0613 kW.

Navigant recommends capturing replacement UEC to calculate savings based on the IMP.

1.00 1.01

Ceiling / Attic and Wall Insulation

Navigant found that SEER CAC was not properly capturing the SEER for AC units. The Navigant calculation used SEERb (when data was available) or the default of 13.

Navigant recommends capturing SEER information for homes with central air conditioning to properly capture savings.

0.93 0.42

Floor Insulation



1.00 1.00



Measure Discrepancy Observations kWh

Realization Rate

kW Realization

Rate

Commercial LED Exit Signs

Navigant calculated savings based on reported HOU, but reported savings were calculated using a default HOU of 8,760.

Navigant recommends defaulting to 8,760 HOU for this measure.

0.99 1.00


Navigant used phone surveys to confirm measure installation. Multifamily projects and large (>1,946 kWh), medium (1,135 kWh-1,945 kWh), and small sized single-family (336 kWh-1,134 kWh) projects were sampled for phone verifications. The team did not sample very small projects (<342 kWh); instead, the small project stratum realization rates were applied to the very small project strata. The Low-Income Whole Home Solution serves income-eligible multifamily properties. Projects are implemented and reported based on meter configuration: projects are reported individually at the apartment level on individually metered buildings. Conversely, projects are reported at the building level for master-metered buildings. The evaluation team worked with the CSP to identify groups of single-family projects that belonged to a larger multifamily building. By sampling multifamily buildings in their own stratum, the team accounted for program implementation differences occurring when tenants rather than property owners are making residential project decisions. For each of the four multifamily buildings in the sample, Navigant phone surveyed a sub-sample of 10 of the 27 apartments. The average measure realization rate of verified apartments was applied to the unverified apartments within the multifamily sample. The sample plan called for phone verification of direct install measures delivered by a new CSP through the Energy Coordinating Agency’s (ECA’s) Heater Hot Line program. However, these savings were not included in PECO’s PY10 reported savings (either for the preliminary or final reports). Therefore, the team did not conduct evaluation activities for these projects. The impact evaluation sampling strategy used a random sample of projects from the population of program participants in the PY10 tracking database. Navigant selected sampled projects based on project size to ensure the sample reflected the participant population. The team used phone verifications to confirm product installation. In cases where a customer could not remember the quantity of products installed, Navigant verified the reported savings. In cases where a customer provided definitive quantity values, the evaluation team used the customer’s reported values and adjusted the verified savings accordingly. Differences between reported and verified savings were due to the following reasons:

• Multifamily strata phone verification: Navigant verified measure installations in 10 apartments within a sample of four multifamily buildings.

o Lighting: Two ENERGY STAR LED bulbs were not in service as the lamps they were installed in had broken. The tenant reported bulbs were in the closet for future use.

o Smart strip plug outlets: One smart strip in the multifamily strata had been removed by the customer because “it shut off the TV.”



• Single-family small, medium, and large strata phone verification: Navigant verified measure installations in 24 single-family home projects.

o Lighting: Two customers reported ENERGY STAR LED bulbs had not been installed. Two other customers reported removing one of their ENERGY STAR LED bulbs from use due to performance issues (brightness, color).

o Smart strip plug outlets: One customer reported the smart strip had not been installed. One customer reported only one of two reported smart strips had been installed.

Navigant conducted file reviews for a sample of project files. The team did not identify any discrepancies between the sampled project files and reported savings. The evaluation team calculated final program realization rate by applying realization rates determined through phone verification to the adjusted reported savings (reported savings adjusted by the TRM tracking database review).

F.1.2 Process Evaluation

As described in the Phase III Evaluation Plan21, Navigant did not complete any in-depth process evaluation activities for the Whole Home Solution. Instead, the team interviewed the PECO program manager to identify significant implementation changes to inform the impact evaluation activities. No significant changes were found. The team carried out in-depth process evaluations in PY8 and plans to conduct additional activities in PY11.



©2019 Guidehouse Inc. Page G-1

APPENDIX G. SMALL AND LARGE C&I EE PROGRAMS This appendix provides additional detail on the evaluation sample design, methods, and activities deployed in PY10 for select Small and Large C&I EE Program solutions (listed below). The reader should refer to Sections 3.3 and 3.4 in the main body report for evaluation findings, results, and conclusions.

• Equipment and Systems Solution

• New Construction Solution

• Whole Building Solution

• Data Centers Targeted Market Segment

G.1 Equipment and Systems Solution

The Equipment and Systems Solution offers incentives for existing building retrofit projects with either deemed, partially deemed, or custom measures. Typical measures include lighting, variable frequency drives (VFDs), HVAC systems, refrigeration, and controls. Participation is defined as an activity with a unique project number. More than one measure per participant is permitted, with the impact sample defined on the project level. In addition, a single customer is permitted to participate in multiple projects with unique project numbers. PECO’s C&I and Government/Education/Non-Profit (G/E/NP) customers that own or rent their space are eligible to participate in this solution. Participating customers must first identify EE projects at their facility, including deemed, partially deemed, or custom measures. Next, the customer must submit a pre-application to ICF, the CSP, before completing the project. Once approved, each project is implemented by the customer’s own contractor; either the customer or the contractor submits the rebate paperwork to the CSP. In PY10, ICF introduced the Express pathway for program applications. Only retrofit lighting with fewer than 750 MWh savings or non-lighting projects with fewer than 250 MWh savings are eligible for the Express pathway. The Express pathway allows participants to:

• Skip the pre-application step, during which participants submit an initial application to verify equipment eligibility

• Defer submitting documentation of existing equipment, Appendix C from the PA TRM22, and new equipment specs until the final application is submitted

ICF introduced the PECO Instant Lighting Discounts (PILD) pathway for select lighting measures in early PY9 and expanded the number of participating distributors in PY10. The PILD pathway allows customers to receive discounts on qualified products without completing application paperwork. Customers must provide basic information on the facility where lighting will be installed, including verification that the facility is associated with a PECO C&I account. The distributor completes a simplified application on behalf of the customer, and the customer must submit proof of installation within 45 days of purchase.




G.1.1 Gross Impact Evaluation

G.1.1.1 Methodology

Phone Verification Navigant conducted desk reviews for all projects in the evaluation sample. The desk reviews used project applications, project-specific analysis files and associated calculation sheets, measure invoices, measure specification sheets, construction plans, and other construction documents provided by PECO. Documentation included scanned files of hard copy forms, as well as electronic files of CSP inspection reports, photos of installed measures, important emails, and memoranda. The evaluation team supplemented desk reviews with phone verification, which Navigant assigned to projects in accordance with the sampling memo.23 Phone verifications consisted of interviews with customers about their projects. Common discussion points included information about the quantities and type of each measure installed, the operating status of the measures, equipment nameplate data, operating schedules, a careful description of site conditions, and overall verification of the information contained in the project files. Onsite Verification Navigant conducted onsite verification for sampled projects in accordance with the sampling memo.24 Projects assigned an onsite visit first received a desk review to create the Site-Specific Measurement and Verification Plan (SSMVP). The primary objective of the site visits was to collect the data identified in the Phase III Evaluation Framework,25 including verifying the quantities and type of each measure, equipment nameplate data, and operating schedules, as well as carefully describing the site conditions. Navigant achieved the verification through visual inspection of the measures and by interviewing the customers. Onsite Verification with Metering For projects that surpassed the expected energy (kWh) savings thresholds set in Table 1-2 of the 2016 PA TRM,26 the evaluation team conducted onsite verification and collected site-specific information for open variables used to calculate energy and demand savings. Site-specific information included end-use metered data and trend data from building management systems (BMSs). Thirteen projects received onsite verification supplemented with metering or trend data. Handling Non-Response and Customer Refusal Navigant made every attempt to complete its verification efforts. The team made repeated attempts via email and phone calls to schedule site visits or complete phone interviews. For non-responding customers, Navigant assigned field staff in the area to drop in to complete the verification effort. In the case of customer refusal, via phone or in-person, the team dropped the sample point and replaced it with 23 PECO, “PY10 Small and Large C&I EE Impact Sampling Design.” Dated April 16, 2019. 24 PECO, “PY10 Small and Large C&I EE Impact Sampling Design.” Dated April 16, 2019. 25 SWE. Evaluation Framework for Pennsylvania Act 129 Phase III Energy Efficiency and Conservation Programs. October 21, 2016. http://www.puc.pa.gov/Electric/pdf/Act129/SWE_PhaseIII-Evaluation_Framework102616.pdf 26 Pennsylvania Public Utility Commission, Technical Reference Manual; State of Pennsylvania Act 129 Energy Efficiency and Conservation Program & Act 213 Alternative Energy Portfolio Standards, dated June 2016, errata update February 2017.




another from the same stratum. Four projects that were assigned phone verification were ultimately converted to file review only after making at least five attempts to call or drop in at the customer site.

G.1.1.2 Sampling

Using tracking data from PY10, Navigant obtained the total number of projects and the total amount of energy savings in the population. With this project data on hand, Navigant created seven strata of sampled projects. Navigant first created a census stratum (Stratum 1 – Very Large Projects) for projects exceeding the kilowatt-hour thresholds described below. Next, the evaluation team excluded all projects making up the lowest 2% of total solution energy savings. Projects completed through the midstream pathway were then separated and put into their own strata. Finally, the team sorted the remaining projects by size and divided the population into three additional strata: those projects making up the top third, the middle third, and the lowest third of the total energy savings. Small Equipment and Systems

• Stratum 1: Very high impact projects over 1 million kWh/yr energy savings

• Stratum 2: High impact projects between 157,000 kWh/yr and 1 million kWh/year energy savings

• Stratum 3: Medium impact projects between 53,000 kWh/yr and 157,000 kWh/yr energy savings

• Stratum 4: Low impact projects less than 53,000 kWh/yr energy savings

• Stratum 5: Very low impact projects making up less than 2% of total solution savings

• Stratum 6: Midstream high impact projects greater than 25,000 kWh/yr

• Stratum 7: Midstream low impact projects less than 25,000 kWh/yr Large Equipment and Systems:

• Stratum 1: Very high impact projects over 3 million kWh/yr energy savings

• Stratum 2: High impact projects between 978,000 kWh/yr and 3 million kWh/yr energy savings

• Stratum 3: Medium impact projects between 240,000 kWh/yr and 978,000 kWh/yr energy savings

• Stratum 4: Low impact projects less than 240,000 kWh/yr energy savings

• Stratum 5: Very low impact projects making up less than 2% of total solution savings

• Stratum 6: Midstream high impact projects greater than 23,000 kWh/yr

• Stratum 7: Midstream low impact projects less than 23,000 kWh/yr For Small C&I Equipment and Systems, of the 45 projects evaluated:

• 44 included lighting or lighting control retrofits

• 1 was classified as a VFD retrofit

The SWE conducted detailed desk reviews on two projects and site visits on an additional two projects.



For Large C&I Equipment and Systems, of the 44 projects evaluated:

• 37 included lighting or lighting control retrofits

• 4 were classified as custom

• 2 included VFD retrofits

• 1 was classified as HVAC The SWE conducted site visits for three of the projects and desk reviews on an additional three projects. Details of the impact sample, by stratum, can be seen in Table 3-21 and Table 3-32 in the main body report.

G.1.1.3 Findings

The majority of sampled projects for the Equipment and Systems Solution achieved realization rates for both demand and energy within 20% of the expected values. Thirteen projects had verified energy savings values fall below 80% of the reported values, while 11 projects had verified energy savings values above 120% of reported values. For demand savings, 10 projects fell below 80% of reported values while 18 projects were above 120% of reported values. Navigant analyzed these projects to capture any trends in the verified data. The following factors led to variation between the reported and verified savings and to the observed project-level realization rates:

• The most common discrepancy between ex ante and ex post calculations was in the annual HOU for lighting measures. The evaluation team uncovered discrepancies both higher and lower than reported. In most of these cases, the verified HOU was greater than 10% different from deemed HOU.27 In a few cases, the building type was misapplied in the ex ante calculations, and the difference resulted from the evaluators changing the building type. Discrepancies discovered during peak demand hours or summertime operating schedules had the additional consequence of changing the demand calculation, sometimes significantly.

• Other primary drivers for lighting measure realization rates differing from 1.00 include the following:

o 10 sampled projects revealed fixture wattages that differed from reported values

o 8 sampled projects verified the heating fuel type or space cooling type to differ from reported values

o In a few cases, the evaluation team found fixture quantities, control types, and baseline fixture wattages to differ from reported values

• The evaluation team adjusted the power factor for two custom projects. In one case, the ex ante calculations did not account for a VFD power factor correction. In the other project, the ex ante calculations assumed a power factor higher than appropriate, and the evaluation team adjusted it to the nameplate value.

• For two custom projects, the implementer used trend data to determine ex ante savings. The analysis relied on second order polynomial regression curves to fit the data. This approach yields a good fit in the middle ranges of the data, but it distorts the results at the extreme ends. Navigant applied piecewise linear regressions to account for the end ranges.

27 As allowed by the evaluation protocols described in Section 3.1.1 of the 2016 PA TRM.



G.1.2 Net Impact Evaluation

As described in the Phase III Evaluation Plan28, Navigant did not complete any NTG research for the Equipment and Systems Solution in PY10. Instead, the team used the NTG ratios developed in PY9 and applied them to the PY10 gross verified impact results to arrive at PY10 net verified impact results. Navigant plans to update the NTG ratios during the PY11 evaluation.

G.1.3 Process Evaluation

As described in the Phase III Evaluation Plan29, Navigant did not complete any in-depth process evaluation activities for the Equipment and Systems Solution. Instead, the team interviewed the PECO program manager and CSP to identify significant implementation changes to inform the impact evaluation activities. In addition to providing participants more ways to apply to the program (described in Appendix G.1), PECO and ICF implemented a variety of changes in PY10 to increase program participation:

• Adjusted incentives for key lighting and custom measures aimed at improving savings generation

• Offered limited-time bonus incentives to trade allies for installing qualifying projects

• Recruited more distributors to PILD The team will further evaluate the effect of these changes on the customer experience during the planned PY11 process evaluation.

G.2 New Construction Solution

The C&I New Construction Solution is designed to instill and accelerate adoption of energy efficient design and construction practices so new C&I facilities in the PECO territory are more energy efficient than the current stock. The program covers both new construction and buildings undergoing major renovation; major renovation is defined as construction projects that involve the complete removal, redesign, and replacement of two or more major building systems. The program provides facility designers and builders with training, design assistance, and financial incentives to incorporate energy efficient systems into their building designs. Many of the projects within the C&I New Construction Solution involve efficient lighting and heating and cooling technologies and controls. The eligible customer population for the program includes all C&I and G/E/NP new construction and major renovation projects in the PECO service territory or accounts provided with electricity by PECO. Participation is defined as an activity with a unique project number. More than one measure per participant is permitted, with the impact sample defined on the project level. A single customer is permitted to participate in multiple projects with unique project numbers. ICF is the CSP for the C&I New Construction Solution.





G.2.1.1 Methodology

Phone Verification Navigant conducted desk reviews for all projects in the evaluation sample. The desk reviews used project applications, project-specific analysis files and associated calculation sheets, measure invoices, measure specification sheets, construction plans, and other construction documents provided by PECO. Documentation included scanned files of hard copy forms, as well as electronic files of CSP inspection reports, photos of installed measures, important emails, and memoranda. In the case of whole building projects and some new construction projects, PECO provided executable modeling files and related model output files as applicable. The evaluation team supplemented the desk reviews with phone verifications, which Navigant assigned to projects in accordance with the sampling memo.30 Phone verifications consisted of interviews with customers about their projects. Common discussion points included information about the quantities and type of each measure installed, the operating status of the measures, equipment nameplate data, operating schedules, a careful description of site conditions, and overall verification of the information contained in the project files. Onsite Verification Navigant conducted onsite verification for all sampled projects in the very high impact stratum and approximately two-thirds of the sampled projects in the high impact stratum. Projects assigned an onsite visit first received a desk review to create the SSMVP. The primary objective of the site visits was to collect the data identified in the Phase III Evaluation Framework,31 including verifying the quantities and type of each measure, equipment nameplate data, and operating schedules, as well as carefully describing the site conditions. Navigant achieved the verification through visual inspection of the measures and by interviewing the customers. Onsite Verification with Metering For projects that surpassed the expected energy (kWh) savings thresholds set in Table 1-2 of the 2016 PA TRM, the team conducted onsite verification and collected site-specific information for open variables used to calculate energy and demand savings. Site-specific information included end-use metered data and trend data from BMSs. There were no metered projects in PY10.

G.2.1.2 Sampling

Using tracking data from PY10, Navigant obtained the total number of projects and the total amount of energy savings in the population. With this project data in hand, Navigant created four strata of sampled projects.

30 PECO, “PY10 Small and Large C&I EE Impact Sampling Design.” Dated April 16, 2019. 31 SWE. Evaluation Framework for Pennsylvania Act 129 Phase III Energy Efficiency and Conservation Programs. October 21, 2016. http://www.puc.pa.gov/Electric/pdf/Act129/SWE_PhaseIII-Evaluation_Framework102616.pdf




All projects above 2 million kWh per year of annual energy savings in the Large program or 1 million kWh per year in the Small program made up a census stratum (Stratum 1 – Very High Impact Projects). In the C&I New Construction Solution, no Small C&I projects fit this description. Navigant then sorted the projects by size, excluded all projects making up the lowest 2% of total solution energy savings, and divided the population into two strata. Small New Construction

• Stratum 1: Very high impact projects over 1 million kWh/yr energy savings. There were no projects in this stratum.

• Stratum 2: High impact projects between 100,000 kWh/yr and 1 million kWh/year energy savings.

• Stratum 3: Low impact projects less than 100,000 kWh/yr energy savings.

• Stratum 4: Very low impact projects making up less than 2% of total solution savings. Large New Construction

• Stratum 1: Very high impact projects over 2 million kWh/yr energy savings.

• Stratum 2: High impact projects between 300,000 kWh/yr and 2 million kWh/year energy savings.

• Stratum 3: Low impact projects less than 300,000 kWh/yr energy savings.

• Stratum 4: Very low impact projects making up less than 2% of total solution savings. For Small C&I New Construction, Navigant verified 14 projects in PY10. These projects included the following:

• 11 projects including lighting and/or lighting controls

• 3 projects including HVAC or refrigeration measures

• 6 projects including Custom measures32 The SWE conducted one site visit and one desk review for Small C&I New Construction projects in PY10. For Large C&I New Construction, Navigant verified 10 projects in PY10. These projects included the following:

• 5 projects including custom HVAC or refrigeration measures

• 4 projects including whole building energy models

• 1 project including lighting or lighting controls The SWE conducted three site visits and two additional desk reviews for Large C&I New Construction projects in PY10. Details of the New Construction participation numbers and impact sample, by stratum, can be seen in Table 3-21 and Table 3-32 in the main body report.

32 Some projects included more than one measure type.



G.2.1.3 Findings

The majority of sampled New Construction projects achieved realization rates for both demand and energy within 20% of expected values. Seven projects had verified energy savings values fall above 120% or below 80% of the reported values, while six fell outside the same zone for demand savings only. Navigant analyzed these projects to capture any trends in verified data. The following factors led to variation between the reported and verified savings and to the observed project level realization rates:

• The most common discrepancy between ex ante and ex post calculations was in the annual HOU for lighting measures. The evaluation team uncovered discrepancies both higher and lower than reported values. The CSP typically uses TRM deemed HOU in ex ante calculations. If the evaluation team verifies the customer’s reported HOU more than 10% greater or less than the TRM deemed HOU, the customer-reported site-specific HOU are used. Navigant’s analysis revealed that six of the 13 projects flagged above had runtimes that were more than 10% different from deemed values.

• The realization rates for residential HVAC systems were driven down due to changes in effective full load hours (EFLH). These three projects included hundreds of individual HVAC systems in residential spaces. Ex ante calculations used EFLH for Hotel – Lodging spaces. Navigant used EFLH for residential areas for HVAC systems installed in residential areas, reducing EFLH and therefore energy savings by about half.

• For Whole Building energy simulation projects, ex ante peak demand savings were calculated as annual energy savings divided by 8,760. This calculation represents the average annual demand savings but does not represent peak demand savings. Because building simulation software produces 8,760 hourly analyses, Navigant used these outputs to estimate peak demand savings rather than using average annual demand. This change in peak demand calculation methodology typically resulted in verified peak demand savings of more than double the reported savings.


As described in the Phase III Evaluation Plan33, Navigant did not complete any NTG research for the New Construction Solution in PY10. Instead, the team used the NTG ratios developed in PY9 and applied them to the PY10 gross verified impact results to arrive at PY10 net verified impact results. Navigant plans to update the NTG ratios during the PY11 evaluation.


As described in the Phase III Evaluation Plan34, Navigant did not complete any in-depth process evaluation activities for the New Construction Solution. Instead, the team interviewed the PECO program manager and CSP to identify significant implementation changes to inform the impact evaluation activities. No significant changes were found. The team carried out in-depth process evaluations in PY9 and plans to conduct additional activities in PY11.




G.3 Whole Building Solution

The Whole Building Solution offers the direct installation of comprehensive EE measures to customers who want to understand how to improve the overall energy performance of their small businesses. PECO and SmartWatt, the CSP, identify eligible small C&I customers with a monthly demand of less than 100 kW.35 The CSP conducts a comprehensive audit of the customer’s site and creates a proposal detailing the potential project upgrades, costs, and simple payback estimates. On average, PECO covers between 30% and 40% of the upgrade costs, up to a minimum of a 1-year simple payback to the customer. For the remainder of the project cost, PECO offers two financing options: 0% interest for the first 12 months, and long-term financing at 7% interest. The customer and CSP then agree on the project details and funding; the customer signs a formal contract; and the CSP begins installation. The CSP records the project details into PECO’s eTrack system to track program participation and claim energy savings. The Whole Building Solution defines participation in eTrack as an activity at a customer premise with a unique project number. A project can include more than one measure, with the impact evaluation sample defined at the project level.


As described in the Phase III Evaluation Plan36, Navigant used the realization rates calculated in PY9 and applied them to the PY10 reported energy and demand savings results to arrive at PY10 gross impact results. Navigant plans to update realization rates during the PY11 evaluation, including updating the impact evaluation activities to reflect changes recommended by the SWE during the PY9 evaluation effort.

G.3.1.1 Methodology

Navigant conducted a desk review of the Whole Building Solution in PY10, including record-level TRM review and an administrative cost analysis. Navigant’s desk review process included quarterly verification of program-reported savings in the program tracking database. The evaluation team applied energy and demand savings algorithms to verify that the input parameters, as specified in the PA TRM, used to calculate impacts were accurately reported within the program tracking data. The team completed desk reviews for a census of projects completed by Whole Building participants.

G.3.1.2 Sampling

As described in the Phase III Evaluation Plan37, Navigant did not conduct sampling activities for the Whole Building solution in PY10.

35 Religious institutions and non-profits with monthly demand of 200 kW or less are also eligible for participation in the Whole Building Solution. 36 PECO. Phase III Evaluation Plan, Energy Efficiency and Conservation Portfolio. Revised February 15, 2019. 37 PECO. Phase III Evaluation Plan, Energy Efficiency and Conservation Portfolio. Revised February 15, 2019.



G.3.1.3 Findings

Table G-1 summarizes the results of the desk review activities.

Table G-1. Small C&I Whole Building Solution Summary of Desk Review Activities

Impact Activity Finding Description


CFs on demand savings (kW) estimates were not aligned with the building types defined in the TRM.

The CSP applied a default CF for demand (kW) savings estimates instead of the building type CFs as defined in the TRM. This caused demand realization rates to be approximately 0.50 for lighting improvement and lighting control projects.

Administrative Cost Review No issues.

Navigant reviewed the administrative (direct install) costs to confirm that the amounts in the PECO tracking database matched those from the CSP data. Navigant found no discrepancies.



As described in the Phase III Evaluation Plan38, Navigant did not complete any NTG research for the Whole Building Solution in PY10. Instead, the team used the NTG ratios developed in PY9 and applied them to the PY10 gross verified impact results to arrive at PY10 net verified impact results. Navigant plans to update the NTG ratios during the PY11 evaluation.


As described in the Phase III Evaluation Plan39, Navigant did not complete any in-depth process evaluation activities for the Whole Building Solution. Instead, the team interviewed the PECO program manager and CSP to identify significant implementation changes to inform the impact evaluation activities. The team carried out in-depth process evaluations in PY9 and plans to conduct additional activities in PY11. Interviews with the PECO and CSP program managers covered topics such as CSP staff turnover at SmartWatt and greater emphasis on promoting non-lighting technologies. Findings from the interviews included the following:

• The CSP increased the emphasis on promoting non-lighting technologies through the solution. The CSP reported implementing new training opportunities for sales auditor staff including small business refrigeration technologies and vending machine improvement technologies. Sales auditors will continue to increase their familiarity with these types of upgrade opportunities and recommend them during the onsite energy audits.

• Staff turnover at the CSP affected the number of audits performed in PY10. Staff turnover required the CSP to train new personnel on the technical specifications of the measures offered through the solution and the sales skills needed to entice customers to install the measures. This training and ramp-up period took time away from calling customers and conducting energy audits.




G.4 Data Centers Targeted Market Segment

Projects in the Data Centers Targeted Market Segment are eligible to participate in the Equipment and Systems or C&I New Construction Solutions, depending on the details of the project. Data centers, on account of their high energy usage profiles and specialized technologies, are given special attention from the Small and Large C&I EE programs, allowing for tailored recruitment and implementation of such projects. Much of the energy savings in this segment come from cooling technologies, although the implementation of control systems and lighting are also possible. Participation is defined as an activity with a unique project number. More than one measure per participant is permitted, with the impact sample defined on the project level. ICF is the CSP for data center projects.


Navigant evaluated all three data center projects that participated in the program in PY10. Two of these projects fell under the Small C&I EE Program and both involved the installation of computer room air conditioning (CRAC) units. The remaining project fell under the Large C&I EE Program and involved the upgrade of an uninterruptible power supply (UPS). Navigant attempted site visits for all three sites. One customer in the Small C&I EE Program did not respond to requests for a site visit,40 so Navigant converted the project to a desk review. The remaining two projects both received onsite verification. The SWE conducted a desk review for one project in the Small C&I EE Program.


Navigant has not completed NTG research for the Data Centers Targeted Market Segment because participation has not been high enough to complete a full NTG calculation. The evaluation team plans to calculate NTG in PY11, pending participation.


As described in the Phase III Evaluation Plan41, Navigant did not complete any in-depth process evaluation activities for the Data Centers Targeted Market Segment. Instead, the team interviewed the PECO program manager and CSP to identify significant implementation changes to inform the impact evaluation activities. No significant changes were found. The evaluation team does not anticipate enough participation in PY11 to conduct further process evaluation research, however Navigant is working with PECO to further explore the reasons behind lack of participation.

40 Navigant contacted this customer seven times with a mix of emails and phone calls. 41 PECO. Phase III Evaluation Plan, Energy Efficiency and Conservation Portfolio. Revised February 15, 2019.


©2019 Guidehouse Inc. Page H-1

APPENDIX H. MULTIFAMILY TARGETED MARKET SEGMENT This appendix provides additional detail on the evaluation sample design, methods, and activities deployed in PY10 for the Multifamily Targeted Market Segment. The reader should refer to Section 3 in the main body report for evaluation findings, results, and conclusions. The Multifamily Targeted Market Segment is unique in that it contributes savings to the Residential EE, Small C&I EE, and Large C&I EE Programs. The decision makers for the projects in this targeted market segment consist of condominium owners, small multifamily building owners, property managers of large multifamily complexes, and executives at real estate investment companies that own multiple buildings in the PECO territory. Franklin Energy is the CSP for this program. The program offers direct install and prescriptive measures that are installed in common areas and in-unit tenant spaces of participating multifamily buildings.

H.1 Gross Impact Evaluation

H.1.1 Methodology

H.1.1.1 Desk Reviews

Navigant’s reviews included quarterly verification of program-reported savings in the program tracking database. The evaluation team applied energy and demand savings algorithms to verify that the input parameters, as specified in the PA TRM, were reported within the program tracking data. The team also compared the sampled project files and program tracking data to identify any discrepancies in measure locations, quantities, and reported savings.

H.1.1.2 Engineering File Reviews with Onsite Verification

Navigant conducted ex post verification activities for a sample of PY10 projects. The verification included an engineering project file review followed by onsite visits. Section H.1.2 details the sampling methodology. The engineering file reviews involved evaluating the input assumptions used to perform the ex ante calculations for a sub-sample of projects in the impact evaluation sample. Additionally, the team collected project-specific data such as make, model, count, and installation location of each measure, which Navigant’s field technicians later verified during onsite visits. The evaluation team conducted onsite verification of a sub-sample of projects within each of the sampled buildings. After the onsite verifications were completed, Navigant performed the ex post energy and demand savings calculations using the PA TRM algorithms, along with the input assumptions based on findings from the onsite verification effort. The detailed impact evaluation results by program are available in Sections 3.1.2, 3.3.2, and 3.4.2 in the main body report.



H.1.2 Sampling

Because there were no changes to the program structure in PY10, the sampling methodology remained relatively unchanged from PY9. The only change was in the sample size, which reflects the PY10 participation levels in each stratum. Navigant submitted a revised sampling plan42 with updated sample size for the impact evaluation during the beginning of PY10’s verification activities. The complex blend of market segments and audience types requires a comprehensive sampling methodology for evaluation, measurement, and verification (EM&V). Navigant stratified the population to develop a sample representative of all three program types (Residential EE, Small C&I EE, and Large C&I EE), buildings of all sizes and ownership structures, and direct install versus prescriptive measures. The team selected projects at random from each stratum to avoid biasing the sample. Table H-1 shows the final stratification.

Table H-1. Multifamily Targeted Market Segment Impact Evaluation Strata

Stratum Name Stratum Description Market Segments

Unit Basis

Impact Verification Method

Large – C&I Buildings in the large C&I market segment Large C&I Building File Review and

Onsite

Small – C&I Buildings in the small C&I market segment Small C&I Building File Review and

Onsite


Buildings with common areas in the C&I segments and units in the residential segment

Small C&I, Large C&I, Residential

Building File Review and Onsite

Large - Residential

Buildings in the residential market segment with a single decision maker for all projects in the building

Residential Building File Review and Onsite

Small - Residential

Projects in the residential market segment with individual decision makers

Residential Project (in-unit)

File Review and Onsite

Source: Navigant Navigant further sub-subsampled apartments and common area projects for engineering file review and onsite visits from each building in the sample. The PY10 sampling activities targeted the following confidence and precision levels for impact verification activities:

• Overall multifamily targeted market segment minimum: 85% confidence at 15% precision level

• Individual stratum minimum: 85% confidence at 50% precision level

• Individual sampled building or project minimum: 80% confidence at 20% precision level The target sample size provided in the sampling plan is based on estimated program participation levels. Table H-2 summarizes the estimated population size, the actual population size, and the updated sample size for each stratum needed to meet the target precision requirements.

42 PECO “PY10 Small and Large C&I EE Impact Sampling Design.” Dated April 16, 2019.



Table H-2. Multifamily Targeted Market Segment Updated Sample Size

Stratum Name

Unit Basis

Estimate Population

Size

Original Sample

Size

Actual Population

Size

Updated Sample

Size

Large – C&I Building 40 6 39 6

Small – C&I Building 60 6 69 7

Multisector – C&I and Res Building 26 6 19 5

Large – Residential Building 64 5 51 5

Small – Residential

Project (in-unit) 3,282 8 3,327 6

Total Building 3,472 31 3,505 29 Source: Navigant

The detailed gross impact evaluation sample design is provided in Table 3-3, Table 3-20, Table 3-30 in the main body report.

H.1.3 Findings

Navigant conducted onsite verification of 29 buildings43 making up 117 in-unit and 17 common area and exterior projects. The biggest factor influencing the ability to schedule site visits was availability of maintenance staff to dedicate time to the site visit. Tracking information about the installation locations inside the multifamily units and common areas was not always descriptive, resulting in extra time spent finding and verifying the installations while onsite. Overall, Navigant did not find any major discrepancies as a result of its engineering file reviews and onsite verification efforts. One discrepancy in the ex ante and verified savings came from small mismatches in the quantities of expected and verified lighting measures. Additionally, the CSP used 2.5 HOU and a 0.101 CF for all projects, regardless of space type. Navigant calculated the verified savings using weighted average HOU and CF values based on the space types in which the lamps were installed. This method leads to a more accurate estimation of the verified energy and demand savings.

H.2 Net Impact Evaluation

H.2.1 Methodology

Navigant surveyed multifamily tenants and landlords to gather information to inform free ridership and spillover for the Multifamily Targeted Market Segment. The evaluation team developed survey instruments consistent with the Phase III Evaluation Framework’s44 guidance on net impact evaluation

43 Due to difficulties scheduling, Navigant completed one extra site visit in the Small C&I stratum and one fewer site visit in the Large Residential stratum. 44 SWE. Evaluation Framework for Pennsylvania Act 129 Phase III Energy Efficiency and Conservation Programs. October 21, 2016. http://www.puc.pa.gov/Electric/pdf/Act129/SWE_PhaseIII-Evaluation_Framework102616.pdf




techniques45 and guidance from the Uniform Methods Project on estimating net savings.46 Navigant contracted with a third-party survey company to conduct phone surveys with a sample of participating tenants and a census of participating landlords. Using the Evaluation Framework methodology, Navigant asked tenants and landlords identified as decision makers47 if they would have purchased all, some, or none of the same program measures in the absence of the direct install program; participants were also asked to rate the influence of several key program elements in their decision to participate. The key questions determining free ridership focus on the influence of key program interventions such as free equipment, program information regarding efficient products, cost savings associated with the efficient equipment, the program’s educational materials and marketing, as well as the interviewee’s perception of what they would most likely have done in the absence of the program. The NTG evaluation also estimates spillover. Tenant and landlord interviews typically assess spillover by asking interviewees the extent to which additional energy efficient purchases were influenced by their experience with PECO’s multifamily EE offerings. Navigant quantified spillover from the interviews based on the quantity and type of efficient equipment purchased without a rebate, the degree of self-reported influence of the program on the decision to purchase the equipment, and confirmation via the survey data, the program tracking data, and online lookups that the product in question was not rebated. The team summed all quantifiable spillover and divided it by the total reported savings by strata to estimate the spillover rate.

H.2.2 Sampling

The NTG evaluation used the same sampling methodology as the process evaluation effort described in Appendix H.3.2.

H.2.3 Findings

Navigant calculated NTG results for tenants and landlords participating in the Multifamily Targeted Market Segment. Table H-3 shows the results by strata, as well as for the high impact measures identified for the PY10 analysis.

45 Pennsylvania PUC. Phase III Evaluation Framework. Section 3.4. http://www.puc.pa.gov/Electric/pdf/Act129/SWE_PhaseIII-Evaluation_Framework102616.pdf 46 The Uniform Methods Project. Estimating Net Savings: Common Practices. NREL. https://www.nrel.gov/docs/fy14osti/62678.pdf 47 Navigant only counted owners as decision makers in the residential tenant sample and treated renters as non-free riders for NTG purposes as in the PY8 evaluation. The evaluation team assumes renters would not conduct EE upgrades to a rented apartment outside of the multifamily solution offering.






Table H-3. Net-to-Gross Results for the Multifamily Targeted Market Segment

Strata Free Ridership Result

Estimated Participant Spillover NTG Ratio

Residential [1]

Small – Residential 0.08 0.00 0.92

Large – Residential 0.07 0.00 0.93

Multisector – Residential 0.13 0.00 0.87

Residential LEDs (HIM) 0.08 0.00 0.92

Low-Flow Faucet Aerators (HIM) 0.03 0.00 0.97

Low-Flow Showerheads (HIM) 0.06 0.00 0.94

C&I [1]

Small and Large C&I [2] 0.19 0.00 0.81

C&I LEDs (HIM) 0.19 0.00 0.81 [1] For the rolled up Residential EE and C&I EE Program NTG results, see Sections 3.1.3, 3.3.3, and 3.4.3 in the main body report. [2] For the Multifamily Targeted Market Segment NTG analysis, Navigant attempted a census among participating landlords generating a single NTG estimate for both the Small and Large C&I EE Programs. Source: Navigant analysis

Compared to the NTG results calculated in PY8, free ridership has decreased across all strata in the Residential and C&I multifamily sectors, suggesting the program is performing better in PY10.48 The move to offer more prescriptive measures such as lighting fixtures in the Large – Residential and C&I sectors may explain the decrease in free ridership because these customers would likely not have paid for the costly upgrades to lighting fixtures without the assistance of the program. Spillover for residential tenants dropped to zero in PY10 as participants did not report installing additional measures outside of the program. C&I participants did report installations of energy efficient or low flow toilets, but the quantifiable electric savings were minimal and had no significant impact on the NTG result.

H.3 Process Evaluation

H.3.1 Methodology

Navigant interviewed PECO and CSP program managers to get a better understanding of the following:

• Changes to the program design

• Market segments targeted by the program

• Improvements to measure mix

• Successes and challenges faced by CSP staff while conducting outreach

• Tactics used to close more prescriptive projects

• Barriers to participation

48 PY8 free ridership results: Sm. Residential – 0.12; Lg. Residential – 0.20; Residential LED HIM – 0.19; Sm. and Lg. C&I – 0.35.



Navigant also surveyed landlords, property managers, tenants, and condominium owners who participated in the program in PY10. The objective of these interviews was to gain insight into participant awareness about the program, assess their satisfaction with different aspects of the program, determine if the CSPs are successfully channeling participants toward other solutions and programs, and identify potential barriers to participation in the future. The evaluation team thoroughly reviewed the PY10 program tracking data to identify the tenant and landlord population before developing the phone survey samples. The following section provides more details about the sampling methodology.

H.3.2 Sampling Stratification

Based on the tracking database review, Navigant determined that the contact information for program participants varied among the three EE programs. The team found that tenant-level contact information was only available for units that were assigned to the Residential EE Program. Projects in the Small and Large C&I EE Programs only had property manager or site contact information in the tracking database. As a result, the evaluation team stratified the tenant population using the same approach as the impact evaluation sampling. For the Small and Large C&I EE Programs, Navigant grouped the landlord population into a single stratum. Table H-4 and Table H-5 provide an overview of the stratifications used for the tenant and landlord population.

Table H-4. Multifamily Targeted Market Segment Tenant Customer Experience Strata

Stratum Name Stratum Description Market

Segments Unit Targeted Respondent

Research Activities

Large – Residential

Projects in the residential market segment with a single decision maker for all projects in the building

Residential Project (in-unit) Tenant

NTG and Process


Buildings with common areas in the C&I segments and units in the residential segment


Project (in-unit) Tenant NTG and

Process

Small – Residential

Projects in the residential market segment with individual decision makers

Residential Project (in-unit) Tenant

NTG and Process

Source: Navigant

Table H-5. Multifamily Targeted Market Segment Landlord Customer Experience Strata

Stratum Name

Stratum Description

Market Segments Unit Targeted

Respondent Research Activities

Landlords

C&I and residential segment landlords


Building (landlord) Landlord

NTG and Process

Source: Navigant



The evaluation team designed both the landlord and tenant survey samples to meet a relative precision of 10% at the 90% confidence interval assuming a CV of 0.5.

H.3.3 Findings

Navigant completed the following tenant phone surveys:

• Large – Residential stratum: 36 (target 30)

• Small – Residential stratum: 24 (target 26)

• Multisector – C&I and Res stratum: 5 (target 18) Navigant experienced low response rates from the Multisector population. The team tried to compensate for the low responses by conducting additional Large – Residential surveys because the ownership structure of the buildings in those two strata are similar and most of the multisector properties are rental units. The team reached out to a census of the landlord population, with eight landlord surveys completed. Navigant experienced low response rates for the landlord survey despite making concerted efforts to achieve the target sample size without burdening participants. Partway through the survey effort, the team began offering a $100 incentive for a completed interview. Landlords were contacted an average of seven times. Multifamily Targeted Market Segment staff and CSP interviews revealed that some structural changes have been made to the program to streamline the implementation of prescriptive projects. More prescriptive lighting measures have been added to the measure mix and approximately 50% of program savings are now from prescriptive measures. PECO and CSP staff reported that while this measure mix adjustment has boosted program participation levels the program is still experiencing barriers in participation due to limited availability of maintenance staff at the buildings to accompany the installation teams. Overall, both tenants and landlords continue report high satisfaction with the program. The participants who indicated lower satisfaction with the program mostly attributed it to them not seeing a reduction in their electric bills and not having enough information about the program. This suggests an opportunity for PECO to improve their communication of the benefits of participating in the program. More details about program satisfaction can be found in Sections 3.1.4, 3.3.4, and 3.4.4 in the main body report. Navigant also asked Multifamily Targeted Market Segment tenants and landlords about their satisfaction with specific components of the program. Figure H-1 shows that tenants expressed overall satisfaction with the length of time to complete a visit (average of 4.4 out of a 5-point rating scale) and their overall experience during the installation visit (4.2 out of 5). Tenants provided slightly lower satisfaction scores for energy advisor’s communication with the tenant prior to the visit (3.8 out of 5) and the energy advisor’s ability to explain ways to make homes more efficient (3.9 out of 5).



Figure H-1. Tenant Multifamily Targeted Market Segment Participant Satisfaction with Program Components, n=65

Question: "On a scale of 1 to 5, how would you rate your satisfaction with the following: [program component]?" Source: Navigant analysis

Figure H-2 shows that landlords expressed high overall satisfaction with an energy advisor’s communication prior to their visit (average of 4.8 out of a 5-point rating scale) and with program information available on PECO’s website (4.7 out of 5). Landlords provided lower satisfaction scores for the length of time to complete the installation, quality of energy efficient products installed through the program, thoroughness of the energy advisor’s quality control, and the assessment report (4.3 out of 5).



Figure H-2. Landlord Multifamily Targeted Market Segment Participant Satisfaction with Program Components, n=8

Question: "On a scale of 1 to 5, how would you rate your satisfaction with the following: [program component]?" Source: Navigant analysis

Figure H-3 and Figure H-4 show that both tenants and landlords expressed satisfaction with PECO, in general, after their participation in the program. More than half of tenants and three of eight landlords stated they were extremely satisfied with PECO as a utility. Figure H-3. Tenant Multifamily Targeted Market Segment Participant Satisfaction with PECO, n=65

Question: "On a scale of 1 to 5, with 5 meaning Extremely Satisfied and 1 meaning Extremely Dissatisfied, how satisfied are you with PECO in general?" Source: Navigant analysis



Figure H-4. Landlord Multifamily Targeted Market Segment Participant Satisfaction with PECO, n=8

Question: " On a scale of 1 to 5, with 5 meaning Extremely Satisfied and 1 meaning Extremely Dissatisfied, how satisfied are you with PECO in general?" Source: Navigant analysis

Additionally, Figure H-5 and Figure H-6 show that nearly all of tenants and landlords indicated that they were likely to recommend the multifamily program to others.

Figure H-5. Tenant Likelihood of Recommending the Multifamily Targeted Market Segment to Others, n=65

Question: "On a scale of 1-5, with 5 meaning extremely likely and 1 meaning extremely unlikely, overall, how likely are you to recommend PECO’s program to others?" Source: Navigant analysis



Figure H-6. Landlord Likelihood of Recommending the Multifamily Targeted Market Segment to Others, n=8

Question: "On a scale of 1-5, with 5 meaning extremely likely and 1 meaning extremely unlikely, overall, how likely are you to recommend PECO’s Multifamily energy efficiency offering to others?" Source: Navigant analysis

final annual report to the pennsylvania public utility ...final annual report to the pennsylvania...

Documents