2018 indiana residential portfolio em&v report · 2018 indiana residential portfolio . em&v...

2018 Indiana Residential Portfolio EM&V Report

Volume I of II

Prepared for:Indiana Michigan Power

May 2019

Prepared by:

ADM Associates, Inc. 3239 Ramos Circle

Sacramento, CA95827 916.363.8383

Indiana Michigan Power Company

Table of Contents

1. Introduction .......................................................................................................................................... 1

1.1 Impact Evaluation Findings ........................................................................................................................ 1

1.2 Evaluation Findings and Recommendations ............................................................................................... 3

1.3 Cost Effectiveness Evaluation Findings ................................................................................................... 14

1.4 Organization of Report ............................................................................................................................. 14

2 Home Appliance Recycling ............................................................................................................... 16

2.1 Program Description ................................................................................................................................. 16

2.2 Estimation of Ex Post Gross Savings ....................................................................................................... 17

2.3 Estimation of Ex Post Net Savings ........................................................................................................... 25

2.4 Process Evaluation .................................................................................................................................... 32

2.5 Conclusions and Recommendations ......................................................................................................... 41

3 Home Energy Products - Appliances ................................................................................................. 43






4 Home Energy Products - Lighting...................................................................................................... 70






5 Home Energy Reports ........................................................................................................................ 91


5.2 Estimation of Ex Post Savings .................................................................................................................. 91


5.4 Conclusions and Recommendations ....................................................................................................... 109


6 Low Income Home Energy Reports ................................................................................................. 111

6.1 Program Description ............................................................................................................................... 111

6.2 Estimation of Ex Post Savings ................................................................................................................ 111

6.3 Process Evaluation .................................................................................................................................. 113


7 Usage Alerts ..................................................................................................................................... 116


7.2 Estimation of Ex Post Energy Savings ................................................................................................... 116


8 Residential Online Energy Check-up ............................................................................................... 120


8.2 Estimation of Ex Post Gross Savings ..................................................................................................... 121

8.3 Estimation of Ex Post Net Savings ......................................................................................................... 129



9 Home Weatherproofing .................................................................................................................... 146






10 Residential Income Qualified Weatherproofing ............................................................................... 174






11 Schools Energy Education ............................................................................................................... 191







12 Home Energy Management .............................................................................................................. 204






13 Residential New Construction .......................................................................................................... 237






14 Cost Effectiveness Evaluation .......................................................................................................... 253


Table of Tables

Table 1-1 Savings-Related Terminology ........................................................................................ 1

Table 1-2 Components of Impact Evaluation Accounted for in Savings Variables ....................... 2

Table 1-3 Summary of Energy Savings – PY2018 ......................................................................... 2

Table 1-4 Summary of Peak Demand Impacts – PY2018 .............................................................. 3

Table 1-5 Summary of PY2018 Benefit-Cost Ratios ................................................................... 14

Table 2-1 Participant Survey Sample Design ............................................................................... 17

Table 2-2 Ex Post Gross kWh Savings ......................................................................................... 20

Table 2-3 Ex Post Gross kW Savings ........................................................................................... 21

Table 2-4 Gross Realization Summary ......................................................................................... 21

Table 2-5 Verification Rates by Appliance Type ......................................................................... 22

Table 2-6 Recycled Appliances Verified to be in Working Condition ......................................... 22

Table 2-7 Uniform Methods Project UEC Regression Details ..................................................... 22

Table 2-8 2018 Average Refrigerator Characteristics .................................................................. 23

Table 2-9 Unit Energy Savings Adjusted for Partial Use ............................................................. 24

Table 2-10 Ex Post Net kWh Savings........................................................................................... 28

Table 2-11 Ex Post Net kW Savings............................................................................................. 28

Table 2-12 Prior Consideration of Disposal ................................................................................. 30

Table 2-13 Refrigerator Discard/Keep Distribution ..................................................................... 30

Table 2-14 Freezer Discard/Keep Distribution ............................................................................. 30

Table 2-15 Summary of Data Collection Activities for the Home Appliance Recycling Program............................................................................................................................................... 32

Table 2-16 Summary of Orders .................................................................................................... 32

Table 2-17 Summary of Unit Configuration ................................................................................. 33

Table 2-18 Interactions with Individuals Who Collected Appliances .......................................... 37

Table 2-19 Satisfaction Level with I&M as an Electric Service Provider .................................... 38

Table 2-20 Customers Suggestions to Improve the Appliance Recycling Program ..................... 39

Table 3-1 Estimated Energy Savings Factor – Wi-Fi Smart Thermostat ..................................... 53

Table 3-2 Home Energy Products Verified .................................................................................. 54

Table 3-3 Program-level Annual Gross kWh Savings .................................................................. 55

Table 3-4 Measure-level Annual Gross kWh Savings .................................................................. 55


Table 3-5 Program-level Gross kW Reduction ............................................................................. 56

Table 3-6 Measure-level Gross kW Reduction ............................................................................. 56

Table 3-7 Calculation of Trade Ally Influence Score ................................................................... 59

Table 3-8 Timing Adjustment Score............................................................................................. 59

Table 3-9 Survey Response Count and Average Free Ridership Score by Measure .................... 61


Table 3-11 Net kW Demand Reduction ........................................................................................ 61

Table 3-12 Summary of Data Collection Activities for the Home Energy Products – Appliances Program ................................................................................................................................. 62

Table 3-13 Summary of Program Activity ................................................................................... 62

Table 3-14 Time of Awareness of I&M’s Home Energy Products Program ............................... 65

Table 3-15 Sources of Information Consulted when Selecting Measure ..................................... 65

Table 3-16 Type of Store or Contractor Measure Purchased From .............................................. 66

Table 4-1 Baseline Wattage Determination .................................................................................. 72

Table 4-2 LED In-Service Rate – Lighting ................................................................................... 73

Table 4-3 Program-level Annual Gross kWh Savings – Lighting ................................................ 74

Table 4-4 Measure-level Annual Gross kWh Savings – Lighting ................................................ 74

Table 4-5 Program-level Gross kW Reductions – Lighting ......................................................... 74

Table 4-6 Program-level Gross kW Reductions – Lighting ......................................................... 74

Table 4-7 Free Ridership: Self-report Survey Results .................................................................. 78

Table 4-8 Price-Response Model Final Specification – Standard LEDs ...................................... 78

Table 4-9 Price-Response Model Final Specification – Specialty LEDs ..................................... 79

Table 4-10 Free Ridership: Price-response Model Results .......................................................... 79

Table 4-11 Final Free Ridership Estimation ................................................................................. 80

Table 4-12 Ex Post Net kWh Savings – Lighting ......................................................................... 80

Table 4-13 Ex Post Net kW Demand Reduction – Lighting ........................................................ 81

Table 4-14 Summary of Data Collection Activities for the Home Energy Products – Lighting Program ................................................................................................................................. 81

Table 4-15 Summary of Lamp Sales by Type .............................................................................. 82

Table 4-16 Distribution of Sales across Retailers ......................................................................... 82

Table 4-17 Retail Locations where Participants’ Purchased LED lightbulbs ............................... 85


Table 4-18 Existing Bulb Types Identified by Respondents ........................................................ 86

Table 4-19 Important Factors in Deciding What Light Bulb Type to Purchase ........................... 87

Table 4-20 How LEDs Bulbs Compare to Other Bulbs on Factors Important to Customers ....... 88

Table 4-21 2017 and 2018 Lighting Discount Awareness and LED Purchases Comparison ...... 89

Table 5-1 Pre-Period Average Daily Consumption, by Wave ...................................................... 92

Table 5-2 Description of Variables Used in the Regression Model ............................................. 93

Table 5-3 Summary of Energy Savings – PY2018 ....................................................................... 94

Table 5-4 Summary of Peak Demand Impacts – PY2018 ............................................................ 94

Table 5-5 HER Regression Model Output, by Wave ................................................................... 95

Table 5-6 kWh Savings, by Wave ................................................................................................ 95

Table 5-7 kW Peak Reduction, by Wave ...................................................................................... 96

Table 5-8 Summary of Data Collection Activities for the Home Energy Reports Program ........ 97

Table 5-9 Frequency of Reading Reports ................................................................................... 100

Table 5-10 Home Comparison and Expectations ....................................................................... 100

Table 5-11 Difficulty in Understanding the Information ............................................................ 101

Table 5-12 Accuracy of Information .......................................................................................... 101

Table 5-13 Why Customers Believe the Information on the HER is Inaccurate ........................ 101

Table 5-14 Usefulness of Recommendations ............................................................................. 103

Table 5-15 Why Customers did not find Recommendations Useful .......................................... 103

Table 5-16 Reasons for Not Accessing the Portal ...................................................................... 103

Table 5-17 Receipt of Challenge Emails .................................................................................... 104

Table 5-18 Response to Challenge Emails ................................................................................. 104

Table 5-19 Knowledge of Ways to Save Energy ........................................................................ 105

Table 5-20 Effort to Save Energy ............................................................................................... 105

Table 5-21 Actions Taken to Save Energy ................................................................................. 106

Table 5-22 Awareness of Efficiency Programs .......................................................................... 106

Table 5-23 Satisfaction with I&M as their Electricity Provider ................................................. 108

Table 6-1 Pre-Period Average Daily Consumption, by Wave .................................................... 112

Table 6-2 Summary of Energy Savings – PY2018 ..................................................................... 112

Table 6-3 Summary of Peak Demand Impacts – PY2018 .......................................................... 112

Table 6-4 HER Regression Model Output .................................................................................. 113


Table 6-5 kWh Savings ............................................................................................................... 113

Table 6-6 kW Peak Reduction .................................................................................................... 113

Table 6-7 Summary of Data Collection Activities for the Home Energy Reports Program ...... 114

Table 7-1 T-Test of Treatment and Control Group Baseline Period Energy Consumption ....... 118

Table 7-2 PY2018 I&M 5CP Hours ........................................................................................... 118

Table 7-3 Ex Post kWh Savings ................................................................................................. 119

Table 7-4 Ex Post Gross Peak kW Reduction –Usage Alerts ..................................................... 119

Table 7-5 Net kW Demand Reduction........................................................................................ 119

Table 8-1 Baseline Wattage Determination ................................................................................ 124

Table 8-2 In-Service Rates per OEC Measure by Kit Type ....................................................... 127

Table 8-3 Program-level Annual Gross kWh Savings ................................................................ 128

Table 8-4 Measure-level Annual Gross kWh Savings ................................................................ 128

Table 8-5 Ex Post Gross kWh Savings by Kit Type ................................................................... 128

Table 8-6 Program-level Gross kW Reduction ........................................................................... 129

Table 8-7 Survey Response Count and Average Free Ridership Score by Measure .................. 132



Table 8-10 Summary of Data Collection Activities for the Residential Online Energy Check-up............................................................................................................................................. 133

Table 8-11 Summary of Returned and Damaged Kits ................................................................ 134

Table 8-12 Reason for Completing the Residential Online Energy Check-up Survey and Receiving the Energy Efficiency Kit ................................................................................................... 136

Table 8-13 Ease of Taking the Survey in the Residential Online Energy Check-up Program ... 136

Table 8-14 Actions Taken to Reduce Energy Use in Home due to Recommendation ............... 137

Table 8-15 Other Actions Taken to Reduce Energy Use in Home due to Recommendation ..... 138

Table 8-16 Awareness of Other I&M Programs ......................................................................... 139

Table 8-17 Satisfaction Level with I&M as an Electric Service Provider .................................. 141

Table 8-18 2017 and 2018 Ease of Completing the Online Checkup Comparison .................... 143

Table 8-19 2017 and 2018 Usefulness of Recommendations Comparison ................................ 143

Table 9-1 Gross Impact Summary .............................................................................................. 156

Table 9-2 Measure Level Gross kWh Savings Summary ........................................................... 156


Table 9-3 Measure Level Gross Demand Reduction Summary ................................................. 157

Table 9-4 Timing Adjustment Score........................................................................................... 161

Table 9-5 Survey Response Count and Average Free Ridership Score by Measure .................. 162



Table 9-8 Summary of Data Collection Activities for the Home Weatherproofing Program .... 163

Table 9-9 Summary of Services Received .................................................................................. 163

Table 9-10 Summary of Recommendations Received ................................................................ 164

Table 9-11 Summary of Measures Installed ............................................................................... 164

Table 9-12 Reasons for Participation .......................................................................................... 166

Table 9-13 Experience with Home Energy Assessor ................................................................. 167

Table 9-14 Satisfaction with I&M as Electricity Service Provider ............................................ 170

Table 9-15 2017 and 2018 Experience with Home Energy Assessor ......................................... 172

Table 10-1 Gross Impact Summary ............................................................................................ 179

Table 10-2 Measure Level Gross kWh Savings Summary ......................................................... 180

Table 10-3 Measure Level Gross Demand Reduction Summary ............................................... 180

Table 10-4 Ex Post Net kWh Savings......................................................................................... 181

Table 10-5 Net kW Demand Reduction ...................................................................................... 181

Table 10-6 Summary of Data Collection Activities for the Income Qualified Weatherproofing Program ............................................................................................................................... 182

Table 10-7 Summary of Services Received ................................................................................ 182

Table 10-8 Summary of Measure Savings .................................................................................. 182

Table 10-9 Reasons for Participation .......................................................................................... 184

Table 10-10 Experience with Home Energy Assessor ............................................................... 185

Table 10-11 Experience with Home Energy Assessor ............................................................... 189

Table 11-1 Baseline Wattage Determination .............................................................................. 194

Table 11-2 Installation Rates per SEE Measure ......................................................................... 198

Table 11-3 Program-level Annual Gross kWh Savings .............................................................. 198

Table 11-4 Ex Post Gross kWh Savings by Measure ................................................................. 199

Table 11-5 Program-level Gross kW Demand Reduction .......................................................... 199

Table 11-6 Summary of Evaluations Reviewed ......................................................................... 200


Table 11-7 Measure-Level Net-to-Gross Ratios ......................................................................... 200



Table 11-10 Summary of Data Collection Activities for the Schools Energy Education Program............................................................................................................................................. 202

Table 12-1 Number of Data Points by Thermostat Mode ........................................................... 206

Table 12-2 Energy Savings Factor – Wi-Fi Smart Thermostat .................................................. 208

Table 12-3 Key Variables ........................................................................................................... 208

Table 12-4 PY2018 I&M 5CP Hours ......................................................................................... 211

Table 12-5 Annual Gross kWh Savings – Wi-Fi Smart Thermostats ......................................... 212

Table 12-6 Parameter Estimates for Regression Model Estimating Runtime ............................ 212

Table 12-7 kWh Savings ............................................................................................................. 213

Table 12-8 Ex-Post Gross kWh Savings - Orchestrated Energy ................................................ 213

Table 12-9 Demand Response kW Reduction for Locations with One Thermostat - Orchestrated Energy ................................................................................................................................. 214

Table 12-10 Demand Response kW Reduction for Locations with Two Thermostats - Orchestrated Energy ................................................................................................................................. 214

Table 12-11 Program-Level Ex Post Gross kWh Savings .......................................................... 215

Table 12-12 Ex Post Gross Peak kW Reduction – Orchestrated Energy ................................... 216

Table 12-13 Ex Post Gross kW Savings ..................................................................................... 216

Table 12-14 Timing Adjustment Score ....................................................................................... 218

Table 12-15 Survey Response Count and Average Free Ridership Score .................................. 219

Table 12-16 Ex Post Net kWh Savings....................................................................................... 219

Table 12-17 Ex Post Net kW Savings......................................................................................... 220

Table 12-18 Summary of Data Collection Activities for the Home Energy Management Program............................................................................................................................................. 220

Table 12-19 Ease of Enrolling in the Home Energy Management Program .............................. 222

Table 12-20 Summary of Concerns about Participating ............................................................. 223

Table 12-21 Type of Thermostat Wi-Fi Thermostat Replaced ................................................... 224

Table 12-22 Ease of Use of the I&M Home App ....................................................................... 225

Table 12-23 Home Comfort Level Since Participating and During Peak Events ....................... 226

Table 12-24 Customer Cross-Program Awareness ..................................................................... 227


Table 12-25 Customer Comments Regarding Dissatisfaction with the Program ....................... 230

Table 12-26 Satisfaction Level with I&M as an Electric Service Provider ................................ 232

Table 12-27 2017 and 2018 Concerns About Participating When Signing Up Comparison ..... 233

Table 13-1 New Construction Database Savings by Home Type............................................... 239

Table 13-2 Gross Impact Summary ........................................................................................... 241

Table 13-3 kWh Savings by House Tier ..................................................................................... 242

Table 13-4 Demand Reductions by House Tier .......................................................................... 243

Table 13-5 Average Free Ridership Score by Home Builder ..................................................... 246



Table 13-8 Summary of Data Collection Activities for the Residential New Construction Program............................................................................................................................................. 247

Table 13-9 Number and Share of Completed Homes by Tier and Home Type ......................... 247

Table 14-1 Summary of Benefits and Costs Included in each Cost Effectiveness Test ............. 253

Table 14-2 Home Appliance Recycling Program Cost Test Inputs and Results ........................ 254

Table 14-3 Home Energy Products Program Cost Test Inputs and Results ............................... 254

Table 14-4 Home Energy Engagement Program Cost Test Inputs and Results ......................... 254

Table 14-5 Home Weatherproofing Program Cost Test Inputs and Results .............................. 255

Table 14-6 Residential Income Qualified Weatherproofing Program Cost Test Inputs and Results............................................................................................................................................. 255

Table 14-7 Schools Energy Education Program Cost Test Inputs and Results .......................... 255

Table 14-8 Home Energy Management Program Cost Test Inputs and Results ........................ 255

Table 14-9 Residential New Construction Program Cost Test Inputs and Results .................... 256


Table of Figures

Figure 2-1 Average Daily Load Profile ........................................................................................ 25

Figure 2-2 Net Savings Calculation Summary Diagram .............................................................. 28

Figure 2-3 NTGR Calculation – Refrigerators ............................................................................. 31

Figure 2-4 NTGR Calculation – Freezers ..................................................................................... 31

Figure 2-5 Summary of Age of Recycled Units ........................................................................... 33

Figure 2-6 Weekly and Cumulative Units Recycled .................................................................... 34

Figure 2-7 Initial Source of Program Awareness ......................................................................... 36

Figure 2-8 Customer Satisfaction with Residential Appliance Recycling Program ..................... 38

Figure 2-9 2017 and 2018 Program Awareness Comparison ....................................................... 40

Figure 2-10 2017 and 2018 Scheduling Comparison ................................................................... 40

Figure 3-1 Initial Source of Program Awareness ......................................................................... 64

Figure 3-2 Reason for Choosing Model or Type of Measure ....................................................... 66

Figure 3-3 Home Energy Products Appliances Satisfaction......................................................... 67

Figure 3-4 2017 and 2018 Program Awareness Comparison ....................................................... 68

Figure 3-5 2017 and 2018 Reasons for Selecting Measure (Model or Type) Comparison .......... 68

Figure 4-1 Free ridership Scoring for Targeted Random Dialing Survey Respondents ............... 76

Figure 4-2 Summary of Weekly and Cumulative Sales During the Program Year ...................... 83

Figure 4-3 Quantity of Lightbulbs Purchased ............................................................................... 85

Figure 4-4 Sources of Awareness ................................................................................................. 87

Figure 4-5 2017 and 2018 Retail Stores Comparison ................................................................... 88

Figure 4-6 2017 and 2018 LED Replacements Comparison ........................................................ 89

Figure 5-1: 2018 Home Energy Report with New Template ........................................................ 99

Figure 5-2 Assessment of the WattsUp Energy Portal ............................................................... 104

Figure 5-3 Source of Awareness ................................................................................................. 107

Figure 5-4 Satisfaction with Home Energy Report Program Components................................. 107

Figure 5-5 2017 and 2018 Accuracy of Home Energy Usage Comparison ............................... 108

Figure 5-6 2017 and 2018 Usefulness of Home Energy Report’s Recommendations Comparison............................................................................................................................................. 109

Figure 5-7 2017 and 2018 Home Energy Report Satisfaction Comparison ............................... 109

Figure 8-1 Kits Shipped by Month ............................................................................................. 133


Figure 8-2 Initial Source of Program Awareness ....................................................................... 135

Figure 8-3 Number of Times Survey Participants Logged on to Audit Tool ............................. 136

Figure 8-4 Usefulness of Residential Online Energy Check-up Program Recommendations ... 137

Figure 8-5 Customer Knowledge of Energy Usage and Savings in Household ......................... 140

Figure 8-6 Customer Efforts to Save Energy in the Home ......................................................... 140

Figure 8-7 Customer Satisfaction with the Online Energy Check-up Service, Kit Items, and Information Provided .......................................................................................................... 141

Figure 8-8 2017 and 2018 Program Awareness Comparison ..................................................... 142

Figure 8-9 2017 and 2018 Program Rebate and Incentive Awareness Comparison .................. 143

Figure 8-10 2017 and 2018 Comparison of Satisfaction with Kit Items and Information Provided by the Online Energy Checkup ........................................................................................... 144

Figure 9-1 Sources of Program Awareness ................................................................................ 166

Figure 9-2 Experience with Scheduling Energy Assessment ..................................................... 167

Figure 9-3 Helpfulness of Home Energy Report ........................................................................ 168

Figure 9-4 Energy Saving Actions Taken by Program Participants ........................................... 168

Figure 9-5 Program Influence Ratings ........................................................................................ 169

Figure 9-6 Program Satisfaction ................................................................................................. 170

Figure 9-7 2017 and 2018 Ease of Scheduling Home Energy Assessment Comparison ........... 171

Figure 9-8 2017 and 2018 Implemented Energy Saving Actions ............................................... 171

Figure 10-1 Sources of Program Awareness .............................................................................. 184

Figure 10-2 Experience with Scheduling Energy Assessment ................................................... 185

Figure 10-3 Helpfulness of Home Energy Report ...................................................................... 186

Figure 10-4 Energy Saving Actions Taken by Program Participants ......................................... 186

Figure 10-5 Program Influence Ratings ...................................................................................... 187

Figure 10-6 Program Satisfaction ............................................................................................... 188

Figure 10-7 2017 and 2018 Ease of Scheduling Home Energy Assessment Comparison ......... 188

Figure 10-8 2017 and 2018 Implemented Energy Saving Actions Comparison ........................ 189

Figure 11-1 Distribution of Net-to-Gross Ratios by Program Measure ..................................... 200

Figure 12-1 May 31st Demand Response Event ......................................................................... 213

Figure 12-2 Initial Source of Program Awareness ..................................................................... 221

Figure 12-3 Reason for Participating in Home Energy Management Program .......................... 222


Figure 12-4 Clarity of Information Regarding the Home Energy Management Program .......... 223

Figure 12-5 Usefulness of the IM Home App’s features ............................................................ 225

Figure 12-6 Peak Events ............................................................................................................. 226

Figure 12-7 Customer Knowledge of Energy Use and Savings in Household ........................... 227

Figure 12-8 Customer Efforts to Save Energy in their Homes ................................................... 228

Figure 12-9 Overall Customer Satisfaction with the Sign-up Process, IM Home App, Thermostat and Installation .................................................................................................................... 229

Figure 12-10 Overall Customer Satisfaction with Home Energy Management Program .......... 229

Figure 12-11 2017 and 2018 Program Awareness Comparison ................................................. 232

Figure 12-12 2017 and 2018 Reasons for Participating in Home Energy Management Program Comparison ......................................................................................................................... 233

Figure 12-13 2017 and 2018 Ease of Enrolling in the Home Energy Management Program Comparison ......................................................................................................................... 233

Figure 12-14 2017 and 2018 Thermostat Installation Comparison ............................................ 234

Figure 13-1 2009 IECC UDRH Specifications ........................................................................... 240

Figure 13-2 Ex Post kWh Savings vs. HERS Score - Electric ................................................... 241

Figure 13-3 Ex Post kWh Savings vs. HERS Score - Gas/Electric ............................................ 241

Figure 13-4 Summary of Number of Homes Completed by Builders ........................................ 248


1. Introduction Under contract with the Indiana Michigan Power (I&M), ADM Associates, Inc., (ADM) performed evaluation, measurement and verification (EM&V) activities that confirmed the energy savings (kWh) and demand reduction (kW) realized through the energy efficiency programs that I&M implemented in Indiana during the during January 2018 through December 2018 (PY2018).

This chapter provides a summary of evaluation findings for the residential program portfolio and presents information regarding the organization of the report.

1.1 Impact Evaluation Findings

The savings variables presented in this evaluation report are defined in Table 1-1.

Table 1-1 Savings-Related Terminology

Variable Definition

kWh Savings Goal kWh Savings Goal is the energy savings goal cited in the applicable portfolio plan.

Ex Ante Gross kWh Savings Ex Ante Gross kWh Savings are the annual energy savings reported by I&M and are typically obtained from I&M’s DSM/EE Program Scorecard documents.

Gross Audited kWh Savings Gross Audited kWh Savings are determined by reviewing tracking data presenting for any errors, and adjusting Ex Ante Gross kWh Savings accordingly.

Gross Verified kWh Savings

Gross Verified kWh Savings are determined by applying an installation rate to the Gross Audited kWh Savings. The installation rate is defined as the ratio of units that were installed (verified) to the number of units reported (claimed).

Ex Post Gross kWh Savings Ex Post Gross kWh Savings are the realized annual gross kWh savings reflecting all adjustments made by ADM, without accounting for free ridership or spillover.

Ex Post Net kWh Savings Ex Post Net kWh Savings are equal to Ex Post Gross kWh Savings, adjusted to account for free ridership and spillover.

Ex Post Net Lifetime kWh Savings

Ex Post Net Lifetime kWh Savings is the Ex Post Net kWh Savings occurring over the course of the applicable measure effective useful life (EUL).

Gross Realization Rate Gross Realization Rate is equal to Ex Post Gross kWh Savings divided by Ex Ante Gross kWh Savings.

Net-to-Gross Ratio Net-to-Gross Ratio is equal to Ex Post Net kWh Savings divided by Ex Post Gross kWh Savings.

Based on the definitions presented in Table 1-1, Table 1-2 presents a summary of the components of the impact evaluation that are accounted for in savings variables presented in this report.


Table 1-2 Components of Impact Evaluation Accounted for in Savings Variables

Category Tracking

Data Review

In-Service Rates

Ex Post Gross

Analysis

Net-to-Gross

Analysis Gross Audited ✓ Gross Verified ✓ ✓ Ex Post Gross ✓ ✓ ✓ Ex Post Net ✓ ✓ ✓ ✓

ADM performed EM&V activities for 13 residential programs offered by I&M during PY2018. Total residential portfolio ex post gross energy savings are 54,065,670 kWh, while ex post net energy savings are 45,029,916 kWh, as shown in Table 1-3.

Table 1-3 Summary of Energy Savings – PY2018

Program

Ex Ante Annual

kWh Savings

Gross Audited

kWh Savings

Gross Verified

kWh Savings

Ex Post Annual

Gross kWh Savings

Gross Realization

Rate

Ex Post Annual

Net kWh Savings

Net-to-Gross Ratio

Lifetime Net Ex Post

kWh Savings

Home Appliance Recycling 3,794,742 3,794,742 3,728,343 3,866,926 102% 1,873,967 48% 14,991,732

Home Energy Products - Appliances 1,233,192 1,233,106 1,231,521 1,302,679 106% 792,901 61% 11,402,084

Home Energy Products - Lighting 19,618,027 19,618,027 14,953,713 14,965,413 76% 9,167,354 61% 86,391,744

Home Energy Reports 23,449,092 23,449,092 23,449,092 24,419,945 104% 24,419,945 100% 24,419,945

Low Income Home Energy Reports 329,697 329,697 329,697 292,738 89% 292,738 100% 292,738

Usage Alerts 779,957 779,957 779,957 449,852 58% 449,852 100% 449,852 Residential Online Energy Check-up 2,532,116 2,532,116 1,820,350 3,498,139 138% 3,033,683 87% 31,142,298

Home Weatherproofing 533,363 533,363 522,541 527,876 99% 489,865 93% 6,633,226

Residential Income Qualified Weatherproofing

294,440 294,440 273,262 299,907 102% 299,907 100% 4,231,179

Schools Energy Education 2,140,600 2,140,600 719,747 2,785,095 130% 2,680,178 96% 27,411,597

Home Energy Management 429,205 429,205 421,752 480,810 112% 519,446 108% 7,791,691

Residential New Construction 1,029,206 1,029,206 1,029,206 1,176,290 114% 1,010,080 86% 25,252,012

Residential Portfolio Total 56,163,637 56,163,551 49,259,181 54,065,670 96% 45,029,916 83% 240,410,099

Total residential portfolio ex post gross peak demand savings are 7,604.58 kW, while ex post net peak demand savings are 6,382.91 kW, as shown in Table 1-4.


Table 1-4 Summary of Peak Demand Impacts – PY2018

Program

Ex Ante Gross

kW Savings

Gross Audited

kW Savings

Gross Verified

kW Savings

Ex Post Gross kW Savings

Gross Realization

Rate

Ex Post Net kW Savings

Net-to-Gross Ratio

Home Appliance Recycling 447.86 447.86 440.06 456.58 102% 221.34 48% Home Energy Products - Appliances 312.50 312.50 312.14 252.05 81% 153.42 61% Home Energy Products - Lighting 2,687.37 2,687.37 2,048.43 2,049.64 76% 1,255.55 61% Home Energy Reports 2,673.20 2,673.20 2,673.20 2,787.66 104% 2,787.66 100% Low Income Home Energy Reports 37.59 37.59 37.59 33.42 89% 33.42 100% Usage Alerts 88.92 88.92 88.92 88.41 99% 88.41 100% Residential Online Energy Check-up 157.51 157.51 123.16 218.36 139% 180.62 83% Home Weatherproofing 39.99 39.99 39.43 46.62 117% 42.76 92% Residential Income Qualified Weatherproofing 24.62 24.62 23.75 28.69 117% 28.69 100%

Schools Energy Education 242.00 242.00 76.87 254.47 105% 237.30 93% Home Energy Management 1,008.91 1,008.91 1,008.91 577.80 57% 662.78 115% Residential New Construction 595.29 595.29 595.29 810.87 136% 690.97 85% Residential Portfolio Total 8,315.76 8,315.76 7,467.75 7,604.58 91% 6,382.91 84%

1.2 Evaluation Findings and Recommendations

The evaluation findings are summarized by program below.

Home Appliance Recycling

Findings and Conclusions:

The program exceeded its goal. The Appliance Recycling program experienced the highest level of participation since inception and exceeded goals for 2018. The program exceeded savings targets in Indiana and collected 4,170 units during PY2018.

Diverse approaches were used to market the program. Primary marketing approaches include bill inserts, monthly e-newsletter, email campaigns, social media, internet searches, I&M’s website, and cross-promotion through other I&M programs. Customers most frequently reported learning of the program from a bill insert (24%) followed by brochures from I&M (20%) and the program website (13%).

Revised online scheduling process. The Appliance Recycling program revised the online scheduling process in Spring 2018 based on analysis of what point in completing the enrollment form customers stop using the online form. Staff reported that the revision lead to an increase in online enrollments. The participant survey responses indicated that 48% of participants enrolled online in PY2018 as compared to 41% in PY2017.

Few customers reported difficulty with the online and telephone sign-up process. Ninety-five percent or more of customers who enrolled in the program online stated that it was easy to find the sign-up screen, that the website answered all their questions about


participating, and that they received a confirmation that their sign-up was successful. Additionally, all respondents who signed by phone reported that the representative was polite and courteous.

Pick up process working as designed. Nearly all (99%) of customers reported that the pick-up crew was courteous and professional. Additionally, 94% reported that the appliance was plugged in at the time of pick-up and 95% reported that the crew checked that the appliance was working.

Survey responses indicate that about 48% of customers who recalled the sign-up process enrolled in the program online. The majority of these respondents did not encounter any issues with the sign-up process but approximately two percent reported that they had encountered issues with the program website, such as pages not loading, or submission of the application not working properly. Open-ended commentary from these respondents also suggests that some customers found the information on the program website to be unclear. Sixty percent of respondents that signed up by phone were aware that they could sign up online. These customers generally did not indicate any problems during the phone sign up though 12% indicated that they had to call more than one time.

Program satisfaction is high. Ninety-five percent of people were satisfied with the program overall. Ninety-six percent of all respondents reported that they were able to schedule a convenient appliance pick-up time and 99% of respondents indicated that the people picking up the appliances were courteous and professional.

Recommendations:

ADM recommends that ARCA continues to ensure compliance with program rules, particularly with regard to the number of units recycled per customer per year.

Home Energy Products – Appliances


The program exceeded its goal. The Home Energy Products – Appliances program exceeded its savings goals in 2018.

An extensive effort was undertaken to increase uptake of HVAC measures. I&M program staff organized five training seminars on ductless systems in spring 2018 and partnered with Mitsubishi to provide the trainings for contractors. Additionally, I&M heavily focused on relationship building with distributors and contractors in 2018 and developed relationships with Mitsubishi and Fujitsu. Through the relationship, there was marketing and promotion of ductless heat pump systems. Moreover, the outreach to contractors is a positive development as 51% of the respondents learned of the rebates from contractors and 36% purchased the equipment because the contractor recommended it. The Home Energy Products – Appliances program also offered a double rebate for the ductless


heat pump systems in the spring and fall. The overall impact of these actions was that the share of ex ante savings for HVAC systems replacements increased by approximately 16 percentage points, and the share of savings from ductless heat pumps increased by 18 percentage points.

Multiple marketing channels were used in PY2018. The program also marketed rebates directly to customers including email campaigns, Google ads, and a post-card campaign to promote ductless heat pumps. Contractors and plumbers (mentioned by 51%) were the most commonly mentioned sources of program awareness, followed by I&M website (21%) and retail stores (8%).

Application process was streamlined. Based on meetings with contractors, I&M modified and streamlined the application process for the program. Additional changes are being considered for 2019 including the development of a contractor web portal for submitting applications, checking the status of rebates, and reviewing the number of jobs scheduled.

Participants are satisfied with the program. Both participants and trade allies are satisfied with the program. Ninety-two percent of participants were satisfied with the program overall and 88% were satisfied with the application process. Additionally, 86% were satisfied on the savings on their electricity bills since installing the rebated equipment.

Recommendations:

Consider reducing incentives for central air conditioners. The cost per ex ante kWh saved was somewhat high ($1.62) and considerably higher than other measures.

Home Energy Products – Lighting


The savings goal was met. The lighting program met an increased savings goal for PY2018. The program-level gross realization rate is 76%. The discrepancy between ex ante and ex post kW savings is almost exclusively accounted for by the differences in in-service rates referenced in ex ante and ex post savings calculations.

Program changes were made to decrease free ridership. The program made several changes to increase its influence on customer lighting purchases. These changes included redesigning point-of-purchase marketing materials to make them stand out more to customers, increasing the number of store events, and increasing visits from program staff to improve awareness of discounts. Additionally, the program discounts were promoted through other channels such as social media campaigns to targeted zip codes and bill inserts. Survey results indicate that in-store materials and the I&M website were the most effective at increasing customer awareness of the discounts. Based on findings from the general population survey, of those customers who reported purchasing LED lightbulbs


and who were aware of I&M discounts, 33% learned of the discounts through bill inserts, 20% through an email, 13% from I&M website, and 13% from in-store signage.

Online lighting pilot store added. The program added an online lighting store. The store was a success and generated 5.2% of the bulb sales for the year.

A retailer training guide was developed. The program developed a guide to use in training retail staff on lighting products. The guides are provided to participating retailers.

Additional stores were added to target harder to reach customers. The program enrolled stores such as Ace Hardware, Walgreens, and Habitat for Humanity Restores, and Goodwill to target hard to reach customers. Menards and Walmart were the most common retailers where customers indicated they purchased light bulbs.

Recommendations:

Collect and provide customer contact information for online bulb purchasers. Because the online store presents a different and more direct way to influence customer purchases, the net impacts of sales through this channel may differ from the midstream discount channel. Consequently, ADM recommends a survey of these customers to assess net savings in the future. This would require requesting customers’ contact information at the time of the online purchase.

Home Energy Reports


Efforts were taken to increase program savings impacts. The program modified the home energy reports to increase impact. These changes included language changes (i.e., referring to household instead of home and referring to energy savings instead of energy efficiency), and adding a caption to chart of electricity use over time. Additionally, efforts were made to make the appearance of the report, the energy challenge emails, and the high bill alerts more similar so that customers recognized each as a communication from I&M.

Additional control over I&M content did not adversely impact opt-out rates. A enhancements allows customers to opt-in or opt-out of all the possible Home Energy Report communications to give customers more control over the content they receive from I&M. Staff reported that that opt-out rates for the HERs remained largely unchanged.

Most customers report frequently reading the home energy reports. Fifty-one percent of customers stated that they read all the reports and another 24% reported that they had read at least 75% of the reports.

Most customers find the information easy to understand. Ninety-one percent of customers reported that the information in the report was very or somewhat easy to understand.


Many customers reported that the information on their home was accurate, but some noted inaccuracies. Sixty-two percent reported that the information on energy usage was very or somewhat accurate. However, 18% rated it as somewhat or very inaccurate. Common inaccuracies noted was the information on the home size and a lack of information on the type of fuel their heating system or other appliance used. Additionally, 54% of customers found the recommendations to be somewhat or very useful.

Satisfaction with the reports is generally high. Sixty-nine percent of the treatment group reported that they were very or somewhat satisfied with the home energy reports and 72% were very or somewhat satisfied with WattsUp Energy portal. Satisfaction with the challenge emails and contests was somewhat lower, but this was largely due to larger shares of customers who were not particularly satisfied or dissatisfied rather than higher levels of dissatisfaction.

Recommendations:

Use focus groups and/or A/B testing to develop and test customized marketing and messaging approaches. I&M expressed a desire to increase customized marketing and targeting messaging to customers. Focus groups can be an effective means to developing messaging and identifying programs that fit customer market segments. A/B testing can be used to assess the effectiveness of these approaches by monitoring open and click through rates.

Consider adding a mechanism to allow customers to correct information they believe to be inaccurate or to provide additional information on their home characteristics. Some customers cited inaccuracies presented in the home energy report and customers were less likely to act on information that they do not believe is credible. Consider developing a portal or other approach that would allow customers to modify or provide additional information on their home to improve the accuracy of the customer information.

Low Income Home Energy Reports


The low-income program is similar in design to the mass market HER program, but messaging is specific to low-income customers. The Low Income HER program runs and is operated much like mass-market program, but the messaging in the report is targeted towards lower income customers (i.e., the reports do not recommend expensive retrofits). Tendril used I&M data on customers receipt of energy assistance along with data provided by Experian to identify likely low-income customers.

The QA/QC process was modified. There have been a few changes to quality control/quality assurances procedures that apply to the Low Income program as well as the mass-market program. Tendril has built more automated quality control checks and have


added personnel to the operations. There was more of a focus on QC/QA in 2018 to streamline processes and procedures. Under the process, there is a QC checklist (some manual and some automated) that is completed prior to sending a HER. There are over 100 QC checks for a sample set of reports. About 100 to 150 reports are examined manually and the rest are examined at through an automated process.

Recommendations:

ADM does not have any recommendations for the Low Income Home Energy Reports Program.

Residential Online Energy Check-up


The number of kits distributed increased, and the program nearly met its distribution goal. The program achieved 98% of its revised goal for the number of kits distributed in 2018. Although it missed this target by a small amount, the number of kits distributed represents an increase from 2017.

Multiple marketing channels are used to promote the program. The program is promoted through the home energy reports program, the I&M program website, bill inserts, email and post card campaigns, and cross promoted to participants in the weatherization program. Staff noted that the program activity is responsive to marketing efforts.

Few changes made to the program. The program added an insert in the kits to cross-promote other programs. The kit contents remained the same.

Kit delivery is efficient and effective. Review of program tracking data found that the rate of returned or damaged kits was 1%, indicating that that the process of delivering the kit items is efficient and effective.

Most participants found it easy to enroll and the recommendations useful. Ninety-three percent of participants responded that enrolling was very or somewhat easy. Additionally, 74% found the recommendations to be very or somewhat useful.

Nearly all participants reported taking actions based on a recommendation in the report to save energy. Ninety-six percent of respondents reported taking one or more actions based on a recommendation made in the report received. Turning lights off more frequently, limiting washing to full loads, and using cold water were the three most frequently reported actions taken.

Recommendations:

ADM does not have any evaluation recommendations for the Online Energy Check-up program.


Home Weatherproofing


The program fell short of its savings goal but the number of participants increased. Ex ante savings totaled 533,363 kWh as compared to an adjusted goal of 851,554 kWh. The number of participants increased from 297 households in 2017 to 313 households in 2018. I&M will continue to improve the program in 2019, but no major changes are planned. Program staff will conduct more workshops with various organizations around the state (e.g., food banks, schools) in order to reach more customers.

The program was marketed through multiple means. These included promotion in the I&M newsletter, bill inserts, digital content, targeted emails, social media, post-cards, and cross-promotion through other programs. The program also ran an advertisement that generated a high number of calls.

The Home Weatherproofing program began charging a $99 assessment fee to increase the conversion rate. Staff indicated that an assessment fee was charged to increase uptake of major measures. This does not apply to the Income Qualified Weatherproofing Program. Customers who elect to do major measures have the full amount of the assessment fee applied to the rebate for the measures. The change may have been a factor in the increase in the share of households that received a rebate for a major measure (33% in 2018 vs. 14% in 2017). As such, the program is operating more efficiently with the addition of the $99 fee and the addition of the fee does not seem to have adversely impacted participation.

Most participants are satisfied with the program overall. Eighty-seven percent of participants reported that they were satisfied with the program overall. In addition, 94% of participants were very or somewhat satisfied with the energy audit and 96% were very or somewhat satisfied with the quality of work performed.

Recommendations:

Consider additional marketing outreach efforts to increase participation. Although the program was extensively market in 2018, the nonparticipant survey found low-levels of program awareness, including among customers with electric heating. Staff should explore the feasibility of targeting marketing to customers with electric heating. Possible approaches to this include leveraging data sources such as the Online Check-up responses provided by customers or targeting customers with energy usage during the heating season.

Residential Income Qualified Weatherproofing


The program fell short of its savings goals. The program achieved 91% of its savings goal for the year. Staff have focused on increasing program engagement and awareness.


For example, I&M hired a residential liaison manager in March, who is managing programs for income qualified customers. Although staff reported that participation in the Income Qualified Weatherization Program remained relatively the same from previous program years, the development of relationships take time and the impact of this change may become more apparent in 2019. Additionally, program staff will conduct more workshops with various organizations around the state (e.g., food banks, schools) in order to reach more customers in 2019.

The participation process is working well for participants. Eighty-seven percent of participants reported that scheduling the home energy assessment was very or somewhat easy. Eighty-eight percent of respondents reported that they received an energy efficiency report and 95% reported that the assessor discussed savings that may result from implementing the recommendations. Most (87%) respondents thought the report was helpful.

Most participants were satisfied with the program overall. Eighty-seven participants were very or somewhat satisfied with the program overall. Additionally, 89% of participants were very or somewhat satisfied with the performance of the items installed, 90% were very or somewhat satisfied with the energy audit, and 90% were very or somewhat satisfied with I&M as their service provider.

Schools Energy Education


The goal for the number of kits distributed was exceeded. The program distributed 11,000 kits students during the program year.

Minor updates to school kits. No measures were changed in the school kits. The box design was updated and there were some changes made to the material included in the kit that is used to cross-promote the programs.

Kit content changes are planned for 2019. Staff will change the kit contents in 2019 by adding an additional light bulb and including two faucet aerators instead of one. Additionally, the kit will include a handle on it.

Ex Post Gross kWh Savings were less than ex ante savings. The difference was largely a function of the ex post application of an electric water heating adjustment factor to water heating measures.

Recommendations:

ADM does not have any recommendations for the Schools Energy Education Program.

Home Energy Management



Program ex ante savings fell short of the energy saving and demand saving targets. Program ex ante kWh and kW savings equaled 12% of the program goal. In total, 1,215 thermostats were rebated, and 1,493 customers had the Orchestrated Energy software installed.

The program is marketed through multiple channels. The marketing efforts included email campaigns, social media campaigns, direct mail advertising, and newsletters. Thirty-one percent of customers reported that they learned of the program from the program website. Additionally, 17% from an I&M newsletter, 15% learned of it through a bill insert or mailing, and 10% learned of it from a friend or relative.

The program is exploring strategies to increase enrollments including increasing incentives and reviewing marketing materials. Staff reported that they are reviewing incentive levels as one approach to increase enrollments, as well as reviewing marketing materials, messaging, and the enrollment process. Tendril indicated they are currently working with I&M on these changes and exploring ways to simplify (e.g., using less data to verify customers) the enrollment process.

A large share of participants reported awareness of other I&M program offerings. Seventy-seven percent of respondents reported being aware of other I&M program offerings. Of these customers, 5% reported that they became aware of the I&M programs through the IM Home App. The program seems to be reaching customers who are generally aware of I&M program offerings, or the participants are becoming aware of the programs during the process of learning about the program or enrolling.

Participant concerns about enrolling centered on comfort and giving up control of the thermostat settings. Twenty-six participants reported that they had concerns with allowing I&M to alter their thermostat settings during event periods and the same number stated concerns about home comfort. The website materials include information on how the software manages home comfort.

Enrollment process is working well for participants. Eighty-one percent of respondents indicated that enrolling in the program was very or somewhat easy. Most participants found the information provided about the program to be clear. One area where the information was less clear was the information on how the IM Home would adjust your thermostat settings – 19% found this to be unclear. Nevertheless, 61% reported the information on rebates available as somewhat or very clear.

Few customers opted out of the program or the demand response events. Staff reported that the program opt-out rate was around 2 to 4% and that the event opt-out rate was 2%. Tendril stated that the event opt-out rate was “phenomenal” and that they have seen other thermostat program opt-out rates in the range of 20-50%. Most survey respondents (81%) also reported that they had never or infrequently changed the thermostat settings during an event.


Participants reported some impacts on home comfort. Thirty-eight percent of customers reported that their home was somewhat less comfortable during the events and 12% reported that it became a lot less comfortable.

Sixty-three percent of the participants reported that they were satisfied with the program overall, but some dissatisfaction noted. While two-thirds of respondents were satisfied with the program overall, one-third of respondents (n = 73) reported dissatisfaction with the program overall or one or more aspects of the program. Areas of lower satisfaction included the satisfaction with the IM Home app and the information provided about the program. Additionally, several comments cited frequent changes to thermostat settings, beyond those made during peak events, as an area of dissatisfaction.

Recommendations:

Consider using messaging that emphasizes that control participants have over their thermostat. Issues of control were one of the more frequently mentioned concerns customers had about enrolling in the program. An approach to addressing this is to emphasize how the thermostat and the IM Home App gives customers control over their thermostat and energy use.

Clarify that IM Home will adjust temperatures during events and nonevents. A theme amongst dissatisfied customers was a lack of awareness and dislike of the frequent temperature changes. One approach may be to split the IM Home Saves Energy section of the website into two section on how it saves energy every day and how it saves energy during events.

Consider offering a second track that only alters thermostat settings during peak events. A second track may appeal to customers who want to limit the number of adjustments made to their thermostats.

Residential New Construction


Program operations, design, and incentive amounts remained unchanged from PY2017. There were not any significant changes to design or operational procedures to the Residential New Home Construction program in 2018. There were no changes to the incentive amounts for builders.

Program exceeded savings goals for PY2018. The Residential New Construction program ex ante savings equaled 121% of the program goal and was under budget. 2018 was described as the “largest and best year in terms of applications and participation” and “very smooth year”. While there were greater energy savings in 2017, there were additional builders in 2018, which helped to further “spread out” the program.


Increased number of all-electric homes were built compared to prior years. There was a larger share of all-electric homes in 2018 compared to other years and there are now four all-electric builders participating. There were 55 all-electric homes completed in 2018, a number higher than any other year of program operations.

Strong housing market and economy help program succeed. I&M noted that the housing market has remained strong in 2018 and has not observed any downturn. Participation in the program increased and staff believe it is due to economic conditions in the region (e.g., low unemployment and strong housing market). Staff did not see a downturn in the housing market in 2018 and speculated that 2019 will be a strong year like 2018.

New builders came on board in 2018 and additional networking opportunities. The majority (~60%) of houses are completed by five builders. I&M staff discussed the importance of ongoing networking opportunities with builders, such as golf outings or other social events sponsored by associations.

Future challenges include reduced savings from LEDs and cost of HERS raters. Future challenges may include adjusting for reduced savings from LED and that impact to the HERS ratings. CLEAResult is focusing on designing a program that does not include LEDs, while still achieving savings, for 2019. CLEAResult suggested more builders utilize heat pumps, however, expressed it is difficult for builders to implement due to costs.

No changes to communication structure, data management, or QA/QC procedures in PY2018. There have not been any significant changes to the communication structure or any concerns in 2018. There were no changes to data management for the program and staff indicated the data are kept current to manage the program. There have not been any significant changes to the QA/QC procedures in 2018 and no concerns were stated.

Program is well-received by builders. All survey respondents indicated they were very satisfied with the efficiency requirements of the I&M Residential New Construction program, the program application process, and the program overall.

Recommendations:

Consider adjusting program parameters to focus on electricity savings: While a home's HERS score is directly correlated to its overall efficiency, there is not necessarily a linear relationship between the HERS score and electricity usage for homes that use both electricity and natural gas. The HERS score is a function of natural gas usage, electricity usage, and water usage, and two homes with the same HERS score may achieve significantly different electricity savings depending on which efficient measures were implemented. ADM recommends that I&M consider more specific eligibility requirements for gas and electric homes, such as the presence of a high efficiency air conditioner, efficient lighting, or air sealing improvements, in order to improve the predictability of electricity savings as a function of HERS score for gas and electric homes that participate in the program.


1.3 Cost Effectiveness Evaluation Findings

The following cost effectiveness tests were performed for the programs: Total Resource Cost (TRC) test, Utility Cost Test (UCT), Participant Cost Test (PCT), and Ratepayer Impact Measure (RIM) test. A test score above one signifies that, from the perspective of the test, the program benefits were greater than the program costs. The test results for each program are presented in Table 1-5.

The Home Energy Products scores account for the costs and benefits associated with the program's components: Home Energy Products - Appliances and Home Energy Products - Lighting. The Home Energy Engagement scores account for the costs and benefits associated with the Home Energy Reports, Usage Alerts, and Residential Online Energy Check-up Programs. Additionally, the Residential Income Qualified Weatherproofing Program scores account for that program’s costs and benefits as well as those associated with the Low Income Home Energy Reports Program.

Table 1-5 Summary of PY2018 Benefit-Cost Ratios

Program Utility Cost Test

Total Resource Cost Test

Ratepayer Impact

Measure

Participant Cost Test

Home Appliance Recycling Program

1.06 1.39 0.40 N/A

Home Energy Products Program 1.54 1.54 0.50 3.21 Home Energy Engagement Program

1.41 1.41 0.41 N/A

Home Weatherproofing Program 0.80 0.76 0.37 3.12 Residential Income Qualified Weatherproofing Program

0.39 0.42 0.25 2.41

Schools Energy Education Program

2.04 2.04 0.51 N/A

Home Energy Management Program

0.80 0.84 0.58 2.49

Residential New Construction Program

3.65 1.95 1.04 1.47

1.4 Organization of Report

This report is divided into two volumes that provide information on the impact, process, and cost effectiveness evaluation of the Indiana Michigan Power portfolio of residential programs implemented in Indiana during the 2018 program year. Volume I is organized as follows:

Chapter 2: Home Appliance Recycling

Chapter 3: Home Energy Products - Appliances

Chapter 4: Home Energy Products - Lighting

Chapter 5: Home Energy Reports


Chapter 6: Low Income Home Energy Reports

Chapter 7: Usage Alerts

Chapter 8: Residential Online Energy Check-up

Chapter 9: Home Weatherproofing

Chapter 10: Residential Income Qualified Weatherproofing

Chapter 11: Schools Energy Education

Chapter 12: Home Energy Management

Chapter 13: Residential New Construction

Chapter 14: Cost Effectiveness Evaluation

See report Volume II for chapters presenting survey instruments and tabulated survey response information.


2 Home Appliance Recycling This chapter presents the results of both the impact and process evaluations of the 2018 Home Appliance Recycling Program that Indiana Michigan Power (I&M) offered to its Indiana residential customers during the period of January 2018 through December 2018.

The objectives of the evaluation were to:

Assess gross and net energy (kWh) savings and peak demand (kW) reductions resulting from participation in the program during the program year;

Document sources of program awareness among participants;

Characterize levels of awareness among non-participants and barriers to participation;

Determine if there are significant differences between and among participant groups;

Assess satisfaction among participating customers and trade allies;

Document and assess quality assurance and control procedures; and

Provide recommendations for program improvement as appropriate.

2.1 Program Description

The Home Appliance Recycling Program is designed to help customers reduce their energy consumption by removing old, working refrigerators and freezers from their homes for recycling. There is a limit of two refrigerators and/or freezers per household per calendar year. I&M generates energy savings because the old appliances, which are generally inefficient, are permanently removed from the system. The environment also benefits from the recycling process through safe disposal of environmentally harmful material.

The goal of the program is to reduce the number of old, inefficient refrigerators. Removing old, inefficient refrigerators and freezers prevents them from being resold or transferred to another utility customer.

I&M contracts with Appliance Recycling Centers of America (ARCA) to implement the program, which is configured as a turnkey, stand-alone energy efficiency initiative. The customer receives no-cost pick-up and removal services in addition to a $40 rebate per recycled refrigerator or freezer. To be eligible for the program, appliances to be recycled must be in working condition, plugged in and cooling at the time of pick-up. Additionally, the program limits residential customers to recycle a maximum of two units per household per calendar year.


2.2 Estimation of Ex Post Gross Savings

2.2.1 Methodology for Estimating Ex Post Gross Energy Savings

2.2.1.1 Sampling Plan

For the Appliance Recycling Program participant survey, ADM used stratified random sampling to develop a sample of program participants to be surveyed as part of the impact evaluation effort. The sample was stratified by appliance type recycled (refrigerator or freezer). The sample size for verification surveys was calculated to meet 90% confidence and ±10% precision at the program level (90/10). Quotas were set based on the proportion of each appliance type in the program population to ensure the desired confidence and precision level was achieved.

The sample size to meet 90/10 requirements is calculated based on the coefficient of variation of savings for program participants. Coefficient of Variation (CV) is defined as:

𝐶𝐶𝐶𝐶(𝑥𝑥) = 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝑆𝑆𝑆𝑆𝐷𝐷𝐷𝐷𝑆𝑆 (𝑥𝑥)

𝑀𝑀𝐷𝐷𝑆𝑆𝑆𝑆(𝑥𝑥)

Where x is the average kWh savings per participant. Without data to use as a basis for a higher value, it is typical to apply a CV of 0.5 in residential program evaluations. Using a CV of 0.5, sampling quotas were set at 300 surveys with customers who recycled refrigerators, and 100 surveys with customers who recycled freezers. The sample design achieved sample size, and corresponding precision at the 90% confidence level is shown in Table 2-1 below.

Table 2-1 Participant Survey Sample Design

Strata Population Size

Survey Quota

Completed Surveys

Precision (90% CI)

Refrigerators 3,197 300 329 4.29%

Freezers 973 100 120 7.03%

Totals 4,170 400 449 1.74%

ADM drew the sample of customers for the telephone survey effort by first organizing by appliance type recycled, then assigning a random number to each participant. The participant call list was then prioritized by the random number, making up to five call attempts per sampled customer. Ultimately, the telephone survey effort resulted in 449 completed surveys (329 from participants who recycled refrigerators and 120 from participants who recycled freezers).

2.2.1.2 Review of Documentation

I&M’s implementation contractor developed and maintained a participant tracking database that includes a full list of all customers, the make and model numbers of the refrigerators and freezers that were recycled, and a number of other important appliance and household characteristics. The first step in the evaluation effort was to review this program tracking system and other relevant program materials.


ADM reviewed the tracking system data on reported recycled units to determine that all reported units were eligible for the program, and that no duplicate or erroneous entries are present. Additionally, the tracking system was reviewed to ensure that the proper data fields required to support this evaluation as well as future evaluations were included. Finally, the program tracking data and the associated summary data provided in the I&M DSM EE Program Scorecard were reviewed for consistency.

2.2.1.3 Data Collection

Data used for the gross impact evaluation included:

Program tracking data from the main tracking database;

I&M DSM EE Program Scorecard summary data;

Participant survey data; and

Data from relevant secondary sources.

ADM conducted surveys with a sample of program participants. The focus of these surveys was to verify that customers listed in the program tracking database did indeed participate and that the number of appliances claimed to be recycled is accurate. Survey respondents were also asked a series of questions to verify the working condition of their recycled appliances; it is a program requirement that collected units be in working condition at the time of pick-up.

2.2.1.4 Procedures for Estimating Measure-Level Gross Energy Savings

ADM conducted the gross energy savings analysis in accordance with the Department of Energy (DOE) Uniform Methods Project (UMP) Refrigerator Recycling Protocol.1 The UMP is a DOE initiative aimed at developing a consistent framework and set of protocols for determining the energy savings from specific energy efficiency measures and programs. The project represents a refinement of the body of knowledge supporting energy efficiency EM&V activities, and each protocol was written by technical experts within the field and peer-reviewed by industry experts.

The protocols presented in the UMP provide a straightforward method for evaluating gross and net energy savings for common residential and commercial measures offered in ratepayer-funded initiatives in the United States.2 The UMP specifies a regression model developed by The Cadmus Group that uses data from a metering study of 472 refrigerators across five utilities to relate the unit energy consumption (UEC) of refrigerators – metered in situ operating conditions – to various characteristics of the appliance.

1 Keeling, J.; Bruchs, D. (2017). Chapter 7: Refrigerator Recycling Evaluation Protocol. The Uniform Methods Project: Methods for Determining Energy-Efficiency Savings for Specific Measures. Golden, CO; National Renewable Energy Laboratory. NREL/SR-7A40-68563. http://www.nrel.gov/docs/fy17osti/68563.pdf. 2 Office of Energy Efficiency & Renewable Energy, https://energy.gov/oe/downloads/uniform-methods-project-methods-determining-energy-efficiency-savings-specific-measures, accessed: 3 January 2017.


In accordance with the UMP Refrigerator Recycling Protocol, ADM’s statistical model for determining annual kWh considered the following independent variables:

Unit age;

Unit capacity (cubic feet);

Dummy indicator for configurations (top freezer, side-by-side, etc.);

Primary/Secondary usage designation;

Location in conditioned/unconditioned space; and

Weather (cooling degree days, heating degree days).

ADM used the UMP regression model developed by Cadmus to estimate the UEC for refrigerators recycled through the Home Appliance Recycling Program. Specifically, the average characteristics of recycled refrigerators were multiplied by the associated regression coefficients from the Cadmus model and summed to produce an estimated average in situ UEC for refrigerators.

It is important to note that the Cadmus model only considers refrigerators. Accordingly, ADM used a refrigerator-to-freezer ratio factor to determine the average UEC for freezers recycled through the program. This refrigerator-to-freezer factor methodology is similar to that used by the NMR Group, Inc. in their recent evaluation of the Massachusetts Appliance Turn-in program.3 Using relevant secondary sources, ADM concluded that freezers on average use 15% less energy annually than refrigerators. This implies a refrigerator-to-freezer factor of 0.85. The analysis supporting this refrigerator-to-freezer factor is detailed in the previously mentioned Massachusetts Appliance Turn-In program evaluation performed by NMR Group, Inc.4

Additionally, ADM relied upon participant survey data to develop average partial use factors for both refrigerators and freezers. The partial use factor is designed to account for the fact that not all refrigerators and freezers are plugged in year-round. Secondary appliances are more likely to be unplugged for a portion of the year than primary appliances, and since the program targets secondary appliances, the partial use factor is an important consideration when developing gross savings estimates.

Finally, gross energy savings were measured in accordance with the algorithms presented in the UMP Refrigerator Recycling Program Evaluation Protocol. The estimated average UECs for refrigerators and freezers were extrapolated to the population of program participating units to obtain a program level estimate of gross kWh energy savings resulting from refrigerator and freezer recycling. Specifically, the following algorithm was used:

𝐺𝐺𝐺𝐺𝐺𝐺𝑆𝑆𝑆𝑆_𝑘𝑘𝑘𝑘ℎ = 𝑁𝑁 × 𝐸𝐸𝐸𝐸𝐸𝐸𝑆𝑆𝐸𝐸𝐸𝐸𝑁𝑁𝐺𝐺_𝑈𝑈𝐸𝐸𝐶𝐶 × 𝑃𝑃𝑃𝑃𝐺𝐺𝐸𝐸_𝑈𝑈𝑆𝑆𝐸𝐸

3 NMR Group, Inc. Massachusetts Appliance Turn-in Program Impact Evaluation, Final. June 15th, 2011. Available at: http://ma-eeac.org/wordpress/wp-content/uploads/Impact-Evaluation-Final-Report.pdf 4 Ibid.


Where:

GROSS_kWh = Annual electricity savings of refrigerators/freezers measured in kilowatt-hours (kWh)

N= The number of refrigerators/freezers recycled through the program

EXISTING_UEC = The average annual unit energy consumption of participating refrigerators/freezers

PART_USE = The portion of the year the average refrigerator/freezer would likely have operated if not recycled through the program

Gross peak demand savings were calculated based on the critical peak demand definition provided by I&M. Measure specific normalized 8,760 hour load shapes were used to identify the average demand during this on-peak period. These load shapes assign a portion of estimated gross kWh energy savings to each hour of the year. After identifying the total kWh savings that fall into the defined on-peak hours, dividing by the total number of hours in the peak period results in the average gross peak demand reduction. The specific appliance load shapes that were used were originally developed as part of the End-Use Load and Consumer Assessment Program (ELCAP) – a major end-use data collection program undertaken by the Bonneville Power Administration.5

2.2.2 Results of Ex Post Gross Impact Evaluation

ADM estimated ex post gross electric savings and peak demand reductions through detailed analysis of program tracking data and participant survey data. The estimated gross impacts resulting from the 2018 Appliance Recycling Program are summarized in Table 2-2 and Table 2-3. Gross realization rates and lifetime energy savings (kWh) are summarized in Table 2-4. The gross energy savings (kWh) and gross peak demand reduction (kW) realization rate is 102%.

Table 2-2 Ex Post Gross kWh Savings

Appliance Type

Claimed Appliances Recycled

Ex Ante Gross kWh

Savings

Gross Audited kWh

Savings

Gross Verified kWh

Savings

Ex Post Gross kWh

Savings

Gross Realization

Rate

Refrigerators 3,197 3,076,473 3,076,473 3,010,074 3,084,367 100% Freezers 973 718,269 718,269 718,269 782,560 109% Total 4,170 3,794,742 3,794,742 3,728,343 3,866,926 102%

5 Pratt RG, CC Conner, EE Richman, KG Ritland, WF Sandusky, and ME Taylor. 1989. Description of Electric Energy Use in Single-Family Residences in the Pacific Northwest. (End-Use Load and Consumer Assessment Program [ELCAP]). DOE/BP-13795-21, prepared for Bonneville Power Administration by Pacific Northwest Laboratory, Richland, Washington.


Table 2-3 Ex Post Gross kW Savings

Appliance Type

Claimed Appliances Recycled

Ex Ante Gross kW Savings

Gross Audited kW

Savings

Gross Verified kW

Savings


Gross Realization

Rate

Refrigerators 3,197 361.26 361.26 353.46 362.85 100% Freezers 973 86.6 86.6 86.6 93.73 108% Total 4,170 447.86 447.86 440.06 456.58 102%

Table 2-4 Gross Realization Summary

Savings Variable Ex Ante Ex Post Realization Rate

Annual Energy Savings (kWh) 3,794,742 3,866,926 102% Annual Demand Reduction (kW) 447.86 456.58 102% Lifetime Energy Savings (kWh) NA 30,935,414 NA

The calculations leading to these results are detailed in the sub-sections to follow.

2.2.2.1 Database Review

As a first step toward estimating program level kWh and kW impacts, ADM reviewed program tracking data provided by ARCA for accuracy. ADM confirmed that the tracking database included all necessary information to conduct the impact analysis, including appliance and household characteristics. The review did not identify any duplicate or obviously erroneous entries. However, there were two customers who recycled three appliances during 2018 (it is a program requirement that no customer recycle more than two appliances in a given year).

The tracking database was compared to summary data presented in the I&M DSM EE Program Scorecard. The number of units recycled through the program in 2018 was consistent across the two documents. Overall, the tracking database was well organized and complete.

2.2.2.2 Verification of Units Recycled

To verify that the number of units claimed in the program tracking database was accurate, ADM administered a telephone survey with a sample of program participants.

All of the respondents who completed the participant survey verified that they had in fact participated in the program during 2018. However, for participating appliances to accrue energy savings by being taken out of service, the units must be in working condition at the time of pick-up. Seven respondents who recycled a refrigerator reported that their units were not in working condition at the time they were collected for recycle. All respondents who recycled freezers indicated that the units were in working condition at the time of pickup. Based on these results, the verification rates shown in Table 2-5 were determined for each appliance type.


Table 2-5 Verification Rates by Appliance Type

Refrigerator (n=322)

Freezer (n=117)

97.8% 100.0%

Based on these verification rates, Table 2-6 reports the numbers of refrigerators and freezers recycled through the program during 2018 that were verified as being in working condition when recycled and therefore were program-eligible.

Table 2-6 Recycled Appliances Verified to be in Working Condition

Unit Type Quantity

Reported as Recycled

Verification Rate

Quantity of Recycled Units

Verified as program Eligible

Refrigerator 3,197 97.8% 3,128 Freezer 973 100.0% 973

2.2.2.3 Per-Unit Gross Annual kWh Savings Estimates

Per-unit gross annual kWh savings were calculated as described in 2.2.1.4. The details and results of these calculations are presented in this section.

For refrigerators, Unit Energy Consumption (UEC) estimates were derived using the DOE monitoring procedure-based regression model developed by Cadmus in the development of the Uniform Methods Project Refrigerator Recycling Evaluation Protocol. The model specification and estimated coefficients of the Cadmus model are shown in Table 2-7.

Table 2-7 Uniform Methods Project UEC Regression Details6

(Dependent Variable – UMP Estimated In Situ UEC)

Independent Variables Coefficient

Intercept 0.582

Appliance Age (years) 0.027

Dummy: Manufactured Pre-1990 1.055

Appliance Size (square feet) 0.067

Dummy: Single-Door Configuration -1.977

Dummy: Side-by-Side Configuration 1.071

6 Source: Cadmus et al. (2013). The Uniform Methods Project: Methods for Determining Energy Efficiency Savings for Specific Measures. April 2013.


Independent Variables Coefficient

Dummy: Primary Usage Type 0.6054

Interaction: Uncooled Space x CDDs 0.02

Interaction: Uncooled Space x HDDs -0.045

The program tracking database included information regarding configuration, size, and age for all refrigerators collected during 2018. Of these 3,197 refrigerators, 25.2% were side-by-side models and 3.1% were single-door models; the average size was 19.9 cubic feet and the average age was 21.2 years old. Finally, 24.9% of refrigerators were manufactured before 1990. Table 2-8 shows the relevant refrigerator characteristics used to estimate UEC.

Table 2-8 2018 Average Refrigerator Characteristics

Appliance Characteristics Refrigerators

Population Size 3,197 Appliance Age (years) 21.16 Manufacture Pre-1990 24.9% Average Size (Cubic Feet) 19.90 Single-Door Configuration 3.1% Side-by Side Configuration 25.2% Primary Usage 45.9% Interaction: Uncooled x CDD 1.27 Interaction: Uncooled x HDD 9.06

The refrigerator characteristics shown above were used in conjunction with the model coefficients in Table 2-7 to calculate annual energy consumption estimates for program participating refrigerators. The refrigerator-to-freezer factor of 0.85 was applied to develop annual energy consumption estimates for freezers. These calculations are shown below:

Refrigerator UEC (kWh)

�. 582 + .027 ∗ 21.16 (𝑃𝑃𝐴𝐴𝐷𝐷) + 1.055 ∗ .249 (𝑃𝑃𝑆𝑆𝐷𝐷 − 1990) + .067 ∗ 19.90 (𝑆𝑆𝐷𝐷𝑆𝑆𝐷𝐷) − 1.977∗ .031 (𝑆𝑆𝐷𝐷𝑆𝑆𝐴𝐴𝑆𝑆𝐷𝐷 𝐷𝐷𝐷𝐷𝐷𝐷𝑆𝑆) + 1.071 ∗ .252 (𝑆𝑆𝐷𝐷𝑆𝑆𝐷𝐷 𝑏𝑏𝑏𝑏 𝑆𝑆𝐷𝐷𝑆𝑆𝐷𝐷) + .6054 ∗ .459 (𝑈𝑈𝑈𝑈𝑆𝑆𝐴𝐴𝐷𝐷) + .02 ∗ 1.27 (𝐶𝐶𝐷𝐷𝐷𝐷 𝐸𝐸𝑆𝑆𝑆𝑆𝐷𝐷𝑆𝑆𝑆𝑆𝐼𝐼𝑆𝑆𝐷𝐷𝐷𝐷𝑆𝑆) − .045 ∗ 9.06 (𝐻𝐻𝐷𝐷𝐷𝐷 𝐸𝐸𝑆𝑆𝑆𝑆𝐷𝐷𝑆𝑆𝑆𝑆𝐼𝐼𝑆𝑆𝐷𝐷𝐷𝐷𝑆𝑆)� ∗ 365.25= 1,042 𝑘𝑘𝑘𝑘ℎ

Freezer UEC (kWh)

1,042 ∗ 0.85 (𝑆𝑆𝐷𝐷𝑟𝑟𝑆𝑆𝐷𝐷𝐴𝐴𝐷𝐷𝑆𝑆𝑆𝑆𝑆𝑆𝐷𝐷𝑆𝑆 − 𝑆𝑆𝐷𝐷 − 𝑟𝑟𝑆𝑆𝐷𝐷𝐷𝐷𝑆𝑆𝐷𝐷𝑆𝑆 𝑟𝑟𝑆𝑆𝐼𝐼𝑆𝑆𝐷𝐷𝑆𝑆) = 886 𝑘𝑘𝑘𝑘ℎ


A final adjustment was made to account for the fact that not all refrigerators and freezers are plugged in year-round. Secondary appliances are more likely to be unplugged for a portion of the year than primary appliances, and since there was a large presence of secondary appliances in the program, the partial use adjustment is an important consideration when developing gross savings estimates. The partial use adjustment is based on participant survey responses regarding participants’ usage of the recycled units, and assigns different “use factors” based on three categories into which recycled appliances fall:

Some units that were recycled were not being used at all before being sent for recycling. The use factor for such units therefore would be zero. That is, these units were not being used and therefore had no baseline energy usage.

Other units were being used, but for only part of the year. For these units, the use factor is calculated by dividing the number of months in the past year that the unit had been in use by the number of months in the year. Based on data collected through the survey of participants, the average number of months in use for a refrigerator that was being partly used was 4.98 months, implying a use factor of 0.41 (i.e., 4.98/12). For freezers in this category, the use factor was calculated to be 0.54, reflecting an average of 6.5 months in use for freezers being partly used.

Units which are constantly in use have a use factor of one (1).

The overall use factor and the corresponding overall Unit Energy Savings (UES) are calculated as a weighted average across the three categories, where the weights are determined by the percentages of units falling into the three categories. Table 2-9 shows the calculation of the overall UES for refrigerators and freezers when partial use is considered.

Table 2-9 Unit Energy Savings Adjusted for Partial Use

Operating Status of Unit

Percentage of Recycled Units in

Category Use Factor

Calculation of UES to Adjust for Part Use

Refrigerators Not running 0.62% 0 0 Running part time 8.10% 0.41 432 Running all time 91.28% 1.00 1,042

Weighted Average UES for Refrigerators 986 Freezers

Not running 6.84% 0 0 Running part time 5.13% 0.54 480 Running all time 88.03% 1.00 886

Weighted Average UES for Freezers 804

Based on the findings detailed in this section, the ex post gross per-unit annual kWh savings for refrigerators recycled through the program is estimated to be 986 kWh; the ex post gross per-unit annual kWh savings for freezers recycled through the program is estimated to be 804 kWh.


2.2.2.4 Per-Unit Peak kW Reduction Estimates

Appliance load shapes for refrigerators and freezers were used to estimate the average kW reduction occurring during I&M’s defined on-peak period. These load shapes were normalized versions of load shapes originally developed as part of the End-Use Load and Consumer Assessment program (ELCAP).7 The average daily load profile for each appliance type recycled through the program is shown in Figure 2-1.

Figure 2-1 Average Daily Load Profile

Using these normalized ELCAP load shapes, the Evaluation Team determined that approximately 2.3% of the annual gross kWh savings attributable to a recycled refrigerator occurs during the on-peak period. This is equivalent to 23.0 kWh; dividing by the number of on-peak hours (198) results in an average on-peak demand reduction of 0.12 kW per recycled refrigerator.

Similarly, it was determined that approximately 2.4% of a freezer’s energy consumption occurs during on-peak hours (19.1 kWh). Average on-peak demand reduction is thus 0.10 kW per recycled freezer.

2.3 Estimation of Ex Post Net Savings

2.3.1 Methodology for Estimating Ex Post Net Energy Savings

This section explains ADM’s net savings estimation methodology which is based on the UMP protocol. The two effects discussed in this section are free ridership and secondary market impacts.

7 Ibid.


Responses from the participant survey where used to estimate each effect. The UMP protocol used to recommend estimating a third effect, induced replacement, but no longer includes this recommendation due to the difficulty of estimating the affect and the small impact on savings overall.

The UMP protocol does not recommend estimation of participant spillover for appliance recycling programs because there are limited opportunities for “like” spillover (i.e., spillover resulting from measures similar to those incentivized through the program) and the program does not provide energy assessments or education to encourage adoption of additional measures. As such, ADM did not estimate participant spillover for the Home Appliance Recycling Program.

Net savings for recycled appliances are calculated relative to UMP gross savings using the formula below.

𝑁𝑁𝐷𝐷𝑆𝑆 𝑆𝑆𝑆𝑆𝐷𝐷𝐷𝐷𝑆𝑆𝐴𝐴𝑈𝑈 = 𝐺𝐺𝑆𝑆𝐷𝐷𝑈𝑈𝑈𝑈 𝑆𝑆𝑆𝑆𝐷𝐷𝐷𝐷𝑆𝑆𝐴𝐴𝑈𝑈 − 𝐹𝐹𝑆𝑆𝐷𝐷𝐷𝐷𝑆𝑆𝐷𝐷𝑆𝑆𝐷𝐷𝑆𝑆𝑈𝑈ℎ𝐷𝐷𝑖𝑖 − 𝑆𝑆𝐷𝐷𝐼𝐼𝐷𝐷𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑏𝑏 𝑀𝑀𝑆𝑆𝑆𝑆𝑘𝑘𝐷𝐷𝑆𝑆 𝐸𝐸𝐼𝐼𝑖𝑖𝑆𝑆𝐼𝐼𝑆𝑆𝑈𝑈

Where:

Gross Savings = The evaluated in situ UEC for the average recycled unit, adjusted for part use (UMP definition of gross savings);

Free ridership = Program savings from units that would have been destroyed even in the absence of the program;

Secondary Market Impacts = Program savings that would have occurred in the absence of the program based on the estimated/assumed counterfactual actions of appliance acquirers.

2.3.1.1 Methodology for Estimating Free Ridership

Free ridership occurs when an appliance recycled through the program would have been taken off the grid even in the absence of the program. The first step of the free ridership analysis was to ask participants if they had considered discarding the program appliance before learning about the program. If the participant indicated no previous consideration of unit disposal, they are categorized as non-free-riders and removed from the subsequent free ridership analysis.

Next, the remaining participants (i.e., those who had previously considered discarding the program appliance) were asked a series of questions to determine the distribution of program appliances that would have been kept within participant households versus those that would have been discarded. If one considers the counterfactual scenario where there is no program intervention, there are essentially three outcomes for participating appliances:

The appliance would have been kept in use by the participant household.8

8 Note that units kept by participant households but not used are accounted for in the estimation of part-use factors and therefore discounted from gross savings.


The appliance would have been discarded in such a way that it was transferred to another customer for continued use.

The appliance would have been discarded in such a way that it would be taken out of service.

Of the three outcomes, participants who responded that their appliance would have been discarded and taken out of service is indicative of free ridership. This is because the recycled units would have been removed from the grid even without program intervention.

2.3.1.2 Methodology for Estimating Secondary Market Impacts

Secondary market impacts refer to the effect the program has on would-be acquirers of program participating units. In the event that a program unit would have been transferred to another customer (sold, gifted, donated), the question then becomes what other appliance acquisition decisions are made by the would-be acquirer of the program unit now that it is decommissioned and unavailable. The would-be acquirer could:

Not purchase/acquire another unit.

Purchase/acquire a different non-program used appliance.

Purchase a new appliance instead.

Ultimately, the true market level outcome in the absence of the program is difficult to assess. As a result, this evaluation will take a midpoint approach, as recommended by the UMP protocol. That is, 50% of would-be acquirers of program avoided transfers are assumed to find an alternate unit. The next question of interest is whether the alternative units acquired would be used (similar to those recycled by the program) or new. Again, this market distribution is difficult to estimate with any certainty. This evaluation will take the UMP recommendation and assumes that 50% of the alternative units would be used and 50% would be new, standard efficiency units. Energy consumption for a standard efficiency new refrigerator is assumed to be 490 kWh based on sales-weighted appliance data from the Association of Home Appliance Manufactures (AHAM).9 Similarly, energy consumption for a standard new freezer is assumed to be 344 kWh.

2.3.1.3 Complete Net-to-Gross Calculation

Figure 2-2 summarizes the complete net-to-gross calculation that was used in this evaluation. Note that this diagram depicts net savings as calculated using the UMP protocol.

9 AHAM Energy Efficiency and Consumption Trends 2015


Figure 2-2 Net Savings Calculation Summary Diagram

2.3.2 Results of Ex Post Net Impact Evaluation

ADM estimated net-to-gross ratios (NTGRs) for both refrigerators and freezers based on results from the participant survey and applying the methodology described in Section 2.3.1. Applying the estimated NTGR of 47% for refrigerators and the estimated NTGR of 53% for freezers to the gross savings presented in Section 2.2.2 results in the net savings detailed in Table 2-10 and Table 2-11 below.

Table 2-10 Ex Post Net kWh Savings

Appliance Type

Ex Ante Gross kWh

Savings

Gross Audited

kWh Savings

Gross Verified

kWh Savings

Ex Post Gross kWh

Savings

Gross Realization

Rate

Ex Post Net kWh Savings

Net-to-

Gross Ratio

Ex Post Net

Lifetime kWh

Savings

Refrigerators 3,076,473 3,076,473 3,010,074 3,084,367 100% 1,459,772 47% 11,678,175 Freezers 718,269 718,269 718,269 782,560 109% 414,195 53% 3,313,557 Total 3,794,742 3,794,742 3,728,343 3,866,926 102% 1,873,967 48% 14,991,732

Table 2-11 Ex Post Net kW Savings

Appliance Type


Gross Audited kW

Savings

Gross Verified

kW Savings


Gross Realization

Rate


Net-to-Gross Ratio

Refrigerators 361.26 361.26 353.46 362.85 100% 171.73 47% Freezers 86.6 86.6 86.60 93.73 108% 49.61 53% Total 447.86 447.86 440.06 456.58 102% 221.34 48%


The calculations leading to these estimated NTGRs are detailed in the sub-sections to follow.

2.3.2.1 Calculation of NTGRs for Recycled Appliances

ADM used the formula shown below to estimate net savings for recycled refrigerators and freezers. Note that this definition considers gross savings under the UMP definition. Each component of the net savings calculation is described in Section 3.1.4 of this report. Spillover and induced replacement effects were not considered as part of the net savings analysis for this evaluation.

𝑁𝑁𝐷𝐷𝑆𝑆 𝑆𝑆𝑆𝑆𝐷𝐷𝐷𝐷𝑆𝑆𝐴𝐴𝑈𝑈 = 𝐺𝐺𝑆𝑆𝐷𝐷𝑈𝑈𝑈𝑈 𝑆𝑆𝑆𝑆𝐷𝐷𝐷𝐷𝑆𝑆𝐴𝐴𝑈𝑈 − 𝐹𝐹𝑆𝑆𝐷𝐷𝐷𝐷𝑆𝑆𝐷𝐷𝑆𝑆𝐷𝐷𝑆𝑆𝑈𝑈ℎ𝐷𝐷𝑖𝑖 − 𝑆𝑆𝐷𝐷𝐼𝐼𝐷𝐷𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑏𝑏 𝑀𝑀𝑆𝑆𝑆𝑆𝑘𝑘𝐷𝐷𝑆𝑆 𝐸𝐸𝐼𝐼𝑖𝑖𝑆𝑆𝐼𝐼𝑆𝑆𝑈𝑈

Where:

Gross Savings = The evaluated in situ UEC for the average recycled unit, adjusted for part use (UMP definition of gross savings);

Free-ridership = Program savings from units that would have been destroyed even in the absence of the program;

Secondary Market Impacts = Program Savings that would have occurred in the absence of the program based on the estimated/assumed counterfactual actions of appliance acquirers.

Net savings are essentially calculated using a decision tree. The decision tree is populated with estimated percentages of appliance disposition in the absence of the program based on responses to the participant survey. In other words, participants’ actions concerning discarded equipment are used to estimate savings values under all possible scenarios. The weighted average of savings under these scenarios is then used to calculate the net savings attributable to the program.

Participant survey respondents were first asked if they had considered discarding the program appliance before learning about the program. Respondent answers to this question are shown in Table 2-12.


Table 2-12 Prior Consideration of Disposal

Had you already considered disposing of the [refrigerator, freezer] before you heard about [I&M]’s

appliance recycling program?

Measure Response Percent of Respondents (n=324 (ref), 119 (frz))

Refrigerator Yes 73% No 23%

Don’t know 4%

Freezer Yes 80% No 19%

Don’t know 1%

Respondents who indicated they had not considered disposal before learning about the program were considered non-free-riders. That is, for these respondents it was assumed they would have kept the appliance in use absent the program, since they hadn’t considered disposal before learning about the program. Respondents who indicated they had considered disposal or “didn’t know” if they had considered disposal were asked additional questions to determine whether the appliances they recycled were indicative of free-ridership.

Table 2-13 shows appliance disposition based on participant survey responses. Table 2-14 shows the same calculation for freezers.

Table 2-13 Refrigerator Discard/Keep Distribution

Discard/Keep

Proportion of

Participant Sample

(n = 297)

Discard Scenario Proportion of Discards

Overall Proportion

Discard 70% Transfer 40% 28% Destroy 60% 42%

Keep 30% 30%

Table 2-14 Freezer Discard/Keep Distribution

Discard/Keep

Proportion of

Participant Sample

(n = 115)

Discard Scenario Proportion of Discards

Overall Proportion

Discard 74% Transfer 56% 42% Destroy 44% 32%

Keep 26% 26%

Secondary market impacts account for program effects on would-be acquirers of program units (since they are no longer available to acquire program units). Only units that would have been


transferred absent the program are considered in the secondary market impact analysis. As detailed in Section 2.3.1.2, a midpoint approach is taken in this evaluation, based on the recommendation of the UMP protocols. That is, 50% of would-be acquirers of program avoided transfers are assumed to find an alternate unit. Of those who are assumed to find an alternative unit, 50% are assumed to find a similar used unit, while 50% are assumed to purchase a new unit.

ADM determined net savings as UMP gross savings less free-ridership and secondary market impacts. Figure 2-4 depicts the complete net-to-gross ratio calculation for refrigerators. Figure 2-3 shows the same calculation for freezers. As the figures show, ADM’s estimated NTGRs are 0.47 for recycled refrigerators and 0.53 for recycled freezers.

Figure 2-3 NTGR Calculation – Refrigerators

Figure 2-4 NTGR Calculation – Freezers


2.4 Process Evaluation

ADM completed a limited process evaluation of the PY2018 program. The limited process evaluation focused on notable program changes and developments, findings from a review of the program database, and findings from a survey of participating customers.

Data were collected from program staff responsible for administering and implementing the program and from a sample of participant surveys.

Data was collected from multiple data sources to support the process evaluation of the program. The data collection activities and sample sizes are presented in Table 2-15.

Table 2-15 Summary of Data Collection Activities for the Home Appliance Recycling Program

Data Collection Activity Mode of Data Collection Targeted Sample Size Participant survey Online 449 Staff interviews Telephone in-depth interviews 2

2.4.1 Process Evaluation Findings

2.4.1.1 Summary of Program Participation Data

ADM reviewed the program tracking data and summarized the program activity in this section. As part of the review, ADM identified one customer who received a rebate for three units during the program year despite the program limit of recycling three units. ADM recommends that the implementation contractor review processes to prevent customers from receiving rebates for more than two units.

As shown in Table 2-16, refrigerators accounted for the majority of units recycled through the program.

Table 2-16 Summary of Orders

Equipment Type Number of Orders

Percent of Orders with Two Units

Number of Units

Refrigerator 2,949 3% 3,051 Freezer 805 3% 827 Freezer, Refrigerator 146 100% 292

The configuration of recycled units is displayed in Table 2-17.


Table 2-17 Summary of Unit Configuration

Configuration Number of Units

Top Freezer 2,142 Side-by-Side 807 Upright 663 Chest 310 Bottom Freezer 148 Single Door 100

Figure 2-5 displays the age range of units recycled. As shown, the ages varied substantially and a significant share (approximately one-quarter) were manufactured more than 28 years ago (i.e., pre-1990 units) prior to when changes in efficiency standards lead to increased efficiency of units.

Figure 2-5 Summary of Age of Recycled Units

Figure 2-6 shows that the number of units recycled during the year was very consistent during the year.


Figure 2-6 Weekly and Cumulative Units Recycled

2.4.1.2 Program Operations Perspective

This section summarizes the core findings of interviews conducted with I&M program staff and ARCA for the purposes of learning about any changes to the program structure, identifying program objectives, and assessing the extent to which there are future opportunities for program improvement. ADM evaluators interviewed the I&M senior energy efficiency consumer program coordinator and two account managers from ARCA to gain insights in the Appliance Recycling program for 2018.

The key findings from the interview are summarized below.

The online scheduling process was revised. The Appliance Recycling program revised the online scheduling process in Spring 2018. The program had previously had less than 8% participation online (scheduling appointments). Program staff analyzed the online process to identify why customers are not using the online scheduling system. They identified the section that required users to enter their account number as the place where most people leave the system. Staff believed this was the main barrier to scheduling online (not having an account number) and it was changed to allow customers to schedule without this information. Now the program participant enters three points of identification (house/street number, zip code, and last name), which is validated using a database of account numbers to verify eligibility.

The program exceeded its goal. The Appliance Recycling program experienced the highest level of participation since inception and exceeded goals for 2018. The 2018 goal for Indiana was 3,214 units and the program realized 4,170 units.

In addition to reaching savings goals, staff reported that the program met all key performance indicators for transportation and the call center.


Marketing approach was unchanged in 2018. There were two email campaigns in the Fall and Spring in the Indiana territory. The emails focused on specific zip codes, where ARCA could open appointments for those areas to ensure that enrolling customers have a range of appointment time options. The primary marketing activities included: bill inserts, energy efficiency newsletters, cross promotion, promoted on the home energy reports, speaking at various events (e.g., home shows), the Electric Ideas website, social media campaigns, and Google ads. I&M provides ARCA with the marketing plan at the beginning of the year, which staff find helpful for planning. During pick-ups, ARCA staff cross promotes by providing a brochure with information about the energy efficiency programs.

For 2019, I&M is planning an “Ugliest Fridge” contest to market and promote the program around Earth Day.

Data management and QA/QC procedures were not changed in 2018. There were no major changes to data management for the program, but I&M did note that ARCA now uploads data weekly to EECP. I&M and ARCA both indicated the data is kept current to manage the program. There have not been any significant changes to the QA/QC procedures in 2018 and no concerns were stated.

Income-qualified pilot program planned for 2019. The program will pilot an income- qualified program in 2019. The program will include a refrigerator replacement along with the recycling component for income-qualified customers. I&M will provide a list of customers to ARCA for a refrigerator replacement. ARCA will buy the refrigerator from the income-qualified customer and bill I&M for the replacement. The pilot was launched, with the first refrigerator drop-offs/pick-ups occurring in January 2019.

Energy efficiency conservation kits may be a potential program enhancement. ARCA staff has recommended that they provide the kits to customers while they are in the home doing a pick-up, in addition to the cross promotion that is currently occurring.

2.4.1.3 Participant Survey Findings

ADM completed a survey of program participants to collect information regarding their experience with the Home Appliance Recycling Program.

The survey was administered online, and participants were contacted by email. In total, 449 participants completed the survey.

Sources of Program Awareness

Thirty-five percent of survey respondents first became aware of the Residential Appliance Recycling Program through bill inserts or an I&M mailing. Figure 2-7 summarizes the initial source of program awareness for responding customers, with most learning of the program by the bill inserts, a brochure from I&M, and the I&M website.

Respondents who selected the response option of “Other” most commonly reported that they learned of the program through an email from I&M (13 respondents).


Figure 2-7 Initial Source of Program Awareness

Seventy-six percent of respondents reported that they had learned of the Appliance Recycling Program through more than one source. The most commonly cited secondary sources of program awareness were bill inserts (18% of respondents) and the I&M website (14% of respondents).

Fifty-two percent of respondents who recalled the sign-up process indicated they signed up for the Appliance Recycling Program via the phone and 48% signed up online (n = 402). Among those who signed up online (n = 192):

96% of respondents indicated it was easy to find the sign-up screen on the I&M website; 97% stated the website was able to answer all their questions about the Residential

Appliance Recycling Program; 97% received a confirmation that their online sign up was successful; and 83% of those who received a confirmation (n = 186) indicated the confirmation included a

scheduled pickup date.

Most (63%) who signed up online did not contact a program representative after signing up online and among those who did, 58% contacted to confirm their appointment date and time, and 35% contacted to cancel or reschedule the appointment (n = 48).

Among those who signed up via the phone (n = 210):

60% were aware they could have signed up online; 90% found the representative to be polite and courteous; 86% did not have to call more than once, with 25 respondents indicating they had to call

more than once.

Experience with Appliance Pickup


Overall, 96% of program participants were able to schedule a day and time that was convenient for them. Eighty-three percent of respondents stated they interacted with the person who collected their old refrigerator and 14% stated they did not (n = 446). Table 2-18 summarizes the interactions that customers had with the pick-up crew, with 99% of survey participants finding them to be courteous and professional.

Ninety-three percent of respondents indicated that their appliance was plugged in at the time of pickup, and 95% of respondents reported that the pick-up crew verified that the appliance was operational upon pick-up.

Additionally, 40% of respondents reported that they observed the pick-up crew disable their appliance, such as by cutting the cord, at the time of pick-up. It may be the case that some customers did not observe the disabling of the device or realize that the actions taken by the crew were disabling the device.

Table 2-18 Interactions with Individuals Who Collected Appliances

Response Percentage of Respondents n

People who collected were courteous and professional 99% 367 Appliance plugged in at time of pick-up 93% 363 People who collected check to see if it still works 95% 328 People who collected disabled appliance at time of pick-up 40% 298

Customer Satisfaction

The survey collected information regarding customers’ satisfaction with various aspects of the Appliance Recycling Program. Participants rated their satisfaction on a scale from 1 (very dissatisfied) to 5 (very satisfied).

Generally, participants were satisfied with the Residential Appliance Recycling Program and I&M as their electric service provider (see Figure 2-8 and Table 2-19). Most respondents reported being very satisfied with each listed element of their program experience, with the overall portion of satisfied participants ranging from 87% to 95% across program elements. 10

On average, it took 3.4 weeks from the time of appliance(s) pickup to the time customers received their rebate (n = 265). Eighty-seven percent of respondents reported being satisfied with the time it took to receive their rebate check.

The two percent of respondents who indicated dissatisfaction with their overall program experience cited appointment scheduling issues, issues with the pick-up process, issues related to communicating with I&M, and issues receiving rebate checks (n = 8).

Ninety-one percent of respondents reported that they are satisfied with I&M as their electric service provider.

10 Respondents who selected ratings of 4 or 5 on the satisfaction scale are considered to be satisfied.


Figure 2-8 Customer Satisfaction with Residential Appliance Recycling Program

Table 2-19 Satisfaction Level with I&M as an Electric Service Provider

Response Percentage of Respondents

(n=435) Very Satisfied 62% Somewhat Satisfied 29% Neither Satisfied or Dissatisfied 7% Somewhat Dissatisfied 1% Very Dissatisfied 7%

Table 2-20 summarizes the suggestions provided by customers on ways to improve the Residential Appliance Recycling Program. Respondents provided commentary related to various aspects of the program experience, including recommendations for the incentive level and eligibility, communication from I&M, improvements to the pick-up and scheduling process, and other areas.


Table 2-20 Customers Suggestions to Improve the Appliance Recycling Program

Type of Comment n Examples

Incentive changes 12

Expanding it to include other appliances- ACs, heat pumps, water heaters, etc.

The refrigerator was worth more than $40. That would be a start to improving the program.

Recycling dishwasher or any other appliance. More choices.

General communication improvements

9

Explain why the appliance must be in working order.

Be sure all telephone service people are trained with the same information.

Better coordination between recycling company and I&M.

Pick-up scheduling

improvements 8

Offer a shorter time window in the initial scheduling process.

Sooner pick up dates. It took over a week to get it picked up.

Offer off-time pick up hours. weekends or evenings.

Website improvements 5

Get the website working for everyone.

Fix the web signup.

Rebate processing

improvements 4 Issue checks faster, and in larger amounts.

Program awareness

improvements 4

Just keep making more people aware of it, I know I have been telling all my friends!

Pick-up process improvements 3

Tell the drivers not to be so quick to leave without picking them up.

Tell pick up people not to request to use the customer's bathroom.

Other 3 Keep better records.

Make it easier to get the form.

When asked to provide additional information regarding their experience with the program, many participants responded in the form of positive commentary regarding their program experiences. Examples of this type of commentary included:

“Everything was done professionally and on time. Workers were very courteous.”

“[It was] convenient, easy and everyone I dealt with knew the program inside and out. Very good customer service.”

“Everyone was courteous and prompt, and accomplished the removal quickly.”

“They picked up when they said they would and the refund was received in a timely manner.”

Yearly Trends

ADM compared 2018 survey results to those in 2017 to observe any differences from year to year. Key findings are presented in this section. The increases or decreases are presented as the


difference in the percentage of customers providing a response in 2018 as compared to 2017. (i.e., an increase or decrease in percentage points from 2017 to 2018).

In both years the most common responses for how participants learned of the Appliance Recycling program included bill inserts, I&M website, word-of-mouth, and an I&M brochure. There was an nine percentage-point increase in those who reported learning about the program from an I&M brochure and a three percentage-point decrease in learning of the program through word-of-mouth (see Figure 2-9).

Figure 2-9 2017 and 2018 Program Awareness Comparison

There was a seven percentage-point increase among survey participants who reported they scheduled their appliance pick-up online (see Figure 2-10).

Figure 2-10 2017 and 2018 Scheduling Comparison


2.5 Conclusions and Recommendations

2.5.1 Conclusions

Below is a summary of the key findings of the evaluation.

The program exceeded its goal. The Appliance Recycling program experienced the highest level of participation since inception and exceeded goals for 2018. The program exceeded savings targets in Indiana and collected 4,170 units during PY2018.

Diverse approaches were used to market the program. Primary marketing approaches include bill inserts, monthly e-newsletter, email campaigns, social media, internet searches, I&M’s website, and cross-promotion through other I&M programs. Customers most frequently reported learning of the program from a bill insert (24%) followed by brochures from I&M (20%) and the program website (13%).

Revised online scheduling process. The Appliance Recycling program revised the online scheduling process in Spring 2018 based on analysis of what point in completing the enrollment form customers stop using the online form. Staff reported that the revision lead to an increase in online enrollments. The participant survey responses indicated that 48% of participants enrolled online in PY2018 as compared to 41% in PY2017.

Few customers reported difficulty with the online and telephone sign-up process. Ninety-five percent or more of customers who enrolled in the program online stated that it was easy to find the sign-up screen, that the website answered all their questions about participating, and that they received a confirmation that their sign-up was successful. Additionally, all respondents who signed by phone reported that the representative was polite and courteous.

Pick up process working as designed. Nearly all (99%) of customers reported that the pick-up crew was courteous and professional. Additionally, 94% reported that the appliance was plugged in at the time of pick-up and 95% reported that the crew checked that the appliance was working.

Survey responses indicate that about 48% of customers who recalled the sign-up process enrolled in the program online. The majority of these respondents did not encounter any issues with the sign-up process but approximately two percent reported that they had encountered issues with the program website, such as pages not loading, or submission of the application not working properly. Open-ended commentary from these respondents also suggests that some customers found the information on the program website to be unclear. Sixty percent of respondents that signed up by phone were aware that they could sign up online. These customers generally did not indicate any problems during the phone sign up though 12% indicated that they had to call more than one time.

Program satisfaction is high. Ninety-five percent of people were satisfied with the program overall. Ninety-six percent of all respondents reported that they were able to


schedule a convenient appliance pick-up time and 99% of respondents indicated that the people picking up the appliances were courteous and professional.

2.5.2 Recommendations

The following recommendations are based on the evaluation findings.

ADM recommends that ARCA continues to ensure compliance with program rules, particularly with regard to the number of units recycled per customer per year.


3 Home Energy Products - Appliances This chapter presents the results of both the impact and process evaluations of the 2018 Home Energy Products – Appliances Program that Indiana Michigan Power (I&M) offered to its residential customers during the period of January 2018 through December 2018.






Assess satisfaction among participating customers and trade allies; and



The Home Energy Products – Appliances program seeks to increase demand for energy-efficient products through cash-back rebates designed to cover a portion of the incremental cost of upgrading to efficient technologies. In addition, the program aims to educate customers about the energy saving and non-energy benefits associated with efficient HVAC and self-install products that reduce energy consumption.

The objectives of the program include lowering electricity consumption in the residential market sector through the purchase and installation of eligible energy efficiency measures and attributing electric energy savings to those purchases that receive a rebate through the program and encouraging equipment vendors, retailers, and contractors to actively market eligible energy efficient technologies to residential customers.

The Home Energy Products – Appliances program provides cash-back rebates to residential customers who upgrade to more efficient HVAC products such as air conditioners and heat pumps (central split systems or mini split ductless units), energy efficient appliances such as ENERGY STAR® dehumidifiers, or various other measures such as heat pump water heaters and pool pumps. The list of eligible measures incentivized includes:

Ductless Heat Pumps;

Air Source Heat Pumps;

Central Air Conditioning (CAC) Systems;

Wi-Fi Smart Thermostats;

Heat Pump Water Heaters;


Dehumidifiers;

Ceiling Fans;

Electronically Commutated Furnace (ECM) Fan Motors; and

Variable Speed Pool Pumps.


The following section presents the methodology that was used for estimating the gross energy and demand impacts resulting from the Home Energy Products – Appliances Program in 2018.


The M&V approach for the Home Energy Products – Appliances Program focused on determining the following:

Number of appliances rebated and sold through the program;

Average annual energy savings per purchased appliance; and

Average kW reduction per purchased appliance.


ADM used simple random sampling to develop a sample of program participants to be surveyed as part of the impact evaluation effort. The sample size for these participant surveys was calculated to meet 90% confidence and 10% precision (90/10). The sample size to meet 90/10 requirements was calculated based on the coefficient of variation of savings for program participants. The coefficient of variation (CV) is defined as:



Where x is the average kWh energy savings per participant. Without data to use as a basis for a higher value, it is typical to apply a CV of .5 in residential program evaluations. The resulting sample size was estimated at:

𝑆𝑆0 = �1.645 ∗ 𝐶𝐶𝐶𝐶

𝐺𝐺𝑃𝑃�

2

Where,

1.645 = Z Score for 90% confidence interval in a normal distribution

CV = Coefficient of Variation

RP = Required Precision, 10% in this evaluation


With 10% required precision (RP), this called for a minimum sample of 68 participants. ADM exceeded this sampling target with a sample of 102 participants.


As a first step, ADM reviewed data tracking systems associated with the program to ensure that the data provided sufficient information to identify unique customers for surveying and to calculate energy and demand impacts in accordance with the 2015 Indiana Technical Reference Manual (TRM) Version 2.211. ADM further reviewed the program data to verify that the fields required for performing the evaluation were tracked and populated (i.e., the data was not missing) and that the values were reasonable. ADM took several steps in verifying the number of appliances rebated, which consisted of the following:

Validating program tracking data by checking for duplicate or erroneous entries; and

Conducting verification surveys with a sample of program participants to verify that customers listed in the program tracking database did indeed participate, that the number of appliances claimed to be rebated was accurate, and that appliances were rebated according to the process I&M had in place.




Program applications and supporting documentation;


Non-participant survey data.

ADM conducted surveys with a sample of 102 program participants. For the purposes of the gross impact evaluation, the surveys were used to develop measure-level in-service rates.


ADM’s approach for the calculation of gross energy impacts depended largely on the types of measures installed. Deemed values and algorithms from the Indiana TRM were used to calculate verified gross energy and demand impacts for most program measures. ADM referenced the Illinois Statewide Technical Reference Manual Version 6.0 in the calculation of ex post energy savings and peak demand reduction for rebated Ductless Heat Pumps, as the Indiana TRM did not characterize the program measure.

11 The Illinois Statewide Technical Reference Manual Version 6.0 was referenced in the calculation of ex post energy impacts for Ductless Heat Pumps.


The following sections describe the specific algorithms and inputs used to calculate savings for each program measure.

Central Air Conditioner: From the Indiana TRM, the Central Air Conditioning (Time of Sale) section was used to calculate energy impacts for the installation of high efficiency air conditioning units. The following equation was used to calculate custom annual kWh energy savings dependent on participant information:

∆𝑘𝑘𝑘𝑘ℎ =� 1

𝑆𝑆𝐸𝐸𝐸𝐸𝐺𝐺𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏− 1

𝑆𝑆𝐸𝐸𝐸𝐸𝐺𝐺𝑏𝑏𝑏𝑏� × 𝐸𝐸𝐹𝐹𝐸𝐸𝐻𝐻𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 × 𝐵𝐵𝑆𝑆𝑢𝑢𝐻𝐻

1,000

Where:

SEERbase = SEER of baseline air conditioning equipment, 13

SEERee = SEER of installed air conditioning equipment, actual

EFLHcool = Equivalent Full Load Cooling Hours, dependent upon location; TRM table was be referenced

BtuH = Size of equipment in Btuh, actual; 28,994 was assumed otherwise

Following this, ADM calculated the peak demand reduction using the following Indiana TRM defined equation:

∆𝑘𝑘𝑘𝑘 =� 1

𝐸𝐸𝐸𝐸𝐺𝐺𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏− 1

𝐸𝐸𝐸𝐸𝐺𝐺𝑏𝑏𝑏𝑏� × 𝐵𝐵𝑆𝑆𝑢𝑢𝐻𝐻 × 𝐶𝐶𝐹𝐹

1,000

Where:

EERbase = Efficiency in EER of baseline Air Conditioning equipment, 11

EERee = Efficiency in EER of installed Air Conditioning equipment, actual; if actual EER was not available, ADM approximated efficient EER using the efficient SEER of the installed air conditioning unit and an engineering formula (EER = 1.12 * SEER - 0.02 * SEER2)

BtuH = Size of equipment in Btuh, actual; 28,994 was assumed otherwise

CF = Summer Peak Coincidence Factor for measure, 0.88

ENERGY STAR Ceiling Fan: From the Indiana TRM, the Ceiling Fan with ENERGY STAR Light Fixture (Time of Sale) section was used to calculate energy impacts for the installation of new ceiling fans. The following equation, which results in deemed energy savings of 108 kWh, was used to calculate the annual kWh energy savings:


∆𝑘𝑘𝑘𝑘ℎ = �%𝑐𝑐𝑐𝑐𝑙𝑙 × (𝐸𝐸𝐷𝐷𝐿𝐿𝑘𝑘𝑘𝑘𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏 − 𝐸𝐸𝐷𝐷𝐿𝐿𝑘𝑘𝑘𝑘𝑏𝑏𝑏𝑏) + %𝑚𝑚𝑏𝑏𝑚𝑚 × (𝑀𝑀𝐷𝐷𝑆𝑆𝑘𝑘𝑘𝑘𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏 − 𝑀𝑀𝐷𝐷𝑆𝑆𝑘𝑘𝑘𝑘𝑏𝑏𝑏𝑏) + %ℎ𝑖𝑖𝑖𝑖ℎ

× (𝐻𝐻𝐷𝐷𝐴𝐴ℎ𝑘𝑘𝑘𝑘𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏 − 𝐻𝐻𝐷𝐷𝐴𝐴ℎ𝑘𝑘𝑘𝑘𝑏𝑏𝑏𝑏)� × 𝐻𝐻𝐷𝐷𝑢𝑢𝑆𝑆𝑈𝑈𝑓𝑓𝑏𝑏𝑓𝑓

+ �(𝐸𝐸𝑆𝑆𝐼𝐼𝑘𝑘𝑘𝑘 − 𝐶𝐶𝐹𝐹𝐸𝐸𝑘𝑘𝑘𝑘) × 𝐻𝐻𝐷𝐷𝑢𝑢𝑆𝑆𝑈𝑈𝑐𝑐𝑖𝑖𝑖𝑖ℎ𝑡𝑡 × (1 + 𝑘𝑘𝐻𝐻𝐹𝐹𝑏𝑏)�

Where:

%low = Percent time on low speed, 40%

%med = Percent time on medium speed, 40%

%high = Percent time on high speed, 20%

LowkWbase = Low speed baseline ceiling fan demand, 0.0152 kW

LowkWee = Low speed ENERGY STAR ceiling fan demand, 0.0117 kW

MedkWbase = Medium speed baseline ceiling fan demand, 0.0348 kW

MedkWee = Medium speed ENERGY STAR ceiling fan demand, 0.0314 kW

HighkWbase = High speed baseline ceiling fan demand, 0.0725 kW

HighkWee = High speed ENERGY STAR ceiling fan demand, 0.0715 kW

Hoursfan = Typical fan operating hours, 1,022

InckW = Incandescent bulb kW, 0.129 kW

CFLkW = CFL bulb kW, 0.042 kW

Hourslight = Typical lighting operating hours, 1,277.5

WHFE = Waste Heat Factor for Energy to account for HVAC interactions with efficient lighting; Appendix B was referenced

Following this, ADM calculated the peak demand reduction (deemed at 0.013 kW) using the following Indiana TRM defined equation:

∆𝑘𝑘𝑘𝑘 = ��%𝑐𝑐𝑐𝑐𝑙𝑙 × (𝐸𝐸𝐷𝐷𝐿𝐿𝑘𝑘𝑘𝑘𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏 − 𝐸𝐸𝐷𝐷𝐿𝐿𝑘𝑘𝑘𝑘𝑏𝑏𝑏𝑏) + %𝑚𝑚𝑏𝑏𝑚𝑚 × (𝑀𝑀𝐷𝐷𝑆𝑆𝑘𝑘𝑘𝑘𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏 − 𝑀𝑀𝐷𝐷𝑆𝑆𝑘𝑘𝑘𝑘𝑏𝑏𝑏𝑏) + %ℎ𝑖𝑖𝑖𝑖ℎ

× (𝐻𝐻𝐷𝐷𝐴𝐴ℎ𝑘𝑘𝑘𝑘𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏 − 𝐻𝐻𝐷𝐷𝐴𝐴ℎ𝑘𝑘𝑘𝑘𝑏𝑏𝑏𝑏)� + [(𝐸𝐸𝑆𝑆𝐼𝐼𝑘𝑘𝑘𝑘 − 𝐶𝐶𝐹𝐹𝐸𝐸𝑘𝑘𝑘𝑘) × (1 + 𝑘𝑘𝐻𝐻𝐹𝐹𝑚𝑚)]� × 𝐶𝐶𝐹𝐹

Where:

%low = Percent time on low speed, 40%

%med = Percent time on medium speed, 40%

%high = Percent time on high speed, 20%


LowkWbase = Low speed baseline ceiling fan demand, 0.0152 kW

LowkWee = Low speed ENERGY STAR ceiling fan demand, 0.0117 kW

MedkWbase = Medium speed baseline ceiling fan demand, 0.0348 kW

MedkWee = Medium speed ENERGY STAR ceiling fan demand, 0.0314 kW

HighkWbase = High speed baseline ceiling fan demand, 0.0725 kW

HighkWee = High speed ENERGY STAR ceiling fan demand, 0.0715 kW

InckW = Incandescent bulb kW, 0.129 kW

CFLkW = CFL bulb kW, 0.042 kW

WHFD = Waste Heat Factor for Demand to account for HVAC interactions with efficient lighting; Appendix B was referenced


ENERGY STAR Dehumidifier: From the Indiana TRM, the ENERGY STAR Dehumidifier (Time of Sale) section was used to calculate energy impacts for the installation of ENERGY STAR dehumidifiers. The following equation was used to calculate the annual kWh energy savings:

∆𝑘𝑘𝑘𝑘ℎ =𝐶𝐶 × 0.473 × 𝐻𝐻𝐷𝐷𝑢𝑢𝑆𝑆𝑈𝑈

24 × 𝐸𝐸/𝑘𝑘𝑘𝑘ℎ

Where:

C = Average capacity in pints per day, actual

Hours = Run hours per year, 1,620

L/kWh = Liters of water consumed per kWh, based on capacity; TRM tables were referenced


∆𝑘𝑘𝑘𝑘 =∆𝑘𝑘𝑘𝑘ℎ × 𝐶𝐶𝐹𝐹

𝐻𝐻𝐷𝐷𝑢𝑢𝑆𝑆𝑈𝑈

Where:

ΔkWh = Annual energy savings for ENERGY STAR dehumidifier

Hours = Run hours per year, 1,620



Air Source Heat Pump: From the Indiana TRM, the Central Air Source Heat Pumps (Time of Sale) section was used to calculate energy impacts for the installation of high efficiency central air source heat pumps (ASHP). The following equation was used to calculate custom annual kWh energy savings dependent on participant information:



𝑆𝑆𝐸𝐸𝐸𝐸𝐺𝐺𝑏𝑏𝑏𝑏� × 𝐸𝐸𝐹𝐹𝐸𝐸𝐻𝐻𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 × 𝐵𝐵𝑆𝑆𝑢𝑢𝐻𝐻𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐

1,000

+� 1

𝐻𝐻𝑆𝑆𝑃𝑃𝐹𝐹𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏− 1

𝐻𝐻𝑆𝑆𝑃𝑃𝐹𝐹𝑏𝑏𝑏𝑏� × 𝐸𝐸𝐹𝐹𝐸𝐸𝐻𝐻ℎ𝑏𝑏𝑏𝑏𝑡𝑡 × 𝐵𝐵𝑆𝑆𝑢𝑢𝐻𝐻ℎ𝑏𝑏𝑏𝑏𝑡𝑡

1,000

Where:

SEERbase = Seasonal average efficiency in SEER of baseline heat pump, 13

SEERee = Seasonal average efficiency in SEER of installed heat pump, actual

EFLHcool = Equivalent Full Load Cooling Hours, dependent upon location; TRM table was referenced

BtuHcool = Cooling capacity of equipment in Btuh, actual

BtuHheat = Heating capacity of equipment in Btuh, actual

HSPFbase = Heating Seasonal Performance Factor of baseline heat pump, 7.7

HSPFee = Heating Seasonal Performance Factor of installed heat pump, actual

EFLHheat = Equivalent Full Load Heating Hours, dependent upon location; TRM table was referenced




𝐸𝐸𝐸𝐸𝐺𝐺𝑏𝑏𝑏𝑏� × 𝐵𝐵𝑆𝑆𝑢𝑢𝐻𝐻𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 × 𝐶𝐶𝐹𝐹

1,000

Where:

EERbase = Efficiency in EER of baseline heat pump, 11

EERee = Efficiency in EER of installed heat pump, actual; if actual EER was not available, ADM approximated efficient EER using the efficient SEER of the installed heat pump and an engineering formula (EER = 1.12 * SEER - 0.02 * SEER2)

BtuHcool = Cooling capacity of equipment in Btuh, actual



Ductless Heat Pumps: From the Illinois TRM, the Ductless Heat Pumps section was used to calculate energy impacts for the installation of high efficiency ductless heat pumps (DHP). Indiana-specific equivalent full load cooling and heating hours were obtained from the Indiana TRM. The following equation was used to calculate custom annual kWh energy savings dependent on participant information:



𝑆𝑆𝐸𝐸𝐸𝐸𝐺𝐺𝑏𝑏𝑏𝑏� × 𝐸𝐸𝐹𝐹𝐸𝐸𝐻𝐻𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 × 𝐶𝐶𝑆𝑆𝑖𝑖𝑆𝑆𝐼𝐼𝐷𝐷𝑆𝑆𝑏𝑏𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐

1,000

+𝐸𝐸𝑆𝑆𝐷𝐷𝐼𝐼ℎ𝐷𝐷𝑆𝑆𝑆𝑆 × � 1

𝐻𝐻𝑆𝑆𝑃𝑃𝐹𝐹𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏− 1

𝐻𝐻𝑆𝑆𝑃𝑃𝐹𝐹𝑏𝑏𝑏𝑏� × 𝐸𝐸𝐹𝐹𝐸𝐸𝐻𝐻ℎ𝑏𝑏𝑏𝑏𝑡𝑡 × 𝐶𝐶𝑆𝑆𝑖𝑖𝑆𝑆𝐼𝐼𝐷𝐷𝑆𝑆𝑏𝑏ℎ𝑏𝑏𝑏𝑏𝑡𝑡

1,000

Where:

SEERbase = Seasonal average efficiency in SEER of baseline heat pump, dependent on existing cooling system; 14 for ASHP and 13 for CAC or no previous central cooling unit

SEERee = Seasonal average efficiency in SEER of installed heat pump, actual

EFLHcool = Equivalent Full Load Cooling Hours, dependent upon location; Indiana TRM table was referenced

Capacitycool = Cooling capacity of equipment in Btuh, actual

Elecheat = 1 if building is electrically heated, 0 otherwise

HSPFbase = Heating Seasonal Performance Factor of baseline heat pump, 8.2 or 3.41 (for heat pumps displacing electric resistance heating)

HSPFee = Heating Seasonal Performance Factor of installed heat pump, actual

EFLHheat = Equivalent Full Load Heating Hours, dependent upon location; Indiana TRM table was referenced

Capacityheat = Heating capacity of equipment in Btuh, actual

Following this, ADM calculated the peak demand reduction using the following Illinois TRM defined equation:



𝐸𝐸𝐸𝐸𝐺𝐺𝑏𝑏𝑏𝑏� × 𝐶𝐶𝑆𝑆𝑖𝑖𝑆𝑆𝐼𝐼𝐷𝐷𝑆𝑆𝑏𝑏𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 × 𝐶𝐶𝐹𝐹

1,000

Where:


EERbase = Efficiency in EER of baseline heat pump, dependent on existing cooling system; 11.8 for ASHP and 11 for CAC or no previous central cooling unit

EERee = Efficiency in EER of installed heat pump, actual; if actual EER was not available, ADM approximated efficient EER using the efficient SEER of the installed heat pump and an engineering formula (EER = 1.12 * SEER - 0.02 * SEER2)

Capacitycool = Cooling capacity of equipment in Btuh, actual


ECM Furnace Fan: From the Indiana TRM, the Residential Electronically Commutated Motors section was used to calculate energy impacts for the installation of ECMs on HVAC fans. The TRM provides deemed energy impacts of 415 kWh and 0 kW for electronically commutated motors, therefore no additional calculations will be needed for this measure.

Heat Pump Water Heaters: From the Indiana TRM, the Heat Pump Water Heaters (Time of Sale) section was used to calculate energy impacts for the installation of a heat pump water heater replacing an existing electric water heater. The following equation was used to calculate custom annual kWh energy savings dependent on participant information:

∆𝑘𝑘𝑘𝑘ℎ = 𝑘𝑘𝑘𝑘ℎ𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏 × �𝐶𝐶𝐺𝐺𝑃𝑃𝑓𝑓𝑏𝑏𝑙𝑙 − 𝐶𝐶𝐺𝐺𝑃𝑃𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏

𝐶𝐶𝐺𝐺𝑃𝑃𝑓𝑓𝑏𝑏𝑙𝑙� + 𝑘𝑘𝑘𝑘ℎ𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑖𝑖𝑓𝑓𝑖𝑖 − 𝑘𝑘𝑘𝑘ℎℎ𝑏𝑏𝑏𝑏𝑡𝑡𝑖𝑖𝑓𝑓𝑖𝑖

Where:

kWhbase = Average electric DHW consumption, 3,460

COPnew = Coefficient of Performance of Heat Pump Water Heater, 2.0

COPbase = Coefficient of Performance of standard electric water heater, 0.904

kWhcooling = Cooling savings from the conversion of heat in home to water heat, 180

kWhheating = Heating cost from the conversion of heat in home to water heat, dependent on home heating fuel; 1,577 for electric resistance, 779 for heat pump (COP of 2.0), and 0 for fossil fuel


∆𝑘𝑘𝑘𝑘 =∆𝑘𝑘𝑘𝑘ℎ × 𝐶𝐶𝐹𝐹

𝐻𝐻𝐷𝐷𝑢𝑢𝑆𝑆𝑈𝑈

Where:

ΔkWh = Annual energy savings for heat pump water heater


Hours = Equivalent full load hours of hot water heater, 2,533


Wi-Fi Programmable Thermostats: From the Indiana TRM, the Wi-Fi Connected Smart Thermostats (Time of Sale, Direct Install) section was used to calculate energy impacts for the installation of programmable thermostats. The following set of equations was used to calculate the annual kWh energy savings:

∆𝑘𝑘𝑘𝑘ℎ = ∆𝑘𝑘𝑘𝑘ℎ𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑖𝑖𝑓𝑓𝑖𝑖 + ∆𝑘𝑘𝑘𝑘ℎℎ𝑏𝑏𝑏𝑏𝑡𝑡𝑖𝑖𝑓𝑓𝑖𝑖

∆𝑘𝑘𝑘𝑘ℎ𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑖𝑖𝑓𝑓𝑖𝑖 =1

𝑆𝑆𝐸𝐸𝐸𝐸𝐺𝐺 × 𝐸𝐸𝐹𝐹𝐸𝐸𝐻𝐻𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 × 𝐵𝐵𝑆𝑆𝑢𝑢𝐻𝐻𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 × 𝐸𝐸𝑆𝑆𝐹𝐹𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐

1,000

∆𝑘𝑘𝑘𝑘ℎℎ𝑏𝑏𝑏𝑏𝑡𝑡𝑖𝑖𝑓𝑓𝑖𝑖 =𝐸𝐸𝐹𝐹𝐸𝐸𝐻𝐻ℎ𝑏𝑏𝑏𝑏𝑡𝑡 × 𝐵𝐵𝑆𝑆𝑢𝑢𝐻𝐻ℎ𝑏𝑏𝑏𝑏𝑡𝑡 × 𝐸𝐸𝑆𝑆𝐹𝐹ℎ𝑏𝑏𝑏𝑏𝑡𝑡

𝜂𝜂𝐻𝐻𝐷𝐷𝑆𝑆𝑆𝑆 × 3,412

Where:

SEER = Seasonal average efficiency in SEER of central air conditioning equipment, actual; TRM table was referenced otherwise


BtuHcool = Cooling system capacity in Btuh, actual; 28,994 was assumed otherwise

ESFcool = Cooling energy savings fraction (see below)


BtuHheat = Heating system capacity in Btuh, actual

ESFheat = Heating energy savings fraction (see below)

ηHeat = Efficiency in COP of Heating equipment, actual; TRM table was referenced otherwise

The Indiana TRM attributes no peak demand reduction to the installation of Wi-Fi Smart Thermostats.

The Wi-Fi Connected Smart Thermostats (Time of Sale, Direct Install) section from the Indiana TRM provides deemed energy savings factors of 0.139 for cooling and 0.125 for heating, assuming a baseline condition of a standard, non-programmable thermostat. The Programmable Thermostat (Time of Sale, Direct Install) section from the Indiana TRM provides deemed energy savings factors of 0.09 for cooling and 0.068 for heating, also assuming a baseline condition of a standard,


non-programmable thermostat. ADM assigned a cooling energy savings factor of 0.049 (= 0.139 - 0.09) and a heating energy savings factor of 0.057 (= 0.125 - 0.068) to Wi-Fi Programmable Thermostats replacing Programmable Thermostats.

Because participants’ existing thermostat type was not tracked in I&M’s main program tracking system for most Wi-Fi thermostat participants, ADM referenced participant survey data to approximate the appropriate energy savings factors from installing a Wi-Fi Smart Thermostat. Among the survey respondents that identified their existing thermostat type, 47% indicated a manually-controlled thermostat, 53% indicated a programmable thermostat, and the remainder indicated a different Wi-Fi Smart Thermostat. Table 3-1 below presents the cooling and heating energy savings factors ADM applied to all eligible Wi-Fi Smart Thermostats.

Table 3-1 Estimated Energy Savings Factor – Wi-Fi Smart Thermostat

Wi-Fi Smart Thermostat Baseline Type ESF_cool ESF_heat Survey

Weights Estimated ESF_cool

Estimated ESF_heat

Manual Thermostat 0.139 0.125 47% 0.091 0.089 Programmable Thermostat 0.049 0.057 53%

Wi-Fi Smart Thermostat 0 0 2%

Variable Speed Pool Pump: From the Indiana TRM, the Residential Two Speed / Variable Speed Pool Pumps (Time of Sale) section was used to calculate energy impacts for the installation of variable speed pool pumps replacing standard single speed pumps. The following equation was used to calculate custom annual kWh energy savings dependent on participant information:

∆𝑘𝑘𝑘𝑘ℎ =𝐻𝐻𝑃𝑃 × 𝐸𝐸𝐹𝐹 × 0.746 × 𝐻𝐻𝑆𝑆𝑈𝑈/𝐷𝐷𝑆𝑆𝑏𝑏 × 𝐷𝐷𝑆𝑆𝑏𝑏𝑈𝑈/𝑌𝑌𝑆𝑆 × 𝐸𝐸𝑆𝑆𝐹𝐹

𝜂𝜂𝑃𝑃𝑢𝑢𝐼𝐼𝑖𝑖

Where:

HP = Horsepower of pump motor, 1.5

LF = Load factor of pump motor, 0.66

𝜂𝜂Pump = Efficiency of pump motor, 0.325

Hrs/Day = Assumed hours of pump operation per day, 6

Days/Yr = Assumed number of days pool is in use per year, 100

ESF = Energy savings factor for variable speed pool pump, 0.86


∆𝑘𝑘𝑘𝑘 =𝐻𝐻𝑃𝑃 × 𝐸𝐸𝐹𝐹 × 0.746 × 𝐶𝐶𝐹𝐹 × 𝐷𝐷𝑆𝑆𝐹𝐹

𝜂𝜂𝑃𝑃𝑢𝑢𝐼𝐼𝑖𝑖


Where:

HP = Horsepower of pump motor, 1.5

LF = Load factor of pump motor, 0.66

𝜂𝜂Pump = Efficiency of pump motor, 0.325

DSF = Demand savings factor for variable speed pool pump, 0.91


3.2.1.5 In-service Rates (ISR)

Ex post annual gross kWh savings and peak kW reductions resulting from the Home Energy Products – Appliances Program were adjusted by applying the estimated measure-level installation rates of program-incented products to the calculated measure-level gross energy impacts.

To estimate the measure-level ISRs for the program, ADM obtained a sample of 102 program participants. Of the 102 participants that completed the survey, 25 received a rebate for and answered questions regarding dehumidifiers. One of the 25 dehumidifiers survey respondents stated that the dehumidifier for which they received a rebate was not installed and working at the time of the survey.

This assessment results in a verification rate of 96% for dehumidifiers. All other respondents who completed the participant survey verified measures to be installed and working. Table 3-2 below displays the quantity of program-incented products reported, their associated in-service rates, and the resulting quantity of program-incented products verified.

Table 3-2 Home Energy Products Verified

Measure Type Quantity of Measures Reported

In-service Rate

Quantity of Measures Verified

Air Conditioner 236 100% 236 Air Source Heat Pump 75 100% 75 Ceiling Fan 3 100% 3 Dehumidifier 195 96% 187 Ductless Heat Pump 89 100% 89 ECM Retrofit 1,010 100% 1,010 Heat Pump Water Heater 33 100% 33 Variable Speed Pool Pump 39 100% 39 Wi-Fi Smart Thermostat 457 100%* 456 Total 2,137 100% 2,128 * This table presents rounded values. The actual in-service rate for thermostats is less than 100% because 456 of 457 units were verified.


3.2.2 Results of Ex Post Gross Savings Estimation

ADM estimated ex post gross electric savings and peak demand reductions for the Home Energy Products – Appliances Program through detailed analysis of program tracking data and participant survey data. This section presents the results of the gross savings calculation activities.

3.2.2.1 Ex Post Gross kWh Savings

The estimated annual gross energy savings resulting from the 2018 Home Energy Products – Appliances Program are summarized in Table 3-3 below. The gross kWh realization rate for the program is 106%.

Table 3-3 Program-level Annual Gross kWh Savings

Ex Ante Gross kWh Savings

Gross Audited kWh Savings


Ex Post Gross kWh Savings

Gross Realization Rate

1,233,192 1,233,106 1,231,521 1,302,679 106%

Table 3-4 below shows the estimated measure-level annual gross energy savings resulting from the program. The measure level realization rates were close to 100% for most measures.

Table 3-4 Measure-level Annual Gross kWh Savings

Measure Type Ex Ante Gross kWh Savings




Gross Realization

Rate

Air Conditioner 47,571 47,571 47,571 51,449 108% Air Source Heat Pump 99,073 99,073 99,073 95,193 96% Ceiling Fan 324 324 324 324 100% Dehumidifier 39,627 39,627 38,042 36,217 91% Ductless Heat Pump 443,566 443,566 443,566 492,115 111% ECM Retrofit 419,150 419,150 419,150 419,150 100% Heat Pump Water Heater 45,483 45,483 45,483 51,895 114% Variable Speed Pool Pump 43,134 43,134 43,134 51,632 120% Wi-Fi Smart Thermostat 95,264 95,178 95,178 104,704 110% Total 1,233,192 1,233,106 1,231,521 1,302,679 106%

3.2.2.2 Ex Post Gross kW Reductions

The estimated gross demand reduction resulting from the 2018 Home Energy Products – Appliances Program is summarized in Table 3-5 below. The gross kW realization rate for the program is 81%.


Table 3-5 Program-level Gross kW Reduction


Gross Audited kW Savings

Gross Verified kW Savings



312.50 312.50 312.14 252.05 81%

Table 3-6 below shows the estimated measure-level gross demand reduction resulting from the program.

Table 3-6 Measure-level Gross kW Reduction

Measure Type Ex Ante Gross kW Savings




Gross Realization

Rate

Air Conditioner 94.51 94.51 94.51 102.28 108% Air Source Heat Pump 33.67 33.67 33.67 25.65 76% Ceiling Fan 0.06 0.06 0.06 0.04 65% Dehumidifier 9.04 9.04 8.68 8.24 91% Ductless Heat Pump 32.28 32.28 32.28 33.17 103% ECM Retrofit 66.66 66.66 66.66 0.00 0% Heat Pump Water Heater 6.63 6.63 6.63 7.09 107% Variable Speed Pool Pump 69.65 69.65 69.65 75.58 109% Wi-Fi Smart Thermostat 0 0 0 0 - Total 312.50 312.50 312.14 252.05 81%


The following section presents the methodology that was used for estimating the net energy impacts resulting from the Home Energy Products – Appliances Program in 2018.

3.3.1 Survey Data Collection

A survey of program participants was administered to collect data for use in estimating participant free ridership and spillover. Responses to the free ridership questions were collected through an online survey.


The net savings analysis is used to determine what part of the gross energy savings achieved by program participants can be attributed to the effects of the program. The net savings attributable to program participants are the gross savings less free ridership, plus spillover. ADM estimated free ridership and participant spillover through a survey of program participants. Non-participant spillover was estimated through a survey of non-participants.



Survey respondents were asked a series of questions designed to elicit information regarding the following factors:

Financial ability and plans and intentions to implement the efficiency measure;

The program influence on the decision to implement the efficiency measure;

The program’s influence on the timing of the measure installation.

The calculation of a free ridership score was based on the responses to questions about the participants’ prior plans and intentions, program influence on measure selection, and program influence on timing of measure implementation.

3.3.2.1.1 Financial Ability and Plans and Intentions

Two indicator variables were developed based on responses to the survey questions on plans and intentions. The first corresponds to financial ability. Respondents were considered to have not been financially able to install the efficient equipment if they answer “no” to either of the two questions below:

FR1: Would you have been able to afford to purchase the efficient [MEASURE] if the rebate was not available from the program?

FR2: [IF YES] Just to confirm, if the rebate was not available through the program, would you still have paid the additional cost to purchase an [EFF_MEASURE] instead of a [STAND_MEASURE]?

The second indicator variable is related to whether the customer had plans to implement the efficiency measure. Respondents were considered to have had plans if they answer “yes” to the following two questions:

FR3: Were you planning to purchase an [EFF_MEASURE] before you learned of I&M’s rebate program?

FR4: [IF YES] Just to be clear, did you have plans to specifically purchase an [EFF_MEASURE] as opposed to a [STAND_MEASURE]?

Respondents who were found to not have plans or the financial ability to implement the measures were deemed to not be free riders.

3.3.2.1.2 Program Influence on Decision to Implement Energy Efficiency Measure

Participants were asked about the direct influence of the program on their decision to implement the energy efficiency measures. Specifically, participants were asked:

FR5: Now we would like to know how likely you would have been to install the [MEASURE] if the program was not available. Using a scale where 0 is “not at all likely” and 10 is “very likely”, how likely is it that you would have installed the same


[EFF_MEASURE] if you had not received the financial or information assistance through the program?

A program influence score was developed based on this response in the following manner:

Program Influence = FR5 / 10

An aspect of program influence is the indirect influence of trade allies on customer decisions. This indirect influence occurs when the program influences the recommendations made by trade allies, and the trade allies recommendations were influenced by the program. To account for this type of influence, customers that installed efficient HVAC equipment were asked to report on the extent to which their decisions were influenced by the recommendations provided by the contractor that they worked with. Specifically, respondents were asked the following questions:

FR7: Did the contractor that you worked with provide you with information, marketing material or a recommendation to purchase or install the [EFF_MEASURE]?

FR8: Using a scale where 0 is “not at all influential” and 10 is “very influential, how influential was the information, marketing material, or recommendation provide by this contractor in your decision to purchase the [EFF_MEASURE]?

Participants’ program influence scores were substituted with a trade ally influence score if they provide a response of 7 or greater to FR8, indicating that the indirect influence on the participant through the trade ally was greater than the direct influence on the participant. The score used in the calculation of participant free ridership was equal to the trade ally influence score for the customers’ trade ally if that trade ally completed the survey. If the participants’ trade ally did not complete the survey, an average of trade ally influence scores based on all trade ally responses was used.

The trade ally influence scores were based on the responses provided by trade allies to the following questions:

TA1: On a scale of 0 to 10 where 0 is “not at all important” and 10 is “extremely important,” how important was the [PROGRAM], including the incentive and information provided through the program, in influencing your level of marketing and selling of energy efficient products to I&M customers during [YEAR]?

TA2: [IF GREATER THAN 2]: Please briefly describe the most significant ways in which the program influenced your level of marketing and selling of energy efficient items to customers during [YEAR].

TA3: Thinking about the projects that you completed as part of the PROGRAM in [YEAR], did the availability of incentives from the program influence the type, quantity, or efficiency level of the items that you recommended to customers? In other words, would you have made different recommendations if the program were not available?

TA4: [IF YES] Please briefly describe the most significant ways in which the program influenced your decision to recommend energy efficient item to customers during [YEAR].


The trade ally score was calculated as shown in Table 3-7.

Table 3-7 Calculation of Trade Ally Influence Score

Response to TA3 Trade Ally Score Y 1 - TA1/10 N (TA1/10)*.5

3.3.2.1.3 Program Influence on Project Timing

To account for deferred free ridership due to the program’s effect on the timing of the implementation of the efficiency measure, respondents were asked the following two questions:

Did you purchase and install the [EFF_MEASURE] sooner than you would have if the information and financial assistance from the program had not been available?

When might you have purchased or installed the same [EFF_MEASURE] if you had not participated in the program?

Based on the responses to those questions a timing adjustment was calculated as shown in Table 3-8.

Table 3-8 Timing Adjustment Score

Likely Timing of Project in Absence of the Program Timing Score

Within 6 months 1 Between 6 months and 1 year .67 In more than 1 year to 2 years .33 In two years or more 0

3.3.2.1.4 Free Ridership Scoring

For respondents that did not have plans or intentions, an overall free ridership score was developed based on the program influence score and timing score. An overall project free ridership score is calculated by combining the scores described above using the following equation:

Free Ridership = Program Influence * Timing Score

3.3.2.2 Methodology for Estimating Spillovers

Program participants may implement additional energy saving measures without receiving a program incentive because of their participation in the program. The energy savings resulting from these additional measures constitute program participant spillover effects.

To assess participant spillover savings, survey respondents were asked whether they implemented any additional energy saving measures for which they did not receive a program incentive.


Respondents were also asked to provide information on the attributes of the measures implemented for use in estimating the associated energy savings.

Participants who report implementing on one or more efficiency measures are then asked two questions for use in developing a spillover score:

SO1: On a scale of 0 to 10, where 0 represents “not at all important” and 10 represents “extremely important”, how important was your experience with [PROGRAM] in your decision to purchase the items you just mentioned?

SO2: On a scale of 0 to 10, where 0 represents “not at all likely” and 10 represents “extremely likely” how likely would you have been to make the additional purchases you just mentioned even if you had not participated in the [PROGRAM]?

The response to these questions were used to develop a spillover score as follows:

Spillover = Average(SO1, 10 – SO2)

All of the associated measure savings were considered attributable to the program if the resulting score was equal to or greater than 7.

3.3.2.3 Methodology for Estimating Non-Participant Spillover

Non-participant spillover was estimated using data collected through the PY2017 non-participant survey. To estimate PY2018 nonparticipant spillover, the PY2017 non-participant spillover ratio was applied to the 2018 customer annual kWh usage. That is, total non-participant spillover (NPSO) savings for the portfolio was estimated as follows:

NPSO = (2017 Survey Spillover / 2017 Survey Usage) * 2018 Population Usage

Where,

Survey Spillover = Total kWh associated with 2017 spillover measures

Survey Usage = Total 12-month kWh usage for the 2017 survey sample

Population Usage = Total 12-month kWh usage for the 2018 population

The total non-participant spillover was allocated to the Home Energy Products – Appliances Program in proportion to its share of the 2018 expenditures.

Non-participant peak demand spillover impacts were estimated by multiplying the non-participant kWh savings by the ratio of the program ex post gross kW savings to ex post gross kWh savings.

3.3.3 Results of Ex Post Net Energy Savings Estimation

Table 3-9 summarizes the number of survey responses and average free-ridership scores by measure for the Home Energy Products - Appliances Program.


Table 3-9 Survey Response Count and Average Free Ridership Score by Measure

Measure Survey Response Count

Average Free Ridership Score

ECM Retrofit 43 31% Wi-Fi Smart Thermostat 27 47% Variable Speed Pool Pump 9 32% Air Conditioner 28 46% Dehumidifier 26 36% Heat Pump Water Heater 3 49% Ductless Heat Pump 7 51% Heat Pump 11 38% Ceiling Fan 1 100%

3.3.3.1 Ex Post Net kWh Savings

Table 3-10 summarizes the ex post annual net kWh savings of the Home Energy Products – Appliances Program. The annual net savings totaled 792,901 kWh. The net-to-gross ratio is 61%.


Ex Ante Gross kWh

Savings

Gross Audited

kWh Savings

Gross Verified

kWh Savings

Ex Post Gross kWh

Savings

Gross Realization

Rate

Ex Post Net kWh

Savings

Net-to-Gross Ratio

Ex Post Net

Lifetime kWh

Savings

1,233,192 1,233,106 1,231,521 1,302,679 106% 792,901 61% 11,402,084

3.3.3.2 Ex Post Net kW Demand Reductions

Table 3-11 summarizes the ex post net kW demand reduction of the Home Energy Products -- Appliances Program. The net demand reduction equaled 153.42 kW. The net-to-gross ratio is 61%.

Table 3-11 Net kW Demand Reduction

Ex Ante Gross

kW Savings

Gross Audited

kW Savings

Gross Verified

kW Savings

Ex Post Gross

kW Savings

Gross Realization

Rate


Net-to-Gross Ratio

312.50 312.50 312.14 252.05 81% 153.42 61%


ADM completed a limited process evaluation of the PY2018 program. The limited process evaluation documented notable program changes and developments, findings from a review of the program database, and findings from a survey of participating customers.


Data was collected from multiple data sources to support the process evaluation of the program. The data collection activities and the sample sizes are presented in Table 3-12.

Table 3-12 Summary of Data Collection Activities for the Home Energy Products – Appliances Program

Data Collection Activity Mode of Data Collection Sample Size Participant survey Email and telephone 102 Staff interviews Telephone in-depth interviews 1



Table 3-13 summarizes the number of measures, associated savings, incentives and acquisition costs for the measures rebated through the Home Energy Products – Appliances Program.

Table 3-13 Summary of Program Activity

Measure Number Installed

Ex Ante kWh Savings Incentives Dollars per

kWh Ductless Heat Pump 89 443,566 $38,520 $0.09 ECM Retrofit 1,010 419,150 $151,450 $0.36 Air Source Heat Pump 75 99,073 $27,150 $0.27 Wi-Fi Smart Thermostat 457 95,264 $31,438 $0.33 Air Conditioner 236 47,571 $76,900 $1.62 Heat Pump Water Heater 33 45,483 $11,550 $0.25 Variable Speed Pool Pump 39 43,134 $5,800 $0.13 Dehumidifier 195 39,627 $4,875 $0.12 Ceiling Fan 3 324 $75 $0.23


This section summarizes the core findings of interviews conducted with I&M program staff for the purposes of learning about any changes to program structure, identifying program objectives, and assessing the extent to which there are future opportunities for program improvement. ADM evaluators interviewed the I&M senior energy efficiency consumer program coordinator to gain insights in the Home Energy Products program for 2018.

The key findings from the interviews are summarized below.

The program ex ante kWh savings exceeded its savings goal. The Home Energy Products program ex ante savings exceeded the savings goals in Indiana in 2018. Staff reported that uptake of HVAC measures such as heat pumps was a key strength of the program. Multiple program actions likely lead to an increase in uptake of these measures. I&M program staff organized five training seminars on ductless systems in spring 2018 and partnered with Mitsubishi to provide the trainings for contractors. Additionally, I&M


developed relationships with Mitsubishi and Fujitsu that increased marketing and promotion of ductless heat pump systems. Furthermore, the Home Energy Products program offered a double rebate for the ductless heat pump systems in the spring and fall. The promotions resulted in more than twice the amount of participation.

Multiple marketing channels were used to promote program rebates. Primary marketing activities included: email campaigns, cross promotion, speaking at events (e.g., home shows), promoted in our home energy reports, promoted on our energy efficiency newsletter, website, social media campaigns, Google ads, and postcards to promote the ductless heat pumps.

I&M staff increased outreach to distributors and contractors in 2018 compared to efforts in 2017. I&M staff focused on HVAC distributors and noted the importance of their relationship with contractors. I&M staff spent a good amount of time during the second half of the program year with relationship building and development. Relationship building with distributors was noted a lesson learned for the program and I&M because of the distributor’s relationship with contractors. I&M staff was able to leverage this relationship to promote the program to contractors at distributor sponsored training events. I&M would like to partner with the distributors for the marketing and promoting the program. This is in development and discussion with internal I&M marketing.

Outreach to pool dealers and motor repair shops to promote pool pump rebates was also increased in 2018.

Program staff reported increased communication (emails, phone calls, and in person visits) with contractors in 2018. Staff noted that contractors appreciated the increased communication efforts.

Application process streamlined based on contractor feedback. I&M met with the contractors last program year to gather feedback and suggestions for the program. As a result, I&M made changes to the online application process to streamline and simplify it. I&M will be requiring the contractors to submit their applications either online or through US mail, in 2019. This change will create a streamlined internal process and customers will receive their rebate sooner.

Changes planned for the application process in 2019. In 2019, I&M is exploring the addition of a contractor web portal, where they can submit applications, check status of rebates, and review the number of jobs they have with I&M. Additionally, I&M is exploring and developing a form (rebate reassignment form) for contractors to allow customers to receive an instant rebate by signing over their rebate amount to contractor. This will be piloted in 2019 for contractors.

Rebate levels will change to create parity across service territories. I&M will adjust rebate levels and measure offerings so that they are more similar in Michigan and Indiana. The goal is to make it easier for contractors who work in both territories. The change in


rebate amounts will take effect in 2019 but the preliminary work took place in 2018. Additionally, the program has removed the ceiling fan rebates for 2019.

Data management and QA/QC procedures were not changed in 2018. Staff indicated the data is kept current to manage the program.


ADM completed a survey of program participants to collect information regarding their experience with the Home Energy Products – Appliances Program in Indiana. The survey was administered by email and telephone. A total of 102 participants completed the survey.


A majority of respondents (51%) first became aware of the HEP Program through a contractor or plumber. Contractors or plumbers were also the most common source of program awareness during the prior program year. Figure 3-1 summarizes the initial source of program awareness among respondents, with other common sources of program awareness being the I&M website (20%), a bill insert or I&M mailing (9%), and retail stores (8%).


Forty-one percent of survey participants indicated they knew about I&M’s HEP Program before starting the process of purchasing the measure (see Table 3-14).


Table 3-14 Time of Awareness of I&M’s Home Energy Products Program


(n = 95) Before starting the process of purchasing the measure 41% At the time you made the purchase decision 18% After deciding to purchase the measure 11% After researching the product but before deciding to purchase 31%

Selection of Measures

Table 3-15 summarizes the sources of information that participants consulting when selecting a measure to purchase. Respondents were able to select multiple respondents. A majority of respondents indicated that they consulted installation contractors (55%), while 21% of respondents consulted the internet and 12% of respondents consulted retail stores. Four percent of respondents indicated that they did not consult any information sources prior to selecting their equipment.

Table 3-15 Sources of Information Consulted when Selecting Measure

Response Percent (n = 102)

Installation contractors 55% Internet 21% Retailers 12% Friend, neighbor, relative or co-worker 6% Other 4% Did not look for any information about what to buy 4% Consumer reports or other product magazines 3% Utility 2%

Figure 3-2 summarizes the reasons for purchasing the specific equipment model and respondents were able to select multiple responses. Respondents most commonly reported that the equipment was a good price (37%). Thirty-six percent of respondents indicated that they purchased their specific measure because the contractor or retailer had recommended it, and 29% of respondents stated that their purchase was due to the equipment having the features they wanted.


Figure 3-2 Reason for Choosing Model or Type of Measure

Ninety-one percent of respondents reported that they did not have any concerns about purchasing the measure. Initial concerns expressed by respondents included the cost of equipment (2 respondents), the value of the energy savings (2 respondents), the appearance of the equipment (1 respondent), and being unfamiliar with the equipment’s technology (1 respondent).

Table 3-16 summarizes the type of stores where the measures were purchased from, with a majority of respondents (58%) indicating that they purchased the measure from a heating or cooling contractor.

Table 3-16 Type of Store or Contractor Measure Purchased From


Heating/ cooling contractor 58% Home improvement store 15% Appliance store 12% Internet 7% Swimming pool contractor 4% Local hardware store 2% Other 2%

Forty-eight percent of survey participants obtained their rebate application from a contractor, 44% obtained it from the I&M website, and two percent obtained it from another website. Sixty-three percent of survey participants indicated they filled out the rebate themselves, and 34% indicated that a contractor or salesperson fill out the rebate application for them.

Satisfaction


ADM used a 5-point satisfaction scale ranging from 1 (very dissatisfied) to 5 (very satisfied) to assess respondents’ satisfaction with the HEP and with I&M as their electric utility provider.

Satisfaction ratings were high across all program elements, with a majority of respondents reporting that they were satisfied with each listed aspect of the program (see Figure 3-3). The portion of satisfied respondents ranged from 88% to 96% for the rated aspects.

Since installing the rebated equipment, 43% of respondents indicated that they have noticed an energy savings on their electric bill. Of these respondents, 88% reported that they were satisfied with the savings that they had observed.

The rebate application process received the highest number of dissatisfied ratings, and three respondents provided additional commentary explaining that they had encountered difficulties when attempting to fill out forms or obtain the rebate for their equipment.

Figure 3-3 Home Energy Products Appliances Satisfaction

Eighty-nine percent of respondents indicated they were satisfied with I&M as an electricity service provider (i.e. provided a rating of 4 or 5) with three percent of respondents indicating that they were dissatisfied (i.e. providing a rating of 1 or 2).

Yearly Trends

ADM compared 2018 survey results to those in 2017 to observe any differences from year to year. Key findings are presented in this section. The increases or decreases are presented as the difference in the percentage of customers providing a response in 2018 as compared to 2017. (i.e., an increase or decrease in percentage points from 2017 to 2018).


From 2017 to 2018, there was an 11 percentage-point increase in those who indicated they learned of the program from a contractor or plumber and a 12 percentage point increase who learned of the program from the I&M website (see Figure 3-4).


Across all categories, there was an increase in the share of customers citing various factors as the reason for selecting a specific measure (Figure 3-5). The largest increase was in the share of respondents who indicated they selected the specific measure because it was a good price (18 percentage point increase).

Figure 3-5 2017 and 2018 Reasons for Selecting Measure (Model or Type) Comparison


3.5.1 Conclusions



The program exceeded its goal. The Home Energy Products – Appliances program exceeded its savings goals in 2018.

An extensive effort was undertaken to increase uptake of HVAC measures. I&M program staff organized five training seminars on ductless systems in spring 2018 and partnered with Mitsubishi to provide the trainings for contractors. Additionally, I&M heavily focused on relationship building with distributors and contractors in 2018 and developed relationships with Mitsubishi and Fujitsu. Through the relationship, there was marketing and promotion of ductless heat pump systems. Moreover, the outreach to contractors is a positive development as 51% of the respondents learned of the rebates from contractors and 36% purchased the equipment because the contractor recommended it. The Home Energy Products – Appliances program also offered a double rebate for the ductless heat pump systems in the spring and fall. The overall impact of these actions was that the share of ex ante savings for HVAC systems replacements increased by approximately 16 percentage points, and the share of savings from ductless heat pumps increased by 18 percentage points.

Multiple marketing channels were used in PY2018. The program also marketed rebates directly to customers including email campaigns, Google ads, and a post-card campaign to promote ductless heat pumps. Contractors and plumbers (mentioned by 51%) were the most commonly mentioned sources of program awareness, followed by I&M website (21%) and retail stores (8%).

Application process was streamlined. Based on meetings with contractors, I&M modified and streamlined the application process for the program. Additional changes are being considered for 2019 including the development of a contractor web portal for submitting applications, checking the status of rebates, and reviewing the number of jobs scheduled.

Participants are satisfied with the program. Both participants and trade allies are satisfied with the program. Ninety-two percent of participants were satisfied with the program overall and 88% were satisfied with the application process. Additionally, 86% were satisfied on the savings on their electricity bills since installing the rebated equipment.


The following summarizes the recommendations based on the evaluation findings.

Consider reducing incentives for central air conditioners. The cost per ex ante kWh saved was somewhat high ($1.62) and considerably higher than other measures.


4 Home Energy Products - Lighting This chapter presents the results of both the impact and process evaluations of the 2018 Home Energy Products – Lighting Program that Indiana Michigan Power (I&M) offered to its residential customers during the period of January 2018 through December 2018.



Assess marketing and outreach approaches;

Assess coverage of retailer and lamp types;

Assess satisfaction among participating retailers; and



The Home Energy Products – Lighting program increases demand for energy-efficient lighting through upstream incentives and by providing discounts for online lighting purchases.

The objectives of the program include lowering electric consumption in the residential market sector through the purchase and installation of eligible energy efficient lighting measures. The program works with lighting manufactures and retailers to reduce the cost of efficiency lighting technologies and to encourage retailers to promote the purchase of efficient lighting.


The following section presents the methodology that was used for estimating the gross energy and demand impacts resulting from the Home Energy Products – Lighting Program in 2018.


The M&V approach for the Home Energy Products – Lighting Program focused on determining the following:

Number of bulbs discounted and sold through the program;

Average annual kWh savings per purchased bulb type; and

Average kW reduction per purchased bulb type.

4.2.1.1 Data Collection and Sampling Plan




Program summary metrics from the 2018 I&M DSM EE Program Scorecard;

Participant survey data collected through targeted random digit dialing; and


Customer surveys were conducted using a targeted random dialing framework. Sampling frames based on random telephone numbers within the I&M Indiana service territory were developed to ensure all contacted individuals are I&M Indiana customers.

The targeted random dialing procedure was carried out in zip codes prioritized according to the volume of retail sales. The interviewers used screening questions to confirm that the respondent was (a) an I&M Indiana utility customer, (b) had purchased products incentivized as part of the program, and (c) that the purchase(s) took place between January and December of 2018.

The strength of this approach is the ability to obtain a random and relatively large sample size cost-effectively. Parameters used in gross and net savings calculations were developed from the collected survey data. ADM obtained a final sample size of 150 Indiana residents who had purchased lighting products during 2018 through the targeted random dialing survey effort.


As a first step, ADM reviewed data tracking systems associated with the program to ensure that the data provided sufficient information to identify bulb types/characteristics and to calculate energy and demand impacts in accordance with the 2015 Indiana Technical Reference Manual (TRM) Version 2.2. ADM further reviewed the program data to verify required fields were populated (i.e., the data was not missing) and that the program measures were appropriately categorized by measure type. Finally, the program tracking data and the associated summary data provided in the I&M DSM EE Program Scorecard were reviewed for consistency and duplicate or erroneous entries.

ADM’s review of the program data tracking data found there to be sufficient information to estimate energy and demand impacts in accordance with the Indiana TRM. The data included details of the LED bulbs discounted through the program, including bulb types and wattages, information on retail pricing and program discounts, manufacturer and retailer information, and ex ante savings and demand reductions. ADM did not discover erroneous or duplicative entries in the primary tracking data.


ADM referenced the appropriate methodologies and deemed inputs from the Indiana TRM to calculate the gross annual energy savings and gross peak demand reduction from distributed program LEDs. Care was taken to assure any assumptions were reasonable and current, and that there were no errors in the algorithms.

The following sections describe the specific algorithms and inputs that were used to calculate energy and demand impacts for all program-discounted LED lamps.


CFL and LED Lighting: The following algorithm was used to determine annual kWh energy savings, in accordance with the Residential ENERGY STAR Lighting (CFL and LED) section from the Indiana TRM:

∆𝑘𝑘𝑘𝑘ℎ =(𝑘𝑘𝑆𝑆𝑆𝑆𝑆𝑆𝑈𝑈𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵 − 𝑘𝑘𝑆𝑆𝑆𝑆𝑆𝑆𝑈𝑈𝐵𝐵𝐸𝐸𝐸𝐸)

1,000× 𝐸𝐸𝑆𝑆𝐺𝐺 × 𝐻𝐻𝐺𝐺𝑈𝑈𝐺𝐺𝑆𝑆 × (1 + 𝑘𝑘𝐻𝐻𝐹𝐹𝐵𝐵)

Where:

WattsBASE = Baseline bulb wattage defined in the TRM table below

WattsEFF = Wattage of efficient lamp, actual

ISR = In Service Rate or percentage of distributed units that are installed, as determined through analysis of customer survey response data; 76%

HOURS = Average hours of use per year; 902

WHFE = Waste Heat Factor for Energy to account for cooling savings from efficient lighting; -0.059

ADM calculated ex post peak demand reduction using the following Indiana TRM defined equation:

∆𝑘𝑘𝑘𝑘 =(𝑘𝑘𝑆𝑆𝑆𝑆𝑆𝑆𝑈𝑈𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵 − 𝑘𝑘𝑆𝑆𝑆𝑆𝑆𝑆𝑈𝑈𝐵𝐵𝐸𝐸𝐸𝐸)

1,000× 𝐸𝐸𝑆𝑆𝐺𝐺 × (1 + 𝑘𝑘𝐻𝐻𝐹𝐹𝐷𝐷) × 𝐶𝐶𝐹𝐹

Where:

WattsBASE = Baseline bulb wattage defined in the TRM table below

WattsEFF = Wattage of efficient lamp, actual

ISR = In Service Rate or percentage of distributed units that are installed, as determined through analysis of customer survey response data; 76%

WHFD = Waste Heat Factor for Energy to account for cooling savings from efficient lighting; 0.057

CF = Summer peak coincidence factor, 0.11

Table 4-1 below, taken directly from the 2015 Indiana TRM, was used to determine baseline wattages used to calculate energy savings and peak demand reduction estimates as specified in the algorithms above.

Table 4-1 Baseline Wattage Determination

Efficient Technology Watts (efficient) Watts (baseline)

CFL 15W or less 3.05*Watts(EFF) 16W - 20W 3.00*Watts(EFF)

21W or more 3.06*Watts(EFF)


Efficient Technology Watts (efficient) Watts (baseline)

LED 9W or less 3.38*Watts(EFF) 10W - 17W 3.41*Watts(EFF)

18W or more 4.04*Watts(EFF)

4.2.1.4 Methodology for Calculating of In-service Rates (ISR)

To calculate the ISR for ex post impact estimates, ADM surveyed I&M customers residing in the Indiana service territory using a targeted random dialing framework carried out in zip codes prioritized according to the volume of retail sales presented in the main tracking database. Survey respondents were asked a series of questions to the installation quantities and timing of recent bulb purchases. First, they were asked about the types and quantities of light bulbs purchased within the past six months. Then they were asked the quantity of bulbs they (a) installed within one week of purchase, (b) saved for future installation, and (c) still hadn’t installed at the time of the survey. ADM used this information to calculate a first-year ISR equal to:

𝐸𝐸𝑆𝑆𝐺𝐺 = ∑ 𝐵𝐵𝑢𝑢𝑆𝑆𝑏𝑏𝑈𝑈 𝐷𝐷𝑆𝑆𝑈𝑈𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝐷𝐷𝑆𝑆 𝑆𝑆𝑆𝑆 𝑆𝑆𝐷𝐷𝐼𝐼𝐷𝐷 𝐷𝐷𝑟𝑟 𝑈𝑈𝑢𝑢𝑆𝑆𝐷𝐷𝐷𝐷𝑏𝑏

∑ 𝐵𝐵𝑢𝑢𝑆𝑆𝑏𝑏𝑈𝑈 𝑖𝑖𝑢𝑢𝑆𝑆𝐼𝐼ℎ𝑆𝑆𝑈𝑈𝐷𝐷𝑆𝑆 𝑆𝑆𝑢𝑢𝑆𝑆𝐷𝐷𝑆𝑆𝐴𝐴 2018

Ex post kWh savings and kW demand reductions resulting from the lighting component of the Home Energy Products Program were adjusted by applying the estimated first-year installation rates of program-discounted bulbs to the pertinent Indiana TRM energy savings and peak demand reduction algorithms.

Table 4-2 below displays the first-year in-service rate of program LEDs developed from the targeted random dialing survey effort.

Table 4-2 LED In-Service Rate – Lighting

Measure ISR

LED 76%


ADM estimated ex post gross energy and peak demand impacts for the Home Energy Products – Lighting Program through detailed analysis of program tracking data and participant survey data. This section presents the results of the gross savings calculation activities.


Table 4-3 below shows the ex ante annual kWh savings reported in the program tracking database and the ex post annual gross kWh savings resulting from ADM’s evaluation. The overall gross kWh realization rate for the program is 76%. The discrepancy between ex ante and ex post savings values is explained completely by differences in ISRs referenced in ex ante and ex post savings calculations. If the estimated first-year ISR of program discounted bulbs was 100%, the gross kWh realization rate would be 100%.


Table 4-3 Program-level Annual Gross kWh Savings – Lighting






19,618,027 19,618,027 14,953,713 14,965,413 76%

Table 4-4 below shows the estimated measure-level annual gross energy savings resulting from the program. Standard LEDs comprised approximately 80% of the total ex post gross kWh savings.

Table 4-4 Measure-level Annual Gross kWh Savings – Lighting

Measure Verified

Number of Bulbs

Ex Ante Gross kWh

Savings

Gross Audited

kWh Savings

Gross Verified

kWh Savings

Ex Post Gross kWh

Savings

Gross Realization

Rate

Standard LED 600,045 15,745,669 15,745,669 12,002,033 12,010,338 76% Specialty LED 190,211 3,872,358 3,872,358 2,951,680 2,955,076 76% Total 790,256 19,618,027 19,618,027 14,953,713 14,965,413 76%


Table 4-5 below shows the total ex ante peak kW reduction reported in the program tracking database and the total ex post peak kW reduction resulting from ADM’s evaluation. The overall gross kW realization rate for the program is 76%. The discrepancy between ex ante and ex post kW savings is almost exclusively accounted for by the differences in in-service rates referenced in ex ante and ex post savings calculations.

Table 4-5 Program-level Gross kW Reductions – Lighting






2,687.37 2,687.37 2,048.43 2,049.64 76%

Table 4-6 below shows the estimated measure-level annual gross energy savings resulting from the program.

Table 4-6 Program-level Gross kW Reductions – Lighting

Measure Verified

Number of Bulbs


Gross Audited kW

Savings

Gross Verified

kW Savings


Gross Realization

Rate

Standard LED 600,044.92 2,156.91 2,156.91 1,644.09 1,644.92 76% Specialty LED 190,211.00 530.45 530.45 404.34 404.72 76% Total 790,255.92 2,687.37 2,687.37 2,048.43 2,049.64 76%



4.3.1 Methodology for Estimating Ex Post Net Impacts

Determining the net effects of the lighting discounts required estimating the percentage of energy savings from efficient lighting purchases that would have occurred without program intervention. Ideally, participating retailers could provide light bulb sales data for non-program time periods or from similar non-program retail locations. This data would provide adequate information from which to calculate the lift in efficient lighting sales attributable to the program price mark downs. However, retailers are reluctant to release sales data for this purpose because of the possibility that the data may be exposed to competitors or otherwise misused.

As a result, evaluating the net effects of the price discounts required estimating free ridership without non-program sales data. ADM used two approaches to estimating lighting discount free ridership: self-reported responses from a random sample of customers who have purchased efficient light bulbs in the past six months and a consumer demand modeling approach based on sales of discounted lighting through the program.

In addition to free ridership, ADM’s net impact estimates include an adjustment for non-participant spillover.

4.3.1.1 Free Ridership: Self-report Survey Methodology

ADM administered surveys about light bulb purchases made in the previous six months to a random sample of I&M customers. The goal of these surveys as it relates to net impacts was to elicit information from which to estimate the number of bulbs that the customer would have purchased in the counterfactual scenario where the efficient light bulbs were not discounted.

The survey effort was conducted with a random sample of I&M customers through a random digit dialing framework. The strength of this approach is that it also allows for further questioning regarding the fate of recently purchased bulbs (e.g., installed immediately, stored for future use, location of installation, etc.). The biggest drawback to the approach is the potential for respondent recall bias. It may be difficult to get accurate responses to questions about the number of bulbs the respondent recently purchased and whether they were discounted through the program, for example. This may be especially true for upstream price markdown programs, where the counterfactual scenario of regular retail prices may be difficult to explain or grasp.

Survey respondents were asked a series of questions to elicit feedback regarding influences on their light bulb purchasing decisions. Each respondent was then assigned a free ridership score based on a consistent free ridership scoring algorithm.

The final respondent FR score was calculated as follows:

Final Respondent FR Score = 0.3 * [Prior Experience FR] + 0.7 * [Behavior w/o Discount FR] – Mitigating Factors

The free ridership scoring algorithm for light bulb purchases from the targeted random dialing surveys is shown in Figure 4-1.


Figure 4-1 Free ridership Scoring for Targeted Random Dialing Survey Respondents

4.3.1.2 Free Ridership: Price-Response Model Methodology

ADM also estimated free ridership using a price-response modelling approach. This approach estimated sales in response to changes in pricing of the bulbs. ADM developed a regression to estimate the relationship between price and quantity sold. ADM used a negative binomial model to account for the right-skewed relationship between prices and quantities. The dependent variable was number of packages sold by the program. Independent variables used to predict sales included, month, promotion dummy (e.g. presence of a promotional event on the sales date), program price, and a dummy variable for each model type.

Model types were defined as a combination of bulb type (i.e. specialty LED vs. standard LED), bulb shape (i.e. A19 vs BR40), wattage range (i.e. 0-5, 5-10, etc.…), and the number of bulbs per package. Quantities of bulbs sold in the absence of the program were then predicted using pre-program prices and the price-response model, and program quantities were also predicted using program prices and the model coefficients. The final price response model is used to estimate a free ridership as described in the equation below:

𝐹𝐹𝑆𝑆𝐷𝐷𝐷𝐷 𝑆𝑆𝐷𝐷𝑆𝑆𝐷𝐷𝑆𝑆𝑈𝑈ℎ𝐷𝐷𝑖𝑖 𝑆𝑆𝑆𝑆𝑆𝑆𝐷𝐷𝐷𝐷 = ∑ (𝐸𝐸�𝐵𝐵𝑢𝑢𝑆𝑆𝑏𝑏𝑈𝑈𝑁𝑁𝑐𝑐𝑁𝑁𝑁𝑁𝑐𝑐𝑖𝑖𝑁𝑁𝑏𝑏𝑚𝑚𝑖𝑖�)𝑓𝑓

𝑖𝑖

∑ (𝐸𝐸�𝐵𝐵𝑢𝑢𝑆𝑆𝑏𝑏𝑈𝑈𝑁𝑁𝑁𝑁𝑐𝑐𝑖𝑖𝑁𝑁𝑏𝑏𝑚𝑚𝑖𝑖�)𝑓𝑓𝑖𝑖

Where:

𝐸𝐸�𝐵𝐵𝑢𝑢𝑆𝑆𝑏𝑏𝑈𝑈𝑁𝑁𝑐𝑐𝑁𝑁𝑁𝑁𝑐𝑐𝑖𝑖𝑁𝑁𝑏𝑏𝑚𝑚𝑖𝑖� = the expected number of bulbs of type, i, purchased given original retail pricing (as predicted by the model).


𝐸𝐸�𝐵𝐵𝑢𝑢𝑆𝑆𝑏𝑏𝑈𝑈𝑁𝑁𝑁𝑁𝑐𝑐𝑖𝑖𝑁𝑁𝑏𝑏𝑚𝑚𝑖𝑖� = the expected number of bulbs of type, i, given program discounted pricing (as predicted by the model).

The price-response modeling approach is advantageous in that it is built upon actual sales data from participating retailers (as opposed to relying on consumer self-report surveys). There are, however, a number of limitations for the approach. Most importantly, non-program sales data is unavailable for inclusion in the model. As a result, the modeling of price impacts may fit program sales data well, but it is uncertain whether those price effects apply well to prices outside of program ranges. Additionally, there are likely variables that affect sales levels for CFLs and LEDs that are not captured by the program tracking data; thus, there is a risk of omitted variable bias in addition to the inherent amount of error from statistical modeling.

4.3.1.3 Overall Estimation of Free Ridership Methodology

Due to the advantages and disadvantages associated with each of the above methods, ADM averaged the free ridership scores from each method to obtain a combined free ridership score.


Non-participant spillover was estimated using data collected through the PY2017 non-participant survey. To estimate PY2018 nonparticipant spillover, the PY2017 non-participant spillover ratio was applied to the 2018 customer use age. That is, total non-participant spillover (NPSO) savings for the portfolio was estimated as follows:


Where,




The total non-participant spillover was allocated to the Home Energy Products – Lighting Program in proportion to its share of the 2018 expenditures.



4.3.2.1 Free Ridership: Self-report Survey Results

All 150 survey respondents reported purchasing LEDs during the preceding six months and answered all necessary questions to estimate free ridership. This group of 150 respondents served as the sample for estimating free ridership according to the calculation shown in Figure 4-1. Once a free ridership value was calculated for each respondent, the responses were weighted by bulb


type and summed. The resulting free ridership estimate from the self-report survey was 51%, as shown in Table 4-7.

Table 4-7 Free Ridership: Self-report Survey Results

Estimation Methodology

Bulb Type Free Ridership

%

Self-Report Survey

Standard LED 57%

Specialty LED 33%

Overall 51%

4.3.2.2 Free Ridership: Price-Response Model Results

ADM estimated the overall free ridership rate for upstream LEDs to be 29% using the price response model. The estimated free ridership rate for specialty LEDs is 36%, while the estimated free ridership rate for standard LEDs is 26%. ADM ran separate models for each bulb type (i.e. LED Standard and LED Specialty). The coefficients on program price are negative (the expected direction) and statistically significant at the 99% level, while the coefficients on promotional events are positive and statistically significant at the 99% level. The latter result indicates that promotional events have a positive effect on bulb sales. The following tables present the final Standard and Specialty LED model specifications used to estimate overall free ridership of 29%.

Table 4-8 Price-Response Model Final Specification – Standard LEDs

Coefficient Estimate Std Err Statistic p.value CI-low CI-high (Intercept) 1.759 0.092 19.045 0.000 1.578 1.940 Promotional Event 0.252 0.067 3.771 0.000 0.121 0.382 LEDstd_A-Line_0-5_0-500_2 0.127 0.144 0.883 0.377 -0.155 0.410

Aug -0.036 0.036 -0.993 0.321 -0.106 0.035 Dec 0.287 0.035 8.142 0.000 0.218 0.356 Feb 0.232 0.043 5.334 0.000 0.147 0.317 Jan 0.562 0.043 13.096 0.000 0.478 0.646 July -0.027 0.037 -0.736 0.462 -0.099 0.045 June -0.198 0.037 -5.426 0.000 -0.270 -0.127 Mar 0.021 0.046 0.464 0.642 -0.069 0.112 May -0.101 0.041 -2.460 0.014 -0.182 -0.021 Nov -0.013 0.033 -0.380 0.704 -0.077 0.052 Oct -0.013 0.033 -0.380 0.704 -0.078 0.052 Sept 0.325 0.035 9.172 0.000 0.256 0.395


Table 4-9 Price-Response Model Final Specification – Specialty LEDs

Coefficient Estimate Std Err Statistic p.value CI-low CI-high

(Intercept) 2.548 0.284 8.958 0.000 1.991 3.105 Promotional Event 0.429 0.058 7.413 0.000 0.316 0.543 LEDspec_Bulged_Reflector_10-15_1000-1500_2 0.703 0.289 2.429 0.015 0.136 1.270

Aug -0.117 0.041 -2.827 0.005 -0.198 -0.036 Dec 0.145 0.040 3.639 0.000 0.067 0.224 Feb 0.131 0.048 2.729 0.006 0.037 0.225 Jan 0.211 0.048 4.399 0.000 0.117 0.305 July -0.155 0.042 -3.663 0.000 -0.238 -0.072 June -0.163 0.043 -3.763 0.000 -0.248 -0.078 Mar 0.245 0.051 4.829 0.000 0.146 0.345 May -0.036 0.046 -0.778 0.437 -0.127 0.055 Nov -0.041 0.039 -1.060 0.289 -0.118 0.035 Oct -0.042 0.039 -1.055 0.292 -0.119 0.036 Sept 0.082 0.041 2.020 0.043 0.002 0.162

Table 4-10 below summarizes the resulting standard and specialty LED free ridership estimates from the price-response model.

Table 4-10 Free Ridership: Price-response Model Results

Estimation Methodology

Bulb Type Free

Ridership %

Price-Response Model

Standard LED 26%

Specialty LED 36%

Overall 29%

4.3.2.3 Overall Estimation of Free Ridership Results

Due to the inherent advantages and disadvantages of each approach, ADM applied a simple average of the two methodologies to determine the final free ridership score for standard LEDs and specialty LEDs in the program. The results, as shown in Table 4-11, is a free ridership score of 41% for standard LEDs and 34% for specialty LEDs.


Table 4-11 Final Free Ridership Estimation

Estimation Methodology Free Ridership %: Standard

LED

Free Ridership %: Specialty

LED

Self-Report Survey 57% 33%

Price-Response Model 26% 36%

Final FR estimate (average) 41% 34%

4.3.2.4 Non-Participant Spillover Results

Non-participant spillover was estimated through a survey of non-participants. The results of the non-participant spillover analysis assigned 173,719 kWh and 23.79 kW to the lighting component of the Home Energy Products Program.


Table 4-12 summarizes the net ex post kWh savings of the Home Energy Products -- Lighting Program. The annual net savings totaled 9,167,354 kWh. The net-to-gross ratio is 61%.

Table 4-12 Ex Post Net kWh Savings – Lighting

Ex Ante Gross kWh

Savings

Gross Audited

kWh Savings

Gross Verified

kWh Savings

Ex Post Gross kWh

Savings

Gross Realization

Rate

Ex Post Net kWh

Savings

Net-to-Gross Ratio

Ex Post Net Lifetime

kWh Savings

19,618,027 19,618,027 14,953,713 14,965,413 76% 9,167,354 61% 86,391,744


Table 4-13 summarizes the net ex post kW demand reduction of the Home Energy Products – Lighting Program. The net demand reduction equaled 1,255.55. kW. The net-to-gross ratio is 61%.


Table 4-13 Ex Post Net kW Demand Reduction – Lighting


Gross Audited kW

Savings

Gross Verified kW

Savings


Gross Realization

Rate


Net-to-Gross Ratio

2,687.37 2,687.37 2,048.43 2,049.64 76% 1,255.55 61%


ADM completed a limited process evaluation of the PY2018 program. The limited process evaluation documented notable program changes and developments, findings from a review of the program database, and findings from a general population survey of I&M customers.

Data were collected from multiple data sources to support the process evaluation of the program. The data collection activities and sample sizes are presented in Table 4-14.

Table 4-14 Summary of Data Collection Activities for the Home Energy Products – Lighting Program

Data Collection Activity Mode of Data Collection Sample Size General population targeted random dialing survey Telephone 150

Staff interviews Telephone in-depth interviews 3



Table 4-15 summarizes the volume of lamps sold and the average discounts provided. As shown, multipack standard (A-line) LEDs accounted for the largest share of lamp sales and were discounted by $1.50 per lamp, on average. The program also included several types of specialty lamps in both single and multipacks. The types of specialty lamps included PAR, candelabra, reflector lamps, BR, and globe lamps. Discounts were provided for dimming and non-dimming lamps.


Table 4-15 Summary of Lamp Sales by Type

Lamp Type Total Lamps Sold Average

Discount Per Lamp

Specialty LED Single lamp 47,155 $0.88 Multipack 740,076 $0.74

Standard CFL Single lamp 20,924 $1.08 Multipack 213,598 $1.21

Standard LED Single lamp 8,940 $1.04 Multipack 6,084 $1.21

As shown in Table 4-16, the discounts were offered a variety of store types including do-it-yourself (DIY) stores, big box retailers, and discount stores (e.g., dollar chain stores). The top two big box stores by sales volume accounted for 32.8% of the lamps sold.

Notable for 2018, an added online sales channel accounted for 5.2% of program sales.

Table 4-16 Distribution of Sales across Retailers

Retailer Percent of Lamps Sold

Big Box 1 19.6% Big Box 3 13.2% DIY 3 12.0% Big Box 2 10.2% DIY 1 7.8% Discount 1 7.0% Big Box 4 7.0% Online 5.2% DIY 2 4.3% Discount 2 3.8% Discount 3 2.8% Big Box 5 2.3% DIY 4 1.8% Other 2 1.8% Independents 0.7% Hard to Reach 0.3% Other 1 0.3%


Figure 4-2 Summary of Weekly and Cumulative Sales During the Program Year


This section summarizes the findings of interviews conducted with I&M program staff and CLEAResult for the purposes of learning of any changes to the program structure, identifying program objectives, and assessing the extent to which there are future opportunities for program improvement. ADM evaluators interviewed the I&M senior energy efficiency consumer program coordinator and the senior program manager and field supervisor from CLEAResult to gain insights in the Lighting program for 2018.

The key findings of the interviews are presented below.

The savings goal increased. The program expanded its savings goals from 10 million to over 19 million kWh (40% increase) and this change was finalized in September. With the savings goal expansion, there were additional objectives: 1) Increase customer attribution and 2) improve the program net impacts. Along with the increase in the savings goal, steps were taken to increase attribution and improve the program net impacts. To increase customer attribution, there were redesigns to the point-of-purchase materials at the participating stores to make them stand out more, an increase in the number of store events, and visits to increase awareness of the discounts. Both I&M and CLEAResult indicated it is important to continuously improve and increase the visibility and awareness of the Lighting program. I&M noted that CLEAResult is effective at accomplishing this and believes that word-of-mouth has also helped increase awareness. The program ex ante savings met the savings goal.

An online lighting store was piloted. The program piloted an online store, which staff referred to as limited time offers (LTOs). The program partnered with a third-party vendor (TechniArt) to offer an online option to purchase program-eligible bulbs. The first LTO,


which was offered in August (referred to as proof of concept), focused on the harder to reach customers in targeted zip codes. The second LTO, which was offered in October, focused on all I&M customers who they had an email address. Both LTOs performed well. The second LTO was highly successful and exhausted the budget in the first weekend. The initial budget for the LTOs was ~$30,000 and due to the success, the program had to increase it to $100,000. Program staff is working on expanding the number of LTOs in 2019 due to their success in 2018.

Changes were made to participating stores and bulb manufacturers. There were additional stores added in 2018, which targeted harder to reach customers and areas. The added stores included Ace Hardware locations, Walgreen’s, Habitat for Humanity, and Goodwill. Program staff indicated some retailers may face legal barriers to participating and that these can delay when discounts are offered at the retailer’s stores.

Lowe’s switched from Sylvania as their primary lighting manufacturer to GE. At year’s end, GE was producing many bulbs, which helped the program.

Multichannel marketing was used in addition to point-of-purchase materials. Staff reported that marketing activities included social media campaigns with targeted zip codes, email campaigns for the limited time offers, bill inserts, and I&M provided marketing materials to help promote other programs at events.

Additional customer education materials were developed. A retailer guide book was created in 2018 for purposes of training retail staff. Those are provided to stores that are participating.

Future program improvements considered. Staff indicated they have proposed to add fixtures to the program and one staff noted there were a lot of general-purpose bulbs sold in 2018 and that they would like to focus program efforts on other measures that increase impact (e.g., flood lights).

There were no changes to data management or QA/QC procedures. Staff indicated the data is kept current to manage the program. There have not been any significant changes to the quality assurance and quality control (QA/QC) procedures in 2018 and no concerns were cited. Staff stated they only use ENERGY STAR SKUs. In addition, they validate the point of sale reports that are provided by the manufacturing partners and ensure that only eligible measures are incented.

4.4.1.3 General Population Survey Findings

The survey was administered by telephone to I&M customers residing in the Indiana service territory. A total of 150 customers completed the survey.

Recent Light Bulb Purchases and Bulb Familiarity

Figure 4-3 summarizes the number of bulbs purchased. First customers discussed their recent light bulb purchases. As shown, 38% of survey respondents indicated they had purchased between six


and ten lightbulbs, 33% purchased less than six, and 12% purchased between eleven and 15 lightbulbs in the previous six months Figure 4-3 displays the results.

Figure 4-3 Quantity of Lightbulbs Purchased

Survey respondents identified the retail locations where they purchased the LED bulbs, Table 4-17 summarizes the results. Most respondents identified Menards and Walmart, 45% and 32% respectively, as the retail locations where they purchased LED bulbs, followed by Lowes (29%), Meijer (13%), Costco (5%), and The Home Depot (4%).

Table 4-17 Retail Locations where Participants’ Purchased LED lightbulbs

Retail Location Percent (n = 148)

Menards 45% Walmart 32% Lowes 29% Meijer 13% Costco 5% The Home Depot 4% Walgreens 3% Dollar Tree 2% Target 1% Batteries Plus Bulbs 1% Do it Best 1%

Survey respondents provided feedback regarding the location where they installed the LED bulbs and what type of bulbs they replaced. Ninety-five percent indicated they were not installed in a business or commercial building, indicating that cross-sector leakage is low.

Most respondents (55%) indicated the LED bulbs they installed replaced incandescent bulbs, 22% indicated it was a variety of bulb types, 9% replaced existing LEDs replaced CFLs bulbs, and 8%


indicated they replaced existing CFLs. Table 4-18 below summarizes the full distribution of responses.

Table 4-18 Existing Bulb Types Identified by Respondents


Incandescent/Traditional 55% Mixture 22% Existing LEDs 9% CFLs 8% Other 6%

Program Awareness

Survey respondents provided feedback regarding whether they recall recently purchasing LED light bulbs at discounted prices, if they were aware that the utility-discounted lighting was available at participating retailers, and finally if they recall purchasing light bulbs that they knew were discounted by the utility.

Regarding whether or not they recall purchasing LED light bulbs in the past 6 months, at a discounted price,

42% of survey respondents recalled that the LED bulbs were discounted from their normal pricing;

87% indicated they recalled who offered the discounts; and

Of those, 11% indicated they were aware that the utility-sponsored the lighting discounts.

Of those respondents that were aware of the utility lighting discounts (n = 15), the most frequent source of program awareness identified by 33% of respondents was informational material provided in their utility bills (e.g., bill inserts). Twenty percent of respondents became aware of the discounts through emails from utility, followed by utility website (13%). Figure 4-4 displays the results.


Figure 4-4 Sources of Awareness

Installation Rates and Prior Purchases

Customers provided feedback regarding any CFL or LED bulbs they purchased in the last 6 months that they have not installed. Seventy-five percent of customers surveyed indicated that some bulbs were still uninstalled and 25% indicated they installed all the bulbs they purchased.

Survey participants provided feedback about their decisions when purchasing various light bulb types and what factors are important to them. How long a lightbulb will last was the most frequently reported factor, cited by 30% of the respondents, followed by energy efficiency (22%), and price (19%). Table 4-19 below summarizes the full distribution of responses.

Table 4-19 Important Factors in Deciding What Light Bulb Type to Purchase


How long the lightbulb will last 30% Energy efficiency 22% Price 19% Brightness 11% Quality/color of light 10% Other 7% Warranty 0%

Survey participants were asked to provide feedback on how they think LED light bulbs compare to other bulbs (CFLs, halogen, or incandescent) based on the factors that are important to them. When compared to an CFL light bulb, 46% of survey respondents believe an LED light bulb is a lot better, followed by 27% who believe it is a little better. When compared to a halogen or incandescent light bulb, 45% of survey respondents believe an LED light bulb is a lot better,


followed by 23% who believe it is a little better. Table 4-20 below summarizes the full distribution of responses.

Table 4-20 How LEDs Bulbs Compare to Other Bulbs on Factors Important to Customers

Response LED compared to CFL

Percent (n = 140)

LED compared to halogen or incandescent

Percent (n = 121) Much worse 2% 0% A little worse 4% 7% About the same 17% 21% A little better 27% 23% A lot better 46% 45% Don't know what a CFL is 4% 3%

Most survey respondents (89%) indicated they were either very or somewhat likely to purchase an LED light bulb the next time they purchase a light bulb for their house, while 2% were somewhat unlikely and 9% indicated they were very unlikely. This suggests that customers have a high opinion of LED light bulbs and are likely to purchase them in the future.

Yearly Trends


Menards was the top retailer for both program years, with a three percentage-point increase from 2017 to 2018. Lowe’s Home Improvement saw a four percentage-point decrease over the same period.

Figure 4-5 2017 and 2018 Retail Stores Comparison

There was a 13 percentage-point increase among those respondents who first learned of the lighting discounts through an in-store promotional event and a 21 percentage-point decrease among those who learned from the I&M website.


There was an eight-point increase in the number of people who reported mixture for the LED replacement. There was a two-point decrease in the number of respondents who indicated they replaced incandescent bulbs with LED and a five point decrease among those who replaced CFLs (see Figure 4-6).

Figure 4-6 2017 and 2018 LED Replacements Comparison

Between 2017 and 2018, there was a five point increase among those who reported they were aware of utility sponsored lighting discounts and a 20 point percentage increase among survey respondents who had ever purchased LED light bulbs (see Table 4-21)

Table 4-21 2017 and 2018 Lighting Discount Awareness and LED Purchases Comparison

Response Percentage by

Response (2017)

Percentage by Response

(2018)

Point Percentage Difference

Aware of utility lighting discounts 6% 11% 5% Previously purchased LEDs 60% 80% 20%


4.5.1 Conclusions


The savings goal was met. The lighting program met an increased savings goal for PY2018.

Program changes were made to decrease free ridership. The program made several changes to increase its influence on customer lighting purchases. These changes included redesigning point-of-purchase marketing materials to make them stand out more to customers, increasing the number of store events, and increasing visits from program staff to improve awareness of discounts. Additionally, the program discounts were promoted through other channels such as social media campaigns to targeted zip codes and bill inserts. Survey results indicate that in-store materials and the I&M website were the most


effective at increasing customer awareness of the discounts. Based on findings from the general population survey, of those customers who reported purchasing LED lightbulbs and who were aware of I&M discounts, 33% learned of the discounts through bill inserts, 20% through an email, 13% from I&M website, and 13% from in-store signage.

Online lighting pilot store added. The program added an online lighting store. The store was a success and generated 5.2% of the bulb sales for the year.

A retailer training guide was developed. The program developed a guide to use in training retail staff on lighting products. The guides are provided to participating retailers.

Additional stores were added to target harder to reach customers. The program enrolled stores such as Ace Hardware, Walgreens, and Habitat for Humanity Restores, and Goodwill to target hard to reach customers. Menards and Walmart were the most common retailers where customers indicated they purchased light bulbs.



Collect and provide customer contact information for online bulb purchasers. Because the online store presents a different and more direct way to influence customer purchases, the net impacts of sales through this channel may differ from the midstream discount channel. Consequently, ADM recommends a survey of these customers to assess net savings in the future. This would require requesting customers’ contact information at the time of the online purchase.


5 Home Energy Reports This chapter presents the results of both the impact and process evaluations of the 2018 Home Energy Reports Program that Indiana Michigan Power (I&M) offered to its residential customers during the period of January 2018 through December 2018.


Assess net energy (kWh) savings and peak demand (kW) reductions resulting from participation in the program during the program year;

Characterize levels of awareness of the home energy reports, online portal, and challenge emails among participants;

Assess satisfaction with the report and information provided;




I&M contracted with implementation contractor Tendril to deliver a behavioral-based program targeting residential customers. The program is designed to generate greater awareness of energy use and ways to manage energy use through energy efficiency education in the form of home energy reports (HER). The scope of the HERs program includes informational messaging about energy use as well as recommendations for energy efficiency improvements that customers can implement in their homes.

The evaluation approach for the HER Program is aimed at determining:

The number of participants that received reports during PY2018;

Total kWh savings achieved under the program;

Total kW demand reduction achieved under the program; and

Assessment of program operations.

5.2 Estimation of Ex Post Savings

5.2.1 Methodology for Estimating Ex Post Energy Savings

To complete the evaluation of energy savings impacts, I&M and program implementation staff provided ADM with the following data:

Billing data which covers at least one year prior to the first HER (for a given wave), as well as all of 2018.

Customer lists for each customer associated with a HER Program treatment or control group, when the first HER was received, whether the customer opted out or stopped


electrical service. At least one such report should be retrieved at the end of 2018, but reports pulled at different times during the year are also desirable.

ADM verified that the treatment and control customers had similar average daily consumption during the pre-program period. Those results are reported in Table 5-1 below.

Table 5-1 Pre-Period Average Daily Consumption, by Wave

Wave Program Start Date

Controls Treatment

Number of Customers

Pre-Program Average Daily Consumption

Number of Customers

Pre-Program Average Daily Consumption

1 7/2012 - 12/2012 14,094 58.0 31,748 58.1 2 10/2013 - 12/2013 10,275 41.0 22,553 41.0 3 12/2013 - 1/2014 9,383 31.9 17,339 31.9 4 1/1/2015 6,541 39.3 12,802 39.3 5 6/1/2015 18,709 21.4 47,761 21.3 6 2/15/2017 2,738 12.6 6,671 12.4 7 11/1/2017 3,316 26.3 8,311 26.1 8 12/1/2017 4,199 29.6 10,453 29.6

Total 70,947 35.1 161,888 34.5

5.2.1.1 Regression Model for Estimating Energy Savings

Incorporating feedback from I&M and Opower (the implementation contractor prior to 2015) in 2013, ADM's approach closely followed the guidelines laid out in a document authored by the Department of Energy's State & Local Energy Efficiency Action (DOE SEE) Network that provides general recommendations on how behavioral energy-efficiency programs like the Home Energy Reports Program should be evaluated. 12

ADM conducted a regression analysis for the delivery waves using a census of program participant billing data and control group. The billing data for participants included two years of monthly observations for each customer. The pre-period data was the billing data for one year prior to treatment start period for the wave and the “post” period as 2018. At least 10 months of monthly billing must be available for both the pre and post periods for a customer to be included in the regression.

12 State and Local Energy Action Network. 2012. Evaluation, Measurement, and Verification (EM&V) of Residential Behavior-Based Energy Efficiency Programs: Issues and Recommendations. Prepared by A. Todd, E. Stuart, S. Schiller, and C. Goldman, Lawrence Berkeley National Laboratory. http://behavioranalytics.llbl.gov.


To serve as a baseline, ADM used data from a control group of randomly selected customers. This group was also screened for duplicate entries.

The mixed effects panel regression model13 used during 2018 is specified as follows:

𝑘𝑘𝑘𝑘ℎ𝑖𝑖,𝑡𝑡 = 𝛽𝛽1𝐻𝐻𝐷𝐷𝐷𝐷65𝑖𝑖,𝑡𝑡 + 𝛽𝛽2𝐶𝐶𝐷𝐷𝐷𝐷75𝑖𝑖,𝑡𝑡 + 𝛽𝛽3𝑃𝑃𝐷𝐷𝑈𝑈𝑆𝑆𝑖𝑖,𝑡𝑡 + 𝛽𝛽4�𝑃𝑃𝐷𝐷𝑈𝑈𝑆𝑆𝑖𝑖,𝑡𝑡 ∗ 𝐻𝐻𝐷𝐷𝐷𝐷65𝑡𝑡� + 𝛽𝛽5�𝑃𝑃𝐷𝐷𝑈𝑈𝑆𝑆𝑖𝑖,𝑡𝑡 ∗ 𝐶𝐶𝐷𝐷𝐷𝐷75𝑡𝑡�

+ 𝛽𝛽6�𝑃𝑃𝐷𝐷𝑈𝑈𝑆𝑆𝑖𝑖,𝑡𝑡 ∗ 𝐸𝐸𝑆𝑆𝐷𝐷𝑆𝑆𝑆𝑆𝐼𝐼𝐷𝐷𝑆𝑆𝑆𝑆𝑖𝑖� + 𝛼𝛼𝑖𝑖𝐶𝐶𝑢𝑢𝑈𝑈𝑆𝑆𝐷𝐷𝐼𝐼𝐷𝐷𝑆𝑆𝑖𝑖 + 𝜀𝜀𝑖𝑖,𝑡𝑡

Where T(i) represents the number of bills available for i. The model is defined as “mixed effects” because the model decomposes its parameters into fixed-effects (i.e. HDD65, CDD75, Post, Treat, and its various interactions) and random effects (i.e. the individual customer’s base usage). Put simply, a fixed effect is assumed to be constant and independent of the sample, while random effects are assumed to be sources of variation (other than natural measurement error) that are uncorrelated with the fixed effects. The approach is similar to others that treat the individual customer as a fixed-effect but is more computationally efficient as the number of individuals in the sample becomes very large.

While the results of this model are expected to be consistent with a pooled regression (which ignores the individual customer effect), controlling for the individual effect will achieve some improvement in the model’s fit to the data. The variables included in the regression models are specified in Table 5-2 below.

Table 5-2 Description of Variables Used in the Regression Model

Variable Description

Customer random intercept Unique identifier for each customer to control for any customer specific differences.

Heating Degree Days (HDD) Average Heating Degree Days per day within each billing period. This will be calculated by summing up the number of heating degree hours per day, and then averaging over the number of days in the billing period.

Cooling Degree Days (CDD) Average Cooling Degree Days per day within each billing period. This will be calculated by summing up the number of cooling degree hours per day, and then averaging over the number of days in the billing period.

Post Indicator if an observation is post audit (=1 if post, =0 otherwise).

kWh The average daily kWh usage for account i during billing period t.

Post * Treatment Indicator that adjusts for the interactive effect between whether customer i’s monthly billing data in period t in the pre or post period and whether customer i was in the treatment or control group during period t.

13 This was implemented in R using the lme4 package. The syntax used for model specification is lmer(avg.kw ~ 1 + hdd + cdd + post + post*hdd + post* cdd + treat*post + (1 | ACCOUNT_NUMBER), data=dataset)


The HDD and CDD was calculated on a daily basis so they can be applied to each customer’s billing period, however long that may be. It is rare that a customer’s billing dates are on the first of each month, so this ensures that no estimation of usage must occur to match weather data with the billing data.

The coefficient estimate on β6 from the regression model output was used to determine the annual Net kWh and kW savings for the program. The calculation steps are as follows:

1) Scale the daily kWh savings from the regression model up to the annual level, by multiplying by a factor of 365.

2) kW savings are calculated by applying a flat load shape (i.e. 1/8760) to the kWh energy savings.

3) Multiply by the number of program participants (including customers that closed their accounts in 2018 and with insufficient data) to arrive at a program level kWh energy savings and kW peak reduction numbers.

5.2.1.2 Estimating Net Savings

A free rider in the Home Energy Reports Program would be a customer who would have reduced energy consumption regardless of the program’s influence. The experimental design for this study excludes customers who are known to be enrolled in other energy-efficiency programs, and controls for attributes that may correlate with energy conservation via the randomization. A free rider then would have been equally likely to have been in the treatment or control groups, and hence Net-to-Gross is 1. There are no assumed free riders.

5.2.2 Results of Ex Post Savings Estimation

The estimated energy savings impacts are summarized in Table 5-3 and the estimated peak demand impacts are summarized in Table 5-4 resulting from the PY2018 Home Energy Reporting program.



Gross Audited

kWh Savings

Gross Verified

kWh Savings

Ex Post Gross kWh

Savings

Gross Realization

Rate


Net-to-

Gross Ratio

Ex Post Net Lifetime kWh

Savings

23,449,092 23,449,092 23,449,092 24,419,945 104% 24,419,945 100% 24,419,945


Ex Ante Gross

kW Savings

Gross Audited

kW Savings

Gross Verified

kW Savings

Ex Post Gross

kW Savings

Gross Realization

Rate


Net-to-Gross Ratio

2,673,20 2,673,20 2,673,20 2,787.66 104% 2,787.66 100%


Differences in data cleaning steps, as well as a variety of factors related to modeling details will contribute to the variance between ex ante estimates and ex post verified savings. Below is a summary of a data cleaning steps employed by ADM in preparing the model dataset utilized by ADM to determine ex post kWh savings:

Filtering – ADM employed consumption and billing duration filters for outliers.

CDD Base - ADM used a CDD base of 75°F

HDD Base – ADM used a HDD base of 65°F

In addition to likely differences in data preparation and model specification, ex ante estimated kWh savings were developed on a monthly basis while ex post verified kWh savings were determined on an annual basis.

5.2.2.1 Ex Post kWh Savings

The impacts were calculated from the regression outputs for each wave. The regression results are reported in Table 5-5 below. Daily kWh savings were equal to 0 for wave 6 because the treatment effect parameter for that wave was not statistically significant.

Table 5-5 HER Regression Model Output, by Wave

Statistic Wave 1

Wave 2

Wave 3

Wave 4

Wave 5

Wave 6

Wave 7

Wave 8

Daily kWh Savings 0.83 0.55 0.38 0.47 0.16 0.00 0.22 0.19 Number of Treatment Customers 31,748 22,553 17,339 12,802 47,761 6,671 8,311 10,453 Number of Control Customers 14,094 10,275 9,383 6,541 18,709 2,738 3,316 4,199 R-Squared 0.42 0.40 0.38 0.53 0.66 0.70 0.63 0.62

The annualized kWh savings were calculated from the daily kWh savings estimate reported above. The kWh savings by wave are reported in Table 5-6 below.

Table 5-6 kWh Savings, by Wave14

Wave Number of Treatment Customers

Daily kWh Savings per Customer

Annual kWh

Savings per Customer

Ex Post Gross kWh

Savings

1 35,043 0.83 305 10,678,620 2 22,782 0.55 200 4,531,898 3 17,534 0.38 140 2,446,757 4 13,232 0.47 171 2,257,754 5 51,891 0.16 60 3,095,315 6 7,218 0.00 0 0 7 8,459 0.22 81 685,024

14 Ex post gross kWh savings presented in this table account for the incremental impact of program uplift.




Annual kWh


Ex Post Gross kWh

Savings

8 10,710 0.19 68 724,577 Total 166,869 0.40 146 24,419,945

Wave 6 has the smallest treatment group size and lowest pre-program average daily consumption of the program waves. Energy savings were not identified through analysis of wave 6.

5.2.2.2 Ex Post kW Reductions

The annualized kW Peak Reductions were calculated by dividing the annualized kWh savings by 8,760. The results by wave are reported in Table 5-7 below.

Table 5-7 kW Peak Reduction, by Wave


Ex Post Gross kW

Peak Reduction

1 35,043 1,219.02 2 22,782 517.34 3 17,534 279.31 4 13,232 257.73 5 51,891 353.35 6 7,218 0.00 7 8,459 78.20 8 10,710 82.71

Total 166,869 2,787.66


ADM completed a limited process evaluation of the PY2018 program. The limited process evaluation documented notable program changes and developments, findings from a survey of control and treatment group customers.

Data was collected from multiple data sources to support the process evaluation of the program. The data collection activities and the expected sample sizes are presented in Table 5-8.


Table 5-8 Summary of Data Collection Activities for the Home Energy Reports Program

Data Collection Activity Mode of Data Collection Sample Size Treatment group survey Online 203 Control group survey Online 6 Staff interviews Telephone in-depth interviews 2 interviews, 3 respondents



This section summarizes the findings of interviews conducted with I&M and Tendril program staff for the purposes of learning about any changes to the program structure, identifying program objectives, and assessing the extent to which there are future opportunities for program improvement. ADM evaluators interviewed the I&M energy efficiency consumer program coordinator and a project manager and decision science manager from Tendril to gain insights in the Home Energy Reports (HERs) programs for 2018.

The findings of the interviews are discussed below.

The program increased consistency of program materials. Tendril and I&M collaborated to better align digital products and email templates with the I&M brand. Customers now see more consistency across the multiple products (e.g., the report, challenge emails, high bill alerts). The change was rolled out in waves, with the first change to the challenge, then to report, and last to high bill alerts. The goal was to improve the user experience and ensure the customer recognizes the communication is from I&M. Tendril is interested in the feedback from this change.

Indiana customers, who were not in either the treatment or control group, could previously access the portal and high use alerts. While it does not apply to the treatment or control group, the contract was closed out in 2018 so those customers can no longer access the portal or receive a high use alert.

Home energy reports were modified to increase impacts. Tendril conducted usability testing and provided four recommendations from those results. I&M implemented 4 of the 5 recommendations provided by Tendril to align with customers’ preferences. The program changed “home” to “household” because it is more associated with behaviors. Added a caption to the electric use over time chart. The HERs no longer use “energy efficient,” as customers preferred energy savings. Additional changes may be made in 2019 as Tendril will be running a pilot on the Michigan HERs in February 2019. Program staff will be conducting AB testing on the home comparison content. Staff is interested in understanding how this may impact overall savings at the end of the year. Additionally, a customer satisfaction survey designed to get feedback on the template was administered in 2018 and will continue in 2019. The redesigned 2018 report is shown in Figure 5-1.

Additional control provided to customers over the content they received from I&M. I&M launched a new IT Enhancement, which allows customers to control the level of


communication they receive from the Home Energy Report Program. The customer can opt-in or opt-out of paper or email HERs, Challenge Emails, and High Usage Alerts. Despite this change, the opt-out rates remained consistent in 2018 from 2017. Tendril provided ADM evaluators a copy of the Service Level Agreement Report for additional information about engagement rates. According to the report, the December open rates for eHERs was about 41% and the click rate was less than 4%.

The QA/QC process was modified. There have been a few changes to quality control/quality assurances procedures. Tendril has built more automated quality control checks and have added personnel to the operations. There was more of a focus on QC/QA in 2018 to streamline processes and procedures. Under the process, there is a QC checklist (some manual and some automated) that is completed prior to sending a HER. There are over 100 QC checks for a sample set of reports. About 100 to 150 reports are examined manually and the rest are examined through an automated process.


Figure 5-1: 2018 Home Energy Report with New Template



ADM completed a telephone survey of program participants (report recipients) to collect information regarding their experience with the Home Energy Reports Program. A total of 203 customers in the treatment group (i.e., received home energy reports) and 6 customers in the control group completed the survey. Due to the low response rate in the control group, the summary of findings is omitted from the following section.

ADM also invited customers in the Low Income Home Energy Reports Program. The survey was sent to 59 customers in the treatment group with email addresses available. Of these customers, two completed the survey. Because of the low response rate, the survey responses were combined with the responses from customers in the standard Home Energy Reports program treatment group.

Receipt of Reports

As shown in Table 5-9, 51% of respondents reported that they had read all or most of the reports they received, and 45% stated they had read some (between 25 and 75%) of the reports. A minority, 5%, reported that they had not read any of them.

Table 5-9 Frequency of Reading Reports


You have read all of them 51% You have read about 75% of them 24% You have read about 50% of them 11% You have read about 25% of them 10% You have not read any of them 5%

Ninety-eight (98%) percent of survey respondents in the treatment group recalled seeing the information on their home’s electricity usage compared to similar homes. Among those who recalled seeing the home comparisons, 59% indicated it was about what they would have expected and 41% indicated it was different than what they would have expected (see Table 5-10).

Table 5-10 Home Comparison and Expectations


About what you expected 59% Different than what you expected 41%

As shown in Table 5-11 and Table 5-12, a clear majority of participants found the reports easy to understand (90% rated it as a 4 or 5) and accurate (66% rated it as a four or five).


Table 5-11 Difficulty in Understanding the Information


1 (Very difficult) 0% 2 1% 3 9% 4 24% 5 (Very easy) 67%

Table 5-12 Accuracy of Information


1 (Not at all accurate) 6% 2 12% 3 20% 4 38% 5 (Very accurate) 24%

Among those survey respondents who did not find the information on their home energy report accurate, they were provided feedback as to why they believed it was inaccurate. Table 5-13 summarizes and categorizes their responses.

Table 5-13 Why Customers Believe the Information on the HER is Inaccurate Type of

Comment n Comments Made

Home comparison issues or inaccuracies with home specifications

9

Because it shows the sizes of houses, they are comparing mine to much larger than my house is.

Obviously, the most efficient homes have gas water heaters and kitchen ranges, something you don't take into account. You should only compare my home to similar homes with electric hot water and ranges.

It was comparing my house to a larger one.

There is no way we are so much higher than similar houses...no way

The square footage of my home is smaller than the homes in my group.

Size of my house much smaller than what they have listed=2400 sq.ft. actually only 1480.

The facts about our house are wrong. I've tried to correct them in the past. I don't think it counts the fact that we have electric water heaters and not gas.

It is based on the size of the home not the number of or age of the occupants

It does not take into account our out building that has electrical and heating usage.

Above average electricity usage

3

Because my husband has a shop that he works in at least 8 hours a day 5 days a week. A lot of electricity is used in it. The same meter is used for both the shop and house.

Because of the electricity that I have to use.

Does not take into consideration true usage. Other factors are not interpreted in the figures, like outside lighting on motion sensors, cordless tool battery chargers in garage, electric hedge trimmers, chain saw, and other electric tools used often at differing times of the year. Consistent


Type of Comment n Comments Made

use of air compressor, a low heat source in shed to help start lawn tractor, snow removal equipment, and occasional use of a 220V Lincoln stick welder.

Increased bill amounts or other issues with bill

4

Because ever since they installed that Smart thermostat my bills have increased 25%

It says my usage was lower, but my bills kept increasing.

My bill seems high in comparison to my neighbors with similar homes and appliances

It's WAY more than what it used to be, however, I keep the house SIGNIFICANTLY cooler now, so it should be WAY lower

All-electric home 3

Our home is all electric - of course our usage is more than a home that has gas heat. Don't think/know if the comparisons are valid, or at least we don't know the heat systems, # of occupants, size of home and other factors of compared-to homes to feel confident that the comparisons are valid at all. Our thermostat has been set @ 62 deg all winter and don't appreciate being shown how excessive our energy use is compared to other homes.

I live in an apartment that is old and that is all electric. The upkeep from the apartment company is horrible.

We have an all-electric home so our usage will obviously be higher.

Already energy efficient

4

I live by myself and try very hard to save electricity.

I built my house and know how it is insulated. I installed a new furnace last spring.

I have done most of the recommendations. Also have a heat pump/ furnace less than 5 years old.

Because we have insulated whole house, sealed all windows when we moved in in 2013 first gas heating bill was over $400 a month now gas bill is $51 a month. All appliances replaced with energy efficient, but we have a hot tub, but it is inside.

Other 4

I live in a rented property where insulation and wall structure needs work. Also, when the neighbor has their window open in winter (yes, winter) I feel a draft on my side. In the summer their window air conditioner is right next to my open windows.

I do not live on site. My farm is located on the property and I only have animals there. Most of the information is totally inaccurate.

Because a lot of times just guess what the meter is reading.

Are medium use electric appliances considered when comparing households such as electric dryers, ranges, hot water heaters along with number of adults residing in the household?

Not sure I just wish there were more ideas on how to test items in my home for energy use and ideas on how to better help conserve energy.

Most survey respondents (93%) recalled receiving recommendations in their HER on ways to save energy in their home. More than half of survey respondents in the treatment group believed the recommendations were very or somewhat useful, while 24% did not find them useful (Table 5-14).


Table 5-14 Usefulness of Recommendations


1 (Not at all useful) 11% 2 13% 3 23% 4 38% 5 (Very useful) 16%

Among those respondents that did not find the recommendations useful, 50% indicated they were not relevant or applicable to their home and 32% indicated they were not practical (see Table 5-15).

Table 5-15 Why Customers did not find Recommendations Useful


They were not relevant or applicable to my home 50% They were not practical – could not take the recommended actions 32% Not interested in saving energy 5% Other 42%

Use of WattsUp Energy Portal

Seventeen percent of respondents (n = 35) reported that they accessed the WattsUp Energy Portal. The most commonly mentioned reason for not accessing the portal was a lack of awareness of it (63%). The remainder of the respondents gave reasons such not believing the portal would provide useful information (10%), and not having time to access it (9%). Table 5-16 summarizes the full distribution of responses.

Table 5-16 Reasons for Not Accessing the Portal


Was not aware of the portal 63% Did not think the portal would provide useful information 10% Did not have the time to use the portal 9% Did not know how to access the portal 5% Did not know how to use the portal 5% Not interested in saving energy right now 2% Do not use or have internet/computer 1% Other 6%

Figure 5-2 summarizes the assessments of the portal by those who accessed it. As shown, most respondents provided positive feedback and indicated that the information was easy to understand, the portal was easy to navigate, the website was visually appealing, the set-up process was easy, and that the information about the portal was easy to understand. Two respondents indicated that they did not think the website was easy to navigate and that the information was easy to understand.


Figure 5-2 Assessment of the WattsUp Energy Portal

Sixty-eight (68%) percent of respondents reported that they had received the challenge emails (Table 5-17).

Table 5-17 Receipt of Challenge Emails


Yes 68% No 33%

Forty-nine percent reported that they had taken one or more of the challenges and 3% had read and taken all the challenges. Most survey respondents (75%) indicated they typically understand the challenges, followed by 22% who understand some but not all challenges, and 3% who typically do not understand the challenges.

Table 5-18 Response to Challenge Emails


I read them but have never taken the challenge 43% I read them and have taken at least one challenge 49% I read them and have taken all challenges 3% I do not read them 6%


Energy Efficiency Knowledge and Attitudes

Table 5-19 and Table 5-20 summarize survey participants’ self-reported ratings of their knowledge of ways to save energy and their efforts made to save energy. As shown, most customers receiving the home energy reports provided high ratings of their knowledge and their efforts.

Table 5-19 Knowledge of Ways to Save Energy

How knowledgeable are you about ways to save energy in

your home?

Received Reports

(n = 198) 1-3 (Low) 4% 4-7 (Moderate) 25% 8-10 (High) 72% Average 7.6

Table 5-20 Effort to Save Energy

How would you rate your household's efforts to save

energy in your home?

Received Reports

(n = 200) 1-3 (Low) 5% 4-7 (Moderate) 31% 8-10 (High) 65% Average 7.2

Energy Efficiency Behaviors

Many respondents reported taking actions to save energy. As shown in Table 5-21, turning off lights when not in use was the most frequently reported action (93% in the treatment group mentioned doing this) followed by adjusting thermostats (78%), and using cold water when doing laundry more often (70%).

Forty-eight survey respondents indicated they took other actions not mentioned to save energy. Eighteen respondents indicated they purchased new equipment (HVAC, new appliances, lighting), followed by nine who performed some weatherization upgrades, six who added insulation, four who reduced their heating usage, and two who recycled their older equipment.


Table 5-21 Actions Taken to Save Energy

Action

Count who Took

Action

Percent of Respondents

who took Action

Turned off lights when not in use more often 196 93% Adjusted thermostat settings 199 78% Used cold water when doing laundry more often 195 70% Closed or covered windows to reduce heat gain or loss 202 68% Unplug electric equipment or appliances when not in use 200 61% Added weather stripping or caulking 198 39% Turned down water heater temperature 197 21%

Most participants (69%) reported that they were aware of I&M’s rebate and discount programs. As shown in Table 5-22, appliance rebates was the program that respondents most often reported knowing about (76%).

Table 5-22 Awareness of Efficiency Programs

Which programs have you heard of? Received Reports

(n = 132) Appliance rebates 76% Appliance recycling/disposal 75% Thermostat rebates 61% Lighting discounts through participating retailers 31% Rebates for making weatherization improvements like air sealing and adding insulation 28% Another program 4%

Figure 5-3 summarizes how respondents learned of the programs. Bill inserts and mailings from I&M, along with Home energy reports, were the source of awareness most often mentioned (cited by 54% and 52%, respectively), followed by I&M newsletters (31%), and the I&M website (21%).


Figure 5-3 Source of Awareness

Satisfaction

As shown in Figure 5-4, participants were generally satisfied with their home energy reports, challenge emails and contests, and the WattsUp Energy portal. The challenge emails and contests were the aspect of the program that the largest share of respondents reported dissatisfaction with (17% provided a rating of 1 or 2 indicating a measure of dissatisfaction).

Figure 5-4 Satisfaction with Home Energy Report Program Components


Approximately 80% of survey respondents rated their satisfaction with I&M as their electricity provider as very or somewhat satisfied on a 5-point scale.

Table 5-23 Satisfaction with I&M as their Electricity Provider

Response Received Home Energy Reports

(n = 201) Very dissatisfied 2% Somewhat dissatisfied 3% Neither 14% Somewhat satisfied 34% Very satisfied 46%

Yearly Trends


There was a 13 percentage-point decrease in those who indicated the information on their home energy usage was somewhat or very accurate by (see Figure 5-5).

Figure 5-5 2017 and 2018 Accuracy of Home Energy Usage Comparison

There was 13 percentage-points increase among those who indicated the recommendations included in the home energy reports were either somewhat or very useful by 13 percentage points (see Figure 5-6). Among survey respondents who had not visited the WattsUp Energy Portal, there was a 35 percentage-point increase among those who stated they were not aware there was a portal.


Figure 5-6 2017 and 2018 Usefulness of Home Energy Report’s Recommendations Comparison

Among those who recalled receiving the Energy Challenge emails from I&M, there was a 26 percentage-point increase among respondents who read them and have taken at least one challenge.

There was a six percentage-point increase among those who indicated they were aware that I&M offers rebates and incentives. The program that saw the greatest increase in awareness was the appliance recycling program (38 percentage-point increase), followed by a 17 point increase for appliance rebates, and 12 point increase for thermostat rebates.

There was an eleven percentage-point decrease among survey respondents who were somewhat or very satisfied with the home energy report (see Figure 5-7).

Figure 5-7 2017 and 2018 Home Energy Report Satisfaction Comparison


5.4.1 Conclusions

Efforts were taken to increase program savings impacts. The program modified the home energy reports to increase impact. These changes included language changes (i.e., referring to household instead of home and referring to energy savings instead of energy


efficiency), and adding a caption to chart of electricity use over time. Additionally, efforts were made to make the appearance of the report, the energy challenge emails, and the high bill alerts more similar so that customers recognized each as a communication from I&M.

Additional control over I&M content did not adversely impact opt-out rates. A enhancements allows customers to opt-in or opt-out of all the possible Home Energy Report communications to give customers more control over the content they receive from I&M. Staff reported that that opt-out rates for the HERs remained largely unchanged.

Most customers report frequently reading the home energy reports. Fifty-one percent of customers stated that they read all the reports and another 24% reported that they had read at least 75% of the reports.

Most customers find the information easy to understand. Ninety-one percent of customers reported that the information in the report was very or somewhat easy to understand.

Many customers reported that the information on their home was accurate, but some noted inaccuracies. Sixty-two percent reported that the information on energy usage was very or somewhat accurate. However, 18% rated it as somewhat or very inaccurate. Common inaccuracies noted was the information on the home size and a lack of information on the type of fuel their heating system or other appliance used. Additionally, 54% of customers found the recommendations to be somewhat or very useful.

Satisfaction with the reports is generally high. Sixty-nine percent of the treatment group reported that they were very or somewhat satisfied with the home energy reports and 72% were very or somewhat satisfied with WattsUp Energy portal. Satisfaction with the challenge emails and contests was somewhat lower, but this was largely due to larger shares of customers who were not particularly satisfied or dissatisfied rather than higher levels of dissatisfaction.



Use focus groups and/or A/B testing to develop and test customized marketing and messaging approaches. I&M expressed a desire to increase customized marketing and targeting messaging to customers. Focus groups can be an effective means to developing messaging and identifying programs that fit customer market segments. A/B testing can be used to assess the effectiveness of these approaches by monitoring open and click through rates.

Consider adding a mechanism to allow customers to correct information they believe to be inaccurate or to provide additional information on their home characteristics. Some customers cited inaccuracies presented in the home energy report and customers were less likely to act on information that they do not believe is credible. Consider developing a portal or other approach that would allow customers to modify or provide additional information on their home to improve the accuracy of the customer information.


6 Low Income Home Energy Reports This chapter presents the results of both the impact and process evaluations of the 2018 Low Income Home Energy Reports Program that Indiana Michigan Power (I&M) offered to its residential customers during the period of January 2018 through December 2018.


Assess net energy (kWh) savings and peak demand (kW) reductions resulting from participation in the program during the program year;

Characterize levels of awareness of the home energy reports, online portal, and challenge emails among participants;

Determine if there are significant differences between and participant and control groups in energy efficiency knowledge, attitudes, and actions taken to save energy;

Assess satisfaction with the report and information provided;




I&M contracted with implementation contractor Tendril to deliver a behavioral-based program targeting low-income residential customers. The program uses data on customers’ receipt of energy assistance and third-party data to identify likely low-income customers to target. The program is designed to generate greater awareness of energy use and ways to manage energy use through energy efficiency education in the form of home energy reports (HER). The scope of the HERs program includes informational messaging about energy use as well as recommendations for energy efficiency improvements that customers can implement in their homes.

The evaluation approach for the HER Program is aimed at determining:

The number of participants that received reports during PY2018;

Total kWh savings achieved under the program;

Total kW demand reduction achieved under the program; and

Assessment of program operations.

6.2 Estimation of Ex Post Savings


The methodology used to estimate ex post energy savings for the Low Income Home Energy Reports Program was the same as the approach used for the Home Energy Reports Program outlined in section 5.2 on page 91.


The count of customers and the average pre and post consumption results are reported in Table 6-1 below.

Table 6-1 Pre-Period Average Daily Consumption, by Wave

Controls Treatment

Number of Customers

Pre-Program Average

Daily Consumption

Number of Customers

Pre-Program Average

Daily Consumption

1,692 30.4 4,250 30.5

6.2.2 Results of Ex Post Savings Estimation

The estimated energy savings impacts are summarized in Table 6-2 and the estimated peak demand impacts are summarized in Table 6-3 resulting from the PY2018 Low Income Home Energy Reporting program.


Ex Ante Gross kWh

Savings

Gross Audited

kWh Savings

Gross Verified

kWh Savings

Ex Post Gross kWh

Savings

Gross Realization

Rate

Ex Post Net kWh

Savings

Net-to-Gross Ratio


kWh Savings

329,697 329,697 329,697 292,738 89% 292,738 100% 292,738






Gross Realization

Rate


Net-to-Gross Ratio

37.59 37.59 37.59 33.42 89% 33.42 100%

Differences in data cleaning steps, as well as a variety of factors related to modeling details will contribute to the variance between ex ante estimates and ex post verified savings. Below is a summary of a data cleaning steps employed by ADM in preparing the model dataset utilized by ADM to determine ex post kWh savings:

Filtering – ADM employed consumption and billing duration filters for outliers.

CDD Base - ADM used a CDD base of 75°F

HDD Base – ADM used a HDD base of 65°F

In addition to likely differences in data preparation and model specification, ex ante estimated kWh savings were developed on a monthly basis while ex post verified kWh savings were determined on an annual basis.


6.2.2.1 Ex Post kWh Savings

The impacts were calculated from the regression outputs for each wave. The regression results are reported in Table 6-4 below.

Table 6-4 HER Regression Model Output

Statistic Low Income

Daily kWh Savings 0.19 Number of Treatment Customers 4,250 Number of Control Customers 1,692 R-Squared 0.62

The annualized kWh savings was calculated from the daily kWh savings estimate reported above. The kWh savings are reported in Table 6-5 below.15

Table 6-5 kWh Savings

Number of Treatment Customers


Annual kWh


Ex Post Gross kWh

Savings

4,369 0.19 68 292,738

6.2.2.2 Ex Post kW Reductions

The annualized kW Peak Reductions were calculated by dividing the annualized kWh savings by 8,760. The results are reported in Table 6-6 below.

Table 6-6 kW Peak Reduction

Number of Treatment Customers

Ex Post Gross kW

Peak Reduction

Ex Post Net kW Peak

Reduction

4,369 33.42 33.42



Data was collected from multiple data sources to support the process evaluation of the program. The data collection activities and the expected sample sizes are presented in Table 6-7.

15 Ex post gross kWh savings presented in this table account for the incremental impact of program uplift.


Table 6-7 Summary of Data Collection Activities for the Home Energy Reports Program

Data Collection Activity Mode of Data Collection Sample Size Participant survey Online 2 Staff interviews Telephone in-depth interviews 2

Due the small number of customer survey responses obtained, the results were combined with the Home Energy Reports results and are presented in section 5.3.1.2 on page 100.



The low-income program is similar in design to the mass-market HER program, but messaging is specific to low-income customers. The Low Income HER program runs and is operated much like mass-market program. The implementation contractor allows for different messaging in the low-income report that is targeted towards lower income customers (i.e., the reports do not recommend expensive retrofits). However, staff noted that the messaging was not varied for low-income customer reports in PY2018.

To identify low-income customers, I&M provided Tendril with account records of customers that receive energy assistance. Tendril used this data, along with data purchased from Experian, to develop the population of likely low-income customers.

The QA/QC process was modified. There have been a few changes to quality control/quality assurances procedures that apply to the Low Income program as well as the mass-market program. Tendril has built more automated quality control checks and have added personnel to the operations. There was more of a focus on QC/QA in 2018 to streamline processes and procedures. Under the process, there is a QC checklist (some manual and some automated) that is completed prior to sending a HER. There are over 100 QC checks for a sample set of reports. About 100 to 150 reports are examined manually and the rest are examined at through an automated process.


Due the small number of customer survey responses obtained (n = 2), the results were combined with the Home Energy Reports results and are presented in section 5.3.1.2 on p. 100.


6.4.1 Conclusions

The low-income program is similar in design to the mass market HER program. In PY2018, the messaging and design of the reports was the same for the low-income component as for the mass market HER program.

The QA/QC process was modified. There have been a few changes to quality control/quality assurances procedures that apply to the Low Income program as well as the


mass-market program. Tendril has built more automated quality control checks and have added personnel to the operations. There was more of a focus on QC/QA in 2018 to streamline processes and procedures. Under the process, there is a QC checklist (some manual and some automated) that is completed prior to sending a HER. There are over 100 QC checks for a sample set of reports. About 100 to 150 reports are examined manually and the rest are examined at through an automated process.


ADM does not have any recommendations for the Low Income Home Energy Reports Program.


7 Usage Alerts This chapter presents the results of the impact evaluation of the 2018 Usage Alerts Program that Indiana Michigan Power (I&M) offered to its residential customers during the period of January 1, 2018 through July 31, 2018. Evaluable PY2018 activity for this program ended on July 31, 2018.

The objective of the evaluation was to assess net energy (kWh) savings and peak demand (kW) reductions resulting from participation in the program during the program year.


The Usage Alerts Program provided customers with high bill alerts sent to customers in the heating and cooling seasons, notifying them that their usage is increasing due to weather changes. Eligible customers had an email address, have a current bill, and meet the requirements of 30% more usage and a $20 higher bill.

The Usage Alerts Program also provided customers with access to the online energy portal, available through I&M’s bill pay website. The portal provided customers with a summary of the energy use over time, suggestions for reducing energy use, and challenges to engage customers in energy use topics.

7.2 Estimation of Ex Post Energy Savings


Usage Alerts Program impacts were determined through comparison of a treatment group comprised of recipients of high bill alerts and email challenges, and a control group comprised of customers that did not receive such program communications. All I&M Indiana customers with known email addresses, except those part of a Home Energy Reports Program control group, were eligible to receive high bill alerts and email challenges. ADM constructed a Usage Alerts Program control group from Indiana accounts part of Home Energy Reports Program control groups.16

7.2.1.1 Calendarization of Consumption Data

ADM calendarized consumption data for each billing period by performing the following steps:

1. An average daily consumption value was determined by dividing the consumption by the number of days during the billing period.

2. If the billing record spanned more than one month, then the consumption will be split between the months occurring during the billing period. a. For example:

i. Reading Date: 01/23/2018

16 See section 5.2.1 on page 113 for information on the Home Energy Reports Program impact evaluation method.


ii. Reading Duration: 33 iii. Consumption: 613

b. Since the reading duration is 33 days, the start date of the bill is 12/21/2017. This bill period contains days during both January 2018 and December 2017: 23 days in January and 10 days in December.

c. Attribute 23/33 or about 70% of the consumption to January and 10/33 or about 30% of the consumption to December.

d. For January 2018, the consumption was about 427.24 kWh. For December 2017, the consumption was about 185.76 kWh.

3. This process was implemented for all billing records. 4. After all consumption is associated with a month, all consumption data for each month for

each account is summed to determine monthly account-level consumption.

7.2.1.2 Treatment and Control Treatment Group

To account for cross-participation between the Usage Alerts Program and the Home Energy Reports Program, regression analysis was only performed using accounts that were not part of Home Energy Report Program treatment groups. For accounts in the treatment group, the “start date” of program intervention was the date for which the account received its first email alert.

The program did not have a randomized control trial (RCT) design. Therefore, a control group was constructed to use as a counterfactual baseline against which energy savings were measured. The pool of accounts used for the selection of the Usage Alerts control group is the I&M Home Energy Report Indiana control group. Since the program was operational between January 2018 and July 2018, the January 2017 – July 2017 period was used as the baseline period to match on and was incorporated into the regression analysis.

The matching methodology used was a combination of propensity score matching and Mahalanobis Distance matching.17 The Mahalanobis distance is the distance between two points in multivariate space. Matching was based upon three variables: summer pre-treatment consumption, winter pre-treatment consumption, and the propensity score based on pre-treatment consumption. The matching was done with no replacement.

The following equation show the specifications for Mahalanobis Distance matching.

𝑆𝑆𝑆𝑆𝐷𝐷𝑆𝑆𝑆𝑆𝐼𝐼𝐷𝐷𝑆𝑆𝑆𝑆 ~ summerkWh + winterkWh + 𝑖𝑖𝑆𝑆𝐷𝐷𝑖𝑖𝐷𝐷𝑆𝑆𝑈𝑈𝐷𝐷𝑆𝑆𝑏𝑏𝑆𝑆𝐼𝐼𝐷𝐷𝑆𝑆𝐷𝐷

The following equation show the specification for determining the propensity score used in the Mahalanobis Distance matching.

𝑆𝑆𝑆𝑆𝐷𝐷𝑆𝑆𝑆𝑆𝐼𝐼𝐷𝐷𝑆𝑆𝑆𝑆 ~ summerkWh + winterkWh

17 This method can be found in Propensity Scores: A Practical Introduction Using R by Antonio Olmos and Priyalatha Govindasamy on page 72-74.


Once a match is made between the treatment and control group, the start date for the treatment account is used as the start date for the corresponding control account.

Table 7-1 presents results of a t-test of the treatment and control group baseline period energy consumption. The relatively high p-value may indicate goodness of match between the control and treatment group.

Table 7-1 T-Test of Treatment and Control Group Baseline Period Energy Consumption Type Value

P-Value 0.54 90% Confidence Interval [-0.25, 0.11] Mean of Treatment 31.16 Mean of Control 31.23

7.2.1.3 Econometric Model

ADM used the following econometric model to estimate Usage Alerts Program energy savings:

𝑘𝑘𝑘𝑘ℎ𝑖𝑖,𝑡𝑡 = 𝛽𝛽1𝐻𝐻𝐷𝐷𝐷𝐷65𝑖𝑖,𝑡𝑡 + 𝛽𝛽2𝐶𝐶𝐷𝐷𝐷𝐷75𝑖𝑖,𝑡𝑡 + 𝛽𝛽3𝑃𝑃𝐷𝐷𝑈𝑈𝑆𝑆𝑖𝑖,𝑡𝑡 + 𝛽𝛽4�𝑃𝑃𝐷𝐷𝑈𝑈𝑆𝑆𝑖𝑖,𝑡𝑡 ∗ 𝐻𝐻𝐷𝐷𝐷𝐷65𝑡𝑡�

+ 𝛽𝛽5�𝑃𝑃𝐷𝐷𝑈𝑈𝑆𝑆𝑖𝑖,𝑡𝑡 ∗ 𝐶𝐶𝐷𝐷𝐷𝐷75𝑡𝑡� + 𝛽𝛽6�𝑃𝑃𝐷𝐷𝑈𝑈𝑆𝑆𝑖𝑖,𝑡𝑡 ∗ 𝐸𝐸𝑆𝑆𝐷𝐷𝑆𝑆𝑆𝑆𝐼𝐼𝐷𝐷𝑆𝑆𝑆𝑆𝑖𝑖� + 𝛼𝛼𝑖𝑖𝐶𝐶𝑢𝑢𝑈𝑈𝑆𝑆𝐷𝐷𝐼𝐼𝐷𝐷𝑆𝑆𝑖𝑖 + 𝜀𝜀𝑖𝑖,𝑡𝑡

7.2.1.4 Peak Savings

Usage Alerts peak kW savings are calculated as the average savings occurring during the I&M 5CP hours referenced in Table 7-2.

Table 7-2 PY2018 I&M 5CP Hours Date Hour Start Hour End

5/29/2018 1:00 PM 2:00 PM 6/18/2018 2:00 PM 3:00 PM 7/10/2018 12:00 PM 1:00 PM 8/28/2018 3:00 PM 4:00 PM 9/4/2018 4:00 PM 5:00 PM

7.2.2 Results of Ex Post Energy Savings Estimation

Energy savings attributed to the Usage Alerts Program were generated only by customers who did not participate in the Home Energy Reports Program.

Table 7-3 summarizes the ex post gross and net kWh savings of the Usage Alerts Program. The annual net savings totaled 449,852 kWh.


Table 7-3 Ex Post kWh Savings

Ex Ante Gross kWh

Savings

Gross Audited

kWh Savings

Gross Verified

kWh Savings

Ex Post Gross kWh

Savings

Gross Realization

Rate

Ex Post Net kWh

Savings

Net-to-Gross Ratio


kWh Savings

779,957 779,957 779,957 449,852 58% 449,852 100% 449,852


The average Usage Alerts peak kW reduction was calculated during the I&M 5CP periods and reported in Table 7-4.

Table 7-4 Ex Post Gross Peak kW Reduction –Usage Alerts

Date Hour Start Hour End kW Reduction

5/29/2018 1:00 PM 2:00 PM 88.41 6/18/2018 2:00 PM 3:00 PM 88.41 7/10/2018 12:00 PM 1:00 PM 88.41 8/28/2018 3:00 PM 4:00 PM - 9/4/2018 4:00 PM 5:00 PM -

Maximum Peak kW Reduction 88.41

Average Peak kW Reduction 88.41

Table 7-5 summarizes the ex post gross and net kW demand reduction of the Usage Alerts Program. The net demand reduction equaled 88.41 kW.



Gross Audited kW

Savings

Gross Verified kW

Savings


Gross Realization

Rate


Net-to-Gross Ratio

88.92 88.92 88.92 88.41 99% 88.41 100%


No process evaluation was performed for the PY2018 Usage Alerts Program.


8 Residential Online Energy Check-up This chapter presents the results of both the impact and process evaluations of the 2018 Residential Online Energy Check-up Program that Indiana Michigan Power (I&M) offered to its residential customers during the period of January 2018 through December 2018.



Document sources of program awareness and marketing activities;



Assess satisfaction among participants; and



The Residential Online Energy Check-up Program (OEC) identified energy saving opportunities through a web-based self-service assessment tool where customers answer basic questions about their homes and how they use energy. Upon completion of the questions online, the program generates a printable report that includes:

Useful details about energy consumption of the customer’s home;

Customized energy-saving recommendations;

Potential savings from making the suggested improvements; and

Environmental impact of implementing suggested improvements.

In addition, the customer is mailed a kit of low-cost energy efficiency measures dependent on their water heater type. Kits are limited to one per account every three years.

Energy efficient kits for participants with gas water heaters included:

Three (3) 9W LEDs;

Two (2) .5W LED night lights; and

One (1) Digital thermometer.

Energy efficient kits for participants with electric water heaters included:

Three (3) 9W LEDs;

One (1) 1.5 GPM Kitchen faucet aerator;

Two (2) 1.0 GPM Bathroom faucet aerators;


Two (2) 1.5 GPM High-efficiency showerheads; and

One (1) Digital thermometer.

Tendril provides the online audit tool and the kits are distributed by Resource Action Programs.


The following section presents the methodology that was used for estimating the gross energy and demand impacts resulting from the Residential Online Energy Check-up Program in 2018.


The M&V approach for the impact evaluation of the Online Energy Check-up Program focused on determining the following:

Numbers of kits distributed;

Percent of kit components installed; and

Average annual energy (kWh) savings and demand (kW) reduction per kit measure.


ADM used simple random sampling to develop a sample of program participants to be surveyed as part of the impact evaluation effort. The sample size for these participant surveys is calculated to meet 90% confidence and 10% precision (90/10). The sample size to meet 90/10 requirements is calculated based on the coefficient of variation of savings for program participants. The coefficient of variation (CV) is defined as:



Where x is the average kWh energy savings per participant. Without data to use as a basis for a higher value, it is typical to apply a CV of .5 in residential program evaluations. The resulting sample size is estimated at:

𝑆𝑆0 = �1.645 ∗ 𝐶𝐶𝐶𝐶

𝐺𝐺𝑃𝑃�

2

Where,




With 10% required precision (RP), this calls for a sample of 68 participants. ADM exceeded this sampling target with a sample of 221 participants.



As a first step, ADM reviewed data tracking systems associated with the program to ensure that the data provided sufficient information to identify unique customers for surveying and to calculate energy and demand impacts in accordance with the 2015 Indiana Technical Reference Manual (TRM) Version 2.2. ADM further reviewed the program data to verify that the fields required for performing the evaluation were tracked and populated (i.e., the data are not missing) and that the values were reasonable. ADM took several steps in its verification efforts, which consisted of the following:

Validating program tracking data by checking for duplicate or erroneous entries;

Verifying that gas and electric kits were sent to the appropriate participants and according to the agreed-upon process by I&M; and

Conducting verification surveys with a sample of program participants to verify that customers listed in the program tracking database did indeed participate and that the correct total number of measures in the kit was received.

ADM also performed a review of the deemed savings estimates used to calculate ex ante energy impacts for installed kit measures. This evaluation activity served to verify that the ex ante calculations were consistent with algorithms and values specified in the Indiana TRM.

8.2.1.2.1 Number of Kits Mailed

The total number and type of OEC kits mailed and installed at participant homes in PY2018 was determined by (1) reviewing the program tracking system and related documentation from I&M and (2) administering a telephone survey with program participants. Specifically, the tracking system was checked to assure that: (1) duplicate shipments to the same account number did not exist (2) the ex-ante kWh savings are reasonable and (3) that appropriate kits types were sent to customers. Under program rules, kits were limited to one customer account every three years. ADM found no instances in which customers received more than one kit during the program year.

Additionally, ADM administered a telephone survey to 221 program participants who received one of the two types of energy savings kits distributed through the program. All 221 survey respondents verified that they had participated in the program during 2018.

Based on the results of the verification survey effort and the review of the main program tracking system, ADM determined the total number of kits distributed to participants during PY2018 was 11,802 kits (7,593 gas kits and 4,209 electric kits).




Program supporting documentation;




ADM conducted surveys with a sample of 135 gas water heater participants and 86 electric water heater participants for a total of 221 households. For the purposes of the gross impact evaluation, the surveys were used to develop measure-level in-service rates.


Gross energy savings and peak demand reductions for the Online Energy Check-up Program were calculated by kit measure using deemed values and algorithms from the Indiana TRM. The following sections describe the specific algorithms and inputs used to calculate energy impacts for each measure in the kits.

LED Lighting: From the Indiana TRM, the Residential ENERGY STAR Lighting (CFL and LED) section was used to calculate energy impacts for the installation of LED lamps. The following equation was used to calculate the annual kWh energy savings:

∆𝑘𝑘𝑘𝑘ℎ =(𝑘𝑘𝑆𝑆𝑆𝑆𝑆𝑆𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵 − 𝑘𝑘𝑆𝑆𝑆𝑆𝑆𝑆𝐵𝐵𝐸𝐸𝐸𝐸)

1,000× 𝐸𝐸𝑆𝑆𝐺𝐺 × 𝐻𝐻𝐷𝐷𝑢𝑢𝑆𝑆𝑈𝑈 × (1 + 𝑘𝑘𝐻𝐻𝐹𝐹𝐵𝐵)

Where:

WattBASE = Wattage of baseline lamp (see table below)

WattEFF = Wattage of efficient lamp, 9

ISR = In Service Rate or percentage of units that are installed, as determined through analysis of customer survey response data

Hours = Average hours of use per year, 1,135

WHFE = Waste Heat Factor for Energy to account for HVAC interactions with efficient lighting, dependent on location of participant residence

Following this, ADM calculated the peak demand reduction using the following TRM defined equation:

∆𝑘𝑘𝑘𝑘 =(𝑘𝑘𝑆𝑆𝑆𝑆𝑆𝑆𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵 − 𝑘𝑘𝑆𝑆𝑆𝑆𝑆𝑆𝐵𝐵𝐸𝐸𝐸𝐸)

1,000× 𝐸𝐸𝑆𝑆𝐺𝐺 × 𝐶𝐶𝐹𝐹 × (1 + 𝑘𝑘𝐻𝐻𝐹𝐹𝐷𝐷)

Where:




CF = Summer Peak Coincidence Factor, 0.11


WHFD = Waste Heat Factor for Demand to account for HVAC interactions with efficient lighting, dependent on location of participant residence

Table 8-1 below from the 2015 Indiana TRM was used to determine the baseline wattages of the LED lamps included in the energy efficiency kits.


Efficient Technology WattEFF Watts (baseline)

CFL 15W or less 3.05 * WattEFF 16W - 20W 3.00 * WattEFF

21W or more 3.06 * WattEFF

LED 9W or less 3.38 * WattEFF 10W - 17W 3.41 * WattEFF


LED Night Light: From the Indiana TRM, the LED Night Lights section was used to calculate energy impacts for the installation of LED night lights. The following equation was used to calculate the annual kWh energy savings:

∆𝑘𝑘𝑘𝑘ℎ =(𝑘𝑘𝑆𝑆𝑆𝑆𝑆𝑆𝑈𝑈𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵 − 𝑘𝑘𝑆𝑆𝑆𝑆𝑆𝑆𝑈𝑈𝐿𝐿𝐵𝐵𝐷𝐷)

1,000× 𝐸𝐸𝑆𝑆𝐺𝐺 × 𝐻𝐻𝐺𝐺𝑈𝑈𝐺𝐺𝑆𝑆

Where:

WattsBASE = Wattage of incandescent night light, 5

WattsLED = Wattage of LED night light, 0.5

ISR = In Service Rate or percentage of distributed units that are installed, as determined through analysis of customer survey response data

HOURS = Average hours of use per year, 2,920

The Indiana TRM attributes no peak kW reduction to the installation of LED night lights.

Low Flow Faucet Aerator: From the Indiana TRM, the Low Flow Faucet Aerator (Time of Sale or Early Replacement) section was used to calculate energy impacts for the installation of energy saving kitchen and bathroom faucet aerators. The following equation was used to calculate the annual kWh energy savings:

∆𝑘𝑘𝑘𝑘ℎ =𝐸𝐸𝑆𝑆𝐺𝐺 × (𝐺𝐺𝑃𝑃𝑀𝑀𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏 − 𝐺𝐺𝑃𝑃𝑀𝑀𝑐𝑐𝑐𝑐𝑙𝑙) × 𝑀𝑀𝑃𝑃𝐷𝐷 × 𝑃𝑃𝐻𝐻/𝐹𝐹𝐻𝐻 × 𝐷𝐷𝐺𝐺 × 8.3 × (𝐸𝐸𝑚𝑚𝑖𝑖𝑚𝑚 − 𝐸𝐸𝑖𝑖𝑓𝑓) × 365

𝐺𝐺𝐸𝐸 × 3,412

Where:



GPMbase = Gallons per Minute of baseline faucet, 2.44 (kitchen), 1.9 (bathroom)

GPMlow = Gallons per Minute of low flow faucet, 1.49 (kitchen) and 1.01 (bathroom)

MPD = Average minutes per day used by each faucet in home, 4.5 (kitchen) and 1.6 (bathroom)

PH = Average number of people per household, as determined through analysis of customer survey response data; 2.64

FH = Average number of faucets per household, 1.0 (kitchen, single family) and 2.04 (bathroom, single family)

DR = Percentage of water flowing down drain, 50% (kitchen), 70% (bathroom)

Tmix = Average mixed temperature of water used by faucet, 93oF (kitchen), 86oF (bathroom)

Tin = Assumed temperature of water entering house, dependent on climate, dependent on location of participant residence

RE = Recovery efficiency of electric hot water heater, 0.98


∆𝑘𝑘𝑘𝑘 =𝐸𝐸𝑆𝑆𝐺𝐺 × (𝐺𝐺𝑃𝑃𝑀𝑀𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏 − 𝐺𝐺𝑃𝑃𝑀𝑀𝑐𝑐𝑐𝑐𝑙𝑙) × 60 × 𝐷𝐷𝐺𝐺 × 8.3 × (𝐸𝐸𝑚𝑚𝑖𝑖𝑚𝑚 − 𝐸𝐸𝑖𝑖𝑓𝑓) × 𝐶𝐶𝐹𝐹

𝐺𝐺𝐸𝐸 × 3,412

Where:


GPMbase = Gallons per Minute of baseline faucet, 2.44 (kitchen), 1.9 (bathroom)

GPMlow = Gallons per Minute of low flow faucet, 1.49 (kitchen) and 1.01 (bathroom)





CF = Summer Peak Coincidence Factor for measure, 0.0033 (kitchen), 0.0012 (bathroom)


High-efficiency Showerheads: From the Indiana TRM, the Low-Flow Showerhead (Time of Sale or Early Replacement) section was used to calculate energy impacts for the installation of high efficiency showerheads. The following equation was used to calculate the annual kWh energy savings:

∆𝑘𝑘𝑘𝑘ℎ =𝐸𝐸𝑆𝑆𝐺𝐺 × (𝐺𝐺𝑃𝑃𝑀𝑀𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏 − 𝐺𝐺𝑃𝑃𝑀𝑀𝑐𝑐𝑐𝑐𝑙𝑙) × 𝑀𝑀𝑆𝑆 × 𝑆𝑆𝑃𝑃𝐷𝐷 × 𝑃𝑃𝐻𝐻/𝑆𝑆𝐻𝐻 × 8.3 × (𝐸𝐸𝑏𝑏𝑚𝑚𝑖𝑖𝑚𝑚 − 𝐸𝐸𝑖𝑖𝑓𝑓) × 365

𝐺𝐺𝐸𝐸 × 3,412

Where:


GPMbase = Gallons per Minute of baseline showerhead, 2.63

GPMlow = Gallons per Minute of energy saving showerhead, 1.5

MS = Average number of minutes per shower event, 7.8

SPD = Average number of showers per person per day, 0.6


SH = Average number of showerheads per household, as determined through analysis of customer survey response data; 1.65

Tmix = Average mixed temperature of water used for shower, 101oF




∆𝑘𝑘𝑘𝑘 =𝐸𝐸𝑆𝑆𝐺𝐺 × (𝐺𝐺𝑃𝑃𝑀𝑀𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏 − 𝐺𝐺𝑃𝑃𝑀𝑀𝑐𝑐𝑐𝑐𝑙𝑙) × 60 × 8.3 × (𝐸𝐸𝑚𝑚𝑖𝑖𝑚𝑚 − 𝐸𝐸𝑖𝑖𝑓𝑓) × 𝐶𝐶𝐹𝐹

𝐺𝐺𝐸𝐸 × 3,412

Where:



GPMlow = Gallons per Minute of energy saving showerhead, actual; 1.5

Tmix = Assumed temperature of water used for shower, 101oF





Digital Thermometer: ADM assigned zero energy impacts to this measure.


Ex post annual gross kWh savings and peak kW reductions resulting from the Online Energy Check-up Program were adjusted by applying the estimated measure-level installation rates of kit measures to the calculated measure-level gross energy impacts.

The program relies on participants’ installation of the measures, and some of the items may have been uninstalled or perhaps were never installed upon receiving the kit. In the development of measure-level ISRs for the program, ADM surveyed a sample of 221 program participants (135 gas water heater participants and 86 electric water heater participants) who received energy conservation kits through the program.

Table 8-2 below displays the in-service rates developed from the participant survey responses.

Table 8-2 In-Service Rates per OEC Measure by Kit Type

Measure ISR ISR with Planned Install in Next 6

Months LED 83% 95% Showerhead 42% 63% Kitchen Aerator 63% 72% Bath Aerator 54% 67% Nightlight 47% 50%


The ex post gross energy and demand impacts resulting from the 2018 Online Energy Check-up Program are reported in the following sections.


The estimated annual gross energy savings resulting from the Online Energy Check-up Program are summarized in Table 8-3 below. The gross kWh realization rate for the program is 138%.








2,532,116 2,532,116 1,820,350 3,498,139 138%

Table 8-4 below shows the measure-level estimated annual gross energy savings resulting from the program.

Table 8-4 Measure-level Annual Gross kWh Savings

Measure Type Ex Ante Annual kWh Savings



Ex Post Annual Gross kWh

Savings

Gross Realization

Rate

9w LED 616,892 616,892 585,183 754,259 122% Kitchen Aerator 434,158 434,158 313,849 566,121 130% Bathroom Aerator 118,483 118,483 78,989 192,784 163% Showerhead 1,216,948 1,216,948 769,756 1,881,781 155% LED Nightlight 145,634 145,634 72,572 103,193 71% Total 2,532,116 2,532,116 1,820,350 3,498,139 138%

Table 8-5 below displays a breakdown of ex ante and ex post gross kWh savings by kit type. The program-level ex post annual gross kWh savings for electric water heater kits is 2,909,681 kWh, whereas the program-level ex post annual gross kWh savings for gas water heater kits is 588,458 kWh.

Table 8-5 Ex Post Gross kWh Savings by Kit Type

Kit Type Kit Quantity Ex Ante

Annual kWh Savings

Gross Audited kWh

Savings

Gross Verified kWh

Savings

Ex Post Annual

Gross kWh Savings

Gross Realization

Rate

Electric 4,209 2,007,819 2,007,819 1,371,291 2,909,681 145% Gas 7,593 524,297 524,297 449,059 588,458 112% Total 11,802 2,532,116 2,532,116 1,820,350 3,498,139 138%


The estimated gross demand reduction resulting from the Online Energy Check-up Program is summarized in Table 8-6 below. The gross kW realization rate for the program is 139%.


Table 8-6 Program-level Gross kW Reduction






157.51 157.51 123.16 218.36 139%



A survey of program participants was administered to collect data for use in estimating participant free ridership and spillover.


The net savings analysis is used to determine what part of the gross energy savings achieved by program participants can be attributed to the effects of the program. The net savings attributable to program participants are the gross savings less free ridership, plus spillover. ADM estimated free ridership and participant spillover through a survey of program participants. Non-participant spillover was estimated through a survey of non-participants.


The calculation of a free ridership is based on the responses to questions on the following topics:

Prior experience with similar energy saving equipment;

Prior planning to purchase energy efficiency measures that were provided through the program; and

Likelihood of installing similar equipment without the program.

8.3.2.1.1 Prior Experience

The program is designed to encourage customers to try efficiency measures that they previously did not have experience with by providing them at no cost to the customer. As such, a primary indicator of the likelihood that a participant is a free rider, is whether he or she has previously purchased a similar measure. Previous experience is used as an indicator of whether the customer would have coincidently purchased a similar measure on their own.

Prior experience is assessed through the following question:

FR1: Thinking back to before you completed the Online Energy Check-up, had you purchased and installed any of the following items in your home in the last three years?

Respondents indicating that they had not purchased a given measure in the past three years are considered to have minimal to no prior experience with that measure, meaning that the intervention of the program is likely significantly influential in the energy savings resulting from the measure.


These respondents receive an overall free ridership score of 0 for this measure. Otherwise, free ridership is assessed using the following three factors.

8.3.2.1.2 Prior Plans and Intentions

Customers were asked as to any plans they had to purchase any of the measures. This is addressed in the following question:

FR2: Before you heard of the program, did you have specific plans to purchase any of these kit items that were sent to you? If so, which items did you plan to purchase?

For LEDs, night lights, shower heads, and bathroom faucet aerators, customers that respond that they planned to install the measures are asked the following question:

FR3: Of the [MEASURE COUNT] [MEAUSRE] provided in the kit, how many did you plan to purchase on your own?

Respondents who indicate that they had plans to purchase the measure on FR2, are given a plans score of 1. The response to FR3 is used to adjust the plans score to reflect the number of items the respondent planned to purchase. For example, if the respondent planned to purchase one of the two items received, the plans score is adjusted to .5.

8.3.2.1.3 Likelihood of Purchasing Measure

Once customers learn of the program, it is possible that this knowledge will sway their decision-making process to install these energy efficient measures in their homes. Additionally, the information and measures provided through the program may help to overcome existing barriers to energy efficiency improvements. To address this, participants were asked the following questions:

FR4: Using a scale where 0 means “not at all likely” and 10 means “very likely”, if you had not completed the Online Energy Check-up or received the energy conservation kit, how likely would you have been to purchase any of the following items on your own within 12 months of when you received them?

FR5: [IF FR4 > 0] Based on your response, there is some likelihood that you would have purchased some of the kit items in the next 12 months. Given that, we would like to know why you had not already purchased the items on your own. Had you not already purchased [MEASURE] because 1) you didn’t want to spend the money, 2) you had not gotten around to it, 3) you didn’t know where to purchase [MEASURE], 4) you didn’t know enough about [MEASURE], or 6) another reason?

Respondents who indicated in FR4 that they had not already purchased a given measure because they did not want to spend the money, did not know where to purchase the measure, or did not know enough about the measure are considered to have had significant barriers to implementing these energy efficiency improvements and receive a score of 0% free ridership for the measure under this component. Otherwise, the likelihood of purchasing is scored as:

Likelihood of Purchasing = FR4/10


8.3.2.1.4 Free Ridership Scoring

For respondents who demonstrated prior experience with a measure, the scores for the prior plans and likelihood of purchasing the measures were averaged to assign a measure-level free ridership score to each respondent.



To assess participant spillover savings, survey respondents were asked whether they implemented any additional energy saving measures for which they did not receive a program incentive. Respondents were also asked to provide information on the attributes of the measures implemented for use in estimating the associated energy savings.

Participants who reported implementing on one or more efficiency measures were then asked two questions for use in developing a spillover score:

SO1: On a scale of 0 to 10, where 0 represents “not at all important” and 10 represents “extremely important”, how important was your experience with the Online Energy Check-up in your decision to purchase the items you just mentioned?

SO2: On a scale of 0 to 10, where 0 represents “not at all likely” and 10 represents “extremely likely” how likely would you have been to make the additional purchases you just mentioned even if you had not completed the Online Energy Check-up?



All of the associated measure savings were considered attributable to the program if the resulting score was equal or greater than 7.


Non-participant spillover was estimated using data collected through the PY2017 non-participant survey. To estimate PY2018 nonparticipant spillover, the PY2017 non-participant spillover ratio was applied to the 2018 customer usage. That is, total non-participant spillover (NPSO) savings for the portfolio was estimated as follows:


Where,





The total non-participant spillover was allocated to the Online Energy Check-up Program in proportion to its share of the 2018 expenditures.



Table 8-7 summarizes the number of survey responses and average free-ridership scores by measure for the Residential Online Energy Check-up Program.


Measure Survey

Response Count


LED Bulbs 221 35% Bathroom Faucet Aerator 86 3% Kitchen Faucet Aerator 86 9%

Showerhead 86 9% LED Nightlight 135 7%

Four customers reported installing spillover measures. The measures installed were energy efficient light bulbs, an ENERGY STAR refrigerator, and a room air conditioner. The spillover ratio was less than 1%.


Table 8-8 summarizes the net ex post kWh savings of the Residential Online Energy Check-up Program. The annual net savings totaled 3,033,683 kWh and the net-to-gross ratio is 87%.


Ex Ante Gross kWh

Savings

Gross Audited

kWh Savings

Gross Verified

kWh Savings

Ex Post Gross kWh

Savings

Gross Realization

Rate

Ex Post Net kWh

Savings

Net-to-Gross Ratio


kWh Savings

2,532,116 2,532,116 1,820,350 3,498,139 138% 3,033,683 87% 31,142,298


Table 8-9 summarizes the net ex post kW demand reduction of the Residential Online Energy Check-up Program. The net demand reduction totaled 180.62 kW and the net-to-gross ratio is 83%.




Gross Audited kW

Savings

Gross Verified kW

Savings


Gross Realization

Rate


Net-to-Gross Ratio

157.51 157.51 123.16 218.36 139% 180.62 83%



Data was collected from multiple data sources to support the process evaluation of the program. The planned data collection activities and the expected sample sizes are presented in Table 8-10.

Table 8-10 Summary of Data Collection Activities for the Residential Online Energy Check-up




In total, 11,802 kits were shipped through the Online Energy Check-up Program. The monthly number of kits shipped is shown by kit type in Figure 8-1.

Figure 8-1 Kits Shipped by Month


The incidence of returned and damaged kits remained low in PY2018. A very small number of damage kits reported (0%) were reported, and the return rate was less than 1%.

Table 8-11 Summary of Returned and Damaged Kits

Kit Delivery Percent of All Kits

Returned kits 0.94% Damaged kits 0.00%


The I&M program coordinator and Resource Action Programs (RAP) program manager were interviewed to gain insight into PY2018 program performance and operations. Specifically, the interviews focused on changes to program activities, the overall effectiveness of the program process, goals, marketing/outreach, quality assurances and control, and the identification of areas for future program improvement. ADM evaluators interviewed the I&M energy efficiency and consumer program coordinator and a program manager from Resource Action Programs (RAP) to gain insights in the Online Check-up program for 2018.

Findings of the interviews are summarized below.

Program kit distribution goals increased in October. There was an expansion of the goal for the number of kits and subsequently the energy savings because of the deemed savings per kit. The goal was increased 8,225 to 12,079 kits. In total, 11,802 kits were distributed – 98% of the revised goal. The number of kits distributed in 2018 represents an increase from 2017.

Customers will receive a notification if they are not eligible to receive a kit. I&M has requested that Tendril alert customers who request a kit online and who received a kit in the past 36 months to be notified they are not eligible to receive a kit. Currently, there is no notification for those customers who are not eligible to receive a kit.

Multiple marketing channels used to promote kits. The Online Check-up Program is cross promoted through the Home Energy Reports and Challenges, electric ideas website, newsletter, bill inserts, marketing collateral for the Weatherization program, email campaigns, and post card campaigns. I&M noted this is a marketing driven program and participation is linked to the amount of marketing, for example, if the program needed additional participants in a service territory, increased marketing would likely increase participation. Staff did not indicate there are any barriers to participating in the program and did not see any challenges to meeting future goals. Additionally, I&M provided RAP with their planned marketing approach, which staff indicated is helpful in terms of planning for increased participation and forecasting.

No changes were made to the kit measures. The measures offered remained the same, but an insert was added to the kit to cross-promote other programs.

No changes were made to data management or QA/QC processes. There were no changes to data management for the program and program staff indicated the data is kept


current to manage the program additionally, there have not been any significant changes to the QA/QC procedures in 2018 and no concerns were stated.


ADM completed a survey of program participants to collect information regarding their experience with the Online Energy Check-up Program.

The survey was administered online and a total of 221 customers completed the survey. Eighty-six respondents received electric water heater kit and the remaining 135 received gas water heater kits.


The I&M website and newsletter were the most common ways that survey participants learned of the Online Energy Check-up Program, followed by bill inserts and home energy reports. Figure 8-2 summarizes the initial source of program awareness for responding customers.


Survey participants were also asked if they heard about the program from other sources. The majority (52%) responded they had not heard of the program from another source, followed by 12% who indicated bill inserts, 10% who said the I&M newsletter, and 9% who stated the I&M website. Table 8-12 summarizes the reasons customers gave for completing the check-up and receiving the energy efficiency kit. Sixty-six (66%)percent said they participated to learn ways to save money on energy bills and 25% stated the participated to receive the free items.


Table 8-12 Reason for Completing the Residential Online Energy Check-up Survey and Receiving the Energy Efficiency Kit


To learn about ways to save money on energy bill(s) 66% The items were provided free of charge 25% Environmental reasons 7% Other 2%

Experience Using the Audit Tool

Respondents reported how many times they had logged on to the audit tool, with most logging on one time (see Figure 8-3). The results suggest that while 56% logged on once, a significant share revisited the tool at future times.

Figure 8-3 Number of Times Survey Participants Logged on to Audit Tool

Customers rated the usefulness of the recommendations for those who recalled receiving the pdf file: The ratings were on a scale from 1 (not at all relevant/useful) to 5 (very relevant/useful). Overall, most respondents indicated the recommendations provided to them as a result of taking the survey were relevant (see Table 8-12).

Table 8-13 Ease of Taking the Survey in the Residential Online Energy Check-up Program


Very easy 68% Somewhat easy 25% Neither easy nor difficult 6% Somewhat difficult 1% Very difficult 0%


I&M customers who completed the check-up were sent a PDF report which provides recommendations for saving energy within their home. Fifty-nine percent recalled receiving the PDF, with 41% who did not recall (n = 220).

The usefulness of those the recommendations were assessed among customers on a scale from 1 (not at all relevant/useful) to 5 (very relevant/useful). Overall, most respondents indicated the recommendations provided to them regarding the program were relevant (see Figure 8-4).

Figure 8-4 Usefulness of Residential Online Energy Check-up Program Recommendations

Of those who did not find the recommendations useful, the most common reason was because those customers were already implementing energy savings in their households. Below are the issues with the recommendations raised by survey respondents:

88% indicated they are already doing the things recommended (n = 7)

50% indicated they are too generic and/or not applicable to their home (n =4)

38% indicated they are not realistic (n = 3)

Behavioral Actions, Knowledge and Attitudes

Customers specified the actions they have taken since completing the tool. The most commonly implemented recommendation was more frequently turning off lights when not in use, followed by washing only full loads in the clothes washer, and using cold water when doing laundry more often (see Table 8-14).

Table 8-14 Actions Taken to Reduce Energy Use in Home due to Recommendation

Response Number of Respondents

Percentage of

Respondents

Turned off lights when not in used more often 191 89% Wash only full loads in the clothes washer 174 84% Used cold water when doing laundry more often 143 71% Adjusted thermostat settings or used heating and cooling equipment less often 170 79% Closed or cover windows to reduce heat gain or loss 154 74% Unplugged appliances not in use 123 58% Added weather stripping or caulking around doors and windows 81 38%


Response Number of Respondents

Percentage of

Respondents

Turned down water heater temperature 74 36% Increased freezer or refrigerator temperature 47 23%

Table 8-15 summarizes the other actions taken by customers to reduce energy use in their households.

Table 8-15 Other Actions Taken to Reduce Energy Use in Home due to Recommendation


Added insulation 9

Had insulation blown into my attic. Old water heater died. Purchased a new energy efficient water heater.

Assessing insulation factors

Added insulation to my attic

Insulated the attic door.

New windows and doors 6

Installed new windows

Re-installed the door at the bottom of my stairs (that lead to a second floor that doesn't get much heat) to reduce the draft from upstairs.

Replaced 2 wooden entry doors for insulated steel doors, replaced 1 window for a double pane window

Replaced several windows

Window treatments to reduce energy loss 6

Added 6 cellular honeycomb blinds to 6 windows

Plastic over windows during the cold months.

I cover my windows at night to prevent heat loss.

Plastic on windows

Open drapes/blinds on sunny days to warm house inside

Furnace tune up added window film on basement egress window

Other 5

Added humidifier to put more moisture in air. Set ceiling fans to down draft warm air.

Removed an old refrigerator, reduced hot water heater temp

Primarily using cold water when doing dishes, not showering every day and trying to reduce the amount of water used, using efficiency appliances

Keep garage door closed.

Weather strip in garage doors. Turned down the shop heat to 40 degrees unless I'm working there.



Heating alternatives 3

Used fireplace to keep the bill for our terrible radiant ceiling heat, down.

Wood fireplace insert for heating.

Fire place use

LED lighting 3 We changed out our other light bulbs to LEDs.

We use only LED lighting (as much as possible)

Added LED lights to other areas of our home

Reduce appliance usage 3

Take clothes out of dryer sooner and hang when possible.

Hang clothes to dry

I have gone through my entire house and unplugged ANY Item that was not being used. I have a whirlpool tub and once I unplugged that my bill dropped a LOT.

Reduce heating usage 2 Run ceiling fans to make room feel warmer.

Turn heat off at night

Customer’s Cross Program Awareness

Sixty-four (64%) percent of survey participants were aware that I&M offers rebates and discounts for energy efficiency equipment and home improvement (n = 218). Thirty-seven (37%) percent of survey respondents indicated they first learned of the rebate or discount program when they completed their check-up (n = 137).

Of those who had heard of the other programs, 71% indicated knowledge of the appliance rebates, 71% of the appliance recycling/disposal, and 64% of the rebates for thermostats (see Table 8-16).

Table 8-16 Awareness of Other I&M Programs

Response

Percentage of

Respondents (n = 132)

Appliance rebates 71% Appliance recycling/disposal 71% Thermostat rebates 64% Rebates for central air conditioners and heat pumps 43% Lighting discounts through participating retailers 27% Rebates for making weatherization improvements (air sealing, adding insulation) 18% Other 1%

Survey participants rated their knowledge on ways to save and use energy in their homes on a scale from 1 (not at all knowledgeable) to 10 (very knowledgeable). The information was collapsed into three groups (1-3 – not very knowledgeable; 4-6 – somewhat knowledgeable; 7-10 – very knowledgeable). Respondents self-reported they were very knowledgeable in terms of use and savings of energy in their home (see Figure 8-5).


Figure 8-5 Customer Knowledge of Energy Usage and Savings in Household

In addition, participants self-reported their own household’s efforts to save energy in their home on a scale from 1 (you have not done much) to 10 (you have done almost everything you can to lower your monthly energy bill in your home). Again, this data was collapsed into three groups (1-3 – made little to no effort; 4-6 – made some effort; 7-10 – made a lot of effort).

Figure 8-6 Customer Efforts to Save Energy in the Home


Overall customer satisfaction was assessed with the Online Energy Check-up Program. Participants used a scale from 1 (very dissatisfied) to 5 (very satisfied) to indicate their satisfaction with various aspects of the program and their overall impressions. Generally, participants were satisfied with the program and I&M as an electricity service provider (see Figure 8-7 and Table 8-17).


Figure 8-7 Customer Satisfaction with the Online Energy Check-up Service, Kit Items, and Information Provided

The customers who indicated dissatisfaction with the program stated:

They did not receive the kit items (n = 2); The recommendations were not useful (n = 2); The recommended actions cost too much (n = 1); and They did not like the kit items (n =1).


Responses Percentage of Respondents

(n = 220) Very Satisfied 56% Somewhat Satisfied 29% Neither Satisfied or Dissatisfied 12% Somewhat Dissatisfied 2% Very Dissatisfied 0%

Customers provided additional comments at the end of the survey. The majority of comments (13) were positive comments about the program or I&M. For example,

Great program.

Love the service, I am just very energy savvy. I was hoping there might be something new. I think this is a great service to move people to energy savings. Wonderful for our environment and lowering our footprint.


Of all the bills we pay, customer representatives we've communicated with, & services we receive, we have been most satisfied with I&M. Never had any problems & communication has always been efficient.

Other comments were related to:

Interest in solar or renewables (n = 2); Interest in other programs or services (n = 2); Concern that they have not seen a decrease in their electricity costs (n = 1); and Interest in customizing kit contents so that they are appropriate for their home (n = 2).

Yearly Trends


More people learned about the Online Energy Checkup program from home energy reports in 2018 compared to 2017 (12 point increase, see Figure 8-8). There was also a 17 point increase in survey respondents who indicated they decided to complete the Online Energy Check-up because they wanted to learn about ways to save money on energy bills.


There was an overall decrease among those who indicated it was somewhat or very easy to complete the online energy checkup by three percentage points (see Table 8-18). Additionally, there was an overall decrease among those who indicated the recommendations for their home were somewhat or very useful by four percentage points (see Table 8-19).


Table 8-18 2017 and 2018 Ease of Completing the Online Checkup Comparison

Response

Percentage by

Response (2017)

Percentage by

Response (2018)


1 - Very difficult 2% 0% -2% 2 – Somewhat difficult 0% 1% 1% 3 – Neither 2% 6% 4% 4 – Somewhat easy 16% 25% 9% 5 - Very easy 81% 68% -12%

Table 8-19 2017 and 2018 Usefulness of Recommendations Comparison

Response

Percentage by

Response (2017)

Percentage by

Response (2018)


1 – Not at all useful 1% 2% 2% 2 4% 4% 0% 3 17% 20% 2% 4 22% 35% 13% 5 - Very useful 56% 39% -17%

There was a 5 percentage-point increase in the share of survey respondents who indicated they were aware of I&M programs that offer rebates and discounts for energy efficient improvements. The largest increase was among thermostat rebates, followed by appliance recycling and rebates for central air conditioners and heat pumps (see Figure 8-9).

Figure 8-9 2017 and 2018 Program Rebate and Incentive Awareness Comparison

Between 2017 and 2018, there was an overall decrease in survey respondents who indicated they were either somewhat or very satisfied with the kit items they received (4 point decrease) and the information provided by the online energy checkup (2 point decrease).


Figure 8-10 2017 and 2018 Comparison of Satisfaction with Kit Items and Information Provided by the Online Energy Checkup


8.5.1 Conclusions


The number of kits distributed increased, and the program nearly met its distribution goal. The program achieved 98% of its revised goal for the number of kits distributed in 2018. Although it missed this target by a small amount, the number of kits distributed represents an increase from 2017.

Multiple marketing channels are used to promote the program. The program is promoted through the home energy reports program, the I&M program website, bill inserts, email and post card campaigns, and cross-promoted with weatherization advertisements to participants. Staff noted that the program activity is responsive to marketing efforts.

Few changes made to the program. The program added an insert in the kits to cross-promote other programs. The kit contents remained the same.

Kit delivery is efficient and effective. Review of program tracking data found that the rate of returned or damaged kits was 1%, indicating that that the process of delivering the kit items is efficient and effective.

Most participants found it easy to take the survey and the recommendations useful. Ninety-three (93%) percent of participants responded that taking the survey was very or


somewhat easy. Additionally, 74% found the recommendations to be very or somewhat useful.

Nearly all participants reported taking actions based on a recommendation in the report to save energy. Ninety-six (96%) percent of respondents reported taking one or more actions based on a recommendation made in the report received. Turning lights off more frequently, limiting washing to full loads, and using cold water were the three most frequently reported actions taken.

Participants were generally satisfied with the kit items and the program overall. Ninety-two (92%) percent of survey participants indicated they were very or somewhat satisfied with the kit items they received, and 89% were very or somewhat satisfied with the program overall. Eighty-five (85%) percent were very or somewhat satisfied with the I&M as their electricity service provider.


ADM does not have any evaluation recommendations for the Online Energy Check-up program.


9 Home Weatherproofing This chapter presents the results of both the impact and process evaluations of the 2018 Home Weatherproofing Program that Indiana Michigan Power (I&M) offered to its residential customers during the period of January 2018 through December 2018.










The Home Weatherproofing Program is offered to residential customers who would benefit from higher level standard home weatherization measures such as ceiling insulation, home infiltration, and duct sealing. The program provides an in-home energy assessment, during which the assessor identifies energy efficiency improvements and recommends measures and installs low-cost efficiency measures such as LED light bulbs and low-flow devices. The customer initially pays a reduced cost of $99 for the energy assessment. The cost of the assessment may be applied towards the incentive for implemented measures. The home provides incentives for efficiency improvements up to 50% of the cost of the measure. Total incentives are capped at $3,000.

The program is implemented by I&M and I&M staff perform the energy assessments. The program works with a group of contractors to install recommended measures in customer homes.








Where x is the average kWh energy savings per participant. The resulting sample size is estimated at:

𝑆𝑆0 = �1.645 ∗ 𝐶𝐶𝐶𝐶

𝐺𝐺𝑃𝑃�

2

Where,




On-site visitations were performed by ADM to verify and supplement the provided tracking data. ADM used a sample size of 17 homes for this verification activity.


I&M maintains program tracking information that includes a full list of all participants, the measures that were installed in their home, and the kWh and kW savings associated with each measure. The first aspect of conducting measurements of program activity was to verify that the tracking data report of participants and measures was accurate. To this end, ADM reviewed the program data to verify that the fields required for performing the evaluation are tracked and populated (i.e., the data is not missing) and that the values are reasonable. ADM took several steps in verifying the number of weatherproofing measures installed, which consists of the following:

Validating program tracking data by checking for duplicate or erroneous entries;

Performing site visits to participant’s homes to confirm that measures were installed in the quantity and specifications claimed, and

Conducting verification surveys with a sample of program participants to verify that customers listed in the program tracking database did indeed participate and that the number of measures claimed to be installed is accurate.

ADM also performed a review of the savings estimates used to calculate ex ante energy impacts for installed measures. This evaluation activity is intended to verify that the ex ante calculations are consistent with algorithms and values specified in the 2015 Indiana Technical Reference Manual (TRM).








ADM surveyed a sample of program participants for developing measure-level in-service rates. Additionally, ADM conducted on-site verification visits to weatherized homes. Data collected from these evaluation activities were used to develop measure-level in-service rates and to verify that equipment specifications (capacity, efficiency) match the tracking data and are properly recorded.


Gross energy savings and demand reductions for the program were calculated (by measure) in accordance with the Indiana TRM. ADM reviewed the Indiana TRM and assessed the appropriateness of the engineering algorithms.

The following sections describe the specific algorithms and inputs used to calculate energy impacts for each measure in the program.

Air Infiltration Reduction: From the Indiana TRM, the Air Sealing – Reduce Infiltration (Retrofit) section was used to calculate energy impacts for the reduction of home air infiltration. The following equation was used to calculate the annual kWh energy savings:

∆𝑘𝑘𝑘𝑘ℎ =𝐶𝐶𝐹𝐹𝑀𝑀50𝐵𝐵𝑚𝑚𝑖𝑖𝑏𝑏𝑡𝑡 − 𝐶𝐶𝐹𝐹𝑀𝑀50𝑁𝑁𝑏𝑏𝑙𝑙

𝑁𝑁 − 𝐹𝐹𝑆𝑆𝐼𝐼𝑆𝑆𝐷𝐷𝑆𝑆×

∆𝑘𝑘𝑘𝑘ℎ𝐶𝐶𝐹𝐹𝑀𝑀

Where:

CFM50Exist = Existing Cubic Feet per Minute at 50 Pascal pressure differential as measured by the blower door before air sealing

CFM50New = New Cubic Feet per Minute at 50 Pascal pressure differential as measured by the blower door after air sealing

N-Factor = Conversion factor to convert 50-pascal airflow to natural airflow, dependent on exposure level

ΔkWh/cfm = kWh impacts per CFM of infiltration rate reduction, dependent on home cooling and heating types


∆𝑘𝑘𝑘𝑘 =𝐶𝐶𝐹𝐹𝑀𝑀50𝐵𝐵𝑚𝑚𝑖𝑖𝑏𝑏𝑡𝑡 − 𝐶𝐶𝐹𝐹𝑀𝑀50𝑁𝑁𝑏𝑏𝑙𝑙

𝑁𝑁 − 𝐹𝐹𝑆𝑆𝐼𝐼𝑆𝑆𝐷𝐷𝑆𝑆×

∆𝑘𝑘𝑘𝑘𝐶𝐶𝐹𝐹𝑀𝑀

× 𝐶𝐶𝐹𝐹

Where:


CFM50Exist = Existing Cubic Feet per Minute at 50 Pascal pressure differential as measured by the blower door before air sealing

CFM50New = New Cubic Feet per Minute at 50 Pascal pressure differential as measured by the blower door after air sealing

N-Factor = Conversion factor to convert 50-pascal airflow to natural airflow, dependent on exposure level.

ΔkW/cfm = kW impacts per CFM of infiltration rate reduction


Attic/Wall Insulation: From the Indiana TRM, the Envelope Insulation (Retrofit) section was used to calculate energy impacts for the installation of additional insulation. The following equation was used to calculate the annual kWh energy savings:

∆𝑘𝑘𝑘𝑘ℎ = 𝑘𝑘𝑆𝑆𝐹𝐹 ×∆𝑘𝑘𝑘𝑘ℎ

𝑘𝑘𝑆𝑆𝐹𝐹

Where:

kSF = Area of installed insulation (1,000 sq. ft.)

ΔkWh/kSF = Unit energy savings from lookup table, based on insulation type, home location, and installation location (ceiling or wall)


∆𝑘𝑘𝑘𝑘 = 𝑘𝑘𝑆𝑆𝐹𝐹 ×∆𝑘𝑘𝑘𝑘𝑘𝑘𝑆𝑆𝐹𝐹

× 𝐶𝐶𝐹𝐹

Where:

kSF = Area of installed insulation (1,000 sq. ft.)

ΔkW/kSF = Unit demand savings from lookup table, based on insulation type, home location, and installation location (ceiling or wall)


Customer Education: From the 2012 Energizing Indiana Programs EM&V Report, the Home Audit Recommendations section was used as reference to calculate energy savings for customer education. Evaluation staff asked participants during the telephone survey which recommendations from the education were implemented. Savings were attributed to the portion of sampled participants who indicated that they had engaged in these behaviors. These behavior engagement rates were then extrapolated to the participant population. Estimated savings attributed to each of the behaviors listed in the 2012 Energizing Indiana Programs EM&V Report are as follows:


Turning off lights when not in use: 158 kWh, 0.01 kW

Unplugging unused appliances: 21 kWh, 0.00 kW

Installing water heater tank wrap: 79 kWh, 0.01 kW

Washing clothes in cold water: 58 kWh, 0.00 kW

Programming an existing thermostat: 26 kWh, 0.00 kW

Installing a programmable thermostat: 131 kWh, 0.00 kW

Duct Sealing: From the Indiana TRM, the Duct Sealing and Insulation (Retrofit) section was used to calculate energy impacts for performing duct sealing. The following equations was used to calculate the annual kWh energy savings:


∆𝑘𝑘𝑘𝑘ℎ𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑖𝑖𝑓𝑓𝑖𝑖 =�

𝐷𝐷𝐸𝐸𝑏𝑏𝑓𝑓𝑡𝑡𝑏𝑏𝑁𝑁 − 𝐷𝐷𝐸𝐸𝑏𝑏𝑏𝑏𝑓𝑓𝑐𝑐𝑁𝑁𝑏𝑏𝐷𝐷𝐸𝐸𝑏𝑏𝑓𝑓𝑡𝑡𝑏𝑏𝑁𝑁

� × 𝐸𝐸𝐹𝐹𝐸𝐸𝐻𝐻𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 × 𝐵𝐵𝑆𝑆𝑢𝑢𝐻𝐻𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐

𝑆𝑆𝐸𝐸𝐸𝐸𝐺𝐺 × 1,000

∆𝑘𝑘𝑘𝑘ℎℎ𝑏𝑏𝑏𝑏𝑡𝑡𝑖𝑖𝑓𝑓𝑖𝑖 =�

𝐷𝐷𝐸𝐸𝑏𝑏𝑓𝑓𝑡𝑡𝑏𝑏𝑁𝑁 − 𝐷𝐷𝐸𝐸𝑏𝑏𝑏𝑏𝑓𝑓𝑐𝑐𝑁𝑁𝑏𝑏𝐷𝐷𝐸𝐸𝑏𝑏𝑓𝑓𝑡𝑡𝑏𝑏𝑁𝑁

� × 𝐸𝐸𝐹𝐹𝐸𝐸𝐻𝐻ℎ𝑏𝑏𝑏𝑏𝑡𝑡 × 𝐵𝐵𝑆𝑆𝑢𝑢𝐻𝐻ℎ𝑏𝑏𝑏𝑏𝑡𝑡


Where:

ΔkWhcooling = Cooling energy savings due to duct sealing

ΔkWhheating = Heating energy savings due to duct sealing

DEafter = Distribution Efficiency after duct sealing

DEbefore = Distribution Efficiency before duct sealing

EFLHcool = Equivalent Full Load Cooling Hours, dependent upon location

EFLHheat = Equivalent Full Load Heating Hours, dependent upon location

BtuHcool = Cooling capacity of equipment in BtuH

BtuHheat = Heating capacity of equipment in BtuH

SEER = Seasonal average efficiency in SEER of Air Conditioning equipment

ηHeat = Efficiency in COP of Heating equipment

3,412 = Conversion from Btuh to kW



∆𝑘𝑘𝑘𝑘 =�

𝐷𝐷𝐸𝐸𝑝𝑝𝑝𝑝,𝑏𝑏𝑓𝑓𝑡𝑡𝑏𝑏𝑁𝑁 − 𝐷𝐷𝐸𝐸𝑝𝑝𝑝𝑝,𝑏𝑏𝑏𝑏𝑓𝑓𝑐𝑐𝑁𝑁𝑏𝑏𝐷𝐷𝐸𝐸𝑝𝑝𝑝𝑝,𝑏𝑏𝑓𝑓𝑡𝑡𝑏𝑏𝑁𝑁

� × 𝐵𝐵𝑆𝑆𝑢𝑢𝐻𝐻𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 × 𝐶𝐶𝐹𝐹

𝐸𝐸𝐸𝐸𝐺𝐺 × 1,000

Where:

DEpk,after = Distribution Efficiency under peak summer conditions after duct sealing

DEpk,before = Distribution Efficiency under peak summer conditions before duct sealing

BtuHcool = Cooling capacity of equipment in Btuh

EER = Peak efficiency in EER of Air Conditioning equipment


Low Flow Faucet Aerator: From the Indiana TRM, the Low Flow Faucet Aerator (Time of Sale or Early Replacement) section was used to calculate energy impacts for the installation of energy saving faucet aerators. The following equation was used to calculate the annual kWh energy savings:


𝐺𝐺𝐸𝐸 × 3,412

Where:


GPMbase = Gallons per Minute of baseline faucet

GPMlow = Gallons per Minute of low flow faucet

MPD = Average minutes per day used by each faucet in home, 4.5 (kitchen), 1.6 (bathroom)

PH = Average number of people per household, 2.64 (single family)

FH = Average number of faucets per household, 1.00 (kitchen), 2.04 (bathroom)



Tin = Assumed temperature of water entering house, dependent on climate; groundwater temperature referenced





𝐺𝐺𝐸𝐸 × 3,412

Where:

CF = Summer Peak Coincidence Factor for measure, 0.0033 (kitchen), 0.0012 (bathroom)

Low Flow Showerhead: From the Indiana TRM, the Low-Flow Showerhead (Time of Sale or Early Replacement) section was used to calculate energy impacts for the installation of energy saving showerheads. The following equation was used to calculate the annual kWh energy savings:


𝐺𝐺𝐸𝐸 × 3,412

Where:


GPMbase = Gallons per Minute of baseline showerhead

GPMlow = Gallons per Minute of energy saving showerhead

MS = Average minutes per shower, 7.8

SPD = Average showers per day, 0.6

PH = Average number of people per household, 2.64 (single-family)

SH = Average number of showerheads per household, 1.6 (single-family)


Tin = Assumed temperature of water entering house, dependent on climate; TRM table will be referenced




𝐺𝐺𝐸𝐸 × 3,412

Where:






Tin = Assumed temperature of water entering house, dependent on climate; TRM table will be referenced




∆𝑘𝑘𝑘𝑘ℎ =(𝑘𝑘𝑆𝑆𝑆𝑆𝑆𝑆𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏 − 𝑘𝑘𝑆𝑆𝑆𝑆𝑆𝑆𝐿𝐿𝐵𝐵𝐷𝐷)

1,000× 𝐸𝐸𝑆𝑆𝐺𝐺 × 𝐻𝐻𝐷𝐷𝑢𝑢𝑆𝑆𝑈𝑈 × (1 + 𝑘𝑘𝐻𝐻𝐹𝐹𝑏𝑏)

Where:

Wattbase = Wattage of baseline lamp, actual; TRM table referenced if missing

WattLED = Wattage of efficient lamp, actual


Hours = Average hours of use per year, 902

WHFE = Waste Heat Factor for Energy to account for HVAC interactions with efficient lighting; TRM table based on home location was referenced


∆𝑘𝑘𝑘𝑘 =(𝑘𝑘𝑆𝑆𝑆𝑆𝑆𝑆𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏 − 𝑘𝑘𝑆𝑆𝑆𝑆𝑆𝑆𝐿𝐿𝐵𝐵𝐷𝐷)

1,000× 𝐸𝐸𝑆𝑆𝐺𝐺 × 𝐶𝐶𝐹𝐹 × (1 + 𝑘𝑘𝐻𝐻𝐹𝐹𝑚𝑚)

Where:


WHFD = Waste Heat Factor for Demand to account for HVAC interactions with efficient lighting; TRM table based on home location was referenced

Domestic Hot Water Pipe Insulation: From the Indiana TRM, the Domestic Hot Water Pipe Insulation (Retrofit) section was used to calculate energy impacts for the installation of insulation on the hot water pipe of the water heater. The following equation was used to calculate the annual kWh energy savings:



𝐺𝐺𝑏𝑏𝑚𝑚𝑖𝑖𝑏𝑏𝑡𝑡− 1

𝐺𝐺𝑓𝑓𝑏𝑏𝑙𝑙� × 𝐸𝐸 × 𝐶𝐶 × ∆𝐸𝐸 × 8,760

𝜂𝜂𝐷𝐷𝐷𝐷𝐷𝐷 × 3,412

Where:

Rexist = Pipe heat loss coefficient of uninsulated pipe (oF*hr*ft2/Btu), 1.0

Rnew = Pipe heat loss coefficient of insulated pipe (oF*hr*ft2/Btu)

L = Length of pipe from water heating source covered by pipe wrap (ft)

C = Circumference of pipe (ft)

ΔT = Average temperature difference between supplied water and outside air temperature, 65oF

𝜂𝜂DHW = Recovery efficiency of electric hot water heater, 0.98


∆𝑘𝑘𝑘𝑘 =∆𝑘𝑘𝑘𝑘ℎ8,760

Where:

ΔkWh = Annual kWh energy savings due to the installation of the pipe wrap

Shower Start: From the IL TRM, the Thermostatic Restrictor Shower Valve section was used to calculate energy impacts for the installation of smart start showerheads. The following equation was used to calculate the annual kWh energy savings:

∆𝑘𝑘𝑘𝑘ℎ = %𝐸𝐸𝑆𝑆𝐷𝐷𝐼𝐼𝑆𝑆𝑆𝑆𝐷𝐷𝐼𝐼𝐷𝐷𝐻𝐻𝑘𝑘 × �𝐺𝐺𝑃𝑃𝑀𝑀𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑆𝑆 × 𝐸𝐸𝐵𝐵ℎ𝑐𝑐𝑙𝑙𝑏𝑏𝑁𝑁𝑚𝑚𝑏𝑏𝑜𝑜𝑖𝑖𝑐𝑐𝑏𝑏 × 𝐻𝐻𝐷𝐷𝑢𝑢𝑈𝑈𝐷𝐷ℎ𝐷𝐷𝑆𝑆𝑆𝑆 × 𝑆𝑆𝑃𝑃𝐶𝐶𝐷𝐷 × 365.25

𝑆𝑆𝑃𝑃𝐻𝐻 �

× 𝐸𝐸𝑃𝑃𝐺𝐺_𝐷𝐷𝑆𝑆𝐷𝐷𝐼𝐼𝑆𝑆𝑆𝑆𝐷𝐷𝐼𝐼 × 𝐸𝐸𝑆𝑆𝐺𝐺

Where:

%ElectricDHW = Proportion of water heating supplied by electric resistance heating, 1.0

GPM_base_S = Flow rate of the basecase showerhead, 2.67

L_showerdevice = Hot water waste time avoided due to thermostatic valve, 0.89

Household = Average number of people per household, 2.56

SPCD = Showers per capita per day, 0.6

SPH = Showerheads per household, 1.79


EPG_electric = Energy per gallon of hot water supplied by electric, 0.177


∆𝑘𝑘𝑘𝑘 =∆𝑘𝑘𝑘𝑘ℎ𝐻𝐻𝐷𝐷𝑢𝑢𝑆𝑆𝑈𝑈

× 𝐶𝐶𝐹𝐹

Where:

ΔkWh = Annual kWh energy savings due to the installation of the smart start

Hours = Annual electric DHW recovery hours for wasted showerhead use prevented by device, 27.51

CF = Coincidence factor, 0.0022

Thermostat Adjustment: From the IL TRM, the Water Heater Temperature Setback section was used to calculate energy impacts for the temperature adjustment of the water heater. The following equation was used to calculate the annual kWh energy savings:

∆𝑘𝑘𝑘𝑘ℎ =𝑈𝑈 × 𝑃𝑃 × (𝐸𝐸𝑖𝑖𝑆𝑆𝐷𝐷 − 𝐸𝐸𝑖𝑖𝐷𝐷𝑈𝑈𝑆𝑆) × 𝐻𝐻𝐷𝐷𝑢𝑢𝑆𝑆𝑈𝑈 × 𝐸𝐸𝑆𝑆𝐺𝐺

3412 × 𝐺𝐺𝐸𝐸_𝐷𝐷𝑆𝑆𝐷𝐷𝐼𝐼𝑆𝑆𝑆𝑆𝐷𝐷𝐼𝐼

Where:

U = Overall heat transfer coefficient of tank, 0.083

A = Surface area of storage tank, 24.99

Tpre = Hot water setpoint prior to adjustment, 135

Tpost = New hot water setpoint, 120

Hours = Number of hours in a year, 8766

RE_electric = Recovery efficiency of electric hot water heater, 0.98



× 𝐶𝐶𝐹𝐹

Where:

ΔkWh = Annual kWh energy savings due to the temperature adjustment

CF = Coincidence factor, 1



The estimated gross impacts resulting from the PY2018 Home Weatherproofing program are summarized in Table 9-1.

Table 9-1 Gross Impact Summary






533,363 533,363 522,541 527,876 99%






39.99 39.99 39.43 46.62 117%


The annual energy savings for the Indiana Residential Home Weatherproofing Program was 527,876 kWh with a realization rate of 99%.

Table 9-2 Measure Level Gross kWh Savings Summary



Gross Verified kWh

Savings

Ex Post Gross kWh

Savings

Realization Rate

Bath Faucet Aerator 3,683 3,683 3,397 5,434 148% Kitchen Faucet Aerator 19,484 19,484 17,725 13,878 71%

Pipe Insulation 22,969 22,969 22,544 22,292 97% Showerhead 45,348 45,348 37,790 40,200 89% Shower Start 972 972 810 218 22% Duct Sealing 23,112 23,112 23,112 23,208 100% LED Bulb 90,989 90,989 90,358 88,782 98% Air Infiltration 233,299 233,299 233,299 233,299 100% Ceiling Insulation 56,586 56,586 56,586 56,586 100% Wall Insulation 11,309 11,309 11,309 11,309 100% Thermostat Adjustment 87 87 87 82 93%

Customer Education 25,524 25,524 25,524 32,587 128% Total 533,363 533,363 522,541 527,876 99%



Table 9-3 summarizes the gross peak demand reductions of the Home Weatherproofing Program. The annual demand savings for the Indiana Residential Home Weatherproofing Program was 46.62 kW with a realization rate of 117%.

Table 9-3 Measure Level Gross Demand Reduction Summary


Gross Audited

kW Savings

Gross Verified kW

Savings


Realization Rate

Bath Faucet Aerator 0.80 0.80 0.74 0.52 65%

Kitchen Faucet Aerator 0.41 0.41 0.37 0.63 155%

Pipe Insulation 2.58 2.58 2.53 2.54 99% Showerhead 2.25 2.25 1.87 1.97 88% Shower Start 0.06 0.06 0.05 0.01 24% Duct Sealing 8.22 8.22 8.22 8.22 100% LED Bulb 3.84 3.84 3.81 12.21 318% Air Infiltration 11.47 11.47 11.47 11.47 100% Ceiling Insulation 6.37 6.37 6.37 6.37 100% Wall Insulation 1.16 1.16 1.16 1.19 102% Thermostat Adjustment 0.01 0.01 0.01 0.01 93%

Customer Education 2.83 2.83 2.83 1.47 52%

Total 39.99 39.99 39.43 46.62 117%



A survey of program participants was administered to collect data for use in estimating participant free ridership and spillover.


The net savings analysis is used to determine what part of the gross energy savings achieved by program participants can be attributed to the effects of the program. The net savings attributable to program participants are the gross savings less free ridership, plus spillover. ADM estimated net savings using survey responses from a sample of program participants.


Direct Install Measures

The calculation of free ridership was based on the responses to questions about the participants’ prior plans and intentions, program influence on measure selection, and program influence on timing of measure implementation.


Two different sets of questions were used to estimate saving resulting from the no-cost direct install measures and from the major, rebated measures (e.g., insulation, air sealing). The differences in the questions reflect differences in the two types of measures as well as the incentive strategy. The direct install free ridership questions do not incorporate the customer’s financial ability to purchase and install the measures because the items are generally low cost. Additionally, free ridership for the direct install measures is not addressed because it is less likely that a respondent would have had long term plans to install these low-cost items.

The calculation of direct install measure free ridership was based on the responses to questions on the following topics:

Prior experience with similar energy saving equipment;

Prior planning to purchase energy efficiency measures that were provided through the program; and

Likelihood of installing similar equipment without the program.

9.3.2.2 Prior Experience

Because the program provides the measures at no cost to the customer and installs them in the customer’s household as part of an energy assessment and potential larger efficiency project, the primary indicator of the likelihood that a participant is a free rider is whether he or she has previously purchased a similar measure. Previous experience is used as an indicator of whether the customer would have coincidently purchased a similar measure on their own.

Previous experience with the measure is assessed through the following question:

DI_FR1: Thinking back to before you participated in the [PROGRAM], had you purchased and installed any of the following items in your home in the last three years?

Respondents indicating that they had not purchased a given measure in the past three years are considered to have minimal to no prior experience with that measure, meaning that the intervention of the program is likely significantly influential in the energy savings resulting from the measure. These respondents receive an overall free ridership score of 0 for this measure. Otherwise, free ridership is assessed using the following factors.

9.3.2.3 Prior Plans and Intentions

Customers were asked as to any plans they had to purchase any of the measures. This is addressed in the following question:

DI_FR2: Before you heard of the program, did you have specific plans to purchase the no-cost [ALL_DI_MEASURES] installed in your residence? If so, which items did you have planned?

For LEDs, night lights, shower heads, and bathroom faucet aerators, customers that respond that they planned to install the measures were asked the following question:


DI_FR3: Of the [MEASURE COUNT] [MEAUSRE] provided in the kit, how many did you plan to purchase on your own?

Respondents who indicated that they had plans to purchase the measure on DI_FR2, were given a plans score of 1. The response to DI_FR3 was used to adjust the plans score to reflect the number of items the respondent planned to purchase. For example, if the respondent planned to purchase one of the two items received, the plans score was adjusted to .5.

9.3.2.4 Likelihood of Purchasing Measure

In the absence of specific plans to purchase and installs the direct install items Once customers learn of the program, it is possible that this knowledge will sway their decision making process to install these energy efficient measures in their homes. Additionally, the information and measures provided through the program may help to overcome existing barriers to energy efficiency improvements. To address this, participants receive the following questions to inform the Importance of Decision Making variable:

DI_FR4: Using a scale where 0 means “not at all likely” and 10 means “very likely”, how likely would you have been to purchase any of the following items provided for free through the program on your own within 12 months of when you received them?

DI_FR5: [IF DI_FR4 > 0] Based on your response, there is some likelihood that you would have purchased [DI_MEASURE] in the next 12 months. Given that, we would like to know why you had not already purchased [DI_MEASURE] on your own. Had you not already purchased [DI_MEASURE] because 1) you didn’t want to spend the money, 2) you had not gotten around to it, 3) you didn’t know where to purchase [DI_MEASURE], 4) you didn’t know enough about [DI_MEASURE], 5) (for lighting) you were waiting for a bulb to burn out, or 6) another reason?

Respondents who indicated in DI_FR4 that they had not already purchased a given measure because they did not want to spend the money, did not know where to purchase the measure, or did not know enough about the measure are considered to have had significant barriers to implementing these energy efficiency improvements and receive a score of 0% free ridership for the measure under this component. Otherwise, the likelihood of purchasing was scored as:

Likelihood of Purchasing = DI_FR4/10

9.3.2.5 Free Ridership Scoring – Direct Install Measures

For respondents who demonstrated prior experience with a measure, the scores for the prior plans and likelihood of purchasing indicator variables were averaged to develop a measure-level free ridership score to each respondent.

Major (Rebated) Measures

The calculation of a free ridership score for the major measures was based on the responses to questions about participants’ prior plans and intentions, program influence on measure selection, and program influence on timing of measure implementation.


9.3.2.6 Financial Ability and Plans and Intentions

Two indicator variables were developed based on responses to the survey questions on plans and intentions. The first corresponds to financial ability. Respondents were considered to have not been financially able to implement the efficiency measure if they answer “no” to the question below (FR1)

FR1: Would you have been able to afford to implement the [MEASURE] if the rebate was not available from the program?

The second indicator variable is related to whether or not the customer had plans to implement the efficiency measure. Respondents were considered to have had plans if they answer “yes” to the following two questions:

FR3: Were you planning to implement the [MEASURE] before you learned of I&M’s [PROGRAM]?

Respondents who were found to not have plans or the financial ability to implement the measures are deemed to not be free riders.

9.3.2.7 Program Influence

Participants were asked two questions about the direct influence of the program on their decision to implement the energy efficiency measures. Specifically, participants were asked:

FR5: Using a scale where 0 is “not at all influential” and 10 is “very influential,” how influential was the home energy assessment available through program in your decision to implement the [MEASURE]?

FR6: Using the same scale, how influential were the rebates available through program in your decision to implement the [MEASURE]?

FR7: Now we would like to know how likely you would have been to implement the [MEASURE] if the program was not available. Using a scale where 0 is “not at all likely” and 10 is “very likely”, how likely is it that you would have implemented the same [MEASURE] if you had not received the rebate through the program?

FR8: Using the same scale, how likely is it that you would have implemented the same [MEASURE] if you had not received the home energy assessment through the program?

A program influence score was developed based on these two responses in the following manner:

Program Influence = Average (MAX(FR5, FR6), (10 – MIN(FR7, FR8))) / 10

9.3.2.8 Program Influence on Project Timing


Did you purchase and install the [EFF_MEASURE] sooner than you would have if the information and financial assistance from the program had not been available?


When might you have purchased or installed the same [EFF_MEASURE] if you had not participated in the program?

Based on the responses to those questions a timing adjustment was calculated as shown in Table 9-4.



Within 6 months 1 Between 6 months and 1 year .67 In more than 1 year to 2 years .33 In two years or more 0

9.3.2.9 Free Ridership Scoring – Major (Rebated) Measures

For respondents that did not have plans or intentions, an overall free ridership score was developed based on the program influence score and timing score. An overall project free ridership score was based by combining the scores described above using the following equation:

Free Ridership = Program Influence * Timing Score



To assess participant spillover savings, survey respondents were asked whether they implemented any additional energy saving measures for which they did not receive a program incentive. Respondents were also asked to provide information on the attributes of the measures implemented for use in estimating the associated energy savings.

Participants who report implementing on one or more efficiency measures were then asked two questions for use in developing a spillover score:

SO1: On a scale of 0 to 10, where 0 represents “not at all important” and 10 represents “extremely important”, how important was your experience with [PROGRAM] in your decision to purchase the items you just mentioned?

SO2: On a scale of 0 to 10, where 0 represents “not at all likely” and 10 represents “extremely likely” how likely would you have been to make the additional purchases you just mentioned even if you had not participated in the [PROGRAM]?








Where,




The total non-participant spillover was allocated to the Home Weatherproofing Program in proportion to its share of the 2018 expenditures.



Table 9-5 summarizes the number of survey responses and average free-ridership scores by measure for the Home Weatherproofing Program.




Air sealing 26 16%

Insulation 20 16%

Duct sealing 7 14%

LED light bulbs 40 32%

Bathroom Aerators 20 2%

Kitchen Aerators 13 8%

Showerheads 18 7%

Pipe wrap 35 3%

Six customers reported installing LED light bulbs that qualified as program spillover. The spillover ratio is 2.01% for kWh savings and 2.94% for kW savings.


Table 9-6 summarizes the net ex post kWh savings of the Home Weatherproofing Program. The annual net savings totaled 489,865 kWh. The net-to-gross ratio is 93%.



Ex Ante Gross kWh

Savings

Gross Audited

kWh Savings

Gross Verified

kWh Savings

Ex Post Gross kWh

Savings

Gross Realization

Rate

Ex Post Net kWh

Savings

Net-to-Gross Ratio


kWh Savings

533,363 533,363 522,541 527,876 99% 489,865 93% 6,633,226


Table 9-7 summarizes the net ex post kW demand reduction of the Home Weatherproofing Program. The net demand reduction equaled 42.76 kW. The net-to-gross ratio is 92%.



Gross Audited kW

Savings

Gross Verified kW

Savings


Gross Realization

Rate


Net-to-Gross Ratio

39.99 39.99 39.43 46.62 117% 42.76 92%




Table 9-8 Summary of Data Collection Activities for the Home Weatherproofing Program

Data Collection Activity Mode of Data Collection Sample Size Participant survey Online and Telephone 54 Staff interviews Telephone in-depth interviews 2



Table 9-9 summarizes the service received by participants in the program. As shown, 333 homes received services through the program, all received assessments, 92% received direct install measures, and 33% received rebated measures.

Table 9-9 Summary of Services Received

Number of Homes Assessment Direct Install Rebated Measures Conversion Rate* 313 88% 92% 33% 29%


*The conversion rate is equal to the number of customers who implemented a major measure in 2018 divided by the number of participants who received an assessment in 2018. It may undercount additional conversions resulting from customers receiving a recommendation in 2017 and implementing a major measure in 2018.

Table 9-10 summarizes the recommendations participants received for the program’s rebated measures and the percent implementing the recommendations. As shown, approximately one-half of participants received any recommendations, with air sealing being the most common recommendation.

Table 9-10 Summary of Recommendations Received

Duct Sealing Air Sealing Insulation Received Any Recommendations

21% 51% 45% 51%

The number of homes receiving measures, total ex ante savings, and the average ex ante savings are summarized in Table 9-11.

Table 9-11 Summary of Measures Installed

Measure Number of Homes

Total Ex Ante Savings

Average Ex Ante Savings

Incentive Dollars

per kWh1 Air sealing 102 233,299 2,287 $0.18 Lighting 236 90,989 386 Showerheads 165 57,630 349 $0.36 Insulation 74 56,586 765 $0.87 Customer education 283 25,524 90 Faucet aerators 163 23,167 142 Duct sealing 46 23,112 502 $0.16 Pipe wrap 206 22,969 112 Water heater thermostat adjustment 1 87 87 Assessment 277 0 0 1. Incentive amounts were not included in tracking data for direct install measures


This section summarizes the findings of interviews conducted with I&M program staff for the purposes of developing internal program management perspective. ADM evaluators interviewed the I&M energy efficiency residential accounts manager to gain insights in the Weatherproofing program for 2018.

The findings or the interviews are summarized below.

The program fell short of its savings goal. Ex ante savings totaled 533,363 kWh as compared to an adjusted goal of 851,554 kWh. The number of participants increased from 297 households in 2017 to 313 households in 2018.


The program was marketed through multiple means. These included promotion in the I&M newsletter, bill inserts, digital content, targeted emails, social media, postcards, and cross-promotion through other programs. The program also ran an advertisement that generated a high number of calls.

An assessment fee was added to increase the conversion rate. In the spring, the program added a $99 fee for the energy assessment. The fee was added to increase the conversion rate and it is counted towards the rebate paid for major measures. Staff indicated they struggle with increasing the conversion rate and finding ways for customers to install major measures. Customers are receptive to the recommendations in the energy audits but do not always install measures due to upfront costs.

Improvements made to the scheduling system. Beginning in 2018, energy audits were scheduled through the call center.

No significant changes were made to QA/QC processes. There have not been any significant changes to the QA/QC procedures in 2018 and no concerns were stated.

Auditing template was changed in 2018. I&M worked with a vendor to develop an audit template for the auditors. Previously an excel workbook was used and now there is a system that is incorporated into the EECP platform. Auditors use tablets in the field during audits, which uploads data in real time to the current data system. There were no other changes to data management for the program and program staff indicated the data are kept current to manage the program.

Communication structure remained the same. Program staff did not indicate they had any concerns about the current internal communication.


ADM completed a survey of program participants to collect information regarding their experience with the Residential Home Weatherproofing Program in Indiana. The survey was administered by email and telephone. A total of 54 participating customers completed the survey.

Program Awareness and Motivation

Forty-six percent of respondents indicated that they first learned of the I&M’s Home Weatherproofing Program through a bill insert or I&M mailer and 21% learned about the program from the I&M website. The full distribution of responses is displayed in Figure 9-1 below.

Survey participants were asked if they heard of the program from another source. Most had not heard about the program from any other sources (76%), followed by also learning of the program from the I&M website (10%), the home energy report (10%), bill insert/I&M mailer (7%), the I&M newsletter (5%), newspaper ad (2%), TV ad (2%), an I&M representative (2%), community event (2%), and social media (2%).


Figure 9-1 Sources of Program Awareness

Program participants discussed reasons for participating in the program. The most frequent reason for program participation, mentioned by 74% of respondents, was to save money on energy bills. Other reasons customers participated in the program was I&M’s financial assistance for home improvements (see Table 9-12).

Table 9-12 Reasons for Participation

Reason Percent (n = 54)

To save money on energy bill(s) 74% I&M financial assistance for making the home improvements 20% Environmental reasons 2% Other 4%

Respondents provided feedback on whether they would have hired a professional contractor to perform a home audit if I&M did not offer the Home Weatherproofing Program. Most respondents (81%) indicated they probably would not have or definitely would not have had the audit without the support of the program, while 19% indicated they probably or definitely would have hired a professional contractor to perform a home audit. Among those who indicated they would have paid a professional, the amount they would have paid ranged from $100 to $1000.

Program Experiences

Participants rated their experience with scheduling the home energy assessment using a 5-point scale, from 1 (very difficult) to 5 (very easy). A large share (96%) of survey respondents indicated scheduling with either somewhat or very easy, Figure 9-2 displays the results.


Figure 9-2 Experience with Scheduling Energy Assessment

During the home energy assessment 90% of respondents indicated that the assessor asked if there were any specific issues with their home that they wanted to address, 98% received a report with energy efficiency recommendations, and 96% indicated the assessor also discussed the potential energy savings that may be achieved by implementing those recommendations (see Table 9-13).

Table 9-13 Experience with Home Energy Assessor

Did the home energy assessor? Percentage of Respondents

Ask you if there were any specific issues with your home you wanted to address? (n = 50)

Yes 90% No 10%

Provide an energy assessment report with energy efficiency recommendations? (n = 51)

Yes 98% No 2%

Discuss the potential energy savings you might achieve by implementing those recommendations in your home? (n = 48)

Yes 96% No 4%

Respondents rated how helpful the home energy report was using a five-point scale, where 1 meant “not at all helpful” and 5 meant “very helpful”. The responses, shown in Figure 9-3 below, indicate that 84% of respondents thought the report was somewhat helpful or very helpful. All four of the respondents who did not find the HER helpful either stated that no recommended actions were provided, or that they would be unable to implement most or all of the provided recommendations.


Figure 9-3 Helpfulness of Home Energy Report

Energy Efficiency Recommendations and Actions

Seventy-six percent of survey respondents indicated that they learned tips for reducing energy use during the home energy assessment, while 24% said they did not. Of those that received energy savings tips (n = 36), 86% said they have implemented some of the energy saving tips that they learned during the home energy assessment. Figure 9-4 displays the energy savings actions taken by respondents.

Figure 9-4 Energy Saving Actions Taken by Program Participants

Respondents provided additional feedback regarding the overall level of influence the program had on their decision to take energy savings actions. On a scale of 0 to 10 respondents rated the likelihood that they would have implemented the energy savings actions had they not participated in the Home Weatherproofing Program, where 0 was “not at all likely” and 10 was “extremely likely.” Respondents also rated the importance of the program in their decision to implement the


energy savings actions, were 0 was “not at all important” and 10 was “extremely important”. Figure 9-5 displays the results.

Key findings from this question include:

37% of respondents indicated there was a low likelihood they would implement the energy savings actions had they not participated in the program, 37% were somewhat likely, and 26% were highly likely to do so without the program.

95% of respondents indicated the program was very important in their decision to implement the energy savings actions provided by the home energy assessor.

Figure 9-5 Program Influence Ratings

Satisfaction

Program participants provided feedback regarding their satisfaction with various aspects of the Home Weatherproofing Program, as displayed in Figure 9-6.

Key satisfaction findings include:

Performance of the items installed – 85% of participants provided a rating of 4 or 5 on the satisfaction scale, indicating they were somewhat or very satisfied.

Effort required participating in the program – 90% of participants provided a rating of 4 or 5 on the satisfaction scale, indicating they were somewhat or very satisfied.

Information provided by I&M – 87% of participants provided a rating of 4 or 5 on the satisfaction scale, indicating they were somewhat or very satisfied.

The energy audit – 94% of participants provided a rating of 4 or 5 on the satisfaction scale, indicating they were somewhat or very satisfied.

The quality of the work – 96% of participants provided a rating of 4 or 5 on the satisfaction scale, indicated they were somewhat or very satisfied.

The program overall – 87% of participants provided a rating of 4 or 5 on the satisfaction scale, indicating they were somewhat or very satisfied with the program overall.


Figure 9-6 Program Satisfaction

Participants who indicated dissatisfaction were asked to provide additional information about the reasons for their dissatisfaction. The five clarifying responses related to concerns about the expected energy savings and the quality of the installation.

Most survey respondents (87%) indicated they were somewhat or very satisfied with I&M as their electricity service provider, followed by 8% who were neither satisfied nor dissatisfied, and 6% who were somewhat or very dissatisfied (see Table 9-14).

Table 9-14 Satisfaction with I&M as Electricity Service Provider


Very dissatisfied 4% Somewhat dissatisfied 2% Neither satisfied nor dissatisfied 8% Somewhat satisfied 28% Very satisfied 58%

Yearly Trends


There was a four percentage-point increase in the share of survey respondents who indicated they participated in the program to save money on their energy bills and a two percentage-point increase in those who participated for environmental reasons. From 2017 to 2018, there was an overall


increase in the number of survey respondents who indicated scheduling the home energy assessment was somewhat or very easy by 18 percentage points (see Figure 9-7).

Figure 9-7 2017 and 2018 Ease of Scheduling Home Energy Assessment Comparison

More survey respondents indicated they implemented energy saving tips they learned from their home energy assessment (15 percentage point increase) in 2018. In addition, there was an increase in each of the actions taken to save energy, with the greatest increase occurring among those who unplugged unused appliances (see Figure 9-8).

Figure 9-8 2017 and 2018 Implemented Energy Saving Actions

There was an increase among survey respondents who indicated the assessor asked if there were any specific issues in their homes they wanted to address by 21 percentage points (see Table 9-15).


Table 9-15 2017 and 2018 Experience with Home Energy Assessor

Response Percentage by Response

(2017)


(2018)


Ask you if there were any specific issues with your home you wanted to address? 62% 83% 21 Provide an energy assessment report with energy efficiency recommendations? 95% 93% -2 Discuss with you the potential energy savings you might achieve by implementing those recommendations in your home? 86% 87% 1


9.5.1 Conclusions

The program fell short of its savings goal but the number of participants increased. Ex ante savings totaled 533,363 kWh as compared to an adjusted goal of 851,554 kWh. The number of participants increased from 297 households in 2017 to 313 households in 2018. I&M will continue to improve the program in 2019, but no major changes are planned. Program staff will conduct more workshops with various organizations around the state (e.g., food banks, schools) in order to reach more customers.

The program was marketed through multiple means. These included promotion in the I&M newsletter, bill inserts, digital content, targeted emails, social media, post-cards, and cross-promotion through other programs. The program also ran an advertisement that generated a high number of calls.

The Home Weatherproofing program began charging a $99 assessment fee to increase the conversion rate. Staff indicated that an assessment fee was charged to increase uptake of major measures. This does not apply to the Income Qualified Weatherproofing Program. Customers who elect to do major measures have the full amount of the assessment fee applied to the rebate for the measures. The change may have been a factor in the increase in the share of households that received a rebate for a major measure (33% in 2018 vs. 14% in 2017). As such, the program is operating more efficiently with the addition of the $99 fee and the addition of the fee does not seem to have adversely impacted participation.

Most participants are satisfied with the program overall. Eighty-seven percent of participants reported that they were satisfied with the program overall. In addition, 94% of participants were very or somewhat satisfied with the energy audit and 96% were very or somewhat satisfied with the quality of work performed.



Consider additional marketing outreach efforts to increase participation. Although the program was extensively market in 2018, the nonparticipant survey found low-levels


of program awareness, including among customers with electric heating. Staff should explore the feasibility of targeting marketing to customers with electric heating. Possible approaches to this include leveraging data sources such as the Online Check-up responses provided by customers or targeting customers with energy usage during the heating season.


10 Residential Income Qualified Weatherproofing This chapter presents the results of both the impact and process evaluations of the 2018 Income Qualified Weatherproofing Program that Indiana Michigan Power (I&M) offered to its residential customers during the period of January 2018 through December 2018.










The Income Qualified Weatherproofing Program is offered to residential customers who would not otherwise have the ability to make energy efficiency improvements on their own. The program provides energy audits, direct install measures, and weatherization services to qualifying customers at no additional cost.

Eligible customers must reside in a single-family home or duplex with electric heating and have a household income below 200% of the Federal Poverty Level.

The program is implemented by I&M and I&M staff perform the energy assessments. The program works with a group of contractors to install recommended measures in customer homes.








Where x is the average kWh energy savings per participant. Without data to use as a basis for a higher value, it is typical to apply a CV of 0.5 in residential program evaluations. The resulting sample size is estimated at:

𝑆𝑆0 = �1.645 ∗ 𝐶𝐶𝐶𝐶

𝐺𝐺𝑃𝑃�

2

Where,




On-site visitations were performed by ADM to verify and supplement the provided tracking data. ADM used a sample size of 17 homes for this verification activity.


The approach ADM took in reviewing program M&V and due diligence for the Income Qualified Weatherproofing Program is consistent with that of the Home Weatherproofing Program. See section 9.2.1.2 of this document on page 147 for a discussion of this approach.







ADM surveyed a sample of program participants for developing measure-level in-service rates.

Additionally, ADM conducted on-site verification visits to weatherized homes. Data collected from this evaluation activity were also used to develop measure-level in-service rates and to verify that equipment specifications (capacity, efficiency) match the tracking data and are properly recorded.


See section 9.2 of this document on page 146 for a discussion of the approach ADM took in determining gross energy savings for the measures included in the Income Qualified Weatherproofing Program. The following measures were also included in the Income Qualified Weatherproofing Program.


Ductless Heat Pump: From the Illinois TRM, the Ductless Heat Pumps section was used to calculate energy impacts for the installation of a ductless heat pump. The following equation was used to calculate the annual kWh energy savings:

∆𝑘𝑘𝑘𝑘ℎ =𝐸𝐸𝑆𝑆𝐷𝐷𝐼𝐼ℎ𝐷𝐷𝑆𝑆𝑆𝑆 × 𝐶𝐶𝑆𝑆𝑖𝑖𝑆𝑆𝐼𝐼𝐷𝐷𝑆𝑆𝑏𝑏_ℎ𝐷𝐷𝑆𝑆𝑆𝑆 × 𝐸𝐸𝐹𝐹𝐸𝐸𝐻𝐻_ℎ𝐷𝐷𝑆𝑆𝑆𝑆 × � 1

𝐻𝐻𝑆𝑆𝑃𝑃𝐹𝐹_𝐷𝐷𝑥𝑥𝐷𝐷𝑈𝑈𝑆𝑆 − 1𝐻𝐻𝑆𝑆𝑃𝑃𝐹𝐹_𝐷𝐷𝐷𝐷�

1000

+𝐶𝐶𝑆𝑆𝑖𝑖𝑆𝑆𝐼𝐼𝐷𝐷𝑆𝑆𝑏𝑏_𝐼𝐼𝐷𝐷𝐷𝐷𝑆𝑆 × 𝐸𝐸𝐹𝐹𝐸𝐸𝐻𝐻_𝐼𝐼𝐷𝐷𝐷𝐷𝑆𝑆 × � 1

𝑆𝑆𝐸𝐸𝐸𝐸𝐺𝐺_𝐷𝐷𝑥𝑥𝐷𝐷𝑈𝑈𝑆𝑆 − 1𝑆𝑆𝐸𝐸𝐸𝐸𝐺𝐺_𝐷𝐷𝐷𝐷�

1000

Where:

Elecheat = Existing building is electrically heated, 1

Capacity_heat = Heating capacity of ductless heat pump

EFLH_heat = Equivalent full load hours for heating

HSPF_exist = Heating system performance factor for first 6 years, 3.412

HSPF_base = Heating system performance factor for next 12 years, 3.41. Replaces HSPF_exist in equation above

HSPF_ee = Heating system performance factor for new equipment

Capacity_cool = Cooling capacity of ductless heat pump

SEER_exist = SEER rating for existing equipment for first 6 years, 9.3

SEER_base = SEER rating for next 12 years, 13. Replaces SEER_exist in equation above

SEER_ee = SEER rating for new equipment

HSPF_cooling = Equivalent full load hours for cooling


∆𝑘𝑘𝑘𝑘 =𝐶𝐶𝑆𝑆𝑖𝑖𝑆𝑆𝐼𝐼𝐷𝐷𝑆𝑆𝑏𝑏_𝐼𝐼𝐷𝐷𝐷𝐷𝑆𝑆 × � 1

𝐸𝐸𝐸𝐸𝐺𝐺_𝐷𝐷𝑥𝑥𝐷𝐷𝑈𝑈𝑆𝑆 − 1𝐸𝐸𝐸𝐸𝐺𝐺_𝐷𝐷𝐷𝐷�

1000× 𝐶𝐶𝐹𝐹

Where:

EER_exist = Energy efficiency ratio of existing equipment for first 6 years. EER = (-0.02 * SEER2) + (1.12 * SEER)

EER_base = Energy efficiency ratio for next 12 years

EER_ee = Energy efficiency ratio for new equipment


CF = Summer peak coincident factor, 0.72

Refrigerators: From the Indiana TRM, the Refrigerator Replacement (Low Income, Early Replacement) section was used to calculate energy impacts for the installation of a new efficient refrigerator. The following equation was used to calculate the annual kWh energy savings:

∆𝑘𝑘𝑘𝑘ℎ = 𝑈𝑈𝐸𝐸𝐶𝐶𝑏𝑏𝑚𝑚𝑖𝑖𝑏𝑏𝑡𝑡𝑖𝑖𝑓𝑓𝑖𝑖 − 𝑈𝑈𝐸𝐸𝐶𝐶𝐵𝐵𝐵𝐵

Where:

UECexisting = Unit energy consumption of existing refrigerator, 1,696

UECES = Unit energy consumption of efficient refrigerator, 397

UECBASE = Unit energy consumption of baseline refrigerator, 453


∆𝑘𝑘𝑘𝑘 =∆𝑘𝑘𝑘𝑘ℎ8760

× 𝐸𝐸𝑃𝑃𝐹𝐹 × 𝐸𝐸𝑆𝑆𝑃𝑃𝐹𝐹

Where:

TAF = Temperature adjustment factor, 1.21

LSAFexist = Load shape adjustment factor for existing unit, 1.063

LSAFnew = Load shape adjustment factor for existing unit, 1.124

Heat Pump Water Heater: From the Indiana TRM, the Heat Pump Water Heaters section was used to calculate energy impacts for the installation of a heat pump water heater. The following equation was used to calculate the annual kWh energy savings:

∆𝑘𝑘𝑘𝑘ℎ = 𝑘𝑘𝑘𝑘ℎ𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏 ×𝐶𝐶𝐺𝐺𝑃𝑃𝑓𝑓𝑏𝑏𝑙𝑙 − 𝐶𝐶𝐺𝐺𝑃𝑃𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏

𝐶𝐶𝐺𝐺𝑃𝑃𝑓𝑓𝑏𝑏𝑙𝑙+ 𝑘𝑘𝑘𝑘ℎ𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑖𝑖𝑓𝑓𝑖𝑖 − 𝑘𝑘𝑘𝑘ℎℎ𝑏𝑏𝑏𝑏𝑡𝑡𝑖𝑖𝑓𝑓𝑖𝑖

Where:


COPnew = Coefficient of performance of heat pump water heater, 2.0

COPbase = Coefficient of performance of standard water heater, 0.904

kWhcooling = Cooling savings from conversion of heat in home to water heat, 180

Kwhheating = Heating cost from conversion of heat in home to water heat




× 𝐶𝐶𝐹𝐹

Where:

Hours = Equivalent full load hours of hot water heater, 2,533


Water Heater Wrap: From the Indiana TRM, the Water Heater Wrap section was used to calculate energy impacts for the installation of water heater insulation. The following equation was used to calculate the annual kWh energy savings:

∆𝑘𝑘𝑘𝑘ℎ = 𝑘𝑘𝑘𝑘ℎ𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏 ×𝐸𝐸𝐹𝐹𝑓𝑓𝑏𝑏𝑙𝑙 − 𝐸𝐸𝐹𝐹𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏

𝐸𝐸𝐹𝐹𝑓𝑓𝑏𝑏𝑙𝑙

Where:


EFnew = Assumed efficiency of electric tank with wrap, 0.88

EFbase = Assumed efficiency of electric tank without wrap, 0.86


∆𝑘𝑘𝑘𝑘 =∆𝑘𝑘𝑘𝑘ℎ8760

HVAC Tune-Up: From the Indiana TRM, the Residential HVAC Maintenance/Tune-Up section was used to calculate energy impacts for HVAC maintenance. The following equation was used to calculate the annual kWh energy savings:

∆𝑘𝑘𝑘𝑘ℎ𝐶𝐶𝐵𝐵𝐶𝐶 = 𝐸𝐸𝐹𝐹𝐸𝐸𝐻𝐻𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 × 𝐵𝐵𝐸𝐸𝑈𝑈𝐻𝐻𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 ×1

𝑆𝑆𝐸𝐸𝐸𝐸𝐺𝐺𝐶𝐶𝐵𝐵𝐶𝐶 × 1000× 𝑀𝑀𝐹𝐹𝑏𝑏

∆𝑘𝑘𝑘𝑘ℎ𝐵𝐵𝐵𝐵𝐷𝐷𝑁𝑁 = �𝐸𝐸𝐹𝐹𝐸𝐸𝐻𝐻𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 × 𝐵𝐵𝐸𝐸𝑈𝑈𝐻𝐻𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 ×1

𝑆𝑆𝐸𝐸𝐸𝐸𝐺𝐺𝐵𝐵𝐵𝐵𝐷𝐷𝑁𝑁+ 𝐸𝐸𝐹𝐹𝐸𝐸𝐻𝐻ℎ𝑏𝑏𝑏𝑏𝑡𝑡 × 𝐵𝐵𝐸𝐸𝑈𝑈𝐻𝐻ℎ𝑏𝑏𝑏𝑏𝑡𝑡 ×

1𝐻𝐻𝑆𝑆𝑃𝑃𝐹𝐹𝐵𝐵𝐵𝐵𝐷𝐷𝑁𝑁

� ×𝑀𝑀𝐹𝐹𝑏𝑏

1000

Where:

EFLHcool = Equivalent full load cooling hours

BTUHcool = Cooling capacity of equipment, 28,994 if not available

SEERCAC = SEER efficiency of existing central air conditioning unit, 11.15 if not available


MFe = Maintenance energy savings factor, 0.05

SEERASHP = SEER efficiency of existing heat pump, 11.15 if not available

EFLHheat = Equivalent full load heating hours

BTUHheat = Heating capacity of equipment

HSPFASHP = Heating season performance factor of heat pump, 6.8 if unavailable


∆𝑘𝑘𝑘𝑘 = 𝐵𝐵𝐸𝐸𝑈𝑈𝐻𝐻𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 ×1

𝐸𝐸𝐸𝐸𝐺𝐺 × 1000× 𝑀𝑀𝐹𝐹𝐷𝐷 × 𝐶𝐶𝐹𝐹

Where:

EER = EER efficiency of existing unit, SEER * 0.9

MFD = Maintenance demand reduction factor, 0.05



The estimated gross impacts resulting from the PY2018 Home Weatherproofing program are summarized in Table 10-1.







294,440 294,440 273,262 299,907 102%






24.62 24.62 23.75 28.69 117%


Table 10-2 summarizes the gross kWh savings of the Income Qualified Weatherization Program.


Table 10-2 Measure Level Gross kWh Savings Summary


Gross Audited kWh

Savings

Gross Verified kWh

Savings


Realization Rate

Bath Faucet Aerator 810 810 675 1,120 138% Kitchen Faucet Aerator 6,683 6,683 6,683 5,663 85% Pipe Insulation 5,055 5,055 5,055 5,224 103% Showerhead 11,086 11,086 11,086 11,353 102% Duct Sealing 7,854 7,854 7,854 7,940 101% LED Bulb 20,189 20,189 17,436 17,066 85% Air Infiltration 146,320 146,320 128,030 127,621 87% Ceiling Insulation 28,421 28,421 28,421 28,421 100% Wall Insulation 6,133 6,133 6,133 6,133 100% Customer Education 631 631 631 842 133% Ductless Heat Pump 20,802 20,802 20,802 28,819 139% Refrigerators 29,274 29,274 29,274 44,166 151% Heat Pump Water Heater 10,677 10,677 10,677 15,032 141% Water Heater Wrap 241 241 241 236 98% HVAC Tune-up 264 264 264 270 102% Total 294,440 294,440 273,262 299,907 102%


Table 10-3 summarizes the gross kW demand reductions of the Income Qualified Weatherization Program.

Table 10-3 Measure Level Gross Demand Reduction Summary


Gross Audited kW

Savings

Gross Verified kW

Savings


Realization Rate

Bath Faucet Aerator 0.18 0.18 0.15 0.11 59% Kitchen Faucet Aerator 0.15 0.15 0.15 0.26 172% Pipe Insulation 0.60 0.60 0.60 0.60 99% Showerhead 0.52 0.52 0.52 0.56 107% Duct Sealing 2.98 2.98 2.98 2.98 100% LED Bulb 0.85 0.85 0.74 2.35 276% Air Infiltration 5.82 5.82 5.09 5.08 87% Ceiling Insulation 3.10 3.10 3.10 3.10 100% Wall Insulation 0.62 0.62 0.62 0.66 107% Customer Education 0.07 0.07 0.07 0.05 66% Ductless Heat Pump 1.06 1.06 1.06 4.28 404% Refrigerators 6.66 6.66 6.66 6.37 96% Heat Pump Water Heater 1.46 1.46 1.46 2.05 141% Water Heater Wrap 0.03 0.03 0.03 0.03 90% HVAC Tune-up 0.53 0.53 0.53 0.24 45% Total 24.62 24.62 23.75 28.69 117%




The NTG ratio for the Income Qualified Weatherproofing Program was assumed to be 1.0 in line with common practice for estimation of low-income program net savings.18



Table 10-4 summarizes the net ex post kWh savings of the Residential Income Qualified Weatherproofing Program. The annual net savings totaled 299,907 kWh. The net-to-gross ratio is 100%.


Ex Ante Gross kWh

Savings

Gross Audited

kWh Savings

Gross Verified

kWh Savings

Ex Post Gross kWh

Savings

Gross Realization

Rate

Ex Post Net kWh

Savings

Net-to-Gross Ratio


kWh Savings

294,440 294,440 273,262 299,907 102% 299,907 100% 4,231,179


Table 10-5 summarizes the net ex post kW demand reduction of the Residential Income Qualified Weatherproofing Program. The net demand reduction equaled 28.69 kW. The net-to-gross ratio is 100%.



Gross Audited kW

Savings

Gross Verified kW

Savings


Gross Realization

Rate


Net-to-Gross Ratio

24.62 24.62 23.75 28.69 117% 28.69 100%



18 See Violette and Rathbun, Chapter 21: Estimating Net Savings: Common Practices. The Uniform Methods Project: Methods for Determining Energy Efficiency Savings for Specific Measures, available electronically at https://www.nrel.gov/docs/fy17osti/68578.pdf, p. 45



Table 10-6 Summary of Data Collection Activities for the Income Qualified Weatherproofing Program

Data Collection Activity Mode of Data Collection Sample Size Participant survey Telephone 29 Staff interviews Telephone in-depth interviews 1



As shown in Table 10-7, 116 homes received services through the program in 2018. Seventy-two percent of homes received air sealing, 6% received duct sealing, and 63% received insulation.

Table 10-7 Summary of Services Received

Number of Homes Assessment Direct Install

Rebated Measures

117 78% 69% 84%

Air sealing accounted for the largest share of program savings (58%), as shown in Table 10-8.

Table 10-8 Summary of Measure Savings

Measure Number

of Homes

Total Ex Ante

Savings

Average Ex Ante Savings

Air sealing 71 146,320 2,061 Refrigerator 33 29,274 887 Insulation 59 28,421 482 Ductless heat pump 11 20,802 1,891 Lighting 52 20,189 388 Showerheads 54 17,219 319 Heat pump water heater 8 10,677 1,335 Duct sealing 13 7,854 604 Faucet aerators 41 7,492 183 Pipe wrap 50 5,055 101 Customer education 7 631 90 HVAC tune-up 3 264 88 Tank wrap 3 241 80 Assessment 91 0 0 Health and safety 43 0 0


This section summarizes the findings of interviews conducted with I&M program staff for the purposes of developing internal program management perspective. ADM evaluators interviewed


the I&M energy efficiency residential accounts manager to gain insights in the Income Qualified Weatherproofing program for 2018.

The findings or the interviews are summarized below.

The Income Qualified Weather Proofing Program fell slightly short of ex ante savings meeting the program goal The Income Qualified Weatherproofing Program ex ante savings amounted to 91% of the adjust plan goal.

There was a significant effort to engage community stakeholders in the Income Qualified Weatherproofing in the last program year. I&M hired a residential liaison manager in March, who is managing programs for income qualified customers. Staff reported that participation in Income Qualified Weatherproofing remained relatively the same from previous program years.

I&M is working to develop partnerships with additional community agencies and their combined reach. I&M will continue to improve the program in 2019, though no major changes are planned. Program staff will conduct more workshops with various organizations around the state (e.g., food banks, schools) in order to reach the target customer base.


ADM completed a survey of program participants to collect information regarding their experience with the Income Qualified Residential Home Weatherproofing Program in Indiana. The survey was administered by email. A total of 29 participants completed the survey.

Program Awareness and Motivation

Participants provided feedback regarding how they first learned of I&M’s Income Qualified Home Weatherproofing Program. The results reflect the diverse range of marketing efforts undertaken to promote the program as no single source of awareness was endorsed by a large share of participants.

As shown in Figure 10-1, 18% of respondents indicated they learned about the program at a community event and 18% learned about it from a bill insert or I&M mailing. Other frequently mentioned sources of awareness were learning of the program from a friend or relative (14%), a home energy report (11%), an I&M newsletter (11%), and a bill insert (11%).

Participants indicated other sources where they had heard about the program and most had not heard about the program from other sources (67%). Respondents who had heard of the program from a secondary source cited bill inserts (14%), friends or relatives (5%), and an I&M representative (5%).


Figure 10-1 Sources of Program Awareness

Program participants discussed reasons for participating in the program. The most frequent reason for program participation, mentioned by 59% of respondents, was to save money on energy bills. Other reasons customers participated in the program included they wanted to know what measures were available, interested in what improvements their home needed, energy assistance, and mold in their basement.

Table 10-9 Reasons for Participation


To save money on energy bill 59% Other 15% I&M financial assistance 22% Environmental reasons 4%

Program Experiences

Participants rated the scheduling of the home energy assessment on a scale of 1 to 5, where 1 is very difficult and 5 is very easy. Most survey respondents (87%) found scheduling their home energy assessment as somewhat or very easy.


Figure 10-2 Experience with Scheduling Energy Assessment

During the home energy assessment 73% of respondents indicated that the assessor asked if there were any specific issues with their home that they wanted to address, 88% of the survey respondents received a report with energy efficiency recommendations, and 95% indicated the assessor also discussed the potential energy savings that may be achieved by implementing those recommendations (see Table 10-10).


Did the home energy assessor? Percentage of Respondents

Ask you if there were any specific issues with your home you wanted to address? (n = 22)

Yes 73% No 27%

Provide an energy assessment report with energy efficiency recommendations? (n = 26)

Yes 88% No 12% Discuss the potential energy savings you might achieve

by implementing those recommendations in your home? (n = 22)

Yes 95% No 5%

Respondents rated how helpful the home energy report was using a five-point scale, where 1 meant not at all helpful and 5 meant very helpful. Eighty-seven percent thought the report was somewhat helpful or very helpful (Figure 10-3).


Figure 10-3 Helpfulness of Home Energy Report

Energy Efficiency Recommendations and Actions

Eighty-five percent of survey respondents indicated they received tips for reducing energy use during the home energy assessment. Of those that received energy savings tips (n = 22), 90% said they made changes in their home to conserve energy. Figure 10-4 displays the energy savings actions taken by respondents. Participants most frequently reported turning off lights when they leave the room more frequently.

Figure 10-4 Energy Saving Actions Taken by Program Participants

Respondents provided additional feedback regarding the overall level of influence the program had on their decision to take energy savings actions. Using a scale of 0 to 10, respondents rated the likelihood that they would have implemented the energy savings actions had they not participated in the Income Qualified Home Weatherproofing Program, where 0 was not at all likely and 10 was extremely likely. Respondents also rated the importance of the program in their decision to implement the energy savings actions, were 0 was not at all important and 10 was extremely important. These ratings were categorized and displayed in Figure 10-5.

Key findings from this question include:


53% of respondents indicated they were not very likely to implement the energy savings actions had they not participated in the program, 16% were somewhat likely, and 32% were very likely to do so without the program.

77% of respondents indicated the program was very important in their decision to implement the energy savings actions provided by the home energy assessor, 18% indicated it was somewhat important, and 5% indicated it was not very important.

Figure 10-5 Program Influence Ratings

Satisfaction

Program participants provided feedback regarding their satisfaction with various aspects of the Income Qualified Home Weatherproofing Program, as displayed in Figure 10-6.

Key satisfaction findings include:

Performance of the items installed – 89% of participants provided a rating of 4 or 5 on the satisfaction scale, indicating they were somewhat to very satisfied.

Effort required for the application process– 87% of participants provided a rating of 4 or 5 on the satisfaction scale, indicating they were somewhat to very satisfied.

Information provided by I&M – 77% of participants provided a rating of 4 or 5 on the satisfaction scale, indicating they were somewhat to very satisfied.

The energy audit – 90% of participants provided a rating of 4 or 5 on the satisfaction scale, indicating they were somewhat to very satisfied.

The quality of the work – 87% of participants provided a rating of 4 or 5 on the satisfaction scale, indicating they were somewhat to very satisfied.

The program overall – 87% of participants provided a rating of 4 or 5 on the satisfaction scale, indicating they were somewhat to very satisfied.

I&M as their service provider – 90% of participants provided a rating of 4 or 5 on the satisfaction scale, indicating they were somewhat to very satisfied and 10% provided a rating of 3.


Figure 10-6 Program Satisfaction

Yearly Trends


There was a two percentage point increase among survey respondents who indicated they participated in the program to save money on their energy bills and a three percentage point decrease in those who participated for environmental reasons. From 2017 to 2018, there was an overall increase in the number of survey respondents who indicated scheduling the home energy assessment was somewhat or very easy by 18 percentage points (see Figure 10-7).

Figure 10-7 2017 and 2018 Ease of Scheduling Home Energy Assessment Comparison


Fewer survey respondents indicated they implemented energy saving tips they learned from their home energy assessment (9 percentage point decrease) in 2018. In addition, there was an increase among two of the energy saving actions that survey respondents indicated they implemented, with the greatest increase occurring among those who installed a water heater tank wrap (see Figure 10-8).

Figure 10-8 2017 and 2018 Implemented Energy Saving Actions Comparison

There was an eight percentage-point increase in the share of survey respondents who indicated the assessor provided an energy assessment report with energy efficiency recommendations (see Table 10-11).


Response Percentage by Response

(2017)


(2018)


Ask you if there were any specific issues with your home you wanted to address? 65% 55% -9 Provide an energy assessment report with energy efficiency recommendations? 71% 79% 8 Discuss with you the potential energy savings you might achieve by implementing those recommendations in your home? 81% 75% -6


10.5.1 Conclusions

Many of the process evaluation findings and recommendations for the Home Weatherproofing Program summarized in Section 9.5 are applicable to the Income Qualified Weatherproofing Program. The findings specific to the Income Qualified Home Weatherproofing program are summarized below.


The program fell short of its savings goals. The program achieved 91% of its savings goal for the year. Staff have focused on increasing program engagement and awareness. For example, I&M hired a residential liaison manager in March, who is managing programs for income qualified customers. Although staff reported that participation in the Income Qualified Weatherization Program remained relatively the same from previous program years, the development of relationships take time and the impact of this change may become more apparent in 2019. Additionally, program staff will conduct more workshops with various organizations around the state (e.g., food banks, schools) in order to reach more customers in 2019.

The participation process is working well for participants. Eighty-seven percent of participants reported that scheduling the home energy assessment was very or somewhat easy. Eighty-eight percent of respondents reported that they received an energy efficiency report and 95% reported that the assessor discussed savings that may result from implementing the recommendations. Most (87%) respondents thought the report was helpful.

Most participants were satisfied with the program overall. Eighty-seven participants were very or somewhat satisfied with the program overall. Additionally, 89% of participants were very or somewhat satisfied with the performance of the items installed, 90% were very or somewhat satisfied with the energy audit, and 90% were very or somewhat satisfied with I&M as their service provider.


11 Schools Energy Education This chapter presents the results of both the impact and process evaluations of the 2018 Schools Energy Education Program that Indiana Michigan Power (I&M) offered to its residential customers during the period of January 2018 through December 2018.




Characterize barriers to participation;


Assess satisfaction among participants; and



The Schools Energy Education Program (SEE) is an educational offering targeting 5th grade elementary school students, their parents/guardians, and their teachers, in the I&M service territory. The program provides school teachers with an energy education curriculum as well as energy conservation kits to distribute to their students. The energy conservation kits contain a variety of low-cost energy efficiency measures, as follows:

Three (3) 9W LEDs;

One (1) .5W LED night light;

One (1) 1.5 GPM Kitchen faucet aerator;

Two (2) 1.5 GPM High-efficiency showerhead; and

One (1) Filter Tone Alarm.

Additionally, the energy conservation kits include informational literature detailing energy-saving tips, measure installation instructions, and information on I&M efficiency programs. The kits also include supplies that students can use to test their home energy use and make minor improvements to the home’s energy management, including a flow rate test bag, and a digital thermometer for testing hot water and fridge/freezer temperature.

The program is designed to not only result in the installation of the low-cost energy conservation kit measures, but also to improve student and parent/guardian awareness of energy-saving behaviors and equipment, as well as to incorporate energy education into the elementary school curriculum.





The first aspect of conducting measurements of program activity was to verify that the tracking data report of participants and measures is accurate. To this end, ADM reviewed the program data to verify that the fields required for performing the evaluation were tracked and populated (i.e., the data was not missing) and that the values were reasonable. ADM took several steps in verifying the number of kits distributed and kit measures installed, which consisted of the following:

Validating program tracking data provided by Resource Action Programs, the program implementation contractor, by checking for duplicate or erroneous entries;

Verifying that the conservation kits were distributed to instructors and subsequently distributed to students as per the agreed upon process between Resource Action Programs and I&M; and

Examining the parent/guardian survey data collected by Resource Action Programs during 2018 to verify that customers listed in the program tracking database did indeed participate and that the number of measures reported in the kit were received.

ADM also performed a review of the deemed savings estimates used to calculate ex ante energy impacts for installed kit measures. This evaluation activity served to verify that the ex ante calculations were consistent with algorithms and values specified in the Indiana TRM.

11.2.1.1.1 Number of Kits Mailed

The total number of kits distributed to instructors and students during PY2018 was determined by (1) reviewing the program tracking system and related documentation from I&M and (2) examining the parent/guardian survey data collected by Resource Action Programs during 2018.

Resource Action Programs administered a survey to 3,960 parents/guardians during 2018. The survey served to verify that the student had received the kit and informed the measure-level installation rate estimates.

Based on the review of the main program tracking system and all related documentation from I&M and Resource Action Programs, ADM determined the total number of kits distributed to students during PY2018 was 11,000.




Program applications and supporting documentation; and

Participant survey data.


The program implementation contractor conducted surveys with a sample of 3,960 participating parents and guardians whose children received energy conservation kits through the program. ADM referenced the collected survey data in the development of measure-level in-service rates.

The collected survey data was also used in estimating the percentage of program participants with electric hot water heaters in their homes. The resulting estimate was used to adjust the ex post energy savings and demand reduction attributed to the water heater measures included in the SEE kits during 2018 (i.e., kitchen faucet aerators and high-efficiency showerheads).


Gross energy savings and demand reductions for the Schools Energy Education Program were calculated (by kit measure) using the appropriate methodologies and deemed savings algorithms specified in the Indiana TRM. ADM reviewed the TRM and assessed the appropriateness of the engineering algorithms.

The following sections describe the specific algorithms and inputs used to calculate energy impacts for each measure in the kit.


∆𝑘𝑘𝑘𝑘ℎ =(𝑘𝑘𝑆𝑆𝑆𝑆𝑆𝑆𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵 − 𝑘𝑘𝑆𝑆𝑆𝑆𝑆𝑆𝐵𝐵𝐸𝐸𝐸𝐸)

1,000× 𝐸𝐸𝑆𝑆𝐺𝐺 × 𝐻𝐻𝐷𝐷𝑢𝑢𝑆𝑆𝑈𝑈 × (1 + 𝑘𝑘𝐻𝐻𝐹𝐹𝐵𝐵)

Where:




Hours = Average hours of use per year, 1,135

WHFE = Waste Heat Factor for Energy to account for HVAC interactions with efficient lighting, dependent on location of the schools whose teachers received program kits

Following this, ADM calculated the peak demand reduction using the following TRM defined equation:

∆𝑘𝑘𝑘𝑘 =(𝑘𝑘𝑆𝑆𝑆𝑆𝑆𝑆𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵 − 𝑘𝑘𝑆𝑆𝑆𝑆𝑆𝑆𝐿𝐿𝐵𝐵𝐷𝐷)

1,000× 𝐸𝐸𝑆𝑆𝐺𝐺 × 𝐶𝐶𝐹𝐹 × (1 + 𝑘𝑘𝐻𝐻𝐹𝐹𝐷𝐷)

Where:






WHFD = Waste Heat Factor for Demand to account for HVAC interactions with efficient lighting, dependent on location of the schools whose teachers received program kits

Table 11-1 below from the 2015 Indiana TRM was used to determine the baseline wattages of the LED lamps included in the energy efficiency kits.


Efficient Technology WattEFF Watts (baseline)

CFL 15W or less 3.05 * WattEFF 16W - 20W 3.00 * WattEFF


LED 9W or less 3.38 * WattEFF 10W - 17W 3.41 * WattEFF


LED Night Light: From the Indiana TRM, the LED Night Lights section was used to calculate energy impacts for the installation of LED night lights. The following equation was used to calculate the annual kWh energy savings:

∆𝑘𝑘𝑘𝑘ℎ =(𝑘𝑘𝑆𝑆𝑆𝑆𝑆𝑆𝑈𝑈𝐵𝐵𝐵𝐵𝐵𝐵𝐵𝐵 − 𝑘𝑘𝑆𝑆𝑆𝑆𝑆𝑆𝑈𝑈𝐿𝐿𝐵𝐵𝐷𝐷)

1,000× 𝐸𝐸𝑆𝑆𝐺𝐺 × 𝐻𝐻𝐺𝐺𝑈𝑈𝐺𝐺𝑆𝑆

Where:

WattsBASE = Wattage of incandescent night light, 5

WattsEFF = Wattage of LED night light, 0.5

ISR = In Service Rate or percentage of distributed units that are installed, as determined through analysis of customer survey response data

HOURS = Average hours of use per year, 2,920

The Indiana TRM attributes no peak kW reduction to the installation of LED night lights.

Low Flow Kitchen Faucet Aerator: From the Indiana TRM, the Low Flow Faucet Aerator (Time of Sale or Early Replacement) section was used to calculate energy impacts for the installation of energy saving kitchen faucet aerators. The following equation was used to calculate the annual kWh energy savings:



𝐺𝐺𝐸𝐸 × 3,412

Where:


GPMbase = Gallons per Minute of baseline faucet, 2.44

GPMlow = Gallons per Minute of low flow faucet, 1.5

MPD = Average minutes per day used by each faucet in home, 4.5

PH = Average number of people per household, 2.64 (single family)

FH = Average number of faucets per household, dependent on sink and housing type; 1.0 (kitchen)

DR = Percentage of water flowing down drain, 50%

Tmix = Average mixed temperature of water used by faucet, 93oF

Tin = Assumed temperature of water entering house, dependent on climate, dependent on location of the schools whose teachers received program kits




𝐺𝐺𝐸𝐸 × 3,412

Where:


GPMbase = Gallons per Minute of baseline faucet, 2.44

GPMlow = Gallons per Minute of low flow faucet, 1.5

DR = Percentage of water flowing down drain, 50%

Tmix = Average mixed temperature of water used by faucet, 93oF





High-efficiency Showerhead: From the Indiana TRM, the Low-Flow Showerhead (Time of Sale or Early Replacement) section was used to calculate energy impacts for the installation of high-efficiency showerheads. The following equation was used to calculate the annual kWh energy savings:


𝐺𝐺𝐸𝐸 × 3,412

Where:

ISR = In-Service Rate or percentage of units that are installed, as determined through analysis of customer survey response data



MS = Average minutes per shower, 7.8

SPD = Average showers per day, 0.6


SH = Average number of showerheads per household, as determined through analysis of customer survey response data; 1.65






𝐺𝐺𝐸𝐸 × 3,412

Where:









Filter Tone Alarm: The Indiana TRM does not dedicate a savings section to filter tone alarms. I&M has previously provided ADM with a savings algorithm for the filter tone alarm measure based on the 2014 evaluation of this program.

The algorithms provided to ADM that were used to estimate the annual energy impacts for the installation of filter tone alarms are as follows:

∆𝑘𝑘𝑘𝑘ℎ𝐶𝐶𝑏𝑏𝑓𝑓𝑡𝑡𝑁𝑁𝑏𝑏𝑐𝑐𝐵𝐵𝑖𝑖𝑁𝑁𝐶𝐶𝑐𝑐𝑓𝑓𝑚𝑚𝑖𝑖𝑡𝑡𝑖𝑖𝑐𝑐𝑓𝑓𝑏𝑏𝑁𝑁 = 𝐹𝐹𝐸𝐸𝐻𝐻𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 × 𝐵𝐵𝑆𝑆𝑢𝑢𝐻𝐻𝐶𝐶𝐵𝐵𝐶𝐶 × (1/𝑆𝑆𝐸𝐸𝐸𝐸𝐺𝐺)/1000 × 𝐸𝐸𝐹𝐹𝑏𝑏𝑐𝑐𝑏𝑏𝑐𝑐

∆𝑘𝑘𝑘𝑘ℎ𝐷𝐷𝑏𝑏𝑏𝑏𝑡𝑡𝑁𝑁𝐻𝐻𝑚𝑚𝑝𝑝 = 𝐹𝐹𝐸𝐸𝐻𝐻𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 × 𝐵𝐵𝑆𝑆𝑢𝑢𝐻𝐻𝐶𝐶𝐵𝐵𝐶𝐶 × (1/𝑆𝑆𝐸𝐸𝐸𝐸𝐺𝐺)/1000 + 𝐹𝐹𝐸𝐸𝐻𝐻ℎ𝑏𝑏𝑏𝑏𝑡𝑡 × 𝐵𝐵𝑆𝑆𝑢𝑢𝐻𝐻𝐷𝐷𝑁𝑁 × (1/𝐻𝐻𝑆𝑆𝑃𝑃𝐹𝐹)/1000

× 𝐸𝐸𝐹𝐹𝑏𝑏𝑐𝑐𝑏𝑏𝑐𝑐

∆𝑘𝑘𝑘𝑘𝐶𝐶𝑏𝑏𝑓𝑓𝑡𝑡𝑁𝑁𝑏𝑏𝑐𝑐𝐵𝐵𝑖𝑖𝑁𝑁𝐶𝐶𝑐𝑐𝑓𝑓𝑚𝑚𝑖𝑖𝑡𝑡𝑖𝑖𝑐𝑐𝑓𝑓𝑏𝑏𝑁𝑁 = 𝐵𝐵𝑆𝑆𝑢𝑢𝐻𝐻𝐶𝐶𝐵𝐵𝐶𝐶 × (1/𝐸𝐸𝐸𝐸𝐺𝐺)/1000 × 𝐸𝐸𝐹𝐹𝑏𝑏𝑐𝑐𝑏𝑏𝑐𝑐 × 𝐶𝐶𝐹𝐹

∆𝑘𝑘𝑘𝑘𝐷𝐷𝑏𝑏𝑏𝑏𝑡𝑡𝑁𝑁𝐻𝐻𝑚𝑚𝑝𝑝 = 𝐵𝐵𝑆𝑆𝑢𝑢𝐻𝐻𝐷𝐷𝑁𝑁 × (1/𝐸𝐸𝐸𝐸𝐺𝐺)/1000 × 𝐸𝐸𝐹𝐹𝑏𝑏𝑐𝑐𝑏𝑏𝑐𝑐 × 𝐶𝐶𝐹𝐹

Where:

EFelec = Efficiency savings for natural gas furnace

SEER = Seasonal energy efficiency ratio

EER = Energy efficiency ratio

BtuHCAC = Size of central AC units

HSPF = Heating season performance factor

BtuHHP = Size of heat pump

CF = Summer peak coincidence factor for heat pump/central AC

FLHcool = Full load cooling hours

FLHheat = Full load heating hours

ADM previously reviewed the input assumptions used by the Statewide Core evaluation for this measure and found most of the assumptions to be reasonable. However, ADM found that the EFelec value of 0.035 was only applicable to units with poorly maintained filters, and the original study that developed this value applied a 1:1 ratio of properly maintained to poorly maintained appliances. This results in an average EFelec of 0.0175 for a whole population, rather than the 0.035 value assumed for the Statewide Core evaluation. ADM applied this adjusted EFelec to the savings algorithm for this measure, resulting in 18.65 kWh, and 0.036 kW per filter tone alarm.



Ex post annual kWh savings and kW demand reductions resulting from the Schools Energy Education Program were calculated by applying the estimated measure-level installation rates of kit measures to the calculated measure-level gross energy impacts.

The program relies on direct installation by the participant, and some of the items may have been uninstalled or perhaps were never installed by students and their parents/guardians upon receiving the kit. In the development of measure-level ISRs for the program, ADM referenced the collected survey data provided by Resource Action Programs. These surveys were distributed by students’ teachers for students and their parents/guardians to fill out after the kit measures had been installed. The ISR estimates were based on a sample of 3,960 participating parents and guardians whose children received the energy conservation kits through the program.

Table 11-2 below displays the installation rates developed from the collected survey data.

Table 11-2 Installation Rates per SEE Measure

Measure Type In-service Rate

Filter Alarm 25% LED Nightlight 85% Showerhead 22% Kitchen Aerator 20% 9W LED 62%


The ex post gross energy and demand impacts resulting from the 2018 Schools Energy Education Program are reported in the following sections.


The estimated annual gross energy savings resulting from the Schools Energy Education Program are summarized in Table 11-3

Table 11-3 below. The gross kWh realization rate for the program is 130%.







2,140,600 2,140,600 719,747 2,785,095 130%

Table 11-4 below shows the measure-level estimated annual gross energy savings resulting from the program. ADM calculated ex post energy impacts for each kit measure using deemed values


and algorithms from the Indiana TRM and then factored the resulting gross energy impacts by the estimated measure-level installation rates. For kitchen faucet aerators and high-efficiency showerheads, gross energy impacts were also adjusted by the estimated percentage of participants with an electric hot water heater in their homes.19

The primary contributor to ex post gross program savings being higher than ex ante program savings is the low flow showerhead measure. The low flow showerhead gross realization rate equaled 150% and the measure accounted for approximately 63% of program ex ante savings.

Table 11-4 Ex Post Gross kWh Savings by Measure

Measure Type Ex Ante Annual kWh Savings



Ex Post Annual Gross kWh

Savings


Filter Alarm 59,432 59,432 15,112 52,164 88%

LED Nightlight 114,886 114,886 97,972 127,916 111%

Showerhead 1,161,378 1,161,378 260,168 1,741,736 150%

Kitchen Aerator 364,211 364,211 71,357 400,318 110%

9W LED 440,693 440,693 275,139 462,961 105% Total 2,140,600 2,140,600 719,747 2,785,095 130%


The estimated gross demand reduction resulting from the Schools Energy Education Program is summarized in Table 11-5 below. The gross kW realization rate for the program is 105%.

Table 11-5 Program-level Gross kW Demand Reduction






242.00 242.00 76.87 254.47 105%



The net savings analysis is used to determine what part of the gross energy savings achieved by program participants can be attributed to the effects of the program. The net savings attributable to program participants are the gross savings less free ridership, plus spillover. ADM estimated net savings for the I&M Schools Energy Education Program by performing a literature review of secondary net-to-gross values for other similar energy efficient school kit programs. Table 11-6 summarizes the studies reviewed. ADM reviewed the 2017 I&M evaluation results as well as five other evaluations of programs operating in the South and Midwest.

19 ADM referenced the parent/guardian survey data collected by Resource Action Programs to estimate the percentage of program participants in Indiana that have an electric hot water heater in their homes at 50%.


Table 11-6 Summary of Evaluations Reviewed

Utility State Year Ameren Missouri Missouri 2017 Duke Energy North and South Carolina 2015 ComEd Illinois 2018 I&M Indiana 2017 Duke Kentucky 2015 Energy New Orleans Louisiana 2015

Figure 11-1 summarizes the findings of the review of program measure net-to-gross ratios. As shown, the findings were reasonably consistent across studies for most measures. LED light bulbs were the exception. For this measure, two net-to-gross ratios were found: 55% and 119%. The latter value was the finding of the 2017 I&M evaluation. It is worth noting that this estimate was based on a small sample size due to the limited availability of contact information and the response rate.

Figure 11-1 Distribution of Net-to-Gross Ratios by Program Measure

ADM applied the average net-to-gross ratio found across studies for use in estimating the net savings of the I&M Schools Energy Education Program. Table 11-7 presents the measure-level net-to-gross ratios that will be referenced.

Table 11-7 Measure-Level Net-to-Gross Ratios

Program Measure Number of Studies I&M 2017 Result Average Value LED light bulbs 2 119% 87% LED night lights 4 97% 98% Kitchen Faucet Aerators 6 88% 98% High-efficiency showerheads 6 91% 95% Filter tone alarm 2 100% 97%



Because none of the studies researched included non-participant spillover in the calculation of the net-to-gross ratio, ADM added non-participant spillover savings to the estimate of net savings based on applying the researched net-to-gross ratios.

Non-participant spillover was estimated using data collected through the PY2017 non-participant survey. To estimate PY2018 nonparticipant spillover, the PY2017 non-participant spillover ratio was applied to the 2018 customer usage. That is, total non-participant spillover (NPSO) savings for the portfolio was estimated as follows:


Where,




The total non-participant spillover was allocated to the Schools Energy Education Program in proportion to its share of the 2018 expenditures.




Table 11-8 summarizes the net ex post kWh savings of the Schools Energy Education Program. The net savings value is based on the application of the researched net-to-gross ratios, plus non-participant spillover. The annual net savings totaled 2,680,178 kWh. The net-to-gross ratio is 96%.


Ex Ante Gross kWh

Savings

Gross Audited

kWh Savings

Gross Verified

kWh Savings

Ex Post Gross kWh

Savings

Gross Realization

Rate

Ex Post Net kWh

Savings

Net-to-Gross Ratio


kWh Savings

2,140,600 2,140,600 719,747 2,785,095 130% 2,680,178 96% 27,411,597


Table 11-9 summarizes the net ex post kW demand reduction of the Schools Energy Education Program. The net demand reduction equaled 237.30 kW. The net-to-gross ratio is 93%.




Gross Audited kW

Savings

Gross Verified kW

Savings


Gross Realization

Rate


Net-to-Gross Ratio

242.00 242.00 76.87 254.47 105% 237.30 93%


ADM completed a limited process evaluation of the PY2018 program. The limited process evaluation documented notable program changes and developments.

The PY2018 process evaluation consisted of program staff interviews. This data collection activity and sample size are presented in Table 11-10.

Table 11-10 Summary of Data Collection Activities for the Schools Energy Education Program

Data Collection Activity Mode of Data Collection Targeted Sample Size

Staff interviews Telephone in-depth interviews 2



This section summarizes the findings from interviews with I&M program management staff and Resource Action Programs (RAP) staff for the purposes of learning of any changes to the program structure, identifying program objectives, and assessing the extent to which there are future opportunities for program improvement. ADM evaluators interviewed the I&M energy efficiency consumer program coordinator and the program manager from Resource Action programs to gain insights in the School Energy Education program for 2018.

The findings of the interviews are summarized below.

The goal for the number of kits distributed was met. The program distributed 11,000 kits to students during the program year. Indiana achieved the number of kits but did not reach the savings goal (ex ante kWh savings equaled 67% of the planning goal). Staff indicated the savings goal were not met because per kit savings were in lower than originally planned as the result of applying evaluated savings adjustments.

No additional schools were added. The same schools participating in 2017 participated in 2018. The classroom retention rate is approximately 80% and RAP reported that the level of interest remains the same. Staff stated that discontinuation of participation is typically the result of teachers moving to another grade rather than factors related to the program. RAP also noted that the primary barrier to teacher participation is the time required of them to enroll and “get up and running.” Since this a barrier primarily to initial enrollment, it is not very impactful on subsequent enrollment.


The program updated the kit box design. Although the design changed, the kit contents remained the same. Staff noted that a few changes were made to information included in the kit used to cross-promote other I&M programs.

I&M noted there will be changes to the kits beginning January 2019. There will be one additional light bulb, two faucet aerators, and a handle added to the kit box to facilitate carrying. The cut-out where the measures are placed to prevent movement in the box is now removable. This allows the box with the handle to be used as a storage box for the student and/or parents. The insert added inside the kit cross-promotes other I&M energy efficiency programs and is presented in English and Spanish to facilitate understanding in bilingual homes. RAP has offered these changes to the kits at no additional cost to I&M. These enhancements to the kit are to offset the falling price of LED light bulbs, and to pass those savings on to the SEE Program.

Data management and QA/QC process remain consistent with prior years. Staff reported that there were not changes made to data management and QA/QC during the year and that there are no significant concerns at this time.


11.5.1 Conclusions


The goal for the number of kits distributed was exceeded. The program distributed 11,000 kits students during the program year.

Minor updates to school kits. No measures were changed in the school kits during 2018. The box design was updated and there were some changes made to the material included in the kit that is used to cross-promote the programs.

Kit content changes are planned for 2019. Staff will change the kit contents in 2019 by adding an additional light bulb and including two faucet aerators instead of one. Additionally, the kit box will include a handle.


ADM does not have any recommendations for the Schools Energy Education Program.


12 Home Energy Management This chapter presents the results of the impact and process evaluation of the PY2018 Home Energy Management Program offered by Indiana Michigan Power (I&M) during the period of January 1, 2018 through December 31, 2018.


Assess gross and net energy (kWh) savings and demand (kW) reductions resulting from participation in the program during the period of program implementation;




Assess satisfaction among participating customers;




The overall objective of the Home Energy Management Program is to provide I&M’s residential Indiana customers a means to maintain their unique comfort preferences while at the same time improve their home HVAC demand and energy consumption through more efficient operation.

The program provides rebates for smart thermostats and deploys software that alters the smart thermostat set points – Tendril’s Orchestrated Energy (OE). With OE, Tendril uses proprietary algorithms and a home simulation model together with a controllable thermostat to manage energy use in a customer’s house while ensuring a customer’s comfort preferences are satisfied. The controllable device used for the program is an Ecobee or Honeywell smart thermostat.

The Home Energy Management Program is available to all I&M customers in single family or multifamily residences who have existing central air conditioning and/or heat pump equipment and Wi-Fi and internet capability. For non-owner occupied residences, I&M requires written permission from the owner prior to participation. To qualify, participants of the program must agree to:

Install a program-qualifying smart thermostat in their home;

Connect and maintain a Wi-Fi internet connection to the thermostat; and

Allow the program’s IM Home platform to operate and manage their home heating and cooling equipment.

Participating customers receive:

A rebate of up to $150 towards the purchase of a qualifying smart thermostat;


A discount of up to $150 towards the professional installation of the qualifying smart thermostat; and

A bill credit of $1.95 per demand reduction event (up to 15 events per season).

The program implementation is supported by a third-party contractor (Tendril) that provides the Orchestrated Energy software. Third party technicians provide installation of thermostats to customers.




ADM surveyed a sample of 218 program participants as part of the program evaluation effort. Participant surveys were administered to supplement and verify tracking data provided from Tendril and I&M and assess levels of participant free ridership with regards to purchase and installation of smart thermostats.


The first aspect of conducting measurements of program activity were to verify that the reported number of participants is accurate and complete. ADM took several steps in verifying participation, which consists of the following:

Validating program tracking data provided by Tendril and I&M by checking for duplicate or erroneous entries;

Verifying that participants were part of the program according to the agreed-upon process between Tendril and I&M; and

Conducting verification surveys with a sample of program participants. A purpose of these surveys is to verify that customers listed in the program tracking data received a smart thermostat through the program.


The following sub-sections outline the data used to complete the evaluation of the Home Energy Management program.

Data for Estimating Smart Thermostat Savings: ADM used the following data sources to estimate annual energy savings for the Ecobee smart thermostats:

Home Energy Management (HEM) Program tracking data. HEM Program tracking data was referenced to obtain participants’ central heating and cooling equipment type and central cooling equipment capacity in Btuh. ADM also used the data to obtain participants’ existing thermostat type.


Home Energy Products – Appliances (HEP) Program tracking data. ADM used this data to impute the required central heating equipment type (Electric Resistance, Heat Pump, or Gas and Other), central heating capacity in Btuh, and central heating equipment efficiency in COP where applicable for all HEM thermostat participants that did not have this information tracked in the primary HEM tracking system.

Survey responses of HEM thermostat participants collected in 2018. The HEM thermostat participant survey response data was used to estimate thermostat energy savings factors for participants that did not have existing thermostat type recorded (see Table 12-2 below). ADM also referenced the HEM thermostat survey response data to develop in-service rates of program thermostats.

Data for Estimated Orchestrated Energy Savings: ADM received HVAC equipment nameplate information from the program that will allow for estimating equipment kW load.

For determining baseline run-time for the cooling equipment (i.e., RTbaseline), information was developed through statistical analysis using data collected through the Ecobee thermostats.20 These data included the following for each participant home on an hourly basis:

Equipment runtimes

Indoor temperatures

Ambient outdoor temperatures

On/off status of OE software

On/off status of vacation mode

The dataset that ADM received included data collected from 2,132 devices with a total of 5,432,169 data points. ADM removed ineligible, inoperative, and learning thermostat mode data, after which 2,608,675 data points remained.

Table 12-1 Number of Data Points by Thermostat Mode

Thermostat Mode

Number of Observations Number of Devices Prior to Data

Cleaning Post Data Cleaning

Prior to Data Cleaning

Post Data Cleaning

Active 1,956,350 1,814,540 1,763 1,259 Demand Response 234,816 191,086 1,608 1,254

20 This approach builds on the Draft ENERGY STAR Climate Controls Framework Field Savings Metric, U.S. Environmental Protection Agency, January 12, 2015. Ecobee has used a proprietary formula to infer furnace run-time at a 72 degrees Fahrenheit reference temperature. This formula infers a home’s furnace run-time based on thermal parameters observed after the thermostat is installed and operating. Ecobee derives these thermal parameters by curve-fitting the relationship between average daily indoor-outdoor temperature difference and furnace run-time (and perhaps some other factors). Ecobee uses this relationship to extrapolate the run-time for operation at a hypothetical 72 degrees Fahrenheit set point. Other thermostat providers use similar curve-fitting approaches but apply different assumptions about time steps, heat loss, thermal mass and internal loads for this inference. See Lawrence Berkeley National Laboratory, Improving Residential Programmable Thermostats, Final Project Report CEC-500-2015-019, Prepared for California Energy Commission, December 2014.


Thermostat Mode

Number of Observations Number of Devices Prior to Data

Cleaning Post Data Cleaning

Prior to Data Cleaning

Post Data Cleaning

Ineligible 1,169,223 0 1,939 0 Inoperative 1,320,500 0 1,855 0 Learning 751,184 603,049 1,869 1,259 Unknown 96 0 1,193 0

Total 5,432,169 2,608,675


Smart Thermostats: ADM referenced the 2015 Indiana Technical Reference Manual (TRM) Version 2.2 to calculate energy savings of Ecobee thermostats installed under the program.

From the Indiana TRM, the Wi-Fi Connected Smart Thermostats (Time of Sale, Direct Install) section was used to calculate energy impacts for the installation of smart thermostats in the HEM program. The TRM provides the following equations:


∆𝑘𝑘𝑘𝑘ℎ𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑖𝑖𝑓𝑓𝑖𝑖 =1

𝑆𝑆𝐸𝐸𝐸𝐸𝐺𝐺 × 𝐸𝐸𝐹𝐹𝐸𝐸𝐻𝐻𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 × 𝐵𝐵𝑆𝑆𝑢𝑢𝐻𝐻𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 × 𝐸𝐸𝑆𝑆𝐹𝐹𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐

1,000

∆𝑘𝑘𝑘𝑘ℎℎ𝑏𝑏𝑏𝑏𝑡𝑡𝑖𝑖𝑓𝑓𝑖𝑖 =𝐸𝐸𝐹𝐹𝐸𝐸𝐻𝐻ℎ𝑏𝑏𝑏𝑏𝑡𝑡 × 𝐵𝐵𝑆𝑆𝑢𝑢𝐻𝐻ℎ𝑏𝑏𝑏𝑏𝑡𝑡 × 𝐸𝐸𝑆𝑆𝐹𝐹ℎ𝑏𝑏𝑏𝑏𝑡𝑡


Where:

SEER = Seasonal average efficiency in SEER of Central Air Conditioning equipment, actual; 10.77 (EUL-weighted average of the “Before 2006” and “After 2006” assumed SEER values presented in Indiana TRM table) was assumed otherwise.


BtuHcool = Cooling system capacity in Btuh, actual; 28,994 was assumed otherwise

ESFcool = Cooling energy savings fraction (see Table 12-2)


BtuHheat = Heating system capacity in Btuh, actual; estimated from HEP participant data otherwise

ESFheat = Heating energy savings fraction (see Table 12-2)


ηHeat = Efficiency in COP of Heating equipment, actual; estimated from HEP participant data otherwise

The Indiana TRM attributes no peak reduction savings to the installation of Wi-Fi Smart Thermostats.

The Wi-Fi Connected Smart Thermostats (Time of Sale, Direct Install) section from the Indiana TRM provides deemed energy savings factors of 0.139 for cooling and 0.125 for heating, assuming a baseline condition of a standard, manually-controlled thermostat. The Programmable Thermostat (Time of Sale, Direct Install) section from the Indiana TRM provides deemed energy savings factors of 0.09 for cooling and 0.068 for heating, also assuming a baseline condition of a standard, manually-controlled thermostat. ADM assigned a cooling energy savings factor of 0.049 (= 0.139 - 0.09) and a heating energy savings factor of 0.057 (= 0.125 - 0.068) to instances in which Ecobee thermostats replaced programmable thermostats.

The existing thermostat type that the Ecobee thermostats replaced was tracked in I&M’s main program tracking system for most participants. For unknown existing thermostat types, ADM referenced PY2018 HEM thermostat participant survey response data to approximate energy savings factors.

Among the HEM Program survey respondents that identified their existing thermostat type, 37% indicated a manually-controlled thermostat, 58% indicated a programmable thermostat, and the remainder indicated a different Wi-Fi Smart Thermostat. Table 12-2 below presents the cooling and heating energy savings factors ADM applied to all eligible Ecobee thermostats.

Table 12-2 Energy Savings Factor – Wi-Fi Smart Thermostat

Wi-Fi Smart Thermostat Baseline Type ESF_cool ESF_heat

Home Energy Products

Survey Weights

Unknown Baseline

Thermostat Type

ESF_cool

Unknown Baseline

Thermostat Type

ESF_heat

Wi-Fi Smart Thermostat 0 0 5% 0.080 0.079 Manual Thermostat 0.139 0.125 37%

Programmable Thermostat 0.049 0.057 58%

Orchestrated Energy: The OE software has several operational states. During the “counterfactual” state, the software is not controlling the operation of the HVAC equipment. For this reason, the time series observations associated with the “counterfactual” state are identified as a baseline against which the incremental impact of OE software can be measured.

The time series data included information on HVAC operational status, OE software operational state, and ambient temperature. These data were used to construct several variables. Key variables are identified in Table 12-3.

Table 12-3 Key Variables

Variable Name Variable Value

OE 1 if OE software is active; otherwise 0.


Variable Name Variable Value

Morning 1 if time is 6:00 AM - 9:59 AM; otherwise 0. Day 1 if time is 10:00 AM - 2:59 PM; otherwise 0. Afternoon 1 if time is 3:00 PM - 5:59 PM; otherwise 0. Evening 1 if time is 6:00 PM - 11:59 PM; otherwise 0. Weekday 1 if weekday; otherwise 0. CDH70 MAX(Average Outdoor Temperature - 70°F, 0) CDH70_Lag3 Average CDH70 for preceding three hours. CDH70_Lag48 Average CDH70 for preceding 48 hours. MorningWeekday 1 if time is 6:00 AM - 9:59 AM during weekday; otherwise 0. DayWeekday 1 if time is 10:00 AM - 2:59 PM during weekday; otherwise 0. AfternoonWeekday 1 if time is 3:00 PM - 5:59 PM during weekday; otherwise 0. EveningWeekday 1 if time is 6:00 PM - 11:59 PM during weekday; otherwise 0. MorningCDH_Lag3 Equals CDH70 if time is 6:00 AM - 9:59 AM; otherwise 0. DayCDH_Lag3 Equals CDH70 if time is 10:00 AM - 2:59 PM; otherwise 0. AfternoonCDH_Lag3 Equals CDH70 if time is 3:00 PM - 5:59 PM; otherwise 0. EveningCDH_Lag3 Equals CDH70 if time is 6:00 PM - 11:59 PM; otherwise 0.

Cooling energy use was calculated as follows:

Equation 12-1

𝐶𝐶𝐷𝐷𝐷𝐷𝑆𝑆𝐷𝐷𝑆𝑆𝐴𝐴 𝐸𝐸𝐸𝐸𝑢𝑢𝐷𝐷𝑖𝑖𝐼𝐼𝐷𝐷𝑆𝑆𝑆𝑆 𝐶𝐶𝑆𝑆𝑖𝑖𝑆𝑆𝐼𝐼𝐷𝐷𝑆𝑆𝑏𝑏 = 1

𝑆𝑆𝐸𝐸𝐸𝐸𝐺𝐺 × 𝐵𝐵𝑆𝑆𝑢𝑢𝐻𝐻𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐

1,000

Equation 12-2

Cooling Energy Use = RT * Cooling Equipment Capacity

Where;

RT = cooling equipment run-time.

Cooling energy savings was calculated as follows:

Equation 12-3

Cooling Energy Savings = RTOE - baseline* Cooling Equipment Capacity

Where:

RTOE - baseline = the difference run-time for the cooling equipment if OE is “off” and OE is “on”, adjusting for the incremental impact of the margins.


To estimate the impact of OE software on HVAC equipment run-times, a regression equation referencing the variables identified in Table 12-3 was fit to the thermostat time series data to provide an estimate of equipment run-times for customer i at time t:

Equation 12-4

Run-timeit = β0 + β1OEit + β2Morningit + β3Dayit + β4Afternoonit + β5Eveningit + β6Weekdayit + β7CDH70it + β8CDH70_Lag3it + β9CDH70_Lag48it + β10MorningWeekdayit +

β11DayWeekdayit + β12Afternoon Weekdayit + β13EveningWeekdayit + β14MorningCDH_Lag3it + β15DayCDH_Lag3it + β16AfternoonCDH_Lag3it +

β17EveningCDH_Lag3it + eit

In the equation give above, β denotes the intercept term.21

Because run-time will typically have a value of zero for a significant number of observations, estimating a relationship between run-times and the other factors through ordinary least squares (OLS) regression may produce inconsistent estimates for the model coefficients.

To address this problem, a Tobit regression approach was used. Tobit regression modeling for estimation of energy use has been used in several prior studies22. This methodology has also been applied in papers from the International Association of Energy Economics (IAEE.org) for models of energy pricing and consumption.

Tobit regression is a method of correcting for data that is top or bottom censored, i.e., data that either due to physical or practical limitations has a floor or ceiling on its range and also displays a high concentration of data points at the censored boundary. It does so by means of an intermediate step in which values of the dependent variable are temporarily allowed to breach their censoring guideline. In the case at hand, this means allowing for run-time values to be negative. In doing so, the Tobit estimation procedure corrects the estimation of points where run-times are positive. In effect, a high proportion of zero values in the data set no longer biases the slope coefficients downward.

The coefficient β1 will provide an estimate of the incremental impact of OE software on equipment run-time. The incremental impact is applied to the predicted runtime to account for the marginal impacts.

21 β is inclusive of the usual intercept of the Tobit model and the log-standard deviation of the latent variable. 22 For examples, see the following:

Lucas, W. Davis, “Durable Goods and Residential Demand for Energy and Water: Evidence From a Field Trial”, RAND Journal of Economics, Vol. 39, No. 2, Summer 2008, pp. 530–546.

KEMA, Final Report, Pacific Gas and Electric SmartAC Load Impact Evaluation, Prepared for Pacific Gas and Electric Company, April 24, 2008.


Demand Response Events: An econometric model based on the run-time of demand response participants during counterfactual hours was used to estimate counterfactual run-times for HVAC equipment subject to a demand response event. The model used is described in Equation 12-5.

Equation 12-5

Run-timeit = β0 + β1hourit + β2Morningit + β3Dayit + β4Afternoonit + β5Eveningit + β6Weekdayit + β7CDH70it + β8CDH70_Lag3it + β9CDH70_Lag48it + eit

The difference between the counterfactual run-time and the actual run-time is the estimated incremental impact of OE during the demand response event. For each event, multiplying this value by the total kW load associated with the controlled equipment provided for an estimate of the kW reductions caused by the OE software.

Peak kW Savings: OE peak kW savings are calculated as the average savings occurring during the I&M 5CP hours referenced in Table 12-4. The coefficient values associated with Equation 12-4 are applied to estimate the counterfactual kW during these hours in order to estimate the incremental impact of OE during each hour.

Table 12-4 PY2018 I&M 5CP Hours Date Hour Start Hour End

5/29/2018 1:00 PM 2:00 PM 6/18/2018 2:00 PM 3:00 PM 7/10/2018 12:00 PM 1:00 PM 8/28/2018 3:00 PM 4:00 PM 9/4/2018 4:00 PM 5:00 PM


The ex post energy and demand impacts resulting from the Home Energy Management Program in 2018 are reported in the following sections.

12.2.2.1.1 Smart Thermostats

The estimated annual gross energy savings resulting from Wi-Fi Smart Thermostat measures in the Home Energy Management Program in 2018 are summarized in Table 12-5 below. The gross realization rate for the program measure is 127%.

The Indiana TRM attributes no peak demand reduction to the installation of Wi-Fi Smart Thermostats.


Table 12-5 Annual Gross kWh Savings – Wi-Fi Smart Thermostats

Program Component Ex Ante Gross kWh Savings

Gross Audited kWh

Savings

Gross Verified kWh

Savings


Gross Realization

Rate

Wi-Fi Smart Thermostats 181,352 181,352 173,899 230,583 127%

12.2.2.1.2 Orchestrated Energy

The parameter estimates for each variable in the regression model are reported in Table 12-6 below. The estimates for the individual devices were not included. Table 12-7 presents the inputs to the calculation of ex post gross kWh savings, and Table 12-8 presents the ex post and ex ante energy savings of Orchestrated Energy, along with the gross realization rate.

Table 12-6 Parameter Estimates for Regression Model Estimating Runtime Variable Name Estimate t value p value (Intercept): 1 -0.45 -198.72 0.0000 (Intercept): 2 -0.46 -542.60 0.0000

OE -0.16 -141.26 0.0000 Morning -0.08 -26.08 0.0000

Day 0.11 36.04 0.0000 Afternoon 0.28 73.67 0.0000 Evening 0.38 131.63 0.0000 Weekday -0.04 -17.94 0.0000 CDH70 0.02 65.73 0.0000

CDH70_Lag3 0.03 63.89 0.0000 CDH70_Lag48 0.04 162.97 0.0000

MorningWeekday -0.06 -14.89 0.0000 DayWeekday -0.11 -35.67 0.0000

AfternoonWeekday -0.01 -1.89 0.0594 EveningWeekday 0.04 13.69 0.0000

MorningCDH_Lag3 0.02 22.65 0.0000 DayCDH_Lag3 0.01 15.20 0.0000

AfternoonCDH_Lag3 0.00 -6.14 0.0000 EveningCDH_Lag3 -0.01 -14.10 0.0000

Model r2 0.23


Table 12-7 kWh Savings

Number of Thermostats

in Home

Thermostat Mode

Cooling Capacity (kWh per

Unit)

Reduction in Runtime

(Hours)

kWh Savings

One Active

3 81,278 243,834 Two 1.85 3455.64 6,393 Total 250,227

Table 12-8 Ex-Post Gross kWh Savings - Orchestrated Energy





Gross Realization

Rate

Orchestrated Energy 247,853 247,853 247,853 250,227 101%

Demand response events occurred on ten days identified in Table 12-9. Figure 12-1 shows the counterfactual and actual runtimes for the event on May 31st.

Figure 12-1 May 31st Demand Response Event

The run-time reductions from those events for locations with one thermostat and locations with two thermostats are reported in Table 12-9 and Table 12-9, respectively. Per participant run-time reduction is factored by the count of participants and the estimated per unit cooling capacity to calculate total hourly kW reduction. The maximum kW reduction (1,944.00 kW) appeared on August 28th between the hours 3:00 PM – 4:00 PM.


Table 12-9 Demand Response kW Reduction for Locations with One Thermostat - Orchestrated Energy

Date Hour Count of

Participants (N)

Run-time Reduction

per Participant

(RT)

Total Hourly

kW Reduction (N * RT *

3.00) 5/29/2018 3:00 PM - 4:00 PM 997 0.40 1,196.39 5/29/2018 4:00 PM - 5:00 PM 994 0.37 1,117.71 5/29/2018 5:00 PM - 6:00 PM 992 0.28 844.84 5/31/2018 4:00 PM - 5:00 PM 991 0.30 878.88 5/31/2018 5:00 PM - 6:00 PM 996 0.25 740.31 6/18/2018 2:00 PM - 3:00 PM 886 0.41 1,090.33 6/18/2018 3:00 PM - 4:00 PM 973 0.49 1,421.12 6/18/2018 4:00 PM - 5:00 PM 978 0.42 1,233.57 6/19/2018 1:00 PM - 2:00 PM 1,047 0.24 746.53 6/19/2018 2:00 PM - 3:00 PM 1,047 0.23 722.45 6/29/2018 3:00 PM - 4:00 PM 1,083 0.36 1,171.74 6/29/2018 4:00 PM - 5:00 PM 1,083 0.32 1,045.30 7/2/2018 2:00 PM - 3:00 PM 1,061 0.30 951.50 7/2/2018 3:00 PM - 4:00 PM 1,070 0.38 1,222.94 7/2/2018 4:00 PM - 5:00 PM 1,068 0.31 997.50 7/3/2018 3:00 PM - 4:00 PM 1,080 0.41 1,315.97 7/3/2018 4:00 PM - 5:00 PM 1,079 0.36 1,153.71 7/3/2018 5:00 PM - 6:00 PM 1,078 0.27 866.98

7/13/2018 3:00 PM - 4:00 PM 1,122 0.37 1,230.23 7/13/2018 4:00 PM - 5:00 PM 1,121 0.33 1,104.43 8/27/2018 1:00 PM - 2:00 PM 1,304 0.30 1,170.37 8/27/2018 2:00 PM - 3:00 PM 1,303 0.32 1,254.91 8/27/2018 3:00 PM - 4:00 PM 1,300 0.38 1,474.52 8/28/2018 2:00 PM - 3:00 PM 1,313 0.40 1,586.36 Average kW Reduction 1,157.00

Table 12-10 Demand Response kW Reduction for Locations with Two Thermostats - Orchestrated Energy


Date Hour Count of

Participants (N)

Run-time Reduction

per Participant

(RT)

Total Hourly

kW Reduction (N * RT *

1.85) 5/29/2018 3:00 PM - 4:00 PM 37 0.38 25.98 5/29/2018 4:00 PM - 5:00 PM 37 0.38 26.10 5/29/2018 5:00 PM - 6:00 PM 37 0.24 16.28 5/31/2018 4:00 PM - 5:00 PM 35 0.26 16.94 5/31/2018 5:00 PM - 6:00 PM 34 0.19 11.91 6/18/2018 2:00 PM - 3:00 PM 35 0.45 29.34 6/18/2018 3:00 PM - 4:00 PM 43 0.50 40.08 6/18/2018 4:00 PM - 5:00 PM 46 0.39 33.05 6/19/2018 1:00 PM - 2:00 PM 49 0.20 18.56 6/19/2018 2:00 PM - 3:00 PM 49 0.19 16.96 6/29/2018 3:00 PM - 4:00 PM 46 0.34 29.11 6/29/2018 4:00 PM - 5:00 PM 44 0.32 25.70 7/2/2018 2:00 PM - 3:00 PM 46 0.27 22.96 7/2/2018 3:00 PM - 4:00 PM 46 0.36 30.87 7/2/2018 4:00 PM - 5:00 PM 46 0.28 23.97 7/3/2018 3:00 PM - 4:00 PM 44 0.30 24.25 7/3/2018 4:00 PM - 5:00 PM 44 0.28 22.86 7/3/2018 5:00 PM - 6:00 PM 44 0.27 21.66

7/13/2018 3:00 PM - 4:00 PM 44 0.33 26.69 7/13/2018 4:00 PM - 5:00 PM 44 0.32 26.40 8/27/2018 1:00 PM - 2:00 PM 68 0.31 38.65 8/27/2018 2:00 PM - 3:00 PM 66 0.33 40.02 8/27/2018 3:00 PM - 4:00 PM 68 0.35 43.95 8/28/2018 2:00 PM - 3:00 PM 65 0.40 47.91 Average kW Reduction 29.56

12.2.2.1.3 Program-Level Ex Post Gross kWh Savings

The estimated annual gross energy savings resulting from the Home Energy Management Program in 2018 are summarized in Table 12-11 below. The overall gross realization rate for the program is 112%.

Table 12-11 Program-Level Ex Post Gross kWh Savings






Wi-Fi Smart Thermostats 181,352 181,352 173,899 230,583 127% Orchestrated Energy 247,853 247,853 247,853 250,227 101% Total 429,205 269,207 261,754 480,810 112%


12.2.2.2 Program-Level Ex Post Gross kW Reduction

The average OE peak kW reduction was calculated during the I&M 5CP periods and reported in Table 12-12.

Table 12-12 Ex Post Gross Peak kW Reduction – Orchestrated Energy

Date Hour Start Hour End

Number of Active

Participants


5/29/2018 1:00 PM 2:00 PM 1004 -395.64 6/18/2018 2:00 PM 3:00 PM 921 1119.67 7/10/2018 12:00 PM 1:00 PM 854 41.74 8/28/2018 3:00 PM 4:00 PM 1376 1944 9/4/2018 4:00 PM 5:00 PM 1141 179.24

Maximum Peak kW Reduction 1944.00 Average Peak kW Reduction 577.80

The Indiana TRM attributes no peak reduction savings to the installation of Wi-Fi Smart Thermostats, and so the entirety of HEM Program peak kW savings are presented in Table 12-13.

Table 12-13 Ex Post Gross kW Savings






1,008.91 1,008.91 1,008.91 577.80 57%



A survey of program participants was administered to collect data for use in estimating free ridership associated with the rebated thermostat.


As the calculation of peak demand reductions attributable to demand response events includes a control group, the kW impacts associated with this program component were considered to be net impacts, with a net-to-gross ratio of 1.0. Similarly, kWh energy savings associated with the implementation of Orchestrated Energy technology were considered to be net impacts as this technology is only available through the Home Energy Management Program.

However, the kWh energy savings associated with the purchase and installation of an Ecobee smart thermostat are subject to possible free ridership effects, as it is possible that customers would have purchased and installed an Ecobee thermostat in the absence of the program.


To estimate the net savings associated with the gross kWh energy savings achieved from the installation of Ecobee thermostats, ADM estimated net savings using self-reported information collected through a survey of program participants.


Free ridership for smart thermostats was estimated from a survey of 56 customers who received a smart thermostat rebate through the program.

12.3.2.2 Financial Ability and Plans and Intentions

Two indicator variables were developed based on responses to the survey questions on plans and intentions. The first corresponds to financial ability. Respondents were considered to have not been financially able to install the efficient equipment if they answer “no” to the first question below (FR1) and “yes” to the second question (FR2):

FR1: Would you have been able to afford to purchase the smart thermostat if the rebate was not available from the program?

FR2: [IF YES] Just to confirm, if the rebate was not available through the program, would you would still have paid the additional cost to purchase a smart thermostat instead of another type of thermostat?

The second indicator variable is related to whether or not the customer had plans to implement the efficiency measure. Respondents were considered to have had plans if they answer “yes” to the following two questions:

FR3: Were you planning to purchase a smart thermostat before you learned of I&M’s rebate program?

FR4: [IF YES] Just to be clear, did you have plans to specifically purchase a smart thermostat as opposed to another type of thermostat?

Respondents who indicated that they did not have plans or could afford thermostat were deemed to not be free riders.

12.3.2.3 Program Influence

Participants were asked two questions about the direct influence of the program on their decision to implement the energy efficiency measures. Specifically, participants were asked:

FR5: Using a scale where 0 is “not at all influential” and 10 is “very influential,” how influential was the information and rebates available through program in your decision to purchase the [EFF_MEASURE]?

FR6: Now we would like to know how likely you would have been to install the smart thermostat if the program was not available. Using a scale where 0 is “not at all likely” and 10 is “very likely”, how likely is it that you would have installed the same [EFF_MEASURE] if you had not received the financial or information assistance through the program?


A program influence score is developed based on these two responses in the following manner:

Program Influence = Average (FR5 , (10 – FR6)) / 10

12.3.2.4 Program Influence on Project Timing


Did you purchase and install the smart thermostat sooner than you would have if the information and financial assistance from the program had not been available?

When might you have purchased or installed the smart thermostat if you had not participated in the program?

Based on the responses to those questions a timing adjustment is calculated as shown in Table 12-14.



Within 6 months 1.0 Between 6 months and 1 year 0.67 In more than 1 year to 2 years 0.33 In two years or more 0

12.3.2.5 Free Ridership Scoring

For respondents who reported that they had plans and could have afforded the thermostat without the program rebate, an overall project free ridership score was calculated using the following equation:

Free Ridership = (1 – Program Influence Score) * Timing Score


Home Energy Management participant spillover savings would be incorporated in the billing analysis of net savings. That is, participant spillover savings were not estimated through self-reported information.





Where,




The total non-participant spillover was allocated to the Home Energy Management Program in proportion to its share of the 2018 expenditures.



Table 12-15 summarizes the number of survey responses and average free-ridership scores by measure for the Home Energy Management Program.

Table 12-15 Survey Response Count and Average Free Ridership Score



EcoBee Thermostat 141 21%


Table 12-16 summarizes the net ex post kWh savings of the Home Energy Management Program. The ex post net kWh savings of the program are inclusive of the net savings impacts estimated from the billing analysis, the gross savings of the thermostats less free ridership, and the estimated non-participant spillover. The program-level annual net savings totaled 519,446 kWh.


Ex Ante Gross kWh

Savings

Gross Audited

kWh Savings

Gross Verified

kWh Savings

Ex Post Gross kWh

Savings

Gross Realization

Rate


Net-to-Gross Ratio

Ex Post Net

Lifetime kWh

Savings 429,205 429,205 421,752 480,810 112% 519,446 108% 7,791,691


Table 12-17 summarizes the net ex post peak kW reduction of the Home Energy Management Program.


Table 12-17 Ex Post Net kW Savings

Ex Ante Gross

kW Savings

Gross Audited

kW Savings

Gross Verified

kW Savings

Ex Post Gross

kW Savings

Gross Realization

Rate


Net-to-Gross Ratio

1,008.91 1,008.91 1,008.91 577.80 57% 662.78 115%




Table 12-18 Summary of Data Collection Activities for the Home Energy Management Program




ADM evaluators interviewed the director of customer delivery for Demand Management at Tendril to gain insights in the Home Energy Management (HEM) program for 2018.

Below is a summary of the key findings from the staff interview:

Tendril added a web-based enrollment option for Orchestrated Energy. Tendril noted that they observed an increase in enrollment after the launch.

The program is marketed through multiple channels. The marketing efforts included email campaigns, social media campaigns, direct mail advertising, and newsletters.

Few customers opted out of the program or the demand response events. Staff reported that the program opt-out rate was around 2 to 4% and that the even opt-out rate was 2%. Tendril stated that the event opt-out rate was “phenomenal” and that they have seen other thermostat program opt-out rates in the range of 20-50%.

Future efforts to improve customer education are planned. Staff is currently reviewing the customer enrollment and education process to identify ways to more effectively communicating with customers about what to expect of the program.

Various strategies to increase customer enrollment are under consideration. Staff reported that they are reviewing incentive levels as one approach to increase enrollments, as well as reviewing marketing materials, messaging, and the enrollment process. Tendril


indicated they are currently working with I&M on these changes and exploring ways to simplify (i.e., using less data to verify customers) the enrollment process.

Efforts to include Nest thermostats will continue. Tendril is continuing to work with Nest to enable those devices to work with the Tendril software.


ADM completed a survey of program participants to collect information regarding their experience with the Home Energy Management Program. The survey was administered by email and telephone. In total, 218 customers responded to the survey.


Survey participants provided feedback on how they became aware of the Home Energy Management program. Figure 12-2 summarizes the initial source of program awareness for responding customers, with most learning of the program by the I&M website, I&M newsletter and word-of-mouth from friends or relatives. As shown, the largest share of customers (31%) reported learning of the program by visiting the I&M website. Other commonly reported sources of awareness were an I&M newsletter (17%) and a bill insert or mailing (15%).



Survey participants were also asked if they heard about the program from other sources. Again, the I&M website was most often cited (32%), followed by a bill insert (16%), an I&M newsletter (11%), and a home energy report (11%).

Figure 12-3 summarizes the reasons customers chose to participate in the Home Energy Management Program, with 43% indicating that they participated in order to receive thermostat rebate followed by participating in order to save money on energy costs (28%).

Figure 12-3 Reason for Participating in Home Energy Management Program

Participation Process

Respondents indicated overall satisfaction with the ease of enrolling in the Home Energy Management program, with 56% indicating enrollment was very easy (rated as 5) and 25% indicating it was somewhat easy (rated as a 4) (see Table 12-19).

Table 12-19 Ease of Enrolling in the Home Energy Management Program


(n = 216) 5 (Very easy) 56% 4 25% 3 13% 2 3% 1 (Very difficult) 2%

Survey participants rated the clarity of information on various aspects of the Home Energy Management program using a scale that ranged from 1 (not at all clear) to 5 (very clear). Overall, most respondents indicated the information provided to them regarding the program was either


very or somewhat clear (see Figure 12-4). However, some found the information on how IM Home would adjust to the thermostat unclear. Specifically, 9% rated this aspect as a 1, meaning not at all clear, or a 2 (10%).

Figure 12-4 Clarity of Information Regarding the Home Energy Management Program

Survey participants were asked if they had any concerns about participating in the program when they first signed up. Less than half (40%) stated they had concerns and 60% did not have concerns when they initially signed up for the program. Most concerns were related to maintaining control of the thermostat and comfort in the home. Table 12-22 summarizes all the concerns expressed by respondents in the survey.

Table 12-20 Summary of Concerns about Participating

Type of Comment n Example comments

Losing control of temperature adjustments 26

The ability for an outside agency to control my home.

I may not be able to control for my own comfort temperature by myself

Just someone having control of a device over the internet

Comfort issues 26

That we would be uncomfortably hot during it.

Just what does it mean during the summer cooling season. We will see.

Making my home uncomfortably warm during the summer


Type of Comment n Example comments

Needing additional program information 11

Was not clear what will happen

How it was going to work and how I would be notified.

Exactly how it would work

Concerns that energy use has or would increase 4

It seems like a greater waste of energy to cool the house down ahead of time.

Is it costing me more electricity to cool my house colder before the event instead of keeping constant through the event.

How notifications would work 3 How it was going to work and how I would be notified.

Impact of changing settings on participation 1 If I didn't like the thermostat setting that I&M set it to and changed it myself, would that affect my requirements for the program.

Time to receive the rebate 1 Timeline to receive the rebate.

The new thermostat 1 New Thermostat had me concerned and it still does. NO instructions for set up and operation. Very unhappy with this aspect of program

Other 4

That I wouldn't agree with it

How would it conflict with the times we are away and set the thermostat to a temperature where the cooling equipment wouldn't typically run.

Twelve percent of respondents indicated they had their thermostat professionally installed, while 87% self-installed (n = 120). A majority of respondents indicated the thermostat replaced was a programmable thermostat (58%), followed by a standard thermostat (37%) (see Table 12-21). Of those who replaced a programmable thermostat, 74% indicated it was programmed at the time of replacement.

Table 12-21 Type of Thermostat Wi-Fi Thermostat Replaced


(n = 139) A programmable thermostat that allows you to schedule the temperature settings for different times of the day 58%

A standard thermostat that lets you set on/off temperatures 37% A different Wi-Fi smart thermostat 5%

Almost all survey participants (98%) indicated they personally installed the I&M Home app on their smartphone. Participants who installed the app rated how easy or difficult it was to use. About half (56%) found the IM Home App either very or somewhat easy to use (see Table 12-22).


Table 12-22 Ease of Use of the I&M Home App


(n = 185) 5 (Very easy to use) 23% 4 31% 3 26% 2 12% 1 (Very difficult to use) 7%

Survey participants provided feedback on the IM Home App features: Personalized Energy Insights and how their home uses energy, energy saving tips and recommendations, and the app’s news feed. Respondents were asked to rate how useful each feature is on a scale from 1 (not at all useful) to 5 (very useful).

Personalized Energy Insights was the feature that respondents found most useful with 54% indicating the feature was very or somewhat useful. In comparison, 36% of respondents found the app’s news feed to be very or somewhat useful (see Figure 12-5).

Figure 12-5 Usefulness of the IM Home App’s features

Customer Experience with the Home Energy Management Program during Peak Events

Survey participants were asked a series of questions related to their experience with the Home Energy Management Program during peak events. Program participants were asked if they recalled receiving a notification of when an event would occur through the IM Home App. The majority (61%) reported they recalled receiving notification and 39% did not recall receiving the notice. Among those who recalled receiving a notice, 84% believed they typically get enough advance notice of the events.

Survey participants were asked if the comfort level of their home changed (more or less) since they began participating in the program. Forty-eight percent of respondents stated the level of


comfort of their home remained the same, while 22% indicated they are somewhat or a lot more comfortable now and 29% indicated the comfort level decreased somewhat or a lot (see Table 12-23). Survey respondents provided feedback on the comfort level in their home during events. Fifty-two percent reported their home was somewhat or a lot less comfortable during events, while 44% reported the level of comfort stayed about the same and 4% indicated it was somewhat more comfortable.

Table 12-23 Home Comfort Level Since Participating and During Peak Events

Response

Percentage of Respondents Since

Participating in HEM (n = 213)

Percentage of Respondents During

Peak Event (n =91)

A lot more comfortable 3% 0% Somewhat more comfortable 19% 4% The level of comfort stayed about the same 48% 44% Somewhat less comfortable 19% 40% A lot less comfortable 10% 12%

Participants were asked about what share of time was either themselves or someone else in the household during the event and if anyone in the home changed the temperature of the thermostat. Sixty-four percent of respondents indicated that neither they nor anyone else changes the thermostat (see Figure 12-6).

Figure 12-6 Peak Events

Customer’s Cross Program Awareness


Most survey respondents indicated were aware of the I&M programs that provide rebates and discounts for energy efficient equipment and home improvements (77%). Participants were asked if they first learned of the rebate/discount programs from IM Home app and 5% responded yes, with 95% responding no.

Table 12-24 summarizes survey participants awareness of various energy efficiency programs. The thermostat rebates was the most common program that participant had heard of (84%), followed by appliance rebates (75%) and appliance recycling (74%).

Table 12-24 Customer Cross-Program Awareness

Rebate or Service Percentage of

Responses (n = 157)

Thermostat rebates 84% Appliance rebates 75% Appliance recycling/disposal 74% Lighting discounts through participating retailers 53% Rebates for central air conditioners and heat pumps (HVAC equipment) 52% Rebates for making weatherization improvements like air sealing and adding insulation 48% Other 1%

Survey participants were asked to rate their knowledge on ways to save and use energy in their homes on a scale from 1 (not at all knowledgeable) to 10 (very knowledgeable). The information was collapsed into three groups (1-3 – not very knowledgeable; 4-6 – somewhat knowledgeable; 7-10 – very knowledgeable). Respondents self-reported they were very knowledgeable in terms of use and savings of energy in their home (see Figure 12-7).

Figure 12-7 Customer Knowledge of Energy Use and Savings in Household

In addition, participants were asked to rate their own household’s efforts to save energy in their home on a scale from 1 (you have not done much) to 10 (you have done almost everything you


can to lower your monthly energy bill in your home). Again, these data were collapsed into three groups (1-3 – made little to no effort; 4-6 – made some effort; 7-10 – made a lot of effort). Nearly all respondents indicated they have made some effort to a lot of effort (see Figure 12-8).

Figure 12-8 Customer Efforts to Save Energy in their Homes


Participants were asked several questions using a scale from 1 (very dissatisfied) to 5 (very satisfied) indicating their satisfaction with the program overall or one or more of the rated program aspects (see Figure 12-9 and Figure 12-10).

Sixty-three percent of the participants reported that they were satisfied with the program overall, although one-third of respondents (n = 73) reported dissatisfaction with the program overall or one or more aspects of the program. As shown satisfaction was highest with the smart thermostat and the installation of it – nearly all respondents reported that they were satisfied with these aspects. The areas of the program customers were most dissatisfied with were:

The IM Home app (18% were dissatisfied);

The assistance provided by the I&M customer care team (18% were dissatisfied); and

The information provided about the program (14% were dissatisfied).


Figure 12-9 Overall Customer Satisfaction with the Sign-up Process, IM Home App, Thermostat and Installation

Figure 12-10 Overall Customer Satisfaction with Home Energy Management Program

Table 12-25 summarizes the types of comments made by customers who indicated dissatisfaction with one or more aspects of the program. The most commonly mentioned issue centered on the frequency of events or changes to the settings or schedule. Several customers noted that adjustments were frequently made to their thermostat settings. This behavior, likely resulting from


Orchestrated Energy optimization processes during non-event periods, seems to have not been well understood by participants. Several remarks indicated that participants did not know this would happen. Additionally, two respondents noted dissatisfaction with changes being applied to the settings during the night that required them to get out of bed to override them.

Table 12-25 Customer Comments Regarding Dissatisfaction with the Program

Type of Comment n Example Comments

Too many events/changes to setting or schedule 18

Almost every day was an "energy event," so I could almost never use my A/C like I wanted to.

What annoys me is the daily adjustments to my schedule which I find uncomfortable.

Having an "energy saving event" nearly every single day was not what I signed up for.

Not all of the information is accurate. They had random events that would increase the set point of my air conditioner. These were unannounced and unneeded.

Issues with the app 14

The app isn't very user friendly.

App has errors and it is not useful.

Since I changed my email address my IM Home app is not working properly.

No savings 13

Did not notice any savings during summer time.

Did not see any real savings

My bill seemed to increase this summer and my house was warmer inside than previous years.

Home too warm 9

It added programs and changed my temperature settings even when there was no "event" making the house unbearably hot.

In the summer it changed my program every day at midnight and I would wake up hot. Sometimes it raises the Thermostat too high when I'm home. Then I have to turn it down again.

Cools home too much 8

The second was settings on my thermostat that were that were lower when I was away from home then what I had programmed into the thermostat.

I do not like it being pre-cooled to such a low temp that I have to put a sweater on.

The program didn't work as I work from home and on hot days when energy reduction was taking place the system would lower the home temps too much and was freezing.

Customer support issues 7

The assistance provided by the customer care team do not understand my complaints and simply offer to unenroll me not make the program better.

I have talked to the support team and they have been no help.


Type of Comment n Example Comments

When calling for assistance the customer service rep just referred my to the website.

Issue with thermostat 5

The Ecobee thermostat is not very good

The thermostat worked for cooling, but once switched over to heating, it no longer works. Working with Honeywell to exchange it.

I don't know how to operate the equipment. I don't know how to program temperature and times and days.

Not enough information on rebate/credits 4

Still have no idea what rebated and credits amount to. Info is apparently hidden.

I'd like to know how much money I'm saving and I'd like to see the savings for each Event.

Did not get rebate or credit 4

I thought I signed up for the program correctly but never received the rebate.

I never saw any kind of a rebate check or credit, so after some months had passed, I reached out to see what was up only to learn that I no longer qualified.

Did not receive rebate for thermostat. Seemed to be too many "do this and this and this then this"

Other 4 It was just too hot this past summer

Unease about how to deregister thermostat/IM account at the end of rental contracts.

Not enough information 2 I didn't realize this was a program I was signing up for. I just thought I was signing up for a rebate.

Lack of information

Disliked creation of OE profiles 1 Really the thing I liked the "least" was that each time there was an event, a new "profile" was created on my Ecobee smart thermostat.

Issues with scheduling install 1 3 appointments before installed

Although several customers were dissatisfied with some aspect of their program experience, this did not result in dissatisfaction with I&M overall. A majority of customers (79%) reported that they were satisfied with I&M overall and few reported dissatisfaction with the company.




(n = 214) Very Satisfied 41% Somewhat Satisfied 38% Neither Satisfied or Dissatisfied 14% Somewhat Dissatisfied 4% Very Dissatisfied 2%

Yearly Trends


More people learned about the Home Energy Management program from the I&M website in 2018 compared to 2017 (nine percentage point increase) and less learned from a bill insert (21 percentage point decrease). Figure 12-11 provides a full summary.


There was a 23 point increase in survey respondents who indicated they decided to participate in the program because they wanted to get the thermostat rebate and a 15 point decrease among those who wanted to save money on energy costs (see Figure 12-12).


Figure 12-12 2017 and 2018 Reasons for Participating in Home Energy Management Program Comparison

From 2017 to 2018, there was a 29 percentage-point increase among survey respondents who had concerns prior to first signing up (see Table 12-27). During this time there was also a decrease in the share of survey respondents who indicated it was somewhat or very easy to enroll in the program (see Figure 12-13).

Table 12-27 2017 and 2018 Concerns About Participating When Signing Up Comparison

Response Percentage by

Response (2017)


(2018)


Yes 11% 40% 29

Figure 12-13 2017 and 2018 Ease of Enrolling in the Home Energy Management Program Comparison


There was an increase (67 percentage points) from 2017 to 2018 among survey respondents who indicated they self-installed their thermostat and a decrease (67 percentage points) among those who had the thermostat professionally installed (see Figure 12-14).

Figure 12-14 2017 and 2018 Thermostat Installation Comparison


12.5.1 Conclusions


Program ex ante savings fell short of the energy saving and demand saving targets. Program ex ante kWh and kW savings equaled 12% of the program goal. In total, 1,215 thermostats were rebated, and 1,493 customers had the Orchestrated Energy software installed.

The program is marketed through multiple channels. The marketing efforts included email campaigns, social media campaigns, direct mail advertising, and newsletters. Thirty-one percent of customers reported that they learned of the program from the program website. Additionally, 17% from an I&M newsletter, 15% learned of it through a bill insert or mailing, and 10% learned of it from a friend or relative.

The program is exploring strategies to increase enrollments including increasing incentives and reviewing marketing materials. Staff reported that they are reviewing incentive levels as one approach to increase enrollments, as well as reviewing marketing materials, messaging, and the enrollment process. Tendril indicated they are currently working with I&M on these changes and exploring ways to simplify (e.g., using less data to verify customers) the enrollment process.

A large share of participants reported awareness of other I&M program offerings. Seventy-seven percent of respondents reported being aware of other I&M program offerings. Of these customers, 5% reported that they became aware of the I&M programs through the IM Home App. The program seems to be reaching customers who are


generally aware of I&M program offerings, or the participants are becoming aware of the programs during the process of learning about the program or enrolling.

Participant concerns about enrolling centered on comfort and giving up control of the thermostat settings. Twenty-six participants reported that they had concerns with allowing I&M to alter their thermostat settings during event periods and the same number stated concerns about home comfort. The website materials include information on how the software manages home comfort.

Enrollment process is working well for participants. Eighty-one percent of respondents indicated that enrolling in the program was very or somewhat easy. Most participants found the information provided about the program to be clear. One area where the information was less clear was the information on how the IM Home would adjust your thermostat settings – 19% found this to be unclear. Nevertheless, 61% reported the information on rebates available as somewhat or very clear.

Few customers opted out of the program or the demand response events. Staff reported that the program opt-out rate was around 2 to 4% and that the event opt-out rate was 2%. Tendril stated that the event opt-out rate was “phenomenal” and that they have seen other thermostat program opt-out rates in the range of 20-50%. Most survey respondents (81%) also reported that they had never or infrequently changed the thermostat settings during an event.

Participants reported some impacts on home comfort. Thirty-eight percent of customers reported that their home was somewhat less comfortable during the events and 12% reported that it became a lot less comfortable.

Sixty-three percent of the participants reported that they were satisfied with the program overall, but some dissatisfaction noted. While two-thirds of respondents were satisfied with the program overall, one-third of respondents (n = 73) reported dissatisfaction with the program overall or one or more aspects of the program. Areas of lower satisfaction included the satisfaction with the IM Home app and the information provided about the program. Additionally, several comments cited frequent changes to thermostat settings, beyond those made during peak events, as an area of dissatisfaction.

Information about the program and the number of schedule changes were key factors in customers intentions to remain enrolled. Sixteen percent of respondents indicated that they were not very likely (rated the likelihood as less than 4) to remain enrolled in summer PY2019. Analysis of customer survey responses found that key predictors of remaining enrolled were their satisfaction with the information provided about the program, satisfaction with the signup process, and if they mentioned the number of schedule changes as a reason for dissatisfaction. Of interest is that perceived home comfort was not strongly related to intent to remain in the program.




Consider using messaging that emphasizes that control participants have over their thermostat. Issues of control were one of the more frequently mentioned concerns customers had about enrolling in the program. An approach to addressing this is to emphasize how the thermostat and the IM Home App gives customers control over their thermostat and energy use.

Clarify that IM Home will adjust temperatures during events and nonevents. A theme amongst dissatisfied customers was a lack of awareness and dislike of the frequent temperature changes. One approach may be to split the IM Home Saves Energy section of the website into two section on how it saves energy every day and how it saves energy during events.

Consider offering a second track that only alters thermostat settings during peak events. A second track may appeal to customers who want to limit the number of adjustments made to their thermostats.


13 Residential New Construction This chapter presents the results of both the impact and process evaluations of the 2018 New Construction Program that Indiana Michigan Power (I&M) offered to its residential customers during the period of January 2018 through December 2018.



Document sources of program awareness among participants (builders);

Assess differences in level of participation among builders;

Assess satisfaction among participating builders;




The New Construction Program is offered to home builders that construct their homes to exceed minimum building code standards (IECC 2009). Participating homes fall into one of three tiers of energy efficiency based on HERS index score; I&M Silver, I&M Gold, and I&M Platinum. Participants receive incentives ranging from 20 to 63 percent of the cost to upgrade and certify each home, depending on the home’s primary heating fuel type (natural gas or electric).

In addition to paying cash incentives, this program also represents a market transformation program, aimed at reducing multiple barriers to this higher level of construction standards. The program implementer will promote the program using the following strategies:

Direct mail campaign and trade ally rollout meetings;

Direct contact with home builders;

Home builder advisory group meetings;

Web based marketing via email and I&M Company website; and

Direct marketing to trade allies and builders.

Program participation is contingent upon an internal eligibility review and verification process. This process provides a first layer of assurance to I&M and the participating builders that the homes will meet program specifications and be more efficient than required by code (IECC 2009).





ADM conducted a stratified random sampling approach for each of the three efficiency tiers (I&M Silver, I&M Gold, and I&M Platinum), whereby we first selected a sample of representative home builders in each tier and then select a random sample of homes within each builder. Detailed engineering reviews will be completed on this sample of homes. The final survey sample was determined, in part, by the program participation level. ADM obtained a sample of 19 homes from the stratified random sampling approach.


The first aspect of conducting measurements of program activity was to verify the number of homes participating in the program. ADM reviewed the tracking system data on reported homes to verify that all homes were eligible for the program. Additionally, the tracking system was reviewed to ensure that the proper data fields required to support this evaluation as well as future evaluations are included. The tracking system was reviewed for completeness, accuracy, and efficiency.

Home and energy saving data were provided using Ekotrope. ADM reviewed the data provided by to ensure compatibility with REMrate. Ekotrope allows the export to REMRate, which allowed for easy comparison between the two data types.


A stratified random sample was used to capture the expected savings for each of the different Tiers of remodeled homes and for the program overall. Fuel summary reports from REMRate were collected for each home, which contain the savings for the efficient home relative to the user defined reference home (UDRH). In addition, the REMRate building file for each sampled home was inspected to ensure that energy efficiency improvements were consistent with typical upgrades found in other residential new construction programs. Homes with data in the Ekotrope format were exported to REMRate for analysis.


Through the review of the database of program homes, ex ante savings for the New Construction Program were found to be developed through the assignment of set values for each of the six types of efficient homes present in the PY2018 program tracking data, namely Silver Star All Electric, Silver Star Gas & Electric, Gold Star All Electric, Gold Star Gas & Electric, Platinum Star All Electric, and Platinum Star Gas & Electric. The database savings for each home type are summarized below in Table 13-1.


Table 13-1 New Construction Database Savings by Home Type

Home Tier Fuel Type kWh Savings in Database

Average kW Reduction in

Database

Silver All Electric 4,696 0.91

Silver Natural Gas & Electric 1,438 0.86

Gold All Electric 8,116 1.39

Gold Natural Gas & Electric 1,618 1.25

Platinum Natural Gas & Electric 1,644 1.29

These savings estimates were developed through the use of EnergyGauge home models. EnergyGauge incorporates models for each efficient home and compares them to the energy consumption of baseline homes. The savings in the database reflects the difference between the EnergyGauge home model outputs for the baseline home and the efficient home for each home tier and fuel type.

ADM determined verified kWh and kW savings using the previously described sample for each tier. Annual energy savings were calculated by determining the difference in energy usage between each home in the sample with a UDRH. According to the Indiana TRM, the UDRH is an exact replica of the rated home in size, structure, and climate zone, but the energy characteristics are defined by local code or building practices. The UDRH is the assumed baseline efficient home, which for I&M is a home that meets the minimum standards of IECC 2009, which is shown in Figure 13-1. The newly developed savings were then factored by the number of homes in the appropriate energy tier in order to determine program level energy savings. Annual demand reduction was calculated in the same manner, but also accounted for a coincident factor (CF) of 0.5, as per the Indiana TRM.


Figure 13-1 2009 IECC UDRH Specifications


The following figures present the kWh savings as a function of HERS rating for the All Electric homes and the Gas & Electric homes that were sampled for the evaluation effort. There appears to be a fairly strong relationship between HERS score and kWh savings for the All Electric homes, and less of a correlation for the Gas & Electric category. Table 13-2 summarizes the gross impact results of the Residential New Construction Program.


Figure 13-2 Ex Post kWh Savings vs. HERS Score - Electric

Figure 13-3 Ex Post kWh Savings vs. HERS Score - Gas/Electric


Home Type Number

of Homes

Number of Sampled Homes

Ex Ante kWh

Savings per Home

Ex Ante Total kWh

Savings

Ex Post kWh

Savings per Home

Ex Post Total kWh

Savings

Gross Realization

Rate

Platinum – Gas & Electric 29 3 1,644 47,676 3,512 101,846 214%

Gold – Gas & Electric 311 10 1,618 503,198 1,843 573,152 114%

Gold – Electric 8 1 8,116 64,928 7,339 58,710 90%


Home Type Number

of Homes

Number of Sampled Homes

Ex Ante kWh

Savings per Home

Ex Ante Total kWh

Savings

Ex Post kWh

Savings per Home

Ex Post Total kWh

Savings

Gross Realization

Rate

Silver- Gas & Electric 134 3 1,438 192,692 1,466 196,444 102%

Silver- Electric 47 2 4,696 220,712 5,237 246,137 112% Total 529 19 1,029,206 1,176,290 114%


Table 13-3 below summarizes the annual energy savings by housing energy tier and fuel type. The largest contributor to savings was the gold tier housing, which represents approximately 55% of ex ante savings with a realization rate of 111%.

The annual energy savings for the Indiana New Construction Program totaled 1,029,206 kWh with a realization rate of 114%.

Table 13-3 kWh Savings by House Tier

Home Type Ex Ante

Total kWh Savings

Ex Post Total kWh

Savings

Realization Rate

Platinum – Gas & Electric 47,676 101,846 214% Gold – Gas & Electric 503,198 573,152 114% Gold – Electric 64,928 58,710 90% Silver- Gas & Electric 192,692 196,444 102% Silver- Electric 220,712 246,137 112% Total 1,029,206 1,176,290 114%

Ex Ante Gross kWh

Savings Gross Audited kWh

Savings Gross Verified kWh

Savings Ex Post Gross kWh

Savings Gross Realization

Rate 1,029,206 1,029,206 1,029,206 1,176,290 114%


The total annual demand reduction for the Indiana Residential New Construction Program was 810.87 kW with a realization rate of 136%.


Table 13-4 Demand Reductions by House Tier

Home Type Ex Ante Total kW Reduction

Ex Post Total kW Reduction

Realization Rate

Platinum – Gas & Electric 37.41 31.53 84% Gold – Gas & Electric 388.75 406.30 105% Gold – Electric 11.12 63.59 572% Silver- Gas & Electric 115.24 83.85 73% Silver- Electric 42.77 225.60 527% Total 595.29 810.87 136%

Ex Ante Gross kW

Savings Gross Audited kW

Savings Gross Verified kW

Savings Ex Post Gross kW

Savings Gross Realization

Rate 595.29 595.29 595.29 810.87 136%



A survey of nine program participants (builders) was administered to collect data for use in estimating participant free ridership and spillover.


ADM conducted surveys of participating builders to estimate a net-to-gross ratio for the program. Free ridership scores were developed for each interviewed builder by analyzing responses to three lines of questioning: program influence, building practices in the absence of the program, and co-participation in other rebate programs. The scoring for each line of questioning is detailed below, followed by the algorithm for calculating the overall net-to-gross ratio.


Program Influence

The Program Influence indicator variable were calculated using the response to the following:

FR1: We would like to identify which, if any, aspects of the program were important in your decision to build homes to a higher efficiency standard than is required by code. Please rate each of the following factors on a scale of 0 to 0, where 0 means that the factor was “not at all important” in your decision to build energy efficient homes, and 10 means that the factor was “extremely important” in your decision to build energy efficient homes.

FR 2: How, if at all, have any of the resources offered by the program affected your success in selling energy efficient homes?

FR 3: Could you please tell me, in your own words, the influence the I&M Residential New Construction Program had on your building practices?


Question FR1 provides respondents with a list of factors that are associated with the I&M program. Respondents rated the importance of each in their decision making process. These factors were:

Information from CLEAResult or I&M staff;

Technical assistance from HERS raters;

The incentive provided by the program; and

Program marketing and program informational literature.

The unadjusted Program Influence score was defined as the maximum rating provided by respondents for the above factors in FR1, converted to a percentage by dividing the score by 10. FR2 and FR3 serve as free ridership mitigation variables, where respondents providing open-ended commentary indicating that the program has positively influenced their sales of efficient homes, or has affected their building practices, receive a 50% reduction in free ridership for this variable. For example, a respondent providing a rating of 6 for Information from CLEAResult or I&M staff, and a rating of 8 for the incentive provided by the program, would receive a Program Influence score of (8/10) = 80%. This represents a free ridership level of 20%. If this respondent also states that the program has positively affected their sales of efficient homes or their building practices, their free ridership rate would be adjusted to (0.2/2.0) = 0.1, or 10%, resulting in a final Program Influence Score of 90%.

Behavior Absent Program

The Behavior Absent Program indicator variable was calculated using the response to the following:

FR4: On a scale of 0 to 10, where 0 represents “not at all likely” and 10 represents “extremely likely,” how likely would you be to build your homes to the same efficiency standard if the I&M Residential New Construction Program and incentive were not available?”; and

FR5: If the I&M program and incentive were not available, how likely would your company be to build fewer homes to the same efficiency standard? Please answer on the same 0 to 10 scale where 0 means “not at all likely” and 10 means “extremely likely”.

FR6: What factors influence decisions to include energy efficient equipment/materials/construction practices which exceed IECC 2009 building code requirements?

Responses to FR4 were divided by 10 to calculate the level of unadjusted free ridership for the behavior absent program variable. FR5 and FR6 serve as free ridership mitigation factors, where respondents providing a score of 5 or greater receive a 50% reduction in free ridership for the behavior in the absence of the program indicator and respondents providing an open-ended response to FR6 indicating that their decision to build efficient homes is affected by financial factors receive another 50% reduction in free ridership for this variable. Thus, a respondent meeting both of these mitigation criteria would receive a 100% reduction in free ridership for this variable.


After the adjustment was applied, the behavior in the absence of the program score was calculated by subtracting the adjusted behavior absent program free ridership from 1. For example, a respondent providing a response of 4 to FR4 would receive an unadjusted behavior absent program of the program free ridership value of (4/10) = 0.4, or 40%. If this respondent provides an answer of 6 to FR6, their adjusted behavior in the absence of the program free ridership value would be (0.4/2.0) = 0.2, or 20%. Finally, their behavior absent program score was calculated as (1.0 – 0.2) = 0.8, or 80%.

Builder net-to-gross ratios were based on the Program Influence Score and the Behavior Absent Program Score, as follows, where Program Influence accounts for 60% of the net-to-gross score and Behavior Absent Program accounts for 40% of the net-to-gross score:

Net-to-Gross Score = (0.6 * Program Influence Score) + (0.4 * Behavior Absent Program Score)

The net-to-gross scores were then weighted by the number of participating homes that each responding builder had in the program during PY2018. As some builder respondents may have completed the builder survey as part of the PY2018 EM&V effort, ADM considered these respondents’ prior responses in addition to their PY2018 responses when making the net-to-gross determination, in order to account for prior influence the program may have had on builders’ business practices and energy efficiency engagement.


To estimate potential spillover impacts, builders are asked if they completed any homes in the service territory that did not receive program incentives but would have qualified for program incentives. To determine if this additional construction was influenced by the program, builders are asked the following two questions:

Using a scale where 0 means “not at all important” and 10 means “very important”, how important was your experience with the program in your decision to build these program qualifying homes?

Using a scale where 0 means “not at all likely” and 10 means “very likely”, how likely is that you would have built these additional program qualifying homes if you had NOT participated in the program?

The response to these question are used to develop a spillover score as follows:




Table 13-5 summarizes the free-ridership scores by home builder for the Residential New Construction Program.


Table 13-5 Average Free Ridership Score by Home Builder

Home Builder Survey Year Free Ridership Score

Builder 1 2017 85%

Builder 2 2017 45%

Builder 3 2017 58%

Builder 4 2017 60%

Builder 5 2017 0%

Builder 6 2017 36%

Builder 7 2017 0%

Builder 8 2017 0%

Builder 9 2017 78%

Builder 10 2018 60%

No respondents reported spillover qualifying homes.


Table 13-6 summarizes the net ex post kWh savings of the Residential New Construction Program. The annual net savings totaled 1,010,080 kWh. The net-to-gross ratio is 86%.


Ex Ante Gross kWh

Savings

Gross Audited

kWh Savings

Gross Verified

kWh Savings

Ex Post Gross kWh

Savings

Gross Realization

Rate


Net-to-Gross Ratio

Ex Post Net

Lifetime kWh

Savings

1,029,206 1,029,206 1,029,206 1,176,290 114% 1,010,080 86% 25,252,012


Table 13-7 summarizes the net ex post kW demand reduction of the Residential New Construction Program. The net demand reduction equaled 690.97 kW. The net-to-gross ratio is 85%.







Gross Realization

Rate


Net-to-Gross Ratio

595.29 595.29 595.29 810.87 136% 690.97 85%




Table 13-8 Summary of Data Collection Activities for the Residential New Construction Program

Data Collection Activity Mode of Data Collection Sample Size Builder survey Email 2 Staff interviews Telephone in-depth interviews 2



Table 13-9 summarizes the number and share of homes completed by Tier. As shown, Gold Star homes accounted for 61% of the completed homes. Silver Star and Platinum Star homes accounted for 34% and 5% of homes, respectively. The kWh acquisition cost was notably higher for Platinum gas and electric homes and for Gold gas and electric homes, than other tiers and home types.

Table 13-9 Number and Share of Completed Homes by Tier and Home Type

Tier/Home Type Number of Homes

Percent of Homes

Total Ex Ante kWh Savings

Ex Ante kWh

Savings per Home

Rebate Dollars per Ex

Ante kWh

Gold – Gas & Electric 311 59% 503,198 1,618 $500 $0.31 Silver- Gas & Electric 134 25% 192,692 1,438 $250 $0.17 Silver- Electric 47 9% 220,712 4,696 $600 $0.13 Platinum – Electric 29 5% 47,676 1,644 $600 $0.36 Gold – Electric 8 2% 64,928 8,116 $600 $0.11

Figure 13-4 summarizes the number of homes completed by builders in 2018. As shown, the top four builders completed a large share of program homes.


Figure 13-4 Summary of Number of Homes Completed by Builders



ADM interviewed the I&M energy efficiency consumer program coordinator and a manager and marketing outreach specialist from CLEAResult to gain insights in the Residential New Home Construction program for 2018. The key findings of the interviews are summarized below.


Program exceeded savings goals for PY2018. The Residential New Construction program exceeded energy savings goals by 18% and was under budget. The year-end savings was over one million kWh (121% of goal). 2018 was described as the “largest and best year in terms of applications and participation” and “very smooth year”. While there were greater energy savings in 2017, there were additional builders in 2018, which helped to further “spread out” the program.

Increased number of all-electric homes were built compared to prior years. There was a larger number of all-electric homes in 2018 compared to any other years since the program began and there are now four all-electric builders participating. A total of 504 homes were completed in 2018. There were 55 all-electric homes completed in 2018 compared to 12 in 2015, suggesting a general trend towards increasing all electric home participation. There were a significant number of permits pulled each month and the number of permits pulled was the greatest since the program launched.

Strong housing market and economy helped the program succeed. I&M noted that the housing market has remained strong in 2018 and has not observed any downturn. Participation in the program increased and staff believe it is due to economic conditions in the region (e.g., low unemployment and strong housing market). Staff did not see a downturn in the housing market in 2018 and speculated that 2019 will be a strong year like 2018.


Program is well-received by builders. The program has received positive feedback from builders, and they have indicated they appreciate that the paperwork for the program is light. Staff also suggest continuing to streamline the application process and simplify it as much as possible.

Future challenges include reduced savings from LEDs and cost of HERS raters. Future challenges may include adjusting for reduced savings from LED and that impact to the


HERS ratings. CLEAResult is focusing on designing a program that does not include LEDs, while still achieving savings, for 2019. CLEAResult suggested more builders utilize heat pumps, however, expressed it is difficult for builders to implement due to costs.

CLEAResult staff suggested to get more involvement with more builders and to focus on smaller builders but noted that there may be additional challenges for smaller builders such as the cost of HERS raters.

Another suggestion included providing incentives to the HERS raters, as they are steering builders toward greater energy efficiency savings. Staff believe that increasing the number of HERS raters will be advantageous for the program. CLEAResult and I&M staff did not anticipate any significant changes to 2019 but are gearing up for bigger changes for 2020 as I&M would like to focus on electrically heated homes in that year.

No changes to communication structure, data management, or QA/QC procedures in PY2018. There have not been any significant changes to the communication structure or any concerns in 2018. There were no changes to data management for the program and staff indicated the data is kept current to manage the program. There have not been any significant changes to the QA/QC procedures in 2018 and no concerns were stated.

13.4.1.3 Builder Survey Findings

ADM conducted surveys with builders who built qualifying energy efficiency homes which an incentive was received in 2018. Two builders completed the survey for the Residential New Construction program in PY2018. This sample size limits the inferences that can be made from the responses. The key findings are briefly summarized below and the tabulated responses to all questions are presented in the Volume II report.

The key findings of the survey are as follows:

Both respondents believed buyers are interested in the primary benefits of energy efficient construction, which include reducing energy costs, improving home comfort, and knowing that they purchased a high-quality home. One respondent reported using less energy for environmental reasons is also important to buyers.

Both respondents indicated the incentive provided by the program and the program marketing and informational literature were not very important in their decision to build energy efficient homes – rated as a 2 on a scale from 0 (not at all likely) and 10 (extremely likely).

Both respondents indicated they would not have been very likely to build homes to the same efficiency standard if the program and incentives were not available – rated as a 1 on a scale from 0 (not at all likely) and 10 (extremely likely).

Both respondents indicated they perform duct leakage testing on all their homes and perform infiltration testing on a sample of their homes.


Neither of the respondents utilized a HERS rater for homes in I&M’s service territory prior to their enrollment in the program.

Neither respondent believed there are any potential barriers or disadvantages to participating in I&M’s Residential New Construction Program that might discourage them or other builders from participating in it.

Both respondents were very satisfied (rated as a 5 on a 5-point scale, where 1 means “very dissatisfied” and 5 means “very satisfied”) with the efficiency requirements of the I&M Residential New Construction program, the program application process, and the program overall. Furthermore, both builders plan on participating in 2019.


13.5.1 Conclusions

The following presents a selection of key conclusions from the second year of program operations:


Program exceeded savings goals for PY2018. The Residential New Construction program ex ante savings equaled 121% of the program goal and was under budget. 2018 was described as the “largest and best year in terms of applications and participation” and “very smooth year”. While there were greater energy savings in 2017, there were additional builders in 2018, which helped to further “spread out” the program.

Increased number of all-electric homes were built compared to prior years. There was a larger share of all-electric homes in 2018 compared to other years and there are now four all-electric builders participating. There were 55 all-electric homes completed in 2018, a number higher than any other year of program operations.

Strong housing market and economy help program succeed. I&M noted that the housing market has remained strong in 2018 and has not observed any downturn. Participation in the program increased and staff believe it is due to economic conditions in the region (e.g., low unemployment and strong housing market). Staff did not see a downturn in the housing market in 2018 and speculated that 2019 will be a strong year like 2018.


Future challenges include reduced savings from LEDs and cost of HERS raters. Future challenges may include adjusting for reduced savings from LED and that impact to the


HERS ratings. CLEAResult is focusing on designing a program that does not include LEDs, while still achieving savings, for 2019. CLEAResult suggested more builders utilize heat pumps, however, expressed it is difficult for builders to implement due to costs.

No changes to communication structure, data management, or QA/QC procedures in PY2018. There have not been any significant changes to the communication structure or any concerns in 2018. There were no changes to data management for the program and staff indicated the data are kept current to manage the program. There have not been any significant changes to the QA/QC procedures in 2018 and no concerns were stated.

Program is well-received by builders. All survey respondents indicated they were very satisfied with the efficiency requirements of the I&M Residential New Construction program, the program application process, and the program overall.


Consider adjusting program parameters to focus on electricity savings: While a home's HERS score is directly correlated to its overall efficiency, there is not necessarily a linear relationship between the HERS score and electricity usage for homes that use both electricity and natural gas. The HERS score is a function of natural gas usage, electricity usage, and water usage, and two homes with the same HERS score may achieve significantly different electricity savings depending on which efficient measures were implemented. ADM recommends that I&M consider more specific eligibility requirements for gas and electric homes, such as the presence of a high efficiency air conditioner, efficient lighting, or air sealing improvements, in order to improve the predictability of electricity savings as a function of HERS score for gas and electric homes that participate in the program.


14 Cost Effectiveness Evaluation The following cost effectiveness tests were performed for each program: Total Resource Cost (TRC) test, Utility Cost Test (UCT), Participant Cost Test (PCT), and Ratepayer Impact Measure (RIM) test. A score above one signifies that, from the perspective of the test, the program benefits were greater than the program costs. The benefits and costs associated with each test are defined in Table 14-1.

Table 14-1 Summary of Benefits and Costs Included in each Cost Effectiveness Test

Variable Definition PCT UCT RIM TRC

Benefit Cost Benefit Cost Benefit Cost Benefit Cost

Incentives Incentives paid to customers. ✓ ✓ ✓

Program Installation

Costs

Installation costs paid by program.

✓ ✓ ✓

Bill Savings / Lost

Revenue

Reduction in electricity costs faced

by customers as a result of

implementation of program measures.

Equal to revenue lost to the utility.

✓ ✓

Avoided Energy Costs

Energy-related costs avoided by utility.

✓ ✓ ✓

Avoided Capacity

Costs

Capacity-related costs avoided by utility, including

T&D.

✓ ✓ ✓

Incremental Costs

Incremental costs associated with

measure implementation, as

compared with what would have been

done in absence of program.

✓ ✓

Program Overhead

Costs

Program costs other than incentive or installation costs.

✓ ✓ ✓

Table 14-2 through Table 14-9 summarize key financial benefit and cost inputs for the various tests along as well as the test results for each residential program.

As stated in section 1.3, the Home Energy Products scores account for the costs and benefits associated with the program's components: Home Energy Products - Appliances and Home Energy Products - Lighting. Additionally, the Home Energy Engagement scores account for the costs and


benefits associated with the Home Energy Reports, Usage Alerts, and Residential Online Energy Check-up Programs.

Table 14-2 Home Appliance Recycling Program Cost Test Inputs and Results

Table 14-3 Home Energy Products Program Cost Test Inputs and Results

Table 14-4 Home Energy Engagement Program Cost Test Inputs and Results

Benefit Cost Benefit Cost Benefit Cost Benefit CostIncentives 166,800$ 166,800$ 166,800$ Program Installation Costs -$ -$ -$ Bill Savings (NPV) 949,360$ Lost Revenue (NPV) 1,163,425$ Avoided Energy Costs (NPV) 607,090$ 607,090$ 607,090$ Avoided Capacity Costs (NPV) 152,665$ 152,665$ 152,665$ Avoided T&D Costs (NPV) -$ -$ -$ Incremental Costs -$ -$ Program Overhead Costs 547,917$ 547,917$ 547,917$ Total BenefitsTotal CostsTest Score N/A 1.06 0.40 1.39

547,917$

Variable PCT UCT RIM

714,717$ 1,878,142$ -$

TRC

1,116,160$ 759,755$ 759,755$ 759,755$

Benefit Cost Benefit Cost Benefit Cost Benefit CostIncentives 2,241,034$ 2,241,034$ 2,241,034$ Program Installation Costs -$ -$ -$ Bill Savings (NPV) 4,942,712$ Lost Revenue (NPV) 6,702,569$ Avoided Energy Costs (NPV) 3,862,971$ 3,862,971$ 3,862,971$ Avoided Capacity Costs (NPV) 1,084,460$ 1,084,460$ 1,084,460$ Avoided T&D Costs (NPV) -$ -$ -$ Incremental Costs 2,235,649$ 2,235,649$ Program Overhead Costs 978,735$ 978,735$ 978,735$ Total BenefitsTotal CostsTest Score

7,183,747$ 4,947,431$ 4,947,431$ 4,947,431$

VariablePCT UCT RIM TRC

2,235,649$ 3,219,770$ 9,922,339$ 3,214,384$ 3.21 1.54 0.50 1.54

Benefit Cost Benefit Cost Benefit Cost Benefit CostIncentives -$ -$ -$ Program Installation Costs -$ -$ -$ Bill Savings (NPV) 4,033,915$ Lost Revenue (NPV) 4,563,984$ Avoided Energy Costs (NPV) 2,282,779$ 2,282,779$ 2,282,779$ Avoided Capacity Costs (NPV) 330,674$ 330,674$ 330,674$ Avoided T&D Costs (NPV) -$ -$ -$ Incremental Costs -$ -$ Program Overhead Costs 1,848,745$ 1,848,745$ 1,848,745$ Total BenefitsTotal CostsTest Score

4,033,915$ 2,613,454$ 2,613,454$ 2,613,454$


-$ 1,848,745$ 6,412,729$ 1,848,745$ N/A 1.41 0.41 1.41


Table 14-5 Home Weatherproofing Program Cost Test Inputs and Results

Table 14-6 Residential Income Qualified Weatherproofing Program Cost Test Inputs and Results

Table 14-7 Schools Energy Education Program Cost Test Inputs and Results

Table 14-8 Home Energy Management Program Cost Test Inputs and Results

Benefit Cost Benefit Cost Benefit Cost Benefit CostIncentives 111,630$ 111,630$ 111,630$ Program Installation Costs 11,392$ 11,392$ 11,392$ Bill Savings (NPV) 290,655$ Lost Revenue (NPV) 420,823$ Avoided Energy Costs (NPV) 252,543$ 252,543$ 252,543$ Avoided Capacity Costs (NPV) 43,393$ 43,393$ 43,393$ Avoided T&D Costs (NPV) -$ -$ -$ Incremental Costs 128,927$ 128,927$ Program Overhead Costs 248,865$ 248,865$ 248,865$ Total BenefitsTotal CostsTest Score

402,285$ 295,935$ 295,935$ 295,935$

Variable PCT UCT RIM TRC

128,927$ 371,888$ 792,711$ 389,185$ 3.12 0.80 0.37 0.76

Benefit Cost Benefit Cost Benefit Cost Benefit CostIncentives 215,375$ 215,375$ 215,375$ Program Installation Costs 8,384$ 8,384$ 8,384$ Bill Savings (NPV) 218,192$ Lost Revenue (NPV) 300,372$ Avoided Energy Costs (NPV) 174,599$ 174,599$ 174,599$ Avoided Capacity Costs (NPV) 33,208$ 33,208$ 33,208$ Avoided T&D Costs (NPV) -$ -$ -$ Incremental Costs 180,224$ 180,224$ Program Overhead Costs 304,122$ 304,122$ 304,122$ Total BenefitsTotal CostsTest Score

433,567$ 207,807$ 207,807$ 207,807$


180,224$ 527,881$ 828,253$ 492,729$ 2.41 0.39 0.25 0.42

Benefit Cost Benefit Cost Benefit Cost Benefit CostIncentives -$ -$ -$ Program Installation Costs -$ -$ -$ Bill Savings (NPV) 1,511,700$ Lost Revenue (NPV) 1,979,121$ Avoided Energy Costs (NPV) 1,101,703$ 1,101,703$ 1,101,703$ Avoided Capacity Costs (NPV) 231,678$ 231,678$ 231,678$ Avoided T&D Costs (NPV) -$ -$ -$ Incremental Costs -$ -$ Program Overhead Costs 655,094$ 655,094$ 655,094$ Total BenefitsTotal CostsTest Score

-$ 655,094$ 2,634,215$ 655,094$ N/A 2.04 0.51 2.04

1,511,700$ 1,333,381$ 1,333,381$ 1,333,381$


Benefit Cost Benefit Cost Benefit Cost Benefit CostIncentives 337,301$ 337,301$ 337,301$ Program Installation Costs -$ -$ -$ Bill Savings (NPV) 355,971$ Lost Revenue (NPV) 511,383$ Avoided Energy Costs (NPV) 307,147$ 307,147$ 307,147$ Avoided Capacity Costs (NPV) 753,824$ 753,824$ 753,824$ Avoided T&D Costs (NPV) -$ -$ -$ Incremental Costs 278,677$ 278,677$ Program Overhead Costs 985,032$ 985,032$ 985,032$ Total BenefitsTotal CostsTest Score

693,273$ 1,060,971$ 1,060,971$ 1,060,971$


278,677$ 1,322,334$ 1,833,717$ 1,263,709$ 2.49 0.80 0.58 0.84


Table 14-9 Residential New Construction Program Cost Test Inputs and Results

Benefit Cost Benefit Cost Benefit Cost Benefit CostIncentives 241,800$ 241,800$ 241,800$ Program Installation Costs -$ -$ -$ Bill Savings (NPV) 788,043$ Lost Revenue (NPV) 1,330,032$ Avoided Energy Costs (NPV) 859,386$ 859,386$ 859,386$ Avoided Capacity Costs (NPV) 1,067,841$ 1,067,841$ 1,067,841$ Avoided T&D Costs (NPV) -$ -$ -$ Incremental Costs 700,976$ 700,976$ Program Overhead Costs 286,008$ 286,008$ 286,008$ Total BenefitsTotal CostsTest Score

1,029,843$ 1,927,227$ 1,927,227$ 1,927,227$


700,976$ 527,808$ 1,857,839$ 986,984$ 1.47 3.65 1.04 1.95


2018 indiana residential portfolio em&v report · 2018 indiana residential portfolio . em&v...

Documents