20160914Roy Exhibit 2.pdf

DAVID J. MEYER
VICE PRESIDENT AND CHIEF COUNSEL OF
REGULATORY & GOVERNMENTAL AFFAIRS
AVISTA CORPORATION
P.O. BOX 3727
1411 EAST MISSION AVENUE, MSC 27
SPOKANE, WASHINGTON 99220-3727
TELEPHONE: ( 50 9) 4 95-4316
EMAIL: david .me yer@ a vi stacorp .com
RECE IVED
2ll\6 SEP \ 4 ~M \O: I I
I 1, \
·1 ._. , I
. ._:J1:,L\C
-~011, MISS10t
BEFORE THE IDAHO PUBLIC UTILITIES COMMISSION
IN THE MATTER OF THE APPLICATION
OF AVISTA CORPORATION FOR A
FINDING OF PRUDENCE FOR 2014-2015
EXPENDITURES ASSOCIATED WITH
PROVIDING ELECTRIC AND NATURAL GAS
ENERGY EFFICIENCY SERVICE IN THE
STATE OF IDAHO
CASE NO. AVU-E-16-0~
& rm. AJG 6 IA
EXHIBIT NO. 2
LYNN ROY
REPRESENTING
Nexant, Inc
FOR AVISTA CORPORATION
(ELECTRIC AND NATURAL GAS)
REPORT t,1Nexanr
Reimagine tomorrow.
Impact Evaluation of Idaho 2014-2015
Energy Efficiency Programs
Submitted to Avista Utilities
June 2, 2016
Principal Authors:
Lynn Roy, Mary-Hall Johnson, Patrick Burns, Jesse Smith,
Wyley Hodgson, Cherlyn Seruto, Nathanael Benton,
Greg Sidorov, Shannon Hees; Nexant, Inc.
Ryan Bliss, Paul Schwartz; Research Into Action
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 1 of 212
Contents
1 Executive Summary ............................................................................. 1
1.1 Evaluation Methodology and Activities .................................................... 1
1.2 Summary of Impact Evaluation Results ................................................... 2
1.3 Conclusions and Recommendations ........................................................ 4
1.3.1 Nonresidential Programs ................................................................... 5
1.3.2 Residential Programs ........................................................................ 6
2 Introduction ........................................................................................ 10
2.1 Purpose of Evaluation .............................................................................. 10
2.2 Program Summary ................................................................................... 10
t..1Nexanr
2.2.1 Nonresidential .................................................................................. 1 O
2. 2. 1. 1 Site Specific .......................................................................... 1 O
2.2.1.2 Prescriptive Lighting .............................................................. 12
2. 2. 1. 3 Energy Smart Grocer ............................................................. 14
2.2.1.4 Food Service Equipment ....................................................... 16
2. 2. 1. 5 Green Motor Rewind ............................................................. 17
2.2.1.6 Commercial HVAC Variable Frequency Drive (VFD) Program
.............................................................................................. 18
2.2.1. 7 Commercial Clothes Washers ............................................... 18
2. 2. 1. 8 Power Management for Personal Computer Networks ......... 19
2.2.1.9 Commercial Windows & Insulation ........................................ 19
2. 2. 1. 10 Commercial Water Heaters ................................................... 19
2. 2. 1. 11 Standby Generator Block Heater .......................................... 20
2.2.2 Small Business ................................................................................ 20
2.2.3 Residential ....................................................................................... 21
2.2.3. 1 Appliance Recycling .............................................................. 22
2.2.3.2 HVAC Program ..................................................................... 23
2.2.3.3 Water Heat ............................................................................ 23
2.2.3.4 ENERGY STAR® Homes ....................................................... 23
2. 2. 3. 5 Fuel Efficiency Program ........................................................ 24
Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 2 of 212
2. 2. 3. 6 Residential Lighting ............................................................... 24
2.2.3. 7 Shell Program ....................................................................... 25
2. 2. 3. 8 Home Energy Reports ........................................................... 25
2.2.3.9 Low Income ........................................................................... 25
2.3 Program Participation Summary ............................................................. 26
2.4 Evaluation Goals and Objectives ............................................................ 27
3 Impact Evaluation Methodology ........................................................ 29
3.1 Understanding the Program Context ...................................................... 29
3.2 Designing the Sample .............................................................................. 29
3.3 Database Review ...................................................................................... 32
3.4 Verifying the Sample -Gross Verified Savings ..................................... 33
3.4.1 Document Audit ............................................................................... 33
3.4.2 Telephone Survey ............................................................................ 34
3.4.3 On site Measurement and Verification .............................................. 34
3.4.4 Billing Analysis ................................................................................. 35
4 Nonresidential Impact Evaluation ..................................................... 38
4.1 Overview ................................................................................................... 38
4.2 Prescriptive Lighting ................................................................................ 40
4.2.1 Overview .......................................................................................... 40
4.2.2 Program Achievements and Participation Summary ........................ 40
4.2.3 Methodology .................................................................................... 40
4. 2. 3. 1 Sampling ............................................................................... 41
4.2.3.2 Document Audits ................................................................... 41
4.2.3.3 Field Inspections ................................................................... 41
4.2.3.4 Impact Analysis Methods ...................................................... 42
4.2.4 Findings and Recommendations ..................................................... 44
4.3 Prescriptive EnergySmart Grocer ........................................................... 45
'-"Nexanr
4.3.1 Overview .......................................................................................... 45
4.3.2 Program Achievements and Participation Summary ........................ 45
4.3.3 Methodology .................................................................................... 46
4.3.3.1 Sampling Approach ............................................................... 46
4.3.3.2 Document Audits ................................................................... 46
4.3.3.3 Field Inspections ................................................................... 46
Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs ii
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 3 of 212
4. 3. 3. 4 Impact Analysis Methods ...................................................... 4 7
4.3.4 Findings and Recommendations ..................................................... 48
4.4 Prescriptive Non-Lighting Other Programs ........................................... 49
4.4.1 Overview .......................................................................................... 49
4.4.2 Program Achievements and Participation Study .............................. 50
4.4.3 Methodology .................................................................................... 51
4. 4. 3. 1 Sampling ............................................................................... 51
4.4.3.2 Document Audits ................................................................... 52
4.4. 3. 3 Field Inspections ................................................................... 52
4. 4. 3.4 Impact Analysis Methods ...................................................... 54
4.4.4 Findings and Recommendations ..................................................... 56
4.5 Site Specific .............................................................................................. 58
4.5.1 Overview .......................................................................................... 58
4.5.2 Program Achievements and Participation Summary ........................ 58
4. 5. 3 Methodology .................................................................................... 59
4. 5. 3. 1 Sampling ............................................................................... 60
4. 5. 3. 2 Document Audits ................................................................... 60
4. 5. 3. 3 Field Inspections ................................................................... 60
4. 5. 3. 4 Project-Specific Billing Analysis ............................................ 63
4.5.3.5 Algorithm-Based Impact Analysis Methods ........................... 63
4.5.4 Findings and Recommendations ..................................................... 64
4.6 Nonresidential Sector Results Summary ............................................... 66
5 Small Business Impact Evaluation ................................................... 67
5.1 Overview ................................................................................................... 67
5.2 Program Achievements and Participation Summary ............................ 67
5. 2. 1. 1 Sampling ............................................................................... 67
5.2.2 Document Audits ............................................................................. 68
5.2.3 Onsite Inspections ........................................................................... 68
5.2.4 Impact Analysis Methods ................................................................. 69
5.3 Findings and Recommendations ............................................................ 70
5. 3. 1. 1 Deemed Savings for Faucet Aerators ................................... 70
5.3.1.2 Deemed Savings for Pre-Rinse Spray Valves ....................... 71
6 Residential Impact Evaluation ........................................................... 72
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs iii
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 4 of 212
6.1 Overview ................................................................................................... 72
6.2 Residential Appliance Recycling ............................................................ 74
6.2.1 Overview .......................................................................................... 74
6.2.2 Program Achievements and Participation Summary ........................ 74
6.2.3 Methodology .................................................................................... 75
6.2.4 Findings and Recommendations ..................................................... 75
6.3 HVAC Program .......................................................................................... 76
6.3.1 Overview .......................................................................................... 76
6.3.2 Program Achievements and Participation Summary ........................ 76
6.3.3 Methodology .................................................................................... 77
6. 3. 3. 1 Air Source Heat Pump .......................................................... 78
6.3.3.2 Variable Speed Fan Motor .................................................... 79
6. 3. 3. 3 Smart Thermostat ................................................................. 80
6.3.4 Findings and Recommendations ..................................................... 81
6.3.4.1 Air Source Heat Pump .......................................................... 81
6. 3.4. 2 Variable Speed Fan Motor .................................................... 82
6.3.4.3 Smart Thermostat ................................................................. 83
6.3.5 Program Results .............................................................................. 86
6.4 Water Heat Program ................................................................................. 87
6.4.1 Overview .......................................................................................... 87
6.4.2 Program Achievements and Participation Summary ........................ 87
6.4.3 Methodology .................................................................................... 88
6.4.4 Findings and Recommendations ..................................................... 91
6.5 ENERGY STAR® Homes .......................................................................... 91
6.5.1 Overview .......................................................................................... 91
6.5.2 Program Achievements and Participation Summary ........................ 92
6.5.3 Methodology .................................................................................... 92
6.5.4 Findings and Recommendations ..................................................... 94
6.5.5 Program Results .............................................................................. 94
6.6 Fuel Efficiency .......................................................................................... 95
6.6.1 Overview .......................................................................................... 95
6.6.2 Program Achievements and Participation Summary ........................ 95
6.6.3 Methodology .................................................................................... 96
6.6.4 Findings and Recommendations ................................................... 100
6.6.5 Program Results ............................................................................ 102
6.7 Residential Lighting Program ................................................................ 102
L.-1Nexanr
6.7.1 Overview ........................................................................................ 102
Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs iv
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 5 of212
6.7.2 Program Achievements and Participation Summary ...................... 103
6.7.3 Methodology .................................................................................. 104
6. 7.3. 1 Total Program Bulbs ........................................................... 105
6. 7.3.2 Hours of Use ....................................................................... 107
6. 7.3.3 Delta Watts ......................................................................... 107
6. 7. 3. 4 Interactive Effects ................................................................ 109
6. 7.3.5 Installation Rate .................................................................. 109
6. 7. 3. 6 Cross-Sector Sales Leakage .............................................. 110
6. 7.4 Findings and Recommendations ................................................... 111
6.8 Shell Program ......................................................................................... 114
6.8.1 Overview ........................................................................................ 114
6.8.2 Program Achievements and Participation Summary ...................... 114
6.8.3 Methodology .................................................................................. 115
6.8.4 Findings and Recommendations ................................................... 116
6. 8. 4. 1 Shell Rebate Measures ....................................................... 116
6.8.5 Program Results ............................................................................ 118
6.9 Opower Behavioral Program ................................................................. 119
6.9.1 Overview ........................................................................................ 119
6.9.2 Program Achievements and Participation Summary ...................... 120
6.9.3 Methodology .................................................................................. 120
6. 9. 3. 1 Data Sources and Management ......................................... 120
6.9.3.2 Equivalence Testing ............................................................ 121
6. 9. 3. 3 Regression Analysis ............................................................ 122
6. 9. 3. 4 Overlap Analysis ................................................................. 124
6.9.4 Findings and Recommendations ................................................... 124
6.9.4.1 Per-Home kWh and Percent Impacts .................................. 124
6.9.4.2 Aggregate Impacts .............................................................. 126
6.9.4.3 Precision of Findings ........................................................... 126
6.9.4.4 Savings Patterns ................................................................. 127
6. 9. 4. 5 Gas Savings ........................................................................ 129
6.10 Low lncome ............................................................................................. 131
t-1Nexanr
6.10.1 Overview ........................................................................................ 131
6.10.2 Program Achievements and Participation Summary ...................... 131
6.10.3Methodology .................................................................................. 134
6.10.4 Findings and Recommendations ................................................... 136
6. 10.4. 1 Non-Lighting Conservation and Fuel Conversion Homes ... 136
6.10.4.2Lighting Conservation ......................................................... 137
Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs V
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 6 of 212
6.10.5 Program Results ............................................................................ 138
6.11 Residential Sector Results Summary ................................................... 138
7 Conclusions and Recommendations .............................................. 140
7.1 Summary ................................................................................................. 140
7.2 Impact Findings ...................................................................................... 140
7.3 Conclusions and Recommendations .................................................... 141
7.3.1 Nonresidential Programs ............................................................... 142
7.3.1. 1 Site Specific Program .......................................................... 142
7.3.1.2 Prescriptive Lighting Program ............................................. 142
7.3.1.3 EnergySmart Grocer Program ............................................ 143
7.3.1.4 Prescriptive Non-Lighting Other Programs .......................... 143
7. 3. 1. 5 Small Business Program ..................................................... 143
7.3.2 Residential Programs .................................................................... 143
7. 3. 2. 1 Appliance Recycling ............................................................ 144
7.3.2.2 HVAC Program ................................................................... 144
7.3.2.3 Water Heat .......................................................................... 145
7.3.2.4 ENERGY STAR® Homes ................................................... 145
7. 3. 2. 5 Fuel Efficiency. .................................................................... 145
7.3.2.6 Residential Lighting ............................................................. 146
7. 3. 2. 7 Shell Program ..................................................................... 146
7.3.2.8 Opower Program ................................................................. 147
7.3.2.9 Low Income Program .......................................................... 147
8 Residential Lighting Study .............................................................. 148
8.1 Methodology ........................................................................................... 148
t-1Nexanr
8.1.1 Household Sampling Approach ..................................................... 148
8.1.2 Logger Deployment Sampling Approach ....................................... 151
8.1.3 Primary Data Collection ................................................................. 152
8. 1. 3. 1 Recruitment & Participant Criteria ....................................... 152
8.1.3.2 Lighting Inventory ................................................................ 153
8. 1. 3. 3 Measurement Activities ....................................................... 153
8.1.4 Data Analysis ................................................................................. 154
8. 1. 4. 1 Data Cleaning ..................................................................... 154
8. 1. 4. 2 Development of Weights ..................................................... 155
8. 1. 4. 3 Hours of Use Modeling ........................................................ 156
Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs vi
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 7 of 212
8.1.4.4 Development of Annualized HOU ....................................... 156
8.1.4.5 Hierarchical Model .............................................................. 159
8.1.4.6 Coincidence Factor Modeling .............................................. 159
8.2 Lighting Inventory Findings .................................................................. 160
8.2.1 CFL & LED Saturation by Room Type ........................................... 161
8.2.2 CFL & LED Saturation by Socket and Circuit Type ........................ 162
8.2.3 CFL & LED Saturation by Housing Type and Ownership Status ... 163
8.2.4 CFL & LED Saturation by Region .................................................. 164
8. 2. 4. 1 Program Participation & Misc. Saturation Findings ............. 164
8.3 Lighting Hours of Use Findings ............................................................ 165
8.3.1 Aggregate Hours of Use ................................................................ 165
8.3.2 Hours of Use by Lamp Type .......................................................... 166
8.3.3 Hours of Use by Room Type .......................................................... 167
8.3.4 Peak Coincidence .......................................................................... 167
Appendix A Sampling and Estimation ............................................ A-1
Appendix B Lighting Interactive Factors ........................................ 8-1
Appendix C Billing Analysis Regression Outputs ......................... C-1
Appendix D Net to Gross Methodology and Findings ................... D-1
Appendix E Residential Lighting Logger Study Forms ................. E-1
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs vii
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 8 of 212
List of Figures
Figure 1-1: Idaho Electric Nonresidential Sector Program Gross Verified Saving Shares ............................. 3
Figure 1-2: Idaho Electric Residential Sector Program Gross Verified Saving Shares ................................... 4
Figure 2-1: Site Specific Program Process ................................................................................................... 12
Figure 4-1: Nonresidential Program Reported Energy Savings Shares ....................................................... 39
Figure 4-2: Prescriptive Lighting Reported Energy Savings Shares ............................................................. 40
Figure 4-3: EnergySmart Grocer Reported Energy Savings Shares ............................................................. 45
Figure 4-4: Prescriptive Non-Lighting Other Reported Energy Savings Shares .......................................... 51
Figure 4-5: Site Specific Reported Participation Energy Savings Shares ..................................................... 59
Figure 6-1: Residential Program Reported Energy Savings Shares ............................................................. 73
Participation in the 2014-2015 HVAC program totaled 599 measures. Table 6-7 and .............................. 76
Figure 6-2: 2014-2015 HVAC Program Reported Participation Energy Saving Shares ............................... 77
Figure 6-3: ASHP Distribution of Percent Savings ....................................................................................... 82
Figure 6-4: Variable Speed Motor Distribution of Percent Savings ............................................................ 83
Figure 6-5: 2014-2015 Water Heat Program Reported Participation Energy Saving Shares ..................... 88
Figure 6-6: 2014-2015 ENERGY STAR® Homes Program Reported Energy Saving Shares ......................... 92
Figure 6-7: 2014-2015 Fuel Efficiency Program Reported Energy Saving Shares ...................................... 96
Figure 6-8: Diagram of Fuel Switching Participation ................................................................................... 98
Figure 6-9: Fuel Efficiency Regression Analysis, Example Home .............................................................. 100
Figure 6-10 : Distribution of Lighting Energy Savings by Technology Type ............................................... 104
Figure 6-11: Estimates of Percentage of Products in Commercial Sector ................................................ 111
Figure 6-12: 2014-2015 Shell Program Reported Energy Saving Shares .................................................. 115
Figure 6-13: Participation and Cumulative Opt-outs by Month ............................................................... 120
Figure 6-14: Treatment and Control Energy Usage in the Pre-Period ...................................................... 122
Figure 6-15: Average Monthly Savings per Household with Relative Precision Bounds .......................... 127
Figure 6-16: Average Percent Savings and Control Daily Usage by Month .............................................. 128
Figure 6-17: Household Monthly Savings by Year .................................................................................... 129
Figure 6-18: Average Monthly Gas Savings per Household with Relative Precision Bounds ................... 131
Figure 6-19: 2014-2015 Low Income Program Reported Energy Saving Shares: Measure Category ....... 133
Figure 6-20: 2014-2015 Low-Income Program Reported Energy Saving Shares: Non-Lighting Conservation
.................................................................................................................................................................. 134
Figure 6-21: Distribution of Reported kWh Values by Home Type ........................................................... 135
Figure 6-22: Low-Income Program Impacts by Month ............................................................................. 137
Figure 8-1: Actual Customer Participation by Region ............................................................................... 149
Figure 8-2: Actual Participation by Dwelling Type .................................................................................... 150
Figure 8-3: Actual Participation by Household Income ............................................................................ 150
Figure 8-4: Actual Participation by Geographical Area ............................................................................. 151
Figure 8-5: Percent Deviation from Average Annual Daylight Hours ....................................................... 157
Figure 8-6: Lighting Inventory Summary of Room and Lamp Type .......................................................... 162
Figure 8-7: Aggregate Hours of Use Actual and Annualized Estimate ...................................................... 166
Figure A-1: Comparison of Mean-Per-Unit and Ratio Estimation ............................................................ A-2
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs viii
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 9 of 212
List of Tables
Table 1-1: Summary of Impact Evaluation Activities .................................................................................... 2
Table 1-2: 2014-2015 Idaho Electric Portfolio Evaluation Results ................................................................ 2
Table 1-3: Idaho Electric Nonresidential Program Evaluation Results .......................................................... 3
Table 1-4: Idaho Electric Residential Program Evaluation Results ............................................................... 4
Table 2-1: Site Specific Program Measures ................................................................................................. 11
Table 2-2: Prescriptive Lighting Program Measures ................................................................................... 13
Table 2-3: EnergySmart Program Measures ............................................................................................... 15
Table 2-4: Food Service Equipment Program Measures ............................................................................. 17
Table 2-5: Green Motor Rewinds Program Measures ................................................................................ 18
Table 2-6: Motor Controls HVAC Program Measures ................................................................................. 18
Table 2-7: Motor Controls HVAC Program Measures ................................................................................. 18
Table 2-8: Power Management for PC Networks Program Measures ........................................................ 19
Table 2-9: Commercial Windows & Insulation Measures ........................................................................... 19
Table 2-10: Commercial Water Heater Measures ...................................................................................... 20
Table 2-11: Fleet Heat Measures ................................................................................................................ 20
Table 2-12: Small Business Program Measure Overview ............................................................................ 21
Table 2-13: Residential Program Type and Description .............................................................................. 22
Table 2-14 Appliance Recycling Measures and Incentives ......................................................................... 23
Table 2-15 HVAC Measure Overview .......................................................................................................... 23
Table 2-16 Water Heat Program Measure Overview ................................................................................. 23
Table 2-17 ENERGY STAR0 Homes Measure Overview ............................................................................... 24
Table 2-18 Fuel Efficiency Measure Overview ............................................................................................ 24
Table 2-19 Shell Measure Overview ........................................................................................................... 25
Table 2-20 Low Income Approved Measure List (100% of costs offset by Avista) ..................................... 26
Table 2-21 Low Income Rebate List ............................................................................................................ 26
Table 2-22 Avista Nonresidential Reported Participation and Savings ...................................................... 27
Table 2-23 Avista Residential Reported Participation and Savings ............................................................ 27
Table 3-1: Planned Sampling and Evaluation Rigor for Washington/Idaho Electric Residential Programs 31
Table 3-2: Sampling and Evaluation Rigor for Washington/Idaho Electric Nonresidential Programs ........ 31
Table 3-3: Achieved Sampling and Confidence/Precision for Washington/Idaho Electric Residential
Programs ..................................................................................................................................................... 32
Table 3-4: Achieved Sampling and Evaluation Rigor for Washington/Idaho Electric Nonresidential
Programs ..................................................................................................................................................... 32
Table 3-5: Fixed Effects Regression Model Definition of Terms ................................................................. 37
Table 4-1: Nonresidential Program Reported Savings ................................................................................ 38
Table 4-2: Nonresidential Program Achieved Evaluation Sample .............................................................. 39
Table 4-3: Prescriptive Lighting Reported Energy Savings by Measure ...................................................... 40
Table 4-4: Prescriptive Lighting Achieved Sample ...................................................................................... 41
Table 4-5: Prescriptive Lighting Onsite Data Collection .............................................................................. 42
Table 4-6: Prescriptive Lighting Realization Rate Results ........................................................................... 44
Table 4-7: Baseline Fixture Types for Prescriptive Lighting (lnterior) ......................................................... 44
Table 4-8: Prescriptive Lighting Gross Verified Savings .............................................................................. 44
'-'"Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs ix
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 1 O of 212
Table 4-9: EnergySmart Grocer Reported Energy Savings by Measure ...................................................... 45
Table 4-10: EnergySmart Grocer Achieved Sample .................................................................................... 46
Table 4-11: EnergySmart Grocer Onsite Data Collection ............................................................................ 47
Table 4-12: EnergySmart Grocer Impact Energy Realization Rate Results ................................................. 48
Table 4-13: EnergySmart Grocer Gross Verified Savings ............................................................................ 49
Table 4-14: Prescriptive Non-Lighting Other Program Summaries ............................................................. 50
Table 4-15: Prescriptive Non-Lighting Other Reported Energy Savings by Measure ................................. 50
Table 4-16: Prescriptive Non-Lighting Other Achieved Sample .................................................................. 51
Table 4-17: Prescriptive Non-Lighting Other Achieved Sample by Program .............................................. 52
Table 4-18: Prescriptive Non-Lighting Other Onsite Data Collection ......................................................... 53
Table 4-19: Prescriptive Non-Lighting Other Realization Rate Results ....................................................... 56
Table 4-20: Cooling Season Savings for Window Replacements ................................................................ 57
Table 4-21: Prescriptive Non-Lighting Other Gross Verified Savings .......................................................... 58
Table 4-22: Site Specific Reported Energy Savings by Measure ................................................................. 59
Table 4-23: Site Specific Achieved Sample .................................................................................................. 60
Table 4-24: Site Specific Onsite Data Collection ......................................................................................... 61
Table 4-25: Site Specific Program Realization Rate Results ........................................................................ 64
Table 4-26: Site Specific Measure-Level Gross Verified Savings ................................................................. 65
Table 4-27: Baseline Fixture Types for Site Specific Interior Lighting ......................................................... 65
Table 4-28: Site Specific Gross Verified Savings .......................................................................................... 66
Table 4-29: Non residential Program Gross Impact Evaluation Results ...................................................... 66
Table 5-1: Small Business Program Impact Evaluation Achieved Sample .................................................. 68
Table 5-2: Small Business Program Onsite Data Collection ........................................................................ 69
Table 5-3: Small Business Program Realization Rate Summary .................................................................. 70
Table 5-4: Recommended Deemed Savings Values for Faucet Aerator Measures .................................... 71
Table 5-5: Recommended Deemed Savings Values for Pre-Rinse Spray Valve Measures ......................... 71
Table 6-1: Residential Program Reported Savings ...................................................................................... 72
Table 6-2: Residential Program Achieved Evaluation Sample .................................................................... 74
Table 6-3 Appliance Program Reported Participation and Savings ............................................................ 74
Table 6-4 Appliance Recycling Participation Counts ................................................................................... 75
Table 6-5 Appliance Recycling Reported and Evaluated Savings ................................................................ 75
Table 6-6 Appliance Recycling Gross Verified Savings ................................................................................ 76
Table 6-7: HVAC Program Reported Participation and Savings .................................................................. 77
Table 6-8: ASHP Fixed-Effects Regression Model Definition of Terms ....................................................... 79
Table 6-9: Variable-Speed Motor Fixed-Effects Regression Model Definition of Terms ............................ 80
Table 6-10: Air Source Heat Pump Impact Summary ................................................................................. 81
Table 6-11: Variable Speed Motor Impact Summary .................................................................................. 82
Table 6-12: Comparison of Smart Thermostat Evaluation Results ............................................................. 85
Table 6-13: HVAC Program Gross Verified Savings ..................................................................................... 87
Table 6-14: 2014-2015 Water Heat Reported Participation and Savings .................................................. 87
Table 6-15: Water Heat Program Achieved Sample ................................................................................... 88
Table 6-16: Low-Flow Showerhead Parameters and Data Sources ............................................................ 90
Table 6-17: Water Heat Program Gross Verified Savings ........................................................................... 91
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs X
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 11 of 212
Table 6-18: 2014-2015 ENERGY STAR® Homes Reported Participation and Savings ................................ 92
Table 6-19: Calculation of Consumption Absent Program Definition of Terms ......................................... 93
Table 6-20: ENERGY STAR Home: Results for Stick Built homes in Idaho ................................................... 94
Table 6-21: ENERGY STAR Home: Results for Furnaces in Manufactured Homes ...................................... 94
Table 6-22: ENERGY STAR® Homes Program Gross Verified Savings .......................................................... 95
Table 6-23: 2014-2015 Fuel Efficiency Reported Participation and Savings .............................................. 95
Table 6-24: Fuel Efficiency Electric Billing Analysis Summary Statistics ................................................... 101
Table 6-25: Regression Coefficients from Combined Furnace Conversion Model ................................... 101
Table 6-26: Fuel Efficiency Program Reported and Gross Verified Savings .............................................. 102
Table 6-27: 2014-2015 Residential Lighting Reported Participation and Savings ................................... 103
Table 6-28: Lighting Program Parameters and Sources ............................................................................ 105
Table 6-29: Verified Residential Lighting Unit Counts by Lamp Type and Delivery Stream ..................... 106
Table 6-30: Verified Hours of Use for Residential Lighting ....................................................................... 107
Table 6-31: Standard Lamp Baseline Wattage for Equivalences .............................................................. 108
Table 6-32: Decorative and Globe Lamp Baseline Wattage for Equivalences .......................................... 108
Table 6-33: In-Service Rate Trajectory for Markdown and Giveaway CFL based on UMP ....................... 110
Table 6-34: Nonresidential Lighting Input Parameter Assumptions ......................................................... 111
Table 6-35: Verified Residential Lighting Energy Savings by Lamp Type and Delivery Stream ................ 112
Table 6-36: Residential Lighting Realization Rates and Gross Verified Savings ........................................ 113
Table 6-37: 2014-2015 Shell Program Reported Participation and Savings ............................................ 114
Table 6-38: Shell Rebate Model Coefficients ............................................................................................ 116
Table 6-39: Shell Rebate Gross Verified Savings Summary ...................................................................... 117
Table 6-40: Precision of Findings .............................................................................................................. 117
Table 6-41: Shell Rebate Performance by Measure Category .................................................................. 117
Table 6-42: Shell Rebate Measure Average Annual Usage ....................................................................... 118
Table 6-43: Shell Program Gross Verified Savings .................................................................................... 119
Table 6-44: Difference in Means t-test Values ......................................................................................... 122
Table 6-45: Lagged Dependent Variable Model Definition ofTerms ....................................................... 123
Table 6-46: Opower Behavioral Program Impact Estimates with EE Adjustments .................................. 125
Table 6-47: Opower Program Incremental Annual MWh Savings ............................................................ 126
Table 6-48: Confidence Intervals Associated with Behavioral Program Impact Estimates ...................... 126
Table 6-49: Opower Program Gas Impact Estimates with EE Adjustments .............................................. 130
Table 6-50: Confidence Intervals Associated with Program Gas Impact Estimates ................................. 130
Table 6-51: 2014-2015 Low-Income Program Reported Participation and Savings ................................ 132
Table 6-52: Low Income Billing Analysis Findings ..................................................................................... 136
Table 6-53: Low-Income Lighting Conservation Measures Gross Verified Savings .................................. 138
Table 6-54: Low-Income Program Gross Verified Savings ........................................................................ 138
Table 6-55: Residential Program Gross Impact Evaluation Results .......................................................... 139
Table 7-1: 2014-2015 Idaho Electric Portfolio Evaluation Results ............................................................ 140
Table 7-2: Idaho Electric Nonresidential Program Evaluation Results ...................................................... 141
Table 7-3: Idaho Electric Residential Program Evaluation Results ........................................................... 141
Table 7-4: Opower Acquisition Cost Example ........................................................................................... 147
Table 8-1: Head of Household Age Participant Share ............................................................................... 151
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs xi
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 12 of 212
Table 8-2: Sample Frame of Logger Deployment by Room Type, by Bulb Type ....................................... 152
Table 8-3: Distribution of Loggers Installed by Room with Viable Data ................................................... 155
Table 8-4: Population Weights Applied to Sample Frame ........................................................................ 156
Table 8-5: Lighting Inventory Summary Saturation by Lamp Type ........................................................... 161
Table 8-6: Lighting Inventory Summary CFL Saturation by Room Type .................................................... 161
Table 8-7: Lighting Inventory CFL Saturation by Socket Type ................................................................... 162
Table 8-8: Lighting Inventory CFL Saturation by Circuit Type ................................................................... 163
Table 8-9: Lighting Inventory CFL Saturation by Building Type ................................................................ 163
Table 8-10: Lighting Inventory CFL Saturation by Ownership Type .......................................................... 163
Table 8-11: Lighting Inventory CFL Saturation by Region ......................................................................... 164
Table 8-12: Free CFL Program Participation Findings ............................................................................... 164
Table 8-13: Space Heating Equipment Saturation .................................................................................... 165
Table 8-14: Space Cooling Equipment Saturation .................................................................................... 165
Table 8-15: Space Heating Fuel Share ....................................................................................................... 165
Table 8-16: Aggregate Lighting Socket Hours of Use ................................................................................ 166
Table 8-17: Hours of Use by Lamp Type ................................................................................................... 167
Table 8-18: Hours of Use by Room Type ................................................................................................... 167
Table 8-19: Hours of Use by Room Usage Type ........................................................................................ 167
Table 8-20: Lighting Coincident Factor by Peak Period ............................................................................ 169
Table 8-21: Coincident Factor by Peak Period by Lamp Type ................................................................... 169
Table 8-22: Coincident Factor by Peak Period by Room Type .................................................................. 170
Table A-1: Case Weights Example ............................................................................................................ A-3
Table A-2: Relative Precision Example ..................................................................................................... A-6
Table B-1: Lighting Interactive Factors by Building Type and HVAC System Type .................................... B-1
Table B-2: Lighting Interactive Factors by Building Type and HVAC System Type Cont ........................... B-2
Table C-1: ASHP Fixed-Effects Regression Output .................................................................................... C-1
Table C-2: Variable Speed Fan Motor Fixed-Effects Regression Output ................................................... C-2
Table C-3: Low Income Fuel Switching ...................................................................................................... C-3
Table C-4: Low Income Electric Conservation ........................................................................................... C-4
Table C-5: Shell Rebate Measures ............................................................................................................. C-5
Table C-6: Electric to Gas Furnace Conversion ......................................................................................... C-6
Table C-7: Electric to Gas Water Heater Conversion ................................................................................ C-7
Table C-8: Electric to Gas Furnace and Water Heater Conversion ........................................................... C-8
Table D-1: Free Ridership Change Values ................................................................................................ D-2
Table D-2: Free Ridership Influence Values ............................................................................................. D-3
Table D-3: Appliance Recycling Modified FR Values ................................................................................ D-4
Table D-4: Participant Spillover Program Influence Values ..................................................................... D-5
Table D-5: Example Market Baseline 60-watt Equivalent Lamp .............................................................. D-6
Table D-6: Residential Lighting Net to Gross Ratios and Net Verified Impacts ....................................... D-6
Table D-7: Nonresidential Program Net To Gross Ratios ......................................................................... D-7
Table D-8: Residential Program Net To Gross Ratios ............................................................................... D-7
t.-"1 Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs xii
Exhibit No. 2
L. Roy. Avista
Schedule 1, Page 13 of 212
Equations
Equation 3-1: Regression Model Specification for Electric Measures ........................................................ 36
Equation 3-2: Regression Model Specification for Gas Measures .............................................................. 36
Equation 4-1: Prescriptive Lighting Energy Savings Calculation ................................................................. 42
Equation 4-2: Prescriptive Lighting Base Case Demand Savings Calculation .............................................. 43
Equation 4-3: Prescriptive Retrofit Case Demand Savings Calculation ...................................................... 43
Equation 4-4: HVAC Motor Controls Energy Savings Calculation ............................................................... 55
Equation 4-5: Commercial Windows and Insulation Cooling Savings Calculation ...................................... 55
Equation 4-6: Commercial Windows and Insulation Heating Savings Calculation ..................................... 56
Equation 4-7: VFD Energy Savings Calculation ............................................................................................ 63
Equation 4-8: HVAC Replacement Energy Savings Calculation ................................................................... 64
Equation 5-1: Small Business Program Energy Savings Calculation ............................................................ 69
Equation 6-1: ASHP Fixed-Effects Panel Regression Model Specification .................................................. 78
Equation 6-2: Variable Speed Motor Fixed-Effects Regression Model Specification ................................. 79
Equation 6-3: Low Flow Showerhead Energy Savings Calculation .............................................................. 89
Equation 6-4: Calculation of Consumption Absent Program ...................................................................... 93
Equation 6-5: Calculation of Consumption Absent Program .................................................................... 104
Equation 6-6: Fixed-Effects Panel Regression Model Specification .......................................................... 115
Equation 6-7: Lagged Dependent Variable Model Specification .............................................................. 123
Equation 8-1: Sinusoidal Model Specification ......................................................................................... 158
Equation 8-2: Hierarchical Linear Model for HOU .................................................................................... 159
Equation 8-3: Hierarchical Linear Model for HOU .................................................................................... 160
Equation A-1: Coefficient of Variation ..................................................................................................... A-2
Equation A-2: Coefficient of Variation ..................................................................................................... A-4
Equation A-3: Error Ratio ......................................................................................................................... A-4
Equation A-4: Required Sample Size ........................................................................................................ A-4
Equation A-5: Finite Population Correction Factor .................................................................................. A-5
Equation A-6: Application of the Finite Population Correction Factor .................................................... A-5
Equation A-7: Error Bound of the Savings Estimate ................................................................................ A-5
Equation A-8: Relative Precision of the Savings Estimate ........................................................................ A-6
Equation A-9: Combining Error Bounds across Strata ............................................................................. A-6
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs xiii
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 14 of212
1 Executive Summary
Nexant Inc. and Research Into Action (collectively the evaluation team) conducted an impact
and process evaluation of Avista's 2014 and 2015 residential and nonresidential energy
efficiency programs. This report documents findings from the impact evaluation activities for
Avista's Idaho electric programs. The primary goal of this evaluation was to provide an accurate
summary of the gross energy and demand savings attributable to the following Avista programs
offered in 2014 and/or 2015:
• Nonresidential Prescriptive
• Nonresidential Site Specific
• Residential Appliance Recycling
• Residential Heating, Ventilation and Air Conditioning (HVAC)
• Residential Water Heat
• Residential ENERGY STAR® Homes
• Residential Fuel Efficiency
• Residential Lighting
• Residential Shell
• Residential Opower Behavioral
• Low Income
1.1 Evaluation Methodology and Activities
The evaluation team performed the impact evaluation through a combination of document
audits, customer surveys, engineering analysis and onsite measurement and verification (M&V)
of completed program projects. Because it is not cost-effective to complete analysis and onsite
inspection on a census of the implemented projects, the evaluation team verified energy savings
for a representative sample of projects to draw statistically-measurable results. The gross
verified program savings were adjusted by a realization rate (RR), which is the ratio of
evaluation verified savings to the program-reported savings within the sample.
The evaluation team conducted more than 525 document audits, approximately 360 customer
surveys, and nearly 250 onsite inspections across the residential and nonresidential programs
being evaluated (Table 1-1). In addition, the evaluation team conducted billing regression
analysis to estimate the impacts of five residential programs and on a case-by-case basis for
the nonresidential projects. The samples were designed to meet a 90% confidence and 10%
precision level at the portfolio and sector level and were based upon the expected and actual
significance (or magnitude) of program participation, the level of certainty of savings, and the
variety of measures.
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 1
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 15 of 212
EXECUTIVE SUMMARY
Table 1-1: Summary of Impact Evaluation Activities
Document . Billing Program A d" Surveys Ons1te M&V A 1 . u 1t na ys1s
Residential
Residential Appliance Recycling 70 72 0
HVAC Program 68 68 0
Water Heat Program 24 13 0
ENERGY STAR Homes 19 16 0
Fuel Efficiency 26 25 0
Residential Lighting Program 0 0 75
Shell Program 28 28 0 ..J
Opower Behavioral Program 0 0 0 ..J
Low Income 24 i 0 0 ..J I I
Nonresidential
Prescriptive Lighting 68 22 22
Prescriptive EnergySmart Grocer 44 20 20
Prescriptive Non-Lighting Other 24 15 15
Site Specific 101 84 84 as applicable
Small Business* 31 31
TOTAL 527 363 247
i
*There was no participation in the Small Business program in Idaho in 2015 and the evaluation activities
were conducted on Washington participants.
1.2 Summary of Impact Evaluation Results
Avista's Idaho electric 2014 and 2015 programs achieved more than 80 GWh of savings over
the two year period (Table 1-2). Table 1-3 and Table 1-4 summarize Avista's 2014 and 2015
impact evaluation results by sector and program.
Table 1-2: 2014-2015 Idaho Electric Portfolio Evaluation Results
S Reported Realization Gross Verified ector . . Savings (kWh) Rate(%) Savings (kWh)
Residential 18,772,837 97% 18,281 ,513
Nonresidential 12,379,360 94% 11 ,687,224
Low Income 758,955 147% 1,112,301
PORTFOLIO 31,911,152 97% 31,081,038
L-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 2
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 16 of212
EXECUTIVE SUMMARY
Table 1-3: Idaho Electric Nonresidential Program Evaluation Results
P 2014-2015 Reported R 1. . R 2014-2015 Verified Gross rogram . ea 1zat1on ate Savings (kWh) Savings (kWh)
EnergySmart Grocer 2,387,662 90% 2,138,035
Food Service Equipment 130,946 54% 70,971
Green Motors 43,954 54% 23,823
Motor Controls HVAC 466,340 54% 252,751
Commercial Water Heaters 190 54% 103
Prescriptive Lighting 3,475,049 99% 3,432,865
Prescriptive Shell 54,381 54% 29,474
Fleet Heat 7,228 54% 3,917
Site Specific 5,813,610 99% 5,735,284
TOTAL NONRESIDENTIAL 12,379,360 94% 11,687,224
Figure 1-1: Idaho Electric Nonresidential Sector Program Gross Verified Saving Shares
49%
t-1Nexanr
1%
2%
0%
• EnergySmart Grocer
• Food Service Equipment
• Green Motors
• Motor Controls HVAC
Commercial Water Heaters
• Prescriptive Lighting
• Prescriptive Shell
Fleet Heat
• Site Specific
Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 3
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 17 of 212
EXECUTIVE SUMMARY
Table 1-4: Idaho Electric Residential Program Evaluation Results
2014-2015
Program Adjusted_ Realization Rate 201~~2015 G~oss Reported Savings Ver1f1ed Savings
(kWh)
Appliance Recycling 250,920 166% 416,524
HVAC 872,828 60% 521 ,365
Water Heat 239,267 148% 354,675
ENERGY STAR Homes 140,538 123% 173,120
Fuel Efficiency 5,295,779 60% 3,198,893
Lighting 8,323,842 126% 10,457,288
Shell 903,663 38% 345,048
Opower 2,746,000 102% 2,814,601
Low Income 758,955 147% 1,112,301
TOTAL RESIDENTIAL 19,531,792 99% 19,393,814
Figure 1-2: Idaho Electric Residential Sector Program Gross Verified Saving Shares
2% 3%
16%
2%
54%
• Appliance Recycling
•HVAC
• Water Heat
• ENERGY STAR Homes
• Fuel Efficiency
• Lighting
•Shell
•Opower
• Low Income
1.3 Conclusions and Recommendations
The following outlines the key conclusions and recommendations as a result of the evaluation
activities. Specific details regarding the conclusions and recommendations outlined here, along
'-"'Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 4
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 18 of 212
EXECUTIVE SUMMARY
with additional conclusions and recommendations can be found in the program-specific sections
of this report and in Section 7.
1.3.1 Nonresidential Programs
The overall realization rate for the nonresidential portfolio is 94%. The realization rates ranged
from 99% for the Site Specific and Prescriptive Lighting programs down to 54% for the
"Prescriptive Non-Lighting Other" program. The Site Specific and Prescriptive Lighting programs
are the largest programs in the portfolio, together representing 75% of the portfolio's gross
verified savings. The evaluation team found that the processes Avista is utilizing for estimating
and reporting energy savings for the nonresidential programs are predominantly sound and
reasonable. The following subsections outline specific key conclusions and recommendations
for several of the nonresidential programs.
Conclusion: The Site Specific program constitutes almost 50% of the program energy shares.
Within the last 2 years, Avista has increased their level of quality assurance and review on
projects that participate through the program. The evaluation team's analysis resulted in a 99%
realization rate for the Site Specific program. The high realization rate indicates that Avista's
internal process for project review, savings estimation, and installation verification are working
to produce high quality estimates of project impacts.
Recommendation: The evaluation team recommends that Avista continue to operate
this program with the current level of rigor. For interior lighting projects, Avista should
consider applying the interactive factors deemed by the Regional Technical Forum
(RTF) to quantify the interactive effects between lighting retrofits and their associated
HVAC systems.
Conclusion: Avista's EnergySmart Grocer program is successfully providing retail and
restaurant customers with an avenue to upgrade their refrigeration equipment. Participation in
the program includes both prescriptive and custom projects. The evaluation team's review of
projects in the program resulted in a realization rate of 90%. For prescriptive projects, the
evaluation team determined that RTF deemed savings values were being appropriately applied
in most cases. However, low project-level realization rates for custom projects, which tend to be
larger in size than prescriptive projects, are driving the program realization rate downward.
Recommendation: Avista should consider more internal review of energy savings
estimates submitted by vendors for custom projects under this program. Alternatively,
Avista could consider tracking custom projects under the Site Specific program with
other projects of similar size and complexity.
Conclusion: Avista reported 2014-2015 participation in six other prescriptive programs. Of
these, the HVAC Motor Controls program is the largest, constituting 66% of the energy savings
for this group. The evaluation team's review of projects in these programs resulted in a 54%
realization rate. Cases of ineligible VFD projects receiving incentives were cause of the low
realization rate for these programs.
""Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 5
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 19 of 212
EXECUTIVE SUMMARY
Recommendation: Avista should revise the HVAC Motor Controls program to include
more verification of motor eligibility status. More emphasis should be placed on
confirming motor application and duty status to ensure compliance with the program's
existing eligibility requirements. More specifically, Avista should place specific emphasis
on ensuring VFDs are installed in a manner that saves energy (i.e. not just as "soft
starters") and that incentivized VFDs serve primary-duty motors.
Conclusion: The Small Business reported savings for faucet aerators were found to be
conservatively low based upon the evaluation team's secondary research. The realization rates
for faucet aerators were 126% for electric savings and 204% for natural gas savings.
Recommendation: It is recommended that the modified deemed savings values utilized
by the evaluation team be adopted by the program for future reporting purposes.
1.3.2 Residential Programs
The overall realization rate for the residential and low income portfolio is 99%. The realization
rates varied significantly across the various programs evaluated with the Shell, HVAC, and Fuel
Efficiency programs having the lowest realization rate (38% and 60% respectively). The
evaluation team found that the reported savings for the majority of the programs were
understating the actual impacts found from the evaluation activities. The following subsections
outline specific conclusions and recommendations for several of the residential programs.
Conclusion: The evaluation team found that the reported deemed savings value (per recycled
unit) for the program was lower than estimated gross savings valued from prior studies. Avista
may have aligned their deemed savings values close to the RTF deemed savings values, but it
is important to understand that the RTF is reporting a value that accounts for net market effects
(i.e. free ridership).
Recommendation: If Avista chooses to offer an appliance recycling program in the
future, it is recommended that a clear distinction between gross and net savings values
is noted if Avista reports the most current RTF values.
Conclusion: The evaluation team found, through billing regression analysis, a relatively low
realization rate for the Air Source Heat Pump (ASHP) measures (RR of 49%).
Recommendation: The evaluation team recommends Avista reexamine the
assumptions relating to annual per-home consumption and savings estimates in homes
receiving ASHP installations. In addition, to help better understand the baseline for the
ASHP replacement, Avista could consider requesting that contractors and customers
provide a better description of the replaced unit
Conclusion: For showerheads distributed through the Simple Steps program, Avista allocates
50% of its reported savings to electric savings and 50% to natural gas savings to account for
homes that have different water heating fuel types.
t.-1N&anr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 6
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 20 of 212
EXECUTIVE SUMMARY
Recommendation: The evaluation team recommends Avista update this allocation
assumption to be based on representative water heater fuel type saturation. These data
are available through the Regional Building Stock Assessment study; however, we
recommend Avista base the allocation on data specific to its territory.
Conclusion: The evaluation team conducted a billing regression analysis for the Fuel Efficiency
participants and found realization rates of 57-62% for rebate projects that included the
conversion of a home's heating system from electricity to natural gas. When regression
coefficients were examined in detail, the evaluation team noted that the estimated reduction in
electric heating load was being offset by an increase in estimated base load within participating
homes.
Recommendation: Because the rebate amounts and per-home savings from Fuel
Efficiency are so large and the number of participants is relatively low, the evaluation
team recommends Avista ask participating customers for details on any additional home
renovations that were completed in parallel with the fuel conversion. Home improvement
projects such as an addition, finishing a basement, or adding air conditioning can
drastically change the consumption patterns within a home and render the assumed
baseline inaccurate.
Conclusion: The evaluation team found that over half the homes receiving Fuel Efficiency
rebates in 2014-2015 did not have a gas billing history with Avista prior to the conversion. These
homes realized savings at a higher rate than homes that did have previous gas service.
Recommendation: The evaluation team recommends that Avista consider adding a
field to the program tracking database that indicates the gas meter installation date or
service start date of participating homes. This would more clearly delineate homes that
were previously all electric and became dual-fuel around the same time as the Fuel
Efficiency project, from homes that had been dual-fuel historically. Avista may also want
to consider assuming a more conservative electric savings estimate for homes that had
prior gas service because it's possible that the home was not 100% electrically heated
prior to program participation.
Conclusion: Avista's deemed savings estimates, which were generally the same for all similar
product types and not correlated to the bulb wattage, understated the savings found by the
evaluation team. This was especially the case for Avista's CFL giveaway program.
Recommendation: The evaluation team recommends that Avista consider more
detailed product type deemed values in an effort to be more closely aligned with the
actual participating lamps. Simple Steps has shifted its program tracking to specific
product types by lumen bins in accordance with the most current SPA UES measure list.
Avista should consider using these higher resolution deemed value for internal reporting
with the Simple Steps program and for use with internal residential lighting programs.
'-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 7
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 21 of212
EXECUTIVE SUMMARY
Recommendation: An overarching recommendation related to the Residential Lighting, is that
Avista monitor the LED lamp market for technology cost changes and customer preferences,
and consider increasing LED lamp options from the 2014-2015 portfolio in future DSM planning.
Currently, LED prices are dramatically decreasing and customer preferences are shifting from
CFL to LEDs as a preferred choice as an energy efficient technology. Consequently, CFLs shelf
space share is declining as an abandoned technology, despite its better cost effectiveness
compared to LED lamps.
Conclusion: The evaluation team found a low realization rate (38%) for shell rebate measures
(windows and insulation). This finding indicates that reported savings values were too
aggressive on average. The evaluation team compared the end-use shares estimated via
regression analysis and found that only approximately 5,500 of the 13,000 kWh of average
annual consumption in residential homes in Avista's service territory was assigned to heating
and cooling load. Given this end-use share, the reported savings values claimed by Avista
equate to a 25% reduction in HVAC loads.
Recommendation: The evaluation team recommends Avista examine planning
assumptions about per-home consumption, end-use load shares, and percent reductions
in heating and cooling loads from shell improvements. It may be that the percent
reduction assumptions are sound, but they are being applied to an overstated
assumption of the average electric HVAC consumption per home. Conversely, the
assumed end-use shares may be accurate, but the end-use reduction percentage is
inflated. This investigation should be conducted separately for electrically heated homes
and dual fuel homes as the heating electric end-use share will be different.
Conclusion: The evaluation team found that savings held fairly consistent during the 6 month
interruption in Home Energy Report delivery. The finding reinforces Avista's decision to assume
a multi-year measure life when calculating the cost-effectiveness of the Opower program.
Recommendation: The evaluation team recommends Avista examine the program delivery
model in the 2016-2017 cycle. Given the fixed and volumetric nature of program costs,
measure life assumptions, and mechanisms by which measured savings are counted toward
goal achievement the evaluation team believes there are alternatives to the traditional
delivery model that optimize program achievements relative to costs.
Conclusion: The evaluation team found a high realization rate for the fuel conversion measures
implemented through the Low Income program. One reason for the high realization rate could
be due to the fact that Avista caps the reported savings value to 20% of the contractor
estimated savings. In addition, the evaluation team found that the verified savings for these fuel
conversion measures aligned closely with the verified savings found through the regular-income
Fuel Conversion program .
..,...,Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 8
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 22 of 212
EXECUTIVE SUMMARY
Recommendation: The evaluation team recommends re-evaluating the current savings cap
for fuel conversion projects. In addition, we recommend that Avista align assumptions for
fuel switching savings for the Low Income and Fuel Efficiency programs.
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 9
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 23 of 212
2 Introduction
2.1 Purpose of Evaluation
The purpose of the impact evaluation was to verify the savings attributed to Avista's 2014-2015
rebate programs and to identify areas for future program opportunities. The evaluation team
estimated gross program energy impacts through a combination of documentation audits, and
telephone surveys, as well as engineering analysis and site inspections of completed program
projects.
2.2 Program Summary
The following section provides a description of each program we evaluated in Idaho. Although
the program descriptions outline electric and gas measures, as applicable, the remainder of this
report provides the methodology and findings for the electric-only measures and programs.
2.2.1 Nonresidential
The nonresidential energy efficiency market is delivered through a combination of prescriptive
and site-specific offerings. Any measure not offered through a prescriptive program is
automatically eligible for treatment through the site-specific program, subject to the criteria for
participation in that program. Prescriptive paths for the nonresidential market are preferred for
measures that are relatively small and uniform in their energy efficiency characteristics. The
following subsections provide a summary of Avista's Site Specific and Prescriptive programs,
including a description of program offerings, measures, and incentive amounts.
2.2.1.1 Site Specific
Avista's Site Specific program offers nonresidential customers the opportunity to propose any
energy efficiency project outside the realm of Avista's other programs. Any project with
documentable energy savings (kilowatt-hours and/or therms) and a minimum ten year measure
life can be submitted for a technical review and potential incentive through the Site Specific
program. The majority of projects that participate in this program are appliance upgrades,
compressed air, HVAC, industrial process, motors, shell improvements, custom lighting, and
natural gas multifamily market transformation projects. Multi-family residential developments
may also be treated through the Site Specific program when the majority of the units and
common areas are receiving the efficiency improvement. The determination of incentive
eligibility is based upon the project's individual characteristics as they apply to the Company's
electric Schedule 90 or natural gas Schedule 190 tariffs.
Customers or their representative are required to contact Avista for a Site Specific analysis prior
to any equipment being purchased or installed. Based on the post-verification process,
incentives may not be offered after the installation of energy efficiency equipment or process
under this program design. Table 2-1 shows the incentive levels associated with designated
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 10
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 24 of 212
2 INTRODUCTION
ranges of project simple payback periods. To be eligible for incentive, lighting measures must
have a simple payback period less than 8 years and all other measures must have a simple
payback period less than 13 years. Simple payback is calculated as the incremental cost of a
measure divided by the annual energy savings of the measure, calculated using the customer's
Avista electric and/or gas rate. Incremental costs are only those projects costs necessary for the
energy efficiency improvement.
Table 2-1: Site Specific Program Measures
Category Required Payback Period Incentive Level ($/Saved kWh)
Between 1 and 2 years $0.08
All Measures Between 2 and 4 years $0.12
Between 4 and 6 years $0.16
Between 6 and 8 years $0.20
Most Lighting Measures1
Greater than 8 years Not eligible
Between 6 and 13 years $0.20
All Other Measures
Greater than 13 years Not eligible
1Lighting measures with independently verified lives of less than 40,000 hours.
Avista internally implements the Site Specific program following a multi-stage internal
process outlined in Figure 2-1. To be considered for incentives, Avista must receive
notification of a potential project during the planning stage. Avista engineers generate energy
analyses and savings estimates for each project.
These energy savings estimates are subjected to a rigorous internal review process, with the
level of review dependent on the potential incentive level for the project. Avista's current
internal review guidelines are as follows:
• Measures that have an incentive of $0 and an energy based simple payback of over 20
years require no report and no review, just a form letter to the customer.
• Measures that have incentives between $1 and $2,000 will be processed by the
reporting engineer without any other review.
• Measures that have incentives between $2001 and $25,000 will be reviewed before
going to the customer by another qualified engineer.
• Measures over $25,000 will be reviewed by another qualified engineer with an additional
technical management review prior to releasing to the customer.
• Measures over $40,000 will be reviewed by another qualified engineer, a technical
manager, and an additional director review prior to releasing to the customer.
t..1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 11
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 25 of 212
2 INTRODUCTION
Avista employs the use of a "Technical Review Top Sheet" at each stage of the review
process. The Top Sheet is a checklist intended to ensure that all program processes and
policies have been followed and that project documentation is complete.
An "Energy Efficiency Evaluation Report" is generated for each project that includes a
summary of the project's scope of work, estimated energy savings and incentives. Following
project installation, Avista program staff members perform installation verification on nearly
100% of projects with limited exceptions. Program staff follows an "Incentive Payment Top
Sheet" prior to incentive payment, which is another checklist to ensure that the project has
been appropriately documented, tracked , and finalized.
Figure 2-1: Site Specific Program Process 1
2.2.1.2 Prescriptive Lighting
The Prescriptive Lighting program is designed to make lighting improvement projects more
accessible for Avista's nonresidential customers. This program is implemented internally by
Avista, and existing commercial or industrial facilities with electric service provided by Avista
with rate schedules 11 or above are eligible to participate. The program provides a pre­
determined incentive amount for many common lighting retrofits, as shown in Table 2-2.
Installed LED lighting must comply with nationally recognized specifications set forth by
ENERGY STAR and Design Lights Consortium (DLC) and the Seattle Lighting Design Lab.
1 Washington Demand Side Management Standard Operation Procedures. Avista Utilities. 2015.
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 12
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 26 of 212
2 INTRODUCTION
Avista's regionally-based Account Executives (AEs) are a key part of delivering the Prescriptive
Lighting program along with area vendors and contractors.
Table 2-2: Prescriptive Lighting Program Measures
$
Measure Incentive/
250 watt HID Fixture to 4-Lamp High Performance (HP) T8 Fixture HO or 2-Lamp T5HO Fixture
250 watt HID Fixture to 4-Lamp HP T8 Fixture HO or 2-Lamp T5HO 5-foot Fixture with
occupancy sensor
400 watt HID Fixture to 4-Lamp T5 Fixture
400 watt HID Fixture to 4-Lamp T5 Fixture with oc sensor
400 watt HID Fixture to 6-Lamp HP T8 Fixture
400 watt HID Fixture to 6-Lamp HP T8 with oc sensor
400 watt HID Fixture to 8-Lamp HP T8 Fixture
(4-Foot Lamps)
400 watt HID Fixture to 8-Lamp HP T8 Fixture
(4-Foot Lamps) with oc sensor
40 watt Incandescent to 6-10 watt LED*
60 watt Incandescent to 9-13 watt LED*
75-100 watt Incandescent to 12-20 watt LED*
Over 150 watt Incandescent to 2L HP F32T8 Fixture
20 watt MR16 (GU10 Base) to MR16 LED* 2-4 watt
35 watt MR16 (GU10 Base) to MR16 LED* 4-6 watt
50 watt MR16 (GU10 Base) to MR16 LED* 6-9 watt
75-100 watt Incandescent to LED* Can Light Kit
Fixture with no occupancy sensor to built in to with relays for room control (no switch sensors)
4-Foot 4-Lamp T12/8 to 4-Foot 3-Lamp HP T8 Ballast with 25 or 28 watt Lamps
4-Foot 4-Lamp T12/8 to 4-Foot 2-Lamp HP T8 Ballast with 25 or 28 watt Lamps
4-Foot 3-Lamp T12/8 to 2X4 LED* Fixture
4-Foot 3-Lamp T12/8 to 4-Foot 2-Lamp HP T8 Ballast with 25-28 watt Lamps
4-Foot 2-Lamp T12/8 to 4-Foot 1-Lamp HP T8 Ballast with 25-28 watt Lamps
4-Foot 1-Lamp T12/8 to 1-Lamp HP T8 Ballast with 25-28 watt Lamps
8-Foot 4-Lamp T12/8 to 8-Foot 4-Lamp (8') or 8-Lamp (4') HP T8 Ballast with 25 or 28 watt
Lamps
8-Foot 2-Lamp T12/8 to LED* 2X4 Fixture
8-Foot 1-Lamp T12/8 to LED* 1X4 Fixture
T12 Sign Lighting to LED Retrofit
Exterior-1000 watt HID to 400-575 watt DHID
Exterior-400 watt HID to 250 watt DHD MH
Exterior-400 watt HID to 122-175 watt LED*
Unit
$ 90
$120
$120
$150
$120
$150
$125
$155
$10
$12
$15
$40
$10
$11
$12
$30
$30
$32
$35
$60
$15
$13
$13
$54
$80
$40
$17 / FT2
$225
$150
$255
t.-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 13
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 27 of212
2 INTRODUCTION
$
Measure Incentive/
Unit
Exterior-320 watt to 122-160 watt LED* $180
Exterior-250 watt HID to 85-140 watt LED* & 250 watt HID to New Construction 85-121 watt $145
LED*
Exterior-175 watt HID to 35-85 watt LED* & 175 watt HID to New Construction 35-85 watt LED* $135
Exterior-150 watt HID to 35-50 watt LED* $130
Exterior-90-100 watt HID to 25-50 watt LED* $75
Exterior-70-90 watt HID to 15-35 watt LED $55
Exterior-320 & 400 watt HID to New Construction 122-175 watt LED* $180
Exterior-400 watt Canopy HID to 122-175 watt LED* Canopy Fixture $325
Exterior-325 watt Canopy HID to 122-160 watt LED* Canopy Fixture $250
Exterior-250 watt Canopy HID to 85-140 watt LED* Canopy Fixture $155
2.2.1.3 EnergySmart Grocer
The EnergySmart Grocer program offers a range of proven energy-saving solutions for grocery
stores and other customers with commercial refrigeration. The program was designed to offer
personalized facility assessments to identify efficiency opportunities and incentives to offset the
upfront costs of efficiency projects, making it easy and affordable for participating businesses to
achieve significant savings on their utility bills. EnergySmart Grocer is administered by
CLEAResult with Avista oversight.
The EnergySmart Grocer program is available to electric (Schedule 11, 12, 21 , 25) or natural
gas (Schedule 101, 111, 121) customers. The list of measures incentivized by this program is
fluid and may change at any point in the year. Table 2-3 lists the measures offered at one point
in 2015.
t..1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 14
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 28 of 212
2 INTRODUCTION
Table 2-3: EnergySmart Program Measures
M Incentive easure
$/unit Units
Cases
Low Temp Open Case to Reach-in Case
Medium Temp Open Case to Reach-in Case
Low Temp Reach-in to High Efficiency Reach-in Case
Low Temp Coffin to High Efficiency Reach-in
Medium Temp Open Case to High Efficiency Open Case
Special Doors with Low/No ASH for Low Temperature Reach-in
Add doors to Open Medium Case
Case Lighting
Reach-in Case Light: T12 to Low Power LED, Retrofit
Reach-in Case Light: T8 to Low Power LED, Retrofit
Reach-in Case Light: T8 to Low Power LED, New Case
Reach-in Case Light: Add Motion Sensor to Low Power LED
Reach-in Case Light: Add Motion Sensor to High Power LED
Controls
Anti-Sweat Heat -with Energy Management System
Anti-Sweat Heat -without Energy Management System -Med
Temp
Anti-Sweat Heat -without Energy Management System -Low
Temp
Evaporated Fan -Walk-In ECM Controller -Low Temp -1/10-1/20
HP
Evaporated Fan -Walk-In ECM Controller -Medium Temp -1/10-
1/20 HP
$150
$20
$150
$55
$20
$200
$85
$21
$12
$12
$1.00
$2.00
$14
$40
$40
$35
$35
Strip Curtains, Gaskets & Auto-Closers
Strip Curtains for Supermarket Walk-in Cooler $5
Strip Curtains for Supermarket Walk-in Freezer $5
Strip Curtains for Convenience Store Walk-in Freezer $5
Strip Curtains for Restaurant Walk-in Freezer $5
Gaskets for Walk-in Cooler -Main $25
Gaskets for Walk-in Freezer -Main Door $65
Gaskets for Reach-in Glass Doors, Medium Temp $ 25
Gaskets for Reach-in Glass Doors, Low Temp $ 40
Auto-Closers for Walk-in Freezers $170
Auto-Closers for Walk-in Coolers $25
Auto-Closers for Glass Reach-in Doors -Freezers $35
In ft of case
In ft of case
In ft of case
In ft of case
In ft of case
door
In ft of case
In ft of LED
In ft of LED
In ft of LED
In ft of LED
In ft of LED
In ft of case
In ft of case
In ft of case
Motor controlled
Motor controlled
sq ft
sq ft
sq ft
sq ft
door
door
door
door
Closer
Closer
Closer
t.-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 15
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 29 of 212
2 INTRODUCTION
M Incentive easure
$/unit Units
Auto-Closers for Glass Reach-in Doors -Coolers $35 Closer
Motors
Evaporator Motors -Shaded Pole to ECM in Display cases $55 motor
Evaporator Motors -Shaded Pole To ECM in Walk-in s 23 watts $140 motor
Evaporator Motors -Shaded Pole To ECM in Walk-in > 23 watts $140 motor
Floating Head Pressure on Singles, LT Condensing Unit $100 hp
Floating Head Pressure on Singles, MT Condensing Unit $100 hp
Floating Head Pressure on Singles, LT Remote Condenser $100 hp
Floating Head Pressure on Singles, MT Remote Condenser $100 hp
2.2.1.4 Food Service Equipment
The Food Service Equipment Program provides incentives for the purchase and installation of
energy efficient commercial food service equipment to Avista's electric (Schedule 11 , 12, 21 ,
25) and natural gas (Schedule 101, 111 , 121) customers. Equipment must be commercial grade
and must meet Energy Star or Fish nick specifications. Certified equipment is 10-70% more
efficient than standard equipment, depending on product type. Types of rebated equipment
include fryers, steam cookers, hot food holding cabinets, commercial convection ovens, dish
washers, commercial ice machines, pre-rinse sprayers, and commercial rack ovens. Table 2-4
summarizes the incentives available under the Food Service Equipment program. Avista
implements this program in a prescriptive manner, and incentives are issued to the participating
customer after the measure is installed.
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 16
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 30 of 212
2 INTRODUCTION
Table 2-4: Food Service Equipment Program Measures
Equipment Incentive
Commercial Convection Ovens
Commercial Convection Oven, Natural Gas $700/ Each
Commercial Convection Oven, Electric $225/ Each
Commercial Combination Oven, Natural Gas $1,000/ Each
Commercial Combination Oven, Electric $1 ,000/ Each
Dish Washers
Commercial Low Temp Electric Hot Water $600/ Each
Commercial High Temp Electric Hot Water $650/ Each
Commercial Low Temp Natural Gas Hot Water $300/ Each
Commercial High Temp Natural Gas Hot Water $350/ Each
Commercial Ice Machines
Under 200 LBS/Day Capacity $40/Each
200-399 LBS/Day Capacity $60/Each
400-599 LBS/Day Capacity $80/Each
600-799 LBS/Day Capacity $100/Each
800-999 LBS/Day Capacity $120/Each
1000-1199 LBS/Day Capacity $140/Each
1200-1399 LBS/Day Capacity $160/Each
1400-1599 LBS/Day Capacity $180/Each
1600-> LBS/Day Capacity $200/Each
Pre Rinse Sprayers
1 to 1.00 GPM Electric $25
.61 to .80 GPM Electric $25
.81 to 1.00 GPM Natural Gas $25
.61 to .80 GPM Natural Gas $25
Commercial Rack Ovens
Commercial Rack Ovens, Natural Gas $235
2.2.1.5 Green Motor Rewind
The Green Motors Rewind program is implemented by the Green Motors Practice Group with
Avista oversight. This program is available to electric (Schedule 11, 12, 21, 25, 31) customers
who receive a green motor rewind at a participating service center. To participate, customers
must take an existing motor to a participating service center to have a green rewind done.
Customers receive an automatic rebate applied at the service center of $1 per hp based on the
size of the motor. Motors ranging from 15 to 5,000 hp are eligible to participate. Motor service
centers must meet specific criteria to be qualified for the program.
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 17
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 31 of 212
2 INTRODUCTION
Table 2-5: Green Motor Rewinds Program Measures
Measure Eligible Motor Size Rebate
Green Motor Rewind 15-5,000 hp $1 / hp
2.2.1.6 Commercial HVAC Variable Frequency Drive (VFD) Program
This program encourages customers to increase HVAC pump and fan system efficiency through
the installation of variable frequency drives (VFDs). Incentives are issued after measure
installation . To be eligible for an incentive, a VFD must be installed on commercial heating,
ventilation, and air conditioning equipment that is served by an Avista electric non-residential
rate schedule (Schedule 11 , 12, 21 , 25). New construction projects are not eligible to
participate. Additionally, only VFDs installed on primary pumps and fans are qualified.
Secondary or spare pumps and fans do not qualify. Incentives are paid on a per-horsepower
basis, depending on the application of the VFD, as shown in Table 2-6. Avista implements this
program in a prescriptive manner, and incentives are issued to the participating customer after
the measure is installed.
Table 2-6: Motor Controls HVAC Program Measures
Measure Incentive per HP
VFD Fans $80
VFD Cooling Pump Only $85
VFD Heat Pump only or Combined Heating & Cooling Pump $140
2.2.1.7 Commercial Clothes Washers
The Commercial Clothes Washer Program provides incentives to Avista's electric (Schedule 11 ,
12, 21 , 25) or natural gas (Schedule 101, 111, 121) customers for the purchase and installation
of an energy efficient commercial clothes washers. Clothes washers must be commercial grade
units and must meet ENERGY STAR™ commercial clothes washer specifications. To be
eligible for incentive, the clothes washer must be served by hot water that is generated using an
Avista fuel source (e.g. a natural gas hot water heater on Avista natural gas service). The types
of equipment eligible to participate in this program are listed in Table 2-7. Avista implements this
program in a prescriptive manner, and incentives are issued to the participating customer after
the measure is installed.
t-1Nexanr
Table 2-7: Motor Controls HVAC Program Measures
Equipment Rebate/ unit
ES Washer electric hot water and dryer $75
ES Washer electric hot water and natural gas dryer $75
ES Washer natural gas hot water and natural gas dryer $75
ES Washer -natural gas hot water and electric dryer $75
Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 18
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 32 of 212
2 INTRODUCTION
2.2.1.8 Power Management for Personal Computer Networks
This program encourages implementation of power management software to obtain energy
efficiency. Power management software saves energy by shifting personal computers to a low­
power operating state after a specified period of inactivity. When deployed on a network serving
multiple personal computers, this type of software can achieve significant energy savings.
Eligibility for participation in this program includes confirmation of electric usage, and
submission of pre-and post-install usage data. Post-installation reporting may be required for a
period of three years. The incentive available for this program is $5 per license. Avista
implements this program in a prescriptive manner, and incentives are issued to the participating
customer after the measure is installed.
Table 2-8: Power Management for PC Networks Program Measures
Measure Incentive
PC Power Management Software $5 / license
2.2.1.9 Commercial Windows & Insulation
The Commercial Windows & Insulation program offers incentives to Avista's non-residential
electric (Schedule 11, 12, 21 , 25) or natural gas (Schedule 101 , 111, 121) customers for
improvements to bu ilding envelopes through window upgrades and adding insulation. To
participate in this prescriptive rebate program, customers must submit documentation of the
project that includes post-installation R-values and affected square footage for insulation, and
documentation of U-value, solar heat gain coefficient, and size for window replacements. The
incentive levels for insulation project are dependent on the pre-and post-retrofit level of
insulation. Avista implements this program in a prescriptive manner, and incentives are issued
to the participating customer after the measure is installed .
Table 2-9: Commercial Windows & Insulation Measures
Measure Incentive ($ / sf)
Less than R4 Wall Insulation to R-11-R 18 Retrofit $0.30
Less than R4 Wall Insulation to R19 or above Retrofit $0.35
Less than R11 Attic Insulation to R30-R44 Retrofit $0.20
Less than R11 Attic Insulation to R45 or above Retrofit $0.25
Less than R11 Roof Insulation to R30 or above Retrofit $0.25
Windows U-Factor of .35 or less and SHGC .35 or Less (New Construction) $0.50
Windows U-Factor of .35 or less and SHGC .35 or Less (Retrofit) $0.50
2.2.1.1 O Commercial Water Heaters
The Commercial Water Heaters program provides incentive to electric (Schedule 11 , 12, 21 , 25)
or natural gas (Schedule 101, 111, 121) customers for the purchase and installation of an
energy efficient commercial water heater. Water heaters must be commercial grade units and
must be served by an Avista fuel source. An incentive of $20 per unit is provided for qualified
water heaters. Water heater eligibility guidelines are outlined in Table 2-10. Avista implements
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 19
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 33 of 212
2 INTRODUCTION
this program in a prescriptive manner, and incentives are issued to the participating customer
after the measure is installed.
Table 2-10: Commercial Water Heater Measures
Electric Natural Gas
Tank Size (gal) Energy Energy Incentive
Factor Factor
Greater than or equal to 25 gallons but less than 35 gallons 0.90 0.70
Greater than or equal to 35 gallons but less than 45 gallons 0.90 0.70
Greater than or equal to 45 gallons but less than 55 gallons 0.90 0.70
$20
Greater than or equal to 55 gallons but less than 75 gallons 0.87 0.68
Greater than or equal to 75 gallons but less than 100 gallons 0.87 0.68
Greater than or equal to 100 gallons but less than 120 gallons 0.86 0.68
2.2.1.11 Standby Generator Block Heater
This program provides an incentive to Avista's nonresidential electric customers (Schedule 11,
12, 21 , 25) for the purchase and installation of a more efficient style of engine block heater.
Traditional block heating technology employs a thermosiphon to drive circulation in an engine
block. A more efficient option uses pump driven circulation and results in less wasted heat flow
between the engine block and the ambient environment. This rebate is available for a retrofit
only and requires pre-approval from Avista to do pre and post logging. The available incentive is
$400 per heater.
Table 2-11: Fleet Heat Measures
Measure Incentive
Standby Generator Block Heater $400 I unit
2.2.2 Small Business
The Small-Medium Business (SMB) program is administered by SBW consulting and is a direct
installation/audit program providing customer energy-efficiency opportunities by: (1) directly
installing appropriate energy-saving measures at each target site, (2) conducting a brief onsite
audit to identify customer opportunities and interest in existing Avista programs, and (3)
providing materials and contact information so that customers are able to follow up with
additional energy efficiency measures under existing programs. This program is only available
to customers who receive electric service under Rate Schedule 11 in Washington and Idaho,
and to customers who receive natural gas service under Rate Schedule 101 in Washington.
Schedule 11 customers typically use less than 250,000 kWh per year.
Direct-install measures include faucet aerators, showerheads, pre-rinse spray valves, screw-in
LEDs, smart strips, CoolerMisers, and VendingMisers (Table 2-12). The evaluation team
conducted onsite verification, documentation audits, and engineering analysis to determine
verified gross savings for each measure in the program.
t.-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 20
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 34 of212
2 INTRODUCTION
Table 2-12: Small Business Program Measure Overview
Category Measure Description Cost
Lighting
Hot Water
Cooler Miser
Vending Miser
Tier 1 Smart Power Strip
Screw in LED Lamp (40W Equivalent)
Screw in LED Lamp (60W Equivalent)
Screw in LED Lamp (100W Equivalent)
Screw in LED BR30
Screw in LED BR40
i Screw in LED PAR30
I Screw in LEDPAR38
Low-flow faucet aerator (0.5 gpm) Electric Water Heat
Low-flow faucet aerator (1.0 gpm) Electric Water Heat
Low-flow faucet aerator (0.5 gpm) Gas Water Heat
Low-flow faucet aerator (1 .0 gpm) Gas Water Heat
Pre-Rinse Spray Valve Electric Heat
Pre-Rinse Spray Valve Gas Heat
Shower Head Fitness Electric
: Shower Head Fitness Gas
i Shower Head Electric
Shower Head Gas
Control for glass-front cooler that uses passive
infrared (PIR) sensor to power down machine when
surrounding area is vacant
I Control for refrigerated beverage machine that uses
I passive infrared (PIR) sensor to power down machine
! when surrounding area is vacant
I Eliminate standby power draw of peripheral devices I while continuing to power devices in "hot" outlets
$17 /lamp
$17 /lamp
$31 /lamp
$22 /lamp
$28 /lamp
$28 /lamp
$32 /lamp
$8 /unit
$8 /unit
$8 /unit
$8 /unit
$129 /unit
$129 /unit
$41 /unit
$41 /unit
$41 /unit
$41 /unit
$225 /unit
$225 /unit
$39 /unit
2.2.3 Residential
Avista's residential portfolio is composed of several approaches to engage and encourage
customers to consider energy-efficiency improvements in their homes. Prescriptive rebate
programs are the main component of the portfolio, together with a variety of other interventions.
These include upstream buy-down of low-cost lighting and water-saving measures; select
distribution of low-cost lighting and weatherization materials; an appliance recycling program; a
low-interest loan program; direct-install programs; and a multi-faceted, multichannel outreach
and customer engagement effort.
Throughout 2014 and 2015, Avista provided incentives and services for its residential electric
and gas customers in its Idaho service territory and for residential electric customers throughout
t.-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 21
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 35 of212
2 INTRODUCTION
its Idaho service territory. The evaluation team examined nine core programs in Idaho that
constituted the bulk of Avista 's residential energy-efficiency offerings in 2014 and 2015. Table
2-13 provides a summary of those programs, and the sections below detail each program.
Table 2-13: Residential Program Type and Description
Rebate
Midstream
Behavior
Low-income
Appliance Recycling
ENERGY STAR®
Homes
Fuel Efficiency
HVAC Program
Shell
Water Heater
Residential Lighting:
Simple Steps, Smart
Savings
Home Energy Reports
Low-income Programs
2.2.3.1 Appliance Recycling
JACO
Avista
Avista
Avista
Avista
Avista
CLEAResult
Opower
Rebate for recycling fridge or freezer older than
. 1995. This program was discontinued in June
I 2015.
I I Rebate for purchase of ENERGY STAR® home
I
!I Rebate for conversion of electric to natural gas
furnace and/or water heater
I
i Rebate for purchase of energy efficient and high
efficiency HVAC equipment, including variable I speed motors, air source heat pump, natural gas
I furnace and boiler, and smart thermostat
I Rebate for adding insulation to attic, walls, and
I floor, as well as adding energy efficient windows.
Rebate for installation of high efficiency gas or
electric water heater, natural gas water heater, and
Smart Savings showerhead.
I Direct manufacture discount for purchase of I approved CFLs, LEDs (bulbs and fixtures), and
! low-flow showerheads. ;
The Opower program generates behavioral savings
from a treatment group, which receives Home
Energy Reports, which compares the customers
energy usage to similar homes in Avista's service
territory.
! CAPs within Avista's Washington and Idaho service
Community Action I territories implement the projects. CAPs determine
Partners (CAPs) I energy-efficiency measure installations based on
1 the results of a home energy audit.
The appliance recycling program, administered by JACO Environmental Inc., provided a pick-up
and recycling service for operational refrigerators or freezers manufactured before 1995. JACO
provided the pick-up service free to customers and the $30 rebate was provided for each
operational refrigerator and/or freezer, up to two per household (Table 2-14). JACO provided
the following data points to Avista on a monthly basis: date of pick-up, customer name, address,
city state zip, type of unit collected and number of units collected . The appliance recycling
program ceased operation in June 2015 as a result of revised RTF values that became effective
in July of 2015 causing the program to cease to be cost-effective.
L-'1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 22
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 36 of 212
2 INTRODUCTION
Table 2-14 Appliance Recycling Measures and Incentives
Measure Rebate
Pre-1995 Freezer $30
Pre-1995 Refrigerator $30
2.2.3.2 HVAC Program
Avista internally manages the HVAC program which encourages the implementation of high
efficiency HVAC equipment and smart thermostats through direct incentives issued to the
customer after the measure has been installed (Table 2-15). This program is available to all
residential electric or natural gas customers with a winter heating season usage of 4,000 or
more kilowatt hours, or at least 160 therms of space heating the prior year. Existing or new
construction homes are eligible.
Table 2-15 HVAC Measure Overview
Fuel Efficiency Measures Rebate
Variable speed motor $100
Electric to air source heat pump $900
High efficiency natural gas furnace $250
High efficiency natural gas boiler $250
Smart thermostat $50 or $100
2.2.3.3 Water Heat
Customers replacing their existing electric or natural gas water heater are eligible to receive a
rebate for selecting a high efficiency option. This program also includes discounted
showerheads available at participating retailers throughout Avista's WA and ID service territory
under the Simple Steps, Smart Savings program. Table 2-16 outlines the measures offered and
rebate per unit.
Table 2-16 Water Heat Program Measure Overview
Water Heat Measure Rebate
Electric; 35-55 gallon with 0.94 EF or higher $20
Natural Gas; 40 gallon with 0.62 EF or higher $20
Natural Gas; 50 gallon with 0.60 EF or higher $20
Natural Gas: Tankless with 0.82 EF or higher $130
Simple Steps, Smart Savings Low-flow Showerheads: 1.5-2 GPM buydown
2.2.3.4 ENERGY STAR® Homes
ENERGY STAR® certified home construction is administered by a Northwest Energy Efficiency
Alliance (NEEA) regional program. Avista provides a rebate for homes within their service
territory that successfully make it through this ENERGY STAR® certification process. In addition
to NEEA's program, the manufactured homes industry has established a labeling program for
Energy Star certified manufactured homes, which Avista also incentivizes. New home buyers
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 23
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 37 of212
2 INTRODUCTION
can apply for an $800 rebate for an ENERGY STAR® ECO-rated new manufactured home or
$1 ,000 for an ENERGY STAR® stick-built home. The purchaser must submit the application and
certification paperwork to Avista within 90 days of occupying the residence. The ENERGY
STAR® home rebate may not be combined with other Avista individual measure rebates (e.g.
high efficiency water heaters).
Table 2-17 describes eligible measures available for the program.
Table 2-17 ENERGY STAR® Homes Measure Overview
Energy Star Home Measure Rebate
Stick built -electric $1 ,000
Stick built or manufactured w/ gas only $650
Manufactured w/ furnace $800
Manufactured w/ heat pump $800
2.2.3.5 Fuel Efficiency Program
The fuel efficiency program offers a rebate for the conversion of electric straight resistance heat
to natural gas, as well as the conversion of electric hot water heaters to natural gas models. The
home must have used 4,000 or more kWh of electric space heat during the previous winter
season to be eligible for flat-rate rebates. If natural gas is not available or is not suitable for the
home, the installation of an air source heat pump as a replacement unit is accepted (see electric
to air source heat pump measure under 2.2.3.2 HVAC Program.
Table 2-18 Fuel Efficiency Measure Overview
Fuel Efficiency Measures Savings (kWh) Rebate
Electric to natural gas conversion -space heat 12,012 $2,300
Electric to natural gas conversion -water heat 4,031 $600
Electric to natural furnace and water heat -combo 16,043 $3,200
Electric to natural gas wall heaters -space heat 10,932 $1 ,300
2.2.3.6 Residential Lighting
The Simple Steps, Smart Savings program provides discounts to manufacturers to lower the
price of efficient light bulbs, light fixtures, showerheads, and appliances. This program, launched
by Bonneville Power Administration (BPA) and administered by CLEAResult, operates across
the Pacific Northwest. Utilities are able to select which reduced price items to include in their
territory. Avista's offerings include a selection of general and special CFLs, LED light fixtures,
and LED bulbs2. Retailers such a big box stores and regional and national chains are the
primary recipient of the product and typically select from Avista's approved options what they
will carry at their store location. These products are clearly identified with a sticker indicating
they are part of the Simple Steps, Smart Savings program. Avista also encourages the use of
2 Avista offered LED bulbs in 2014 and the last half of 2015.
t.-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 24
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 38 of 212
2 INTRODUCTION
the LightRecycle CFL recycling locations throughout their Idaho service territory, to further
support the utilization of CFL's.
2.2.3.7 Shell Program
Avista's internally managed shell program incentivizes measures that improve the integrity of
the home's envelope (Table 2-19). For insulation and windows: rebates are issued to the
customer after measure has been installed. Eligibility guidelines for participation include but may
not be limited to: confirmation of electric or natural gas heating usage, itemized invoices
including insulation levels or window values and square footage. Pre and/or post-inspection of
insulation and windows may occur as necessary throughout the year. Customer must
demonstrate a winter heating season electricity usage of 4,000 kilowatt hours or 160 therms to
be eligible for insulation and window program participation. Addition of insulation that increases
the R-value by R-10 or greater for both fitted/batt type and blow-in products are eligible.
Windows with a U-factor of 0.30 or less that replace single or double pane windows are eligible.
Table 2-19 Shell Measure Overview
. . Existing Equipment Fuel Eff1c1ency Measures Eff' . Rebate ($/sf) 1c1ency
Attic insulation
Wall insulation
Floor insulation
Window insulation
2.2.3.8 Home Energy Reports
I R-19 or less
! R-5 or less ;
I R-5 or less
i 0.30 u-factor or lower
i
$0.15
$0.25
$0.20
$4.00
Avista provides peer comparison reports of home energy consumption, termed Home Energy
Reports (HER), through Opower. This is an opt-out program aimed to encourage customers to
save energy. 73,500 customers were initially mailed HERs in June of 2013: 48,300 to
Washington customers and 25,200 to Idaho customers. The cadence of reports began by
sending out a report every month for the first three months followed by a bi-monthly mailing of
reports thereafter, continuing until June 2016. Customers must be a recipient of Avista electricity
to qualify. Reports do not have a gas or dual fuel focus, though approximately 42% of recipients
also have a gas meter.
2.2.3.9 Low Income
Avista leverages Community Action Program (CAP) agencies to deliver energy efficiency
programs to low-income customers. CAP agencies have resources to income qualify, prioritize
and treat homes based upon a number of characteristics . In addition to the Company's annual
fund ing, the Agencies have other monetary resources that they can usually leverage when
treating a home with weatherization and other energy efficiency measures. The Agencies either
have in-house or contractor crews to install many of the efficiency measures of the program.
One CAP agency, Community Action Partnership -Lewiston, serves Avista's Idaho service
territory. Avista provides the CAP agency with an "approved measure list", the items on this list
"., Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 25
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 39 of 212
2 INTRODUCTION
are reimbursed 100% (Table 2-20). Avista also provides a "rebate list" of additional energy
saving measures that the CAP agency can utilize (Table 2-21).
Table 2-20 Low Income Approved Measure List (100% of costs offset by Avista)
Measures
Electric to Gas Furnace Conversion
Electric to Gas Water Heater Conversion
Insulation (ceiling/ attic, floors and walls)
Insulation (duct)/ Duct sealing
Air Infiltration
Energy Star® Doors
Energy Star® Windows (gas heat)
Table 2-21 Low Income Rebate List
Measures
I
Electric to air source heat pump (when natural gas not
viable)
Electric to natural gas water heater
Electric Water Heater (0.93 EF)
Gas Water Heater (0.62 EF)
Air Source Heat Pump
Gas Furnace (>90% AFUE)
Duct insulation (electric heat)
Duct insulation (gas heat)
Energy Star® Windows
Energy Star® Refrigerators
Energy Star® Windows (electric heat)
2.3 Program Participation Summary
Reported participation and savings for Avista's 2014 and 2015 programs is outlined in Table
2-22 and Table 2-23.
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 26
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 40 of 212
2 INTRODUCTION
Table 2-22 Avista Nonresidential Reported Participation and Savings
P 2014-2015 Project 2014-2015 Reported rogram . Count Savings (kWh)
EnergySmart Grocer 149 2,387,662
Food Service Equipment 20 130,946
Green Motors 23 43,954
Motor Controls HVAC 4 466,340
Commercial Water Heaters 3 190
Prescriptive Lighting 327 3,475,049
Prescriptive Shell 9 54,381
Fleet Heat 3 7,228
Site Specific 125 5,813,610
Small Business 0 0
TOTAL 663 12,379,360
Table 2-23 Avista Residential Reported Participation and Savings
2014-2015 Participation 2014-2015 Reported Program . Count Savings (kWh)
Appliance Recycling 400
HVAC 599
Water Heat* 4,306
ENERGY STAR Homes 19
Fuel Efficiency 405
Lighting** 462,144
Shell 370
Opower*** 19,366
Low Income**** 7302
TOTAL 494,911
*Includes counts for both projects and showerheads
**Denotes bulb count and includes Simple Steps, and Giveaway
***Number of participants in the Treatment in January, 2015
****Includes both projects and counts of bulbs
261 ,924
872,828
239,267
140,538
5,290,679
8,323,842
903,663
2,746,000
758,955
19,537,696
2.4 Evaluation Goals and Objectives
"Model Energy-Efficiency Program Impact Evaluation Guide - A Resource of the National Action
Plan for Energy Efficiency," published in November 2007. The report states:
'-',.,Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 27
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page41 of212
2 INTRODUCTION
Evaluation is the process of determining and documenting the results, benefits,
and lessons teamed from an energy-efficiency program. Evaluation results can
be used in planning future programs and determining the value and potential of a
portfolio of energy-efficiency programs in an integrated resource planning
process. It can also be used in retrospectively determining the performance (and
resulting payments, incentives, or penalties) of contractors and administrators
responsible for implementing efficiency programs.
Evaluation has two key objectives:
1. To document and measure the effects of a program and determine
whether it met its goals with respect to being a reliable energy resource.
2. To help understand why those effects occurred and identify ways to
improve.
Avista has identified the following objectives for the evaluation:
• Independently verify, measure and document energy savings impacts from Avista's
electric and natural gas energy efficiency programs, or for program categories
representing consolidated small scale program offerings, by Avista in 2014 and 2015
• Analytically substantiate the measurement of those savings
• Calculate the cost effectiveness of the portfolio and component programs
• Identify program improvements, if any,
• Identify possible future programs .
.... 1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 28
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 42 of 212
3 Impact Evaluation Methodology
The impact evaluation evaluated the gross savings attributable to Avista's 2014 and 2015
energy-efficiency programs. Impact evaluations generally seek to quantify the energy and, when
possible, the non-energy savings that have resulted from DSM program operations. These
savings may be expressed as all of the changes resulting from the program (gross savings), or
only those changes that would not have occurred absent the program (net savings).
The evaluation team verified the gross energy savings of Avista's 2014 and 2015 programs by:
• Understanding the program context
• Designing the impact evaluation sample
• Verifying the project and program savings through document review, telephone surveys,
onsite measurement and verification, and billing analysis
• Comparing Avista-reported savings to savings verified during project-level evaluations to
determine verified gross savings.
3.1 Understanding the Program Context
The first significant step of the evaluation activities was to gain a comprehensive understanding
of the programs and measures being evaluated. Specifically, the team explored the following
documents and data records:
• Avista's 2014 and 2015 Demand Side Management (DSM) Business Plans which detail
processes and energy savings justifications
• Program tracking databases/spreadsheets and participation through December 2014
• Project documents from external sources, such as documents from customers, program
consultants, or implementation contractors.
Based on the initial review, the evaluation team outlined the distribution of program contributions
to the overall portfolio of programs. In addition, the review allowed the evaluation team to
understand the sources for unit energy savings for each measure offered in the programs, along
with the sources for energy-savings algorithms and the internal quality assurance and quality
control (QNQC) processes for large nonresidential projects. Following this review, the
evaluation team designed the sample strategy for the impact evaluation activities, as discussed
in the following section.
3.2 Designing the Sample
Sample development enabled the evaluation team to deliver meaningful, defensible results to
Avista. The sampling methodology used for the impact evaluation was guided by a value of
information (VOi) framework, which allowed the team to target activities and respondents with
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 29
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 43 of 212
3 IMPACT EVALUATION METHODOLOGY
expected high impact and yield, while representing the entire population of interest. In general,
VOi focuses budgets and rigor towards the programs/projects with high uncertainty and high
impact3.
For the sample design, the evaluation team organized the programs into evaluation "bins,"
segmenting the programs based on two metrics:
• Program Uncertainty: The risks associated with a program's reported savings were
broken into three categories: high, medium, and low. Risks included custom vs. deemed
vs. Regional Technical Forum status, delivery mechanism, performance goals, etc.
• Program Size: A determination of size-either large or small-was based on projected
energy savings and planned budget allocations.
Bins were created for: (1) residential and nonresidential programs and (2) electric
(Washington/Idaho) and natural gas (Washington) programs.
In parallel, the evaluation team calculated a "level of rigor" value for each program; based on
assumed measure complexity and Regional Technical Forum (RTF) influence, the team
identified an appropriate level of sampling and evaluation rigor.
• Level of Sampling: Defined as confidence/precision (C/P) for calculating sample sizes,
the evaluation team used three levels for sampling: 90/10, 85/15, or 80/20 C/P.
• Evaluation Rigor: Defined as the level of detail used for the evaluation activities, the
team identified four levels of increasing evaluation rigor: document audit, surveys, onsite
inspections, and billing analysis. In many cases, a combination of these four approaches
was used to both validate savings and provide insights into any identified discrepancies
between reported and verified savings values.
The evaluation bin identified for each program was one factor in determining the sample size
and level of rigor for the evaluation activities. Additional factors that influenced the sample size
and level of rigor included evaluation costs, RTF influence, and findings and recommendations
from previous evaluations.
Table 3-1 and Table 3-2 show the anticipated confidence/precision level, planned sample sizes,
and level of rigor, by program, for the Washington/Idaho electric residential and nonresidential
portfolios. The samples are drawn to meet the specified confidence/precision for each program
and to meet 90% confidence and 10% precision at the portfolio level4. Because programs do not
differ between the Washington and Idaho service territories, the sample approach was
combined for both territories, and the findings from the impact evaluation (i.e. realization rates)
were applied across both states.
3 See Appendix A for detailed discussion on sampling and estimation.
4 See Appendix A for detailed information on the presentation of uncertainty.
'-'"Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 30
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 44 of 212
3 IMPACT EVALUATION METHODOLOGY
Table 3-1: Planned Sampling and Evaluation Rigor for Washington/Idaho Electric
Residential Programs
. . . Document Onsite . . . Electric Res1dent1al Program Target C/P A d" Surveys . B1lhng Analysis u rt lnspecbons
Residential Appliance Recycling 90/10 70
HVAC Program 90/10 67 67
Water Heat Program 1 80/20 11 11
ENERGY STAR Homes 85/15 15 15 census
Fuel Efficiency 85/15 24 24 census
Residential Lighting Program2 90/10 703
Shell Program 85/15 24 24 census
Opower Behavioral Program census census
Low Income 85/15 24 census
TOTAL 165 211 70
Includes Simple Steps, Smart Savings upstream showerhead component
21ncludes Simple Steps, Smart Savings upstream lighting program and CFL giveaway events
3Denotes sample size for residential lighting program logger study
Table 3-2: Sampling and Evaluation Rigor for Washington/Idaho Electric Nonresidential
Programs
Electric Nonresidential Program Target Document Onsite
C/p A d·t Surveys 1 . Billing Analysis u I nspect1ons
Prescriptive Lighting 90/10 68 16 16
Prescriptive EnergySmart Grocer 95/15 44 15 15
Prescriptive Non-Lighting Other 90/15 24 9 9
Cascade Energy Pilot 80/20 5 5
Site Specific 90/10 84 84 84 based on IPMVP5
Small Business 90/15 31 31 31
TOTAL 225 129 124
For the purposes of the evaluation sampling, the evaluation team has bundled the following
nonresidential electric programs into one program titled "Prescriptive Non-Lighting":
•
5 International Performance Measurement and Verification Protocol
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 31
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 45 of 212
3 IMPACT EVALUATION METHODOLOGY
• Food Service Equipment • Power Management for PC
Green Motors Rewind Networks •
• Windows & Insulation • HVAC Variable Frequency Drive
• Standby Generator Block Heater • Clothes Washers
Table 3-3: Achieved Sampling and Confidence/Precision for Washington/Idaho Electric
Residential Programs
. . Achieved Document On site Electric Res1dent1al Program C/P A d" Surveys 1 • u rt nspecbons
Residential Appliance Recycling N/A 70 72
HVAC Program 90/31 68 68
Water Heat Program 1 90/13 24 13
ENERGY STAR Homes 90/14 19 16
Fuel Efficiency 90/7 26 25
Residential Lighting Program2 90/15.3 75
Shell Program 90/33 28 28
Opower Behavioral Program 90/8
Low Income 90/13 24
TOTAL 90/9 259 222 75
Table 3-4: Achieved Sampling and Evaluation Rigor for Washington/Idaho Electric
Nonresidential Programs
Electric Nonresidential Program Achieved Document Onsite . Surveys . C/P Audit Inspections
Prescriptive Lighting 90/13 68 22 22
Prescriptive EnergySmart Grocer 95/14 44 20 20
Prescriptive Non-Lighting Other 90/228 24 15 15
Site Specific 90/7 101 84 84
TOTAL 90/7 237 141 141
Small Business 90/25 31 31
TOTAL INCLUDING SMALL 268 141 172 BUSINESS:
3.3 Database Review
For the Small Business and Residential programs, the evaluation team conducted a review of
the program databases as provided by Avista and its third-party implementers. The purpose of
the review was to look for large outliers in program-reported data and to remove any duplicate
t.-1 Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 32
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 46 of 212
3 IMPACT EVALUATION METHODOLOGY
entries found in the databases. The outcome of the database review was an "adjusted reported"
participation count and savings value for each measure and program. The realization rate that
the evaluation team calculated as part of the gross verified savings activities, described in the
following section, was then applied to the adjusted reported savings value.
3.4 Verifying the Sample -Gross Verified Savings
The next step in the impact evaluation process was to determine the gross impacts, which are
the energy savings that are found at a customer site as the direct result of a program's
operation ; net impacts are the result of customer and market behavior that can add to or
subtract from a program's direct results.
The impact evaluation activities resulted in realization rates, which were applied to the adjusted/
reported savings. The ratio of the savings determined from the site inspections, measurement
and verification (M&V) activities, or engineering calculations to the program-reported savings
was the project realization rate; the program realization rate was the weighted average for all
projects in the sample. The savings obtained by multiplying the program realization rates by the
program-adjusted/reported savings were termed the gross verified savings. These gross verified
savings reflect the direct energy and demand impact of the program's operations.
Total program gross sc1vings were adjusted using the following equation:
Where:
kWhadi
kWhrep
Realization rate
=
=
=
k WhadJ = k Wh,.ep · Realization Rate
kWh calculated by the evaluation team for the program, the gross
impact
kWh reported/adjusted for the program
weighted average kWh adj I kWhrep for the research sample
The estimate of gross verified energy savings occurred through one or more levels of evaluation
rigor, as detailed in the following sections.
3.4.1 Document Audit
The first level of rigor that the evaluation team used was a document audit of all sampled
projects for which documentation existed. Document audits were also a critical precursor for
conducting telephone surveys and onsite inspections and, more specifically, for determining
project-specific variables to be collected during these activities. The document audit for each
sampled project sought to answer three questions:
• Were the data files of the sampled projects complete, well documented, and adequate
for calculating and reporting the savings?
L-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 33
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 47 of 212
3 IMPACT EVALUATION METHODOLOGY
• Were the calculation methods correctly applied, appropriate, and accurate?
• Were all the necessary fields properly populated?
3.4.2 Telephone Survey
A second level of evaluation rigor was through stand-alone telephone surveys with program
participants. Telephone surveys were conducted in conjunction with the process evaluation
activities and were used to gather information on the energy-efficiency measure implemented,
the key parameters needed to verify the assumptions used by RTF for approved values or to
estimate verified energy savings, and any baseline data that may be available from the
participant.
3.4.3 Onsite Measurement and Verification
A sample of projects in the nonresidential sector was selected for onsite measurement and
verification activities. Before conducting site inspections, it was important for field engineers to
understand the project that they were verifying. This understanding built from the document­
aud it task discussed earlier. For all onsite inspections, a telephone survey served as an
introduction to the evaluation activities and was used to confirm that the customer participated in
the program, to confirm the appropriate contact, and to verify basic information such as building
type and building size. All onsite activities were conducted by evaluation team field engineers.
The evaluation team conducted two levels of rigor associated with the onsite inspections -
measurement and verification (M&V) and verification-only (V). Upon review of the project
documents, the evaluation team decided which level of rigor was appropriate for each sampled
project/measure. In cases where the measure had an approved RTF UES value, the evaluation
team's effort focused on verifying the quality and quantity of installation to apply the RTF UES
values to.
An M&V plan was developed for each M&V-designated project. The team based these plans on
a review of the available calculation methods and assumptions used for determining measure­
level energy savings. These plans aided in understanding what data to collect during onsite
visits and telephone surveys to calculate gross verified savings for each sampled project.
M&V methods were developed with adherence to the IPMVP. As defined by IMPVP, the general
equation for energy savings is defined as: 6
Normalized Savings =
(Baseline Energy ± Routine Adjustments to fixed conditions ± Non-Routine Adjustments to
fixed conditions) -( Reporting Period Energy ± Routine Adjustments to fixed conditions ±
Non-Routine Adjustments to fixed conditions)
The broad categories of the IPMVP are as follows:
6 Efficiency Valuation Organization (EVO) "International Performance Measurement and Verification Protocol (IMPVP) Concepts
and Options for Determining Energy and Water Savings Volume 1", April 2007, page 19.
i1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 34
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 48 of 212
3 IMPACT EVALUATION METHODOLOGY
• Option A, Retrofit Isolation: Key Parameter Measurement -This method uses
engineering calculations, along with partial site measurements, to verify the savings
resulting from specific measures.
• Option B, Retrofit Isolation: All Parameter Measurement -This method uses engineering
calculations, along with ongoing site measurements, to verify the savings resulting from
specific measures.
• Option C, Whole Facility: This method uses whole-facility energy usage information,
most often focusing on a utility bill analysis, to evaluate savings.
• Option D, Calibrated Simulation: Computer energy models are employed to calculate
savings as a function of the important independent variables. The models must include
verified inputs that accurately characterize the project and must be calibrated to match
actual energy usage.
In addition, the evaluation team conducted metering tasks on a subset of the onsite inspection
sample chosen for the M&V level of rigor. Projects were selected for metering activities based
on the measure type, project complexity, and the level of information needed to estimate gross
savings for the project.
3.4.4 Billing Analysis
Participants received an assortment of efficiency measures through Avista's residential rebate
programs. Billing analyses are generally considered a best practice for calculating energy
savings resulting from "whole-house" efficiency retrofits. Thus, because of the diverse and
interactive savings profiles associated with the improvements, the evaluation team determined
that a utility bill regression analysis (IPMVP Option C) was the best method for quantifying
energy savings resulting from the programs' treatment measures.
The utility billing analysis used data from participating customers who had sufficient utility-billed
consumption records before and after the measure installation. Specifically, the evaluation team
used a billing analysis approach for estimating gross verified savings for some or all measures
in the following residential programs: Shell, Fuel Efficiency, HVAC, Opower, and Low Income.
The remainder of this section outlines the general approach that the team followed for
conducting the billing analysis. More specific details related to each program and measure
evaluation are provided in Section 6.
The evaluation team requested program tracking data and complete billing histories for Avista's
residential rebate program participants. IPMVP Option C utility bill analysis works best when at
least one full year of utility billing data before and after the measure installation are available for
comparison. This ensures that seasonal effects of the improvements are captured in the savings
estimates. However, because of the timing of measure installations and the nature of certain
programs, some customers had a limited amount of pre-retrofit and/or post-retrofit billing data.
For example, accounts under the ENERGY STAR® Homes program do not have any "pre"
billing data and, as a result, alternative methods were applied.
t.-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 35
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 49 of 212
3 IMPACT EVALUATION METHODOLOGY
Before performing the analysis, utility billing records were assessed for quality and
completeness. Duplicate observations were removed from the billing data. Billing periods of
more than 35 days or less than 26 days were also excluded from the dataset because these
observations are not representative of a typical billing cycle.
In addition to program participation records and customer billing histories, the evaluation team
collected daily temperature records and normal weather conditions (TMY3) from three weather
stations located in Avista's service territory. Observed temperature records were used to
calculate the number of heating degree days (HOD) and cooling degree days (COD) in each
customer's monthly billing period. Weather stations used by the evaluation team include Coeur
d'Alene, Idaho; Lewiston, Idaho; and Spokane, Washington. Each participant was matched to
the nearest weather station based on service address.
Gross verified energy savings were calculated by comparing billed consumption in months prior
to the measure installations to the billed consumption in months after the measure installations.
For most programs the evaluation team required homes to have 12 months of pre-retrofit
consumption and 12-months of post-retrofit consumption for inclusion in the billing analysis. In
cases in which participation was limited, this requirement was relaxed to increase sample sizes,
provided that the participating homes had data from the key seasons. For example, switching
from electric heat to a natural gas furnace will produce the largest savings during winter months.
Because of the March 2016 timing of billing data collection, homes who implemented the fuel
conversion measure in the summer of 2015 might have a full 12 months of pre-retrofit data but
only 6 to 8 months of post-retrofit data. However, the post-retrofit period included the heating
season and gave the regression model sufficient data upon which to establish a mathematical
relationship between weather and consumption.
Table 3-5 defines the terms and coefficients shown in the two equations that follow. Equation
3-1 shows the general regression model specification used for electric measures, Equation 3-2
shows the general model specification used for gas measures. The key difference between
them is the absence of cooling degree day (COD) terms in the gas model. Because residential
gas consumption is predominantly associated with heating, the evaluation team opted to
exclude the COD terms from the gas model, resulting in more robust impact estimates.
Equation 3-1: Regression Model Specification for Electric Measures
kWhi t = ~i + ~1 X Postit + ~2 X CDDit + ~3 (Post X CDD)it + ~4 X HDDit + ~5 (Post X HDD)it + Eit
t.-1Nexanr
Equation 3-2: Regression Model Specification for Gas Measures
Ther msit = ~i + ~1 X Postit + ~2 X HDDit + ~3(Post X HDD\t + Eit
Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 36
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 50 of 212
3 IMPACT EVALUATION METHODOLOGY
Table 3-5: Fixed Effects Regression Model Definition of Terms
Variable Definition
kWh;1 / Therms;1
Post1
CDD;1
HDD;t
[3;
€it
I Estimated consumption in home i during period t (dependent variable)
I indicator variable denoting pre-installation period vs. post-installation period
!
I Average cooling degree days during period t at home i
I Average heating degree days during period tat home i
I Customer specific model intercept representing baseline consumption
!
I Coefficients determined via regression describing impacts associated with independent
I variables
Customer-level random error
The model specifications shown in Equation 3-1 and Equation 3-2 were used to determine the
coefficients describing the relationship between consumption and weather. That relationship
was then applied to normal weather conditions to estimate average annual consumption in the
pre-installation and post-installation periods to calculate weather normalized savings.
The evaluation team used a multi-faceted approach to estimate savings for many of Avista's
programs. The evaluation team used the fixed-effects regression models summarized above,
together with a pooled approach , which combined all participants and billing periods into a
single regression analysis to estimate weather normalized savings at the program or measure
level. In some cases, the team then ran individual customer regressions to obtain weather
normalized savings estimates for each customer, allowing for a more granular assessment of
how savings magnitudes were distributed across the program or measure population. In
addition, for measures with relatively small impact estimates, we included a control group
constructed from homes in the Opower program, to achieve a more stable baseline comparison.
For these measures, estimates were based on a difference-in-differences regression analysis of
billing data from customers in the treatment and comparison groups.
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 37
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 51 of 212
4 Nonresidential Impact Evaluation
This section outlines the impact evaluation methodology and findings for each of the evaluated
nonresidential programs.
4.1 Overview
Avista offered 14 nonresidential programs in their Idaho service territory in 2014 and 2015. The
reported savings for the nonresidential programs are summarized in Table 4-1 .
Table 4-1: Nonresidential Program Reported Savings
. . . 2014-2015 Reported
Idaho Electric Nonres1dent1al Program Savings (kWh)
EnergySmart Grocer 2,387,662
Food Service Equipment 130,946
Green Motors 43,954
Motor Controls HVAC 466,340
Commercial Water Heaters 190
Commercial Clothes Washers
Prescriptive Lighting 3,475,049
Power Mgmt for PC Networks
Prescriptive Shell 54,381
Fleet Heat 7,228
AirGuardian
Site Specific 5,813,610
Cascade Strategic Energy Management
Small Business
TOTAL NONRESIDENTIAL 12,379,360
No participation was reported in five programs: Commercial Clothes Washers, Power
Management for PC Networks, AirGuardian, Cascade Strategic Energy Management, and
Small Business. The Site Specific program contributes the largest share of the reported savings,
47% as shown in Figure 4-1. Prescriptive Lighting is the next largest contributor at 28%.
t.-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 38
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 52 of 212
4 NONRESIDENTIAL IMPACT EVALUATION
Figure 4-1: Nonresidential Program Reported Energy Savings Shares
28%
• Site Specific
• Prescriptive Lighting
• EnergySmart Grocer
• Motor Controls HVAC
Prescriptive Shell
4% • Food Service Equipment
• Green Motors
Fleet Heat
Commercial Water Heaters
The evaluation team designed a sampling strategy for these programs placing the most
emphasis on the Site Specific program because of its large share of savings. The Site Specific
program was divided into two strata based on reported savings. As part of the evaluation
activities, a total of 237 document audits were conducted , and onsite inspections were
conducted on a sub-sample of 141 projects, as shown in Table 4-2. Engineering activities
included review of savings calculation methodology and assumptions, verification of operating
hours through participant surveys and included use of data loggers in some cases, utility bill
analysis, review of energy management system trend data, and energy savings analysis.
Table 4-2: Nonresidential Program Achieved Evaluation Sample
Achieved Document On Site Program/Group C/P A d"t Survey I t· u I nspec ions
Prescriptive Lighting 90/13 68 22 22
EnergySmart Grocer 90/14 44 20 20
Prescriptive Non-Lighting Other 90/228 24 15 15
Site Specific Large(> 275,000 kWh) 17 17 17
90/7
Site Specific Small(< 275,000 kWh) 84 67 67
TOTAL gon 237 141 141
L-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 39
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 53 of 212
4 NONRESIDENTIAL IMPACT EVALUATION
4.2 Prescriptive Lighting
4.2.1 Overview
The Prescriptive Lighting program encourages commercial customers and vendors to make
lighting improvements to their businesses. The program provides many common retrofits to
receive a pre-determined incentive based on baseline and replacement lamp wattages. The
program is internally implemented by Avista.
4.2.2 Program Achievements and Participation Summary
A total of 327 prescriptive lighting projects at 236 unique premises were installed in Idaho
across the 2014 and 2015 program years. Table 4-3 and Figure 4-2 summarize Avista's 2014-
2015 Prescriptive Lighting Program energy impacts by measure.
Table 4-3: Prescriptive Lighting Reported Energy Savings by Measure
Energy Savings . . Measure Type (kWh) % Electric Savmgs
Lighting (Exterior) 2,230,603 64%
Lighting (Interior) 1,244,446 36%
TOTAL 3,475,049 100%
Figure 4-2: Prescriptive Lighting Reported Energy Savings Shares
• Lighting (Exterior)
• Lighting (Interior)
4.2.3 Methodology
The impact evaluation for this program followed the RTF's Nonresidential Lighting Retrofit
Standard Protocol, IPMVP Option A (Retrofit Isolation: Key Parameter Measurement), and DOE
t.-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 40
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 54 of 212
4 NONRESIDENTIAL IMPACT EVALUATION
Uniform Methods Commercial and Industrial Lighting Evaluation Protocol7. Engineering activities
included installation verification, determination of operational hours including spot-metering in
for a sub-sample of projects, and engineering savings calculations.
4.2.3.1 Sampling
The evaluation team conducted document audits for 68 projects. Customer surveys and onsite
inspections were completed on a sub-sample of 22 of these projects (Table 4-4). Because of the
installation of multiple projects at some sites, the achieved sample size for onsite inspections
and surveys was slightly higher than the original sample design of 16 surveys and onsite
inspections as noted in Table 3-2.
Table 4-4: Prescriptive Lighting Achieved Sample
P Document Survey On Site
rogram A d"t I t· u I nspec ions
Prescriptive Lighting 68 22 22
4.2.3.2 Document Audits
Project documentation was requested for each sampled project, including invoices, savings
calculations, work order forms, equipment specification sheets, and any other project records
that may exist. Thorough review of this documentation was the first crucial step in evaluation of
each project.
4.2.3.3 Field Inspections
The telephone surveys conducted as part of the process evaluation were used to recruit
projects for onsite inspection. These onsite inspections provide a more rigorous way to verify
energy savings, and allowed the evaluation team to note any discrepancies between onsite
findings regarding actual measure and equipment performance and the information gathered
through the telephone surveys and project documentation . A survey instrument specific to this
program was created in advance of the site inspections to ensure that the correct information
was gathered.
Table 4-5 summarizes the information that was collected for each project during the onsite
inspection. All parameters needed to support the savings analysis of a project were collected ,
including fixture counts, baseline and post-retrofit wattages, hours of operation, and HVAC
system information (to inform calculation of interactive effects).
7 http://energy.gov/sites/prod/files/2013/11/f5/53827-2.pdf
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 41
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 55 of 212
4 NONRESIDENTIAL IMPACT EVALUATION
Table 4-5: Prescriptive Lighting Onsite Data Collection
End Use Category Baseline Retrofit
All Facilities
Lighting
i Year facility was built
I :~:~:~.:~;~:::"IB
i ! Operating Hours, posted or otherwise
I Total conditioned square footage
! Heating system type/age/efficiency/size/condition
I Cooling system type/age/efficiency/size/condition
I Lamp Type (e.g., TB, T12) i Lamp Type
[ Ballast Type (mag. or elec.) I Confirm Electronic Ballast and Factor
I Lamp Size (4 ft. or 8 ft.) I Lamp Size
! Quantity of Lamps per Fixture
I Wattage per Lamp
l Fixture Quantity
Operating Hours
Control Type
Quantity of Lamps per Fixture
! Wattage per Lamp
! Fixture Quantity
I Operating Hours
I Control Type
I Confirm ENERGY STAR@ rating
Where feasible and appropriate, the evaluation team also used standalone data loggers to
minimize uncertainty in the estimation of lighting operating hours. Evaluation team engineers
installed HOBO® U9-002 light on/off loggers for a minimum of four months. This collected
measured data was supplemented by lighting operating characterization as determined through
onsite interviews and surveys of control strategies (dimmers, timers, etc.) to inform the balance
of the yearly operating hours.
The data collected over the logging duration was tabulated per hour per week to create an
average weekly operation schedule for each measured space with energy efficiency measures.
The weekly hourly profile includes 24 hours of each of eight distinct day types (Sunday,
Monday, Tuesday, Wednesday, Thursday, Friday, Saturday, and holiday). Annual operating
hours were created by extrapolating measured values to a calendar year, adjusted as needed
per the interviews with onsite personnel.
4.2.3.4 Impact Analysis Methods
To calculate the gross verified energy savings of a lighting retrofit, the evaluation utilized the
calculation outlined in Equation 4-1:
Equation 4-1: Prescriptive Lighting Energy Savings Calculation
~kWh=(# fixturesbase * kWbase -# fixturesretrofit * kWretrofit) * Hours* IF
'-"'Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 42
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 56 of 212
4
Where:
NONRESIDENTIAL IMPACT EVALUATION
# fixturesbase orretrofit = Quantity of fixtures installed in baseline or retrofit of a project
Hours = Annual hours of fixture operation
IF = the ratio of heating and cooling electricity reduction per unit of
lighting energy reduction resulting from the reduction in lighting waste heat removed by
an electric HVAC system
Equation 4-1 is based on per fixture energy savings as calculated in Equation 4-2 and Equation
4-3:
Where:
Equation 4-2: Prescriptive Lighting Base Case Demand Savings Calculation
# lampsbase * Wattsbase * BFbase
kWbase = 1000
Equation 4-3: Prescriptive Retrofit Case Demand Savings Calculation
# lampsretrofit * Wattsretrofit * BFretrofit
kWretrofit = lOOO
# /ampsbase or retrofit
Watts base or retrofit
BF base or retrofit
= Quantity of lamps installed in a baseline or retrofit fixture
= Wattage of baseline or retrofit lamp
= Ballast factor of baseline or retrofit light fixture
The analysis utilized a TB baseline for linear fluorescent replacements, since T12 lamps are no
longer compliant under federal regulations (EISA 2007 and EPact 2005).
Interactive Equipment Energy Changes for Lighting Retrofits
The energy consumption of lighting equipment within an enclosed space is not viewed in
isolation. Building systems interact with one another and a change in one system will often
affect the energy consumption of another. This interaction is important to consider when
calculating the benefits provided by lighting equipment because it adopts a comprehensive view
of premise-level energy changes rather than limiting the analysis to the energy change directly
related to the modified equipment. The evaluation team utilized the interactive factors
designated in the RTF's Non-residential Lighting Retrofits protocol8 and included in Appendix B.
Engineers gathered heating and cooling system types serving each space affected by a lighting
retrofit project during the site visit in order to appropriately apply the RTF's factors. For desk
reviews without an accompanying site visit, the evaluation team assumed electric cooling with
gas heating in absence of better information.
8 http://rtf.nwcouncil.org/measures/measure .asp?id=213
'-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 43
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 57 of 212
4 NONRESIDENTIAL IMPACT EVALUATION
4.2.4 Findings and Recommendations
The data collected as a result of the desk reviews and onsite data measurement and verification
activities were utilized to estimate the gross verified savings. The evaluation team's gross
verified savings values for the sample of reviewed projects were very close to Avista's reported
values, resulting in realization rates near 100% for both measures. Individual project realization
rates varied both above and below 100% due to differences in operating hours, baseline and
retrofit fixture wattage, and application of interactive effects; these differences averaged out to
realization rates near 100%. Table 4-6 summarizes the findings of the realization rate for energy
benefits for each measure in the Prescriptive Lighting program.
Table 4-6: Prescriptive Lighting Realization Rate Results
M Sample Unique R r r R t Relative Precision easure Projects ea iza ion a e (90% Confidence)
Lighting (Exterior) 36 104% N/A Lighting (Interior) 32 97%
TOTAL 68 99% 13%
The baseline fixture types for the projects in the evaluated sample for Interior Lighting are
summarized in Table 4-7. Projects with multiple fixture types are counted multiple times. The
majority of evaluated projects were retrofits of incandescent and HID technologies. Linear
fluorescent participation was low, only 4 projects in the evaluation sample.
Table 4-7: Baseline Fixture Types for Prescriptive Lighting (Interior)
Baseline Fixture Type Project Count
TB
3
HID 11
Incandescent 21
Halogen 2
Sensor only project
1 Baseline fixture type may have been T12. Project
documentation does not specify. All T12s are analyzes
using an analogous TB baseline.
2Both Avista and the evaluation team estimated savings for
these projects using the analogous TB technology as the
baseline.
Table 4-8 shows the total gross verified savings for the Prescriptive Lighting program.
Table 4-8: Prescriptive Lighting Gross Verified Savings
P Reported Savings Energy Gross Verified
rogram (kWh) Realization Rate Savings (kWh)
Prescriptive Lighting 3,475,049 99% 3,432,B65
.,..,Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 44
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 58 of 212
4 NONRESIDENTIAL IMPACT EVALUATION
4.3 Prescriptive EnergySmart Grocer
4.3.1 Overview
The EnergySmart Grocer program, implemented by CLEAResult, offers a range of proven
energy-saving solutions for grocery stores and other customers with commercial refrigeration .
This program is intended to prompt the customer to increase the energy efficiency of their
refrigerated cases and related grocery equipment through direct financial incentives. Energy
savings are primarily achieved through installation of high efficiency case lighting and other
refrigeration system efficiency improvements. Some custom projects identified by CLEAResult
are also included in the EnergySmart Grocer program.
4.3.2 Program Achievements and Participation Summary
A total of 149 unique Prescriptive EnergySmart Grocer measures were installed at 68premises
in Idaho in 2014 and 2015. Table 4-9 and Figure 4-3 summarize Avista's 2014-2015
EnergySmart Grocer Program energy impacts by measure. Avista tracks all non-Case Lighting
measures as 'Industrial Process', both prescriptive and custom. Examples include ECMs in
display cases, floating head pressure controls, etc.
55%
e,1Nexanr
Table 4-9: EnergySmart Grocer Reported Energy Savings by Measure
Energy Savings . . Measure Type (kWh) % Electric Savings
Prescriptive Case Lighting 1,322,341 55%
Prescriptive Industrial Process 873,852 37%
Custom Industrial Process 191,470 8%
TOTAL 2,387,662 100%
Figure 4-3: EnergySmart Grocer Reported Energy Savings Shares
• Prescriptive Case Lighting
• Prescriptive Industrial Process
• Custom Industrial Process
8%
Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 45
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 59 of212
4 NONRESIDENTIAL IMPACT EVALUATION
4.3.3 Methodology
Engineering activities for the evaluation of this program included review of project
documentation, review of relevant RTF deemed savings values and workbooks, installation
verification, determination of operational hours, and savings calculations.
4.3.3.1 Sampling Approach
The evaluation team conducted document audits on 44 projects implemented through the
EnergySmart Grocer program. Surveys and onsite inspections were conducted for a sub­
sample of 20 of these projects (Table 4-10). Because of the installation of multiple projects at
some sites, the achieved sample size for onsite inspections and surveys was slightly higher
than the original sample design of 15 surveys and onsite inspections as noted in Table 3-2.
Table 4-10: EnergySmart Grocer Achieved Sample
P Document S On Site rogram . urvey . Aud rt lnspecbons
EnergySmart Grocer 44 20 20
4.3.3.2 Document Audits
Project documentation was requested for each sampled project, including invoices, savings
calculations, work order forms, equipment specification sheets, and any other project records
that may exist. Thorough review of this documentation was the first crucial step in evaluation of
each project.
4.3.3.3 Field Inspections
The telephone surveys conducted as part of the process evaluation were used to recruit
projects for onsite inspection verification. These onsite inspections provide a more rigorous way
to verify energy savings, and allowed the evaluation team to note any discrepancies between
onsite findings regarding actual measure and equipment performance and the information
gathered through the telephone surveys and project documentation review. A survey instrument
specific to this program was created in advance of the site inspections to ensure that the correct
information was gathered.
Table 4-11 summarizes the information that was collected for each project during the onsite
inspection. All parameters needed to support the savings analysis of a project were collected,
including fixture counts, baseline and post-retrofit wattages, hours of operation, and HVAC
system information to inform calculation of interactive effects.
t..1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 46
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 60 of 212
4 NONRESIDENTIAL IMPACT EVALUATION
Table 4-11: EnergySmart Grocer Onsite Data Collection
End Use Category Baseline Retrofit
All Facilities
Case Lighting
Industrial Process
Business Type
Operating Hours, posted or otherwise
Total conditioned square footage
Heating system type/age/efficiency/size/condition
Cooling system type/age/efficiency/size/condition
j Case Temperature
j Lamp Type (e.g., TB , T12)
! Ballast Type (mag. or elec.)
! Lamp Size (linear ft.)
I Quantity of Lamps per Fixture
Wattage per Lamp
Fixture Quantity
Operating Hours
Control Type
: Type of Equipment (e.g., open reach­
I in refrigerated case, closed freezer)
I Operating Temperatures
I Capacity
! Efficiency
I Operating Hours
I Other Parameters (e.g., motor kW or
! hp, linear feet of gaskets, thickness of
i suction line insulation)
!
i Case Temperature
I Lamp Type
[ Confirm Electronic Ballast and Factor
Lamp Size (linear ft.)
! Quantity of Lamps per Fixture
i Wattage per Lamp i Fixture Quantity
i Operating Hours
i Control Type
I Confirm ENERGY STAR© rating
Type of Equipment
Operating Temperatures
Capacity
i Efficiency
I Operating Hours
Other Parameters
4.3.3.4 Impact Analysis Methods
The evaluation team applied deemed energy savings values as published by the Regional
Technical Forum (RTF) where appropriate. Custom analyses were generated for measures not
listed with the RTF.
Active RTF-listed Measures
A majority of the measures installed under the EnergySmart Grocer program are active
measures with deemed energy savings values published by the RTF. For these measures, the
evaluation team reviewed the relevant RTF workbooks9 and the reported measure savings,
verifying eligibility and appropriate application of RTF savings values for each project in the
sample.
9 Grocery -Display Case LEDs (Open Cases) v1 .0, 1.1, 1.2, and 1.3. Grocery -Display Case LEDs (Reach-In Cases) v2.0, 2.2, 3.0,
3.1 , and 3.2. Grocery -ECMs for Display Cases v2.0, 2.1, 2.2, and 3.0. Grocery -ECMs for Walk-ins. V1 .1, 1.2, 2.0, and 2.1.
Grocery -Floating Heat Pressure Controls for Single Compressor Systems v1 .0, 1.1, 1.2, and 1.3. Available from
http://rtf.nwcouncil.org/measures/Default.asp.
'-"Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 47
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 61 of 212
4 NONRESIDENTIAL IMPACT EVALUATION
Non-RTF Measures
For measures not listed with the RTF, the evaluation team analyzed the energy savings using
custom project-specific methods.
4.3.4 Findings and Recommendations
The data collected as a result of the desk reviews and onsite measurement and verification
activities were utilized to estimate the gross verified energy savings for each sampled project.
The gross verified savings values for the sample of projects resulted in a realization rate of 90%
for the EnergySmart Grocer program (Table 4-12).
Table 4-12: EnergySmart Grocer Impact Energy Realization Rate Results
p Sample Unique Energy Relative Precision
rogram Projects Realization Rate (90% Confidence)
EnergySmart Grocer 44 90% 14%
In the following subsections, the evaluation team notes observed reasons for the gross verified
values for this program.
Application of RTF Deemed Savings Values
The RTF's deemed savings values for specific measures are periodically reviewed and updated
based on further research and input from RTF members. For each revision, the RTF publishes a
new workbook, and the current workbook as well as all prior versions are available on the RTF
website. In some cases, different deemed savings values were observed to be used in the
program tracking database for the same measure. The different deemed savings values appear
to have been taken from different versions of the RTF workbooks. The program implementer
appears to be updating its internal measure savings assumptions within the same program year.
Onsite Inspection Case Lighting Findings
The evaluation team found inconsistencies between onsite conditions and the applied RTF
deemed savings values in a few cases. Fewer linear feet of case lighting was noted in one
project of the 12 case lighting projects visited. In three cases, it was observed that projects
reported as occurring in low-temperature cases (i.e. freezers) were actually medium­
temperature cases (i.e. refrigerators). Lighting retrofits in medium-temperature cases result in
lower energy savings because there is less interactive effect with the case refrigeration system
due to the higher temperature. Overall, these finds play a relatively small role in the program
realization rate.
Custom Project Findings
Custom projects incentivized under this program have significantly larger reported savings on
average than the prescriptive projects. The reported energy savings for custom projects were
generally determined using eQuest energy simulation modeling. The evaluation team found
discrepancies in the energy model for one large project -a big box retail store with
t..1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 48
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 62 of 212
4 NONRESIDENTIAL IMPACT EVALUATION
overestimated sales floor lighting hours of operation. Because of the size of the project, this one
finding is a primary driver in reducing the program realization rate to 90%. The evaluation team
recommends tracking atypical custom projects such as this one through the Site Specific
program. This would allow such larger projects access to the QA/QC processes consistent with
the Site Specific program.
Table 4-13 presents the 2014-2015 gross verified savings for the EnergySmart Grocer program.
Table 4-13: EnergySmart Grocer Gross Verified Savings
P Reported Savings Energy Realization Gross Verified rogram . (kWh) Rate Savings (kWh)
EnergySmart Grocer 2,387,662 90% 2,138,035
4.4 Prescriptive Non-Lighting Other Programs
4.4.1 Overview
For evaluation purposes, the evaluation team analyzed several of Avista's smaller prescriptive
electric programs together under a "Prescriptive Non-Lighting Other" category. Table 4-14 lists
brief summaries of the programs included in this group. All are implemented internally by Avista
except Green Motors, which is implemented by the Green Motors Initiative.
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 49
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 63 of 212
-------------------------------------------------
4 NONRESIDENTIAL IMPACT EVALUATION
Table 4-14: Prescriptive Non-Lighting Other Program Summaries
Electric Programs Description
Food Service Equipment i This program offers incentives for commercial customers who purchase or replace
I food service equipment with Energy Star or higher equipment (prescriptive).
Green Motors
! The Green Motors Initiative is to organize, identify, educate, and promote member I motor service centers to commit to energy saving shop rewind practices, I continuous energy improvement and motor driven system efficiency.
i
HVAC Motor Controls
I This program is intended to prompt the customer to increase the energy efficiency I of their fan or pump applications with variable frequency drives through direct
Commercial Clothes
Washers
I financial incentives.
I This program encourages nonresidential customers to improve the efficiency of
I • • • I their clothes washing equipment.
Power Management for
PC Networks
I This program is designed to encourage implementation of power management
! software in networked PC's to obtain energy efficiency.
Commercial Windows &
Insulation
I This program encourages nonresidential customers to improve the envelope of I their building by adding insulation and replacing windows.
Commercial Water
Heaters
Fleet Heat
I
! This program encourages nonresidential customers to improve the efficiency of
! i their water heating equipment.
!
I Installation of technology that reduces standby losses of vehicle engine blocks by
! fleet operators by adding the ability to energize block heaters only when Outside
I Air Temperatu re drops below a temperature set-point and the engine mounted
! thermostat is calling for heat.
4.4.2 Program Achievements and Participation Study
A total of 62 unique measures were installed at 42 premises in Idaho through these
"Prescriptive Non-Lighting Other" programs in 2014 and 2015. Table 4-15 and Figure 4-4
summarize Avista's 2014-2015 reported energy impacts by measure for these programs in
Idaho.
Table 4-15: Prescriptive Non-Lighting Other Reported Energy Savings by Measure
P Energy Savings % Electric
rogram (kWh) Savings
Com Water Heater 190 0%
Com Windows and Insulation i 54,381 8%
Food Service Equipment 130,946 19%
Green Motors Rewind ! 43,954 6%
HVAC Motor Controls ! 466,340 66%
Standby Generator Block i 1,22a 1%
TOTAL i 103,039 100%
t.-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 50
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 64 of 212
4 NONRESIDENTIAL IMPACT EVALUATION
Figure 4-4: Prescriptive Non-Lighting Other Reported Energy Savings Shares
• Com Water Heater
• Com Windows and Insulation
• Food Service Equipment
66% • Green Motors Rewind
• HVAC Motor Controls
• Standby Generator Block
4.4.3 Methodology
Engineering activities for the evaluation of these projects varied by measure and included
review of project documentation, review of relevant RTF deemed savings values and
workbooks, installation verification, determination of operational hours, and savings calculations.
4.4.3.1 Sampling
The evaluation team conducted document audits for 24 projects that were grouped under the
"Prescriptive Non-Lighting Other" category. Surveys and onsite inspections were conducted for
a sub-sample of 15 of these projects (Table 4-16). Because of the installation of multiple
projects at some sites, the achieved sample size for onsite inspections and surveys was slightly
higher than the original sample design of 9 surveys and onsite inspections as noted in Table
3-2. The breakdown by program for the 24 document audits is provided in Table 4-17.
t..1Nexanr
Table 4-16: Prescriptive Non-Lighting Other Achieved Sample
Document OnSite Program A d"t Survey 1 • u I nspecbons
Prescriptive Non-Lighting Other ! 24 15 15
Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 51
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 65 of 212
4 NONRESIDENTIAL IMPACT EVALUATION
Table 4-17: Prescriptive Non-Lighting Other Achieved Sample by Program
Sample Measure 5. 1ze
Commercial Water Heaters 0
Commercial Windows and Insulation 17
Food Service Equipment 2
Green Motors Rewind
Motor Controls HVAC 4
Fleet Heat 0
4.4.3.2 Document Audits
Project documentation was requested for each sampled project, including invoices, savings
calculations, work order forms, equipment specification sheets, and any other project records
that may exist. Thorough review of this documentation was the first crucial step in evaluation of
each project.
4.4.3.3 Field Inspections
The telephone surveys conducted as part of the process evaluation were used to recruit a
sample for onsite inspection verification. These onsite inspections provide a more rigorous way
to verify energy savings, and allowed the evaluation team to note any discrepancies between
onsite findings regarding actual measure and equipment performance and the information
gathered through the telephone surveys and project documentation review. Because of the wide
variety of measures included in this evaluation, site-specific survey instruments were generated
in advance of each site inspections to ensure that sufficient information was gathered to support
the analysis of each measure.
Table 4-18 summarizes the types of information that were collected for each project during the
onsite inspection.
t.-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 52
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 66 of 212
4 NONRESIDENTIAL IMPACT EVALUATION
Table 4-18: Prescriptive Non-Lighting Other Onsite Data Collection
End Use Category Baseline Retrofit
All Facilities
HVAC
Motors
Building Envelope
Appliances
J Year of construction ' ' Business Type
I Number of occupants
! Number of floors
' : Operating Hours, posted or otherwise
I Total conditioned square footage
! Type (e.g., DX, heat pump)
I Age
' ! Heating & Cooling Capacity
! Efficiency
! Operating Hours
I Operating Temperatures (space, supply, I return, including info on setbacks)
i Control Capability / Strategy
i Other Features (e.g. economizer)
Motor size (hp)
Motor Efficiency
Age
Condition
Operating Hours
Insulation Type
' Insulation Thickness
I Type
J Age
! Capacity
: Efficiency
i Operating Hours
! Operating Temperatures
! Control Capability / Strategy
! Features
: Motor size (hp)
! Motor Efficiency
: Age
' Condition
! Operating Hours
I VFD Speed (current settings and load
I profile)
! Insulation Type
I Insulation Thickness
J Window Type (no. of panes, type of glass) i Window Type (no. of panes, type of glass)
! Affected Window/ Wall/ Attic Area (sq ft)
: Manufacturer
: Model Number
I Efficiency
Onsite data collection for HVAC Motor Control (Variable Frequency Drive or VFD) measures
included equipment inspection, interviews with site personnel, and collection of energy
management system (EMS) trend data if available. Topics covered in the interview included:
• Fan operation prior to the installation of the VFD including baseline fan control capability:
• On/Off
• Inlet Guide Vanes
• Discharge Damper
• Control programming associated with the VFD such as (1) facility operations schedule,
(2) temperature setpoints, (3) differential pressure control
t.--1 Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 53
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 67 of 212
4 NONRESIDENTIAL IMPACT EVALUATION
• Minimum and maximum observed operating speeds and associated facility and weather
conditions
• Typical operating speed
• Annual equipment operation schedule and variation on a daily, weekly, and annual basis
• After-hours usage in evenings
• Weekend usage
• Summer shut down
• Night setback
• Availability of trended VFD operating data via building EMS or other control system.
Field engineers gathered the following information from equipment nameplates or as-built
drawings:
• Motor make and model • Motor type
• Motor size (hp) • Fan type
• Motor efficiency • VFD make and model
• Motor speed (RPM)
Field engineers also collected operating parameters from the VFD drive's user interface control
panel (if present). To facilitate this data collection, the field engineers were provided with model­
specific guidance for accessing relevant parameters from the control panel. Although the
availability of these operating parameters varies between different VFDs, common operating
parameters collected include:
• Instantaneous operating parameters:
• Frequency (Hz)
• % speed
• Motor power (W)
• Motor amperage (A)
• Cumulative kWh and associated time interval
4.4.3.4 Impact Analysis Methods
Food Service Equipment
The Food Service Equipment projects included in the evaluation sample were for ENERGY
STAR-rated ice makers. The evaluation team evaluated the energy savings of each ice maker
using the Commercial Kitchen Equipment calculator published by ENERGY STAR10
10 https ://www.energystar.gov/sites/default/files/asset/document/commercial kitchen equipment calculator%2003-15-2016.xlsx
t.-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 54
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 68 of 212
4 NONRESIDENTIAL IMPACT EVALUATION
Green Motor Rewinds
The energy savings for Green Motor Rewind projects were evaluated using the deemed savings
values published by the RTF for this measure 11.
HVAC Motor Controls
The evaluation team assessed the HVAC Motor Control projects by modeling each affected
motor's input power based on motor size, efficiency, and performance curves published by
ASHRAE for various baseline motor control techniques (e.g. inlet guide vanes) as well as VFD
control. The general form of the algorithm used presented in Equation 4-4.
Where:
Equation 4-4: HVAC Motor Controls Energy Savings Calculation
100%
t.kWh = I [kWbaseline,cap -kWefficient,cap] X hourscap
cap=S%
Cap = operating capacity of the motor, ranging from 5% of full capacity to
100%
kWbaseline,cap = Baseline motor power consumption at a specific capacity, based on
ASHRAE performance curves for baseline motor control capability
kWefficient,cap = Post-retrofit motor power consumption at a specific capacity, based on
ASHRAE performance curve for VFDs
hourscap = Number of annual hours operating at each % capacity
Commercial Windows and Insulation
For measures affecting building envelope (attic insulation, wall insulation, and window
replacements), an industry-standard relationship for insulation improvements was applied.
Energy savings during the cooling season were calculated using the algorithm in Equation 4-5
Equation 4-5: Commercial Windows and Insulation Cooling Savings Calculation
Llk W hcooling = (R:re -
Where:
11 http://rtf.nwcouncil.org/measures/measure.asp?id=11 5
~) x Area x 24 x CDD post
1000 X 'Y/cool
'-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 55
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 69 of 212
4 NONRESIDENTIAL IMPACT EVALUATION
Rpre and post
Aattic
COD
'lcool
= Pre-and Post-improvement R-values of insulation or windows
= Affected area (sq ft).
= Annual cooling degree days
= Cooling system efficiency, EER or SEER
For buildings with electric heat sources, including both electric resistance furnaces and heat
pumps, the calculated savings during the heating season using the following algorithm
(Equation 4-6):
Equation 4-6: Commercial Windows and Insulation Heating Savings Calculation
f:ikWhh eating
( l ~) x Area X24 xHDD Rpre -post =--------------
rJheat X 3412
Where:
HOD = Annual cooling degree days
'lheat = Heating system efficiency
4.4.4 Findings and Recommendations
Table 4-19 presents the realization rate based on the gross verified savings values for the
sample of reviewed projects in the Prescriptive Non-Lighting Other category
Table 4-19: Prescriptive Non-Lighting Other Realization Rate Results
p /C t Sample Unique Energy Relative Precision
rogram a egory Projects Realization Rate (90% Confidence)
Prescriptive Non-Lighting Other 24 54% 228%
HVAC Motor Control Findings
The evaluation sample included four prescriptive HVAC Motor Control projects. Of these, a
project for two VFDs was found to have a 50% project-level realization rate because the two
VFDs were found to be serving a pair of motors operating in "Duty / Standby" configuration
where only one of the two operates at a time. A second project for a single VFD was found to be
installed in a non-typical VFD application (workshop dust collection system) and only being used
as a soft-starter, with the motor continuing to operate at 100% speed during occupied hours and
then switched off at night. Thus, this project was found to have zero energy savings. These
findings are the major drivers in the low stratum-level realization rate as well as the high relative
precision of 228% for this stratum. Without these two projects, the stratum's relative precision
t--'1 Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 56
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 70 of 212
4 NONRESIDENTIAL IMPACT EVALUATION
improves to 20% at the 90% confidence interval.
To improve the realization rate, Avista should consider adding additional review processes to
the program to check motor eligibility more stringently. More emphasis should be placed on
verifying each motor's application, confirming the VFD is controlling the speed of the motor in a
variable manner relative to load conditions, and checking that VFDs are not serving standby
motors.
Food Service Equipment Findings
The evaluation team did not find any significant discrepancies in the evaluated sample of Food
Service Equipment findings. Avista's reported energy savings are similar to what the evaluation
team calculated using the ENERGY STAR calculator.
Green Motor Rewind Findings
The evaluation team found that Avista is appropriately applying the deemed values published by
the RTF for Green Motor Rewind projects. No discrepancies were found.
Commercial Window and Insulation Findings
The algorithm the evaluation team utilized for verifying heating savings (both electric and gas)
resulting from window replacements is very similar to what is used by Avista. Both algorithms
estimate the effect of reduced thermal conduction loads on a building's heating system. For
cooling savings, the program utilizes an algorithm that estimates savings based on reduced
solar radiation loads. The evaluation team reviewed the SEEM model outputs included in the
RTF's workbook for Small Commercial Weatherization for Avista's service territory and
determined the program's radiation-based algorithm may be overstating savings. The evaluation
team opted to apply only the conduction-based algorithm, similar to the heating savings
algorithm, because the results aligned more closely with the SEEM values. Table 4-20
summarizes the program-reported and gross verified savings for window replacement cooling
season savings, compared with SEEM results for Heating Zones 1 and 2.
Table 4-20: Cooling Season Savings for Window Replacements
' Cooling Season
Savings (kWh/sqft)
Reported Savings 5.95
Gross Verified Savings 0.20
SEEM Results, Heating Zone 1 * -0.9-0.1
SEEM Results, Heating Zone 2* 0.02-0.68
·values from Small Commercial Weatherization Workbook: SmallCommWx_ProCost_ V2_0.xls
The evaluation team's algorithm resulted in very low realization rates for some projects, but the
average savings for this type of project is small on average, so the overall impact on the
program realization rate is minimal.
The evaluation team recommends that Avista consider alternate algorithms for the cooling
t-'1 Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 57
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 71 of 212
4 NONRESIDENTIAL IMPACT EVALUATION
season or investigate other ways to support the program's current algorithm using energy
modeling, billing analysis, or other third-party sources.
Table 4-21 shows the total gross verified savings for the programs evaluated under the
"Prescriptive Non-Lighting Other" stratum.
Table 4-21: Prescriptive Non-Lighting Other Gross Verified Savings
P Reported Savings R 1. . R Gross Verified rogram (kWh) ea 1zat1on ate 5 . avmgs (kWh)
Com Water Heater 190 103
Com Windows and Insulation 54,381 29,474
Food Service Equipment 130,946 70,971
54%
Green Motors Rewind 43,954 23,823
HVAC Motor Controls 466,340 252,751
Standby Generator Block 7,228 3,917
TOTAL 703,039 381,039
4.5 Site Specific
4.5.1 Overview
Avista's Site Specific program offers commercial customers the opportunity to propose any
energy efficiency project with documentable energy savings (kilowatt-hours and/or therms) for
an incentive. The majority of projects in this program are appliance upgrades, compressed air,
HVAC, industrial process, motors, shell measures, custom lighting projects, and natural gas
multifamily market transformation. The Site Specific program is implemented internally by
Avista , and program staff develop custom energy savings estimates for each project with input
from the customer. Projects must have a simple payback period between one and eight years
for lighting projects and between one and thirteen years for all other projects to be eligible for
incentive.
4.5.2 Program Achievements and Participation Summary
A total of 125 unique measures were installed through the Site Specific program at 102
premises in Idaho throughout 2014 and 2015. Table 4-22 and Figure 4-5 summarize Avista's
reported energy impacts by measure for the Site Specific program.
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 58
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 72 of212
4 NONRESIDENTIAL IMPACT EVALUATION
Table 4-22: Site Specific Reported Energy Savings by Measure
M T Energy Savings % Electric
easure ype (kWh) Savings
Appliances 8,237 0%
Compressed Air 369,035 6%
HVAC Combined 675,442 12%
HVAC Cooling 213,868 4%
HVAC Heating 5,557 0%
Industrial Process 354,318 6%
Lighting (Exterior) 957,055 16%
Lighting (Interior) 2,819,961 49%
Industrial Motor Controls 87,877 2%
Motors 5,351 0%
Multifamily 276,304 5%
Shell 40,605 1%
TOTAL 5,813,610 100%
Figure 4-5: Site Specific Reported Participation Energy Savings Shares
• Appliances
• Compressed Air
• HVAC Combined
• HVAC Cooling
HVAC Heating
• Industrial Process
• Lighting (Exterior)
• Lighting (Interior)
• Industrial Motor Controls
4.5.3 Methodology
• Motors
Multifamily
The impact evaluation for this program followed IPMVP guidance as well as the DOE Uniform
Method Protocol(s). The RTF's Non-Residential Lighting Retrofit Standard Protocol was
followed for lighting projects and IPMVP Option C was used to guide billing analysis for select
projects. Engineering activities included thorough review of the program savings methodology
for each project, installation verification, determination of operational hours including spot­
metering in some cases, collection of energy management system (EMS) trend data, and
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 59
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 73 of 212
4 NONRESIDENTIAL IMPACT EVALUATION
associated energy savings calculations.
• 4.5.3.1 Sampling
The evaluation team conducted 101 document audits on participating projects through the Site
Specific program. Customer surveys and onsite inspections were conducted on a subset of
these projects. Because of sample overlap with the Site Specific gas program, the achieved
sample size for document audits was higher than planned. Within the Site Specific program, the
evaluation team designated projects into two strata based on reported savings. Projects with a
reported savings over 275,000 kWh were designated as Large projects, with all others
designated as Small. This stratified sampling strategy was selected in order to ensure that the
relative impacts of large projects were fairly represented in the program-level results. Table 4-23
outlines the achieved sample for the Site Specific Program.
Table 4-23: Site Specific Achieved Sample
P St t Document 5 OnSite rogram ra a urvey Audtt Inspections
Large (> 275,000 kWh) 17 17 17
Small(< 275,000 kWh) 84 67 67
TOTAL 101 84 84
4.5.3.2 Document Audits
Project documentation was requested for each sampled project, including Avista's 'Top Sheets',
invoices, savings calculations, work order forms, equipment specification sheets, and any other
project records that may exist. The evaluation team's desk review process for Site Specific
projects included tracking the history of each project through the various stages of the program
as documented in the ''Top Sheets". Thorough review of this documentation was the first crucial
step in evaluation of each project.
For projects where Avista estimated savings using energy modeling software such as eQuest,
the evaluation team requested and reviewed the energy models.
4.5.3.3 Field Inspections
The telephone surveys conducted as part of the process evaluation were primarily used to
recruit a sample for onsite inspection verification. Some additional recruitment for this activity
was done by phone separate from the process telephone survey.
The onsite inspections provide a more rigorous way to verify energy savings, and allowed the
evaluation team to note any discrepancies between onsite findings regarding actual measure
and equipment performance and the information gathered through the telephone surveys and
project documentation review. Because of the wide variety of measures included in this
evaluation, project-specific survey instruments were generated in advance of each onsite
inspection to ensure that sufficient information was gathered to support the analysis of each
'-'"Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 60
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 7 4 of 212
4 NONRESIDENTIAL IMPACT EVALUATION
measure.
Table 4-18 summarizes the types of information that were collected for each project during the
onsite inspection. All parameters needed to support the savings analysis of a project were
collected .
Table 4-24: Site Specific Onsite Data Collection
End Use Category Baseline Retrofit
All Facilities
HVAC
Motors
Building Envelope
Appliances
! Year of construction ! I Business Type
! Number of occupants
! Number of floors
I Operating Hours, posted or otherwise
I Total conditioned square footage
Type (e.g., DX, heat pump)
Age
Heating & Cooling Capacity
Efficiency
Operating Hours
Operating Temperatures (space,
supply, return, including info on
setbacks)
Control Capability / Strategy
Other Features (e.g. economizer)
Motor size (hp)
Motor Efficiency
Age
Condition
Operating Hours
Insulation Type
Insulation Thickness
Window Type (no. of panes, type of
glass)
Type
Age
Capacity
Efficiency
Operating Hours
Operating Temperatures
Control Capability / Strategy
Features
Motor size (hp)
Motor Efficiency
Age
Condition
Operating Hours
VFD Speed (current settings and load
profile)
Insulation Type
Insulation Thickness
Window Type (no. of panes, type of
glass)
Affected Window/ Wall / Attic Area
(sq ft)
Manufacturer
Model Number
Efficiency
Onsite data collection for HVAC Motor Control (Variable Frequency Drive or VFD) measures
included equipment inspection, interviews with site personnel, and collection of energy
management system (EMS) trend data if available. Topics covered in the interview included:
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 61
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 75 of 212
4 NONRESIDENTIAL IMPACT EVALUATION
• Fan operation prior to the installation of the VFD including baseline fan control capability:
• On/Off
• Inlet Guide Vanes
• Discharge Damper
• Control programming associated with the VFD such as (1) facility operations schedule,
(2) temperature setpoints, (3) differential pressure control
• Minimum and maximum observed operating speeds and associated facility and weather
conditions
• Typical operating speed
• Annual equipment operation schedule and variation on a daily, weekly, and annual basis
• After-hours usage in evenings
• Weekend usage
• Summer shut down.
• Night setback
• Availability of trended VFD operating data via building EMS or other control system.
Field engineers gathered the following information from equipment nameplates or as-built
drawings:
• Motor make and model • Motor type
• Motor size (hp) • Fan type
• Motor efficiency • VFD make and model
• Motor speed (RPM)
Field engineers also collected operating parameters from the VFD drive's user interface control
panel (if present). To facilitate this data collection , the field engineers were provided with model­
specific guidance for accessing relevant parameters from the control panel. Although the
availability of these operating parameters varies between different VFDs, common operating
parameters collected include:
• Instantaneous operating parameters:
• Frequency (Hz)
• % speed
• Motor power (W)
• Motor amperage (A)
• Cumulative kWh and associated time interval
'-"Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 62
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 76 of 212
4 NONRESIDENTIAL IMPACT EVALUATION
4.5.3.4 Project-Specific Billing Analysis
The evaluation team reviewed utility bill histories for several projects where appropriate. To be a
good candidate for savings estimation using utility bill analysis approach, a project must provide
energy savings equal to at least 10% of the facility's annual consumption. Secondly, at least 9
months but preferably 12 months of post-project utility bill data must be available at the time of
the analysis. Thirdly, conditions at the facility should be relatively static, except for the project of
interest. The installation of other energy efficiency measures or other major changes at the
facility makes billing analysis inappropriate for project-specific savings estimation. If a project
was deemed to be a good candidate for utility bill analysis, then the evaluation team employed
IPMVP Option C to estimate energy savings, normalizing for monthly variation in weather
conditions.
4.5.3.5 Algorithm-Based Impact Analysis Methods
Because of the custom nature of the projects that participated in the Site Specific program, a
wide array of custom analysis methods were utilized and tailored to each individual project. In
many cases, if the evaluation team agreed with the program team's savings methodology, then
the evaluation team used the same methodology for the project evaluation, updating only the
input values and assumptions based on the results of onsite inspections or other data collection.
In some cases, the evaluation team used a different methodology, especially where billing data
or trend data allowed for savings to be calculated from measured data.
The evaluation team applied key algorithms for multiple projects, as described in the following
sections.
Lighting Projects
The evaluation team utilized the same approach for the lighting projects as described in the
methodology section for the Prescriptive Lighting Program (Section 4.2.3.4).
Variable Frequency Drives
Projects involving variable frequency drives (VFDs) were evaluated by modeling each affected
motor's input power based on motor size, efficiency, and performance curves published by
ASHRAE for various baseline motor control techniques (e.g. inlet guide vanes) as well as VFD
control. The general form of the algorithm used is shown in
Equation 4-7:
Where:
cap
t-1Nexanr
Equation 4-7: VFD Energy Savings Calculation
100%
.1kWh = L [kWbaseline,cap -kWefficient,cap] X hourscap
cap=So/o
= operating capacity of the motor, ranging from 5% of full capacity to
Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs63
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 77 of 212
4 NONRESIDENTIAL IMPACT EVALUATION
100%
kWbaseline,cap = Baseline motor power consumption at a specific capacity, based on
ASHRAE performance curves for baseline motor control capability
kWefflcient,cap = Post-retrofit motor power consumption at a specific capacity, based on
ASHRAE performance curve for VFDs
hourscap = Number of annual hours operating at each % capacity
HVAC Replacements
For HVAC projects various permutations of Equation 4-8 were utilized to calculate savings, as
applicable:
Equation 4-8: HVAC Replacement Energy Savings Calculation
~kWh= EFLH X kBtuH x ( 1 --1 -) IEERbase IEERee
Commercial Windows and Insulation
The evaluation team utilized the same approach for the commercial windows and insulation
projects as described in the methodology section for the Prescriptive Non-Lighting Other
Programs (Section 4.4.3.4)
4.5.4 Findings and Recommendations
The evaluation team found that the 2014-2015 Site Specific program achieved energy savings
very close to its reported performance, with a program-level realization rate of 99% (Table
4-25). Although individual project realization rates within the evaluation team's sample vary both
above and below 100%, the high overall average for the program of 99% reflects the high level
of review and scrutiny that Avista places on the projects that participate in the Site Specific
program.
Table 4-25: Site Specific Program Realization Rate Results
. . Energy Relative Precision
Strata Sample Unique ProJects R 1• • R (SO"' C f'd ) ea 1zat1on ate ,o on I ence
Large(> 275,000 kWh) 17 96% 5%
Small(< 275,000 kWh) 84 101% 12%
TOTAL 101 99% 7%
Measure-level realization rates for measures where more than one project was included in the
evaluation sample are presented in Table 4-26.
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs64
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 78 of 212
4 NONRESIDENTIAL IMPACT EVALUATION
Table 4-26: Site Specific Measure-Level Gross Verified Savings
M Sample Unique Energy Realization easure . ProJects Rate
Appliances 3 100%
HVAC Combined 31 95%
Industrial Process 4 87%
Lighting (Exterior) 15 102%
Lighting (Interior) 38 112%
Multifamily 3 86%
Shell 5 35%
Lighting Project Findings
The review of lighting projects in the evaluation sample for the Site Specific program showed
that Avista is generating high quality savings estimates for these projects, with measure-level
realization rates of 102% for Exterior Lighting and 112% for Interior Lighting. The primary factor
driving up the realization rate for Interior Lighting is the calculation of interactive effects. The
program uses a 7.7% interactive factor for air conditioned spaces with gas heat, the most
prevalent HVAC system type in the program, regardless of building type. The evaluation team
applied the interactive factors listed by the RTF, which range from 94% to 116% for that HVAC
system type (Appendix 8). However many of the evaluated projects were in building types at the
higher end of the RTF's range, such as Big Box Retail, Anchor Store Retail, and
College/University.
The baseline fixture types for the projects in the evaluated sample are summarized in Table
4-27. Projects with multiple fixture types are counted multiple times. The evaluation team
observed a distributed participation across several baseline fixture types in the sample.
Table 4-27: Baseline Fixture Types for Site Specific Interior Lighting
Baseline Fixture Type Project Count
TB 9
T12* 7
T5 5
HID 8
Incandescent 3
CFL
New construction
Sensor only project 9
*Both Avista and the evaluation team estimated savings for these projects
using the analogous TB technology as the baseline.
Window and Insulation Findings
As similarly described for prescriptive window replacements in Section 4.4.3.4, the algorithm
t.-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs65
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 79 of 212
4 NONRESIDENTIAL IMPACT EVALUATION
applied for cooling season savings is more conservative than what Avista is using. The program
utilizes an algorithm that estimates savings based on reduced solar radiation loads. The
evaluation team reviewed the SEEM model outputs included in the RTF's workbook for Small
Commercial Weatherization for Avista's service territory and determined the program's
radiation-based algorithm may be overstating savings. We opted to apply only a conduction­
based algorithm, similar to the heating savings algorithm, because the results aligned more
closely with the SEEM values. This difference of approach is the primary driver in the 35%
realization rate for Shell measures. However, since this measure makes up only 1 % of the total
program savings, the impact on the program realization rate is minimal.
Table 4-28 shows the total gross verified savings for the Site Specific program.
Table 4-28: Site Specific Gross Verified Savings
2014-2015 2014-2015 G
Program Reported Savings Realization Rate V ·t· d S . ro(sksWh) (kWh) eri 1e avmgs
Site Specific 5,813,610 99% 5,735,284
The high realization rate for this program indicates that Avista's internal process for project
review, savings estimation, and installation verification are working to produce high quality
estimates of project impacts. The evaluation team recommends that Avista continue to operate
this program with the current level of rigor.
4.6 Nonresidential Sector Results Summary
Table 4-29 lists the gross verified savings for each of Avista's nonresidential programs in Idaho
in 2014-2015. The Idaho electric nonresidential sector achieved a 94% realization rate and the
relative precision of the program-level electric realization rate was ± 7% at the 90% confidence
level.
Table 4-29: Nonresidential Program Gross Impact Evaluation Results
Idaho Electric Nonresidential 2014-2015 Reported R 1• • R 2014-2015 Verified . ea 1zat1on ate Program Savings (kWh) Gross Savings (kWh)
EnergySmart Grocer
Food Service Equipment
Green Motors
Motor Controls HVAC
Commercial Water Heaters
Prescriptive Lighting
Prescriptive Shell
Fleet Heat
Site Specific
NONRESIDENTIAL TOTAL
t.-1Nexanr
2,387,662 90% 2,138,035
130,946 54% 70,971
43,954 54% 23,823
466,340 54% 252,751
190 54% 103
3,475,049 99% 3,432,865
54,381 54% 29,474
7,228 54% 3,917
5,813,610 99% 5,735,284
12,379,360 94% 11,687,224
Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs66
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 80 of 212
5 Small Business Impact Evaluation
5.1 Overview
The Small Business (SB) program is a third-party-administered (SSW Consulting), direct
installation/audit program, providing customer energy efficiency opportunities by:
1) Directly installing appropriate energy-saving measures at each target site
2) Conducting a brief onsite audit to identify customer opportunities and interest in
existing Avista programs
3) Providing materials and contact information so that customers are able to follow up
with additional energy efficiency measures under existing programs.
Direct-install measures include:
• Faucet aerators • Smart power strips
• Showerheads • CoolerMisers
• Pre-rinse spray valves • VendingMisers
• Screw-in LEDs
The evaluation team conducted onsite verification, documentation audits, and engineering
analysis to determine verified gross savings for each measure in the program. Another key
objective for this evaluation was to develop new deemed savings values for faucet aerators and
pre-rinse spray valves based upon secondary research of statewide technical reference
manuals (TRMs) and published third-party data.
5.2 Program Achievements and Participation Summary
There were no Small Business participants in the Idaho service territory in 2015. The evaluation
team conducted impact evaluation activities on a sample of projects implemented in Avista's
Washington service territory. The findings from the evaluation activities conducted in
Washington are presented here and the evaluation team recommends that the findings be
considered for future program planning in Avista's Idaho service territory.
5.2.1.1 Sampling
The evaluation team selected a simple random sample of 31 projects for the impact evaluation
of the Small Business Program. Onsite verification was performed for all 31 sites. The 31
sampled project sites collectively accounted for a total of 191 electric and 46 natural gas saving
measures. Table 5-1 summarizes the achieved sample size.
L-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 67
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 81 of212
5 SMALL BUSINESS IMPACT EVALUATION
Table 5-1: Small Business Program Impact Evaluation Achieved Sample
On-Site . Program Verification Document Audit
Small Business 31 31
5.2.2 Document Audits
The evaluation team conducted a review of the project documentation for each sampled project,
including invoices, savings calculations, work order forms , equipment specification sheets, and
any other project records that may exist. Thorough review of this documentation was the first
crucial step in evaluation of each project.
5.2.3 Onsite Inspections
The impact evaluation activities included telephone surveys, documentation audits, and onsite
inspections for the entire sample. A telephone survey served as an introduction to the
evaluation activities and was used to confirm that the customer participated in the program,
confirm the appropriate contact, and to verify basic information such as building type and
building size. Arrangements for onsite inspections were then made during the telephone survey.
The onsite inspections were used to determine whether:
• The measure tracking database correctly represented the work that was done at each
site
• The measures remained installed and were operational
• There were any opportunities for measure installation that were missed
• There were assumptions embedded in the deemed savings estimates for each installed
measure (e.g. 3,000 lighting hours of use) applicable to the site.
Field engineers were equipped with a custom field data collection tool designed to capture the
relevant data points for each measure included in the SB program. Table 5-2 summarizes the
information that was collected for each measure type during the onsite inspection. All
parameters needed to support the savings analysis of a project were collected, including, but
not limited to, fixture counts, hours of operation, and water heater fuel type.
'-"Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 68
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 82 of 212
5
'
SMALL BUSINESS IMPACT EVALUATION
Table 5-2: Small Business Program Onsite Data Collection
Measure Type Key Parameters
All Facilities
Lighting
Faucet Aerators
Pre-rinse Sprayers
Showerheads
Vending Miser
Cooling Miser
Tier 1 Smart Power Strips
I Number of occupants
I Business Type
! Operating Hours, posted or otherwise
i Water Heater Type (Tank or Tankless)
! ! Water Heater Fuel Type (Natural Gas or Electric)
! Quantity of Lamps Installed
I Quantity of Lamps Decommissioned
I Lighting Hours of Use
I Pre-and Post-retrofit Lamp Wattage
I Quantity of Efficient Fixtures/Aerators Installed
Quantity of Efficient Fixtures/Aerators Decommissioned
Device Flow Rate
Water Heater Type
I Facility Hot Water Load
I Quantity Installed !
Quantity Decommissioned
1 Vending Machine Type
I Occupancy Hours
I Frequency of Use
I Quantity Installed
! I Quantity Decommissioned
! Connected Plug Loads
! Baseline Conditions
5.2.4 Impact Analysis Methods
The evaluation team estimated gross verified savings using the field verified quantities and the
program-specified deemed savings value for each measure. The deemed savings values used
by the program originate from a variety of sources including (UES) measures from the Regional
Technical Forum (RTF), California DEER database 12, and Puget Sound Energy 2014-2015 unit
energy savings values. Verified energy savings were generally calculated for each measure
using Equation 5-1 :
Where:
Equation 5-1: Small Business Program Energy Savings Calculation
.1kWh = Quantity Verified x kWh Saved/Unit
Quantity Verified = Quantity of devices/fixtures/lamps verified onsite
12 http://www.deeresources.com/
t.-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 69
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 83 of 212
5 SMALL BUSINESS IMPACT EVALUATION
kWh Saved = Program-stipulated electric energy (kWh) saved per unit installed
In addition to estimating program-level savings, the evaluation team also conducted a deemed
savings review for each direct-install measure offered by the Small Business Program. This
review process consisted of comparing deemed savings values used by Avista with those used
by similar programs in other jurisdictions and in other statewide TRMs. Recommended updates
to the deemed savings values were developed by the evaluation team for the faucet aerator and
pre-rinse spray valve measure offerings. The deemed savings assumptions used for the
remainder of the measures were deemed appropriate and therefore, were not modified in the
analysis. Additional details on the research conducted and measure-specific findings
determined for faucet aerators and pre-rinse spray valves are discussed in the Findings and
Recommendations section below.
5.3 Findings and Recommendations
The gross verified electric energy savings for the sample of reviewed projects for the Small
Business program resulted in a realization rate of 102% (Table 5-3).
Table 5-3: Small Business Program Realization Rate Summary
Electric : 5 1 d E Relative amp e nergy .. Measure Category i M R 1• • Prec1s1on (90%
1 easures ea 1zat1on C t·d ) I on I ence
1
Rate
Lighting 62 91%
Faucet Aerators 59 126%
Pre-rinse Sprayers 2 85%
Showerheads 0 100%
Vending Miser 9 100%
CoolerMiser 18 95%
Tier 1 Smart Power Strip 41 89%
TOTAL 191 102% 25%
5.3.1.1 Deemed Savings for Faucet Aerators
The evaluation team developed new electric (kWh) and natural gas (therms) deemed savings
values for both 0.5 GPM and 1.0 GPM faucet aerators installed through the program. The newly
developed values were applied on a per device installed basis. They were developed based
upon a comprehensive review of five statewide technical reference manuals 13, assumptions for
similar measures offered in other jurisdictions 14, and assumptions from applicable RTF UES
measures. During the research process, the evaluation team not only compiled the deemed
energy savings values used by each source, but also some of the underlying assumptions such
13 Statewide TRMs reviewed as part of our research included Massachusetts, Pennsylvania, Wisconsin, Minnesota, and Michigan.
14 Programs from other jurisdictions included the ComEd Small Business Energy Savings (SBES) Program and a program offered
by Questar Gas.
t,"1 NexanT Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 70
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 84 of212
5 SMALL BUSINESS IMPACT EVALUATION
as baseline and efficient device flow rates (GPM), frequency of use, hot water temperature, and
inlet water temperature. A summary of key findings and recommendations are provided in Table
5-4.
Table 5-4: Recommended Deemed Savings Values for Faucet Aerator Measures
A A A~ A~
B vg R dvg d Avg Gal Hot H20 Inlet H20 Deemed Deemed Measure ase e uce GPM GPM Reduced/yr Temp (°F) Temp (°F) kWh therms
Savings Savings
Faucet Aerator (1.0) 2.1 1.2 5,460 105 52 176 12
Faucet Aerator (0.5) 2.1 0.5 4,500 105 52 300 21
5.3.1.2 Deemed Savings for Pre-Rinse Spray Valves
The evaluation team also developed verified per-device energy savings estimates for pre-rinse
spray valves using the same approach and data sources described for faucet aerators. Key
findings from this research are provided in Table 5-5.
Table 5-5: Recommended Deemed Savings Values for Pre-Rinse Spray Valve Measures
A A Avg Avg
M Bvg Rvgd d Avg Gal Hot H20 Inlet H20 Deemed Deemed easure ase e uce GPM GPM Reduced/yr Temp (°F) Temp (°F) kWh therms
Pre-Rinse Sprayer
t-1Nexanr
Savings Savings
1.8 1.1 23,617 105 52 1,130 72
Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 71
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 85 of 212
6 Residential Impact Evaluation
The following sections outline the impact evaluation methodology and findings for each of the
evaluated residential programs and the low income program.
6.1 Overview
Avista offered seven electric incentive-based residential programs, one residential behavioral
program (Opower), and the low income program in their Idaho service territory in 2014 and
2015. The reported savings for these residential programs are summarized in Table 6-1 .
Table 6-1: Residential Program Reported Savings
. 2014-2015 Reported Idaho Electric Program S . (kWh avmgs )
Appliance Recycling 261 ,924
HVAC 872,828
Water Heat 239,267
ENERGY STAR Homes 140,538
Fuel Efficiency 5,290,679
Lighting 8,323,842
Shell 903,663
Opower (Home Energy Reports) 2,746,000
Low Income 758,955
TOTAL PORTFOLIO 19,537,696
The Lighting program contributes the largest share of the reported savings, 43% as shown in
Figure 6-1. Fuel Efficiency is the next largest contributor at 27%.
t.-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 72
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 86 of212
6 RESIDENTIAL IMPACT EVALUATION
5%
Figure 6-1: Residential Program Reported Energy Savings Shares
1% 4%
27%
43%
• Appliance Recycling
•HVAC
• Water Heat
• ENERGY STAR Homes
• Fuel Efficiency
• Lighting
•Shell
• Opower
• Low Income
The evaluation team designed a sampling strategy for these programs placing the most
emphasis on the programs with the highest projected savings and the highest level of
uncertainty. As part of the evaluation activities, a total of 259 document audits and 222
telephone surveys were conducted, and onsite inspections were conducted on 75 homes in
support of the Lighting Hours of Use study, as shown in Table 6-2. Engineering activities
included review of savings calculation methodology and assumptions, utility bill analysis and
energy savings analysis.
t-'1 Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 73
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 87 of 212
6 RESIDENTIAL IMPACT EVALUATION
Table 6-2: Residential Program Achieved Evaluation Sample
. . . Achieved Document Onsite Electric Res1dent1al Program C/P A d" Surveys . u rt lnspecbons
Residential Appliance Recycling N/A 70 72
HVAC Program 90/31 68 68
Water Heat Program 90/13 24 13
ENERGY STAR Homes 90/14 19 16
Fuel Efficiency 90/7 26 25
Residential Lighting Program 90/15.3 75
Shell Program 90/33 28 28
Opower Behavioral Program 90/8
Low Income 90/13 24
TOTAL 90/9 259 222 75
6.2 Residential Appliance Recycling
6.2.1 Overview
The appliance recycling program, administered by JACO Environmental Inc., provided a pick-up
and recycling service for operational refrigerators or freezers manufactured before 1995. The
pick-up service was free to customers and a $30 rebate was provided for each operational
refrigerator and/or freezer, up to two per household. JACO provided the following data points to
Avista on a monthly basis: date of pick-up, customer name, address, city state zip, type of unit
collected and number of units collected. The appliance recycling program ceased operation in
June 2015 as a result of revised RTF values that became effective in July of 2015 causing the
program to cease to be cost-effective.
6.2.2 Program Achievements and Participation Summary
The Appliance Recycling Program's reported participation and savings across the 2014-2015
program cycle is presented in Table 6-3.
t-1Nexanr
Table 6-3 Appliance Program Reported Participation and Savings
2014-2015 2014-2015
Measure Reported Reported
Participation Savings (kWh)
Refrigerator 309 198,150
Freezer 91 63,774
TOTAL 400 261,924
Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 74
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 88 of 212
6 RESIDENTIAL IMPACT EVALUATION
6.2.3 Methodology
The evaluation team conducted telephone surveys and document audits for 72 program
participants.
To record participation, Avista totals participation on a monthly basis from data provided directly
by the implementer, JACO. JACO also provided the evaluation team with a total database of all
units recycled in 2014 and 2015 under the Avista program. The evaluation team checked this
database for duplicates (zero found), and cleaned the database of refrigerators and freezers
collected that did not meet the program criteria of being manufactured before 1995 (125
records). The evaluation team then compared these results to Avista's reported values. The
final cleaned database reported 1,288 appliances recycled in WA over 2014 and 2015 (Table
6-4).
Table 6-4 Appliance Recycling Participation Counts
Avista Reported Implementer Adjusted
Measure P rt· . t· Reported Reported a 1c1pa ,on P . . . P . . . art1c1pat1on art1c1pat1on
Refrigerators 309 526 306
Freezers 91 157 92
TOTAL 400 683 398
Avista's deemed savings values reported per recycled freezer and refrigerator are based on
RTF unit energy savings which include the effects of freeridership. For purposes of estimating a
gross savings value for the measures, the evaluation team reviewed the findings from the 2012-
2013 Washington Impact Evaluation 15. The evaluation team then applied the gross verified
savings values reported in the prior evaluation study to the adjusted reported participation
values identified by the evaluation team . Table 6-5 outlines the Avista reported and evaluated
savings per unit for the Appliance Recycling program.
Table 6-5 Appliance Recycling Reported and Evaluated Savings
Avis~ Reported 2012-2013 Evaluated
Measure Savings Value 5 . (kWh/ "t) (kWh/unit) avmgs uni
Refrigerators 636 1,090
Freezers 612 902
6.2.4 Findings and Recommendations
While this program has been cancelled, there are a few findings that may assist Avista in
planning purposes should they implement a similar program in the future.
• The implementer JACO provided each customer with an OrderlD, and collects
datapoints for reporting to Avista including: name, account number, address, the type
15 Avista 2012-2013 Washington Electric Impact Evaluation Report, The Cadmus Group, Inc. May 15, 2014
t,1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 75
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 89 of 212
6 RESIDENTIAL IMPACT EVALUATION
unit recycled, make and model, as well as the year. Due to common place errors in
alternate spelling of names and addresses, it is important for the implementer to record
accurate account numbers. This will assist tracking of participants across programs and
tracking to billing data should that be necessary.
• The roll-up of Avista's reported appliance recycling values included only count of
appliance type per month, which is then applied to the deemed savings values to
estimate the reported program savings. This makes it difficult to determine where any
discrepancies may have occurred between the master implementer database and the
summarized Avista database. Maintaining as many variables as possible would allow for
improved error checking. For example, based on the fact that the JACO database total
counts and Avista reported total counts per appliance are different, it appears some
errors in data transfer may have occurred, and/or some appliances may have been
rebated by Avista that were manufactured after 1994. The cause of the discrepancy is
difficult to determine, however, with the variables reported in Avista's summary.
Table 6-6 outlines the Avista reported savings and the evaluation team's gross verified savings
based on the methodology described above. The program achieved a 165% realization rate
over the 2014 -2015 program cycle, as compared to the adjusted reported savings.
Refrigerator 306 198,150 194,616 171% 333,540
Freezer 92 63,774 56,304 147% 82,984
TOTAL 398 261,924 250,920 166% 416,524
6.3 HVAC Program
6.3.1 Overview
Avista internally manages the HVAC program which encourages the implementation of high
efficiency HVAC equipment and smart thermostats through direct incentives issued to the
customer after the measure has been installed. The evaluation team used a combination of
desk reviews, customer telephone surveys and billing analysis to estimate the gross-verified
savings for the applicable measures and the program as a whole.
6.3.2 Program Achievements and Participation Summary
Participation in the 2014-2015 HVAC program totaled 599 measures. Table 6-7 and
Figure 6-2 summarize Avista's 2014-2015 HVAC program participation and energy impacts.
L-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 76
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 90 of212
6 RESIDENTIAL IMPACT EVALUATION
Table 6-7: HVAC Program Reported Participation and Savings
Electric to Air Source Heat Pump 147 666,736
Smart Thermostat 15 12,493
Variable Speed Motor 437 193,599
TOTAL 599 872,828
Figure 6-2: 2014-2015 HVAC Program Reported Participation Energy Saving Shares
2%
6.3.3 Methodology
• Electric to Air Source Heat
Pump
• Smart Thermostat
• Variable Speed Motor
The evaluation team investigated measures under the residential HVAC program separately,
but utilized similar methods across multiple measures. The following four measure categories
were analyzed:
• Air Source Heat Pump (ASHP)
• Electric Variable Speed Motor
• Smart Thermostat
The evaluation team conducted 68 telephone surveys and document audits with program
participants and a billing analysis was conducted on all of the measures evaluated as well. As
discussed in Section 3.3, these surveys and document audits were conducted to confirm
participation in the program, confirm efficiency levels of installed equipment as applicable, check
that Avista reported data matched project files and that Avista is reporting the correct savings
t..1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 77
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 91 of 212
6 RESIDENTIAL IMPACT EVALUATION
value for each applicable measure. The evaluation team also conducted a review of Avista's
complete 2014 and 2015 program databases to check for errors in measure-level reporting.
The subsections below outline the specific evaluation methodology for estimating the gross
verified impacts for the ASHP, Electric Variable Speed Motor and the Smart Thermostat
measures. The methodology utilized for the natural gas furnaces is presented in the WA Natural
Gas Impact Evaluation Report16.
6.3.3.1 Air Source Heat Pump
To estimate electric savings resulting from participants' installation of air source heat pumps, the
evaluation team utilized the fixed-effects panel regression approach described in Section 3.4.4
Billing Analysis. Gross verified energy savings were calculated by comparing billed consumption
in months prior to the measure installations to the billed consumption in months after the
measure installations.
Utility billing data for participating homes were merged with observed temperature data (HDD
and COD) and program tracking data was used to identify the measure installation dates and
designate the pre-retrofit and post-retrofit periods for each customer. In order to estimate
impacts directly attributable to the heat pumps, the evaluation team isolated the customers who
received an air source heat pump and no additional measures. An indicator variable was
generated to designate billing periods that occurred prior to the measure installation (i.e. "pre"
period) and billing periods that occurred after the measure installation (i.e. "post" period). The
evaluation team required participants to have at least 12 months of "pre" billing data and at least
six months of "post" billing data to be included in the analysis. We then estimated fixed-effects
panel regression models to estimate the relationship between electric consumption and weather
during the "pre" and "post" retrofit periods. Equation 6-1 shows the model specification used to
estimate the relationship.
Equation 6-1: ASHP Fixed-Effects Panel Regression Model Specification
kWhit = ~i + ~1 X Postit + ~2 X HDDit + ~3 (Post X HDD)it + Eit
Table 6-8 provides additional information about the terms and coefficients in Equation 6-1 .
16 WA 2014-2015 Natural Gas Impact Evaluation Report -May 26, 2016
'-'1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 78
Exhibit No. 2
L. Roy, Avista
Schedule 1 , Page 92 of 212
6 RESIDENTIAL IMPACT EVALUATION
kWh;t
Postt
HDD;t
J3;
Table 6-8: ASHP Fixed-Effects Regression Model Definition of Terms
Variable Definition
I Estimated consumption in home i during period t (dependent variable)
J 1ndicator variable denoting pre-installation period vs. post-installation period
! Average heating degree days during period tat home i !
! Customer specific model intercept representing baseline consumption
! I Coefficients determined via regression describing impacts associated with independent
I variables
I Customer level random error
The 131 and 133 terms in Equation 6-1 describe the average change in daily base kWh and daily
kWh per HDD, respectively, in the post-retrofit period. The evaluation team applied these
coefficients to the TMY3 normal weather conditions to estimate weather normalized annual
electric savings resulting from ASHP installation.
6.3.3.2 Variable Speed Fan Motor
A similar approach was used to estimate electric savings associated with variable speed fan
motors. Similar to the ASHP analysis, the evaluation team first isolated the program participants
who received a new variable speed motor and no other measures in order to pinpoint the
savings directly attributable to the motors. Customers' utility billing data was merged with
historic weather records and the pre-installation and post-installation billing periods we
designated using the measure installation date from program tracking data. A fixed-effects panel
regression model was then estimated to develop the relationship between weather and electric
load before and after the variable speed fan improvement was installed.
The model specification used to estimate variable speed motor impacts is slightly different than
the model specification used for ASHP. Because the motor is active during both heating and
cooling seasons, CDD terms were included in the model specification in addition to the HDD
terms. Equation 6-2 shows the model specification used to estimate the impacts of variable
speed fan motors.
Equation 6-2: Variable Speed Motor Fixed-Effects Regression Model Specification
kWhit = l3i + 131 X Postit + 132 X CDDit + 133 (Post X CDD)it + 134 X HDDit + 135 (Post x HDD)it + Eit
Table 6-9 provides additional information about the terms and coefficients in Equation 6-2.
'-'"'Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 79
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 93 of212
6 RESIDENTIAL IMPACT EVALUATION
Table 6-9: Variable-Speed Motor Fixed-Effects Regression Model Definition of Terms
Variable Definition
kWh;t
Post1
CDD;t
HDD;t
~i
~1-5
Eit
Estimated consumption in home i during period t (dependent variable)
I Indicator variable denoting pre-installation period vs. post-installation period
i I Average cooling degree days during period t at home i
i I Average heating degree days during period tat home i
!
I Customer specific model intercept representing baseline consumption
I Coefficients determined via regression describing impacts associated with
I independent variables !
I Customer level random error
!
The 131, 133 and 135 terms in Equation 6-2 represent the average change in daily base load, daily
kWh per COD and daily kWh per HOD, respectively, in the post-installation period. These terms
were then applied to the normal weather conditions (TMY3) to estimate average weather
normalized annual savings associated with variable speed fan motors.
6.3.3.3 Smart Thermostat
Avista offers rebates for the installation of qualified smart thermostat products. These devices
have advance features such as occupancy detection, auxiliary heat lockout, economizer
capability, and "learning" algorithms to adapt to resident behavior. Avista claims savings based
on the heating fuel of the home so electric savings are only claimed for homes that have electric
heating systems. The majority of the smart thermostats rebated in 2014-2015 were in homes
with natural gas heating systems. The other challenge for evaluation was that uptake of the
smart thermostat offering was highest in the fourth quarter of 2015 . This meant that participating
only had a few months of post-installation billing data at the time of this evaluation. Further
complicating the analysis was the fact that a subset of the smart thermostat rebate recipient
also installed other HVAC measures such as variable speed fans and high efficiency furnaces at
the same time as the smart thermostat.
The evaluation team used propensity score matching to develop a comparison group of homes
from the Opower program to serve as a baseline for savings estimates. Only five homes had
sufficient post-retrofit billing data to estimate savings. The sample size wasn't sufficient to
develop a statistically significant per-home verified savings estimate, but two of the five homes
produced savings annual estimates below Avista's per-unit savings value of 961 kWh and three
of the five homes produces savings estimates above the reported savings value. Absent any
information supporting an adjustment of savings, the evaluation team set the gross verified
electric savings equal to reported savings for this measure.
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 80
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 94 of212
6 RESIDENTIAL IMPACT EVALUATION
6.3.4 Findings and Recommendations
6.3.4.1 Air Source Heat Pump
The findings from the telephone surveys, document audit and database review found that all
records matched between the Avista reported database and the project documentation.
Therefore, the reported savings and the adjusted-reported savings for program count and
savings match.
The fixed-effects regression analysis described in Section 6.3.3.1 produced statistically
significant reductions in heating loads in homes where air source heat pumps were installed and
rebated. Table C-1 in Appendix C shows the fixed-effects regression output for ASHP rebates.
Despite showing statistically significant heating impacts, the gross verified annual savings
estimated by the regression approach are well below the deemed savings reported by Avista
prior to the analysis. Whereas the average reported ex ante savings for ASHPs was 4,925 kWh,
the annual savings estimated by the analysis was 2,390 kWh, resulting in a 48.5% realization
rate. The relative precision of the savings estimate for ASHPs was ±19.0% at the 90%
confidence level (Table 6-10).
Table 6-10: Air Source Heat Pump Impact Summary
Precision at
H E A kWh Annual kWh Annual kWh O I RR 900, n omes x nte p p t e ta ,o
re os Confidence
109 4,925 20,574 18,183 2,390 48.5% ±19.0%
The evaluation team also ran individual customer regressions using the model specification
shown in Equation 6-1 in order to assess the distribution of savings at a more granular level
across the measure's participant population. The analysis resulted in an average 12. 7%
reduction in electric consumption in the "post" period as a result of ASHP installation. Figure 6-3
shows a histogram of the distribution of percent savings across the 109 participants receiving
ASHPs.
t.-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 81
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 95 of 212
6 RESIDENTIAL IMPACT EVALUATION
>, u C: Cl)
:::::J C"' Cl) .... LL
C)
<">
C)
N
C)
Figure 6-3: ASHP Distribution of Percent Savings
-.5 0 .5 Percent Savings
The evaluation team recommends Avista reexamine the assumptions relating to annual per­
home consumption and savings estimates in homes receiving ASHP installations.
6.3.4.2 Variable Speed Fan Motor
1
The findings from the telephone surveys, document audit and database review found a few
errors in the program database, resulting in a slight variance between the program reported and
adjusted reported values.
The regression approach produced statistically significant impact estimates in both the heating
and cooling loads of homes who installed a variable speed fan motor in their homes. Table C-2
in Appendix C provides the full regression output. In addition, annual savings estimated by the
regression were nearly at a level consistent with the deemed savings reported by Avista for the
program cycle. Table 6-11 summarizes the impacts and realization rate for variable speed fan
motor installations. On average, homes achieved 414 kWh annual savings compared to 439
kWh annual savings reported by Avista, resulting in a realization rate of 94.4%.
Table 6-11: Variable Speed Motor Impact Summary
H E A t kWh Annual kWh Annual kWh D It RR n omes x n e Pre Post e a
592 439 12,111 11 ,696 414 94.4%
The model specification shown in Equation 6-2 was also used to run separate regressions on
each individual customer receiving a variable speed motor. On average, customers receiving a
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 82
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 96 of 212
6 RESIDENTIAL IMPACT EVALUATION
variable speed motor installation achieved a 1.4% reduction in annual electric consumption.
Figure 6-4 shows the distribution of percent savings for program participants receiving a
variable speed motor rebate.
Figure 6-4 shows a histogram of the distribution of percent savings across the 592 participants
receiving variable speed fan motors.
Figure 6-4: Variable Speed Motor Distribution of Percent Savings
C)
LO
C)
>-~ 0 C: Ill :::s 0-~ u.
C) LO
-1
6.3.4.3 Smart Thermostat
-.5 0 Percent Savings .5 1
Given the inconclusive analysis results for this measure driven by data limitations, the
evaluation team recommends that Avista revisit the analysis of this measure in late 2016 or
early 2017, when a full year of post-installation billing data is available for several hundred
rebate recipients. Table 6-12 compares findings from smart thermostat impact evaluation across
the country. These studies vary in:
• Location (e.g. weather)
• Sample sizes
• Thermostat product installed and type of thermostat replaced
• Robustness of methodology
• Type of installation (utility direct install, professional, self-install).
The impact estimates of these studies also vary considerably. In general, programs that offer
direct replacement of manual thermostats have the highest savings estimates and mass market
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 83
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 97 of212
6 RESIDENTIAL IMPACT EVALUATION
offerings where the replaced thermostat population includes a mix of conventional
programmable and manual devices produce lower savings .
.,,.,Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 84
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 98 of 212
6 RESIDENTIAL IMPACT EVALUATION
Table 6-12: Comparison of Smart Thermostat Evaluation Results
QI
E ns z
>,
"O ::I ... U)
MA PAs: Wi-Fi
Programmable
Controllable Thermostat
Pilot Program Evaluation
(9/12)
National Grid: Evaluation
of 2013-2014 Smart
Thermostat Pilots: Home
Energy Monitoring,
Automatic Temperature
Control, Demand
Response (7 /15)
National Grid: Evaluation
of 2013-2014 Smart
Thermostat Pilots: Home
Energy Monitoring,
Automatic Temperature
Control, Demand
Response (7 /15)
NIPSCO: Evaluation of the
2013-2014 Programmable
and Smart Thermostat
Program (9/14)
NIPSCO: Evaluation of the
2013-2014 Programmable
and Smart Thermostat
Program (9/14)
Vectren: Evaluation of
2013-2014 Programmable
and Smart Thermostat
Program (1/14)
Vectren: Evaluation of
2013-2014 Programmable
and Smart Thermostat
Program (1/14)
Xcel: In-Home Smart
Device Pilot. Public
Service Company of
t..1Nexanr
QI -.I!! U)
MA
MA
MA
IN
IN
IN
IN
co
-ns -VI I-
QI :§
QI VI ns m
Manual &
Programmable
Manual
Programmable
Manual
Manual
Manual
Manual
Not specified
...
.I!! VI I-
"O ..l!:! -; ... VI .5
Ecobee Smart
Ecobee Smart
Ecobee Smart
Nest
Conventional
Programmable
Nest
Conventional
Programmable
Other Smart
or PCT
QI -~ U)
QI C.
E ns U)
66
(Gas)
11
(Elec)
9
(Gas),
15
(Elec)
26
(gas),
48
(elec)
238
>,
C: <
!::
C: ::I 0 ...
(!)
0 ... -C: 0 0
23
469
(Gas)
522
(Elec)
217 469
(Gas) j (Gas)
212 I 522
(Elec) (Elec)
197
(Gas)
191
(Elec)
184
(Gas)
205
(Elec)
2611
! (Gas) I 2114
I (Elec)
I
I(~;;)
1
2714
(Elec)
QI
VI ::I C') 0 C: J: 0 ..l!:! 0 0 0 .c. s: VI C')
VI C:
C') ·;;
C: ns ·;; IJ)
ns ~ U) ..
'if!.
16%
10%
3.9% 1 16%
3.9% 15%
4.0% 13.9%
3.7% ! 13.1%
1,100 N/A 3.3% 4.6%
C')
C: .:::; ns QI
J:
VI C')
C: ·;;
ns
U)
'if!.
8%
13.4%
8.0%
12.5%
5.0%
Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 85
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 99 of212
6 RESIDENTIAL IMPACT EVALUATION
a,
>, Ill s:: ::::1 C') C') < 0 :§ s:: --J: :;::: a, CV .l!! a, :: ~ 0 CV -.!::! 0 a, E Ill Ill ci I-I-(/) 0 () J: CV a, ::::1 ..s:: z -a, -c, a, 0 ~ Ill Ill .l!! .!: ~ ii ... C') C') >, (/) ai ;;; E C) Ill s:: s:: -c,
C') ·s: ·s: ::::1 Ill -CV 0 -Ill s:: CV CV (/) CV .!: (/) ... ·s: (/) (/) in -s::
0 CV ~ ~ (/) () ~ 0
Colorado (4/14)
PG&E: Findings from the conventional Opower/Honeywell Smart CA programmable & Other Smart 423 695 1.0% Thermostat Field or PCT
Assessment (7 /14) manual mix
Puget Sound Energy: I
2014 Impact Evaluation of I
PSE's Web-Enabled WA Not specified Other Smart 1,000 1,000 I
Thermostat (WET) or PCT 0.2% I
Program 8/15) I
Energy Trust of Oregon 75% I Nest Thermostat Heat OR programmable, Nest 185 211 NA 12.0% Pump Control Pilot 4.7% 1 !
Evaluation 25% manual
NV Energy 2013 DR 2477
(T) i Program NV not clear Eco-Factor 2,478 5.4% !
Evaluation+A27: R27 2478
(C)
ComEd Smart 2016 1791
most likely Smart (mostly (T) i Thermostat-Annual and IL 1,887 1.5% I 4.8% 6.7% blended Nest) 1887 Seasonal (C) I
6.3.5 Program Results
Table 6-13 outlines the program reported, adjusted, and gross verified savings value for each
measure in the HVAC program. The evaluation team found a 60% realization rate across the
entire HVAC program. The relative precision of the program level electric realization rate is
±30.5% at the 90% confidence level.
t-1Nexanr Impact Evaluation of Idaho 2014-2015 E~ergy Efficiency Programs 86
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 100 of 212
6 RESIDENTIAL IMPACT EVALUATION
Table 6-13: HVAC Program Gross Verified Savings
Electric to Air Source 147 666,736 Heat Pump
Smart Thermostat 15 12,493
Variable Speed Motor 437 193,599
TOTAL 599 872,828
6.4 Water Heat Program
6.4.1 Overview
666,736 49%
12,493 100%
193,599 94%
872,828 60%
326,196
12,493
182,677
521,365
The evaluation team's assessment of the Water Heat program included analysis and verification
of electric water heating-related measures offered by Avista including clothes washers, electric
water heaters, and low flow showerheads. Both clothes washers and showerhead incentives
were offered through the Simple Steps upstream program.
6.4.2 Program Achievements and Participation Summary
Participation in the 2014-2015 Water Heat program totaled 4,306 measures (includes distinct
measure and bulb counts). Table 6-14 and Figure 6-5 summarize Avista's 2014-2015 Water
Heat program participation and energy impacts.
Table 6-14: 2014-2015 Water Heat Reported Participation and Savings
E Electric Water Heater 19 2,090
Simple Steps Clothes washers 432 57,024
Simple Steps Showerheads* 3,855 180,153
TOTAL 4,306 239,267
*Inclusive of 1.5, 1.6, 1.75, and 2.0 gpm low flow showerheads and includes nonparticipant savings
,1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 87
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 101 of 212
6 RESIDENTIAL IMPACT EVALUATION
Figure 6-5: 2014-2015 Water Heat Program Reported Participation Energy Saving Shares
• E Electric Water Heater
• Simple Steps Clothes washers
• Simple Steps Showerheads
1%
6.4.3 Methodology
The evaluation team performed verification of the program measures through a review of
sampled project documentation and phone survey responses with program participants. Our
review was designed to confirm the program tracking database was aligned with both project
documentation and survey data.
Table 6-15 below presents the sampling completed for the Water Heat evaluation. The
evaluation team collected information on fuel types and baseline equipment data from
participant surveys and compared these data with project applications and supporting invoices.
The evaluation team used this information to assess if the data recorded in the program tracking
database was accurate.
Because we designed and drew our sample in 2014, clothes washers were not included in the
sample as this measure was not offered until 2015.
Table 6-15: Water Heat Program Achieved Sample
Strata Document Audit Phone Survey
Clothes Washers 0 0
Electric Water Heater 13 13
TOTAL 13 13
In addition to the participation verification activities described above, the evaluation team also
conducted an engineering analysis to estimate per unit savings for showerheads for each
efficiency level. The evaluation team estimated savings from low flow showerheads following
Equation 6-3 and the parameters and source for each identified in Table 6-16
t.-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 88
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 102 of 212
6 RESIDENTIAL IMPACT EVALUATION
Equation 6-3: Low Flow Showerhead Energy Savings Calculation
En ergy Savings (kWh/Year)
= People x Show er Ti.me x Days x %Day s x fl GPM X (TsHOWER -Tm) x Cp x Den
3 ,.413 x RE x Showerheads
Where:
People
Shower Time
Days
%Days
LJGPM
TSHOWER
TIN
CP
Den
3,413
RE
= the number of people taking showers (ppl/household)
= the average shower length (min/shower)
= the number of days per year (day/yr)
= the number of showers per day, per person (shower/day-pp!)
= the difference in gallons per minute for the base showerhead
and the new showerhead (gal/min)
= the average water temperature at the showerhead ( oF)
= the average inlet water temperature (oF)
= the specific water heat (BTU/lb-oF)
= the water density (lb/gal)
= the conversion rate between BTU and kWh
= the water heater's energy factor
Total# of Showerheads = the number of showerheads per home
High-Efficiency Showerheads = the number of high-efficiency showerheads installed by
the program
L-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 89
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 103 of 212
6 RESIDENTIAL IMPACT EVALUATION
Table 6-16: Low-Flow Showerhead Parameters and Data Sources
Term Value Source
People 2.51 ! U.S. 2010 Census
Shower Time 8.06 [ Regional Technical Form
Days 365 I Conversion Factor (day/yr)
%Days 0.68 I Regional Technical Form
~GPM I O 3 0 55 0 7 0 8 I Program data (efficient case); Regional
I · ' · ' · ' · : Technical Form (baseline case) ! ;
TSHOWER 105 ; 17 ! Secondary source
TIN 52 ; 18 ! Secondary source
EFelectric 100% [ Regional Technical Form
CP I Constant (BTU/lb-oF)
Den 8.33 [ Constant (lb/gal)
Number of Showerheads 1.91 j U.S. 2010 Census; Regional Technical Form
Because the showerheads were either distributed via an upstream or direct install program, the
evaluation team assumed an installation rate of 1.0.
Per unit savings were estimated based on these parameter inputs and extrapolated total
savings from showerheads based on the measure counts reported by the program
implementers. The Simple Steps database provided the overall number of showerheads sold
through the program in Idaho; however, no program data was available to determine the
proportion of showerheads installed in homes with electric water heating. In order to determine
the proportion of homes with electric water heating, the evaluation team leveraged data
collected through the 2011 Single Family Regional Building Stock Assessment 19. We used data
specific to Idaho to assign the proportion of Simple Steps showerheads that contributed to
electric savings.
Additionally, the Bonneville Power Authority (BPA) reported additional non-participant savings
from showerheads under the Simple Steps program. The evaluation team allocated these
additional savings based on the same assumed electric water heating saturation for Idaho. We
also assigned only a portion of these savings to Idaho as the BPA non-participant savings
represented both Avista's Washington and Idaho territories. The evaluation team based the
portion assigned to Idaho on Avista's Idaho residential customer base relative to its entire
customer base.
17 DeOreo, William, P. Mayer, L. Martien, M. Hayden, A. Funk, M. Kramer-Duffield, and R. Davis (201 1 ). "California
Single-Family Water Use
18 https://www3.epa.gov/ceampubl/learn2model/part-two/onsite/ex/jne henrys map.html
19 http://neea.org/docs/reports/res ide ntia 1-bui Id i ng-stock-a ssessment-si ng le-family-characteristics-and-energy-use. pdf?sfvrs n=8
t.-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 90
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 104 of 212
6 RESIDENTIAL IMPACT EVALUATION
6.4.4 Findings and Recommendations
Based on the review of sampled project documentation and phone survey data, the evaluation
team did not identify any errors or corrections needed to the program tracking database. The
evaluation team assessed and agreed with the savings value being reported for the Simple
Steps clothes washer and electric water heater measures. Therefore, these measures were
assigned a 100% realization rate.
The analysis conducted for the low flow showerheads, as described above, resulted in a
blended realization rate across the 2.0, 1.75, 1.6 and 1.50 GPM Simple Steps showerheads of
164%.
The main reasons for the large realization rate for the Simple Steps showerheads include:
• The per unit savings are lower than the evaluation team's calculated values most likely
due to a difference in some of the parameters discussed in Table 6-16 above.
• The evaluation team assumed that approximately 54% of the showerhead installations
savings are tied to an electric water heater, whereas Avista reports 50% toward electric
water heater savings.
The total program realization rate and savings are presented in Table 6-17. The relative
precision of the program level electric realization rate is ±13.4% at the 90% confidence level.
Table 6-17: Water Heat Program Gross Verified Savings
2014-2015 Gross 2014-2015 2014-2015 Adjusted Realization Verified Measure Participation Reported Reported Rate(%) Savings Count Savings (kWh) Savings (kWh) (kWh)
Electric Water Heater 19 2,090 2,090 100% 2,090
Simple Steps Clothes Washers 432 57,024 57,024 100% 57,024
Simple Steps Showerheads 3,855 180,153 180,153 164% 295,561
TOTAL 4,306 239,267 239,267 148% 354,675
6.5 ENERGY STAR® Homes
6.5.1 Overview
The ENERGY STAR® Homes program provides new home buyers with an $800 rebate for an
ENERGY STAR® ECO-rated new manufactured home or $1 ,000 for an ENERGY STAR® stick­
built home. The evaluation team conducted a document review and engineering analysis for a
sample of the participating homes and attempted to conduct a billing analysis to estimate gross
verified impacts for the program.
'-"1 Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 91
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 105 of 212
6 RESIDENTIAL IMPACT EVALUATION
6.5.2 Program Achievements and Participation Summary
Participation in the 2014-2015 ENERGY STAR® Homes program totaled 19 homes. Table 6-18
and Figure 6-6 summarize Avista's 2014 and 2015 ENERGY STAR® Homes program
participation and energy impacts.
Table 6-18: 2014-2015 ENERGY STAR® Homes Reported Participation and Savings
Energy Star Home -Manufactured, Furnace 16 109,552
Energy Star Home -Stick Built 3 30,986
TOTAL 19 140,538
Figure 6-6: 2014-2015 ENERGY STAR® Homes Program Reported Energy Saving Shares
78%
• E Energy Star Home -Manufactured, Furnace
22% • Energy Star Home -Stick Built
6.5.3 Methodology
The evaluation team initially attempted to use a difference-in-means approach to estimate
savings for the ENERGY STAR® Homes program. Utility billing data was used to compare
average weather normalized annual consumption of newly built ENERGY STAR® Homes to the
weather normalized annual consumption of non-program new meter hookups in Avista service
territory, allowing for an estimate of program-related savings. However, due to the small number
of ENERGY ST AR® Homes participants and absent any detailed characteristics of the homes
(e.g. square footage, single-vs. multi-family, etc.) a reliable non-program comparison group
could not be attained.
t-'1 Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 92
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 106 of212
6 RESIDENTIAL IMPACT EVALUATION
Instead, the evaluation team collected Home Energy Rating System (HERS) Index scores for
participating ENERGY STAR® Homes wherever available. A total of 19 HERS scores were
found, including four ENERGY STAR® Stick Built, WA homes and 15 ENERGY STAR Natural
Gas homes. A baseline HERS Index score of 80 was assumed as standard for non-program
new meter hookups, determined by the 2012 IECC HERS Index Score for climate zone 5.
The evaluation team estimated weather normalized annual consumption for ENERGY STAR®
Homes using the same basic model specification shown in Equation 3-1 and Equation 3-2.
Because these newly built homes do not have a pre-retrofit period, only "post-retrofit"
consumption was estimated by the model (in this case, the "retrofit" occurs upon completion of
the home or at the time of occupancy).
To estimate what the home's consumption would have been, absent the ENERGY STAR®
program, each home's weather normalized annual consumption estimates was scaled up by a
weighting factor calculated as the quotient of the base HERS Index score 80 and the home's
HERS Index score. Equation 6-4 shows the calculation of estimated consumption absent the
program. Note that Equation 6-4 denotes electric consumption for ENERGY STAR® Homes;
estimated natural gas consumption absent the program was calculated in exactly the same
manner, replacing therms for kWh in Equation 6-4 and Table 6-19 below.
Equation 6-4: Calculation of Consumption Absent Program
HERSsase kWhNP = kWhp X ----HERSHo me
Table 6-19 provides additional information about the terms in Equation 6-4.
Table 6-19: Calculation of Consumption Absent Program Definition of Terms
Variable Definition
kWhNP I Estimated electric energy consumption in home absent the program
kWhp I Weather normalized annual consumption of the home
HERSsase ! 2012 IECC HERS Index Score for climate zone 5 = 80
HERSHome HERS Index Score for the home
Estimated savings for the 15 ENERGY STAR Natural Gas Homes (therms) and four ENERGY
STAR® Stick Built, WA Homes (kWh) were calculated individually using each home's specific
HERS Index score and averaged for each cohort. HERS Index scores for the remaining
ENERGY STAR® Homes were not available, so the evaluation team applied the mean HERS
Index score from among the 19 ENERGY STAR® Homes with HERS Index scores and
estimated annual consumption absent the program in the same way for these homes, using
Equation 6-4.
t.-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 93
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 107 of 212
6 RESIDENTIAL IMPACT EVALUATION
6.5.4 Findings and Recommendations
The findings of the HERS Index score approach produced savings estimates exceeding the
deemed ex ante savings reported by Avista for the ENERGY STAR® Homes measures.
Realization rates were calculated at greater than 100% of reported savings across all measures.
While the results of the HERS Index score approach shows positive savings results, a billing
analysis approach with a non-program comparison group would have been the preferred
approach. For future evaluations, the evaluation team recommends that Avista track more
detailed characteristics of the ENERGY STAR® program homes and non-program homes to
allow for a reliable non-participant comparison group billing analysis approach.
Table 6-20 shows calculations for electric savings and realization rate for ENERGY STAR®
Stick Built homes in Idaho. Two of these homes did not have adequate billing data to produce
reliable weather normalized consumption estimates and consequently were dropped from the
analysis. Analysis on these homes estimated approximately 6,861 annual kWh used under
program conditions. The HERS Index weight of 1.7 estimated 11 ,694 kWh annually under non­
program conditions, resulting in 4,833 kWh estimated savings.
Table 6-20: ENERGY STAR Home: Results for Stick Built homes in Idaho
. Realization n Homes Ex Ante kWh Annual kWh Base kWh Delta kWh Weight Rate
2 4,734 6,861 11 ,694 4,833 1.7 102%
The evaluation team calculated an average HERS Index score for the 19 homes having
individual HERS Index scores. The average score of 49.3 was applied to the remaining subset
of ENERGY STAR®-Manufactured, Furnace homes that do not have individual HERS Index
scores. Annual consumption and realization rate for these homes are summarized in Table
6-21. Because of the small participation for the ENERGY ST AR® Manufactured, Heat Pump
homes (one home participated in 2014), the evaluation team applied the same realization to this
one participant.
Table 6-21: ENERGY STAR Home: Results for Furnaces in Manufactured Homes
. Realization n Homes Ex Ante kWh Annual kWh Base kWh Delta kWh Weight R ate
17 6,847 14,173 23,016 8,843 1.6 129%
6.5.5 Program Results
Table 6-22 outlines the program reported, adjusted, and gross verified savings value for each
measure in the ENERGY STAR® homes program. The evaluation team found a 123%
realization rate across the entire program. The relative precision of the program level electric
realization rate is ±14.4% at the 90% confidence level.
t.-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 94
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 108 of 212
6 RESIDENTIAL IMPACT EVALUATION
Table 6-22: ENERGY STAR® Homes Program Gross Verified Savings
2014-2015 2014-2015 2014-2015 Gross
Reported Reported Adjusted Realization Verified Measure Participation Savings Reported Rate Savings
Count (kWh) Savings (kWh) (kWh)
Energy Star
Home: Manufactured, Furnace 16 109,552 109,552 129% 141 ,485
Energy Star Home: Stick Built 3 30,986 30,986 102% 31 ,635
TOTAL 19 140,538 140,538 123% 173,120
6.6 Fuel Efficiency
6.6.1 Overview
The fuel efficiency program offers a rebate for the conversion of electric straight resistance heat
to natural gas, as well as the conversion of electric hot water heaters to natural gas models. The
evaluation team conducted a document review, database review, telephone surveys, and a
billing analysis on a sample of the population in order to estimate the gross verified savings for
the program.
6.6.2 Program Achievements and Participation Summary
Participation in the 2014-2015 Fuel Efficiency program totaled 405 conversions. Table 6-23 and
Figure 6-7 summarize Avista's 2014-2015 Fuel Efficiency program participation and energy
impacts.
Table 6-23: 2014-2015 Fuel Efficiency Reported Participation and Savings
M 2014-2015 Reported 2014-2015 Reported
easure P . . . C S · (kWh) art1c1pat1on ount avmgs
Electric to Natural Gas Furnace & Water Heater 170 2,720,510
Electric to Natural Gas Furnace 202 2,426,424
Electric to Natural Gas Water Heater 30 110,949
Electric to Natural Gas Wall Heater 3 32,796
TOTAL 405 5,290,679
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 95
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 109 of 212
6 RESIDENTIAL IMPACT EVALUATION
Figure 6-7: 2014-2015 Fuel Efficiency Program Reported Energy Saving Shares
51%
• Electric to Natural Gas Furnace & Water Heater
• Electric to Natural Gas Furnace
• Electric to Natural Gas Water Heater
• Electric to Natural Gas Wall Heater
46%
6.6.3 Methodology
The Fuel Efficiency program is a dynamic offering because participants modify the fuel source
used for space heating and/or water heating within their residences. These measures produce a
large reduction in electric consumption, which is offset to some extent by increased
consumption of natural gas. The evaluation team examined both the electric savings and
associated gas penalty using an Option C regression analysis of billing data provided by Avista.
There are two key factors that affect gas penalty analysis -the first simplifies matters, while the
second complicates the analysis and accounting of the gas penalty.
1) Over half of homes that received Fuel Efficiency rebates did not have natural gas
service with Avista prior to participation20. This means the gas furnace or water heater
was installed shortly after gas service was added to the residence. It also makes the
gas usage in the home pre-retrofit intuitive-zero therms per year.
2) Approximately 49% of homes that received fuel efficiency incentives from Avista also
received rebates for the installation of a high efficiency furnace or water heater. For
these homes the observed increase in gas consumption actually overstates the
appropriate gas penalty because the gas meter records the consumption of the rebated
efficient appliance rather than the code minimum furnace or water heater required of the
homeowner to receive a Fuel Efficiency rebate. The difference in consumption between
the code minimum appliance that was not installed and high efficiency appliance that
was installed are credited as savings in the Gas HVAC and Gas Water Heating
programs. This was not the case for Idaho participants because there were no gas
20 The evaluation team used homes with two of fewer months of gas billing history and more than two months of electric billing
history as a proxy for the absence of gas service.
L-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 96
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 11 O of 212
6 RESIDENTIAL IMPACT EVALUATION
program offerings in Idaho in 2014-2015. However, since this factor affected the overall
analysis, it is noted herein.
The evaluation team requested monthly consumption records for each account that received a
Fuel Efficiency rebate (both Washington and Idaho) from Avista in 2014 and 2015. Billing
records were requested for January 2013 through February 2016 to maximize the quantity of
pre-and post-retrofit data available. The team excluded accounts where the meter number
changed during the period as this indicates the customer had moved and the consumption data
was from two different physical residences . Figure 6-8 provides of breakdown of the remaining
901 homes that received Fuel Efficiency rebates.
'-'"Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 97
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 111 of 212
6 RESIDENTIAL IMPACT EVALUATION
Figure 6-8: Diagram of Fuel Switching Participation
Furnace Conversion
Only (n=398)
No Rebate for High
Efficiency Gas
Equipment (n=217)
High Efficiency
Furnace Rebate
(n=181)
No Rebate for High
Efficiency Gas
Equipment (n=178)
High Efficiency
Furnace Rebate
(n=llS)
High Efficiency WH
Rebate (n=l)
High Efficiency
Furnace & WH
Rebate (n=59)
No Rebate for High
Efficiency Gas
Equipment (n=69)
High Efficiency WH
Rebate (n=18)
High Efficiency
Furnace Rebate
(n=29)
High Efficiency
Furnace & WH
Rebate (n=34)
The complexities around secondary rebates for installation of high efficiency rebates were not a
major concern for the electric savings analysis because the high efficiency water heater and
furnace don't significantly affect the electric usage of the home. The evaluation team did
exclude any homes that participated in the Shell rebate program in order to isolate the electric
savings from Fuel Efficiency as much as possible. A small number of homes that converted
from electric heat to natural gas furnaces also received rebates for installation of a variable
speed electric furnace fan , but because the expected fan savings were minimal when compared
to the fuel conversion the evaluation team elected not to exclude them from the analysis.
t..-1 Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 98
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 112 of 212
6 RESIDENTIAL IMPACT EVALUATION
The evaluation team estimated three separate electric regression models, one for each of the
conversion types shown in Figure 6-8. The general form of the electric regression model is
shown in Section 3.4.4 of this report and the detailed regression output is presented in Appendix
C. In order to maximize the number of homes analyzed the evaluation team relaxed the required
number of months for inclusion in the analysis. Homes with at least nine months of pre-retrofit
electric billing history and six months of post-retrofit billing history were included in the electric
analysis.
Figure 6-9 presents a simplified example of the utility bill regression analysis used to estimate
electric savings following receipt of Fuel Efficiency rebates. This example uses a single
customer and relies on only heating degree days (HOD) to explain the variation in monthly
electric usage. During pre-retrofit period electric consumption rises sharply as weather
conditions get colder. In the post-retrofit period the slope of the line is still positive, likely due to
increased use of the furnace fan or lighting within the home during cold winter months, but the
relationship is much less dramatic than the pre-retrofit period . When the slopes of these lines
are applied to an identical expected number of annual HOD, the difference in expected kWh is
interpreted as savings attributable to the program. The evaluation team's regression analysis to
estimate gross verified savings utilized many homes and also incorporated cooling degree days
(COD) as an independent variable, but the underlying principle is the same.
'-'1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 99
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 113 of 212
6 RESIDENTIAL IMPACT EVALUATION
Figure 6-9: Fuel Efficiency Regression Analysis, Example Home
0 0 0
M
0 0 IO • N.
0 0 0
~N-
~o • 0 IO
•
0 0
t-.;: t-• 0 • t-.....
~ t-t-t-t-t-t-t-
0 t-
0 IO
0 500 1000 1500
Monthly HOD
• Pre-Retrofit Fitted (Pre)
t-Post-Retrofit Fitted (Post)
The same process was repeated for homes that converted both furnace and water heater.
Almost all of the homes that converted only the water heating type had previous gas service so
the penalty for that group was determined using a pre\post analysis of gas consumption in those
homes.
In addition, the evaluation team performed verification of the program tracking database and
conducted 26 document audits and telephone surveys with customers who participated in the
program.
6.6.4 Findings and Recommendations
During the document audit and program database review, the evaluation team did find a few
reporting errors, which are reflected in the "adjusted reported" savings value found in the
Program Results section below.
Table 6-24 provides detail on the electric billing analyses for the three different fuel conversion
paths incented by Avista.
t-'1 Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 100
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 114 of 212
6 RESIDENTIAL IMPACT EVALUATION
Table 6-24: Fuel Efficiency Electric Billing Analysis Summary Statistics
Rebate Type Water Heater Furnace Furnace & Water Heater
Number of Homes Analyzed 71 173 102
Average Reported kWh 3,864 12,168 16,211
Average Annual kWh Pre 13,403 19,623 19,355
Average Annual kWh Post 9,647 12,100 10,083
Average Weather Normalized 3,756 7,524 9,272 Annual kWh Savings per Home
ELECTRIC REALIZATION RATE 97% 62% 57%
The "Water Heater" column in Table 6-24 includes both tank and wall heaters. These homes
used significantly less electricity prior to the conversion than the homes who converted heating
systems-likely because a majority of the homes already used fossil fuel heating systems. The
regression coefficients in Table C-8 in Appendix C show an expected pattern of savings. The
coefficients for the change in heating and cooling loads within the homes are small and not
statistically significant. However the coefficient representing the change in daily baseload
(1.treatment) is highly significant and estimates an 8.5 kWh per day reduction in non-weather
dependent electric load.
The homes that converted heating fuel from electricity to natural gas showed similarly large
weather-normalized annual electric pre-retrofit. The furnace-only homes used 19,623 kWh, on
average, and the furnace-and-water heater homes used 19,355 kWh annually. The realization
rates for the two groups were similar, with the group that converted both systems showing a
lower realization rate than the groups that converted just one system.
Appendix C contains the full regression output for these two fuel conversion groups, but the
evaluation team also estimated a combined model using both the furnace and furnace-and­
water heater homes. The regression coefficients from this analysis are presented in Table 6-25.
Table 6-25: Regression Coefficients from Combined Furnace Conversion Model
Model Term Coefficient Lower Bound of 90% Cl Upper Bound of 90% Cl
Intercept 14.69 12.59 16.79
Treatment 8.48 6.65 10.31
hdd_ave 2.01 1.89 2.13
treatment*hdd_ave -1.63 -1 .75 -1.51
cdd_ave 2.57 2.33 2.81
Treatment*cdd_ave -1 .16 -1 .37 -0.95
As expected, this model estimates a dramatic reduction in the electric heating consumption of
homes who replaced their electric heating system with a natural gas furnace. On average
homes go from using 2.01 kWh per HOD (base 65 F) to 0.38 kWh per HOD. Interestingly, the
4.-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 101
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 115 of 212
6 RESIDENTIAL IMPACT EVALUATION
model also estimates a reduction in cooling usage of 1.16 kWh per COD from 2.57 to 1.41.
Another noteworthy result in Table 6-25 is the estimated increase in base load from 14.69 kWh
per day to 23.17 kWh per day. This 3,000 kWh annual increase could be an artifact of the model
fit statistics, either because of small sample size or the 65 (F) degree day base is not accurately
disaggregating loads within all homes. However, another possibility is that participating homes
are undergoing some other fundamental change at the same time as the fuel conversion. Major
home improvement projects such as a home addition or finishing a basement, or a change in
occupancy within the home could drastically alter the consumption patterns within a home. The
evaluation team recommends Avista consider asking participants to indicate on their rebate
application if major home renovations are being completed in parallel with the heating system
fuel conversion. We believe excluding any such homes from future billing analysis would be
justified and limit the possibility of home improvement projects confounding the electric savings
estimates from Fuel Efficiency rebates.
6.6.5 Program Results
The electric realization rate for the Fuel Efficiency program was 60%. This program level
realization rate was developed by taking a weighted average of the realization rates of the Fuel
Efficiency rebate types shown in Table 6-26. The relative precision of the program level electric
realization rate was ±6.9% at the 90% confidence level.
Table 6-26: Fuel Efficiency Program Reported and Gross Verified Savings
2014-2015 2014-2015 2014-2015 Gross
Reported Reported Adjusted Realization Verified Measure Participation Savings Reported Rate Savings
Count (kWh) Savings (kWh) (kWh)
Electric to Natural Gas Furnace & WH i 170 2,720,510 2,725,610 57% 1,558,909
Electric to Natural Gas Furnace 202 2,426,424 2,426,424 62% 1,500,287
Electric to Natural Gas Water Heater 30 110,949 110,949 97% 107,825
Electric to Natural Gas Wall Heater 3 32,796 32,796 97% 31 ,873
TOTAL 405 5,290,679 5,295,779 60% 3,198,893
6.7 Residential Lighting Program
6. 7 .1 Overview
In 2014 and 2015, the Avista residential lighting program was comprised of two delivery
streams: Simple Steps, and the Avista Bulb Giveaway.
The Simple Steps, Smart Savings TM program provides discounts to manufacturers to lower the
price of efficient light bulbs, light fixtures, showerheads, and appliances. This program, launched
by Bonneville Power Administration (BPA) and administered by CLEAResult, operates across
c.-'1 Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 102
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 116 of 212
6 RESIDENTIAL IMPACT EVALUATION
the Pacific Northwest. Utilities may select which reduced-price items to include in their territory.
Avista's offerings included a selection of general and special CFLs, LED light fixtures, and LED
bulbs that were clearly identified with a sticker indicating they were part of the Simple Steps,
Smart Savings program. Retailers-big-box stores, regional chains, and national chains-were
the primary recipients of the products and typically selected from Avista's approved options for
each store location. Additionally, Simple Steps program provided Avista with an allocation of
additional residential lighting savings from non-participating utilities; this subprogram is called
"Simple Steps -NP". Finally, Avista gave its customers free, energy-efficient lighting products,
specifically CFL and LED lamps, at corporate and regional events.
6.7.2 Program Achievements and Participation Summary
Table 6-27 and
Figure 6-7 summarize Avista's 2014 and 2015 residential lighting program participation and
energy impacts.
Table 6-27: 2014-2015 Residential Lighting Reported Participation and Savings
2014:-~01 ~ Reported 2014-2015 Re orted
Measure Part1c1pat1on Count S . (kWP h)
(Bulbs) avmgs
Simple Steps-LED 89,124 1,846,600
Simple Steps-CFL 372,227 6,371,184
Simple Steps -NP-LED 420 6,376
Simple Steps -NP-CFL 4,645 70,970
Giveaway-CFL 1,824 27,360
Giveaway-LED 379 1,352
TOTAL 468,619 8,323,842
t.-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 103
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 117 of 212
6 RESIDENTIAL IMPACT EVALUATION
Figure 6-1 O: Distribution of Lighting Energy Savings by Technology Type
• LED
• CFL
Reported energy savings are based on a per-lamp basis, using a deemed value for each lamp
product type and delivery approach (i.e. retail, direct installation, giveaway) based on legacy
regional technical forum values.
6. 7 .3 Methodology
The lighting program gross impact analysis involved three distinct program components,
although each component ultimately depended on the same calculation and parameters to
estimate gross impacts. The underlying values for the input parameters were the only
differentiation across program components. Therefore, to simplify the approach and
methodology for the program, the evaluation team included a review of each of the key
parameters associated with energy savings. The team relied on savings protocols as specified
in the DOE-UMP. The UMP includes a full chapter on residential lighting evaluation protocols.21
The annual kWh savings for the lighting program are dependent on several key parameters.
The annual energy savings produced when a CFL or LED bulb replaces an incandescent bulb is
calculated as shown in Equation 6-5 :
Equation 6-5: Calculation of Consumption Absent Program
Annual kWh Savings= Total bulbs X .1Watts x 365.25 x HOUvaily x JSR x IE
Where:
21 Residential Lighting Chapter (21) in the UMP: http://energy.gov/sites/prod/files/2013/11/f5/53827-6.pdf. [
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 104
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 118 of 212
6
Annual kWh Savings =
Total bulbs =
~Watts =
HOUoaily =
365.25 =
JSR =
IE =
RESIDENTIAL IMPACT EVALUATION
The average annual energy savings from replacing the
incandescent bulb with a more efficient bulb
The total number of verified program incentivized bulbs
The change in connected load (baseline minus efficient wattage)
The average operating hours per day the light is turned on
Average number of days per year (to annualize daily HOU)
The in-service rate
The interactive effects (loss of inefficient bulb waste heat).
Table 6-28 shows each of the key parameters and the inputs for each parameter for the gross
savings analysis. More detail about the data sources/collection activities and parameter
estimates is presented in the remainder of this section.
Table 6-28: Lighting Program Parameters and Sources
Parameter CFL Retail LED Retail CFL Giveaway
Number of Bulbs Tracking Database Tracking Database Tracking Database
Hours of Use 2015 Light Metering 2015 Light Metering 2015 Light Metering
Study-Evaluation Study-Evaluation Study-Evaluation
Delta Watts Tracking Database, Tracking Database, Participant Survey EISA Mapping EISA Mapping
In-Service Rate Regional Technical Regional Technical Regional Technical
Forum; UMP Forum Forum; UMP
Cross Sector Leakage Retailer Interviews Retailer Interviews Not applicable
Interactive Effects Regional Technical Regional Technical Regional Technical
Forum Forum Forum
6.7.3.1 Total Program Bulbs
The evaluation team verified the number of CFL and LED lamps, product type, location, and the
bulb wattage distributed via the Simple Steps program via a database review for the State of
Idaho. For internal reporting, Avista uses a 70%/30% split to separate the total Simple Steps
units between its Washington and Idaho service territories, respectively. During the review of
the program database, the evaluation team found that 28.2% of the total units were actually in
the Idaho service territory. Because of this 0.2% difference between A vista's internal reporting
method and the numbers in the database, a slight difference appears between the total units
shown in Table 6-27 and in Table 6-29. The actual lamp unit counts in Table 6-29 were used in
the evaluation analysis.
'-"'Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 105
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 119 of 212
6 RESIDENTIAL IMPACT EVALUATION
Table 6-29: Verified Residential Lighting Unit Counts by Lamp Type and Delivery Stream
Program Delivery Stream Lamp Type Unit Counts
Simple Steps
Simple Steps -NP
Giveaway
TOTAL
t-1Nexanr
CFL General Purpose 342,094
CFL Specialty: Reflector 41,637
CFL Specialty: Globe 733
CFL Specialty: Candelabra 737
CFL Specialty: 3-way 648
CFL Fixture 2,104
CFL Subtotal 387,953
LED General Purpose 61 ,803
LED Specialty: Reflector 1,733
LED Specialty: Globe 56
LED Specialty: Candelabra 506
LED Specialty: 3-way 186
LED Fixture 2,639
LED Subtotal 66,923
CFL General Purpose 4,237
CFL Specialty: Reflector 261
CFL Specialty: Globe 13
CFL Specialty: Candelabra 59
CFL Specialty: 3-way 3
CFL Fixture 73
CFL Subtotal 4,645
LED General Purpose 266
LED Specialty: Reflector 98
LED Specialty: Globe 6
LED Specialty: Candelabra 21
LED Specialty: 3-way 0
LED Fixture 26
LED Subtotal 420
LED General Purpose 379
CFL General Purpose 1,824
462,144
Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 106
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 120 of212
6 RESIDENTIAL IMPACT EVALUATION
6.7.3.2 Hours of Use
As part of the evaluation of residential lighting, the team conducted a large-scale residential
lighting hours-of-use (HOU) study by collecting usage data from onsite metering of lighting
fixtures in the homes of Avista customers. The study methodology aligns with the Department of
Energy (DOE) Uniform Measure Project (UMP) for residential lighting. The research team
measured how many hours per day various lighting fixtures were illuminated during a six-month
study period beginning July 2015 and lasting through January 2016, at the residences of 74
Avista customers. An average of seven lamps per home were metered across a random sample
of fixture and room types, with 522 lighting meters deployed across Avista's service territory.
Collecting data for an average of seven lamps per residence enabled the team to gather a large
dataset for analysis across multiple delivery streams, residence, and room types. Metered
lamps included both efficient lamps (CFLs and LEDs) and inefficient lamps (incandescents and
halogens). A full inventory of lighting (fixture, socket, lamp type, etc.) was also performed while
onsite. Chapter 8 details the residential lighting hours-of-use study.
As a study outcome, the measured hours of use for residential lighting bulbs appear in Table
6-30.
Table 6-30: Verified Hours of Use for Residential Lighting
Room (Logger level, weighted Annualized Room-
by event type) Based HOU/day
Kitchen 3.75
Dining 2.48
Living/GreaUFamily 2.41
Foyer/Hall/Stair 1.25
Bedroom 1.25
ToileUBathroom 1.82
Other 1.52
TOTAL WEIGHTED AVERAGE 1.94
Because the room type and previous bulb technology of the installed residential lamp is
unknown, the total weighted average hours of use of 1.94 hours per day was applied for all
residential premises. This value is identical to the Regional Technical Forum value for 60W­
equivalent screw-in lamps delivered through a retail markdown channel in the most current UES
assumptions.
6. 7 .3.3 Delta Watts
Delta watts represent the difference between the wattage of the assumed baseline product and
the wattage of the CFL or LED. For the CFL and LED markdown programs, the evaluation team
first assessed Energy and Independence Security Act (EISA) eligibility for each program bulb
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 107
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 121 of212
,-------------------------------------------------·-
6 RESIDENTIAL IMPACT EVALUATION
product type, segmenting the bulbs into a few groups: EISA-qualified general service lamps
(GSL), EISA-qualified reflectors, decorative lamps, and globes. These categories were assigned
baselines considering lumen equivalency and "bin mapping,"22 as summarized Table 6-31 and
Table 6-32
Table 6-31: Standard Lamp Baseline Wattage for Equivalences
Incandescent Equivalent Wattage
Minimum Lumens Maximum Lumens Baseline (Exempt
B I Baseline (Post-EISA) u bs)
2,000 2,600 150 72
1,600 1,999 100 72
1,100 1,599 75 53
8000 1,099 60 43
450 799 40 29
310 449 25 15
Table 6-32: Decorative and Globe Lamp Baseline Wattage for Equivalences
Lumen Bins Incandescent Equivalent Wattage
Baseline (Exempt . Decorative Shape Globe Shape Bulbs) Baseline (Post-EISA)
1, 100-1 ,300 150 72
650-1 ,099 100 72
575-649 75 53
500-699 500-574 60 43
30Q-499 350-499 40 29
150-299 250-349 25 15
90-149 15 15
70-89 10 10
For some product type, the lumen bin is documented by Simple Steps and is easy to map to
these EISA bins. For other products, only the efficient case wattage of the product type is
known; the evaluation team correlated the wattage to the equivalent lumen bin for each lighting
technology (i.e. CFL or LED) through market research .
For the assessment of gross verified energy savings, the post-EISA baseline was used for each
product type and wattage. Additionally, the evaluation team calculated a market baseline
considering the composition of lamp types found from onsite inspections in the lighting study;
respective EISA equivalent baselines; and efficient case wattage to determine the free-ridership
22 "Bin mapping" refers to the assignment (or "mapping") of lumen-based equivalent bulbs based on ranges (or "bins") to determine
baseline watts.
'-"Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 108
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 122 of212
6 RESIDENTIAL IMPACT EVALUATION
market effects, in which a customer likely replaced an expired efficient technology with a like
technology. Refer to the description in Appendix E for additional information.
6.7.3.4 Interactive Effects
The team considered heating and cooling interactive effects associated with replacing standard
incandescent light bulbs with higher efficiency lighting technology. CFLs and LEDs release
substantially less heat into the room, leading to increased heating and decreased cooling loads
for a home.
The evaluation team used a single, deemed value of 93.4% to estimate the impacts of the
heating, ventilating, and air conditioning (HVAC) system based on assumptions from the most
recent RTF residential lighting UES calculation model. Stated differently, the electric energy
savings of the efficient lamp were effectively reduced by 6.6% because of the necessary
increase in electric heating. However, the evaluation team believes that this reduction factor is
likely high for Avista's service territory, because gas-heated homes are more prevalent there
than in the Pacific Northwest at-large.
6.7.3.5 Installation Rate
The installation rate, also commonly referred to as the in-service rate (ISR), represents the
percentage of program bulbs purchased that are ultimately installed by program participants.
This rate quantifies customers' common practice of waiting to replace a bulb until it has burned
out, which can lead to product storage and deferred installation. Retail and giveaway programs
distribute the bulbs but do not guarantee that customers actually install the bulbs.
For the CFLs distributed as part of the Simple Steps retail program and Avista giveaway
delivery channels, the evaluation team used first-year installation rates of 76% from the most
recent RTF residential lighting UES calculation model and RBSA23. This installation rate only
considers the first-year installation rate; it is well understood that stored lamps will eventually be
installed by the customer24. Because Avista reports program savings on a first-year, annualized
basis, the evaluation discounted the future savings of stored lamps back to present value. The
RTF UES calculation model recognizes that stored lamps will be installed in the future, but
elects to only apply a 109% savings factor in the future and does not provide a present value
that can be used in evaluations with first-year savings values.
The evaluation team followed industry-standard DOE-UMP protocols to forecast the future
installation trajectory for both program components. Trajectory refers to the installation rates to
account for installations that occur in the years following the program year in which the bulb was
purchased . The UMP trajectory leverages a comprehensive multi-year study that tracked
installations for the same group of participants. A review of the trajectory calculations is included
23 24% Storage Rate; Ecotope Inc., "2011 Residential Building Stock Assessment: Single-Family Characteristics and Energy Use",
prepared for the Northwest Energy Efficiency Alliance, September 2012.
24 Section 4.12 Residential Lighting Chapter (21) in the UMP: http://energy.gov/sites/prod/files/2013/11/f5/53827-6.pdf.
'-'1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 109
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 123 of 212
6 RESIDENTIAL IMPACT EVALUATION
in Table 6-33 below. The team used 20-year Treasury bill rates, currently 2.3%, as the rate to
discount future installation savings. Using the 2.3% discount rate and accounting for years two
through four for installations per the UMP, the final estimated CFL markdown installation rate
was 97.5%.
Table 6-33: In-Service Rate Trajectory for Markdown and Giveaway CFL based on UMP
Y Incremental % Total % ISR C 1 1 t· Retail/ ear a cu a 10n Installed Installed Giveaway
Year1 NA NA Researched Value 76%
Year2 41% 41% (Storage % Y1 * 85.8% 41%)+1SR Y1
Year3 28% 69% (Storage % Y1 * 92.6% 69%)+1SR Y1
Year4 NA NA Default to 97% 97.0%
OVERALL ISR NA NA NPV Y1->Y4 97.5%
Consistent with the RTF assumption, the team chose to apply a 100% installation rate for LEDs
because:
• Limited or no applicable or equivalent research has been completed for LED bulbs
• The LEDs were purchased as single packs; the CFLs were purchased as multipacks,
encouraging customers to place them in storage
• The higher prices of LEDs would likely lead to limited, if any, stockpiling.
Finally, consistent with the RTF assumption, the evaluation team applied a 2% removal rate for
all lamps removed before expiration.
6.7.3.6 Cross-Sector Sales Leakage
The Simple Steps, Smart Savings program promotes the sales of CFL and LEDs to residential
customers. Avista currently only reports savings for this offering through their residential lighting
program. However, because of the delivery mechanism of the program via in-store, buy-down
promotions, the evaluation team sought to understand if nonresidential customers were
purchasing bulbs discounted through the program and, if so, what percentage of Simple Steps
bulbs were "leaking" into the nonresidential sector. The evaluation team estimated this "leakage"
into the commercial sector using the responses of customers (participants and nonparticipants),
as well as by conducting a survey of large retailers that sell Simple Steps items. The evaluation
team's activities are outlined in the process evaluation report of Avista Utilities 2014 and 2015
energy efficiency programs. Figure 6-11 summarizes the evaluation team findings from surveys
of customers and retailers for CFL and LED lamps.
t.-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 110
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 124 of 212
6 RESIDENTIAL IMPACT EVALUATION
Figure 6-11: Estimates of Percentage of Products in Commercial Sector
14% 12.6%
12% 11.6% 12.0%
10%
8%
6% 5.3%
4%
2%
0%
CFL LED
• Customer • Retailers
Additionally, the evaluation team used the RTF nonresidential operating characteristics to inform
the nonresidential HOU: 8 hours per day as a weighted average across the business types25.
The commercial parameter assumptions, including operating hours and in-service rates, are
included in Table 6-34.
Table 6-34: Nonresidential Lighting Input Parameter Assumptions
Parameter CFL bulbs LED bulbs
Hours of Use 8.0 8.0
Cross Sector Sales Shares 8.4% 12.3%
6.7.4 Findings and Recommendations
The verified unit counts, verified energy savings, and average savings per lamp are summarized
in Table 6-36 for each product type in the residential lighting program.
25 This value is from market research Nexant conducted for the State of Pennsylvania as the Statewide Evaluator (SWE).
http://www.puc.pa .qov/pcdocs/1340978. pdf
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 111
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 125 of 212
6 RESIDENTIAL IMPACT EVALUATION
Table 6-35: Verified Residential Lighting Energy Savings by Lamp Type and Delivery
Stream
Program Delivery L T U ·t C t Verified Energy Average amp ype m oun s Stream Savings (kWh) kWh/bulb
Simple Steps
Simple Steps -NP
'-"Nexanr
CFL General Purpose 342,094 7,430,156 22.3
CFL Specialty: Reflector 41 ,637 993,917 23.5
CFL Specialty: Globe 733 21 ,811 29.4
CFL Specialty: 737 19,878 26.7 Candelabra
CFL Specialty: 3-way 648 24,476 37.8
CFL Fixture 2,104 96,589 45.9
CFL Subtotal 387,953 8,586,828
LED General Purpose 61 ,803 1,542,708 22.8
LED Specialty: Reflector I 1,733 39,064 32.5
LED Specialty: Globe 56 1,136 20.3
LED Specialty: 506 9,176 18.1 Candelabra
LED Specialty: 3-way 186 8,775 45.1
LED Fixture 2,639 79,369 30.1
LED Subtotal 66,923 1,630,230
CFL General Purpose 4,237 101 ,593 24.0
CFL Specialty: Reflector 261 6,640 25.4
CFL Specialty: Globe 13 373 28.7
CFL Specialty: 59 1,317 22.4 Candelabra
CFL Specialty: 3-way 3 96 32.0
CFL Fixture 73 3,345 45.9
CFL Subtotal 4,645 113,364
LED General Purpose 266 7,992 30.1
LED Specialty: Reflector ! 98 3,348 34.1
LED Specialty: Globe 6 130 20.3
LED Specialty: 21 417 20.3 Candelabra
LED Specialty: 3-way 0 18 41.8
LED Fixture 26 782 30.1
LED Subtotal 420 12,687
Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 112
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 126 of212
6 RESIDENTIAL IMPACT EVALUATION
Program Delivery L T U . C Verified Energy Average amp ype mt ounts Stream Savings (kWh) kWh/bulb
LED General Purpose 379 9,321 30.1
Giveaway
CFL General Purpose 1,824 54,858 24.6
TOTAL 462,144 10,457,288
The electric realization rate for the residential lighting program is 131 %, as shown in Table 6-36.
The relative precision of the program-level electric realization rate is ±13.5% at the 90%
confidence level, largely based on the residential lighting hours-of-use study.
Table 6-36: Residential Lighting Realization Rates and Gross Verified Savings
2014-2015 2014-2015 Gross . . . . 2014-2015 Reported Realization . . . Delivery Stream Part1c1pat1on S . (kWh) R Verified Savmgs . avmgs ate (unit counts) (kWh)
Simple Steps-LED 66,923 1,846,600 91 .0% 1,680,230
Simple Steps-CFL 387,953 6,371 ,184 134.8% 8,586,828
Simple Steps -NP-LED 420 6,376 199.0% 12,687
Simple Steps -NP-CFL 4,645 70,970 159.7% 113,364
Giveaway -CFL 1,824 27,360 200.5% 54,858
Giveaway-LED 379 1,352 689.4% 9,321
TOTAL 462,144 8,323,842 125.6% 10,457,288
The key factors for the realization rates that were greater than 100% are summarized below:
• Avista's deemed savings estimates, which were generally the same for all similar
product types, and not correlated to the bulb wattage, understated the savings, in
particular for the giveaway program; improved data illuminated the actual savings
• For product types where Simple Steps and Avista reported a weighted-average energy
savings value for multiple lamp wattages, the actual weighted-average, verified-lumen
bin was greater than the assumed value, resulting in higher savings
• Verified cross-sector nonresidential sales and the corresponding increase in hours of
use meant realization rates over 100%.
t.-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 113
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 127 of 212
6 RESIDENTIAL IMPACT EVALUATION
6.8 Shell Program
6.8.1 Overview
Avista's internally managed shell program incentivizes measures that improve the integrity of
the home's envelope such as insulation (attic, floor and wall), and window replacements. The
evaluation team conducted a database review, document audits, customer telephone surveys,
and a billing analysis to estimate the adjusted reported and gross verified savings for the
program.
6.8.2 Program Achievements and Participation Summary
Participation in the 2014 and 2015 Shell program totaled 370 projects. Table 6-37 and Figure
6-12 summarize Avista's 2014 and 2015 Shell program participation and energy impacts.
Table 6-37: 2014-2015 Shell Program Reported Participation and Savings
M 2014-2015 Reported 2014-2015 Reported
easure Participation Count Savings (kWh)
Attic Insulation* 45 46,172
Floor Insulation 12 17,946
Wall Insulation 17 35,948
Window Replacement from Single Pane* 146 505,897
Window Replacement from Double Pane 150 297,701
TOTAL 370 903,663
*Includes projects and savings for gas measures that reported electricity savings
t-'1 Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 114
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 128 of212
.-------------------------------~-------------------
6 RESIDENTIAL IMPACT EVALUATION
Figure 6-12: 2014-2015 Shell Program Reported Energy Saving Shares
• Attic Insulation
• Floor Insulation
• Wall Insulation
• Window Replacement
from Single Pane
• Window Replacement
from Double Pane
6.8.3 Methodology
The evaluation team merged electric billing data from participating homes with historic weather
conditions (HOD and COD) and program tracking data was used to code the pre-retrofit and
post-retrofit period for each home. The evaluation team then estimated fixed effects panel
regression models to develop a mathematical relationship between weather and electric load
before and after the Shell improvements were installed. Equation 6-6 shows the form of the
model and the text below defines the model terms.
Equation 6-6: Fixed-Effects Panel Regression Model Specification
kWhit =/Ji+ /31 (Post)it + /32 (CDD)it + /33 (Post x CDD)it + /34 (HDDh + /35 (Post X HDD)it + Eit
Where :
kWhit
Postit
CDDit
HDDit
E
{3;
/31-5
= Estimated energy usage (dependent variable) in home i during period t
= Dummy variable indicating whether period twas pre-or post-retrofit
= Average cooling degree days (base 65 F) during period tat home i
= Average heating degree days (base 65 F) during period tat home i
= Customer-level random error
= The model intercept for home i
= Coefficients determined via regression
The [31, [33, and [35 terms in Equation 6-6 represent the average change in daily baseload, daily
kWh per COD, and daily kWh per HOD respectively. The evaluation team used these
coefficients and normal weather conditions (TMY3) for the three chosen weather stations to
estimate the average weather normalized annual savings.
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 115
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 129 of212
6 RESIDENTIAL IMPACT EVALUATION
In order to construct the electric Shell Rebate analysis data set, the evaluation team
implemented the following data preparation steps. The number of unique homes remaining for
analysis after each filter is shown in parentheses.
• Identify the homes that participated in the Shell program and had billing data provided by
Avista to the evaluation team (2,724)
• Exclude homes that also participated in other Rebate programs to ensure Shell impact
estimates are not confounded with impacts from the Fuel Efficiency, HVAC, or other
programs. (2,514)
• Limit the data set to homes with reported kWh savings and electric billing data (1,991)
• Exclude homes with fewer than 12 months of pre-retrofit billing history (908)
• Exclude homes with fewer than 12 months of post-retrofit billing history (767).
In addition to the billing analysis activities noted above, the evaluation team performed
verification of the program tracking database and conducted 28 document audits of participating
projects.
6.8.4 Findings and Recommendations
6.8.4.1 Shell Rebate Measures
The evaluation team's regression analysis produced statistically significant reductions in both
the cooling and heating loads of homes that implemented the Shell Rebate measures (attic,
floor and wall insulation, and window replacements). Appendix C presents the full regression
output for the Shell Rebate measures, and the key outputs are summarized in Table 6-38. On
average, homes were savings 0.14 kWh per COD and 0.05 kWh per HOD in addition to 0.39
kWh per day reduction in non-weather dependent electric usage.
Table 6-38: Shell Rebate Model Coefficients
Model Term Pre-Retrofit Post-Retrofit Savings
Base Load 20.04 19.65 0.39
Daily kWh per COD 1.77 1.63 0.14
Daily kWh per HOD 0.75 0.70 0.05
Although the electric reductions from Shell Rebate measures are statistically significant in both
the heating and cooling season, the gross verified savings estimate is well below the reported
savings values for the analyzed homes. The average reported savings per home was 1,406
kWh and the average verified savings was 537 kWh. This result equates to a realization rate of
38.2% (Table 6-39) and a 4.1 % average reduction in total weather normalized electric
consumption (Table 6-40).
t.-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 116
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 130 of212
6 RESIDENTIAL IMPACT EVALUATION
Table 6-39: Shell Rebate Gross Verified Savings Sum,mary
# Average Reported Annual kWh Annual kWh Gross Verified kWh Realization
Homes kWh Pre Post Savings Rate
767 1,406 13,021 12,484 537 38.2%
The relative precision of the savings estimate is ± 24.8% at the 90% confidence level. Although
the per-home margin of error is actually reasonably tight at± 133 kWh/year, the precision
suffers when considered on a relative basis because of the lower than expected impacts. Table
6-40 provides some additional relevant measurements of the estimated gross verified energy
savings along with the upper and lower bound of the 90% confidence interval.
Table 6-40: Precision of Findings
I S . t· p . t E t· t Lower Bound of 90% Upper Bound of 90% mpact tat1s 1c om s ,ma e . Confidence Interval Confidence Interval
Gross Verified kWh per Home 537 404 670
Realization Rate 38.2% 28.7% 47.6%
Percent Reduction in Whole House 4.1% 3.1% 5.1% Electric Usage
Percent Reduction in Cooling 7.9% 1.8% 14.0% Usage
Percent Reduction in Electric 6.8% 3.0% 10.5% Heating Usage
The evaluation team also examined the performance of Shell Rebate measure categories
(window upgrade and insulation) to investigate if the low realization was being driven by a
particular measure. Table 6-41 shows the results of this more granular analysis. Savings for
homes that received rebates for insulation and windows, both, were not examined.
Table 6-41: Shell Rebate Performance by Measure Category
Window Upgrade Window Upgrade Insulation Upgrade Parameter . (Electric Heat) (Gas Heat) (Electric Heat)
Number of Homes Analyzed 209 503 27
Average Reported kWh 2,539 737 1,319
Annual kWh Pre 18,762 10,351 18,516
Annual kWh Post 17,993 9,925 18,254
kWh Savings 769 426 262
REALIZATION RATE 30% 58% 20%
Avista claims a modest electric savings from gas heated homes that install efficient windows -
on average 737 kWh per home as shown in Table 6-41 . This group's verified savings estimates
were closest to the reported values of the three categories analyzed , although none of the
differences between groups are statistically significant.
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 117
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 131 of212
6 RESIDENTIAL IMPACT EVALUATION
The regression coefficients summarized in Table 6-42 may also help explain the low realization
rate for Shell Rebate measures. The evaluation team's regression analysis estimates that prior
to retrofit, participating homes were using slightly more than 13,000 kWh annually, but only
approximately 5,500 kWh of this consumption was weather dependent HVAC load.
Table 6-42: Shell Rebate Measure Average Annual Usage
Model Term Pre-Retrofit Coefficient Multiplier Annual Usage (kWh)
Base Load (kWh/day) 20.04 365 (days) 7,513 (57.7%)
Daily kWh per COD 1.77 379 (Spokane COD) 700 (5.4%)
Daily kWh per HOD 0.75 6,707 (Spokane HOD) 4,808 (36.9%)
AVERAGE ANNUAL KWH PER SHELL REBATE PARTICIPANT 13,021
Savings from shell improvements should be realized almost exclusively through reductions in
heating and cooling usage within participating homes. When the average reported savings claim
of 1,406 kWh across the 767 homes analyzed is compared to this estimate of end-use load
shares, we see that the program is claiming a (1,406/5,508) = 25.5% reduction in HVAC loads.
The evaluation team recommends Avista examine planning assumptions about per-home
consumption, end-use load shares, and percent reductions in heating and cooling loads from
shell improvements. It may be that the percent reduction assumptions are sound, but they are
being applied to an overstated assumption of the average electric HVAC consumption per
home.
6.8.5 Program Results
As noted in section 6.8.2, the evaluation team found several significant outliers in Avista's
reported data during the database review for the Shell program. In addition, during the
document audit activities, the evaluation team also found that reported savings values did not
match the project documentation for the majority of the sampled homes that had window
replacement from single pane measures (such as size of window installed and baseline
measure). In addition, the document audit activities found several discrepancies in the heating
fuel type reported for the home and the associated fuel type that the measure is savings. For
example, in a few instances, both the customer survey and the project application state wood
and natural gas as the primary heating source, but the window and attic insulation incentives
were paid based on electric heating. Based on these findings, the evaluation team recommends
that Avista work with local contractors to confirm that the measure savings is tied to the correct
heating fuel source, perhaps conducting verification activities on a percent of applications
received would also help improve the reporting accuracy.
The electric realization rate for the Shell program is 38%. This program level realization rate
was developed by taking a weighted average of the realization rates of the program measures
shown in Table 6-43. The relative precision of the program level electric realization rate is
±33.1 % at the 90% confidence level.
t-'1 Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 118
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 132 of212
6 RESIDENTIAL IMPACT EVALUATION
Table 6-43: Shell Program Gross Verified Savings
Attic Insulation 45 46,172 46,172 38% 17,630
Floor Insulation 12 17,946 17,946 38% 6,852
Wall Insulation 17 35,948 35,948 38% 13,726
Window Replacement from 146 505,897 505,897 38% 193,168 Single Pane
Window Replacement from 150 297,701 297,701 38% 113,672 Double Pane
TOTAL 370 903,663 903,663 38% 345,048
6.9 Opower Behavioral Program
6.9.1 Overview
Home Energy Report (HER) programs have been widely shown to obtain savings through
reduced energy consumption among households that receive them. Avista's Behavioral
program relies on normative comparisons of energy usage to similar homes to increase
awareness of energy consumption levels and stimulate recipients to alter their behavior and
consume less energy. The evaluation approach relies on a combination of large sample sizes
and random assignment to enable straightforward quantification of associated energy savings.
HERs provide residential customers with detailed information about how their home uses
energy and includes charts that compare their energy use to that of similar homes. Participants
receive up to eight home energy reports annually.
The program launched in June 2013 towards the end of the previous biennium. Avista assumed
a three year measure life for savings reported in the 2012-2013 biennium so all program
achievements in the 2014-2015 biennium were incremental to the 2,870,905 kWh reported by
the program in the previous biennium. Due to a change in billing system , reports were
suspended and none were sent out from February to August of 2015. Reports were reinstated in
September 2015; however there was concern about how the gap in reports may affect savings
given the incremental accounting of savings net of the previous biennium's achievements.
Like all of Avista's Idaho DSM offerings, the Opower Behavioral program is operated as an
"electric only" program with the HER messaging designed to stimulate electric conservation
among recipient homes. Because of this, Opower calculated reported savings only on electricity
(kWh usage), and not on gas (therm) usage. Nexant also requested and analyzed the gas
consumption records of treatment and control group homes who receive natural gas service
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 119
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 133 of212
6 RESIDENTIAL IMPACT EVALUATION
from Avista to assess whether the program produced statistically significant reductions in gas
usage.
6.9.2 Program Achievements and Participation Summary
In Idaho, approximately 25,200 treatment and 13,000 control participants were randomly
enrolled in the Behavioral Program. The Opower program is set up as an "opt-out" program, not
an "opt-in" program, meaning that while households are randomly selected to receive the home
energy report, they can also choose to opt out. Figure 6-133 presents the number of treatment
participants and the opt-outs as a cumulative percentage by month in the post-period. The dip in
participants observed in 2015 is most likely a legacy of Avista switching its billing system around
that time. Approximately 2%26 of homes opted out of the program.
30,000
25,000
~ 20,000
C: I'll
-§" 15,000 :e I'll
c. 10,000
5,000
Figure 6-13: Participation and Cumulative Opt-outs by Month
-Cumulative opt-outs{%) -Number of treatment participants
~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~", ;-,, ~"" «"-; ,? ;!:."-; ~"-; :-,; ~"-; «"> ,;-,, ;!:.",, ~", ;-,, ~", '>°" c.,e,~ ~o '>'?f ~'?f ~'?f '>°" c.,e,~ ~o '>'?f ~'?f ~'?f '>°" c.,e,~ ~o
M onth
6.9.3 Methodology
6.9.3.1 Data Sources and Management
5%
4%
... ::I 3% 0
"'C (1j ... C. 2% 0
'*-
1%
0%
To develop estimates of the electric savings attributable to Avista's Behavioral Program, the
evaluation team requested data covering two core components:
1) Participation Record: A list of all billing accounts that are part of the initiative,
treatment\control designation, date assigned, opt-out or move-out data if
applicable, and any demographic or rate code status information available in
Avista's customer information system.
2) Consumption History: Monthly electric and gas billing records for each account in
the treatment and control group including the meter read date and number of days
26 543 opt-outs from a total of 25,200 treatment group homes
i-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 120
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 134 of 212
6 RESIDENTIAL IMPACT EVALUATION
in the billing period. Billing history was requested back to February 2012 to ensure
adequate pre-treatment data for the analysis.
In preparation for the impact analysis, Nexant combined and cleaned the billing data provided
by Avista. The dataset included 38,185 distinct accounts, 25,191 of which were assigned to the
treatment group and 12,994 of which were assigned to the control group. The billing history
dataset included 1,494,134 monthly billing records.
Nexant removed the following data points and customers from the analysis:
• 3 accounts with duplicate billing data.
• 2,023 accounts that had no billing data after program launch
• 3,064 accounts that lacked 12 months of billing data in the pre-period (March 2012-June
2013). Less than 12 months of pre-treatment data is insufficient for the analysis.
For the participation numbers used to calculate the aggregate impacts for each program month,
the number of treatment participants was the number of unique treatment accounts with billing
data that month, before accounts with no post data and accounts with insufficient pre-data were
removed . Treatment group homes that opted out of the program were not removed from the
impact analysis or the participation counts. While this may seem counterintuitive, it is necessary
to preserve the integrity of the RCT design because control group homes do not have the option
to opt-out and there is no way to determine which control group homes would opt-out if they
were assigned to treatment. This approach dilutes the per-home impacts to some extent
because only -98% of the participants were actively receiving HERs at a given time, but this is
negated by including all active accounts in the estimation of aggregate impacts.
Like most utilities, Avista does not bill its customers for usage within a standard calendar month
interval. Instead, billing cycles are a function of meter read dates and vary across accounts.
Since the interval between meter reads vary by customer and by month, the evaluation team
"calendarized" the usage data to reflect each calendar month, so that all accounts represent
usage on a uniform basis. The calendarization process includes expanding usage data to daily
usage, splitting the bill month's usage uniformly among the days between reads. The average
daily usage for each calendar month is then calculated, by taking the average of usage within
the calendar month.
A similar calendarization process was performed on the gas billing data. However, instead of
cleaning individual accounts with bad data, we matched up the accounts with valid electric
billing data to the accounts in the gas billing data and only used those accounts that were also
in the cleaned electric data.
6.9.3.2 Equivalence Testing
The next step in the evaluation team's analysis approach was to perform a detailed review of
the assignment randomization by comparing consumption patterns for the treatment and control
group for the months in the pre-period (March 2012 to June 2013). The purpose of this analysis
t-'1 Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 121
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 135 of212
6 RESIDENTIAL IMPACT EVALUATION
is to determine if structural differences in electricity consumption existed between the treatment
and control group prior to HER exposure. Pre-treatment differences can take the form of total
annual consumption or variation in the seasonality of consumption. The findings of this step are
of critical importance because they will determine the appropriate model specification to
estimate savings. Table 6-4444 displays the results of a difference in means two-sided t-test to
validate the randomization and confirms that there is no significant difference in usage between
the treatment and control groups in the pre-period. The results confirm that the randomization is
robust and that there is no real difference in the energy consumption of the two groups.
Table 6-44: Difference in Means t-test Values
Control Average Daily Treatment Average Daily C . . 1 V 1 ( ) p 1 (95.,) . rit1ca a ue t -va ue ,o Usage: Pre period Usage: Pre period
43.51 43.47 .59 .55
Figure 6-14 examines usage in the pre-treatment visually and echoes the results of the
statistical test.
Figure 6-14: Treatment and Control Energy Usage in the Pre-Period
7/2012 Control
Treatment
8/2012 Control Treatment
9/2012 Control Treatment
10/2012 Control Treatment
11/2012 Control Treatment
12/2012 Control
Treatment
1/2013 Control
Treatment
2/2013 Control
Treatment
3/2013 Control
Treatment
4/2013 Control Treatment
5/2013 Control Treatment
6/2013 Control Treatment
0 50 100 150
Daily Usage (kWh)
excludes outside values
6.9.3.3 Regression Analysis
The evaluation team used a lagged dependent variable (LDV) model to estimate savings. The
LDV model is the preferred analysis approach to use when the randomization of homes to
treatment and control is sound and results in groups with equivalent usage prior to HER
exposure, as presented in the section above. If pre-assignment differences in electric
'-'1 Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 122
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 136 of212
6 RESIDENTIAL IMPACT EVALUATION
consumption are present, a linear fixed effects regression model (LFER) would have been the
more appropriate model.
The LDV model is a category of specifications in which the dependent variable in the equation is
restricted to the post-test period. The customers' usage prior to the onset of treatment for the
same period (i.e., usage in the same monthly period in the prior year) is entered into the
regression model as an independent variable -thus the name lagged dependent variable model
- and the coefficient for the treatment variable is interpreted as the change in consumption due
to treatment. The specification used is shown in Equation 6-7 and the corresponding variables
are defined in Table 6-45.
Equation 6-7: Lagged Dependent Variable Model Specification
12 n
kWhity =/Jo + LL lty * /Jty + kW hi,t,y-n * /Jt,y-n + T * treatmenti * lty + Eit
t =l y=l
Table 6-45: Lagged Dependent Variable Model Definition of Terms
Variable Definition
/Jo
kWhi,t,y-n
/Jt,y-n
I The intercept, or the coefficient on the billing month t, post-period year indicator variable that is left
I out due to collinearity
I Customer i's average daily energy usage in billing month t of the post-period y
I Indicator variable that equals one for each monthly billing period t, post-period y and zero
I otherwise.
The coefficient on the billing month t, post-period year indicator variable
The lagged usage of customer i in the corresponding billing month t, in the pre-period y-n
The coefficient for the corresponding billing month t, in the pre-period y-n
treatmenti I Treatment variable, equal to one if customer i is in the treatment group and zero if control
T I Estimated average daily energy reduction of the treatment group in bill month t for the post-period j y
I Error term for customer i for bill month t
The average daily treatment effect (r) for each billing period of the study is multiplied by number
of active customers in the treatment group times the number of days in that month to estimate
the monthly aggregate savings (MWh). The monthly savings impacts are summed over the
study horizon to produce the total change in energy consumption in treated homes over the
period under study. The results of an overlap analysis discussed below are then subtracted from
this total change in consumption to arrive at the net ex post energy savings attributable to the
Behavioral Program.
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 123
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 137 of 212
6 RESIDENTIAL IMPACT EVALUATION
6.9.3.4 Overlap Analysis
The ability to serve as a marketing tool for other energy efficiency initiatives is an important part
of what makes normative comparison reports so attractive to utilities and agencies. The billing
analysis methodology captures all savings at the meter, even those claimed by other programs.
To the extent that the treatment and control group participate in other Avista programs at a
different rate, the difference in kWh needs to be netted off of the Behavioral Program impact to
prevent any double-counting or under-statement of savings. For measures promoted by Avista
and tracked at the customer level, the amount of savings overlap was estimated by matching
the treatment and control group customers to the energy efficiency program participation data.
Next, the difference between treatment and control groups in rebated savings per home is
calculated and the difference multiplied by the number of treatment group homes.
6.9.4 Findings and Recommendations
6.9.4.1 Per-Home kWh and Percent Impacts
The evaluation team estimates the average home in the Opower Behavioral Program saved
over 579 kWh of electricity from January 2014 through December 2015. This represents a
1.81 % reduction in total electric consumption compared to the control group over the same
period. The 579 kWh and 1.81% impact estimates include HER savings net of savings from
incremental participation in other Avista Energy Efficiency programs. As explained in Section
6.9.3.4, an overlap analysis was performed to prevent double-counting of savings that have
already been attributed to another energy-saving program. The overlap analysis found that
treatment group homes participated in energy efficiency programs at a greater rate than the
control group, necessitating a downward adjustment of the impacts. This means a net decrease
in usage for the Opower Behavioral Program when comparing the treatment to the control.
Therefore , a downward adjustment was applied to each monthly savings estimate based on
differential energy efficiency participation and the greater per-home EE savings for the
treatment group. The dual participation adjustment totaled 1.35 kWh over the 24 month period
of analysis.
Table 6-46 shows the LDV impact estimates in each month for the average treatment
household, totaling 580 kWh over the biennium. The table also shows the program savings after
subsequent adjustment for savings attributed the energy efficiency overlap, totaling 579 kWh
per household over the biennium.
'-"Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 124
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 138 of212
6 RESIDENTIAL IMPACT EVALUATION
Table 6-46: Opower Behavioral Program Impact Estimates with EE Adjustments
Feb-14 21 ,846
Mar-14 21 ,176
Apr-14 20,953
May-14 20,708
Jun-14 20,430
Jul-14 20,138
Aug-14 19,845
Sep-14 19,651
Oct-14 19,447
Nov-14 19,248
Dec-14 19,157
Jan-15 19,366
Feb-15 19,296
Mar-15 19,158
Apr-15 19,001
May-15 18,818
Jun-15 18,635
Jul-15 18,419
Aug-15 18,189
Sep-15 18,021
Oct-15 17,869
Nov-15 17,726
Dec-15 17,635
Biennium Total
t-1Nexanr
24.55 -0.24 24.79 542
18.52 -0.22 18.74 397
15.57 -0.19 15.76 330
13.80 -0.27 14.08 291
15.07 -0.28 15.35 314
22.76 -0.19 22.95 462
19.52 -0.20 19.72 391
16.56 -0.23 16.79 330
22.58 -0.26 22.84 444
30.91 -0.21 31 .12 599
41.28 -0.09 41 .37 792
33.08 -0.07 33.15 642
26.86 -0.33 27.20 525
21 .32 -0.33 21 .65 415
19.89 -0.04 19.93 379
17.42 0.69 16.73 315
20.28 0.38 19.90 371
18.94 0.59 18.35 338
, 19.29 0.50 18.80 342
18.76 0.29 18.47 333
27.23 0.36 26.87 480
36.40 0.81 35.59 631
52.91 1.00 51 .92 916
580.42 1.35 579.08 11,176
Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 125
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 139 of212
6 RESIDENTIAL IMPACT EVALUATION
6.9.4.2 Aggregate Impacts
The total impact of the Opower Behavioral Program is calculated by multiplying the per-home
impacts (adjusted for incremental EE participation) for each calendar month by the number of
treatment group homes in that month. Over the twenty-four month period examined by the
evaluation team in this evaluation , participants saved 11,176 MWh of electricity. The monthly
and annualized aggregate savings are shown in Table 6-46.
Because some of the savings observed in the 2014-2015 biennium were already claimed in the
previous biennium due to the assumed measure life of 3 years, these previous achievements
must be netted out to calculate incremental achievements and prevent double-counting . Table
6-4 7 displays the aggregate savings in 2014 and 2015, respectively, net of savings counted in
the previous year.
Table 6-47: Opower Program Incremental Annual MWh Savings
I
Y Reported MWh impact Verified MWh impact 1 1 W ear . . ncrementa M h (cumulative) (cumulative)
2013 3,184 2,871 0
2014 5,668 5,491 2,620
2015 5,930 5,685 195
BIENNIUM TOTAL 2,814
6.9.4.3 Precision of Findings
The margin of error of the impact estimates are also important to consider. If margin of error is
wide, the true savings value could actually differ from the point estimates by a large amount.
The margin of error for the per-home biennium impact estimate is± 38 kWh at the 90%
confidence level. Table 6-48 presents the upper and lower bounds of the 90% confidence
interval for biennium per-home kWh savings, percent reduction, and aggregate impact
estimates.
Table 6-48: Confidence Intervals Associated with Behavioral Program Impact Estimates
Parameter Lower Bound (90%) Point Estimate Upper Bound (90%)
2014-2015 Program Savings per Home I 541 kWh 579 kWh 617 kWh
Percent Reduction 1.70% 1.81% 1.93%
Aggregate Impact 10,442 MWh 11,176 MWh 11 ,910MWh
The impact estimate has an absolute precision of± 0.12% and a relative precision of± 6.6% at
the 90% confidence interval. The estimates are statistically significant, as the confidence
interval does not include zero. Figure 6-15 shows the monthly savings estimates with relative
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 126
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 140 of212
6 RESIDENTIAL IMPACT EVALUATION
precision upper and lower bounds. The shaded box denotes the period between February and
August 2015 where reports were not being sent out.
Figure 6-15: Average Monthly Savings per Household with Relative Precision Bounds
• Household Mean Monthly Savings -Upper Bound (90%) -Lower Bound (90%)
60
::i: 50 ~ ::.
"' :!:° 40 ! ·s;
! ro VI ~ 30 i i i ... i QJ s:: I I LU 20 ~ • • ..r:: • ... • s:: 0 10 ::
0
'Sl" 'Sl" 'Sl" 'Sl" 'Sl" 'Sl" 'Sl" 'Sl" 'Sl" 'Sl" 'Sl" 'Sl" Lil Lil Lil Lil Lil Lil Lil Lil Lil Lil Lil Lil rl rl rl '2 rl rl rl rl rl rl rl rl rl rl rl rl '-;< rl '( rl '( rl rl '( c ..0 .'.. >-C: ' bl) a. ' > u c ..0 ' ' C: bl) ..:. > +-' .... .... > Cl. u ro (I) ro Cl. ro ::::J ::::J ::::J (I) u 0 (I) ro (I) ro Cl. ro ::::J ::::J ::::J (I) u 0 (I) ...., u. ~ <( ~ <( <J) 0 z 0 ...., u. ~ <( ~ <( <J) 0 z 0
Month
6.9.4.4 Savings Patterns
Avista currently mails out reports to the treatment group on a varying cycle, with participants
receiving 8 reports annually. The blue series in Figure 6-16 depicts the estimated percent
reduction for each month of the treatment period, July 2013 through December 2015. Figure
6-16 also shows the average daily kWh usage of the control group with a green line. The control
group's average daily usage shows highest electricity usage in the winter months.
'-'"Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 127
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 141 of212
6 RESIDENTIAL IMPACT EVALUATION
Figure 6-16: Average Percent Savings and Control Daily Usage by Month
';f!. 2.00%
"' t>.O C:
-~ 1.50%
V,
> e.o CIJ ~ 1.00%
0.50%
0.00%
-Treatment savings(%) -control daily usage (kWh/household)
~0 ~"J "'~") ~ ~ ~ ~~ ~ ~~ ~ ",,<-; <..~ ;.4.~ ~~ ,-:~ "'~<-;
\v c..,q,~ ~o \'b<::-~~ ~'b.:,;. \v <-f~ ~o \'b ~7> ~7> \v ,.f'' ~o
Month
70
60
so -~ s
40 ::::.
CIJ t>.O ni "' 30 => ~
ni
20 C
10
0
There is a seasonal pattern to the savings, where the greatest savings are experienced during
the winter months. It is unusual to see the highest savings on a percent basis when usage is
also peaking. However, we can see roughly the same pattern on an absolute basis in Figure
6-15. Additionally, the significant gas savings during the winter months, which are discussed in
more depth in Section 6.9.4.5, mean that the electricity savings are not entirely offset by an
increase in gas usage. The Opower reports can encourage fuel switching as a way of reducing
electricity usage.
It is important to note what is happening during the period of February to August of 2015, when
home energy reports were not being sent out to customers. The monthly savings by year are
shown in Figure 6-17. With the exception of July and August, each month's estimated savings
grows from 2014 to 2015. It is also important to note that the savings during this period hold
fairly consistent with what was observed in the year before, meaning they do not grow, but do
not diminish significantly either. Additionally, once reports resume in September 2015, monthly
savings surpass what they were in the years previous again.
L-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 128
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 142 of 212
6
"' IIO C:
60
RESIDENTIAL IMPACT EVALUATION
Figure 6-17: Household Monthly Savings by Year
• 2013 Savings • 2014 Savings • 2015 Savings
·s; 40 +---------------------------------
!II Ill
> :S 30
C: -0 ~ 20 -"tl 0 ..c: ~ 10 -:I 0 :I:
-----
-
.. ..... ---..... --
- - -
0 +-......... --,____J_.,_ ___ .--__ ....,___.._.---__ --,~_L,-J _____ ,..........._.....,_...._ __ ...,...~ ........ __,,___L,
Month
6.9.4.5 Gas Savings
While the Behavioral Program set up by Avista and Opower is an electricity-saving program,
Avista is a gas and electric utility and approximately 45% of the homes in Idaho assigned to the
program also receive natural gas service from Avista. The evaluation team used the LDV model
to examine any gas usage differences created by the program. In addition to general
conservation messaging, the Behavioral Program provided information on the benefits of fuel
switching (electric->gas). Although fuel switching impacts would be captured by the overlap
analysis if the switch was rebated by Avista, these interventions would have opposite effects , so
we entered the analysis without a hypothesis about whether gas reductions, increases, or no
effect at all would be found.
The results of the gas impact analysis with overlap analysis adjustments are summarized by
month in Table 6-49. While in certain months, a net increase in usage is observed in the
program participants, over the two year program period a net savings of 8.48 therms per
household is estimated. Program-wide, gas savings during the 2014-2015 biennium totaled
74,579 therms. Figure 6-18 displays the monthly gas savings estimates with relative precision
bounds. The shaded box represents the period between February and August 2015 when no
reports were sent out.
t-'1 Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 129
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 143 of 212
6 RESIDENTIAL IMPACT EVALUATION
The margin of error for the per-home biennium impact estimate is ± 6.0 therms at the 90%
confidence level. Table 6-50 displays the point estimates and the 90% confidence interval
upper and lower bounds for the biennial per home, percent, and aggregate gas savings
estimates. The impact estimate has an absolute precision of± 0.43% and a relative precision of
± 71 % at the 90% confidence interval.
Table 6-50: Confidence Intervals Associated with Program Gas Impact Estimates
Parameter Lower Bound (90%) Point Estimate Upper Bound (90%)
Biannium Savings per Home 2 therms 8 therms 15 therms
Percent Reduction 0.18% 0.61% 1.04%
Aggregate Impact 21.540 therms 74.579 therms 127619 therms
In the summer months, the estimated savings are low and in the case of August and September
of 2014, are slightly negative. However, it is important to note that despite the monthly
fluctuations in gas savings illustrated in Figure 6-18, the estimated gas savings are statistically
significant over the biennium27.
27 t = -2.91, P-value = 0.004
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 130
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 144 of212
6 RESIDENTIAL IMPACT EVALUATION
Figure 6-18: Average Monthly Gas Savings per Household with Relative Precision
Bounds
• Household Mean Monthly Savings -Upper Bound (90%) -Lower Bound (90%)
2.5
2
1.5
1
-1
<:I' <:I' <:I' 'SI' <:I' <:I' <:I' <:I' <:I' <:I' <:I' <:I' I.I') I.I') I.I') I.I') I.I') I.I') I.I') I.I') I.I') I.I') I.I') I.I') ,.... ,.... ,.... ,.... ,.... ,.... ,.... ,.... ,.... ,.... ,.... ,.... ,.... ,.... ,.... ,.... ,.... ,.... ,.... ,.... ,.... ,.... ,.... ,....
' i::, ' ' > ' ' rui 6. ' > u ' i::, ' ' > c ' rui 6. ' > ' C '-'-C :J .... C '-'-:J .... u ro QJ ro a. ro :J :J QJ u 0 QJ ro QJ ro a. ro :J :J QJ u 0 QJ -. u.. ~ <( ~ -. <( Vl 0 z 0 -. u.. ~ <( ~ <( Vl 0 z 0
6.10 Low Income
6.10.1 Overview
Avista's electric Low Income program offers a variety of conservation and fuel efficiency
measures to low income households. Avista leverages Community Action Program (CAP)
agencies to deliver energy efficiency programs to the Company's low income customer group.
CAP agencies have resources to income qualify, prioritize and treat homes based upon a
number of characteristics. In addition to the Company's annual funding, the Agencies have
other monetary resources that they can usually leverage when treating a home with
weatherization and other energy efficiency measures. The Agencies either have in-house or
contractor crews to install many of the efficiency measures of the program. Avista provides CAP
agencies with an "Approved Measure List" of energy efficiency measures. Any measure
installed on this list by the Agency in an income qualified home will receive 100%
reimbursement for the cost for the work.
6.10.2 Program Achievements and Participation Summary
Participation in the 2014-2015 Low Income program totaled 7,302 conservation and fuel
conversion projects. Table 6-51 summarizes the reported participation counts and energy
savings for the measures that make-up the Low Income program. Figure 6-19 presents the
energy savings for non-lighting conservation measures, lighting conservation measures, and the
fuel conversion measures. Non-lighting conservation measures account for 46% of the program
'-"Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 131
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 145 of212
6 RESIDENTIAL IMPACT EVALUATION
savings, with duct sealing and insulation measures accounting for 75% of this category, as
shown in Figure 6-20.
Table 6-51: 2014-2015 Low-Income Program Reported Participation and Savings
Measure Category
. . .
Non-Lighting Conservation
Non-Lighting Conservation
Non-Lighting Conservation
Non-Lighting Conservation
Non-Lighting Conservation
Non-Lighting Conservation
Non-Lighting Conservation
Fuel Conversion
Fuel Conversion
Lighting Conservation
Lighting Conservation
TOTAL
£-1Nexanr
2014-2015 2014-2015
Measure Reported Reported Savings Participation (kWh) Count
. . . .
ENERGY STAR Windows 53 1,418
ENERGY STAR Doors 60 17,241
Air Infiltration 113 58,337
Duct Sealing 97 132,290
ENERGY STAR Refrigerator 14 11,284
Water Heater 107
E to G Furnace Conversion 77 195,948
E to G Water Heat Conversion 60 71,421
E to G Heatpump Conversion 14 41 ,930
LI Giveaway CFL bulbs 4,526 72,855
LI Giveaway LED bulbs 2,123 27,599
7,302 758,955
Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 132
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 146 of212
6 RESIDENTIAL IMPACT EVALUATION
Figure 6-19: 2014-2015 Low Income Program Reported Energy Saving Shares: Measure
Category
41%
46% • Conservation Non-Lighting
• Conservation Lighting
• Fuel Conversion
13%
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 133
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 147 of212
6 RESIDENTIAL IMPACT EVALUATION
Figure 6-20: 2014-2015 Low-Income Program Reported Energy Saving Shares: Non­
Lighting Conservation
38%
17%
• Insulation
a ENERGY STAR W indows
a ENERGY STAR Doors
• Air Infiltration
• Duct Sealing
• ENERGY STAR Refrigerator
• Water Heater
6.10.3 Methodology
The evaluation team organized the analysis for the Low Income Program based on the
measures categories noted in Table 6-51 above. For the non-lighting conservation and fuel
conversion measures, the evaluation team employed a regression analysis. For the lighting
conservation measures, the evaluation team followed the same methodology as outlined in the
Residential Lighting Section (Section 6.7.3). The remainder of this section outlines the
methodology for the non-lighting conservation and fuel conversion measures.
The Low Income program operates as an electric-only program in Idaho with CAP Agencies
targeting electric savings opportunities. Participating homes generally received multiple
improvements so the electric savings values from all measures installed within a given home
were aggregated to arrive at the total reported savings for each home. The evaluation team
relied on a regression analysis of Avista billing data to estimate per-home impacts. Billing
analysis was determined to be an appropriate method because the average annual electric
savings claimed per participating home was almost 2,300 kWh across the 323 treated homes.
Next, homes were assigned to one of two groups for analysis:
1) Electric Conservation Homes -these homes had reported electric savings
.,..,Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 134
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 148 of 212
6 RESIDENTIAL IMPACT EVALUATION
2) Fuel Conversion Homes -these homes had reported electric savings and a negative
reported therm savings. This net gas penalty (and a large share of the electric savings)
resulted from a conversion of the homes heating or water heating system from electricity
to natural gas.
Figure 6-21 shows the distribution of per-home reported electric savings for the two groups.
Reported electric Impacts for the fuel switching homes were generally larger. Within the Electric
Conservation Homes there was a subset of residences that reported limited electric savings
because the primary improvements affected the gas heating system.
0 "q"
0 >-C') u C: Q) :J 0 0-N ~ u..
0 ~
0
0 Cl()
>-0 u CD C: Q) :J 0-0 ~ "q"
u..
0 N
0
Figure 6-21: Distribution of Reported kWh Values by Home Type
2,000
0 2,000
Fuel Switching Homes
4,000 6,000
Reported kWh per Home
Electric Conservation Homes
4,000 6,000
Reported kWh per Home
8,000
8,000
10,000
10,000
As described in Section 3.4.4, each home was matched to nearest weather station and
historical weather records were merged with historical consumption . Homes were required to
have at least 12 months of pre-retrofit and 12 months of post-retrofit billing data for inclusion in
the analysis. The evaluation team used a fixed effects panel regression model to establish the
average relationship between electric consumption and weather before and after service.
Separate models were estimated for fuel conversion homes and electric conservation homes
and both Idaho and Washington homes were used in the analysis to boost the precision of the
results. Regression coefficients were then applied to normal weather conditions (TMY3) for the
region to estimate weather-normalized annual electric savings. The regression coefficients and
relevant goodness of fit statistics are presented in Appendix C.
L-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 135
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 149 of 212
6 RESIDENTIAL IMPACT EVALUATION
The evaluation team also conducted a review of Avista's 2014 and 2015 tracking databases and
a document audit on 24 projects.
6.10.4 Findings and Recommendations
6.10.4.1 Non-Lighting Conservation and Fuel Conversion Homes
Table 6-52 summarizes the key inputs and outputs of the regression analysis. As expected the
fuel switching homes saved significantly more electricity on average than homes that did not
have a primary mechanical system converted from electricity to natural gas. The average
percent reduction in electric consumption for the 67 fuel switching homes analyzed was 55.7%,
meaning the post-retrofit electric consumption was less than half of what it was pre-retrofit.
Electric conservation homes used less electricity on average pre-retrofit than fuel switching
homes (13,278 kWh vs. 17,722 kWh). This group saved less on both an absolute and percent
basis.
Table 6-52: Low Income Billing Analysis Findings
Stratum Fuel Conversion Homes Electric Conservation Homes
Number of Homes Analyzed 67 165
Average Reported kWh per Home 3,909 1,233
Weather Normalized Annual kWh Pre-17,722 13,278 Retrofit
Weather Normalized Annual kWh Post-7,846 12,575 Retrofit
Average kWh Savings per Home 9,876 702
Realization Rate 253% 57%
Relative Precision ±9.2% ± 60.9% (90% confidence level)
Average Percent Reduction in Annual 55.7% 5.3% Electric Consumption
The realization rate for Fuel Conversion Homes was 253%, with homes saving an average of
almost 10,000 kWh annually. It is worth noting that the reported savings assumptions for electric
to gas conversion of heating and water heating in Low Income program were far more
conservative than the Fuel Efficiency program, which assumed 12,012 kWh for furnace
conversions and 4,031 kWh for water heater conversions. Evaluation results actually found a
higher per home impact from fuel switching in the Low Income program than in Fuel Efficiency
program although the difference was not statistically significant. Moving forward, the evaluation
team recommends that Avista align assumptions for fuel switching savings for the Low Income
and Fuel Efficiency programs.
'-"Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 136
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 150 of 212
6 RESIDENTIAL IMPACT EVALUATION
Figure 6-22 shows the evaluation teams estimates of the average Low Income home savings by
month for the last 13 months. Savings from the Low Income program are occurring primarily
during winter months when electric heating loads are highest. Figure 6-22 was created by
comparing the actual metered loads of homes (both fuel conversion and electric conservation)
to the regression estimates of what consumption would have been during the pre-retrofit period
using the actual weather conditions in place January 2015 through January 2016.
0 0 0
0 0 co
(/)
C) 0 C 0 ·;;: CD
ro U)
.c 0 ~ 0 -.;f"
0 0 N
0
2015m1
Figure 6-22: Low-Income Program Impacts by Month
2015m3
--1•--kWh Savings
2015m5 2015m7
Month
················· 90%CI
2015m9 2015m11
... ··
2016m1
6.10.4.2 Lighting Conservation
The 2014 and 2015 Low Income programs CAP agencies conducted multiple "giveaway" events
throughout the program cycle and reported bulb type (CFL/LED) and bulb count for each of the
events and the location of the event so that Avista could allocate the savings attributable to their
Washington and Idaho service territories. Based on the program reported data, the average
kWh savings attributed to the CFL bulbs was 16.1 kWh and 12.5 kWh for LEDs. Based on the
methodology outlined in Section 6.7.3 above, the evaluation team estimates the average
savings for the giveaway CFLs to be 18.7 kWhs and 20.9 kWhs for LEDs (assuming a 60w
equivalent). Table 6-53 presents the realization rate and per-unit gross verified savings.
c..-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 137
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 151 of212
6 RESIDENTIAL IMPACT EVALUATION
Table 6-53: Low-Income Lighting Conservation Measures Gross Verified Savings
Average Reported . . Gross Verified
Bulb Type Savings (kWh/bulb) Realization Rate Savings (kWh/bulb)
CFL G1veway 16.1 116% 18.7
LED Giveaway 12.5 167% 20.9
6.10.5 Program Results
The database review and document audit activities conducted by the evaluation team did not
result in any adjustments to the reported Avista savings values. The overall electric realization
rate for the Low Income program was 147%. This program level realization rate was developed
by taking a weighted average of the realization rates of the measure types shown in Table 6-54.
The relative precision of the program level electric realization rate was ±12.6% at the 90%
confidence level.
Table 6-54: Low-Income Program Gross Verified Savings
2014-2015 2014-2015
Reported Adjusted . . Gross Verified Measure Category P . . t· R d Realization Rate S . kWh art1c1pa 10n eporte avmgs ( )
Count Savings (kWh)
Conservation Non-Lighting 502 349,202 57% 199,045
Conservation Lighting 6,649 100,454 130% 130,730
Fuel Conversion 151 309,299 253% 782,526
TOTAL 7,302 758,955 147% 1,112,301
6.11 Residential Sector Results Summary
Table 6-55 lists the gross verified savings for each of Avista's residential programs in Idaho in
2014 and 2015 and for the overall portfolio. The Idaho electric residential sector achieved a 99%
realization rate and the relative precision of the program-level electric realization rate was
±9.05% at the 90% confidence level
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 138
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 152 of212
6 RESIDENTIAL IMPACT EVALUATION
Table 6-55: Residential Program Gross Impact Evaluation Results
2014-2015 ~;~·20: 2014-2015
Program Reported R Juste Realization Rate Gross Verified
Savings (kWh) Savi~::~:~h) Savings
Appliance Recycling
HVAC
Water Heat
ENERGY STAR Homes
Fuel Efficiency
Lighting
Shell
Opower
Low Income
TOTAL RESIDENTIAL
t1Nexanr
261,924 250,920 166% 416,524
872,828 872,828 60% 521 ,365
239,267 239,267 148% 354,675
140,538 140,538 123% 173,120
5,290,679 5,295,779 60% 3,198,893
8,323,842 8,323,842 126% 10,457,288
903,663 903,663 38% 345,048
2,746,000 2,746,000 102% 2,814,601
758,955 758,955 147% 1,112,301
19,537,696 19,531,792 99% 19,393,814
Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 139
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 153 of 212
----------------------------------·--·----·----------
7 Conclusions and Recommendations
7.1 Summary
The following outlines the evaluation team's conclusions and recommendations for Avista to
consider for future program processes and reporting. Additional details regarding the
conclusions and recommendations outlined here can be found in the program-specific sections
of this report.
7.2 Impact Findings
The evaluation team performed the impact evaluation for Avista's 2014 and 2015 Idaho electric
program through a combination of document audits, customer surveys, engineering analysis
and onsite measurement and verification (M& V) on a sample of participating projects. The
impact evaluation activities resulted in a 97% realization rate across Avista's 2014-2015
portfolio of programs (Table 7-1). Table 7-3 and Table 7-2 summarize Avista's 2014 and 2015
impact evaluation results by sector and program.
t.-1Nexanr
Table 7-1: 2014-2015 Idaho Electric Portfolio Evaluation Results
S t Reported Realization Gross Verified ec or . . Savings (kWh) Rate(%) Savings (kWh)
Residential 18,772,837 97% 18,281 ,513
Nonresidential 12,379,360 94% 11 ,687,224
Low Income 758,955 147% 1,112,301
PORTFOLIO 31,911,152 97% 31,081,038
Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 140
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 154 of 212
7 CONCLUSIONS AND RECOMMENDATIONS
Table 7-2: Idaho Electric Nonresidential Program Evaluation Results
Program
. . . .
Food Service Equipment
Green Motors
Motor Controls HVAC
Commercial Water Heaters
Prescriptive Lighting
Prescriptive Shell
Fleet Heat
Site Specific
TOTAL NONRESIDENTIAL
2014-2015 Reported
Savings (kWh) Realization Rate
2,387,662 90%
54%
43,954 54%
466,340 54%
190 54%
3,475,049 99%
54,381 54%
7,228 54%
5,813,610 99%
12,379,360 94%
2014-2015 Verified Gross
Savings (kWh)
; .
70,971
23,823
252,751
103
3,432,865
29,474
3,917
5,735,284
11,687,224
Table 7-3: Idaho Electric Residential Program Evaluation Results
2014-2015
Adjusted . . 2014-2015 Gross Program R d S . Reahzat,on Rate V ·t· d S . eporte avmgs eri ,e avmgs
(kWh)
Appliance Recycling 250,920 166% 416,524
HVAC 872,828 60% 521 ,365
Water Heat 239,267 148% 354,675
ENERGY STAR Homes 140,538 123% 173,120
Fuel Efficiency 5,295,779 60% 3,198,893
Lighting 8,323,842 126% 10,457,288
Shell 903,663 38% 345,048
Opower 2,746,000 102% 2,814,601
Low Income 758,955 147% 1,112,301
TOT AL RESIDENTIAL 19,531,792 99% 19,393,814
7.3 Conclusions and Recommendations
The following outlines the key conclusions and recommendations as a result of the evaluation
activities. Specific details regarding the conclusions and recommendations outlined here, along
with additional conclusions and recommendations can be found in the program-specific sections
of this report.
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 141
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 155 of212
7 CONCLUSIONS AND RECOMMENDATIONS
7 .3.1 Nonresidential Programs
The overall realization rate for the nonresidential portfolio is 94%. The realization rates ranged
from 99% for the Site Specific and Prescriptive Lighting programs down to 54% for the
"Prescriptive Non-Lighting Other" program. The Site Specific and Prescriptive Lighting programs
are the largest programs in the portfolio, together representing 75% of the portfolio's gross
verified savings. The evaluation team found that the processes Avista is utilizing for estimating
and reporting energy savings for the nonresidential programs are predominantly sound and
reasonable. The following subsections outline specific conclusions and recommendations for
several of the nonresidential programs.
7 .3.1.1 Site Specific Program
Conclusion: The Site Specific program constitutes almost 50% of the program energy shares.
Within the last 2 years, Avista has increased their level of quality assurance and review on
projects that participate through the program. The evaluation team's analysis resulted in a 99%
realization rate for the Site Specific program. The strong realization rate indicates that Avista's
internal process for project review, savings estimation, and installation verification are working
to produce high quality estimates of project impacts.
Recommendation: The evaluation team recommends that Avista continue to operate
this program with the current level of rigor. For interior lighting projects, Avista should
consider applying the interactive factors deemed by the RTF to quantify the interactive
effects between lighting retrofits and their associated HVAC systems. More specifically,
for interior lighting projects, Avista assumes a standard interactive factor of 7.7% for
buildings with air conditioning. The RTF's values for interactive factors vary depending
on heating and cooling system types and building type. For some building types,
especially those that tend to participate in the Site Specific program, the RTF's
interactive factors are higher than Avista's factor
Recommendation: While the impact from the Commercial Windows and Insulation
measures under the Site Specific program are minimal, Avista should further review its
algorithm for cooling season savings achieved by window replacements. The algorithm
that Avista currently uses may be overstating the impacts of these replacements on air
condition energy consumption.
7.3.1.2 Prescriptive Lighting Program
Conclusion: The Prescriptive Lighting program is the second largest program in Avista's
nonresidential portfolio, constituting 28% of the energy savings. The evaluation team's analysis
resulted in a 99% realization rate for the Prescriptive Lighting program, indicating that Avista's
reported energy savings for this program are accurate.
Recommendation: The evaluation team recommends that Avista continue to operate
this program with the current level of rigor. Avista should consider applying the
interactive factors deemed by the RTF to quantify the interactive effects between interior
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 142
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 156 of 212
7 CONCLUSIONS AND RECOMMENDATIONS
lighting retrofits and their associated HVAC systems. More specifically, for interior
lighting projects, Avista assumes a standard interactive factor of 7.7% for buildings with
air conditioning. The RTF's values for interactive factors vary depending on heating and
cooling system types and building type. For some building types, especially those that
tend to participate in the Site Specific program, the RTF's interactive factors are higher
than Avista's factor
7.3.1.3 EnergySmart Grocer Program
Conclusion: Avista's EnergySmart Grocer program is successfully providing retail and
restaurant customers with an avenue to upgrade their refrigeration equipment. Participation in
the program includes both prescriptive and custom projects. The evaluation team's review of
projects in the program resulted in a realization rate of 90%. For prescriptive projects, the
evaluation team determined that RTF deemed savings values were being appropriately applied
in most cases. However, low project-level realization rates for custom projects, which tend to be
larger in size than prescriptive projects, are driving the program realization rate downward .
Recommendation: Avista should consider more internal review of energy savings
estimates submitted by vendors for custom projects under this program. Alternatively,
Avista could consider tracking custom projects under the Site Specific program with
other projects of similar size and complexity.
7 .3.1.4 Prescriptive Non-Lighting Other Programs
Conclusion: Avista reported 2014-2015 participation in six other prescriptive programs. Of
these, the HVAC Motor Controls program is the largest, constituting 66% of the energy savings
for this group. The evaluation team's review of projects in these programs resulted in a 54%
realization rate. Cases of ineligible VFD projects receiving incentives were cause of the low
realization rate for these programs.
Recommendation: Avista should revise the HVAC Motor Controls program to include
more verification of motor eligibility status. More emphasis should be placed on
confirming motor application and duty status to ensure compliance with the program's
existing eligibility requirements. More specifically, Avista should place specific emphasis
on ensuring VFDs are installed in a manner that saves energy (i.e. not just as "soft
starters") and that incentivized VFDs serve primary-duty motors.
7.3.1.5 Small Business Program
Recommendation: It is recommended that the modified deemed savings values utilized by the
evaluation team be adopted by the program for future reporting purposes.
7.3.2 Residential Programs
The overall realization rate for the residential portfolio is 99%. The realization rates varied
significantly across the various programs evaluated with the Shell and Fuel Efficiency programs
having the lowest realization rate (38% and 62% respectively). The evaluation team found that
t.-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 143•
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 157 of212
7 CONCLUSIONS AND RECOMMENDATIONS
the reported savings for the majority of the programs were understating the actual impacts
found from the evaluation activities. The following subsections outline specific conclusions and
recommendations for several of the residential programs.
7 .3.2.1 Appliance Recycling
Conclusion: The evaluation team found that the reported deemed savings value (per recycled
unit) for the program was lower than estimated gross savings valued from prior studies. Avista
may have aligned their deemed savings values close to the RTF deemed savings values, but it
is important to understand that the RTF is reporting a value that accounts for net market effects
(i.e. free ridership).
Recommendation: If Avista choses to offer an appliance recycling program in the
future , it is recommended that a clear distinction between gross and net savings values
is noted if Avista reports the most current RTF values.
Conclusion: The evaluation team found discrepancies when comparing Avista's reported
participation counts against the implementer reported values. The evaluation team believes that
one reason for the discrepancies could be due to overlapping reporting periods and the way
participants are reported and tracked.
Recommendation: Avista should consider tracking the customer account number in
addition to the name/address. It would be easier to track account numbers back to billing
database records than the name /address fields, which are easier misspelled, and often
formatted differently.
7.3.2.2 HVAC Program
Conclusion: The evaluation team found, through billing regression analysis, a relatively low
realization rate for the Air Source Heat Pump measures (RR of 48.5%).
Recommendation: The evaluation team recommends Avista reexamine the
assumptions relating to annual per-home consumption and savings estimates in homes
receiving ASHP installations. In addition, to help better understand the baseline for the
ASHP replacement, Avista could consider requesting that contractors and customers
provide a better description of the replaced unit
Conclusion: For the analysis of the Smart Thermostat measure, only five homes had sufficient
post-retrofit billing data to estimate savings. Therefore, the evaluation team applied a 100%
realization rate to the reported savings due to the small population .
Recommendation: Given the inconclusive analysis results for this measure driven by
data limitations, the evaluation team recommends Avista revisit the analysis of this
measure in late 2016 -early 2017 when a full year of post-installation billing data is
available for several hundred rebate recipients.
t.-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 144
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 158 of 212
7 CONCLUSIONS AND RECOMMENDATIONS
7.3.2.3 Water Heat
Conclusion: For showerheads distributed through the Simple Steps program, Avista allocates
50% of its reported savings to electric savings and 50% to natural gas savings to account for
homes that have different water heating fuel types.
Recommendation: The evaluation team recommends Avista update this allocation
assumption to be based on representative water heater fuel type saturation. These data
are available through the Regional Building Stock Assessment study; however, we
recommend Avista base the allocation on data specific to its territory.
7.3.2.4 ENERGY STAR® Homes
Conclusion: The evaluation team initially attempted to use a difference-in-means approach to
estimate savings for the ENERGY STAR® Homes program. However, due to the small number
of ENERGY STAR® Homes participants and absent any detailed characteristics of the homes
(e.g. square footage, single-vs. multi-family, etc.) a reliable non-program comparison group
could not be attained. Therefore, the evaluation team collected Home Energy Rating System
(HERS) Index scores for participating ENERGY STAR® Homes wherever available to conduct
the impact analysis.
Recommendation: As more participants enter the program, the evaluation team
recommends again attempting a difference-in-means approach to estimating the savings
for the program, if sufficient data is available.
Recommendation: To aid future evaluation efforts, the evaluation team recommends
including the HERS scores in the program tracking documents. In addition, for stick-built
ENERGY STAR homes, application forms could ask for the RESNET Registry ID, which
is now assigned as part of RESNET Archival of all HERS Rated or ENERGY STAR
homes. This will ensure that the home has been certified third party and is recognized by
RESNET, the certifying agency for ENERGY STAR.
7.3.2.5 Fuel Efficiency
Conclusion: The evaluation team conducted a billing regression analysis for the Fuel Efficiency
participants and found realization rates of 57-62% for rebate projects that included the
conversion of a home's heating system from electricity to natural gas. When regression
coefficients were examined in detail, the evaluation team noted that the estimated reduction in
electric heating load was being offset by an increase in estimated base load within participating
homes.
Recommendation: Because the rebate amounts and per-home savings from Fuel
Efficiency are so large and the number of participants is relatively low, the evaluation
team recommends Avista ask participating customers for details on any additional home
renovations that were completed in parallel with the fuel conversion. Home improvement
projects such as an addition, finishing a basement, or adding air conditioning can
e.-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 145
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 159 of 212
7 CONCLUSIONS AND RECOMMENDATIONS
drastically change the consumption patterns within a home and render the assumed
baseline inaccurate.
Conclusion: The evaluation team found that over half the homes receiving Fuel Efficiency
rebates in 2014-2015 did not have a gas billing history with Avista prior to the conversion. These
homes realized savings at a higher rate than homes that did have previous gas service.
Recommendation: The evaluation team recommends that Avista consider adding a
field to the program tracking database that indicates the gas meter installation date or
service start date of participating homes. This would more clearly delineate homes that
were previously all electric and became dual-fuel around the same time as the Fuel
Efficiency project, from homes that had been dual-fuel historically. Avista may also want
to consider assuming a more conservative electric savings estimate for homes that had
prior gas service because it's possible that the home was not 100% electrically heated
prior to program participation .
7.3.2.6 Residential Lighting
Conclusion: Avista's deemed savings estimates, which were generally the same for all similar
product types and not correlated to the bulb wattage, understated the savings found by the
evaluation team. This was especially the case for Avista's CFL giveaway program .
Recommendation: The evaluation team recommends that Avista consider more
detailed product type deemed values in an effort to be more closely aligned with the
actual participating lamps. Simple Steps has shifted its program tracking to specific
product types by lumen bins in accordance with the most current BPA UES measure list.
Avista should consider using these higher resolution deemed value for internal reporting
with the Simple Steps program and for use with internal residential lighting programs.
An overarching recommendation is also for Avista to monitor the LED lamp market for
technology cost changes and customer preferences, and consider increasing LED lamp options
from the 2014-2015 portfolio in future DSM planning. Currently, LED prices are dramatically
decreasing and customer preferences are shifting from CFL to LEDs as a preferred choice as
an energy efficient technology. Consequently, CFLs shelf space share is declining as an
abandoned technology, despite its better cost effectiveness compared to LED lamps.
7.3.2.7 Shell Program
Conclusion: The evaluation team found a low realization rate (38%) for shell rebate measures
(windows and insulation). This finding indicates that reported savings values were too
aggressive on average. The evaluation team compared the end-use shares estimated via
regression analysis and found that only approximately 5,500 of the 13,000 kWh of average
annual consumption in residential homes in Avista's service territory was assigned to heating
and cooling load. Given this end-use share, the reported savings values claimed by Avista
equate to a 25% reduction in HVAC loads.
t.-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 146
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 160 of 212
7 CONCLUSIONS AND RECOMMENDATIONS
Recommendation: The evaluation team recommends Avista examine planning
assumptions about per-home consumption, end-use load shares, and percent reductions
in heating and cooling loads from shell improvements. It may be that the percent
reduction assumptions are sound, but they are being applied to an overstated
assumption of the average electric HVAC consumption per home. Conversely, the
assumed end-use shares may be accurate, but the end-use reduction percentage is
inflated. This investigation should be conducted separately for electrically heated homes
and dual fuel homes as the heating electric end-use share will be different.
7.3.2.8 Opower Program
Conclusion: The evaluation team found that savings held fairly consistent during the 6 month
interruption in Home Energy Report delivery. The finding reinforces Avista's decision to assume
a multi-year measure life when calculating the cost-effectiveness of the Opower program.
Recommendation: The evaluation team recommends Avista examine the program delivery
model in the 2016-2017 cycle. Given the fixed and volumetric nature of program costs,
measure life assumptions, and mechanisms by which measured savings are counted toward
goal achievement the evaluation team believes there are alternatives to the traditional
delivery model that optimize program achievements relative to costs. As an example, Avista
should consider not running the program during the second year of a biennium given the
constraints currently in place. Per the hypothetical example below, the acquisition cost
greatly increases in 2017 when a 2 year measure life with no decay is assumed.
Table 7-4: Opower Acquisition Cost Example
Annual A · · · cqu1s1t1on kWh per Program T H MWh C Incremental C Year x omes ost ost
Home Cost per MWh ($/kWh)
Home
2016 250 $15 50,000 12,500 $750,000 12,500 $0.06
2017 300 $15 46,000 13,800 $690,000 1,300 $0.53
7.3.2.9 Low Income Program
Conclusion: The evaluation team found a high realization rate for the fuel conversion measures
implemented through the Low Income program. One reason for the high RR could be due to the
fact that Avista caps the reported savings value to 20% of the contractor estimated savings. In
addition , the evaluation team found that the verified savings for these fuel conversion measures
aligned closely with the verified savings found through the regular-income
Fuel Conversion program.
Recommendation: The evaluation team recommends re-evaluating the current savings cap
for fuel conversion projects. In addition, we recommend that Avista align assumptions for
fuel switching savings for the Low Income and Fuel Efficiency programs.
'-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 147
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 161 of 212
8 Residential Lighting Study
In order to meet the objectives of the evaluation, the evaluation team collected data in the form
of onsite metering of lighting fixtures in the homes of Avista customers. The study methodology
chosen aligns with the Department of Energy (DOE) Uniform Measure Project (UMP) for
residential lighting. The research team measured how many hours per day various lighting
fixtures were illuminated during a six (6) month study period beginning July 2015 and lasting
through January 2016, at the residences of 74 Avista customers.
An average of seven (7) lamps per home were metered across a random sample of fixture and
room types, with 522 lighting meters deployed across Avista's service territory. Collecting data
for an average of seven lamps per residence allowed for a large dataset to be gathered for
analysis across multiple delivery streams, residence, and room types. Metered lamps included
both efficient lamps (CFLs and LEDs) and inefficient lamps (e.g. incandescents and halogens).
A full inventory of lighting (fixture, socket, lamp type, etc.) was also performed while onsite.
All recovered logger data was compiled into a dataset, analyzed, and summarized for hours of
use and peak coincidence estimation . Total hours per day was calculated from the
measurement results, which included ten-minute time intervals and the associated percent on
for that metered fixture. The hours of use was estimated for each logger across every day of the
metering period. This data was then weighted (by room type) to the inventory population and
regressed against a sinusoidal curve to develop an annualized estimate. This sinusoidal based
regression corrects for (annualizes) the metering period which spanned from July 2015 through
January 2016.
8.1 Methodology
8.1.1 Household Sampling Approach
To develop the sample frame, the evaluation team drew a stratified random sample of potential
participants from Avista Utilities' customer list. This list was used to recruit participants. The
sample was stratified by a proportional share of customer energy load in each state. Customers
consuming less than 2,000 kWh/ year were removed from the list of potential study
candidates2s. The sample frame was further stratified based on geographic region (ID-North, ID­
South , WA-North, WA-Central, and WA-South) and premise type (single family vs. multifamily).
The sample structure was designed to be representative of program participation and the
population at large, as practical. The representativeness controls the research team established
when recruiting participants in the study include:
Participation by geographic region (ID-North, ID-South, WA-North, WA-Central, and WA-South)
28 It is assumed that a typical customer home consumes at least 2,000 kWh per year. This control, therefore, will remove non-home
premises from the sample.
t.-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 148
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 162 of212
8 RESIDENTIAL LIGHTING STUDY
• Participation by dwelling type (single family vs. multifamily)
• Participation by household income level (low income vs. non-low income)
• Participation by geographic type (rural vs. urban)
• Participation by age of head of household
As outlined in the figures below, the evaluation team believes that the controls have been met to
ensure that the sample is representative of the population.
The evaluation team targeted 33% Idaho region (21% ID-North and 12% ID-South) and 67%
Washington region (9% WA-North, 11 % WA-South, and 47% WA-Central) participation in the
study. This split was based on the share of energy consumption by region . Figure 8-1 shows
that the actual split of participants was a representative 30% Idaho (19% ID-North and 11 % ID­
South) and 70% Washington (9% WA-North, 12% WA-South, and 49% WA-Central).
Figure 8-1: Actual Customer Participation by Region
49%
WA-South
12%
11%
9%
• ID-North
• ID-South
• WA-North
• WA-South
• WA-Central
Another important check to ensure a representative sample was to control for housing type
(single family vs. multi-family). We researched the current split of residents in the State of
Washington for these two housing types at 26% multi-family and 74% single family29; with the
State of Idaho researched to be 15% multi-family and 85% single family30_ Figure 8-2 shows
that the research team achieved a representative sample with 81 % single family and 19% multi­
family participants in Washington and 86% single family and 14% multi-family participants in
Idaho.
29 Based on 2015 U.S. Census data for the State of Washington -http://quickfacts.census.gov/qfd/states/53000.html
30 Based on 2015 U.S. Census data for the State of Idaho -http://quickfacts.census.gov/qfd/states/16000.html
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 149
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 163 of 212
----------------------------------------------
8 RESIDENTIAL LIGHTING STUDY
Figure 8-2: Actual Participation by Dwelling Type
Washington Single Family 81% Multi-Family 19°
w }. ' ' .\:' 'l',t, \i
Idaho . 0 Mult1-Famil Single Family 86 Yo • 'J 4%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
A third important factor we took into consideration, and monitored to ensure a proper
representative sample, was the household income level (low income vs. non-low income).The
State of Washington listed 13% within the low income range and 87% non-low income31 _
Similarly, the state of Idaho listed 16% within the low income range and 84% non-low income32_
Figure 8-3 shows that the research team achieved a representative sample with 13% low
income and 69% non-low income participants in Washington (17% of participants declined to
answer the survey question) and 14% low income and 77% non-low income participants in
Idaho, with 9% declining.
Figure 8-3: Actual Participation by Household Income
Washington
Idaho
,.. .,. ~" :: '
L~~-l~c~me Non-Low Income Declined
'',;% .14%". 77% !•9°/oi'~
iMf/;:~', 'l >/,t;;r,~
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Additionally, the evaluation team reviewed and incorporated the delineation of geographical
areas (urban vs. rural) into the sampled homes to further ensure a proper general population
representation. The customer counts within Avista's territory showed 53.6% of the population is
31 Based on 2015 U.S. Census data for the State of Washington -http://quickfacts.census.gov/qfd/states/53000.html
32 Based on 2015 U.S. Census data for the State of Idaho -http://quickfacts.census.gov/qfd/states/16000.html
t..1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 150
Exhibit No. 2
L. Roy, Avista Schedule 1, Page 164 of 212
8 RESIDENTIAL LIGHTING STUDY
considered WA-Urban, while 12.6% is WA-Rural, 23.2% is ID-Urban, and 10.6% is ID-Rural.
Figure 8-4 shows that the research team achieved a representative sample with 58 .1 % WA­
Urban, 12.2% WA-Rural, 23.0% ID-Urban, and 6.8% ID-Rural.
Figure 8-4: Actual Participation by Geographical Area
7%
• ID-Rural
• ID-Urban
• WA-Rural
58% • WA-Urban
Finally, evaluation team also conducted representativeness checks to ensure participants were
from a cross-section of age demographics. The age of the head of household (HOH) was
collected for each home visited. The distribution of study participants is provided in Table 8-1
and is reasonably representative of the age demographics for the States of Washington and
Idaho. 8.1 % of the homes visited declined to provide the age of their head of household, but
confirmed it was over the age of 18.
Table 8-1: Head of Household Age Participant Share
HOH Age Target Participation Actual Participation
by Age33 by Age
18 to 24 12.0% 1.4%
25 to 44 36.0% 23.0%
45 to 64 36.0% 41 .9%
>65 16.0% 25.7%
Declined 0.0% 8.1%
8.1.2 Logger Deployment Sampling Approach
Because the upstream and giveaway components of the Avista lighting program do not target
specific fixtures or high-usage areas in the home, the study metered an average of seven (7)
lamps per home across a random sample of fixture and room types in the homes of 74 Avista
33 Based on combined 2012 U.S. Census data for the State of Washington and the State of Idaho
'-"'Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 151
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 165 of 212
8 RESIDENTIAL LIGHTING STUDY
customers. Metered lamps included CFLs, LEDs, halogens, incandescent lamps and other
misc. lamps. The lighting study targeted annual operating hour results with 9% precision at the
90% confidence level for the 522 loggers successfully deployed in metered homes.
In addition to the controls mentioned above, the research team also sought to achieve
statistically meaningful results for multiple room types, as well as CFL/LED versus incandescent
operating hours. The study intended to place a higher proportion of loggers in high-use room
types (such as family/living room) to provide higher levels of statistical confidence for those
room types. The targeted sample frame of logger deployment by room type is illustrated in
below.
8.1.3
Table 8-2: Sample Frame of Logger Deployment by Room Type, by Bulb Type
# of Loggers
Room Type
CFL/LED Incandescent Total
Bathroom 20 19 39
Bedroom 45 45 90
Dining Room 35 34 69
Foyer/Hallway 20 20 40
Kitchen 35 34 69
Family/Living Room 45 45 90
Garage/Attic/Other 35 34 69
Other 35 34 69
TOTAL 270 265 535
Primary Data Collection
To accurately meet the objectives of this study the evaluation team designed an approach which
utilized a primary data collection approach in the form of onsite surveys & metering of customer
homes. Onsite surveys and metering provides highly accurate data because information is
collected and loggers deployed by trained engineers with experience identifying and properly
deploying metering equipment on lighting fixtures. The methods used to collect data through
onsite visits are detailed below.
8.1.3.1 Recruitment & Participant Criteria
1,500 general population Avista customers were contacted via a mailed letter (Appendix E) to
ask for their participation in the study. Recruitment letters (Appendix E) were mailed to the
sample frame customers. The letter introduced them to the study, and requested they call a toll­
free phone number to speak with an evaluation team representative if they were interested in
participating in the study, or had further questions. Participants were provided a $75 incentive to
participate in the study ($25 at the time of logger installation and $50 when the loggers were
collected) to participate in the study.
L-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 152
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 166 of212
8 RESIDENTIAL LIGHTING STUDY
8.1.3.2 Lighting Inventory
An inventory of all the lighting fixtures and lamps was performed while at each participant's
home. The purpose was not only to provide insightful saturation data on CFL, LEDs and other
lamps, but provided the necessary information to properly weight the hours of use data by room
type. Upon arrival at the home, the field engineer inspected each room and took a full inventory
of all the lighting circuits, fixtures and lamps. Data collected include:
• Circuit Type
• Room Type & Description
• Fixture type and quantity
• Socket type and socket quantity per fixture
• Lamp type, lamp shape, and lamp quantity per fixture
• Watts per lamp (when available)
The categorization utilized to identify fixture, socket and lamp types can be found in the Lighting
Inventory Form in Appendix C.
8.1.3.3 Measurement Activities
An average of seven (7) HOBO® on/off and light intensity data loggers were placed in each of
the 74 customer homes that participated. The data loggers utilized for this study include:
• HOBO UX90-002 Light On/Off
• HOBO U9-002 -Light On/Off
• HOBO U12-012 Temp/RH/Light Intensity
The light on/off loggers simply measure on-off luminosity events that exceed a pre-set
threshold, while the intensity logger measures incremental changes in luminosity. While all
loggers can be calibrated to accurately record data in any setting, the on/off loggers were
targeted for deployment in low ambient lighting settings, while the intensity loggers were
targeted for deployment in high ambient lighting settings. HOBO UX-90 light pipes were also
deployed to help ensure the logger sensors were more effectively recording lamp luminosity,
and not ambient light changes.
The location of loggers placed on the various fixtures and rooms in each home was determined
by a random sampling methodology that was programmed into a smart phone randomizer
application ("app") developed by the evaluation team that deterred the field engineer from
introducing any bias into the where the loggers were deployed. The randomizer app required
the field engineer to enter in the number of lighting circuits34 in a home and identify which ones
34 For the purposes of this study, a circuit is defined as the series of one or more lights controlled by a single switch (e.g. wall
switch). By using circuits as the selection criteria, as opposed to fixtures, the research team was able to collect unique data sets (as
logging data for more than one fixture on a single circuit would provide duplicate results).
t..1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 153
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 167 of212
8 RESIDENTIAL LIGHTING STUDY
had a CFL or LED installed on it; at which point a random sample of lighting circuits would be
provided to the engineer. The field engineer then installed the lighting loggers on one fixture for
the identified circuits. In order to obtain as much data as possible on CFLs and LEDs, the
randomizer app was programmed to automatically include up to four (4) circuits that had
CFL/LED lamp fixtures. The remaining circuits were then randomly selected for the remaining
loggers. Additionally, the sampling algorithm confirmed compliance with the overall target
sample frame to ensure representativeness of the general population with respect to room type.
When room type quotas were reached, the evaluation team engineers refrained from installing
any additional loggers in that room type.
In order to fully estimate the changes in daily operating schedules, the research team sought to
have loggers deployed at least one month in each season (summer, fall and winter). Based on
the delivery schedule of this study, the evaluation team began its six-month metering duration in
July/August 2015 and retrieved all the loggers in January 2016.
8.1.4 Data Analysis
8.1.4.1 Data Cleaning
After removal of the loggers in January 2016, analysts downloaded logger data using
HOBOware software and imported the data into STAT A for generating summary statistics, data
cleaning , hours of use and peak coincidence factor estimation. The research team also
reviewed logger notes documented by the removal team to determine whether to include or
exclude each logger from the HOU analysis. Based on these removal notes, analysts
determined loggers to be excluded from the HOU analysis based on the following
circumstances:
• Participants prematurely removed loggers from metered fixtures
• Participants didn't respond to repeated requests by research team to pick up loggers
• Loggers were damaged at the customer home
• Logger malfunction (e.g. battery) led to incomplete dataset
• Field Engineer didn't correctly "launch" logger during installation
• High ambient light conditions resulted in poor data quality
Initial review of the logger data for viability and outlier behavior was a two-step process based
on the logger type: for intensity loggers the data was exported into histograms for review while
event loggers (on/off events) were reviewed by STATA code. Analysts reviewed all raw intensity
logger data using histograms exported into Excel, specifically targeting minimum thresholds for
what would qualify as a light-on event specific to each logger. Loggers with very low or very high
intensity readings or reading that appeared suspect were reviewed further; ultimately nine
loggers were removed from the analysis due to questionable intensity readings.
Loggers flagged as questionable by the removal team (e.g., the participant removed the logger,
the logger fell off the fixture, poor installation, etc.) were carefully reviewed to ensure that data
'-"'Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 154
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 168 of 212
8 RESIDENTIAL LIGHTING STUDY
represented in situ observations. As poor logger installation did not always result in bad data,
some data from improperly installed loggers were included in the analysis. Some loggers were
immediately coded as "remove" if they recorded data for only a small fraction of metering period
(less than one month of data points), the loggers were damaged, and other anomalies.
To provide a general quality control check, analysts wrote the STATA program to "trim" data
points occurring before or on the day of the install date or on the day or after the removal date.
This check prevented analysis from including events occurring prior to installation, in case a
technician did not reset the logger at the time of installation. The check also prevented the
analysis from including events occurring after the removal date, if logger data were downloaded
on a day other than the removal date.
Once the light logger data was completely cleaned , the data was merged with the household
lighting audit data collected during logger installations. Table 8-3 shows the distribution of total
loggers retained for final analysis (loggers with viable data) by room type. After data cleaning, a
total of 459 loggers were available for the hours of use and coincident factor analysis.
Table 8-3: Distribution of Loggers Installed by Room with Viable Data
Room Loggers with Viable Data35
Kitchen 61
Dining 33
Living/GreaUFamily 79
Foyer/Hall/Stair 42
Bedroom 77
ToileUBathroom 48
Other 119
TOTAL 459
8.1.4.2 Development of Weights
The total number of lamps metered with a data logger was weighted back to the inventory
population based on two primary criteria: 1) the data was weighted to match the entire inventory
sample population's distribution of total lamps by room type, and 2) the entire inventory sample
populations' distribution of total lamps by source of efficient light bulbs (delivery stream).
Population weights were developed by calculating the inverse of a lamp's probability of being
metered with a data logger. This resulted in a different weight for each combination of room type
and source of efficient light bulb, and renders the logger-based lamp sample frame equivalent to
a simple random sample. Table 8-4 shows the population weights calculated using the
inventory-based, and logger-based, lamp counts.
35 This represents the number of loggers included in the analysis after data cleaning.
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 155
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 169 of 212
8 RESIDENTIAL LIGHTING STUDY
Table 8-4: Population Weights Applied to Sample Frame
Inventory-based Logger-based Population
Room Lamp Lamp Count Lamp Count Weight
(A) (B) (A / B)
i CFL 93 54 1.7
Kitchen I Incandescent 316 95 3.3
i LED 94 31 3.0 ! !
! CFL ! 23 18 1.3
Dining I Incandescent 190 89 2.1
[ LED 25 22 1.1
I CFL 155 53 2.9
i Incandescent 326 70 4.7 ! Living/Great/Family
LED 49 11 4.5
i CFL 55 21 2.6 I
Foyer/Hall/Stair j Incandescent 223 33 6.8
I LED 13 8 1.6
i CFL ! 182 50 3.6
Bedroom I Incandescent 432 77 5.6
LED 42 4 10.5
i CFL 144 55 2.6 !
Toilet/Bathroom I Incandescent 461 73 6.3
I LED 24 3 8.0
! CFL 276 83 3.3
Other l Incandescent 753 108 7.0
I LED 26 4 6.5
8.1.4.3 Hours of Use Modeling
Estimates of HOU were developed by first annualizing the logger data, and then applying a
hierarchical linear model. The logger data was annualized to simulate a full year of data for
loggers that were installed for part of the year. The hierarchical linear model was applied, with
the population weights, to estimate HOU with standard errors that reflect the structure of the
sample.
8.1.4.4 Development of Annualized HOU
Residential lighting usage, both frequency and duration-based, is partly a function of ambient
daylight. Lamps used in rooms without access to daylight (closets, basements, and other
windowless rooms), along with lamps with usage independent of daylight (lights on timers or
lights turned on when home from work), can be classified as "base load" lights. Overall, HOU for
homes is based on this base load usage, combined with usage dependent on hours of daylight.
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 156
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 170 of 212
8 RESIDENTIAL LIGHTING STUDY
Overall usage, therefore, fluctuates over the course of a year given fluctuations in daylight
hours.
The average HOU for all lamps during the summer solstice (beginning June 21) is expected to
be the lowest of the year, while HOU usage during winter solstice (beginning December 21) to
be the highest of the year. Average annual use is assumed to be coincident with the spring and
fall equinox, occurring on March 20 and September 22, respectively. For example, the fraction
of the daily percent difference from the average annual daylight hours across one year is
represented as a sinusoid curve. This curve can be represented by the equation
sin(-2rr(284+d)/365), where d is the Julian date of the year (January 1 = 1, December 31 = 365).
Figure 8-5, the peak and trough (at 1 and -1 , respectively) represent the winter and summer
solstices, and O represents the spring and fall equinoxes (effectively the annual average daylight
hours).
Figure 8-5: Percent Deviation from Average Annual Daylight Hours
2 ~---------------------------
cu Winter
bl) 1.5 Solstice nl ...
11.1 > ct 1
E
0 ... 0.5 -:::> C: 0 .2 :c .... -0 .~ nl > ::s 150 200 350 cu C: C c: .... ct -0.5
C: cu u Equinox ... -1 Equinox cu C.
~ 'iii -1.5 +--------------+---------------c
Julian Day of Year {1-365)
Light logger data were collected during a six-month period starting July 2015 and removed from
the homes in January 2016. Basing HOU on these data alone would result in a low estimate, as
lighting HOU and daylight hours are inversely related . In other words, HOU should increase with
decreasing daylight. Annualization of the spring and summer-only HOU estimate was required
to adjust this HOU to an annual value.
The basis for the HOU annualization is the UMP Chapter 6: Residential Lighting Evaluation
Protocol36_ According to the UMP: "Due to the seasonality of lighting usage, logging should be
36 The Uniform Methods Project Methods for Determining Energy Efficiency Savings for Specific Measures, Scott Dimetrosky, Apex
Analytics LLC. April, 2013. https://www1 .eere.energy.gov/wip/pdfs/53827-6.pdf
L-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 157
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 171 of 212
8 RESIDENTIAL LIGHTING STUDY
conducted in total for at least six months and capture summer, winter, and at least one shoulder
season -fall or spring . At a minimum, loggers should be left in each home for at least three
months (that is, two waves of three months each to attain six months of data). All data should be
annualized using techniques such as sinusoidal modeling to reflect a full year of usage." The
UMP goes on to discuss the sinusoid regression: "Sinusoidal modeling assumes that hours of
use will vary inversely with hours of daylight over the course of a year. Sinusoid modeling shows
that (1) hours of use change by season, reflective of changes in the number of daylight hours
and weather and (2) these patterns will be consistent year to year, in the pattern of a sine wave.
An example of this approach is provided in the evaluation of the 2006 -2008 California
Upstream Lighting Program evaluation."
A sinusoid curve, best representing annual changes in daylight hours, was then statistically fit to
weekend and weekday logger data using the following equation:
Where:
HOU
e
d
/Jo
E
Equation 8-1: Sinusoidal Model Specification
HOUd = /30 + /31 sin (Jd + Ed
= hours of use;
= angle, in radians, representing the amount of sunlight on the day. Theta
is -for the spring and autumnal equinoxes, pi/ 2 for the winter solstice,
and -pi / 2 for the summer solstice;
= the day of the year;
= the intercept, representing the annual average HOU estimate (which
coincides with the spring and fall equinox);
= coefficient representing the difference between the HOU on the solstice
and the average HOU (maximum amplitude of the curve); and
= error term.
For the Avista HOU lighting analysis, the evaluation team leveraged this sinusoid model to
calculate the adjusted average annual HOU, based on the available logger data. We used
separate models for weekday and weekend data, and regressed mean daily use for the relevant
days in the metering period on the sin(ect) associated with those days. Drawing on methodology
used in the Pennsylvania 2014 Commercial & Residential Light Metering Study37, a sinusoidal
model was deemed to have a poor fit if one of the following criteria was met:
1) /31 has an absolute value greater than 1 O;
2) The standard error for /31is greater than 1;
3) /Jo is less than or equal to O; and
4) /Jo is greater than 24.
37 Pennsylvania Statewide Act 129 2014 Commercial & Residential Light Metering Study. Prepared by the Pa Statewide Evaluation
Team; GDS Associates, Nexant, Research Into Action, Apex Analytics. January 13, 2014.
L-'1 Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 158
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 172 of 212
8 RESIDENTIAL LIGHTING STUDY
Based on the above criteria, 37 of 916 sinusoidal models were identified as poorly fit. Those 37
represented 30 loggers (because weekend and weekday data was modeled separately, a single
logger had two sinusoidal models associated with it). Rather than using the fitted values for
those 37 models, the average HOU from the logger data was used to estimate annual HOU.
8.1.4.5 Hierarchical Model
A weighted hierarchical (or multilevel) model was developed to estimate average HOU for the
home.38 The key advantage of the hierarchical approach is that the model takes into account in­
home lighting usage covariance in estimating coefficients. This is important as lighting across
multiple loggers in the same home are likely to have some covariance associated with the
usage behavioral patterns of the home's occupants. For instance, during an extended vacation ,
nearly all of the lights in the home may be off, and all of those loggers would record zero usage
during those same dates.
The model includes random effects for the intercept at the household level, which accounts for
correlation among loggers within a home. To estimate HOU for various categories such as room
type, lamp usage category and fixture type, fixed effects variables were included in the model.
The specification shown in equation 2 below features fixed effects for room type, but the model
takes a similar form for other categories.
Where:
Equation 8-2: Hierarchical Linear Model for HOU
HOUh,i = (/Jo + ho,h) + L flrlr + Eh,i
r
HOU = hours of use
bo,1, -N( b;,, (J2u~
h = index for home
i
r
Ir
/Jx
bo,
= index for logger
= index for room type
= indicator variable for room type
= fixed effects coefficients
= random effects coefficients
E = error term
8.1.4.6 Coincidence Factor Modeling
Avista has three peaks for which coincidence factors were calculated: a summer peak from 5 to
6.30 PM, a winter peak from 7 to 8 AM, and a winter peak from 5 to 6 PM. For each peak, the
coincidence factor is average percent of the hour lights are on during the defined peak period of
non-holiday weekdays.
38 Hierarchical models are described very briefly here. For further details, refer to the following: Woltman, Feldstain, MacKay, and
Rocchi, An introduction to hierarchical linear modeling; Goldstein, Harvey, Multilevel Statistical Models; and Sullivan, Dukes, and
Losina, Tutorial in Biostatistics: An Introduction to Hierarchical Linear Modeling.
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 159
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 173 of212
8 RESIDENTIAL LIGHTING STUDY
Since loggers were in place for nearly an entire summer period (July through September), and
nearly an entire winter period (November through January and, in many cases, some part of
February), sinusoidal model estimates were not used in the estimated CF. Average CF was
computed for each peak period for each logger and then a hierarchical model was developed to
estimate CF. The model has a similar form to that used to estimate HOU, featuring random
effects for the intercept at the household level, which accounts for correlation among loggers
within a home. To estimate CF for various categories such as room type, lamp usage category
and fixture type, fixed effects variables were included in the model. The specification shown in
equation 3 below features fixed effects for room type, but the model takes a similar form for
other categories. The CF during each of the three peak periods was estimated separately using
the same specification.
Where:
CF
Equation 8-3: Hierarchical Linear Model for HOU
CFh,i = (Po + bo,h) + L Prlr + Eh,i
r
= coincidence factor during a particular peak period;
bo,h -N( b11, o-2/j,);
h = index for home
i = index for logger
r
J,.
/Jx
bo,h
E
= index for room type
= indicator variable for room type
= fixed effects coefficients
= random effects coefficients
= error term
8.2 Lighting Inventory Findings
An important part of the residential HOU study is the collection of bulb saturation data across
the homes that participated in the study. Saturation studies are useful tools to help gauge the
market penetration of efficient lighting products to assess past program effectiveness and to
determine future potential for continued lighting program efforts. Additionally, collecting
supplemental information about each user and home of where the bulbs were installed allows
segmenting the analysis to frame and design future programs to target these areas of highest
potential.
There were a total of 3,902 lighting sockets reviewed based on the 74 homes surveyed (or an
average of 53 sockets per home). The evaluation team found CFL socket saturation to be
23.8% and LED saturation at 7.0%. The combined less efficient (non CFL/LED) 69.2% bulb
saturation can be viewed as the maximum available potential for future CFL and/or LED
installations.
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 160
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 174 of212
8 RESIDENTIAL LIGHTING STUDY
Table 8-5: Lighting Inventory Summary Saturation by Lamp Type
Lamp Type Total Bulbs Lamp Distribution %
CFL 928 23.8%
Empty Socket 71 1.8%
Halogen/Quartz 152 3.9%
Incandescent 2102 53.9%
LED 273 7.0%
Linear Fluorescent 353 9.0%
Other 6 0.2%
N/A 17 0.4%
TOTAL 3,902 100%
8.2.1 CFL & LED Saturation by Room Type
Knowing which rooms have the most CFL and LED lamps installed helps to understand how
consumers are using and installing energy efficient bulbs. Table 8-6 shows the CFL and LED
saturation by room type, with living/greaUfamily room type having the highest CFL saturation
(29.2% CFL saturation), whereas dining rooms have the lowest CFL saturation (9.7%). Kitchens
had the highest LED saturation (18.7%) and "Other" room s had th e lowest LED saturation
(2.5%). Figure 8-6: Lighting Inventory Summary of Room and Lamp Type shows the complete
lighting inventory represented by room and lamp type.
Table 8-6: Lighting Inventory Summary CFL Saturation by Room Type
CFL LED Room Type Total Bulbs CFLs S LED aturation Saturation
Kitchen
Dining
Living/Great/Family
Foyer/Hall/Stair
Bedroom
Toilet/Bathroom
Other
TOTAL
t.-1Nexanr
503 93 18.5% 94 18.7%
238 23 9.7% 25 10.5%
530 155 29.2% 49 9.2%
291 55 18.9% 13 4.5%
656 182 27.7% 42 6.4%
629 144 22.9% 24 3.8%
1055 276 26.2% 26 2.5%
3,902 928 23.8% 273 7.0%
Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 161
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 175 of 212
8 RESIDENTIAL LIGHTING STUDY
Figure 8-6: Lighting Inventory Summary of Room and Lamp Type
Other
Toilet/Bathroom
Bedroom
Foyer/Hall/Stair
Living/Great/Family
Dining
Kitchen
1
0
•
200 400
, ... ,
I
I
I
600
Number of Lamps
I
I
800 1000 1200
•Incandescent • CFL •Halogen/Quartz • Other • Empty Socket • LED Linear Fluorescent • N/A
8.2.2 CFL & LED Saturation by Socket and Circuit Type
As shown in Table 8-7 the majority (76.8%) of the sockets are medium screw based bulbs,
followed by pin based bulbs (10.4%). CFL saturation is highest for the medium screw based
fixtures (30.2%) and LED saturation is highest for the "Other" socket type at 40.8%. Also shown
below in Table 8-8 is the majority (86.7%) of circuits are represented by the standard on/off
switch. If remote control and other circuits are excluded (since there were only 4 total circuits
represented in this study) circuits with dimmer capabilities have the lowest CFL saturation
(7.6%) and timers have the lowest LED saturation (2.6%).
Table 8-7: Lighting Inventory CFL Saturation by Socket Type
S Socket Type CFL Saturation LED Saturation Socket Type Total ockets 0. . " 1stnbution % 010
Medium Screw Base
(standard)
Small Screw Base
(candelabra)
Pin Base
Other
TOTAL
t.-1Nexanr
2,998 76.8% 30.2% 5.8%
353 9.0% 2.0% 1.1%
404 10.4% 2.5% 8.4%
147 3.8% 4.1% 40.8%
3,902 100.0% 23.8% 7.0%
Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 162
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 176 of212
8 RESIDENTIAL LIGHTING STUDY
Table 8-8: Lighting Inventory CFL Saturation by Circuit Type
Total Sockets c· . T CFL LED
T I C. . c· . 1rcu1t ype Circuit Type ota 1rcu1ts per 1rcu1t D" t .b t· S t t· 0, S t t· o,
G 1s ri u 10n a ura 10n ,o a ura 10n ,o
3-way
Dimmer
Motion/Photo Sensor
On/Off (switch, plug,
string)
Other
Remote Control
Timer
N/A
TOTAL
59
76
26
1460
3
23
36
1,684
roup
198
302
42
3238
2
6
38
76
3,902
3.5% 8.1% 12.1%
4.5% 7.6% 4.3%
1.5% 19.0% 4.8%
86.7% 26.4% 7.2%
0.1% 0.0% 0.0%
0.2% 0.0% 0.0%
1.4% 31 .6% 2.6%
2.1% 19.7% 0.0%
100.0% 23.8% 7.0%
8.2.3 CFL & LED Saturation by Housing Type and Ownership Status
Multi-Family homes have the highest CFL saturation (close to 33%) while mobile homes had the
highest LED saturation at 14.2% (though the level of confidence in this estimate is low since
there were only 5 mobile homes in the sample). Interestingly, CFL saturation was the highest in
rental households (38.3%) while LED saturation was highest in owner-occupied households
(7.7%).
Table 8-9: Lighting Inventory CFL Saturation by Building Type
Number of . .
B -1 · T H . N b f S k t CFL Saturation LED Saturation u1 dmg ype omes m um er o oc e s % %
Sample
Mobile Home 5 218 25.2% 14.2%
Multi-Family (3+ Units) 10 167 32.9% 0.6%
Single Family (1 unit) 57 3,450 23.3% 7.0%
Single Family Attached (2 2 67 19.4% 0.0%
units)
TOTAL 74 3,902 23.8% 7.0%
Table 8-10: Lighting Inventory CFL Saturation by Ownership Type
Number of . . . . CFL Saturation LED Saturation Ownership Status Homes m Number of Sockets % %
Sample
Own
Rent
N/A
TOTAL
t-1Nexanr
56 3,460 21.9% 7.66%
16 376 38.3% 1.06%
2 66 37.9% 6.06%
74 3,902 23.8% 7.00%
Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 163
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 177 of212
8 RESIDENTIAL LIGHTING STUDY
8.2.4 CFL & LED Saturation by Region
Table 8-11 shows the CFL and LED saturation by region. The Avista region with the highest
CFL and LED saturation was WA-Central with 30.3% and 10.7% respectively. The region with
the lowest CFL saturation was WA-North (10.1 %), while WA-South had the lowest LED
saturation (1 .2%).
Table 8-11: Lighting Inventory CFL Saturation by Region
R . Homes in N b f S k t CFL Saturation LED Saturation eg1on S I um er o oc e s 0, 0, ampe ,o ,o
Idaho 22 1,231 18.8% 4.8%
ID-North 14 648 18.4% 5.1%
ID-South 8 583 19.2% 4.5%
Washington 52 2,671 26.1% 8.0%
WA-North 7 317 10.1% 3.5%
WA-South 9 514 20.8% 1.2%
WA-Central 36 1840 30.3% 10.7%
TOTAL 74 3,902 23.8% 7.0%
8.2.4.1 Program Participation & Misc. Saturation Findings
While onsite, evaluation team engineers asked homeowners if they recall receiving free light
bulbs from Avista from the Avista light bulb give-away program. Table 8-12 shows that
percentage of participants that recall receiving the free light bulbs. We also found that of those
customers that recall receiving a free light bulb, 100% of them installed the free light bulb.
Table 8-12: Free CFL Program Participation Findings
B "Id" T Total Homes % of homes that recall m mg ype . . . . . V1s1ted rece1vmg free lights
Mobile Home 5 80.0%
Multi-Family (3+ Un its) 10 40.0%
Single Family (1 unit) 57 56.1%
Single Family Attached (2 units) 2 50.0%
TOTAL 74 55.4%
Engineers also recorded information on household space heating and space cooling equipment,
as well as asked them about the number of portable electronics in the household. The research
team found that 81 % of households have a furnace to provide their space heating needs, while
54% of households use a central A/C systems for space cooling (Table 8-13 and Table 8-14).
5.4% of households were found to have no space cooling equipment present.
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 164
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 178 of 212
8 RESIDENTIAL LIGHTING STUDY
Table 8-13: Space Heating Equipment Saturation
Space Heating . . . T Households Equipment Count Saturation Equipment ype
Baseboard 8 8 10.8%
Furnace 60 60 81.1%
Other 4 4 5.4%
N/A 2 2 2.7%
TOTAL 74 74 100.0%
Table 8-14: Space Cooling Equipment Saturation
Space Cooling . . T Households Equipment Count Fuel Share % Equipment ype
Central A/C 40 40 54.1%
Fan 3 3 4.1%
other 4 4 5.4%
WindowA/C 23 23 31 .1%
None 4 4 5.4%
TOTAL 74 74 100.0%
The share of households that use natural gas as their primary space heating fuel was estimated
at 68.9%, while the share of households that utilize electricity as their primary space heating fuel
was estimates at 24.3% (Table 8-15). The research team also asked the participants to estimate
the number of portable electronics in their household -and found the average number of
portable electronics per household to be 3.7.
Table 8-15: Space Heating Fuel Share
Space Heating Fuel Type Households Fuel Share %
Electric 18 24.3%
Gas 51 68.9%
Oil 2 2.7%
Pellets 1.4%
Wood 1.4%
N/A 1.4%
Total 74 100%
8.3 Lighting Hours of Use Findings
8.3.1 Aggregate Hours of Use
The overall daily lighting hours of use (HOU) annualized across the entire year is estimated to
be 1.94. This value is estimated with a 90% confidence and 15.3% precision. Given a calculated
L-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 165
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 179 of212
8 RESIDENTIAL LIGHTING STUDY
0.18 standard error, the research team estimates this annualized daily HOU value could be as
low as 1.64 hours/day or as high as 2.23 hours/day.
Table 8-16: Aggregate Lighting Socket Hours of Use
Standard Precision (90% Lower
HOU Estimation Mean HOU f'd ) L' . Upper Limit Error con I ence 1m1t
Hierarchical Estimate,
Clustered SE 1.94 0.18 15.3% 1.64 2.23
The predicted and actual aggregated hours of use from August 81h, 2015 through January 101h,
2016 is displayed in Figure 8-7 below.
Figure 8-7: Aggregate Hours of Use Actual and Annualized Estimate
:J
0 I
(Y)
N
0&'01/201 S
•
•
11/01/201S
•
02/01/2016 Date 05/01/2016
• Actual Weekday • Actual Weekend
• Predicted Weekday • Predicted Weekend!
06'01/2016
8.3.2 Hours of Use by Lamp Type
The evaluation team also investigated the differences between bulb types within the homes
metered. Higher efficiency bulbs such as CFLs and LEDs showed considerably higher overall
hours of use (2.21 and 3.37, respectively) relative to inefficient bulbs such as incandescents
(1.69). The results are statistically significant as found in Table 8-17.
'-''1 Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 166
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 180 of 212
8 RESIDENTIAL LIGHTING STUDY
Table 8-17: Hours of Use by Lamp Type
Standard Precision (90% Lamp Type (Logger Level) Mean HOU t· Lower Limit Upper Limit Error con 1dence)
CFL 2.21 0.22 16.8% 1.84 2.58
Incandescent 1.69 0.18 17.7% 1.40 1.99
LED 3.37 0.77 37.7% 2.10 4.64
8.3.3 Hours of Use by Room Type
Finally, the team investigated the differences in lighting hours of use across various room types.
Kitchens were the highest HOU, with well above 3 hours/day, relative to bedrooms and
foyer/hall/stairways, which are lower-use rooms Uust over 1 hour/day). The research team also
calculated the estimated hours of use by high/moderate and low usage room types. The results
are and presented in Table 8-18 and Table 8-19 respectively.
Table 8-18: Hours of Use by Room Type
Room (Logger level . . . ' Annualized Room-Standard Prec1s1on (90% Lower Upper
weighted by event B d HOU E t·d ) L" · L" · ase rror con I ence 1m1t 1m1t type)
Kitchen 3.75 0.45 19.57% 3.02 4.48
Di:1ing 2.48 0.55 36.43% 1.57 3.38
Living/Great/Family 2.41 0.31 21 .31% 1.90 2.93
Foyer/Hall/Stair 1.25 0.37 49.09% 0.63 1.86
Bedroom 1.25 0.18 23.08% 0.97 1.54
Toilet/Bathroom 1.82 0.30 27.46% 1.32 2.32
Other 1.52 0.25 26.53% 1.12 1.92
Table 8-19: Hours of Use by Room Usage Type
Precision
Room Usage Type HOU Standard (SO°/c Lower Upper
(Logger level) N Error . 0 Limit Limit confidence)
High Use 314 3.03 0.31 16.58% 2.53 3.53
Moderate Use 606 1.66 0.20 19.68% 1.33 1.98
Low Use 42 0.36 0.36 166.90% -0.24 0.95
8.3.4 Peak Coincidence
To calculate the peak coincidence factor (CF), the team used the same clean light logger
dataset used for HOU estimates. Analysts calculated the peak coincidence factors based on the
peak period: a summer peak from 5 to 6.30 PM, a winter peak from 7 to 8 AM, and a winter
peak from 5 to 6 PM. Average CF was computed for each peak period for each logger and then
a hierarchical model was developed to estimate CF.
L-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 167
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 181 of212
8 RESIDENTIAL LIGHTING STUDY
The weighted peak coincidence factor for Avista's peak period is 10.2% (Table 8-20). The CF
for the winter 7-8am was calculated at 8.0%, while the 5-6pm winter peak CF was calculated at
14.4% and the 5-6:30pm summer peak CF is estimated at 9.1 %.
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 168
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 182 of212
8 RESIDENTIAL LIGHTING STUDY
Table 8-20: Lighting Coincident Factor by Peak Period
Precision Standard Lower Upper Peak CF Estimation N CF (90% L" . L" . Error f"d ) 1m1t 1m1t con I ence
Winter, 7-8 AM Hierarchical Estimate, 962 8.0% 0.01 22.74% 6.2% 9.8% Robust SE
Winter, 5-6 PM Hierarchical Estimate, 962 14.4% 0.01 14.91% 12.3% 16.6% Robust SE
Summer. 5-6.30 Hierarchical Estimate, 962 9.1% 0.01 18.73% 7.4% 10.8% PM Robust SE
Weighted Average Hierarchical Estimate, 962 10.2% 0.01 15.14% Robust SE
The evaluation team also estimated coincident factor by lamp type and room type. Findings are
presented in Table 8-21 and Table 8-22, but it should be noted that the number of sample points
among some variables is quite low (e.g. metered lamps in hallways), which lead to low
confidence/precision estimates. The reader should be mindful of this uncertainty when
interpreting the results.
Table 8-21: Coincident Factor by Peak Period by Lamp Type
Precision Lamp Type (Logger Standard Lower Upper Peak N CF (90% . . . . Level) Error t·d ) L1m1t L1m1t
Winter, 7-8 AM
Winter, 5-6 PM
Summer, 5-6.30 PM
t.-1Nexanr
con I ence
CFL 334 0.10 0.02 33.54% 0.06 0.13
Incandescent 545 0.07 0.01 27.07% 0.05 0.08
LED 83 0.16 0.06 60.40% 0.06 0.26
CFL 334 0.17 0.02 19.71% 0.13 0.20
Incandescent 545 0.13 0.02 19.49% 0.10 0.15
LED 83 0.22 0.04 29.24% 0.15 0.28
CFL 334 0.10 0.01 21.47% 0.08 0.12
Incandescent 545 0.08 0.01 23.22% 0.06 0.10
LED 83 0.13 0.03 32.78% 0.08 0.17
Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 169
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 183 of212
8 RESIDENTIAL LIGHTING STUDY
Table 8-22: Coincident Factor by Peak Period by Room Type
Precision
Peak Room Type (Logger N CF Standard (go% Lower Upper
level) Error O Limit Limit
Winter, 7-8 AM
Winter, 5-6 PM
Summer, 5-
6.30 PM
t-1Nexanr
confidence)
Kitchen 180 0.16 0.04 37.49% 0.10 0.23
Dining 129 0.06 0.02 53.25% 0.03 0.10
Living/GreaUFamily 134 0.09 0.02 31 .31% 0.06 0.12
Foyer/Hall/Stair 62 0.09 0.03 55.36% 0.04 0.14
Bedroom 131 0.06 0.02 47.75% 0.03 0.09
ToileUBathroom 131 0.10 0.03 42.21% 0.06 0.14
Other 195 0.04 0.02 73.21% 0.01 0.06
Kitchen 180 0.31 0.04 21.03% 0.24 0.38
Dining 129 0.22 0.04 31.27% 0.15 0.29
Livi ng/G reaUF am ily 134 0.24 0.03 18.93% 0.20 0.29
Foyer/Hall/Stair 62 0.12 0.03 42.75% 0.07 0.17
Bedroom 131 0.08 0.02 33.77% 0.05 0.10
ToileUBathroom 131 0.07 0.02 40.35% 0.04 0.09
Other 195 0.11 0.02 30.02% 0.08 0.14
Kitchen 180 0.16 0.02 24.68% 0.12 0.20
Dining 129 0.13 0.03 44.49% 0.07 0.18
Living/GreaUFamily 134 0.09 0.02 32.93% 0.06 0.11
Foyer/Hall/Stair 62 0.06 0.02 69.16% 0.02 0.10
Bedroom 131 0.04 0.01 43.46% 0.02 0.06
ToileUBathroom 131 0.11 0.02 30.86% 0.07 0.14
Other 195 0.09 0.02 34.97% 0.06 0.12
Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 170
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 184 of 212
Appendix A Sampling and Estimation
The gross verified energy savings estimates presented in this report from Avista's electric DSM
programs were generally determined through the observation of key measure parameters
among a sample of program participants. A census evaluation would involve surveying ,
measuring, or otherwise evaluating the entire population of projects within a population .
Although a census approach would eliminate the sampling uncertainty for an entire program , the
reality is that M&V takes many resources both on the part of the evaluation team and the
program participants who agree to be surveyed or have site inspections conducted in their
home or business. When a sample of projects is selected and analyzed, the sample statistics
can be extrapolated to provide a reasonable estimate of the population parameters. Therefore,
when used effectively, sampling can improve the overall quality of an evaluation study. By
limiting resource-intensive data collection and analysis to a random sample of all projects, more
attention can be devoted to each project surveyed.
The nuances and tradeoffs considered by the evaluation team when developing sampling
approaches varied across the portfolio and are discussed in more detail in Section 3.2.
However, several common objectives were shared across sectors and programs. The most
important sampling objective was representativeness -that is the projects selected in the
evaluation were representative of the population they were selected from and will produce
unbiased estimates of population parameters. A second key sampling objective was to consider
the value of information being collected and align sample allocations accordingly. This effort
generally involves considering the size (contribution to program savings) and uncertainty
associated with the area being studied and making a determination about the appropriate level
of evaluation resources to allocate.
The evaluation team used two broad classes of probability estimation techniques to make
inferences about program or stratum performance based on the observations and
measurements collected from the evaluation sample. Auxiliary information refers to the reported
savings estimates stored in the program tracking system.
1) Mean-Per-Unit (or estimation in the absence of auxiliary information): This technique
was used to analyze samples drawn from populations that are similar in size and
scope. This approach was used primarily for residential programs that include a large
number of rebates for similar equipment types where the evaluation objective is to
determine an average kWh savings per rebated piece of equipment. With mean-per­
unit estimation the average kWh savings observed within the sample is applied to all
projects in the population.
2) Ratio Estimation (or estimation using auxiliary information): This technique was used
for nonresidential programs and residential programs with varying savings across
projects. This technique assumes that the ratio of the sum of the verified savings
estimates to the sum of the reported savings estimates within the sample is
t.-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs A-1
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 185 of 212
APPENDIX A SAMPLING AND ESTIMATION
representative of the program as a whole. This ratio is referred to as the realization
rate, or ratio estimator, and is calculated as follows:
Equation A-1: Coefficient of Variation
. . I:? Verified Savings Realzzatwn Rate = "11 d . L...i Reporte Savings
Where n is the number of projects in the evaluation sample. The realization rate is then applied
to the claimed savings of each project in the population to calculate gross verified savings.
Figure A-1 shows the reduction in error that can be achieved through ratio estimation when the
sizes of projects within a program population vary considerably. The ratio estimator provides a
better estimate of individual project savings than a mean savings value by leverag ing the
reported savings estimate.
Figure A-1: Comparison of Mean-Per-Unit and Ratio Estimation
700,000 • 000,000
500,000 •
~ • 1 «Jo,000 • .,, .. 'E 300,000 .. :,,
200,000
100,000
0
Error
ean
500,000 +------------'--_,.."---
~ 1 «lo,000 +--------~'---or--~--.,,
i 30q000 +----------------
0J :,,
0 10(),000 200,000 300,000 4100,000 ~000 f,00,000
Reporti!d Swines
For a measure such as the variable speed house fan , where each of the nearly 1,300 rebated
units claimed an identical savings value of 439 kWh/year ratio estimation would offer no
advantage over mean-per-unit estimation because there is no variability along the x-axis to
leverage.
A.1 Stratification
The evaluation team used sample stratification with both classes of estimation techniques.
Stratification is a departure from simple random sampling (SRS), where each sampling unit
(customer/project/rebate/measure) has an identical likelihood of being selected in the sample.
Stratified random sampling refers to the designation of two or more sub-groups (strata) from
within a program population prior to the selection process. Whenever stratification was
employed the evaluation team took great care to ensure that each sampling unit within the
population belonged to one (and only one) stratum. In each program sample design where
stratification was used, the probability of selection is different between strata and this difference
must be accounted for when calculating results. The inverse of the selection probability is
referred to as the case weight and is used in estimation of impacts when stratified random
t1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs A-2
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 186 of 212
APPENDIX A SAMPLING AND ESTIMATION
samples are utilized. Consider the following simplified example in Table A-1 based on a
fictional program with two measures; refrigerators and clothes washers.
Table A-1: Case Weights Example
Measure Population Size Sample Size Case Weight
Clothes Washer 15,000 30 500
Refrigerator 6,000 30 200
Because refrigerators are sampled at a higher rate (1-in-200) than clothes washers (1-in-500),
each sample point carries less weight in the program results than an individual clothes washer
sample point. In general, the evaluation team designed samples so that strata with high case
weights had low per-unit impacts or were well-understood measures. Low case weights were
reserved for large and complex measures such as the large stratum of the Site Specific
program .
The evaluation team felt that stratification was advantageous and utilized it in the sample design
for a variety of reasons across the portfolio:
1) Increased precision if the within-stratum variability was expected to be small compared
to the variability of the popu lation as a whole. Stratification in this case allows for
increased precision or smaller total sample sizes, which lowered evaluation costs.
2) To ensure that a minimum number of units within a particular stratum will be verified.
This was relevant for small programs like ENERGY STAR® Homes. Although the
program's contribution to portfolio savings was small, the evaluation team felt it was
important to sample enough projects to independently estimate program performance.
3) It is easy to implement a value-of-information approach through which the largest
projects are sampled at a much higher rate than smaller projects by creating size-based
strata.
4) Sampling independently within each stratum allows for comparisons among groups.
Avista and the evaluation team find value in comparing results between strata; e.g.,
comparing the realization rates between measures within a program.
A.2 Presentation of Uncertainty
There is an inherent risk, or uncertainty, that accompanies sampling, because the projects
selected in the evaluation sample may not be representative of the program population as a
whole with respect to the parameters of interest. As the proportion of projects in the program
population that are sampled increases, the amount of sampling uncertainty in the findings
decreases. The amount of variability in the sample also affects the amount of uncertainty
introduced by sampling. A small sample drawn from a homogeneous population will provide a
more reliable estimate of the true population characteristics than a small sample drawn from a
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs A-3
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 187 of 212
APPENDIX A SAMPLING AND ESTIMATION
heterogeneous population. Variability is expressed using the coefficient of variation (Cv) for
programs that use simple random sampling, and an error ratio for programs that use ratio
estimation. The Cv of a population is equal to the standard deviation (a) divided by the mean(µ)
as shown in Equation A-2.
Equation A-2: Coefficient of Variation
(J
C --v -µ
When ratio estimation is utilized, standard deviations will vary for each project in the population.
The error ratio is an expression of this variability and is analogous to the Cv for simple random
sampling.
Equation A-3 provides the formula for estimating error ratio.
Equation A-3: Error Ratio
. rr=l (Ti Error Ratio = ____,.,N,---
Li=l µi
Equation A-4 shows the formula used to calculate the required sample size for each evaluation
sample, based on the desired level of confidence and precision. Notice that the Cv term is in the
numerator, so required sample size will increase as the level of variability increases. For
programs that rely on ratio estimation, error ratio replaces the Cv term in Equation A-4. Results
of the 2012-2013 portfolio evaluation were the primary source of error ratio and Cv assumptions
for the evaluation.
Where:
Equation A-4: Required Sample Size
z * Cv no= (--)2 D
n0 = The required sample size before adjusting for the size of the population
Z = A constant based on the desired level of confidence (equal to 1.645 for 90%
confidence two-tailed test)
Cv = Coefficient of variation (error ratio for ratio estimation)
D = Desired relative precision
The sample size formula shown in Equation A-4 assumes that the population of the program is
infinite and that the sample being drawn is reasonably large. In practice, this assumption is not
always met. For sampling purposes, any population greater than approximately 7,000 may be
considered infinite for the purposes of sampling. For smaller, or finite, populations, the use of a
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs A-4
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 188 of212
APPENDIX A SAMPLING AND ESTIMATION
finite population correction factor (FPC) is warranted. This adjustment accounts for the extra
precision that is gained when the sampled projects make up more than about 5% of the
program savings. Multiplying the results of Equation A-4 by the FPC formula shown in Equation
A-5 will produce the required sample size for a finite population.
Equation A-5: Finite Population Correction Factor
fpc= 1
Where:
N = Size of the population
n0 = The required sample size before adjusting for the size of the population
The required sample size (n) after adjusting for the size of the population is given by Equation
A-6.
Equation A-6: Application of the Finite Population Correction Factor
n = n 0 * f pc
Throughout this report gross verified energy savings are reported with the associated margin of
error. The margin of error can be introduced by sampling or via estimation error from a billing
analysis, or both. Billing analyses rely on consumption data that often contains variability not
explained by weather or other independent variables. This inherent variability in the data
introduces uncertainty because program savings effects must be separated from underlying
noise. The standard errors of coefficients in the regression model quantify this uncertainty and
allow a margin of error to be calculated . Verified savings estimates always represent the point
estimate of total savings, or the midpoint of the confidence interval around the verified savings
estimate for the program. Equation A-7 shows the formula used to calculate the margin of error
for a parameter estimate.
Where:
se
Equation A-7: Error Bound of the Savings Estimate
Error Bound = se * (z -statistic)
=The standard error of the population parameter of interest (proportion of
customers installing a measure, realization rate, total energy savings,
etc.) This formula will differ according to the sampling technique utilized.
z -statistic =Calculated based on the desired confidence level and the standard
normal distribution.
'-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs A-5
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 189 of212
APPENDIX A SAMPLING AND ESTIMATION
The 90% confidence level is a widely accepted industry standard for reporting uncertainty in
evaluation findings. Unless otherwise noted, the confidence levels and precision values
presented in this report are at the 90% confidence level. The z-statistic associated with 90%
confidence is 1.645.
The evaluation team also reports the relative precision value associated with verified savings
estimates. When evaluators or regulators use the term "90/10", the 10 refers to the relative
precision of the estimate. The formula for relative precision shown in Equation A-8:
Equation A-8: Relative Precision of the Savings Estimate
. . . Error Bound(kWh or kW)
Relative Preciswnverified savings = .1. d Ven ie lmpact(kWh or kW)
An important attribute of relative precision to consider when reviewing achieved precision values
is that it is "relative" to the impact estimate. Therefore programs with low realization rates are
likely to have larger relative precision values because the error bound (in kWh) is being divided
by a smaller number. This means two programs with exactly the same reported savings and
sampling error in absolute terms, with have very different relative precision values (example in
Table A-2).
Table A-2: Relative Precision Example
. . Relative . . Error Bound Verified Program Reported kWh Realization Rate k Precision
( Wh) kWh (gQ%)
Program #1 4,000,000 0.5 400,000 2,000,000 ±20%
Program #2 4,000,000 1.0 400,000 4,000,000 ± 10%
In many cases a program-level savings estimate requires summation of the verified savings
estimates from several strata. In order to calculate the relative precision for these program-level
savings estimates, the evaluation team used Equation A-9 to estimate the error bound for the
program as a whole from the stratum-level error bounds.
Equation A-9: Combining Error Bounds across Strata
d2 2 2 Error Boundprogram = Error Baun stratuml + Error Boundstratumz + Error BoundstratumJ
Using this methodology, the evaluation team developed verified savings estimates for the
program and an error bound for that estimate. The relative precision of the verified savings for
the program is then calculated by dividing the error bound by the verified savings estimate.
'-"1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs A-6
Exhibit No. 2
L. Roy, Avista
Schedule 1 , Page 190 of 212
Appendix B Lighting Interactive Factors
Table B-1: Lighting Interactive Factors by Building Type and HVAC System Type
Assembly 93% 82% 102% 91%
Automotive Repair 61% 61% 81% 81%
College or University 72% 53% 96% 77%
Exterior 24 Hour Operation 100% 100% 100% 100%
Hospital 29% 28% 65% 64%
Industrial Plant with One Shift 69% 61% 89% 81%
Industrial Plant with Three Shifts 69% 61% 89% 81%
Industrial Plant with Two Shifts 69% 61% 89% 81%
Library 72% 53% 96% 77%
Lodging 70% 60% 90% 80%
Manufacturing 69% 61% 89% 81%
Office <20,000 sf 72% 53% 96% 77%
Office >100,000 sf 93% 82% 102% 91%
Office 20,000 to 100,000 sf 93% 82% 102% 91%
Other Health, Nursing, Medical Clinic 93% 82% 102% 91%
Parking Garage 100% 100% 100% 100%
Restaurant 43% 41% 73% 71%
Retail 5,000 to 50,000 sf 73% 61% 93% 81%
Retail Anchor Store >50,000 sf Multistory 75% 57% 97% 79%
Retail Big Box >50,000 sf One-Story 86% 67% 103% 84%
Retail Boutique <5,000 sf 82% 69% 98% 85%
Retail Mini Mart 75% 61% 95% 81%
Retail Supermarket 86% 78% 97% 89%
School K-12 62% 52% 86% 76%
Street & Area Lighting (Photo Sensor 100% 100% 100% 100% Controlled)
Warehouse 61% 61% 81% 81%
Other 93% 82% 102% 91%
L-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs B-1
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 191 of212
APPENDIX B LIGHTING INTERACTIVE FACTORS
Table B-2: Lighting Interactive Factors by Building Type and HVAC System Type Cont.
Assembly 98% 111 % 130% 100% -0.0082
Automotive Repair 96% 100% 130% 100% -0.0177
College or University 96% 119% 130% 100% -0.0214
Exterior 24 Hour Operation 100% 100% 130% 100% 0
Hospital 93% 101% 130% 100% -0.0328
Industrial Plant with One Shift 96% 108% 130% 100% -0.0177
Industrial Plant with Three Shifts 96% 108% 130% 100% -0.0177
Industrial Plant with Two Shifts 96% 108% 130% 100% -0.0177
Library 96% 119% 130% 100% -0.0214
Lodging 96% 110% 130% 100% -0.0182
Manufacturing 96% 108% 130% 100% -0.0177
Office <20,000 sf 96% 119% 130% 100% -0.0214
Office >100,000 sf 98% 111% 130% 100% -0.0082
Office 20,000 to 100,000 sf 98% 111% 130% 100% -0.0082
Other Health, Nursing, Medical 98% 111 % 130% 100% -0.0082 Clinic
Parking Garage 100% 100% 130% 100% 0
Restaurant 94% 102% 130% 100% -0.0268
Retail 5,000 to 50,000 sf 96% 112% 130% 100% -0.0177
Retail Anchor Store >50,000 sf
Multistory 96% 118% 130% 100% -0.0196
Retail Big Box >50,000 sf One-Story 97% 119% 130% 100% -0.0150
Retail Boutique <5,000 sf 97% 113% 130% 100% -0.0141
Retail Mini Mart 96% 114% 130% 100% -0.0177
Retail Supermarket 98% 108% 130% 100% -0.0100
School K-12 96% 110% 130% 100% -0.0218
Street & Area Lighting (Photo 100% 100% 130% 100% 0 Sensor Controlled)
Warehouse 96% 100% 130% 100% -0.0177
Other 98% 111 % 130% 100% -0.0082
t..'1 Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs B-2
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 192 of212
Appendix C Billing Analysis Regression Outputs
C.1 HVAC Program
Table C-1: ASHP Fixed-Effects Regression Output
Fixed-effects (within) regression
Group variable : new acct
R-sq: within 0 .6350
0.0705
0 .4841
between
overall
corr(u_i, Xb) 0.0078
daily_kwh
treatment
hdd ave -
c.hdd ave#c.treatment
cons
sigma_u
sigma_e
rho
Coef.
2.953907
1. 813402
-.5409008
24.4846
15 .473174
16.184346
.47754676
(Std. Err.
Robust
Std. Err .
1.051504
.078876
.0624757
1.284606
Number of obs
Number of groups
Obs per group : min
avg
max
F(3 , 108)
Prob> F
adjusted
t
2 .81
22.99
-8 .66
19.06
for 109
P>lt l
0 .006
0 .000
0 .000
0.000
(fraction of variance due to
3826
109
20
35.1
37
193 .04
0 .0000
clusters in
[95% Conf .
. 8696451
1. 657056
-.6647383
21.93829
u i) -
new_acct)
Interval]
5 .03817
1 .969747
-.4170632
27.03091
L-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs C-1
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 193 of212
APPENDIX C BILLING ANALYSIS REGRESSION OUTPUTS
Table C-2: Variable Speed Fan Motor Fixed-Effects Regression Output
Fixed-effects (within) regression
Group variable : new acct
R-sq : within
between
overall
corr(u_i , Xb)
0.1426
0.0007
0.0492
-0.0002
Number of obs
Number of groups
Obs per group : min
avg
max
F (5 ,591)
Prob > F
21036
592
19
35.5
37
168 .92
0 .0000
(Std . Err. adjusted for 592 clusters in new_acct)
daily_kwh
cdd ave
treatment
c .cdd ave#c .treatment
hdd ave
treatment
c.hdd ave#c.treatment
I
cons
sigma_u
sigma_e
rho
Coef .
Robust
Std. Err.
2.240237 .1093027
.7268809 .4911327
-.377199 .088024
.448729 .031888
0 (omitted)
-.0784601 .0243396
22 .42068 .6604954
t
20.50
1. 48
-4.29
14 .07
-3 .22
33 .95
P>l tl
0 .000
0 .139
0.000
0.000
0 .00 1
0.000
[95% Conf. Interval]
2 .025568
-.2376969
-.550077
.3861013
-.126262 7
21 .12348
2 .454906
1 .691459
-.204321
. 5113567
-.0306575
23 .71788
17 .550959
11.904935
.68488454 (fraction of variance due to u_i)
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs C-2
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 194 of 212
APPENDIX C BILLING ANALYSIS REGRESSION OUTPUTS
C.2 Low Income Program
Table C-3: Low Income Fuel Switching
Fixed-effects (within) regression
Group variable : account
R-sq: within
between
overall
corr(u_i, Xb)
0 .6476
0 .0081
0 .5357
-0.0104
dai ly_kwh
treatment
cdd ave
hdd ave
c.cdd ave#c .treatment
c.hdd ave#c.treatment
cons
sigma_u
sigma_e
rho
Coef .
-.2237355
1. 744057
1. 71593
-.4636856
-1 .479525
15.72763
11.082831
14.797874
.35935317
(Std.
Robust
Std. Err .
1.204884
.1989493
.0943928
.1449925
.0896845
1. 486092
Number of obs
Number of groups
Obs per group: min
avg
max
F(5,66)
Prob> F
Err . adjusted for 67
t P>ltl
-0.19 0 .853
8.77 0 .000
18 .18 0 .000
-3 .20 0.002
-16 .50 0.000
10 .58 0 .000
2226
67
25
33 .2
60
107 .35
0 .0000
clusters in
[95% Conf .
-2 .629364
1 .346842
1 .527469
-.7531725
-1.658586
12 .76056
(fraction of variance due to u_i)
account)
Interval)
2 .181893
2 .141272
1 .904392
-.1741987
-1.300464
18 .69471
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs C-3
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 195 of212
APPENDIX C BILLING ANALYSIS REGRESSION OUTPUTS
Table C-4: Low Income Electric Conservation
Fixed-effects (wi thin) regression
Group variable: account
R-sq : within 0.2724
0 .0021
0 .1512
between
overall
corr(u_i , Xb) -0.0079
daily_kwh
treatment
cdd ave
hdd ave
c .cdd ave#c .treatment
c.hdd_ave#c.treatment
cons
sigma_u
sigma_e
rho
Coef .
.0369547
1 . 413486
1. 000256
-.0717039
-.1048216
16.68617
17 .502397
17 .046712
. 5131872
Number of obs
Number of groups
Obs per group : min
avg
max
F(5,164)
Prob> F
(Std. Err . adjusted for 165
Robust
Std . Err . t P>ltl
1 .036704 0.04 0 . 972
.1614987 8 .75 0 .000
.083218 12 .02 0 .000
. 1136132 -0 .63 0 .529
. 0577246 -1. 82 0. 071
1 .478321 11. 29 0.000
(f raction of vari ance due to
5758
165
26
34 .9
70
52 .86
0 .0000
clusters in account)
[95% Conf . Interval]
-2.010053 2 .083963
1. 094601 1 .732371
.8359395 1 .164573
-.296037 .1526293
-.218801 .0091577
13 .76717 19 .60517
u i)
t.-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs C-4
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 196 of212
APPENDIX C BILLING ANALYSIS REGRESSION OUTPUTS
C.3 Shell Program
Table C-5: Shell Rebate Measures
Fixed-effects (within) regression
Group variable: new acct
R-sq : within 0.2066
0.0197
0.0908
between
overall
corr(u_i , Xb) -0 .00 86
daily_kwh
treatment
cdd ave
hdd ave
c .hdd ave#c .trea tment
c .cdd ave#c.treatment
cons
sigma_u
sigma_e
rho
Coef.
-.3911459
1. 767326
.7466587
-.0504493
-.1390177
20 .04061
17.860391
14 .751276
.59447877
Number of obs
Number of groups
Obs per group: min
avg
max
F (5,766 )
Prob> F
(Std . Err . adjusted for 767
Robust
Std . Err. t P>ltl
.4069751 -0. 96 0.337
.1030673 17. 15 0 .000
.0368662 20 .25 0.000
.0170151 -2 .96 0 .003
.0656922 -2.12 0.035
. 727349 27 .55 0 .000
(fraction of variance due to
26568
767
24
34 .6
36
145.62
0.0000
clusters in new_acct)
[95% Conf. Interval]
-1.190065 .407773
1. 564998 1. 969654
.674288 .8190294
-.08 38509 -.0170476
-.2679758 -.0100595
18 .61277 21. 46844
u_i)
t.-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs C-5
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 197 of212
APPENDIX C BILLING ANALYSIS REGRESSION OUTPUTS
C.4 Fuel Efficiency Program
Table C-6: Electric to Gas Furnace Conversion
Fixed-effects (within) regression
Group variable: id
R-sq : within 0 .5869
between 0 .0952
overall O. 4 080
corr(u i , Xb) 0.0217
daily_kwh
hdd ave
l .treatment
treatment#c.hdd_ave
1
cdd ave
treatment#c .cdd_ave
1
cons
sigma_u
sigma_e
rho
Coef .
2 .063256
10.75073
-1. 687934
2 .511148
-1.154795
13.8264
18 .416175
17 .166024
.53509083
Number of obs
Number of groups
Obs per group: min
avg
max
F(5, 172)
Prob> F
5792
173
15
33 .5
37
114 .59
0 .0000
(Std . Err . adjusted for 173 clusters in id)
Robust
Std. Err.
.1090112
1.607743
.1106144
.2011141
. 17 69566
1 .902086
t
18 .93
6 .69
P>ltl
0.000
0.000
-15 .26 0.000
12.49 0 .000
-6 .53 0 .000
7 .27 0 .000
[95% Conf . Interval]
1.848084
7 .577283
2 .278428
13 .92418
-1 .906271 -1 .469598
2.114179 2.908117
-1 .504081 -.8055084
10 .07197 17 .58084
(fraction of variance due to u i)
t-"1 Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs C-6
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 198 of212
APPENDIX C BILLING ANALYSIS REGRESSION OUTPUTS
Table C-7: Electric to Gas Water Heater Conversion
Fixed-effects (within) regression
Group variable: id
R-sq: within
between
overall
0.2691
0.0034
0.1216
corr(u_i, Xb) -0.0141
daily_kwh
hdd ave
!.treatment
treatment#c.hdd_ave
1
cdd ave
treatment#c.cdd_ave
1
cons
sigma_u
sigma_e
rho
Coef.
.4577346
-8.485181
-.1015656
1.617465
.0304397
26.45666
14. 212811
11 . 201992
.61682752
Number of obs 2495
Number of groups 71
Obs per group: min 21
avg 35.1
max 37
F (5, 70) 26. 87
Prob > F 0.0000
(Std. Err. adjusted for 71 clusters in id)
Robust
Std. Err.
. 0671164
1. 34192
.0723047
.3369514
.210502
1. 361088
t
6.82
-6 . 32
-1.40
4.80
0.14
19.44
P>ltl
0.000
0.000
0.165
0.000
0.885
0.000
[95 % Conf. Interval)
.3238751
-11.16156
-. 2457728
.9454364
-.3893 933
23.7 4206
.5915941
-5.808806
.0426416
2.289493
. 4502726
29.17127
(fraction of variance due to u_i)
'-'1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs C-7
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 199 of 212
APPENDIX C BILLING ANALYS IS REGRESSION OUTPUTS
Table C-8: Electric to Gas Furnace and Water Heater Conversion
Fixed-effects (within) regression
Group variabl e : id
R-sq: within
between
overall
0.6718
0.0034
0 .4474
corr(u_i , Xb) -0 .0355
daily_kwh Coef.
hdd ave 1. 952949
l.treatment 6.088577
treatment#c.hdd_ave
1 -1.627161
cdd ave 2.659406 -
treatment#c.cdd_ave
1 -1. 310611
cons 14.03094
sigma_u 18.025111
sigma_e 15. 327112
rho .58036822
Number of obs 3475
Number of groups 102
Obs per group: min 15
avg 34.1
max 37
F(5 ,101) 120.37
Prob > F 0 .0000
(Std . Err. adjusted for 102 clusters in id)
Robust
Std . Err. t P>lt l [95% Conf. Interval)
.0842092 23.19 0.000 1. 785901 2.119998
1.855304 3.28 0.001 2 .408154 9.769001
.0935052 -17.40 0.000 -1.81265 -1.441672
.1870938 14.21 0.000 2.288262 3 .03055
.1920518 -6.82 0.000 -1 .69159 -.9296322
1.437596 9. 76 0.000 11.17914 1 6 .88275
(fraction of variance due to u_i)
t..'1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs C-8
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 200 of 212
Appendix D Net to Gross Methodology and Findings
The evaluation team calculated net-to-gross (NTG) ratios for each program, using data collected
from participant surveys. NTG takes into consideration the levels of both free ridership (FR) and
spillover (SO). Free ridership refers to the portion of energy savings that participants would have
achieved in the absence of the program through their own initiatives and expenditures (EPA,
2007).39 Spillover refers to the program-induced adoption of measures by non-participants and
participants who did not receive financial incentives or technical assistance for installations of
measures supported by the program (EPA, 2007). The evaluation team used the following
formula to calculate a NTG ratio for each program:
NTG = 1 -FR+ SO
D.1 Free Ridership
Subtracting free ridership from gross savings produces an estimate of how much the program
influenced participants to make the energy saving improvements that the program incents. Free
ridership ranges from O to 1, with O being no free ridership (the program induced all of the
reported gross savings), 1 being total free ridership (the program induced none of the savings)
and values in between represent varying degrees of partial free ridership. The evaluation team
used participant survey data to inform free ridership estimates.
With the exception of appliance recycling (which uses a different approach, explained below),
free ridership consists of two components -change (FRC) and influence (FRI) -which both
range from Oto .5.
FR= FRC + FRI
Free Ridership Change (FRC)
Free ridership change is the participant's self-report of what they likely would have done if the
program had not provided an incentive for their energy upgrade. To determine this, the
evaluation team asked participant survey respondents FRC questions specific to the measures
they installed. The question below exemplifies how the evaluation team collected FRC data.
I'd like to ask a few questions about what you most likely would have done had you not received
assistance from Avista for the [Measure Type].
Q1. Which of the following three alternatives is most likely: Would you have:
[SINGLE RESPONSE]
1. Put off buying a new [Measure Type] for at least one year [Includes repairing old or
buying a used one.]
2. Bought a new [Measure Type] that was less expensive or less energy efficient.
39 The Environmental Protection Agency (EPA) (2007). Model Energy Efficiency Program Impact Evaluation Guide. Retrieved June
8, 2015 from http://www.epa.gov/cleanenergy/documents/suca/evaluation_guide.pdf.
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs 0-1
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 201 of212
APPENDIX D NET TO GROSS METHODOLOGY AND FINDINGS
3. Bought the exact same [Measure Type] anyway, and paid the full cost yourself.
[Do not read:]
-96. 96. Other, please specify: [OPEN-ENDED RESPONSE]
-97. 98. Don't know
-98. 99. Refused
The evaluation team then assigned the following FRC values to each respondent, based on
their response to the question above, as shown in the Table D-1.
Table D-1: Free Ridership Change Values
Q1 Response FRC Value
Put off buying a new (Measure Type] for at least one year [Includes
repairing old or buying a used one.]
Bought a new (Measure Type] that was less expensive or less
energy efficient.
Bought the exact same [Measure Type] anyway, and paid the full
cost yourself.
Other
Don't know I Refused
Free Ridership Influence (FRI)
0.00
0.25
0.50
FRC values assigned on a case by case
basis, depending on which pre-coded
response item they most resemble
0.25
Free ridership influence represents how much influence the program had on a participant's
decision to perform the incented energy upgrade. To determine this, the evaluation team asked
participant survey respondents the following question:
Q2. Now I would like to ask about the influence that the program played in your decision to
purchase the energy efficient [Measure Type]. I'm going to read a list of things that may
have influenced your decision to buy the [Measure Type]. For each one, please indicate
how much of an influence it played in your decision, where '1' means it was "not at all
influential" and "5" means it was "extremely influential." Let me know if an item doesn't
apply to you. [Interviewer: do not read 97-99]
MA TRIX QUESTION: SCALE]
[LOGIC] Item 1 2 3 4 5 97 98 99
NA DK RF
[IF INCENTIVE =
REBATE] The rebate
you received
Information on Avista's
website
Advertising and other
information from Avista
A salesperson or
contractor
t.-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs D-2
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 202 of212
--------------------------------------------------
APPENDIX D NET TO GROSS METHODOLOGY AND FINDINGS
I
Aoyth;og else, please
. specify: _____ _ I I I I
The evaluation team then selected the highest rated program-attributable item for each
respondent and assigned the following FRI scores, depending on their high score value (Table
D-2).
Table D-2: Free Ridership Influence Values
Influence Rating FRI Value
0.500
2 0.375
3 0.250
4 0.125
5 0.000
Don't know/ Refused Sector-level measure average
Program-Level Free Ridership
The evaluation team summed FRC and FRI scores for each respondent, yielding participant­
level free ridership (FR) scores. The evaluation team used the participant-level FR scores to
calculate a savings-weighted average FR score for each program, which serves as the
program-level FR score.
Appliance Recycling Free Ridership
The evaluation team developed an approach to calculating net savings for the Appliance
Recycling Program by applying the Department of Energy Uniform Methods Project's (UMP)
methodology. The UMP methodology differs from the NTG methodology for other program
types. Rather than first calculating a NTG value from survey responses and then applying that
to gross savings to yield net savings, the UMP methodology first calculates net savings using
jurisdiction-specific data on the energy consumption of new and recycled appliances, together
with survey data on the participants' decision-making.40
Adding estimated spillover to the net savings and dividing that sum by the program-reported
gross savings yields the NTG ratio.41 The evaluation team developed a modified approach that
40 See The Uniform Methods Project: Methods for Determining Energy Efficiency Savings for Specific Measures, Chapter 7:
"Refrigerator Recycling Evaluation Protocols, National Renewable Energy Laboratory," March 2013 (Download available at:
http://www1 .eere .energy.gov/wip/pdfs/53827-7. pdf).
41 The rationale for the UMP approach is that the actual gross savings for a particular participant depends on whether or not the
participant replaced the recycled unit with a new one. Replacing the recycled unit with a new one yields gross savings equal to the
energy consumption of the recycled unit minus the energy consumption of the replacement unit. Recycling without replacement
yields gross savings equal to the entire energy consumption of the recycled unit. The net savings thus account for the level of free
ridership as well as the mix of replaced and non-replaced appliances.
'-'""'Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs D-3
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 203 of 212
APPENDIX D NET TO GROSS METHODOLOGY AND FINDINGS
did not require estimates of the average consumption of new and recycled appliances.
Surveyed participants reported what they would have done absent the program, and the
evaluation team assigned a free ridership value to each respondent based on the latter
information (Table D-3).42
Table D-3: Appliance Recycling Modified FR Values
Scenario FR Value
The participant would not have recycled appliance without the program
Without the program, the participant would have sold or given away appliance for use in
another home. Some of those would have been removed from the grid, some not.*
Without the program, the participant would have disposed of the appliance in a way that
removed it from the grid.
0.000
0.375
1.000
* The UMP methodology assumes that half the units would have been taken off the grid without replacement, one-quarter of the
units would have been taken off the grid with replacement, and one-quarter of the units would not have been taken off the grid. The
evaluation team assigned free ridership values of 0, .5, and 1.0 to those three subgroups, respectively.
The evaluation team used the participant-level FR scores to calculate a savings-weighted
average FR score for the appliance recycling program, which serves as the program-level FR
score.
0.2 Spillover
Spillover estimates the energy savings from non-rebated energy improvements made outside of
the program that are influenced by the program, and can be used to adjust gross savings by the
additional energy savings garnered and the level of attribution the program is able to claim for
these non-rebated measures. A spillover value of O equates to no spillover and values greater
than O demonstrate the existence and magnitude of spillover.43 The evaluation team used
participant survey data to estimate spillover.
The evaluation team asked participant survey respondents to indicate what energy saving
measures they had implemented since participating in the program to identify potential spillover.
The evaluation team then asked participants to use a 1 to 5 scale, where 1 means "not at all
influential" and 5 means "extremely influential," to indicate how much influence the Avista
program had on their decision to purchase these additional energy saving measures. Table D-4
exhibits how much program influence, ranging from 0% to 100%, is associated with each scale
response to the spillover influence question.
42 The surveyed respondents also reported whether they did or did not replace the recycled appliance. However, the information on
replacement or non-replacement did not enter the free ridership equation, as that only indicates the amount of gross savings
possible.
43 Spillover values can be interpreted as percentages, where 1=100%. Thus, a spillover value of .5 would mean that spillover
savings were 50% of program gross savings.
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs D-4
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 204 of 212
APPENDIX D NET TO GROSS METHODOLOGY AND FINDINGS
Table D-4: Participant Spillover Program Influence Values
Reported Avista Program I fl V 1 Influence n uence a ue
0.0
2 0.0
3 0.5
4 1.0
5 1.0
The evaluation team used the influence value to calculate the participant measure spillover
(PMSO) for each spillover measure that each participant reported. Participant measure spillover
is calculated as follows, with the deemed measure savings values based on the evaluation
teams estimate of the savings for the implemented measure:
PMSO = Deemed Measure Savings* Influence Value
The evaluation team then summed all PMSO values associated with each program and divided
them by the sample's gross program savings to calculate the spillover estimates for each
program:
L Program PMSO Program SO=------------­I.Sample's Gross Program Savings
0.3 Residential Lighting Net to Gross
The estimated free ridership impacts of the residential lighting program-in which a customer
likely replaced an expired, efficient technology with a like technology-was calculated by
constructing a market baseline. The evaluation team developed this baseline by examining the
composition of lamp types found from onsite inspections in the lighting study, respective EISA
equivalent baselines, and efficient case wattage to determine the free ridership market effects.
The evaluation team's methodology is consistent with the RTF, but values are based on primary
data collection from Avista's service territory.
The market share for each lamp technology was determined from the Avista residential lighting
hours-of-use study, in which the existing shares of installed lamps by technology type were
inventoried during onsite inspections; see Table 8-5. For the purposes of assessing the market
baseline for the residential lighting program, the market shares needed to be normalized for
screw-in sockets only. For example, the market share for CFL lamps increased from 23.8% to
26.9% once only screw-in sockets were included. The CFL market share from onsite
inspections is supported by the data from the NEEA 2014-2015 Northwest Residential Lighting
Long-Term Market Tracking Study44, which listed the estimated CFL market share as 28%.
44 https://neea.org/docs/default-source/reports/northwest-residential-lighting-long-term-market-tracking-study.pdf?sfvrsn=4
t-'1 Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs D-5
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 205 of 212
APPENDIX D NET TO GROSS METHODOLOGY AND FINDINGS
To determine the adjusted market baseline for screw-in lamps, the evaluation team multiplied
the market share by the typical technology wattage for each type. To illustrate the approach,
Table D-5 provides a summary of the calculation to estimate the market baseline for a 60-watt
equivalent A-lamp.
Table D-5: Example Market Baseline 60-watt Equivalent Lamp
T h I T Market Share of Typical Technology Contribution to Market ec no ogy ype . Screw-m Sockets Wattage Baseline Wattage
CFL 26.9% 13 3.5
Incandescent 60.8% 43* 26.1
Halogen 4.4% 43* 1.9
LED 7.9% 9.5 0.8
TOTAL 100% 32.3
* The technology wattage for incandescent and halogen lamps was set to the applicable lumen bin EISA
standard.
In this example, the market baseline reduced the savings baseline from the EISA standard
wattage of 43, to the market baseline of 32.3W-a 24.9% reduction of the baseline wattage.
This in turn reduced the gross energy savings impacts by the same percentage reduction . The
evaluation team followed this approach to uniquely calculate and aggregate each lumen bin and
product type.
The net to gross ratio for the residential lighting program was 64.5% as shown in Table D-6.
Table D-6: Residential Lighting Net to Gross Ratios and Net Verified Impacts
Reported . . Gr~~s Net Verified
S . Realization Verified Net to Gross 5 . avmgs . avmgs
(kWh) Rate Savings Ratios (kWh
(kWh) )
Simple Steps-LED 4,308,734 125.2% 5,394,253 65.9% 3,557,152
Simple Steps-CFL 14,866,096 132.5% 19,701 ,850 64.1% 12,623,297
Simple Steps -NP-LED 14,877 199.3% 29,644 65.9% 19,548
Simple Steps -N-CFL 165,598 159.7% 264,478 64.1% 169,456
Giveaway-CFL 3,660 200.5% 7,338 65.9% 4,839
Giveaway-LED 9,995 446.6% 44,637 64.1% 28,600
TOTAL 19,606,228 131.0% 25,689,564 64.5% I 16,561,380
D.4 Net to Gross Findings
The tables below outline the free ridership, spillover, and NTG values estimated for each
program.
1.-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs D-6
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 206 of 212
APPENDIX D NET TO GROSS METHODOLOGY AND FINDINGS
Table 0-7: Nonresidential Program Net To Gross Ratios
FR (savings . Program . h d) Spillover NTG we,g te
Nonresidential Electric
Site Specific 58% 0.4% 58%
Prescriptive Lighting 37% 3% 66%
EnergySmart Grocer NA 0% NA
Prescriptive Non-Lighting Other 24% 6% 82%
Table 0-8: Residential Program Net To Gross Ratios
Program FR (savings Spillover NTG
weighted)
Residential Electric
Appliance Recycling 75% 0% 26%
ENERGY STAR Homes 67% 0% 33%
Fuel Efficiency 27% 0% 73%
HVAC 54% 0% 46%
Shell 45% 0% 55%
Water Heat 74% 0% 26%
t-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs D-7
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 207 of 212
Appendix E Residential Lighting Logger Study Forms
E.1 Lighting Inventory Form
RE IDENTIAL LIGHTING HO R OF U E T DY : 0 -rr
(i&O t ID:
Customer N,me:
Cont ct PhOnt!:
Addr ;
Crtv. Sta c. ZIP·
u,g1n,,er:
Si!~ V rt 01te/Time:
Notes·
rve Ke
N/A = Not Apphcablt!
N)( = Not Available
Site lnfom1ation
I. PT misc Type: -----------
v,.-card tt:
Em II:
(. -fumtl~ :tQ hro I um. ~c-f•mil> ll• hod 2 um , t.h.i · amk 1-u \I ;,hie H
2.
1.
d)
4. II 111.: heatm •quip111.:nt l )l>.:: _
( umn , lln ho.ml I le.11 Pump. Otha-. pecil\)
5. I lom.: . \ir nditil 11111g Cquipm"'lll Typ : __ _
(('cnlral • ir, \\ indm, ,\ (". an, thcr . pcci y,. ooc
6. Ebtimat.:d munl.1<.'T' of portable d.: ni..: J ,·1~es ui;cJ III tJ, hou. c ( •.g. il'hun.:. TaJ,let ' mputc ,
J..:iudki>. l:lC. ?. --------
7. I lom . 11uon: Fe~I (t1ppr-o,. : ----­
y r 11 me 01 tni..:t d: ------­
Program Pa11i ipant Jnfo
9. s lhc c.-u.,;t mer r ..:all ~--c l\10g fr c b · fl igJ11s fr m vista in 2 12'' '\ X OJ... :
lf o. did th, u t m r in t.11 t "~ ligh (Y ~ DK)'?:
'-'1 Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs E-1
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 208 of 212
APPENDIX E
Rooa, Tyrw m Tabl
Koom # Room 1')·pe
I
2
3
" s
6
7
8
9
lO
11
12
13
'" 15
16
17
18
19
20
21
22
23
2-'
25
26
27
28
29
JU
l irch n
2-Oining Room
3= v1ng/Fam1ly/Great Room
4= Office
S-oyer/Hallway/S airway
RESIDENTIAL LIGHTING LOGGER STUDY FORMS
Koom l)es(-ription
Room T e:
6" Utility/Laundry Room
7 = Mast r Bedroom
8= Bedroom
9 =Toilet/Ball room
10-Basement
ll" At ic
l2•G rag
13:: Mech/ lectri'-l! Room
14= Close !/Storage
lS=EJltenor
I -Other. specify
t-'1 Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs E-2
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 209 of 212
APPENDIX E
U 1li11 ln\t"ltlon & IA N' Pl:u "lnt:-nl
•• ,,.l'ltyiw ...... ·~ ,.,. .. , ......
I .1-.,,~U.tiro, ·-~··t-
-·-§ ...... .,.,§ ... -, ... , .. ""Ot' x"•"' ty,-ot' ~ .., .... °'' "'"" .,. ,.,.,~ , ... "'(lf' ~~'1 11"" ..
, ... "'C~ ·-~1..., ..
=~~ .... ~~ .... 1-l----~----J
RESIDENTIAL LIGHTING LOGGER STUDY FORMS
Clblum.., JD. ~,,, ., .. § _,. ., .. § .. ··--•tt,tl •I• _,., lf"'""O~"
'O'tl/f ol. f' 1-yf-,.,.. III ..... V\.oo>..-
•r4:,)f•I r _,_ 1-• ..... •1-r
• .,,,, •• .-..--,., 1,--·>t•w
' -~ ... ,§ .........
I '"~•~I
I f'-f"4'
1.....,.. ......
t.-1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs E-3
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 210 of212
APPENDIX E RESIDENTIAL LIGHTING LOGGER STUDY FORMS
Lamp Shape Lookup Table
Bl.ibS •
1. Stand•clf
Pear/candelabra
(A~amp)
2. TwisV spiral
3. Globe (I Wied
for bathroom
vanity)
4.=Bug lighl
(YII owJblue/red
colot)
L-Kitchen
2.,. inln Room
Code
1
2
3
4
3=Livini,/F milv/Grt' t qoom
4:0f Ct'
I e
l• On/O'f (switch, plug, pull ming. etc.I 5• limer
Bulb e Code Ima e
S. Spot/
reftector/
Flood light
(twlcal In can
lghls)
6. Clrcline
7.
a..11e1Ao,pedo
5
6
7
8
9 • Tcilet/!la[hroom
I Base ent
ll=Attic
lr-C.lra(l('
l•\1~1
IJ#
13-Mech/Electrical R.oom
14:0oset/S:o,ai;:e
-t nor
l6=0tht-r, $f'l-f'Olv
2 Dimmer 6-3·wav
4•Pit,fi l,P
i• C\J se
= Mo io Photo se or
<1 = Remote Cort I
1= inc. d!!!.t
2 (H
LEO
7= 0 hilt, peci V
l•U OOl LightOn/011
2= \J O Dill t.ght Or/011 t
= CA.-llf ng surf,= moun
6 Tract U~tlng
7= 'i ndtd I ndt-H~r J
S= Recr.sscd (c.m II ht)
1,Sb nd>rct/hor/C>n~ obn,/;. t,mp ~-Sp I
; •Twt s t1 a &-<1n;;l lnp.
l=Gtooe 1 "" n<Ypeao
6= Otl'o!-, 1p ilV
, ,poclfy
t.."1 Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs E-4
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 211 of 212
APPENDIX E RESIDENTIAL LIGHTING LOGGER STUDY FORMS
E.2 Recruitment Materials
,,:;tllSTII :#
Ju e 30. 201 5
Dear <customer>,
Avista Utilities is conductina a residential liehtina study in the homes of our customers. You have random If been
selected as a potential participant. In order to better understand how our customers use energy and Improve
our lighting rebates programs for customers like you. Avista Utilities has re ained Nexant, an expert m the
energv eff,clency evaluation f eld, to conduct a llghtlng study on our ~half to meuure how many hours per day
customers iHe usine hehu in vanous areas of their homes
We would hke to offer you t e opportunity o participate in this study. Participation IS Voluntary and
participants of the study will receive $75 In pre-paid Visa sih cards. If you are Interested m part1C1patmg, or
vould lrke more Information, please call l-8S5-828-n4S to spea to a Nexant repr~entatlve. Please re erence
your Study ID: <study Id>.
Availability rs hmlted, so participants will~ adm ted on a first-come, frrst ~ervt! basis. If you decide to
participate In the study, an appointment will be scheduled at your earl lest convenience for a NeXint l!'Jaluator
to vrsit your ome and Insta ll 4 to 8 small light measurina devices ("logers"J which measure only t e amount of
time the Ilg ts are turned on. A follow-up appointment will be scheduled In approxlmatety six months for the
loge rs to be coll~ted.
The resu so this study will elp us underst.tnd ow our customers use t eir I ehts so that we tin improve our
energy eff,ciency proarams in the future. If you ave any quest10ns about the study, please flive me a call.
In addi on, Avista Ut1h les offers several Res.den .al rebates including;
• H eh Efficiency Equipment (Furnace, Boiler, V;iriable Spe d Moton, Smart Thermostats)
• Insulation (Attic. Wall, Floor)
• Windows
• Space & Water Heat conversions from Electric
For a complete hst of rebates and requirements, applicabon forms or to submit an online application go to
Or you can contact rebates@avistautil ies.com or 80().227-9187 with
quest,ons.
T an you aeain for youn llinenessto parucipa e.
Sincerely,
David Schafer ·DSM Proaram Manaeer
A111Sta Ut1ht s -P: 509-49~688 E: Oavid.Schafer@)av1stacorp.com
: ~:111sr11
---
t..1Nexanr Impact Evaluation of Idaho 2014-2015 Energy Efficiency Programs E-5
Exhibit No. 2
L. Roy, Avista
Schedule 1, Page 212 of 212
~1Nexanr
Reimagine tomorrow.
research) into) action ~
Process Evaluation of Avista's 2014-
2015 Energy Efficiency Programs
Submitted to Avista Utilities
May 26, 2016
Principal authors:
Mersiha McClaren, Ryan Bliss, Paul Schwarz, Nathaniel Albers,
Benn Messer, and Zac Hathaway; Research Into Action, Inc.
Lynn Roy and Cherlyn Seruto; Nexant Inc.
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 1 of 151
Contents
1 Executive Summary ............................................................................. 1
1.1 Nonresidential Key Findings ..................................................................... 3
1.1.1 Program Participation , Awareness and Involvement ......................... 3
1.1.2 Influences on Customers Decision Making ........................................ 3
1.1.3 Program Experience .......................................................................... 3
1.1.4 Opportunities for Increasing Program Participation ........................... 4
1.1 .5 Commercial Uptake of Simple Steps Measures ................................. 4
1.1 .6 Small Business Key Findings ............................................................. 4
1.2 Residential Key Findings ........................................................................... 4
1.2.1 Program Delivery ............................................................................... 4
1.2.2 Awareness and Familiarity with Avista's programs ............................ 5
1.2.3 Program Experience .......................................................................... 5
1.2.4 Motivations and Barriers to Participation ........................................... 6
1.2.5 Participation Trends ........................................................................... 6
1.2.6 Future Opportunities .......................................................................... 7
1.3 Conclusions and Recommendations ........................................................ 7
1.3.1 Cross-cutting ...................................................................................... 7
1.3.2 Nonresidential, Including Small Business .......................................... 9
1.3.3 Residential ......................................................................................... 9
2 Introduction ........................................................................................ 12
2.1 Purpose of Evaluation .............................................................................. 12
2.2 Description of Nonresidential Programs ................................................ 14
2.2.1 Prescriptive ...................................................................................... 14
2.2.2 Site Specific ..................................................................................... 15
2.2.3 Energy Smart Grocer ....................................................................... 15
2.2.4 Small Business Program .................................................................. 15
2.3 Description of Residential Programs ...................................................... 16
2.3.1 Appliance Recycling ........................................................................ 17
2.3.2 ENERGY STAR® Homes ................................................................. 17
L,1 Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 2 of 151
2.3.3 Fuel Efficiency ................................................................................. 18
2.3.4 Heating, Ventilation , and Air Conditioning (HVAC) Rebates ............ 18
2.3.5 Water Heat Rebates ........................................................................ 18
2.3.6 Shell Measures ................................................................................ 18
2.3.7 Simple Steps, Smart Savings .......................................................... 18
2.3.8 Home Energy Reports ..................................................................... 18
2.3.9 Low-Income ..................................................................................... 18
3 Methods .............................................................................................. 20
3.1 Cross-cutting activities ............................................................................ 21
3.1.1 Staff and Implementer Interview Methods ....................................... 21
3.1.2 Contractor Sample ........................................................................... 21
3.2 Nonresidential Activities .......................................................................... 23
3.2.1 Participant Survey Sample and Methods ......................................... 23
3.2.2 Nonparticipant Survey Sample and Methods ................................... 25
3.2.3 Small Business Process Evaluation Methods .................................. 26
3.3 Residential Activities ............................................................................... 28
3.4 Special Studies Activities ........................................................................ 30
3.4.1 Declining Participation Rates ........................................................... 30
3.4.2 Participation Rates Among Opower Behavioral Program Participants
and Nonparticipants ......................................................................... 31
3.4.3 Commercial Uptake of Simple Steps Measures Methods ................ 31
3.5 Review of Program Logic Models ........................................................... 32
4 Nonresidential Process Results ........................................................ 33
4.1 Program Administration ............................................................................. 33
4.2 Program Awareness and Involvement.. ...................................................... 34
4.2.1 Contractor Involvement.. .................................................................. 34
4.2.2 Nonresidential Customer Awareness ............................................... 35
4.3 Influences on Customers Decision Making ................................................ 39
4.3.1 Energy Practices and Policies ......................................................... 39
4.3.2 Customer Motives ............................................................................ 41
4.3.3 Contractors' Sales Practices ............................................................ 42
4.4 Program Experience ................................................................................... 43
'-"Nexanr
4.4.1 Participant Program Satisfaction ...................................................... 43
Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs ii
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 3 of 151
4.4.2 Contractor Program Satisfaction ...................................................... 47
4.4.3 Perceived Value of Rebates -Contractor Perspectives .................. 49
4.4.4 Driving lncented Upgrades -Contractor Perspectives .................... 50
4.4.5 Participant Concerns ....................................................................... 52
4.5 Opportunities for Increasing Program Participation ................................... 52
4.6 Freeridership and Spillover ..................................................................... 55
4.6.1 Freeridership .................................................................................... 55
4.6.2 Participant Spillover ......................................................................... 57
5 Small Business Process Results ...................................................... 59
5.1 Small Business Process Evaluation Overview ...................................... 59
5.2 Summary of Program Data ...................................................................... 59
5.3 Staff and Implementer Interviews ........................................................... 61
5.3.1 Program Goals and Plans for the Future ......................................... 61
5.3.2 Implementation and Delivery ........................................................... 62
5.3.3 Marketing and Outreach .................................................................. 62
5.3.4 Program Successes ......................................................................... 63
5.4 Participant Surveys .................................................................................. 63
6 Residential Process Results ............................................................. 70
6.1 Program Administration .......................................................................... 70
6.1.1 Rebate Programs ............................................................................. 70
6.1.2 Appliance Recycling Program .......................................................... 76
6.1.3 Home Energy Reports Behavior Program ........................................ 79
6.1.4 Low-income Program ....................................................................... 82
6.2 Customer Experience with Rebate Programs ........................................ 85
6.2.1 Awareness and Familiarity with Avista Programs ............................ 86
6.2.2 Motivation and Barriers to Participation ........................................... 92
6.2.3 Program Experience ........................................................................ 94
6.2.4 Attitudes toward Energy Use and Conservation .............................. 98
6.3 Customer Experience with Simple Steps, Smart Savings Program ..... 99
6.4 Customer Experience with the Behavior Program .............................. 101
6.5 Freeridership and Spillover ................................................................... 103
t.-1Nexanr
6.5.1 Freeridership .................................................................................. 103
Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs iii
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 4 of 151
6. 5.2 Participant Spillover ....................................................................... 105
7 Special Studies ................................................................................. 106
7.1 Declining Program Participation Rates ................................................ 106
7.1.1 Nonresidential Participation Trends ............................................... 106
7.1.2 Residential Participation Trends .................................................... 112
7.2 Participation Rates Among Opower Behavioral Program Participants
and Nonparticipants ............................................................................... 118
7.2.1 Data and Methods ......................................................................... 118
7.2.2 Findings ......................................................................................... 122
7.2.3 Discussion ..................................................................................... 125
7.3 Commercial Uptake of Simple Steps Lighting .......................................... 126
7. 3. 1 Objective ........................................................................................ 127
7.3.2 Results ........................................................................................... 127
7.3.3 Retailers Experience with Simple Steps ........................................ 128
7.3.4 Other Opportunities for Simple Steps ............................................ 129
8 Conclusions and Recommendations .............................................. 130
8.1 Cross-cutting .......................................................................................... 130
8.2 Nonresidential, Including Small Business ........................................... 131
8.3 Residential .............................................................................................. 132
t..1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs iv
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 5 of 151
List of Figures
Figure 4-1: Nonresidential Contractor Activity Level. ................................................................................. 35
Figure 4-2: Source of Program Awareness (Multiple Responses Allowed) ................................................. 36
Figure 4-3: Relative Association of Participant Awareness with Participant Population ........................... 37
Figure 4-4: Length of Time Energy Related Goals and Policies Have Been In Place at Nonparticipants'
Organizations .............................................................................................................................................. 40
Figure 4-5: Contractor Perspective: Importance of Reasons Nonresidential Customers Implement Avista
Energy Efficiency Projects (n = 29) .............................................................................................................. 42
Figure 4-6: Percentage of Equipment Sold (n = 28) .................................................................................... 42
Figure 4-7: Satisfaction with Program Elements ......................................................................................... 44
Figure 4-8: Reasons for Contact with Avista Representatives .................................................................... 45
Figure 4-9: Who Prepared Application? ...................................................................................................... 46
Figure 4-10: Clarity of Avista Program Information (n = 190) ..................................................................... 47
Figure 4-11: Commercial Contractor Satisfaction with Program Elements (n = 24) ................................... 48
Figure 4-12: Contractor Perceived Value of Avista Rebates (n = 29) .......................................................... 49
Figure 4-13: Freeridership Values Over Time ............................................................................................. 56
Table 4-11: Number of Participants Reporting a Spillover Action .............................................................. 58
Figure 5-1: Areas targeted by SB program in 2015 ..................................................................................... 60
Table 5-5: Satisfaction with Program Elements .......................................................................................... 67
Figure 5-2: Considerations When Making Building Upgrades (n = 34) ....................................................... 68
Figure 6-1: Contractors Number of Avista-Rebated Projects (n=53) .......................................................... 71
Figure 6-2: Residential Contractors Satisfaction with Program Elements (n=46)* .................................... 72
Figure 6-3 : How Program Helps Residential Contractors (n=53) ................................................................ 74
Figure 6-4: Benefits of Efficient Equipment Mentioned During Sales (n=52)* ........................................... 75
Figure 6-5: CAPs Delivery Process to Low-Income Customers ................................................................... 82
Figure 6-6: Source of Program Awareness (2015 Nonparticipants) ........................................................... 87
Figure 6-7: Source of Program Awareness (2014 and 2015 Participants) .................................................. 88
Figure 6-8: Relative Association of Residential Participant Awareness with Participant Population ......... 89
Figure 6-9: Percentage of 2014 and 2015 Participants Familiar with Avista Rebates for Other Programs 90
Figure 6-10: Motivations for Participating in a Rebate Program, by Program (2014 and 2015 Participants;
Multiple Responses Allowed)• ................................................................................................................... 93
Figure 6-11: Satisfaction with Program Elements (2014 and 2015 Participants) ....................................... 95
Figure 6-12 : Satisfaction Rating, by Program (2014 and 2015 Participants) a, b ......................................... 96
Figure 6-13: Clarity of the Program Information by State across 2014 and 2015 (2014 and 2015
Participants) a .............................................................................................................................................. 98
Figure 6-14: Agreement with Eight Statements Associated with Energy Usage and Conservation• ......... 99
Figure 6-15: Ease of Finding Lighting and Low-flow Showerheads (2015 Nonparticipants)" ................... 101
Figure 6-16: Usefulness and Satisfaction with HER (2014 and 2015 Participants and 2015
Nonparticipants; n=28) a ........................................................................................................................... 103
Figure 6-17: Free ridership Over Time* ..................................................................................................... 104
Table 6-10: Number of Participants Reporting a Spillover Action ............................................................ 105
Figure 7-2: Percent Change Year-to-Year by Measure Rebate Type, 2010-2015 Nonresidential Program
Data* ......................................................................................................................................................... 108
t..1Nexanr Process Evaluation of Avista 's 2014-2015 Energy Efficiency Programs V
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 6 of 151
Figure 7-3: Percent of Nonresidential Customers Participating in Multiple Programs or Same Program
Multiple Times, 2010-2015 Nonresidential Program Data ....................................................................... 112
Figure 7-4: Reported Number of Residential Rebates, 2010-2015 Program Data ................................... 113
Figure 7-5: Percent Change Year-to-Year by Measure Rebate Type, 2010-2015 Residential Program Data
.................................................................................................................................................................. 115
Figure 7-6: Percent of Residential Customers Participating in Multiple Programs or Same Program
Multiple Times, 2010-2014 Residential Program Data ............................................................................. 117
Figure 7-7: Monthly Average Daily Energy Usage by Group ..................................................................... 123
Figure 7-8: Average Cumulative Percent Electricity Usage Compared to Nonparticipants ...................... 124
Figure 7-9: Average Daily Percent Electricity Usage for Each Month Compared to Nonparticipants* .... 125
Figure 7-10: Estimates of Percent of Products in Commercial Sector ...................................................... 128
List of Tables
Table 1-1: Overview of Data Collection Activities ......................................................................................... 2
Table 2-1: Process Evaluation Objectives and Information Sources ........................................................... 13
Table 2-2: Key Energy Efficiency Programs ................................................................................................. 14
Table 2-3 : Residential Program Type and Description ................................................................................ 17
Table 3-1: Overview of Data Collection Activities ....................................................................................... 20
Table 3-2: Contractor Population and Sample ............................................................................................ 22
Table 3-4: Nonresidential Participant Survey Completions by Program Type and Fuel. ............................ 24
Table 3-5: Population and Completed Sample Distribution by Program .................................................... 25
Table 3-6: Nonparticipant Nonresidential Population and Survey Completes ........................................... 26
Table 3-7: Overview of Small Business Data Collection Activities .............................................................. 27
Table 3-8: Disposition Summary ................................................................................................................. 28
Table 3-9: Distribution of Population, Sample, and Completed Sample .................................................... 28
Table 3-10: Sample Distribution for Residential Program Participants and Nonparticipants .................... 29
Table 3-11: Residential Participant Surveys ................................................................................................ 30
Table 3-12: Retailer Sales Data in Simple Steps .......................................................................................... 32
Table 4-1: Nonparticipant Awareness of Avista Rebates (n= 70, Multiple Responses Allowed) ................ 38
Table 4-2: Interest in Future Participation (Multiple Responses Allowed) ................................................. 38
Table 4-3: Nonresidential Customer Preferred Method of Receiving Information from Avista (Multiple
Responses Allowed) .................................................................................................................................... 39
Table 4-4: Energy Savings Policies and Practices ........................................................................................ 40
Table 4-5: Reasons for Applying to Program (Multiple Responses Allowed) (n = 305) .............................. 41
Table 4-6: Contractors' and Customers' Roles in Initiating Upgrades ........................................................ 50
Table 4-7: Contractors' and Customers' Roles in Discussing Rebates ........................................................ 51
Table 4-8: Contractors' and Customers' Relative Roles in Driving lncented Upgrades .............................. 52
Table 4-9: Equipment Replacements or Upgrades Made by Nonparticipants in Past Two Years or Planned
for Next Two Years (Count and Percent of Total) ....................................................................................... 53
Table 4-10: Factors Influencing Nonparticipants' Recent or Planned Purchase of Energy Efficient
Upgrades ..................................................................................................................................................... 54
t.-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs vi
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 7 of 151
Table 5-1 Participation to Date Compared to Estimated Participation ...................................................... 61
Table 5-2: Small Business Respondent Characteristics (n = 34) .................................................................. 64
Table 5-3: Reasons for Participating in SB Program (n = 34) ...................................................................... 65
Table 6-1: Contractor Suggestions for Additional Program Measures ....................................................... 76
Table 6-2: Nonparticipant Awareness of Avista Incentives, (n=29; Multiple Responses Allowed) ............ 90
Table 6-3: Additional Energy Saving Information Requested (2014 and 2015 Participants and 2015
Nonparticipants; Multiple Responses Allowed) .......................................................................................... 91
Table 6-4: Preferred Method of Receiving Information from Avista (2014 and 2015 Participants and 2015
Nonparticipants; Multiple Responses Allowed) .......................................................................................... 91
Table 6-5: Motivations for Participating in a Rebate Program (2014 and 2015 Participants; Multiple
Responses Allowed) .................................................................................................................................... 92
Table 6-6: Future Upgrades Planned (2015 Nonparticipants; n=70; Multiple Responses Allowed) .......... 94
Table 6-7: Barriers to Making Energy Efficiency Improvements (2015 Nonparticipants; n=38; Multiple
Responses Allowed) .................................................................................................................................... 94
Table 6-8: Suggestions for Improving the Rebate Program (2014 and 2015 Participants; n=lOO; Multiple
Responses Allowed) a .................................................................................................................................. 97
Table 6-9: Purchases of CFLs, LEDs, or Showerheads in 2015 (2015 Nonparticipants; n=50; Multiple
Response Allowed) • .................................................................................................................................. 100
Table 7-1: Lighting Rebate Amounts By Energy Savings By Measure Type, 2010-2015 Nonresidential
Program Data ............................................................................................................................................ 109
Table 7-2: Theoretical Versus Actual Participation, Accounting for Discontinued Measures, 2010-2015
Nonresidential Program Data ................................................................................................................... 110
Table 7-3: Percent Change in Rebate Amounts and Counts, 2010-2015 Nonresidential Program Data .. 111
Table 7-4: Theoretical Versus Actual Participation, Accounting for Discontinued Measures, 2010-2015
Residential Program Data ......................................................................................................................... 114
Table 7-5: Percent Change in Rebate Amounts and Counts, 2010-2015 Residential Program Data ........ 116
Table 7-6: Number of Opower Participants and Nonparticipants Before and After Removing Random
Sample from Idaho Control Group ............................................................................................................ 119
Table 7-7: Number of Opower and Avista Rebate Participants and Nonparticipants .............................. 120
Table 7-8: Average Daily kWh Usage Before and During the Treatment Period by Group ...................... 123
Table 7-9: Summary Items in the Commercial Sector Attributable to Simple Steps ................................ 127
t-'1 Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs vii
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 8 of 151
1 Executive Summary
Nexant Inc. and Research Into Action (collectively the evaluation team) conducted an impact
and process evaluation of Avista's 2014-2015 residential and nonresidential energy efficiency
programs. This report documents findings from the process evaluation activities. The main
purpose of the process evaluation was to identify any improvements needed at the portfolio
level to increase program effectiveness and efficiency. The evaluation team conducted the
evaluation by reviewing program data and through interviews and surveys with various market
actors. Table 1-1 lists the data collection activities and key topics covered by each data source.
L-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 1
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 9 of 151
EXECUTIVE SUMMARY
Table 1-1: Overview of Data Collection Activities
Data Source Type When Analytic Key Topics
(Sample by Techniques
sector)
Staff (16; 4 nonres.
and 12 res.)
Implementers (7; 1
nonres. and 6 res.)
Interview · Oct. 2015 Qualitative,
thematic
Contractors (82; 29 Survey Aug. 2015, Quantitative,
nonres. and 53
res.)
Participants (680;
305 nonres. and
339 res.)
Nonparticipants
(140; 70 nonres.
and 70 res.)
Retailers (27)
Small Business
staff and
implementer (2)
Small Business
installers (2)
Small Business
participants (34)
Database analysis
Oct. 2015 univariate
Survey
Survey
; May 2015-i Feb. 2016
! Oct.-Nov.
! 2015
Survey I Jan. 2016
Interview ! December
J 2015
Interview I December
! 2015
Survey January
2016
Database Feb. 2015
review j -April
! 2016
and bivariate
frequencies
Quantitative,
l univariate
: and bivariate
frequencies
Quantitative,
univariate
! and bivariate
frequencies
Quantitative
Qualitative,
thematic
Qualitative,
thematic
Quantitative,
univariate
! and bivariate
frequencies
Quantitative
• Program goals and processes
• Communication and coordination
• Data tracking
• Future program opportunities
• Outreach
• Program awareness
• Satisfaction
• Motivations to participate
• EE Sales practices
• Net-to-Gross
• Program awareness
• Satisfaction
• Program experience
• Net-to-Gross
• Commercial uptake of Simple Steps products
• Program awareness
• Experience with EE
• Commercial uptake of Simple Steps products
• Spillover
• Commercial uptake of Simple Steps products
• Program goals and requirements
• Communication and coordination
• Marketing
• Implementation
• Role in outreach
• Data collection and reporting
• Challenges and barriers to participation
• Implementation successes
• Program experience
• Satisfaction
• Future EE plans
• Business characteristics
• Identify participation patterns
• Number of repeat participants
• Assess HER+rebate savings
The 2014-2015 evaluation shows high levels of program awareness among all of Avista's
customers and shows high levels of satisfaction among program participants and contractors.
Program participants and contractors were complementary of Avista staff and generally
appreciated the opportunities to save money, save energy, and improve their properties that the
programs provide. The evaluation also shows that there are areas the programs could enhance
to make them better able to respond to the ever changing market conditions in which these
programs operate.
t-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 2
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 10 of 151
EXECUTIVE SUMMARY
The results of the process evaluation identified the following key findings, organized by sector
and by theme. Conclusions and recommendations follow the key findings.
1.1 Nonresidential Key Findings
1.1.1 Program Participation, Awareness and Involvement
• Program participation declined over the last few years, especially in lighting. The
change to a T8 baseline lowered incentives available for T12 upgrades negatively
effecting participation.
• The Energy Smart Grocer market may need to be expanded to boost participation.
Staff reported that Energy Smart Grocer has seen diminished savings over the last few
years due to the market getting saturated. Program staff is seeking new markets, such
as restaurants and other food service establishments, to boost participation but that
segment alone may not singularly compensate for the savings decline.
• Contractors play a notable role in the acquisition of projects, the implementation of
projects, and in informing customers about rebates. More than half of contractors
reported they play a key role in initiating upgrades and communicating rebate
opportunities to customers. Customer's awareness of the program through contractors
was associated with an increased likelihood of program participation, and contractors
appear to be playing a larger role in preparing applications than in years past.
1.1.2 Influences on Customers Decision Making
• Having a corporate culture that prioritizes energy savings appears associated with
current participation. Participants are twice as likely as nonparticipants to report having
an energy saving policy or practice in place.
• Survey results show that saving money, improving operations and maintenance, and
improving the comfort of facilities are key motivators to participation. Contractors and
participants report that saving money motivates customers to participate. According to
contractors, improving operations and maintenance also was an important motive of
customers. There is also some evidence that improving the comfort of one's building is
an important motivation for participants that implemented a gas project.
1.1 .3 Program Experience
• Participants were largely satisfied with Avista's programs. The large majority of
participants reported high levels of satisfaction with program elements such as the time it
took to apply, the variety of equipment available, and the quality of the products
received. A minority of participants could not rate their satisfaction with their project's
energy savings so soon after project completion.
• Contractors and participants reported high satisfaction with their interactions with
program staff. Most participants sought assistance from staff regarding their application
compared to any other topics.
• Contractors are not engaged or knowledgeable about Avista's marketing efforts.
Among contractors, the quality, and quantity of Avista's marketing received lower
satisfaction scores than any other program element.
t-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 3
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 11 of 151
1 EXECUTIVE SUMMARY
• Contractors value Avista's rebates but there is an opportunity to use the
programs to train contractors. Contractors reported they value Avista's rebates to help
them sell jobs and push customers to install more efficient equipment.
1.1.4 Opportunities for Increasing Program Participation
• Planned equipment upgrades create opportunities for continued program-related
savings. Almost a third of nonparticipants reported they will make an upgrade in the
next two years that could involve an efficiency upgrade, and the majority of those
reported they would make a lighting upgrade.
1.1.5 Commercial Uptake of Simple Steps Measures
• Customers are installing Simple Steps items in commercial buildings. Survey
results show that between 5 and 12% of Simple Steps CFLs and about 12% of Simple
Steps LEDs are purchased for implementation in commercial properties.
1.1.6 Small Business Key Findings
• The program is running smoothly. The program is meeting its overall goals for
measure installation and savings and there were no reports of any systemic problems
with interval communication or administration.
• There is an opportunity to improve the efficiency of small businesses, particularly
in the lighting area. Program data shows and installers reported ample opportunity in
the market to replace T12s. More than a third of 2015 participants had T12 fixtures.
• Staff and participants reported high levels of satisfaction with the measures and
services provided by the program . Very few participants reported removing any of the
installed measures on their own, however the impact evaluation activities did find that a
relatively significant number of participants surveyed did remove on their own at a later
time.
• The outreach model of the program provides Avista with an opportunity to
develop relationships with their customers and engage customers about other
program opportunities. Installers often tell participants about energy saving actions
they could take outside of the scope of the program. Most upgrade recommendations
pertained to lighting and about a third of participants said they plan on making a lighting
upgrade in the next year.
1.2 Residential Key Findings
1.2.1 Program Delivery
• Although rebate programs are running smoothly, there is an opportunity to
engage contractors more with Avista's programs. Avista primarily interacts with
contractors when contractors call to request information on behalf of their customers.
Avista does not currently offer any formal training for contractors on the rebate
programs, and Avista staff only occasionally visit contractor offices to hand out rebate
information, the only face-to-face outreach activity reported by program staff.
t-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 4
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 12 of 151
1 EXECUTIVE SUMMARY
• Rebates are an effective sales tool for contractors. Most contractors agreed that they
always tell customers about rebates and that the rebates help them sell more energy
efficient equipment and services to their customers, a finding that is supported by Avista
staff.
• Simple Steps, Smart Savings, Opower Home Energy Reports, and Low-income are
running smoothly. There were no reports of systemic problems with recruitment,
communication, and implementation. Challenges encountered mainly revolved around
customer databases. For example, smaller retailers in the Simple Steps, Smart Savings
program struggle with reporting sales data because they lack a sophisticated reporting
system that larger retailers typically have and Opower was unable to send Home Energy
reports for about six months in 2015 when Avista changed its customer billing system in
January/February 2015.
1.2.2 Awareness and Familiarity with Avista's programs
• Contractors were aware and familiar with Avista's programs. More than three­
quarters of residential contractors reported completing projects that received Avista
rebates for at least the past five years. Contractors also spent considerable time working
on Avista-rebated projects.
• Avista's marketing efforts are working in generating customer awareness. The
source of program awareness among customers is consistent with Avista's marketing
activities. Of the nonparticipants who were aware of Avista incentives (41 % of the
sample), about half (45%) reported learning about Avista's rebate programs through
channels Avista used for outreach.
• Participants highlighted the importance of contractors in advertising Avista's
programs. Contractors were the main source of awareness for participants. Awareness
through a contractor was greater than any other source and was by far the greatest
predictor of program participation.
• Awareness of other Avista programs among participants varied. Fewer than half of
surveyed participants were familiar with other energy efficiency rebate opportunities from
Avista (besides the program in which they had participated) and this varied by program.
Highest awareness was among Water Heat and Fuel Efficiency participants and lowest
among ENERGY STAR Homes participants.
1.2.3 Program Experience
• Participants were satisfied with the rebate programs. More than four-fifths (84%) of
surveyed participants reported their overall satisfaction with their Avista rebate program
experience as being either "very" or "completely" satisfied.
• Contractors satisfaction with the rebate programs varied. Most (80-85%)
contractors reported being satisfied with the length of time needed to complete the
paperwork and range of qualifying products. The majority (67%) were satisfied with
Avista website and about half (54%) reported being satisfied with the rebate amounts.
• Contractors are unfamiliar with Avista's marketing efforts. Contractors provided the
lowest satisfaction ratings on the marketing aspects of the rebate programs. About one­
tenth (11%) indicated they were dissatisfied with the amount of Avista's marketing and
4-"1 Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 5
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 13 of 151
EXECUTIVE SUMMARY
nearly one-tenth (9%) noted they were dissatisfied with the quality of marketing.
However, in their follow-up comments, these contractors indicated they were largely
unaware of Avista's marketing efforts or only saw the materials sporadically, indicating
that contractors may be more unfamiliar with Avista's marketing of the rebate programs
than they are dissatisfied.
• Nearly all rebate participants found program-related information clear. A majority of
participants reported that program-related information (e.g., website or rebate form) was
clear on how to apply for a rebate, which equipment qualified for a rebate, expected
energy savings of program eligible equipment, and who to contact if any issues arose.
Program materials were less clear about the quality assurance process and regarding
which equipment or items qualified for rebates for Shell participants than for other
program participants.
• Both participants and nonparticipants expressed interest in receiving additional
information on Avista's program offers. About three-quarters (77%) of participants
and more than half (59%) of nonparticipants reported being interested in receiving
energy-saving and/or program information from Avista.
• Home Energy Reports can be effective at engaging customers and motivating
them to take action such as participating in Avista's rebate programs, such as the Fuel
Efficiency program. These findings validate Avista's strategy to promote the rebate
programs via the home energy reports.
1.2.4 Motivations and Barriers to Participation
• Top three motivations for participating in Avista's rebate programs were: increased
comfort, saving energy, and saving money. Between 83-88% of participants reported
these three motivations for participation.
• Up-front cost was the most frequently cited barrier to completing an energy efficiency
upgrade by nonparticipants. This indicates an importance of offering an incentive to
customers for home improvement projects.
• The second most frequently cited barrier was living in a rental property.
Nonparticipants reported that living in a rental property prohibits them from making
improvements to their home. Demographic analysis revealed that 27% of surveyed
nonparticipants and 3% of surveyed participants were renters.
1.2.5 Participation Trends
• Participation in Avista's residential rebate programs increased in the last two
years. The number of rebates declined sharply from 2010 to 2013, and then increased
by 51% from 2013 to 2014 and by 43% from 2014 to 2015. Note that the evaluation
team only examined the number of rebates for these six measures: 1) ENERGY STAR
appliances, 2) shell, 3) HVAC, 4) fuel conversions (or Fuel Efficiency program), 5) water
heater, and 6) ENERGY STAR Homes measures. Shell measure rebates, in particular,
increased by 507% from 2013 to 2014. The decline in the overall number of rebates
examined from 201 Oto 2013 was related to the discontinued rebates for appliance
measures, which accounted for 17,332 of the total decline of 23,453 measures.
'-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 6
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 14 of 151
EXECUTIVE SUMMARY
1.2.6 Future Opportunities
• Program delivery actors suggested that ductless heat pumps, water heating measures,
and plug load technologies could be an opportunity for Avista. Contractors provided
suggestions for additional equipment they would like rebated through the programs, and
ductless heat pumps and hot water saving measures were the most commonly cited.
The CLEAResult representative listed several technologies that Avista could consider if
they wanted to add measure to the program: advanced power strips, new lighting
controls, water heaters, and ductless heat pumps.
• An Opower representative suggested several customer engagement program
opportunities: 1) adding a monthly email report on top of the mail report; 2) alerting
customers of their bills (if high); 3) offering customers a "points and rewards" option
where they can collect points based on how much energy they save and redeem those
points for a gift card; and 4) targeting small and medium businesses or low-income
customers with the reports.
• The Community Action Partners who deliver the low-income program for Avista
also provided several suggestions: 1) offering more in-depth education about saving
energy such as offering a class to customers; 2) providing more funds for safety and
health measures; 3) providing some funding for renewable measures.
1.3 Conclusions and Recommendations
The evaluation team concluded the following and provides several suggestions for Avista's
programs. This section begins with conclusions and recommendations pertinent across all
programs (cross-cutting), followed by nonresidential and small business, and ending with
residential specific conclusions and recommendations.
1.3.1 Cross-cutting
Conclusion 1: Contractors are key program partners.
Contractors are the driving force of Avista's rebate programs, as they inform both nonresidential
and residential consumers about Avista's rebate opportunities and convince them to purchase
qualifying equipment. The nonresidential contractors also initiate a notable portion of work in
comparison to customer-initiated jobs and appear to be playing a larger role in application
preparation than in years past. Both nonresidential and residential customers report being highly
satisfied with contractors and are taking into account contractor's recommendations on what to
install.
Recommendations: Increase support for contractors.
Consider the following suggestions to continue strengthening relationships with contractors and
to improve their effectiveness in generating program savings:
1. Offer an opt-in mailing list to contractors. Contractors subscribed to this mailing list
would receive regular information on program offers, changes, trainings, and other
program supporting information. This list would be open to any interested contractor.
L-"'1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 7
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 15 of 151
EXECUTIVE SUMMARY
2. Promote outreach to contractors: Encourage program staff and account executives
to engage further with contractors by continuing and perhaps increasing their
involvement with contractor-related resources such as the Northwest Lighting
Network. This work can further educate contractors and nudge them to cross­
promote the rebate programs to their customers. Additionally, training may help
contractors' up-sell high efficiency equipment through the program by improving their
understanding of and ability to sell high efficiency solutions. Therefore, Avista should
continue to support contractors attending NEEA's training sessions including their
recently launched comprehensive training for lighting contractors and distributors.
3. Share effective messaging or marketing collateral with contractors. Contractors could
support program and marketing staff by providing insights into how to best target
certain customer types, learn from Avista on how to better target certain customer
segments, and possibly promote cross-program referrals and participation. As
findings from the evaluation show that most contractors specialize in the
nonresidential or residential sectors, even if they serve both, developing sector­
specific messaging may be particularly effective.
4. Investigate offering cooperative (co-op) marketing. Co-op marketing can help
contractors effectively market the program consistent with Avista's objectives and
increase customer perceptions of contractor's credibility and cross-promote other
programs.
Conclusion 2: Avista and its implementation contractors deliver rebate programs
efficiently, and promoting the programs further could help maintain or even increase
participation.
Several indicators suggest program promotions could be optimized. First, participants and
nonparticipants expressed high interest in learning more about Avista 's rebate programs,
indicating that although they may be aware of Avista's offers, their knowledge is limited.
Second, a majority of residential participants who indicated learning primarily about Avista's
offers through contractors were not aware of other program opportunities outside the program
they participated in.
Recommendation: Develop more abilities to target marketing. For example, cross­
promote programs to recent participants by acknowledging their recent participation and
informing them of other program opportunities applicable to their home or business.
Recommendation: For residential customers, continue improving messaging in direct
mail promotions to better communicate program information since residential customers
prefer to receive this information via mail.
t-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 8
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 16 of 151
EXECUTIVE SUMMARY
1.3.2 Nonresidential, Including Small Business
Conclusion 3: Although declining participation rates could threaten Avista's ability to
achieve long-term goals, evaluation results point to opportunities to drive additional
savings.
Developing new strategies to encourage deeper savings or increased participation will be
paramount to reversing the decline in participation and achieving long-term savings goals.
Almost one-third of nonparticipants reported they will make a building upgrade in the next two
years, indicating a continued potential for program participation. In particular, evidence suggests
that much opportunity remains for converting lighting from T12s.
Recommendation: Develop a marketing approach specifically targeting replacement of
T12 lamps.
The switch to a T8 baseline in 2012 had a dramatic effect on participation because the rebates
became far less attractive to customers to upgrade from T12s.1 While it may not be feasible for
Avista to alter the baseline for T12 change-outs, Avista should look into developing targeted
marketing strategies for convincing nonresidential customers with T12s to replace them with
more efficient lighting, focusing not only on savings but improved lighting quality and
performance. Avista could begin by targeting businesses that the Small Business Program has
identified as still having T12s.
Recommendation: Work with nonresidential lighting contractors to promote replacement
of T12 lamps.
Contractors make their living by selling equipment. Avista should work with nonresidential
lighting contractors to make sure they are fully aware of the advantages that more efficient
lighting (including the reduced wattage tube lighting that NEEA is targeting through its Reduced
Wattage Lamp Replacement Initiative) offers their customers.
Recommendation: Consider claiming Simple Steps savings for bulbs purchased for the
nonresidential sector.
The evaluation found that about 12% of Simple Steps LED sales and somewhere from 5% to
12% of Simple Steps CFL sales go to nonresidential customers. The mean hours of use for
such lighting is much higher in a nonresidential than residential settings, meaning that the total
Simple Steps savings is potentially higher than currently estimated, and at a minimum, Avista
should consider claiming the additional savings for these purchases.
1.3.3 Residential
Conclusion 4: Participation in the Avista rebate programs has rebounded since 2013
driven by a fivefold increase in shell program participation.
1 A very similar thing happened to another program administrator in Missouri. See Ameren Missouri BizSavers Process Evaluation
Report 2015.
t-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 9
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 17 of 151
EXECUTIVE SUMMARY
Rebate program participation reached a low point in 2013, after which participation increased
year over year by 51% from 2013 to 2014 and by 43% from 2014 to 2015. This is a positive
sign; however, maintaining or increasing program participation requires cost effective savings
opportunities for residential customers. Avista's residential programs operate in a fast-changing
market. Consumers are adopting LEDs rapidly, 2 retailers are transitioning away from CFLs to
LEDs,3 and the federal government and regulators are mandating higher efficiency standards for
bulbs and other energy efficient technologies.4 The convergence of these forces has
implications for the cost effectiveness of Avista's downstream rebate programs. Program
administrators throughout the United States are exploring and testing alternative program
designs such as upstream and midstream designs in response to the evolving market. Although
Avista is currently participating in the Simple Steps, Smart Savings program (a midstream
program), when asked about future opportunities, program staff did not mention any upcoming
pilots or programs that apply these types of designs.
Recommendation: Continue regularly reviewing the expected savings and cost­
effectiveness of the measures in residential portfolio and exploring the benefits and
costs of other program designs including upstream and/or midstream designs.
Consider these suggestions:
1. Continue monitoring the technological advances and availability of ductless heat pumps
and water heating equipment. Surveyed contractors recommended both of these
categories as candidates for inclusion in Avista's programs. NEEA, for example, has
been working to promote the savings potential of heat pump water heaters in the
Northwest via the Northern Climate Heat Pump Water Heater Specification,5 and The
Northwest Power and Conservation Council has identified both of these measure types
as promising technologies in the recently adopted Seventh Power Plan .6
2. Explore upstream program opportunities outside of the lighting market. Upstream
incentive programs offer the potential to increase the adoption of energy efficient
technologies at a lower cost compared to downstream incentive programs. Program
administrators in California and elsewhere have successfully tested or used upstream
2 1 of 20 A-line bulbs sold nationally was an LED in third quarter of 2014, whereas in the quarter prior to that, it was 1 in 30. This
statistic comes from the 2015 LED Market Intelligence report by Bonneville Power Administration.
https://www.bpa.gov/ee/utility/research-archive/documents/momentum-savings-resources/led_market_intelligence_report.pdf
3 Souza, Kim, 2016. Walmart to transition lighting products away from compact fluorescent to LED. Retrieved from
http :/Ital kbusiness.neU2016/02/walmart-to-transition-lighting-products-away-from-compact-fluorescent-to-led/
4 The lighting standard, established by the Energy Independence and Security Act of 2007, requires that light bulbs use about 25%
less energy by 2014. New efficiency heating and cooling standards from the U.S. Department of Energy, which have gone into effect
Jan. 1, 2015, will increase the efficiency of heating, ventilation, and air-conditioning (HVAC) equipment in certain regions.
5 http://neea.org/northernclimatespec/
6 http://www.nwcouncil.org/energy/powerplan/7/plan/
'-"Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 10
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 18 of 151
EXECUTIVE SUMMARY
program designs for technologies that Avista currently incents, including HVAC
equipment and water heaters.7
Conclusion 5: Residential customers who rent their home are underserved.
Nonparticipants say living in a rental property prohibits them from making improvements. This
was the second most commonly cited barrier to making energy efficient upgrades among
nonparticipants (after the up-front cost barrier). More than a quarter (27%) of nonparticipant
survey respondents were renters, whereas only 3% of the participant survey respondents were
renters. Renters account for about one-third of the population in Avista territory.8
Currently, Avista serves renters via the low-income program. The CAP agencies reported
having difficulty serving the low-income renter population because it is difficult to convince
landlords to participate. Additionally, there appears to be no multifamily program in the Avista
portfolio that could serve this market, although Avista does offer an incentive for a natural gas
space and water heating measures to multifamily property owners.
Recommendation: Investigate energy savings opportunities in the rental market.
Consider the following suggestions:
1. Estimate the number and distribution of rental units in the single family, manufactured
home, and among multifamily buildings. Analyzing these data geographically and by
vintage would likely yield insights regarding the energy saving potential in these markets.
2. Conduct needs assessment research with landlords to understand their needs and
concerns and explore ways to bolster their willingness to make energy efficiency
upgrades on their properties. This research should consider the needs landlords serving
low-income renters as well as renters not eligible for the low income program.
3. Conduct needs assessment research with renters to understand their needs and the
barriers to participation they face. For example, although some energy savings activities
may not be appropriate for renters (for example, HVAC system replacement), other
activities such as installing energy efficient lighting and/or advanced power strips could
be appropriate.
7 Quaid, M. and H. Geller (2014). Upstream Incentive Utility Programs: Experience and Lessons Learned. Retrieved April 14, 2016.
http://www.swenergy.org.
8 US Census Bureau. "825003 : Tenure." 2010 -2014 American Community Survey 5-Year Estimates. Web. 13 April 2016.
c...1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 11
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 19 of 151
2 Introduction
2.1 Purpose of Evaluation
The purpose of the process evaluation was to identify any improvements needed at the portfolio
level to increase program effectiveness, efficiency, and identify opportunities for future
programs. The process evaluation collected interview and survey data from program staff,
implementation contractors, program participants, nonparticipants, contractors, and retailers.
Additionally, the evaluation examined program participation data and Opower data.
Table 2-1 summarizes the primary objectives and specific areas for investigation along with the
information sources the evaluation team used to investigate them.
'-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 12
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 20 of 151
2 INTRODUCTION
Table 2-1: Process Evaluation Objectives and Information Sources
P I I . p . . . Nonpartic-rogram St ff mp ementat1on art1c1patmg . t· a p rt· · f 1pa mg Objective Document . . Contractor a icipa mg Contractor Retailers
Review** mterv1ews Interviews Customer Survey CSustomer
Appropriateness of design, participation
procedures, internal communication, rebate
processing activities (e.g., ease of use, cycle
time)
Participant satisfaction with programs
Barriers to participation, effectiveness of
incentives in motivating action
Effectiveness of marketing and promotional
efforts; status of marketing research activities
Opportunities for process improvement and
potential programs; status of Avista response to
previous evaluation recommendations
Obtain data for net-to-gross analysis***
Understand declining participation rates of
programs
Identify commercial uptake of Simple Steps items
Understand the importance of savings associated
with rebated measures and the Home Energy
Reports
../
../ *
*
../ ../
../ ../
../
../
Review and update program logic models l ../ I ../
Survey urvey
../ ../ ../ ../
* ../ ../
* ../ ../ ../
../ ../ ../ ../
../ ../ ../ *
../ ../ ../
../ ../ ../
*Supporting information; •• Descriptions; procedures; design docs; application forms; participant records; marketing materials; etc.; ••• Net-to-gross results appear in impact report
t.-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 13
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 21 of 151
2 INTRODUCTION
2.2 Description of Nonresidential Programs
Avista provided incentives and services for its nonresidential electric and gas customers
throughout its Washington service territory and nonresidential electric customers in its Idaho
service territory in 2014 and 2015.
Avista uses financial incentives and direct installation of efficient measures to encourage its
commercial and industrial customers to install energy efficiency equipment. The evaluation team
examined three core programs that constitute the bulk of Avista's nonresidential energy
efficiency offerings in 2014 and 2015: the Prescriptive, Site Specific, and Energy Smart Grocer
programs. In addition, the evaluation team examined Avista's new Small Business program
which began in June 2015. Table 2-2 provides a summary of those programs and the sections
below provide greater details about each program.
Table 2-2: Key Energy Efficiency Programs
Program Implementer Summary
Prescriptive Avista i Contractors and account managers work with nonresidential customers to
! identify potential projects, submit paperwork, and process incentive I applications.
Site Specific Avista I Contractors, account managers, and program engineers' work with i nonresidential customers to identify potential projects, submit paperwork, and I verify project savings in order to process incentives. !
Energy Smart CLEAResult I Implementer staff conduct outreach to customers with refrigeration equipment
Grocer ! (primarily grocery stores) and conduct an energy audit that identifies energy
J saving projects. If the customer elects to conduct the project(s), implementer ! staff work with the customer and contractors to install equipment.
I
Small I SBW ! Implementer staff provide small business customer's (rate schedule 11) brief
Business I property assessments and energy efficiency measures such as LED lighting
I and faucet aerators.
I
2.2.1 Prescriptive
Avista's prescriptive program provides incentives and services for the following types of electric­
and gas-using equipment.
• Food service equipment • Multifamily development
• Commercial clothes washers • Motors
• Commercial water heaters • Variable Frequency Drives
• Lighting • Compressed air leak detectors
• HVAC • Power management for PC networks
• Building shell (Windows and
Insulation)
t..1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 14
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 22 of 151
2 INTRODUCTION
These incentives and services are available to customers who purchase eligible equipment,
submit a completed application within 90 days after installation, and provide proof of purchase
for all relevant equipment and labor. Customers typically receive their reimbursement about four
to six weeks after Avista receives a complete application. Avista reserves the right to inspect the
installation before processing the rebate.
2.2.2 Site Specific
Avista provides Site Specific services that include helping customers identify energy saving
opportunities and take action to implement those opportunities. Site specific projects may or
may not include prescriptive measures but will always include measures specific to a facility. For
example, a Site Specific project may include custom controls with prescriptive lighting installed
at a given site Eligible measures must have a simple payback less than 15 years and qualify for
$.20 per first year kWh saved for electricity and $3 per first year therm saved. Incentives are
capped at 70% of the incremental project cost.
2.2.3 Energy Smart Grocer
Grocers, convenience stores, restaurants, and any customers with commercial refrigeration are
eligible to participate in the Energy Smart Grocer program. The program, implemented by
CLEAResult, provides no-cost assessments of eligible facilities that result in recommendations
for prescriptive measures the customer could implement to save energy. Measures include case
lighting, controls, refrigerated case gaskets, and motors. Similar to the prescriptive program, the
customer must submit an application after the installation and usually wait four to six weeks
before receiving their incentive. The customer may opt to release the incentive directly to the
installation contractor.
2.2.4 Small Business Program
The Small Business (SB) program is a third-party-administered program that provides
customer's energy efficiency opportunities by conducting the following activities.
1. Conduct a brief onsite audit to identify customer opportunities and interest in existing
Avista programs,
2. Install appropriate energy-saving measures at each target site, and
3. Provide materials and contact information so that customers are able to follow up with
additional energy efficiency measures under existing programs.
Direct-install measures include: faucet aerators, showerheads, pre-rinse spray valves, screw-in
LED's, smart strips, CoolerMisers, and VendingMisers. In 2015 the SB program was only
available to customers who receive electric service under Rate Schedule 11 in Washington and
natural gas service under Rate Schedule 101 in Washington. The program intends to add
Schedule 11 Idaho customers in 2016. They did not target Idaho in 2015 because they were
waiting to see if Idaho would allow gas saving measures. Schedule 11 customers typically use
less than 250,000 kWh per year. The smaller size and the relatively large number of schedule
t..1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 15
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 23 of 151
2 INTRODUCTION
11/101 customers makes them a notoriously difficult to reach and underserved market segment.
SBW Consulting, Inc., based in Bellevue, WA, started program operations in June 2015 and is
under contract to deliver the program through May 2017.
2.3 Description of Residential Programs
Avista provided incentives and services for its residential electric and gas customers throughout
its Washington service territory and for residential electric customers throughout their Idaho
service territory in 2014 and 2015.
Avista uses financial rebates or discounts, reports on energy usage, and direct installation of
efficient measures to encourage its residential customers to install energy efficiency equipment.
The evaluation team examined eight core programs that constitute the bulk of Avista's
residential energy efficiency offerings in 2014 and 2015.Table 2-3 provides a summary of those
programs and the sections below provider greater details about each program.
t..1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 16
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 24 of 151
2 INTRODUCTION
Table 2-3: Residential Program Type and Description
Type Programs Implementer Description
Rebate
Midstream
Behavior
Low-income
Appliance Recycling
I ENERGY STAR®
: Homes
Fuel Efficiency
HVAC Program
i
/ Shell
Water Heater
I Simple Steps, Smart
' Savings
Home Energy Reports
Low-income Programs
2.3.1 Appliance Recycling
I JACO
! Avista
I Avista
! Avista
I Avista
I ! Avista
I CLEAResult
I
1
Opowe,
I
I Community Action
I Partners (CAPs)
I Rebate for recycling fridge or freezer older than I 1995. This program was discontinued in June
12015.
I Rebate for purchase of ENERGY STAR® home
i
I
Rebate for conversion of electric to natural gas
furnace and/or water heater
I Rebate for purchase of energy efficient and high
1 efficiency HVAC equipment, including variable
I speed motors, air source heat pump, natural gas
I furnace and boiler, and smart thermostat
j Rebate for adding insulation to attic, walls, and
! floor, as well as adding energy efficient windows.
I
' Rebate for duct sealing, program measure
discontinued at end of 2014.
Rebate for installation of high efficiency gas or
electric water heater, natural gas water heater, and
1 Smart Savings showerhead
I
Direct manufacture discount for purchase of
approved CFLs, LEDs (bulbs and fixtures), and
! low-flow showerheads. ! I The Opower program generates behavioral savings
1
1
• from a treatment group, which receives Home
Energy Reports, which compares the customer's
I energy usage to similar homes in Avista's service i territory.
I
CAPs within Avista's Washington and Idaho service
territories implement the projects. CAPs determine
energy-efficiency measure installations based on
1
the results of a home energy audit.
The appliance recycling program ceased operation in June 2015 because it was deemed cost
ineffective. Prior to that, the program provided customers a $40 rebate for recycling a
refrigerator manufactured before 1995.
2.3.2 ENERGY STAR® Homes
New home buyers can apply for an $800 rebate for an ENERGY STAR® ECO-rated new
manufactured home or $1 ,000 for an ENERGY STAR® stick-built home. The purchaser must
submit the application and certification paperwork to Avista within 90 days of occupying the
residence.
t..'1 Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 17
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 25 of 151
2 INTRODUCTION
2.3.3 Fuel Efficiency
Customers interested in switching from electrically fueled heating and water heating equipment
to gas fueled equipment are eligible for flat-rate rebates.
2.3.4 Heating, Ventilation, and Air Conditioning (HVAC) Rebates
Avista offers prescriptive rebates for heating equipment such as efficient furnaces or boilers and
variable speed motors, and smart thermostats.
2.3.5 Water Heat Rebates
Avista offers prescriptive rebates for electric and gas efficient water heaters and water saving
fixtures.
2.3.6 Shell Measures
The Shell program provides prescriptive rebates for shell measures like insulation (attic, wall,
and floor), windows, and duct sealing. Contractors generate most of the participants in this
program, except for duct sealing participants. Duct sealing is primarily implemented by UCONs,
a third party contractor. UCONs offers duct sealing to customers free of charge and is
responsible for duct sealing and installation of any other direct install measure that might be part
of the agreement with Avista. UCONs duct sealing program ceased operating in 2015.
2.3.7 Simple Steps, Smart Savings
The Simple Steps, Smart Savings program provides discounts to manufacturers to lower the
price of efficient light bulbs, light fixtures, showerheads, and appliances. This program ,
administered by CLEAResult, operates across the Pacific Northwest and utilities are able to
select which items they want the price lowered. Avista chose general and special CFLs, LED
light fixtures, LED bulbs,9 and showerheads.
2.3.8 Home Energy Reports
Avista and Opower provide free Home Energy Reports (HERs) to a sample of customers that
compares their energy usage to that of similar homes in their area. Using behavioral science,
the program encourages customers to save energy and offers those that receive HERs with
insights into how they can lower energy use.
2.3.9 Low-Income
Local CAP agencies within Avista's Washington and Idaho service territory implement projects
with qualifying low income customers. CAPs assess homes for energy-efficiency measure
applicability, combining funding from Avista and state/federal programs to apply appropriate
measures to a home, based on the results of a home energy audit. CAPs typically approve the
installation of the following measures: shell upgrades (insulation, air-sealing, etc.), duct sealing,
9 Avista offered LED bulbs in 2014 and the last half of 2015.
t..1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 18
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 26 of 151
2 INTRODUCTION
refrigerator replacements, fuel conversions, low-cost measures (window plastic or lighting
measures), and health and safety measures.
t-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 19
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 27 of 151
3 Methods
To conduct a process evaluation of Avista's energy efficiency programs, the evaluation team
reviewed program data and completed 23 interviews and 902 surveys with market actors. Table
3-1 provides an overview of the data collection activities, including the type of data collection
effort and the key topics covered. All interview and survey guides are provided in Appendix C.
Table 3-1: Overview of Data Collection Activities
Data Source T a When Analytic Key Topics
(Sample by sector) ype Techniques
Staff (16; 4 nonres. I interview : Feb. 2015, & Qualitative, • Program goals
and 12 res.) I Oct. 2015 thematic • Program processes
• Communication and coordination
Implementers (7; 1 Interview i Oct. 2015 Qualitative, • Data tracking
nonres. and 6 res.) thematic • Future program opportunities
• Outreach
Contractors (82; 29 ! Survey ! Aug. 2015, Quantitative, • Program awareness
nonres. and 53 res.) ! i Oct. 2015 univariate and • Satisfaction
! • Motivations to participate I bivariate
frequencies • EE Sales practices
• Net-to-Gross
Participants (680; Survey I May 2015-Quantitative, • Program awareness
305 nonres. and 339 I Feb. 2016 univariate and • Satisfaction
res.) bivariate • Program experience
frequencies • Freeridership & spillover
• Leakage of Simple Steps products into
commercial sector
Nonparticipants I Survey I Oct.-Nov. Quantitative, • Program awareness
(140; 70 nonres. and I 2015 univariate and • Experience with EE
70 res.) bivariate • Leakage of Simple Steps products into
frequencies commercial sector
• Spillover
• Program goals
Staff and Qualitative, • Program requirements
implementer Interview i Dec. 2015 thematic • Communication and coordination
manager (2) • Marketing
• Implementation
• Staff background
I interview Qualitative, • Role in outreach
Installers (2) Dec. 2015 • Data collection and reporting
I thematic i • Challenges and barriers to participation
• Implementation successes
Quantitative, • Program experience
Participants (31) Survey J ~~~6-Feb. univariate and • Satisfaction
bivariate • Future EE plans
frequencies • Business characteristics
a The Nexant survey call center fielded the surveys and Research Into Action staff conducted in-depth interviews.
t.-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 20
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 28 of 151
3 METHODS
The sections below provide a brief overview of the sample and methods used to analyze each
data source. The evaluation team first provides an overview where data collection methods
were the same for both the nonresidential and residential sectors ( cross-cutting) followed by
nonresidential, residential and special study specific methods.
3.1 Cross-cutting activities
3.1.1 Staff and Implementer Interview Methods
The evaluation team carried out two sets of staff interviews pertaining to the nonresidential and
residential portfolios. One, conducted in February 2015, took place in a group setting and
included program, engineering, and planning staff. This set of interviews helped the evaluation
team better understand the residential and nonresidential programs and provided an opportunity
for Avista staff to share questions they had for the evaluation. The evaluation team recorded
each group interview, with the interviewees' permission. These interviews typically lasted 90
minutes.
The second set of interviews, conducted in September and October 2015, focused on key
Avista staff responsible for nonresidential programs (prescriptive lighting, prescriptive non­
lighting, and Site specific), residential programs (rebate programs, Opower HERs, Simple
Steps, Smart Savings, and Low-income) marketing, and data management. Additionally, the
evaluation team interviewed key implementers including a staff person representing the Energy
Smart Grocer program, three implementers representing residential programs, and four
Community Action agencies representing implementation staff of Avista's low income programs.
Each interview lasted 45 to 60 minutes. Interviews covered topics such as roles and
responsibilities, program goals, communication among staff and implementers, program
processes, marketing, program changes, and future program opportunities. The evaluation team
integrated results from these interviews into the findings sections of this report.
In addition to the staff and implementer interviews conducted as part of the nonresidential and
residential portfolios, the team interviewed all staff and installers for the Small Business
program. These interviews took place in December 2015 and lasted about 45 to 60 minutes.
Interviews covered topics such as goals, future program plans, program implementation,
marketing, and key successes and challenges. Results of these interviews are discussed in
section 5.3.
3.1.2 Contractor Sample
The evaluation team elected to focus on high-impact contractors -those involved with projects
that delivered the most savings in program year 2014 and 2015. In the nonresidential sector that
meant interviewing lighting and HVAC contractors. In the residential sector that meant
interviewing HVAC and building shell contractors.
Using data assembled by Avista staff, the evaluation team identified 658 unique contractors
operating in Avista territory. The evaluation team categorized these contractors as lighting
(400), HVAC (89), and Shell (55) contractors. The evaluation team could not classify the
t-1 NexanT Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 21
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 29 of 151
3 METHODS
remaining 114 contractors without additional information. Therefore, the evaluation team based
the initial sample on the 544 categorized records.
About three-quarters of the way through completing surveys, the evaluation team determined
additional sample was necessary to complete HVAC and lighting contractor surveys, particularly
in the nonresidential sector. The evaluation team added 75 additional lighting contractors and
14 uncategorized records to the survey sample. Through additional research, we were able to
identify these 14 records as likely HVAC contractors (Table 3-2).
Table 3-2: Contractor Population and Sample
Initial Population Initial Sample Additional Sample Total Sample
HVAC 89 89 14 103
Lighting 400 54 75 129
Shell 55 55 55
Uncategorized 114
Total 658 198 89 287
While some contractors likely worked in both the residential and nonresidential sectors, to lower
the survey burden, the evaluation team surveyed each contractor about work done in only one
of those sectors. The information available in program records did not identify whether a
contractor worked primarily in the residential or nonresidential sector. To identify the primary
sector served, the survey first asked contractors what percentage of their projects are in each
sector. Those who reported completing 50% or more of their projects in the nonresidential
sector answered questions about work done in the nonresidential sector and the rest answered
questions about work done in the residential sector. A large majority (82%) of the respondents
reported doing at least 70% of their work in one sector or the other, indicating a reasonably
clear distinction between nonresidential and residential contractors.
As Table 3-3 shows, the evaluation team exceeded the total goal by six interviews. Because
fewer contractors specialized in nonresidential work than expected, the evaluation team
achieved fewer than the target number of survey completions for that sector.
HVAC
Lighting
Shell
Total
t.-1Nexanr
Table 3-3: Contractor Survey Target and Completions
Target Completions
Residential Nonres. Total Residential Non res. Total
19
19
38
19 38 35 8 43
19 19 21 21
19 18 18
38 76 53 29 82
Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 22
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 30 of 151
3 METHODS
If the distribution of mainly nonresidential and mainly residential contractors is the same in the
population as in the survey, then there are about 202 mainly nonresidential contractors and 376
mainly residential contractors in Avista's territory.10 The 29 nonresidential completions provides
90/14 confidence and precision and the 53 residential completions provides 90/10 confidence
and precision in the findings.
The evaluation team interviewed all contractors about the following topics:
• Awareness of Avista energy efficiency programs
• Motivations to participate in programs
• Satisfaction with programs
• Sales practices related to energy efficient equipment
The evaluation team carried out the contractor telephone survey in August and October 2015.
The evaluation team analyzed the close-ended data using SPSS and used MS Excel to code all
open-ended responses.
3.2 Nonresidential Activities
Nonresidential data collection activities included surveys with participants and nonparticipants.
The evaluation team describes each activity below.
3.2.1 Participant Survey Sample and Methods
The participant surveys covered the following process evaluation related topics :
• Awareness of Avista programs and incentives
• Awareness of energy efficient equipment
• Satisfaction with staff interactions, equipment, clarity of information, time needed to
participate, and, if relevant, their audit experience.
• Energy efficient policies and practices
The evaluation team administered the survey in phases to provide Avista staff with up-to-date
market feedback throughout the evaluation period. The first participant survey occurred in July
2015, capturing data from 2014 and Q1 and Q2 2015 participants. The next survey, conducted
in October 2015, captured data from Q3 2015 participants and the last participant survey
occurred in January 2016, capturing data from Q4 2015 participants. The evaluation team
analyzed all survey data using SPSS and used MS Excel to code all open-end responses. The
evaluation team examined responses for differences by state (Washington or Idaho) and year of
10 The evaluation team assumed the proportion of the sample that is commercially focused, 35%, represents the population than
there are 202 commercially focused contractors (.35*578 =202) and 376 (.65*578) residentially focused contractors.
'-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 23
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 31 of 151
3 METHODS
participation (2014 or 2015). The final tally of survey completions provides for 95/5 confidence
and precision at the portfolio level.
The evaluation team developed a stratified random sample of participating Avista customers by
program and state that included both electric and gas customers. The evaluation team
estimated the target completions using assumptions about participation as of January 2015.
Actual participation varied from the estimates, resulting in fewer survey completions needed in
some program types and more for other program types. Table 3-4 summarizes the targeted and
actual number of completions by year, and Table 3-5 shows the distribution of the sample
population and survey completes by program.
Table 3-4: Nonresidential Participant Survey Completions by Program Type and Fuel
Target Survey Completions Actual Survey Completions
P t 2014 2015 Total 2014-2014 2015 Total 2014-
rogram ype 2015 2015
Prescriptive Lighting
Prescriptive Energy
Smart Grocer
Prescriptive Non-
Lighting Other
Cascade Energy Pilot
Site Specific
Prescriptive (Appliance)
Prescriptive (Shell)
HVAC
Food Service
Site Specific
TOTAL
t.-1Nexanr
Washington/Idaho Electric
32 36 68 40 42 82
20 24 44 22 13 35
12 12 24 14 14 28
4 4
40 44 84 46 39 85
Washington Gas
5 6 11
12 12 24 15 7 22
12 12 24 9 12 21
5 6 11 2 8 10
20 23 43 5 16 21
158 180 338 154 151 305
Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 24
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 32 of 151
3 METHODS
Table 3-5: Population and Completed Sample Distribution by Program
2014 2015
Program name Sample Survey Sample Survey
Population* Completions Population* Complete
Food Service 53 12 25 13
HVAC 44 9 83 19
Prescriptive Lighting 180 40 235 42
Water Heat 3 2
Windows and Insulation 42 16 10 9
Energy Smart Grocer 57 22 20 13
Green Motors 10 2
Site Specific 101 51 108 55
Standby Generator Block Heater 6
TOTAL 496 154 482 151
* Indicates number of participants in which we were able to draw a sample.
3.2.2 Nonparticipant Survey Sample and Methods
The nonparticipant survey covered the following topics related to the process evaluation:
• Awareness of Avista programs
• Recent history of using energy efficient equipment
• Planned upgrades that will use energy efficient equipment
• Energy efficient policies and practices
• Interest in energy efficiency programs
According to data received from Avista, there were 43,848 unique nonparticipant commercial
accounts throughout Avista's Washington and Idaho territory in 2015. The evaluation team
identified 23,180 unique telephone numbers within the population of accounts, and used that
number as a proxy for the size of the population of nonparticipant contacts. To ensure that the
survey correctly represented the high-and low-density areas of Washington and Idaho, the
evaluation team stratified the random sample on state as well as on population density.11 The
distribution of completed interviews across the four strata closely matched the distribution of the
population across the strata (Table 3-6), and the 70 completes provide for 90/1 O confidence and
precision.
11 The mean population density is 588 people per zip code. The low-density strata included zip codes with population densities
below the mean (588) for all zip codes in Avista territory, and high-density strata included zip codes with population densities greater
than or equal to the mean for all zip codes in Avista territory.
'-"Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 25
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 33 of 151
3 METHODS
Table 3-6: Nonparticipant Nonresidential Population and Survey Completes
Nonparticipant Population of 5 C I t . urvey omp e es Unique Contacts
Count Percent Count Percent
Low Population Density -ID 8,741 38% 25 36%
Low Population Density -WA 6,231 27% 18 26%
High Population Density -ID 772 3% 3 4%
High Population Density -WA 7,436 32% 24 34%
TOTAL 23,180 100% 70 100%
The evaluation team administered the survey in October and November 2015 and analyzed the
data using SPSS for close-ended data and MS Excel to code all open-ended responses. The
evaluation team examined responses for differences by state (Washington or Idaho) and year of
participation (2014 or 2015).
3.2.3 Small Business Process Evaluation Methods
The primary goal of the Small Business (SB) process evaluation was to assess and provide
information on program delivery and implementation and market response to the program. The
evaluation focused on program design and theory, implementation and delivery, and market
feedback.
The evaluation team evaluated the programs through interviews with pertinent program actors
including Avista and third-party implementation staff, auditors/installers, and participants (Table
3-7). Avista engaged the evaluation team to evaluate the SB program after the evaluation of the
rest of the program portfolio had begun, and under a separate contract. Therefore, the
evaluation team conducted specific staff and implementer interviews for the SB program ,
separately from other staff and implementer interviews. The SB-specific interviews are
described in this section rather than in Section 3.1.1, above, as they are not cross-cutting.
t-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 26
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 34 of 151
3 METHODS
Table 3-7: Overview of Small Business Data Collection Activities
Analytic . Source (Sample) Type When T h . Key Topics ec niques
Staff and
implementer
manager (2)
Installers (2)
Participants (34)
I , oteo,lew I Dec. 2015
Interview Dec. 2015
I Survey i Jan. -Feb.
! 2016
Qualitative,
thematic
I Qualitative,
thematic
! Quantitative, I univariate and I bivariate
I frequencies
• Program goals
• Program requirements
• Communication and coordination
• Marketing
• Implementation
• Staff background
• Role in outreach
• Data collection and reporting
• Challenges and barriers to participation
• Implementation successes
• Program experience
• Satisfaction
• Future EE plans
• Business characteristics
Of the 1, 181 SB participants in the program database, 35 had received audits but did not have
any measures installed, leaving 1,146 with measures. Of those, 344 had phone numbers. The
distribution of those with phone numbers did not differ noticeably from the population in terms of
measures received or location; therefore, the evaluation team concluded that sampling from
those with phone numbers would not bias the sample in terms of those key variables. Assuming
a response rate of about 15%, the evaluation team selected a random sample of 200 from the
list of 344 participants with phone numbers.
The evaluation team randomized the sample and called businesses in the random order. To
ensure that the completed survey covered all the areas in which the program was active, the
evaluation team set quotas by location (North Washington, South Washington , and Spokane) to
ensure that distribution of survey completions across the three areas would be similar to the
distribution of the participant population across those areas.
The evaluation team exceeded its assumed response rate, achieving a 32% response rate, and
was able to complete the survey after calling the first 105 businesses in the sample. Table 3-8
shows the disposition of the entire sample.
t-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 27
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 35 of 151
3 METHODS
Table 3-8: Disposition Summary
C t Percent of Sample oun Attempted
Complete 34 32%
Refusal 6 6%
Not reached 63 60%
Leftjob 1%
Bad number 1%
Sampled businesses called 105 100%
Sample businesses not called 95
TOTAL 200
The completed sample closely matched the participant population on the three locations in
which the program was active (Table 3-9). As the table shows, the sample also included a
greater percentage of lighting, water-saving, and non-lighting power-saving measures than the
participant population.12
Table 3-9: Distribution of Population, Sample, and Completed Sample
Population Sample Complete
(n = 1,013) (n = 200) (n = 34)
Count Percent Count Percent Count Percent
Location
North Washington 156 15% 17 9% 7 20%
South Washington 160 16% 28 14% 6 17%
Spokane 697 69% 155 78% 21 62%
Measure Type
Any lighting 303 30% 76 38% 15 44%
Any water saving 949 94% 193 97% 34 100%
Any non-lighting, power-saving 320 32% 71 36% 18 53%
The completed sample achieved at least 14% precision at 90% confidence.
3.3 Residential Activities
Residential data collection activities included surveys with participants and nonparticipants. The
participant and nonparticipant surveys covered the following process evaluation related topics:
• Awareness of Avista programs and rebates
12 This is because it had a higher percentage of participants with multiple measures than did the population.
t-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 28
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 36 of 151
3 METHODS
• Motivations and barriers to participation
• Program experience, if participants
• Attitudes toward Energy Use and Conservation
• Purchases of energy efficient products
The evaluation team received 2014 and 2015 residential customer account data from Avista that
identified rebate and appliance recycling participants and all other residential customers
(nonparticipants). The data contained: 1) measures installed/recycled and the rebate received
for program participants; 2) geographic location (ID or WA); 3) utility services (gas, electric, or
both); and 4) contact information.13 The 2014 and 2015 data included approximately 480,000
residential customers, containing a total of 7,505 participants in 2014 and 11 ,620 participants in
2015.
To facilitate the evaluation team's evaluation of the residential lighting program, Simple Steps,
Smart Savings, and the residential behavior program administered by Opower, the evaluation
team included survey questions asking respondents whether they purchased discounted
products from participating retailers or received Home Energy Reports or HERs to identify
possible participants in these two programs.
The evaluation team developed a stratified random sample of rebate/appliance recycling
participants and nonparticipants. The evaluation team stratified the participant sample by year of
participation (2014 or 2015) and state (WA or ID). Nonparticipant sample was stratified by state
(WA or ID) and urban area (whether living in urban or rural zip codes). Both samples included
electric and gas Avista customers. Table 3-10 summarizes the number of participant and
nonparticipant completes by state and year.
Table 3-10: Sample Distribution for Residential Program Participants and Nonparticipants
2014 Participants 2015 Participants Nonparticipants s~~ . .
Population Sample Population Sample * Total Sample
ID 1,143 29 1,823 59 160,455 23
WA 6,362 124 9,797 127 319,370 47
TOTAL 7,505 153 11,620 186 479,825 70
* 67 interviewed in Quarter 1 (Ql) of 2015, 53 interviewed in Q2, 46 interviewed in Q3, and 20 interviewed in Q4 of 2015.
The evaluation team also monitored the status of the participant survey to ensure the relevant
programs and measures were represented in the survey responses. The evaluation team
exceeded the target samples for all programs except the WA gas water heat and ENERGY
STAR Homes programs (Table 3-11).
13 The evaluation team received contact information for the sample only.
t-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 29
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 37 of 151
3 METHODS
Table 3-11: Residential Participant Surveys
Target Completes Actual Completes
Residential Program
2014 2015 Total 2014 2015 Total
Washington/Idaho Electric
Appliance Recycling 34 36 70 35 37 72
HVAC 32 36 68 32 36 68
Water Heat 5 8 13 5 8 13
ENERGY STAR Homes 7 8 15 11 5 16
Fuel Efficiency 5 20 25 5 20 25
Shell 12 12 24 13 15 28
Washington Gas
Water Heat 5 8 13 5 6 11
ENERGY STAR Homes 5 8 13 10 11
HVAC 22 24 46 24 24 48
SHELL 22 24 46 22 25 47
TOTAL 149 184 333 153 186 339
3.4 Special Studies Activities
The evaluation team conducted several special studies as part of the evaluation. This section
provides a brief description of the methods used for each activity. Details about the methods
used for the declining participation rates and participation rates among Opower participants are
provided in sections 7.1 and 7.2.
3.4.1 Declining Participation Rates
The 2012-2013 process evaluation report14 noted that program participation rates based on the
number of rebated measures have declined since 2010. The 2012-2013 process evaluation
report also suggested that one explanation for the decline in participation was fewer measures
offered through the programs and the reduced incentive amounts that Avista offered in
response to declining avoided costs. The evaluation team examined the list of rebated
measures in both nonresidential and residential 2010-2015 program databases to assess the
potential impact of the fewer rebated measures and the reduced incentive amounts on
participation.
14 Avista 2012-2013 Process Evaluation Report, May 15, 2014, Cadmus.
t--1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 30
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 38 of 151
3 METHODS
3.4.2 Participation Rates Among Opower Behavioral Program Participants and
Nonparticipants
Understanding the importance of savings associated with rebated measures and the Opower
Behavioral Program (Home Energy Reports (HER) program) will enable Avista to better
understand the extent of induced behavioral savings not attributed to rebated measures and the
rebated measure portion of the savings. The evaluation team used residential customer data
and program participant's data to conduct this analysis.
3.4.3 Commercial Uptake of Simple Steps Measures Methods
The evaluation team used two methods to estimate the proportion of the CFL and LED
markdown measures (Simple Steps measures) going to the residential and nonresidential
sectors, respectively. Both methods relied on data collected from the process evaluation. The
first approach relied on data from the nonresidential participant and nonparticipant surveys. The
second approach relied on a survey of store and department managers at the dominant retailers
of Simple Steps items. These following subsections describe these approaches.
3.4.3.1 Nonresidential Customer Surveys
The nonresidential participant (n=305) and nonparticipant surveys (n=70) asked respondents to
estimate the number of light bulbs they purchased for their businesses and if they recalled
seeing Simple Steps marketing materials near or on their(See section 3.2 for discussions of the
sample frame preparation for participants and nonparticipants). The evaluation team analyzed
responses using SPSS and Microsoft Excel®. The evaluation team summed the number of CFL
and LED items attributable to Simple Steps separately for participants and nonparticipants.
3.4.3.2 Retail Store Manager Survey
The survey of retail store managers asked respondents to estimate the proportion of sales of
Simple Steps measures that went to residential and nonresidential customers. In a previous,
similar project, members of the evaluation team determined that the only types of respondents
who were able to answer such questions were those from large chain stores like The Home
Depot, Costco, and Walmart, which have staff devoted to selling lighting products and/or sell
large quantities of incented items.
A review of the Simple Steps sales data in Avista territory showed that those same three
retailers accounted for about 90% of sales (Table 3-12); the sample frame thus included the 28
participating stores from those three retailers. It also included the four participating Lowes
stores; this chain is similar to the three dominant retailers and sold, on average, many times
more units per store than retailers other than Walmart, Costco, and Home Depot. In sum, the
sample frame consisted of 32 stores from one of these four retailers.
'-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 31
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 39 of 151
3 METHODS
Table 3-12: Retailer Sales Data in Simple Steps
Retailer Number of Stores Total Units % Of All Units Mean Units Included in
Sold Sold Sold per Store Sample Frame
Walmart 16 421,376 35% 26,336 Yes
Costco 5 394,185 33% 78,837 Yes
Home Depot 7 266,434 22% 38,062 Yes
Lowes 4 24,046 2% 6,012 Yes
All other stores 102 96,435 8% 945 No
TOTAL 134 1,202,476 100% 8,974 N/A
The evaluation team surveyed representatives from 27 of the 32 stores and reached all four
retailers in January 2016. Surveys took approximately five to 1 O minutes to complete. The
evaluation team analyzed responses using SPSS and Microsoft Excel®.
3.5 Review of Program Logic Models
The evaluation team updated the existing logic models for the residential and nonresidential
programs after speaking with program staff and implementers. Each updated logic model is
located in the Appendix B.
t-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 32
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 40 of 151
4 Nonresidential Process Results
The sections below provide the results of the nonresidential process evaluation of Avista's
nonresidential programs. This section begins with an overview of the administration activities of
the programs and a summary of challenges staff reported facing with administration of
programs. Subsequent sections discuss program awareness, the company culture of market
actors, the experience of market actors within the program, and concludes with possible
opportunities to increase program participation.
4.1 Program Administration
The evaluation team interviewed the leaders of each nonresidential program covered in this
evaluation. The following section describes the key points noted by staff regarding the
administration of the program and possible program changes.
Nonresidential program staff and implementers did not report any systemic problems or issues
of concern in program implementation. During the mid-year interviews, they all stated that data
tracking and reporting was adequate for their needs and all reported smooth internal
communications with one another.
Staff noted that Avista changed customer databases between 2014 and 2015 which did cause
some anticipated difficulties querying customer records over time. However, this change in
databases appeared to be a temporary problem typical of transitioning from one system to
another. The change did not negatively affect program staffs ability to carry out their roles.
However, the customer database does not provide the capabilities that a customer relationship
management tool (CRM) could provide. Marketing staff would like the ability to target customers
with messaging about efficiency opportunities and the new database does not offer this
capability. According to staff, the ability to develop targets will happen at some unspecified point
in the future.
Staff noted the following challenges facing Avista's nonresidential programs and expressed how
they plan on meeting those challenges.
• Lighting: The change to a T8 baseline instead of T12 lowered participation because the
savings are not as large for a T8 to LED replacement as they were for a T12 to LED
replacement. Adding LED replacements for HID fixtures to the list of prescriptive lighting
measures is one way the program plans to make the program attractive to potential
participants.
Additionally, the program is considering simplifying its online lighting calculator to
improve customer satisfaction with that tool. The revised tool will help customers by
providing estimated payback and help them determine whether their project will follow
the prescriptive or site specific path. According to staff, this tool could help overcome
customer frustration that occurs occasionally when a customer incorrectly submits a
L-1 Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 33
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 41 of 151
4 NONRESIDENTIAL PROCESS RES UL TS
prescriptive application instead of site specific. Staff also noted the tool could provide
immediate quality control, making that process less time-intensive for them.
• Energy Smart Grocer: The market appears saturated as the program has delivered less
savings each year over the last few years. Staff noted two possible ways to address this
problem. 1) Develop deemed savings measures that would make it easier for customers
to participate. 2) Encourage more participation among restaurants instead of
concentrating on groceries and convenience stores, the programs traditional key
participants.
• Site specific: Account executives currently play an important role in marketing the
program to customers and contractors. Encouraging additional participation may require
new avenues for marketing and outreach and further supporting account executives in
their outreach role.
4.2 Program Awareness and Involvement
To identify how customers become aware of Avista's programs, the evaluation team asked
participants, nonparticipants, and contractors how they learned about programs and about their
reasons for participating and not participating. The sections below summarize each group's
program awareness and provides some insights into motivations and concerns about program
participation.
4.2.1 Contractor Involvement
Most of the 29 nonresidential contractors have been familiar with Avista programs for many
years. Twenty-two of the 29 contractors surveyed reported having more than five years of
experience implementing Avista-incented jobs. Of the remaining seven, four reported at least
four to five years of Avista experience and three reported two to three years of experience.
The nonresidential contractors represented varying levels of activity. As expected, the lighting
contractors tended to report doing more projects per year than the HVAC contractors (Figure
4-1).
t-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 34
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 42 of 151
4
14
ti 12 C:
Cl)
~ 10
0 ~ 8 Cl) a: o 6 ....
Cl)
.0 4 E
:J Z 2
0
NONRESIDENTIAL PROCESS RESULTS
Figure 4-1: Nonresidential Contractor Activity Level
HVAC lighting Total
• Up to 50 • 51 to 200 • More than 200
Twenty-four of the 29 nonresidential contractors surveyed were able to estimate the proportion
of their commercial jobs that receive an Avista rebate. The evaluation team found that most of
the surveyed contractors' work does not receive Avista rebates, with a mean of only 24% of jobs
receiving a rebate. Most (19) respondents reported a quarter or fewer of their jobs receiving an
Avista rebate. Of the remaining five respondents, one each reported that 50% and 75% of their
work receives rebates and three (two lighting contractors and one HVAC) reported that all of
their work receives Avista rebates.
The above findings indicate there is variability in the degree to which contractors are effectively
using Avista's program, with some using them very effectively but more of them making little
effective use of the programs. Section (4.4.4), below, further explores contractors' role in driving
incented upgrades.
4.2.2 Nonresidential Customer Awareness
Nonresidential customers, 305 participants and 70 nonparticipants, were asked how they
became aware of Avista's programs. Customers were allowed multiple responses.
Compared to nonparticipants that were aware of the program (n = 43), participants were more
likely to have heard about the program through a contractor via the program website, and
through past program experience. Compared to participants, nonparticipants were more likely to
have heard about the program via printed material and other sources of awareness (Figure 4-2).
c...1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 35
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 43 of 151
4 NONRESIDENTIAL PROCESS RES UL TS
Figure 4-2: Source of Program Awareness (Multiple Responses Allowed)
Contractor or vendor*
Past experience with program*
Word of mouth
Avista representative
Avista program website*
Newsletter or other print material*
Trade organization
Program sponsored event
Don't know*
Other
• 4%
0%
0% 25%
32%
33%
40%
• Participants (n=305) • Nonparticipants (n=43)
60%
50% 75%
*Significant (p< .05)
It is difficult to gauge the relative impact of each source of program awareness just by
comparing the percentages of participants and nonparticipants that reported a source. For
example, a fairly substantial percentage of participants reported word of mouth, but so did
nonparticipants, so what does the comparison tell us?
The evaluation team developed a coefficient that better illustrates how strong the association
was between each source of awareness and program participation. For each awareness
source, the coefficient was the ratio between two percentages: 1) the percentage of participants
among those who cited a source of program awareness; and 2) the overall percentage of
participants in the population. For any given coefficient, the greater the value, the more strongly
that source of awareness predicts program participation.
Figure 4-3 shows the coefficient for each source of awareness for program participants. This
shows that awareness through past experience with the program was the greatest predictor of
t-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 36
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 44 of 151
4 NONRESIDENTIAL PROCESS RES UL TS
program participation.15 More noteworthy perhaps is that awareness through a contractor or
vendor was positively associated with program participation, as were awareness through the
program website and through an Avista representative. Compared to the overall population,
those who learned about the program through past experience are four times more likely to be a
participant.
Figure 4-3: Relative Association of Participant Awareness with Participant Population
Past experience 3.85
(l} Program website u 2.93
I... ::i 0 V) Contractor /vendor V'l 2.79
V'l (l}
C
(l} Avista rep. 1.77 I... ro ~ <(
E Word of mouth -0.94 ro bl) e Trade org. -0.87 Cl..
Newsletter/print 1111 0.39
0.00 1.00 2.00 3.00 4.00 5.00
Coefficient of Assocation with Program Participation:
Participant % of Awareness Source/ Participant % of Population
More than three-fifths of nonparticipants (57% in Idaho, 64% in Washington) reported being
familiar with Avista rebates (Table 4-1). Nonparticipants primarily reported familiarity with
prescriptive lighting rebates, followed by shell improvement and appliance rebates. They were
far less aware of rebates for HVAC, and water heating.
15 The evaluation team defined program nonparticipants as those who did not participate in 2014 or 2015, but some nonparticipants
so defined could have participated in 2013 or earlier. This likely explains why some nonparticipants identified past program
experience as their source of program awareness.
t.-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 37
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 45 of 151
4 NONRESIDENTIAL PROCESS RESULTS
Table 4-1: Nonparticipant Awareness of Avista Rebates (n= 70, Multiple Responses
Allowed)
Rebates Familiar With Count Percent
Aware of any rebates 43 61%
Prescriptive Lighting 24 34%
Prescriptive Shell 8 11%
Appliances 7 10%
HVAC Program 4 6%
Motor Controls HVAC 1%
Hot water heater 2 3%
Other 3 4%
Don't know 7 10%
Participants and nonparticipants each expressed interest in future program participation. A
slightly higher percentage of participants than nonparticipants expressed interest in learning
more about efficiency programs and opportunities, but the difference was not statistically
significant. Participants were more likely to express interest in attending a workshop or event
about efficiency than were nonparticipants; this difference was statistically significant by chi­
square (p < .05; Table 4-2).
Table 4-2: Interest in Future Participation (Multiple Responses Allowed)
Nonparticipants (n = 70) Participants (n = 305)
Count % Count %
Interest in any future participation 47 67% 221 72%
Energy efficiency programs 45 64% 217 71%
Energy savings opportunities 45 64% 215 71%
Workshops or events about energy efficiency* 28 40% 170 56%
* Significant (p< .05)
Both nonparticipants and participants expressed interest in receiving Avista program
information. While participants indicated they would prefer to receive program information via
email over any other method, nonparticipants were almost as likely to want information via US
mail (not as part of their bill) and they were more likely than participants to request information
via mail (Table 4-3). As an overall percentage, participants and nonparticipants did not differ
much in their preference for person-to-person contact. However, participants were more specific
than nonparticipants when requesting direct person-to-person contact, reporting five different
methods compared to just one for nonparticipants. The 32 participants who indicated a
preference for person-to-person contact suggested such contact might occur at a webinar,
community event, or training or by telephone -none cited more commonly than others.
t.-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 38
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 46 of 151
4 NONRESIDENTIAL PROCESS RESULTS
Table 4-3: Nonresidential Customer Preferred Method of Receiving Information from
Avista (Multiple Responses Allowed)
Preferred Method of Contact
Nonparticipants (n = 47) Participants (n = 305)
Count % Count %
Email 27 57% 196 64%
By US mail separate from bill insert* 26 55% 72 24%
By US mail via bill insert 16 34% 71 23%
Avista website 5 11% 55 18%
Person-to-person contact 3 6% 32 10%
Through trade associations 5 2%
Don't know 7 2%
Other 2 4% 4 1%
Refused to provide contact method 21 7%
* Significant (p< .05)
4.3 Influences on Customers Decision Making
The evaluation investigated several topics relating to customer decision making, their proactivity
toward energy efficiency and their motives for investing in efficient equipment.
4.3.1 Energy Practices and Policies
More than half of participants (57%) reported that their company had one or more energy­
related policies compared to 40% of nonparticipants; this difference was statistically significant
(Chi-square, p <.05). The most commonly reported specific practice was having an employee or
employees responsible for monitoring or managing energy use, with 44% of participants
reporting this practice compared to 17% of nonparticipants. A significant difference between
groups also exists for purchasing energy efficient equipment and having energy and carbon
related goals (Table 4-4).
t..1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 39
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 47 of 151
4 NONRESIDENTIAL PROCESS RESULTS
Table 4-4: Energy Savings Policies and Practices
Nonparticipants (n = 70) Participants (n = 305)
Count Percent Count Percent
Any policy or practice 28 40% 175 57%
Person(s) responsible for energy use 12 17% 133 44%
Policy requiring energy efficient purchasing 12 17% 92 30%
Defined energy savings goals 4 6% 63 21%
Carbon reduction goals 2 3% 46 15%
Othera 7 10% 4 1%
Don't know/Refused 0 0% 2 1%
• Among nonparticipants that reported other policies, four reported offering general encouragement to staff on reducing
energy, two reported having recycling programs, and one reported replacing current lighting with LEDs
The evaluation team also surveyed nonparticipants about the length of time their energy saving
policies and practices were in place. Most nonparticipants who had policies or practices related
to energy management reported that they had been in place for five years or more, with the
exception of policies related to the purchase of energy efficient equipment (Figure 4-4). Of the
12 nonparticipants who reported awareness of Avista's energy efficiency programs, few
indicated that their awareness influenced their companies' decision to implement energy
management policies or practice (two or fewer providing a rating of 4 or 5 on a 5-point scale
from "not at all influential" to "very influential").
Figure 4-4: Length of Time Energy Related Goals and Policies Have Been In Place at
Nonparticipants' Organizations
Purchase energy efficient equipment (n=12)
Person(s) responsible for energy usage (n=12)
Defined energy savings goals (n=4)
Carbon reduction goals (n=2)
Other policies and practices (n=7)
0% 25% 50% 75% 100%
Don't know • Less than five years • Five years or more
t.-'1 Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 40
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 48 of 151
4 NONRESIDENTIAL PROCESS RESULTS
Another indication of a company culture interested in energy efficiency is having staff with
Builder Operator Certification (BOC). One nonparticipant (1 % of sample) and 12 participants
(4%) reported possessing BOC certification.
4.3.2 Customer Motives
The evaluation team investigated customer motives from the perspectives of both program
participants and contractors.
Participants provided many reasons for applying for the program rebate. Topping the list of
reasons were to save money and to save energy (Table 4-5). Washington participants were
significantly more likely than Idaho participants to say that increasing the comfort of their facility
was the reason for participating in a program (67% of WA participants compared to 54% of
Idaho participants; p < .05).16
Table 4-5: Reasons for Applying to Program (Multiple Responses Allowed) (n = 305)
Count Percent
To save money 297 97%
To save energy 290 95%
Seemed easy to use program 217 71%
General trust of Avista programs 199 65%
Increase comfort of facility 193 63%
Good experience with another Avista efficiency program 190 62%
Contractor recommended 180 59%
Obtain high quality equipment 18 6%
Contractors also indicated that customers carry out incented jobs largely to save on their utility
bills and to increase comfort levels (Figure 4-5). They also indicated that improving building
operations and maintenance is an important motive. Neither contractors nor participants
reported that being "green" was an important motive.
16 We found no statistically significant comparisons for the other seven reasons for applying to the program. To control for Type I
error across the eight comparisons, we examined the probability of finding a chi-square result with at least the observed level of
statistical significance in the eight comparisons. A goodness-of-fit chi-square was not statistically significant, indicating that the one
"significant" effect could have occurred by chance. Nevertheless, we have opted to present this finding as it is possibly meaningful,
reflecting the fact that Washington participants, but not Idaho participants, may have had gas-related projects which are more
commonly HVAC and comfort-related.
t-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 41
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 49 of 151
4 NONRESIDENTIAL PROCESS RES UL TS
Figure 4-5: Contractor Perspective: Importance of Reasons Nonresidential Customers
Implement Avista Energy Efficiency Projects (n = 29)
To save money on utility bills 7% 14% 76% : . I
"' . "',_
To improve the operations and maintenance (O&M) 7% 17% 69% : ~
To improve the comfort of their building 17% 17% . , · 59%: . ' · .• : ~
> I \~ "" ..,~
To improve the looks of the building
To lower their reliance on fossil fuels ("to be green")
0% 25% 50% 75% 100%
• Not important • Neutral • Important • Don't know
4.3.3 Contractors' Sales Practices
All but one surveyed contractor reported they did not recall ever discouraging a customer from
ordering a high efficiency equipment option . (The one contractor who did recall doing so said
that it was because the incentive was not sufficient to produce a good ROI on the higher-cost
equipment.) Nevertheless, contractors varied greatly in how much of the equipment they sold is
high efficient, from 5% to 95% of their sales. Figure 4-6 shows that most contractors fall into two
groups: 1) those whose high efficient equipment sales represent more than 60% of their sales;
and 2) those whose high efficient equipment sales represent 40% or less of total sales, most of
whom reported that high efficient equipment makes up 20% or less of their sales.
12 J!l g 10
"C g 8
Q.
~ 6
0 4 ...
-8 2 E ~ 0
Figure 4-6: Percentage of Equipment Sold (n = 28)17
Oto 20% 21 to 40% 41 to 60% 61 to 80% 81 to 100%
Percent of equipment was EE equipment
17 One respondent did not know the percent of their equipment sold that was high efficient.
t-'1 Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 42
Exhibit No. 2
L Roy, Avista
Schedule 2, Page 50 of 151
4 NONRESIDENTIAL PROCESS RESULTS
When asked how many equipment options they offer customers when bidding work, 26
respondents were able to report a specific number of options. Most respondents (22) reported
offering two or three options, with the other four reporting they offer only one option.
Respondents most frequently cited price (10 respondents) and energy efficiency (9
respondents) as the factors that differentiated the options they offer. Less frequently identified
differentiators were differences in product quality or technical characteristics (4) and non-energy
benefits (3). (Four respondents cited multiple differentiators.)
4.4 Program Experience
The section below describes the experience participants and contractors had using Avista
programs. This includes participants' and contractors' satisfaction with the programs, their
motivations to participate, and possible barriers to participation. This section also describes
nonparticipants' reasons for not participating in the program.
4.4.1 Participant Program Satisfaction
Participants from all programs were generally satisfied with their participation, with no more than
5% of respondents reporting negative satisfaction with any element (Figure 4-7). This did not
differ by year of participation or location (WA vs ID). For all but two elements, responses
indicated that more than 80% of respondents thought the program provided an easy-to-use
process and adequate equipment. The two exceptions were as follows:
• Of the 270 respondents that received rebates for equipment upgrades, 64% agreed that
the project energy savings met or exceeded their expectations. However, many of these
participants (27%) did not know whether the energy savings met or exceeded
expectation, suggesting that it may have been too early for the respondent to know
whether the project was delivering savings. Excluding those that did not know about the
energy savings, 88% agreed the savings met or exceeded expectations.
• Of the 143 respondents that received lighting rebates, 78% reported that the range of
eligible lighting equipment met their needs, while 16% reported some dissatisfaction with
the range of lighting equipment. Shedding some light on this finding, program staff had
noted challenges in keeping up a list of eligible equipment in the rapidly changing
lighting market, particularly with growing interest in LEDs.
t.-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 43
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 51 of 151
4
"' 0 M II ..s
~ C "' C. ·u
NONRESIDENTIAL PROCESS RES UL TS
Figure 4-7: Satisfaction with Program Elements
Our efficient equipment has performed very well 95% i
It was easy to apply for a rebate . 94%
The time it took to receive the rebate was reasonable 5% 88% t
.E ::1 The variety of Avista's rebated equipment meet my energy 8% 86% 5 upgrade needs
The range of incentive-eligible equipment met my energy
upgrade needs
The time between scheduling my audit and when it
occurred was reasonable
We got our audit report in a reasonable amount of time
My efficient lighting has performed very well
The range of incentive-eligible bulbs and light fixtures met
my lighting needs
The amount of time to receive an evaluation report was
reasonable
The energy savings from our project met or exceeded our
expectations
9% 83% 6o/c
% 89% 6%
% ~% i~
'. ·. ... 93% !
11% 78% 6o/c
7% 81% {9%
6% 64% -27%
0% 25% 50% 75% 100%
• Not agree (1 & 2) • Neutral (3) • Agree (4 & 5) • Don't know
To better understand what equipment changes might be useful for the program to consider,
interviewers asked those that did not agree or were neutral about the range of rebate eligible
equipment about possible changes to improve the range of equipment. Responses generally
were not specific. One respondent requested more LED lighting options and three said that the
lists were heavily weighted towards lighting measures but lacked other equipment types. Of the
other 28 respondents, 14 indicated a general desire for more variety of equipment, one said
there was an insufficient range of eligible "electric" equipment, and one said that program
equipment often did not align with the list of equipment approved by their national franchise.
As noted above in Section 4.1, program staff reported possible plans to simplify the online
lighting calculator to provide estimated payback and help identify the appropriate project path,
possibly increasing customer satisfaction. This may be very valuable to customers, but
considering that 94% of customers consider the application process easy, such a revision may
not be completely necessary.
t-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 44
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 52 of 151
4 NONRESIDENTIAL PROCESS RESULTS
4.4.1.1 Satisfaction with Program Representatives
Participants who engaged staff or program representatives reported high levels of satisfaction
across various situations. Just more than half of participant respondents (53%) reported having
contact with an Avista representative, most commonly regarding their application. Other reasons
included concerns or questions about project implementation or the rebate. Far fewer
respondents reported contacting Avista representatives about contractors or other issues
(Figure 4-8). Of the 155 respondents who had contact with an Avista representative, almost all
(96%) agreed that the Avista representatives they worked with were courteous and helpful.
100%
1/) ..... C QI 75% "C C 0 C. 1/)
QI a: .... 50% 0
QI
bO ro ..... C QI 25% u ... QI
Cl.
0%
Figure 4-8: Reasons for Contact with Avista Representatives
I 11 11
Application
assistance
Project
implementation
Rebate
Reason for Contact
--Contractor issues -­Other issues
• Percentage of those contacting Avista (n = 155) • Percentage of all participants (n=305)
Of the 95 participants that received on-site inspections for their prescriptive shell work or site
specific work, all agreed the program representative was courteous and efficient when
conducting the inspection. All 18 participants familiar with the on-site audit reported the auditor
helped them understand energy efficiency opportunities and how to pursue those opportunities.
Audit participants generally reported their program experience would likely result in future
actions. Of the 35 participants who received an audit, most (24) indicated they were in the
process of implementing all (10) or some (13) of the recommendations (one did not know
whether it was some or all). Of the remaining 11 participants, seven did not know whether any
of the audit-related upgrades were planned or under way and four stated they would not
implement any audit recommendations.
4.4.1.2 Application Preparation
Across both years studied, about half of all participants reported they prepared the information
for the rebate, but this percentage was somewhat lower for 2015 participants than for 2014
participants (Figure 4-9). A larger percentage of 2015 respondents reported that their contractor
t.-'1 Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 45
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 53 of 151
4 NONRESIDENTIAL PROCESS RESULTS
was involved in preparing the application than did 2014 respondents (47% vs. 31%; Chi-square,
p < .005).18 Similarly, Washington participants were more likely to receive assistance from their
contractor than Idaho participants (16% vs. 6% respectively, Chi-square p < .05).
2015 (n=151)
2014 (n=154)
Figure 4-9: Who Prepared Application?
42% 10% 28% . . .
{ ,,.,-. • ' -~ "; "'!~ M "' 53% 9% ,; ·,,. 23% ·,.·· ,
r !'.: I r , • ~ '°" , • • ~
0% 25% 50% 75%
• Respondent • Someone else in respondent organization
• Contractor
• Avista representative
• Respondent firm assisted by contractor
•Don't know
100%
The above information is not completely consistent with contractors' reports that customers
typically do not complete rebate applications without assistance from a contractor or distributor.
Almost 80% of surveyed contractors (23) reported that the contractor completes the application
(12), the respondent and the contractor complete the application together (8), or a third party
such as a distributor completes the application (3). Possibly some of the difference between the
participant and contractor responses reflects projects that customers self-installed, which the
contractors would not know about. However, it is unlikely that this accounts for a large part of
the discrepancy.
A total of 190 respondents reported they reviewed Avista program information. Of those, about
three-quarters or more said that information from Avista was clear regarding how to apply, what
equipment was eligible, and how to reach program staff for assistance. A somewhat lower
percentage (67%) reported that the information on potential energy savings was clear (Figure
4-10).
18 These percentages refer to the light green ("Contractor") and purple ("Respondent firm assisted by contractor'') portions of each
bar, combined.
t.-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 46
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 54 of 151
4 NONRESIDENTIAL PROCESS RESULTS
Figure 4-10: Clarity of Avista Program Information (n = 190)
About how to apply for rebates
About how to follow up with program staff if you
had any questions or concerns
About what equipment and energy-saving items
qualify for rebates?
About the fact that someone from the program
may inspect your energy upgrades prior to
payment of the rebate (n=82)
On the energy savings you might expect from
the energy efficient items
9% as% I
8% 83% ~
1R 8~ I
15%, 74% ~
18% 67% ~
0% 25% 50% 75% 100%
• Not clear • Neither clear or unclear • Clear • Don't know
* This applies only to participants of programs with audits. Ther.efore then for this is 82, not 190.
4.4.2 Contractor Program Satisfaction
The 24 contractors that reported any of their jobs received an Avista rebate reported their
satisfaction with nine elements of the program across three areas: program-specific areas like
rebates and measures, interactions with program staff, and program marketing.
Satisfaction levels varied across the program elements. Contractors reported highest
satisfaction with how staff explains the program, the amount of rebates, and the ability of staff to
resolve problems.19 They were less satisfied with marketing and the range of qualifying
products; overall, 22 reported they were less than satisfied with at least one element (Figure
4-11). Program staff reported that marketing is not widely conducted, particularly in the site
specific program. Account executives conduct most customer and contractor outreach , which
means that contractors do not see or at least are not aware of marketing efforts.
19 Here, "'satisfied" means they rated an item as four or five on a satisfaction scale ranging from one ("not at all satisfied") to five
("very satisfied"); "less than satisfied" means a rating of three or lower.
t-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 47
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 55 of 151
4
bD C: ·..:;
OJ ~ "' ~
V, u .;::
NONRESIDENTIAL PROCESS RESULTS
Figure 4-11: Commercial Contractor Satisfaction with Program Elements (n = 24)
Ability of staff to explain how the program works 8% 4% 75% ~ 13%
Ability of staff to resolve problems 8% 8% 67% ~17%
Ability of staff to communicate the status of applications 8% 21% 58% if 13%
Quality of Avista's Marketing 8% • 25% 54% / 13%
Amount of marketing Avista does for the program 17% J• .J '_ , 38% 38% 1 8%
Amount of the rebates 17% .8% 71% t,
·~ Length oftime required to complete program paperwork 13% i?'.21% 63% Ii
C. V,
E
~ bD 0 C:
Avista program website 25% 50% 51:1'21%
Range of qualifying products 25% .",c · 25% :so%
0% 25% 50% 75% 100%
Categorized Responses to 5-Point Satisfaction Scale
• Not satisfied {1 & 2) • Neutral (3) • Satisfied (4 & 5) • Don't know
The evaluation team asked all contractors that were less than satisfied with a program element
to specify what they were dissatisfied with. Contractors identified the following issues:
• Marketing. Of the 14 who were less than satisfied about the amount of marketing Avista
does, one suggested that Avista advertise the program in supply houses. The others
provided no suggestions beyond that Avista should do more marketing directly to
commercial customers.
• Range of qualifying options. Of the 12 that commented about the range of qualifying
options, eight provided specific issues. Three did not approve of using the Design Lights
Consortium (DLC) list because it excluded specific items. Two noted that the exclusion
of T8s from the incented list hurt their business. Two others stated specific lights were
not on the list -one reported specific LED fixtures that require onsite evaluation and the
other noted 1000 watt LEDs are not covered.
• Length of time required to complete program paperwork. Eight contractors
remarked that the time it takes to complete program paperwork, typically 4 to 6 weeks, is
too long. One contractor reported it took his last customer four to five months to receive
their rebate.
'-"'Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 48
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 56 of 151
4 NONRESIDENTIAL PROCESS RES UL TS
• Avista program website. Seven contractors had difficulty finding the information they
needed on the website.
• Amount of the rebates. Six contractors reported the rebates were too small to motivate
some customers to do projects. One of these contractors implied that in order to make
up for the smaller rebates, installers are doing jobs for less profit than in the past.
• Staff responses. Seven contractors reported difficulties with staff not getting back to
them when needed.
• Ability of staff to explain the program or resolve problems. Of five contractors noting
some type of difficulty in this area, two noted staff were inflexible in their interpretations
of installed work, while three simply reported generic problems communicating with staff.
4.4.3 Perceived Value of Rebates -Contractor Perspectives
Three-quarters of the contractors reported they always tell customers about rebates, and nearly
as many said that rebates drive customers to install efficient equipment. Fewer contractors
agreed that Avista rebates help sell jobs. This suggests perhaps that some contractors believe
they would still be able to sell work without the rebates, but the work would not necessarily
involve efficient equipment. Even fewer agreed that the program rebates help keep them
knowledgeable about new technologies (Figure 4-12).
Figure 4-12: Contractor Perceived Value of Avista Rebates (n = 29)
Always tell customers about Avista rebates
The Avista rebates push customers to install more
efficient equipment
Avista rebates help me sell jobs
The Avista rebates help keep me knowledgeable
about new technologies
0% 25% 50%
• Not agree ( 1&2 ) • Neutral (3) • Agree (4&5) • Don't know
75% 100%
The latter finding does not mean that the program does not in some way help to keep
contractors knowledgeable about new technologies, just that the rebates themselves do not
necessarily do that. Staff expressed interest in providing more education and training
opportunities for contractors in the future. If more training occurs, future evaluations may
demonstrate that the program rebates contribute to contractor knowledge.
t-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 49
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 57 of 151
4 NONRESIDENTIAL PROCESS RESULTS
4.4.4 Driving lncented Upgrades -Contractor Perspectives
Contractors reported their and their customers' roles in initiating upgrade projects and
communicating about rebates. When asked what percentage of upgrade jobs they and their
customers initiated, contractors most commonly indicated that customers always or usually (at
least 60% of the time) initiated upgrades, but one-third said that it was close to half customer­
initiated and half-contractor initiated. The least common response was that the contractor
initiates most or all upgrades (Table 4-6).
Table 4-6: Contractors' and Customers' Roles in Initiating Upgrades
Who initiates commercial upgrade jobs? Count Percent
Always contractor 2 7%
Usually contractor* 4 15%
Mixed 9 33%
Usually customer* 6 22%
Always customer 6 22%
TOTAL 27 100%
*Usually = at least 60% of the time.
Surveyed contractors reported the percentage of customer-initiated upgrade jobs in which the
customer asked about rebates and, conversely, the percentage of contractor-initiated upgrade
jobs in which the contractor (or their staff) told the customers about the rebates. This provides
additional information about the importance of whether customers or contractors initiate
upgrades: if customers do not ask about rebates when they initiate upgrades, then it is
important for Avista to ensure that contractors always tell their customers about the rebates.
Findings from the contractor survey show that customers do not commonly ask about rebates
when they come to the contractor with an upgrade idea. By contrast, contractors reported that
they do usually tell their customers about the rebates when they themselves suggest the
upgrade idea (Table 4-7).
t-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 50
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 58 of 151
4 NONRESIDENTIAL PROCESS RESULTS
Table 4-7: Contractors' and Customers' Roles in Discussing Rebates
0% 3 10% 7 24%
1% to 25% 16 55% 0 0%
26% to 50% 5 17% 4 14%
51% to 75% 3% 3%
76% to 100% 4 14% 17 59%
TOTAL 29 100% 29 100%
MEAN 30% 65%
Simultaneously considering both of the above sets of questions -what percentage of projects
are contractor-or customer-initiated and what percentage of contractors and customers take the
initiative in discussing rebates when they initiate the upgrade discussion -provides additional
information, including a more meaningful look at the contractors' role in driving incented
upgrades.
For one thing, looking at all the data together presents a different perspective of the roles that
contractors and customers have in driving the rebate discussion than the above table shows. Of
the 24 contractors who answered all of the pertinent questions, 13 reported both that they told
customers about rebates in at least 75% of the jobs they initiated and that customers asked
about rebates in 25% or fewer of the jobs the customers initiated. Thus, for just more than half
of contractors, the rebates likely would not get discussed unless they brought them up.
So how often do rebates get discussed? For each contractor, the percentage of upgrades in
which rebates are discussed is the sum of two products: 1) the percentage of customer-initiated
jobs times the percentage of those jobs where the customer asks about rebates; and 2) the
percentage of contractor-initiated jobs times the percentage of those jobs where the contractor
tells the customer about rebates. For the 24 respondents that provided all those data, this
analysis indicates that, on average, rebates are discussed in 57% of upgrade jobs. This is more
than double the mean percentage of jobs that actually receive rebates (reported in Section
4.2.1, above), suggesting that less than half of potential jobs in which rebates are discussed
actually become incented upgrades. A deeper investigation into the process from initial
discussions between the contractor and customer through installation of incented high-efficiency
equipment may prove fruitful in future evaluation research.
Table 4-8 shows a final perspective on the relative roles that contractors and customers play in
driving incented upgrades according to contractors. The evaluation team coded responses as
indicating whether customers and contractors each played a large, mixed, or small role.
t1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 51
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 59 of 151
4 NONRESIDENTIAL PROCESS RES UL TS
Responses indicate that contractors are much more likely than customers to play a large role in
driving rebate upgrades, while customers are much more likely to play a small role.
Table 4-8: Contractors' and Customers' Relative Roles in Driving lncented Upgrades
Size of role Customers... Count Contractors... Count
Large ... initiate jobs and ask about 5 ... initiate jobs and tell about 14 rebates at least 50% of the time rebates at least 50% of the time
Mixed ... initiate jobs or ask about 18 ... initiate jobs or tell about 10 rebates at least 50% of the time rebates at least 50% of the time
Small ... initiate jobs and ask about 5 ... initiate jobs and tell about
rebates less than 50% of the time rebates less than 50% of the time
4.4.5 Participant Concerns
Six percent of participants (19 of 305) reported they had had some concerns at some point
about their participation in the program. Twelve reported concerns relating to program
processes, four expressed concern that the rebate would be inadequate, two noted concern
about the quality of products, and two others expressed concern about the range of products
available. Of the 19 expressing some concern about participation, 12 suggested that their
contractor (6) or an Avista representative (6) helped alleviate their concerns about participation.
4.5 Opportunities for Increasing Program Participation
Avista staff have considered ways to increase program participation, such as continuing to
move often-used Site Specific measures into the prescriptive measures list, developing deemed
savings measures for the existing Energy Smart Grocer program and expanding outreach to
restaurants, and mining customer data to better target customers for efficiency programs.
To assess possible opportunities for the program, the evaluation team asked nonparticipants
about recent building upgrades and future plans for upgrades. Lighting and HVAC upgrades
were the most commonly cited recent upgrades, and more nonparticipants reported installing
efficient lighting than efficient HVAC equipment.
A total of 43 of the 70 (61 %) surveyed nonparticipants reported either that they had upgraded
equipment or building features in the past two years (n = 34) or that they planned to do so in the
next two years (n = 20). An additional 17 respondents said they were not sure whether or not
they would upgrade equipment, while about half of the respondents (n = 33) said they do not
plan upgrades in the next two years. Of the past and planned upgrades, a little more than half of
were for lighting or lighting controls, with HVAC representing the next most common equipment
type (Table 4-9).
t..1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 52
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 60 of 151
4 NONRESIDENTIAL PROCESS RES UL TS
Table 4-9: Equipment Replacements or Upgrades Made by Nonparticipants in Past Two
Years or Planned for Next Two Years (Count and Percent of Total)
Equipment or Upgrade Upgraded Plan to Upgrade Upgraded and/or Plan to
(n = 34) (n = 20) Upgrade (n =43)
Lighting or lighting controls 23 (68%) 11 (55%) 29 (67%)
Heating, cooling, HVAC 7 (21%) 6 (30%) 10 (23%)
Building shell a 4 (12%) 2 (10%) 5 (12%)
Water heating 4 (12%) 3 (15%) 6 (14%)
Motors or motor controls 3 (9%) (5%) 3 (7%)
Food processing and storage 2 (6%) 0 (0%) 2 (5%)
Other I 3 (9%) 4 (20%) 7 (16%)
a Includes insulation (attic, ceiling, and wall) and windows.
Of the 34 nonparticipants who reported recent equipment or building upgrades, 25 (74%) said
they selected an energy efficient version.20 Similarly, 17 of the 20 nonparticipants (85%) who
planned future equipment or building upgrades affirmed that they were considering using above­
standard-efficiency equipment, while the other three said they were unsure what equipment they
would select or that they might select energy efficient equipment if the cost was not too high. A
total of 34 -half of the respondents -reported they either did use or planned to incorporate
energy efficiency in an equipment upgrade.
Respondents rated the influence of various factors on their decision to carry out energy efficient
upgrades and/or on their plans to do so. Increasing comfort, reducing O&M costs, and
increasing productivity were most commonly cited as being influential, and Avista marketing was
least influential (achieving a green image and contractor/vendor recommendations had
intermediate levels of influence; Table 4-10).21
2° Four respondents reported that they received financial incentives from utilities or government agencies for their upgrades -three,
for lighting or lighting controls and one, for unknown equipment.
21 Here, "influential" means they rated influence as a 4 or 5 on 1-5 scale, where 1 was "no influence" and 5 was "great influence;"
"'°'Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 53
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 61 of 151
4 NONRESIDENTIAL PROCESS RES UL TS
Table 4-10: Factors Influencing Nonparticipants' Recent or Planned Purchase of Energy
Efficient Upgrades
Lighting/lighting controls (n = 18) 3 14 6 5 7
Non-lighting (n = 9) 7 6 6 4 2 4
Planned upgrades (n = 17)
Any planned upgrade (n = 17) 9 15 9 10 7 6
a "Influential" is defined here as a rating of 4 or 5 on a 5-point scale, from "no influence" to a "great influence."
The small samples sizes argue for caution in comparing the ratings for past and planned
upgrades or those for lighting and non-lighting upgrades. Nevertheless, one comparison is
worth mentioning. Three-quarters of respondents who did non-lighting upgrades cited
"increasing comfort" as influential; by contrast, the proportion was closer to one in six for those
who did lighting-related upgrades. The idea that upgrading HVAC or building envelope can
produce greater comfort may seem obvious, while associating lighting with increased comfort
may seem less so -nobody puts on a sweater because of poor lighting. However, research has
demonstrated that lighting is an important factor in workplace comfort and satisfaction.22 Given
that employee comfort is a motive for upgrading other equipment types, messaging that cites
proper lighting as a comfort issue, and not just a productivity or cost issue, may help motivate
greater uptake of energy efficient lighting.
Of the nonparticipants who reported plans for energy efficiency upgrades in the next two years,
ten reported it was likely their organization would apply for Avista rebates, and two were not
sure whether it was likely or not.23 Of the five who indicated they were unlikely to apply for
Avista rebates, one indicated it was because they rely heavily on propane. The other four did
not provide clear reasons: one said that the use of rebates was "not part of their policy directive"
but did not explain why, two said it was because of lack of awareness of the rebates, but they
did not clarify the likelihood of applying would change now that they were aware of the rebates,
and one did not provide any reason at all.
22 See, for example, a summary of research conducted at Cornell University:
http://ergo.human.cornell.edu/lighting/lilstudy/lilstudy.htm. Accessed on April 6, 2016.
23 "Likely to apply"= a score of 4 or 5 on a scale where 1 equaled not at all likely to participate and 5 equaled very likely to
participate.
t.-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 54
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 62 of 151
4 NONRESIDENTIAL PROCESS RESULTS
The six nonparticipants who did not do efficiency upgrades as part of their equipment
replacements either said they lacked capital (two mentions) or they did not prioritize energy
efficiency, were not aware of efficient options or incentives, or did not find efficient equipment
that matched their needs.
4.6 Freeridership and Spillover
This section summarizes results about freeridership and spillover, two key aspects of energy
efficiency programs. Freeridership represents an estimate of the energy savings that the
program participants would have achieved without the program's assistance, and spillover is
what additional energy saving actions occurred outside the program but as a result of program
influence. For a discussion of the methods used to calculate freeridership and spillover values,
see the 2014-2015 impact report discussion about net-to-gross calculations. Additionally, the
impact report covers how freeridership and spillover rates affect savings.
This section discusses freeridership first and spillover second.
4.6.1 Freeridership
The evaluation team examined freeridership for three nonresidential programs: Prescriptive,
Energy Smart Grocer, and Site-Specific. Figure 4-13 shows the PY2014 and PY2015
freeridership results, weighted by program savings, plotted next to the weighted results reported
in the previous evaluation.24 The figure shows a general trend toward increase freeridership
over time, except for the values for PY2011 .
24 Avista 2012-2013 Process Evaluation Report, May 15, 2014. Submitted by Cadmus to Avista Corporation. The previous
evaluation did not report freeridership values for PY2012.
t-'1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 55
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 63 of 151
4 NONRESIDENTIAL PROCESS RES UL TS
Figure 4-13: Freeridership Values Over Time
100%
80%
60%
42%
40% 35%
32%
25% 26%
20%
14% I I i 13%
I 9% i .:.I 0% II
Prescriptive Energy Smart Grocer Site-Specific
•2010 • 2011 • 2013 • 2014 & 2015
The previous evaluation attributed year-over-year changes in freeridership from 2010 to 2013 to
a small number of participant scores having large effects on the program freeridership score
because of the size of their project savings. Freeridership scores are weighted by savings and
the highest saving projects in the sample can have a strong influence on freeridership scores.
Not discounting the possibility that some of the increase in freeridership in PY2014 and PY2015,
relative to those from prior years, may to some degree reflect different methodologies used to
calculate freeridership, the evaluation team below has identified some possible explanations for
some of the observed variability over time in freeridership. These explanations are hypotheses
that would require additional analysis and research to verify.
4.6.1.1 Prescriptive
The dip in freeridership from 2011 to 2013 could reflect the removal of T12s as a baseline
lighting measure in 2012. Prior to December 2012, freeridership may have increased as
customers interested in replacing their T12s took action in 2011 and 2012 to maximize their
rebate amount before the baseline change to T8s lowered rebate amounts. According to this
hypothesis, many of those customers would have replaced their T12s anyway, and so were
freeriders or partial freeriders. After the baseline changed in 2013, freeridership then declined
(according to this hypothesis) because many of the T12 customers -likely partial freeriders -
were no longer participating, leaving mainly customers who really needed the incentives to carry
out the upgrades.
t-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 56
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 64 of 151
4 NONRESIDENTIAL PROCESS RESULTS
The uptick in freeridership seen in 2014 & 2015 could reflect the success of Avista's programs
in transforming the market over time. Another possibility is that the increase in affordable LEDs
over the last two years, in conjunction with rebates, may be spurring customers - likely partial
freeriders -to take action earlier than they otherwise would.
4.6.1.2 EnergySmart Grocer
The general trend in freeridership for the Energy Smart Grocer program is increasing over time.
This increase in freeridership co-occurs with declining participation rates in the program over the
last five years. In the earlier years of the program, freeridership may have been low because the
program was reaching grocers that were unaware of savings opportunities and were therefore
heavily influenced by the program -they were low freeriders -to take action. With the program
well established after several years of operation, possibly driving an increase in general
awareness of efficiency opportunities, one might expect to see an increase in freeridership as
more grocers are aware of energy saving opportunities and thus more likely to be interested in
participating.
4.6.1.3 Site Specific
The general trend in Site-Specific freeridership shows an increase over time. The explanations
for the increase in Prescriptive program freeridership and Energy Smart Grocer rates also apply
here. As Avista's programs mature, awareness of efficiency opportunities increases in the
market, wh ich in turn drives up freeridership rates.
Additionally, the LED lighting issue discussed in section 4.6.1.1 may also apply to site-specific
participants. The increased affordability of LEDs combined with the rebate prompts customers
considering a lighting upgrade to make that upgrade sooner making them partial freeriders and
driving the freeridership rate up.
4.6.2 Participant Spillover
Participant spillover occurs when program participants elect to conduct energy saving activities
outside of the program as a result of program influence. Because the actions took place outside
of the program, the program has no mechanism to capture these actions other than during
customer surveys. The analysis below shows how many participants reported they took a
spillover action. For an analysis and discussion of what effect these actions had on savings, see
the PY2014 and PY2015 impact report.
Of the 305 participants in the sample, twenty reported they were partially (10) or fully (10)
influenced by the program to undertake an energy efficiency project that did not receive a
rebate. Ten of the spillover participants took part in the prescriptive program (6%) and the other
ten took part in the Site-Specific program (9%). No Energy Smart Grocer participant reported
taking a spillover action (Table 4-11).
t.-"1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 57
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 65 of 151
4 NONRESIDENTIAL PROCESS RES UL TS
Table 4-11: Number of Participants Reporting a Spillover Action
Program Total Participants in Participants Who Did Percent of Participants Who
Prescriptive
Energy Smart Grocer
Site-Specific
TOTAL
t-1Nexanr
Sample Spillover Project Did Spillover Project
164 10 6%
35 0 0%
106 10 9%
305 20 7%
Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 58
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 66 of 151
5 Small Business Process Results
5.1 Small Business Process Evaluation Overview
The primary goal of the Small Business (SB) process evaluation was to assess and provide
information on program delivery and implementation and market response to the program. The
evaluation focused on program design and theory, implementation and delivery, and market
feedback.
5.2 Summary of Program Data
In 2015, the program served 1,181 customers. All 1,181 customers received a basic, HVAC,
and lighting audit to determine savings opportunities and 1,013 (86%) received at least one
direct install measure.
Program staff target specific zip codes when conducting audits and installations. As of the end
of 2015, staff conducted audits largely in the Spokane area (69%) followed by the territory south
of Spokane (16%) and the area north of Spokane (15%) (Figure 5-1). As noted earlier, program
staff did not do work in Idaho but anticipate doing so in 2016.
t.-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 59
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 67 of 151
5 SMALL BUSINESS PROCESS RES UL TS
Figure 5-1: Areas targeted by SB program in 2015
Fraricin
Ferry
Unmn
i><lsns
Wala Walla
Targeted Zips
County
Avista territory
Stevens
Bollldary
Bomer
-
Kodenai
Spd<ane
CleS'\•.eter
Idaho
Overall, the program is meeting its participation estimates by exceeding estimated participation
in some areas and not meeting expectations in other areas. For example, the program
exceeded its overall lighting and audit estimates having installed 2,781 LED bulbs when they
anticipated installing 1,000 in 2015 and conducting 3,543 audits when anticipating 3,000. The
program did not meet its estimates in water saving items by installing 2,851 items compared to
their estimate of 4,325. Including audits as a "measure" the program exceeded the number of all
measures they anticipated for 2015 by 518. Excluding the audits by counting only installed
items, the program almost achieved its 2015 estimate perfectly by installing 15 units shy of the
expected number (Table 5-1).
t.-'1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 60
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 68 of 151
5 SMALL BUSINESS PROCESS RES UL TS
Table 5-1 Participation to Date Compared to Estimated Participation
Pa~~~~!!ion Estimated Part1cipat1on
2015 2015 2016 2017 Total
Water saving measures 2,851 4,325 8,650 4,325 17,300
Faucet Aerator (.5 and 1 GPM) 2,561 4,000 8,000 4,000 16,000
Shower Head (incl. Fitness Center) 147 250 500 250 1,000
Spray Valve 143 75 150 75 300
Plug load devices 778 1,100 2,200 1,100 4,400
CoolerMiser 277 75 150 75 300
Vending Miser 106 25 50 25 100
Tier 1 smart power strip 395 1,000 2,000 1,000 4,000
Lighting 2,781 1,000 2,000 1,000 4,000
Screw-in LED lamp (A-line 40W) 528 a a a a
Screw-in LED lamp (A-line 60W) 508 a a a a
Screw-in LED lamp (A-line 75W) 5 250 500 250 1,000
Screw-in LED lamp (A-line 1 OOW) 129 250 500 250 1,000
Screw-in LED lamp (BR30) 802 125 250 125 500
Screw-in LED lamp (BR40) 180 125 250 125 500
Screw-in LED lamp (PAR30) 393 125 250 125 500
Screw-in LED lamp (Par38) 236 125 250 125 500
Audits 3,543 3,000 6,000 3,000 12,000
Basic 1,181 2,000 4,000 2,000 8,000
HVAC 1,181 500 1,000 500 2,000
Lighting 1,181 500 1,000 500 2,000
Total measures including audits 9,953 9,435 18,850 9,425 37,700
Total measures excluding audits 6,410 6,425 12,850 ! 6,425 25,700
a The program did not provide estimates for these two measures.
5.3 Staff and Implementer Interviews
The evaluation team interviewed the Avista SB program manager, the SBW program manager,
the SBW field manager/auditor/installer, and SBW auditor/installer in December 2015 to better
understand the program. The interviews covered program goals and plans, implementation and
delivery, marketing and outreach, and program successes. The outcomes of the interviews are
summarized in the following subsections.
5.3.1 Program Goals and Plans for the Future
The program aims to serve customers that are typically hard to reach, such as "mom and pop"
operations. Typically this excludes national and regional chains that receive services via
traditional efficiency programs. A primary emphasis of the program is to develop interest in other
t-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 61
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 69 of 151
5 SMALL BUSINESS PROCESS RES UL TS
Avista programs and identify savings opportunities. Staff reported many opportunities for lighting
upgrades, particularly replacing T12s25, and upgrading food service equipment.
Currently, the program's exclusion of national accounts can exclude franchises owned by a
"mom and pop" operator. The program may consider expanding services to franchisees.
The program elected not to extend service to Idaho in 2015 because gas saving measures were
deemed not cost effective and, therefore, the program could not claim gas savings. Idaho is
reassessing gas measure cost effectiveness in 2016, at which point the program hopes to begin
serving the state.
The program is interested in doing a pilot study offering Tier II smart strips to customers.
5.3.2 Implementation and Delivery
The program initially targeted the Spokane area to allow Avista staff to easily attend inaugural
site visits and work out any potential problems that can arise in early implementation of a new
program. Staff reported few problems in the early stages of the program and all reported
successful and adequate amounts of communication between Avista and SBW.
The auditors/installers pass leads to other Avista program staff, relying on their assessment of
the participant's likelihood to proceed with another program. According to both Avista and SBW
staff, the auditors/installers have struck the right balance of providing good leads to Avista
without overwhelming Avista staff with leads unlikely to result in projects.
Auditors/installers pay close attention to the hours a business uses its lights before installing
lighting measures. Lights are not installed where the existing lights are used less than 60 hours
per week, as such replacements would not be cost effective.
5.3.3 Marketing and Outreach
Auditors/installers conduct almost all program marketing through door-to-door outreach efforts.
Occasionally program staff receive leads from other small businesses that heard about the
program via a colleague or neighboring small business.
Avista provides a list of Schedule 11 customers that SBW uses to target potential participants.
In the rare occurrence an auditor/installer sees a business not on the list that looks like it
qualifies, they can seek permission from Avista to reach out to the business. Permission is
typically granted quickly. Auditors/installers try "multiple attempts" to reach targeted small
businesses before giving up on a site.
25 Program data shows that 35% of SB participants had T12 lights in place.
t-"1 Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 62
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 70 of 151
5 SMALL BUSINESS PROCESS RES UL TS
5.3.4 Program Successes
According to program staff, customers rarely reject items and the program data reviewed by the
evaluation team supports their assertion. There were two cases where customers refused a
specific recommended item and 18 decommissioned items26 out of the original 6,428 installed .
Once approached by staff, very few small business participants refuse the service. According to
staff, as of mid-December 2015, 12 of about 1,000 potential participants refused. Typically these
refusals are because the auditor/installer could not ultimately reach a decision maker in the
business or because of general suspicion of the program.
Staff reported the SB program offers strong customer service and relationship building between
Avista and its SB customers. For example, according to staff, businesses were particularly
"grateful" for the outreach from their utility immediately following the November 2015 windstorm
that left 200,000 businesses without power. Staff also noted that a key trait required of the
auditor/installer is someone with "excellent" customer service skills who can serve as an
"ambassador" for the utility by relating to people and meeting with participants when it is
convenient for them.
5.4 Participant Surveys
The participant survey covered how respondents learned about the program, their rationale fo r
participating, energy saving topics discussed with the installer, program satisfaction, and plans
for future upgrades. The evaluation team covers these topics below starting with a profile of
respondents and their businesses.
5.4.1.1 Business Characteristics
Respondents represented a variety business types with a variety of energy using types of
equipment. Retail establishments and offices represented close to half of all survey respondents
followed by warehouses, auto repair shops, and food service establishments. All respondents
had heating equipment and almost all had water heating, computers, and cooling equipment
(Table 5-2).
26 Staff removed eight .5 gpm aerators, eight spray vales, one CoolerMiser, and one VendingMiser after installation due to customer
complaints about the measure. Customers, particularly dishwashers, were dissatisfied with the water pressure post installation. It
was unclear why the participants were dissatisfied with the Misers.
t.-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 63
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 71 of 151
5 SMALL BUSINESS PROCESS RES UL TS
Table 5-2: Small Business Respondent Characteristics (n = 34)
Count Percent
Business Types
Retail 8 24%
General office 7 21%
Warehouse/wholesale 4 12%
Auto/truck repair 4 12%
Food service (restaurants) 3 9%
Personal services (spa, salon, gym) 2 6%
Medical or dental 2 6%
Small production 2 6%
Small grocery 3%
Religious institution 3%
Energy Using Equipment
Heating equipment 34 100%
Water heating equipment 33 97%
Electric water heating 17 50%
Gas water heating 16 47%
Computer and office equipment 33 97%
Cooling equipment 32 94%
Refrigerator 28 82%
Air compressor 8 24%
Ventilation fans 7 21%
Freezer 5 15%
Cooking equipment 4 12%
Other 3 9%
5.4.1.2 Program Marketing and Rationale for Participation
The evaluation team conducted the surveys with SB owners, managers, or other people in a
leadership at the business. Almost all reported learning about the SB program through an in­
person visit (26 of 34) or a phone call from a program representative (5 of 34). The remaining
three respondents did not remember how they heard about the program.
Respondents chose to participate for a myriad of reasons. More than two-thirds of respondents
(23 of 34) reported two or more reasons for participating in the program. Most commonly
t-"1 Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 64
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 72 of 151
5 SMALL BUSINESS PROCESS RES UL TS
respondents elected to participate to save money on their energy bills (59%) or for equipment­
specific reasons (47%; Table 5-3).27
Table 5-3: Reasons for Participating in SB Program (n = 34)
Count Percent
Saving money on energy bills 20 59%
Equipment-related reasons 16 47%
Get free equipment 12 35%
Acquire the latest equipment 2 6%
Seek improved lighting 2 6%
Learn more about energy efficient lighting 2 6%
Conserving energy/protecting the environment 10 29%
Representative was convincing 3 9%
Overall positive for store 3%
In addition to the reasons for participating, shown above, 18 respondents (53%) said they
participated because participation was easy. Ease of participation is not in itself a reason to
participate -it does not offer any specific benefit. But these responses provide important
feedback about the process, namely that an easy participation process encourages
participation. Three respondents gave no reason for participating other than that it was easy.
Respondents largely had not considered installing SB measures prior to the program. Four of
the 34 respondents stated they considered upgrading the efficiency of their lights, and no
respondent noted considering water or power saving upgrades such as aerators or power
misers. Of the four that considered lighting upgrades, three stated it was unlikely they would
have made the change without the program and one reported it was likely.
5.4.1.3 Energy Savings Discussions with Installer
To understand how the interactions with the assessor helped them decide what equipment to
replace, the survey asked respondents what they discussed with the assessor. More than three­
quarters reported discussing lighting upgrades, mainly about the type or quantity of lighting to
be replaced (Table 5-4). A minority of those who mentioned lighting indicated that they had
discussed past Avista-supported lighting upgrades with the assessor. Other common discussion
topics were the expected energy savings from upgrades and water-saving measures. Far fewer
respondents indicated that they discussed prioritization of energy-saving projects or about
equipment cost.
27 Ultimately, the equipment-related reasons likely are not really the ultimate motives. It is likely that these responses signify one of
the other motives that were stated more explicitly, namely, saving money or environmental reasons.
t-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 65
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 73 of 151
5 SMALL BUSINESS PROCESS RES UL TS
Table 5-4: Topics Discussed with Installer (n = 34; Multiple Responses Allowed)
Count Percent
Lighting upgrades 27 79%
Type of lights/fixtures to be replaced 20 59%
Quantity of lights/fixtures to be replaced 13 38%
(Past) fluorescent replacement a 6 18%
Quality of lights/fixtures 4 12%
Energy savings resulting from installed equipment 16 47%
Water measures 11 32%
Prioritization of energy-saving projects 4 12%
Plug load 3%
Equipment cost 3%
a The current SB program does not incent fluorescent lighting; the context of some of the comments indicated that
this refers to a previous fluorescent change-out.
More than half (20 of 34) of respondents reported that the installer recommended energy-saving
projects outside the scope of the SB program. Of those 20, most reported the installer
recommended lighting changes (15), including one specifying motion sensors. Four reported
that the installer recommended HVAC upgrades, two said the installer recommended a
refrigeration control unit, and two did not recall the recommendation. No respondent suggested
the program should supply additional equipment.
5.4.1.4 Program Satisfaction
Participants tended to be satisfied with all aspects of the program other than the energy savings
resulting from program participation. In that case, most participants reported not knowing what
savings, if any, resulted from the program measures.
t-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 66
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 74 of 151
5 SMALL BUSINESS PROCESS RES UL TS
Table 5-5: Satisfaction with Program Elements
Your overall experience with the program
Clarity of information provided by your assessor
Your interaction with program contacts
The ease of the paperwork
The scheduling of the installation of measures
The quality of the installation work*
The performance of the new equipment that was
installed*
The energy savings your business has experienced since
the equipment was installed
6% 94%
94%
94%
94% I
91% 6%
91% 6%
6% 88% 6%
0% 25% 50% 75% 100%
• Dissatisfied (1 & 2) • Neutral (3) • Satisfied (4 & 5) • Don't know
Of the five respondents reporting they were dissatisfied or neutral about an element, three
explained their reasons for not being fully satisfied.
• One respondent was dissatisfied with the water pressure from the program-supplied
spray valve. 28
• One respondent was dissatisfied with all program elements because she was ineligible
to receive many measures because her store did not meet the minimum weekly number
hours of lighting.29
• One respondent reported the auditor never followed up with them or provided
equipment.30
Respondents tended to report that they upgraded all areas they could with program measures.
In the two cases in which a respondent reported not replacing any water-saving equipment, they
reported the measures did not fit.31
28 This respondent may have had their spray valve decommissioned by program staff. Of the 1,013 participants that received an
item, 18, or 1.8% of the population had something decommissioned by staff. This one case out of 34 represents 2.9% of the sample.
29 This respondent's business was open 48 hours per week and the program requires lights to be used 60 hours or more week
before making LED replacements. This respondent received two faucet aerators.
30 The respondent did receive a promised faucet aerator and vending miser. Program staff verified this during a follow-up call on or
about February 5, 2016.
t.-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 67
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 75 of 151
5 SMALL BUSINESS PROCESS RES UL TS
5.4.1.5 Future Upgrades
About two-fifths of respondents (14 of 34) reported plans to make energy saving upgrades
within a year after their SB program participation. Most of these respondents (11) said they plan
to make a lighting change, three reported plans to make an HVAC upgrade, and one said they
plan to install a programmable thermostat32 . Of the 11 indicating they will make a lighting
change, two respondents noted they are making the change to save energy, one of whom is
also interested in improving the light quality in his building. The remaining nine did not provide a
reason why. Almost two-thirds of those who plan to make an upgrade (9 of 14) said their
participation in the SB program influenced this decision. Four respondents stated the program
was not influential in their future upgrade decision and one respondent was neutral about the
program's influence.
Respondents reported financial considerations, like the cost of equipment and the payback
period, were important considerations when making building upgrades. Almost the same
percentage of respondents reported product considerations, such as a robust warranty and
recommendations from contractors, were important. Far fewer respondents reported
environmental attributes of the equipment or labeling was important to them (Figure 5-2).
Figure 5-2: Considerations When Making Building Upgrades (n = 34)
"' C:
- 0 -~ ~ u "' C: ~
"' Cl) C: :-2 ·-"' LL C:
0 u
"' C:
The initial cost of the equipment
The payback period
Lifetime rate of return
A robust warranty
,g A recommendation by a vendor or contractor you trust e Cl)
"O
-~ The equipment's effect on greenhouse gases 0 u .... u ::, -g Having and EStar or other environmental label
a:.
Purchasing a familiar brand
18% 82%
% 15% 79%
%'\:'15% 79%
21% 27% 50%
12% 41% 47%
27% 27% 47%
0% 25% 50% 75%
• Not important • Neutral • Important • Don't know
31 Program data shows these respondents did actually receive aerators.
t-'1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs
100%
68
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 76 of 151
5 SMALL BUSINESS PROCESS RES UL TS
Almost half of respondents (16) said they had known about Avista energy saving programs
before they participated in the SB program, with three to four each reporting their source of
awareness being a contractor/distributor, word-of-mouth, Avista bill stuffer, or regular contact
with an Avista representative and one each citing print advertisements and the Avista website.
Almost all respondents (32) reported they could consider contacting Avista prior to making any
building upgrades; the other two did not know whether they would contact Avista.
t-'1 Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 69
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 77 of 151
6 Residential Process Results
6.1 Program Administration
The evaluation team conducted in-depth interviews with Avista program staff, implementation
contractors, and Community Action Partners (CAPs) and a survey with contractors to obtain an
understanding of how the Avista's residential programs are administered and what challenges
these various actors have faced in delivering these programs to the market. The following
subsections describe the findings from these interviews and the contractor survey.
Note that the evaluation team organized this section by each program covered in this
evaluation. The organization is as follows:
•
•
•
•
•
For the rebate programs, the evaluation team described feedback provided by
contractors and Avista's program manager about administration and experience with
these programs.
For the Appliance Recycling, the team reported feedback by JACO, the program
implementer, on administration and program challenges.
For the Simple Steps, Smart Savings, the team reported feedback by Avista's program
manager and CLEAResult, the program implementer, on administration, program
evolution, and future opportunities.
For Home Energy Reports or HERs, the team reported feedback by Avista's program
manager and Opower, the program implementer, on administration , challenges, and
future opportunities.
The team also reported feedback from CAP agencies -agencies who implement the
low-income program for Avista.
6.1.1 Rebate Programs
This section presents results from the contractor survey and Avista staff interviews related to the
rebate programs (i.e., Shell, HVAC, Fuel Efficiency, Water Heat, and ENERGY STAR Homes).
Contractors were surveyed about their interactions with Avista program staff, their satisfaction
with Avista's residential rebate programs, their sales history and their recommendations for
future program opportunities. Avista staff reported on interactions with contractors and future
program opportunities.
6.1.1.1 Contractors Interaction with Avista and Program Awareness
Almost all contractors reported doing an Avista rebated project in the last year and about half
completed 50 or fewer Avista rebated jobs in 2015. HVAC contractors reported doing more
Avista rebated projects than Shell contractors (Figure 6-1).
t.-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 70
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 78 of 151
6 RESIDENTIAL PROCESS RES UL TS
Figure 6-1: Contractors Number of Avista-Rebated Projects (n=53)
30
25
rn c i 20 C
0 a. rn !!! 15 -0
ai .c 10 E ::I z
5
0
HVAC (n=35) Shell (n=18) Total (n=53)
• Zero jobs • 1 to 50 jobs • 51 to 100 jobs • More than 100 jobs • Don't know
Avista projects constituted a considerable portion of all contractors work. HVAC contractors
reported, on average, that 42% of their work received Avista incentives and shell contractors
reported, on average, that 31 % of their work received Avista incentives.
Surveyed contractors reported being aware and familiar with at least some Avista programs.
More than three-quarters (42) of residential contractors reported completing projects that
received Avista rebates for at least the past five years. Seven more reported completing Avista­
projects for four to five years, and four contractors reported completing rebated projects for
three years or less. Furthermore, almost all (45 of 46) residential contractors who were able to
estimate the amount of Avista-related work they completed in the last year, reported completing
at least one rebated project in the last year. Additional analysis shows contractors spend
considerable time working on Avista-rebated projects. Almost two-fifths (39%) of contractor
completed projects, on average, received Avista rebates.
6.1.1.2 Avista's Interaction with Contractors
Although contractors are familiar with the Avista's rebate programs, there are relatively few
interactions between Avista staff and contractors. According to program staff, Avista primarily
interacts with contractors when contractors call to request information on behalf of their
customers. Avista does not currently offer any formal training for contractors on the rebate
programs, and Avista staff only occasionally visit contractor offices to hand out rebate
information, the only face-to-face outreach activity reported by program staff. This indicates that
there is an opportunity for Avista to engage contractors more with the rebate programs.
6.1.1.3 Contractors' Program Satisfaction
Surveyed contractors reported their satisfaction with nine elements of the program across three
different areas: 1) program specific elements including rebates and measures; 2) their
interactions with program staff; and 3) program marketing (Figure 6-2).
i,1Nexanr Process Evaluation of Avista 's 2014-2015 Energy Efficiency Programs 71
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 79 of 151
6
0)
C ~ ~
RESIDENTIAL PROCESS RES UL TS
Figure 6-2: Residential Contractors Satisfaction with Program Elements (n=46)*
Length of time required to complete program
paperwork
Range of qualifying products
Avista program website
Amount of the incentives
Ability of staff to explain how the program works
Ability of staff to resolve problems
Ability of staff to communicate the status of
applications
Quality of Avista's marketing
85% t11%
13% 80% :l:Jt:
11 % 67% 2;;"!22%
17% '. 26% 54% !
. . ...... -· -65% ml28%
7% 65% 1~~28%
ro ~ Amount of marketing Avista does for the program
0% 25% 50% 75% 100%
• Not satisfied (1 &2) • Neutral (3) • Satisfied (4&5) • Don't know
*n=46 and not 53 because this question was seen only by those who reported a proportion of their projects received an
Avista rebate.
Of the three areas investigated, the program-specific elements had the highest proportion of
satisfied contractors. Most contractors reported being generally satisfied with three of the four
program specific elements included in the survey. The exception was rebate amounts, for which
nearly half reported being satisfied, and, unsurprisingly, nearly one-in-six contractors reported
being dissatisfied33-the single largest area of concern among the nine elements in the survey.
Specific mentions of dissatisfaction by respondents included:
• 22 respondents made unspecific comments about their desire for higher rebate levels.
• Seven respondents reported dissatisfaction with the number of rebate eligible products
in Idaho (2), the lack of geothermal products (1), and the lack of renewable energy
33 The evaluation team has seen across many evaluations that contractors often report wanting higher incentives. Higher incentives
help them sell more jobs.
t.-"1 Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 72
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 80 of 151
6 RESIDENTIAL PROCESS RES UL TS
product rebates (1). The remaining three implied that the existing range of products was
not large enough to attract customers but did not specify products or services.
• Five respondents expressed frustration with the program website finding it "confusing"
and hard to find information.
• Two respondents reported dissatisfaction with the amount of time it takes to complete
program paperwork.
A majority of contractors also reported being satisfied with the interactions with Avista staff. At
the same time, this is also a topic area for which many contractors responded "don't know",
suggesting that they either had no opinion on the topic or were unfamiliar or otherwise unwilling
to answer the survey questions. However, after excluding those respondents who reported
"don't know" about their staff interactions, the results indicate high levels of satisfaction with
Avista staff. Ninety-one percent of contractors (48 of 53) were satisfied with staff's ability to
resolve problems and communicate application status, and 87% were satisfied with program
staff's ability to explain how the program works.
Seven contractors reported some degree of dissatisfaction regarding their interactions with
Avista staff. Five reported communication-related difficulties with staff such as delays in getting
questions answered or problems identifying and contacting the right staff person. One noted
dissatisfaction with the amount of support staff provided in promoting the program and
expressed interest in having staff reach out to contractors more and help contractors promote
the program . The seventh respondent rated their staff interactions as a three (on a five-point
scale) but their comment about staff suggested they were pleased with staff performance.
Of the three satisfaction topic areas investigated, the marketing-related elements had the lowest
share of satisfied contractors. A minority of contractors, about one-tenth (11 %) indicated they
were dissatisfied with the amount of Avista's marketing and nearly one-tenth (9%) noted they
were dissatisfied the quality of marketing. However, in their follow-up comments, these five
contractors indicated they were largely unaware of Avista's marketing efforts or only saw the
materials sporadically. In addition , a notable minority of contractors answered "don't know" to
the two marketing-related questions, and a number of respondents answered the question with
a '3'-the midpoint on the rating scale. Collectively, these results suggest that contractors may
be more unfamiliar with Avista's marketing of the rebate programs more than they are
dissatisfied.
6.1.1.4 Contractors' Sales of Efficient Equipment
Rebates are an effective sales tool for contractors. Most contractors agreed that they always tell
customers about rebates and that the rebates help them sell more energy efficient equipment
and services to their customers, a finding that is supported by Avista staff. However, a relatively
low number of contractors agreed that the Avista rebates were helping them stay up-to-date
about new technologies (Figure 6-3).
4,..1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 73
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 81 of 151
6 RESIDENTIAL PROCESS RES UL TS
Figure 6-3: How Program Helps Residential Contractors (n=53)
I always tell Avista customers about Avista incentives
Avista incentives help me sell jobs
The Avista incentives push customers to install more
efficient equipment
The Avista incentives help keep me knowledgeable
about new technologies
a% as% I
a% 11 % • 11% • _ ._ I
23% . · . 10%. ·: . . . I
0% 25% 50% 75% 100%
• Not agree (1 &2) • Neutral (3) •Agree (4&5) • Don't know
Almost all residential contractors offer customers more than one option when selling products or
seNices. Of the 45 respondents that reported how many options they typically provide
customers34, 89% offered two or more options, and 42% of contractors offered three or more
options. The most commonly cited distinguishing characteristic among the options was energy
efficiency (62%), followed by price (22%), and then quality (18%). Only a few respondents (4%)
reported using non-energy benefits, such as improved comfort, to differentiate the options they
presented.
When discussing high-efficiency equipment options with customers, contractors tended to
mention lower operating costs (69%), higher quality (67%), and the Avista rebate associated
with the equipment (54%) (Figure 6-4).
34 Eight reported don't know
t.-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 74
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 82 of 151
6 RESIDENTIAL PROCESS RESULTS
Figure 6-4: Benefits of Efficient Equipment Mentioned During Sales (n=52)*
Lower operation costs over time
High-quality of equipment
Avista's rebate
Improved comfort
Lower maintenance costs
Warranty
Energy efficiency -8%
Quiet operation -6%
Low ultra-violet windows • 2%
Ease of installation • 2%
0%
12%
44%
54%
50%
25% 50%
Percent of Respondents
69%
67%
75%
* One respondent, excluded from this analysis, did not report mentioning any benefits of efficient equipment.
Three-quarters of contractors reported that they prepare all or most of the rebate application
(55%) or do the application in concert with the customer (21%). About a quarter stated the
customer typically prepares the application.
Six surveyed residential contractors reported discouraging their customers from purchasing
highly efficient equipment. They mentioned the following reasons:
• Three respondents mentioned structural barriers that made it difficult to install high
efficiency equipment. For example, one respondent reported adding additional venting
needed for a high efficiency furnace may add too much to the cost of the project to make
it viable.
• Two respondents reported the customer needed the lowest cost option.
• One respondent did not recommend high efficiency equipment when they knew a
customer would not benefit from the savings. For example, if a customer was not going
to be in the house long enough to realize benefits or savings of efficient equipment.
6.1.1.5 Future Rebate Program Opportunities
Contractors provided suggestions for additional equipment they would like rebated through the
programs, and ductless heat pumps and hot water saving measures were the most commonly
cited (Table 6-1). All 34 contractors that wanted these pieces of equipment added to the
program indicated they thought it would improve or encourage program participation.
t.-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 75
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 83 of 151
6 RESIDENTIAL PROCESS RES UL TS
Table 6-1: Contractor Suggestions for Additional Program Measures
Count Percent
Ductless heat pump 11 21%
Hot water measures 7 13%
Doors 5 9%
Air conditioning 3 6%
Geothermal 3 6%
Thermostats 2%
Furnace 2%
Insulated siding 2%
CO2 demand control ventilation 2%
Non-equipment specific suggestions* 4 9%
TOTAL 34 100%
* Two respondents wanted the lists in Idaho and Washington to be the same, one wanted gas rebates for people in
Kootenai Electric territory, and one wanted unspecified "new" equipment incented.
Avista staff reported investigating several possible future program and/or measure
opportunities, showing that Avista staff are preparing for the future and thinking about market
changes and innovative opportunities:
• Avista is tracking the heat pump water heater technology to assess whether it is an
opportunity in milder climate zones.
• Avista is testing the effectiveness of a smart thermostat pilot to assess whether the pilot
can be scaled-up into a program.
• There is some discussion on reconnecting with contractors.
• Avista is considering offering the manufactured home duct sealing program in Idaho and
increasing certain rebates: 1) water heater tank rebate (from $20 to $50), 2) tankless
water heater rebate (from $130-$180), and 3) high efficiency furnace rebate (from $250
to $300).
• Avista also is planning to install AMI meters in Washington to be able to develop
innovative options for delivering programs or different types of smart-grid programs in
the future.
6.1.2 Appliance Recycling Program
This section describes feedback from the interview with the implementation contractor, JACO.
JACO was interviewed about program administration and challenges.
1.1.1.1 Program Administration and Efficiencies
The Appliance Recycling program launched in 2008, and since then, JACO has worked to
improve the program's administrative processes. In 2014 and 2015, while the program was
operating, there were no major inefficiencies in these processes. As explained by the JACO
t-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 76
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 84 of 151
6 RESIDENTIAL PROCESS RES UL TS
representative, the basic process is as follows: customers call the toll-free number to schedule a
pick-up of a refrigerator or freezer, JACO will ask customers whether the unit is working, the
size of the unit, and the age of the unit to determine whether the unit qualifies for the program. If
the unit is eligible, JACO schedules a pick-up. At the pick-up site, JACO will check whether the
unit is working and the age of the unit prior to loading it onto the truck to decommission it. JACO
records all the information about the unit and the customer in their database. This database
allows JACO to have automated reporting to Avista and an automated dashboard that Avista
staff can access to view program progress. Additionally, customers receive an incentive check
in about four to six weeks from the pick-up date. The JACO representative reported that this
process has been refined and optimized over the years.
The vintage requirement for eligible appliances is 1995 or before, and while on-site JACO also
checks the age of the unit. If the unit is determined to not meet the eligibility requirements,
JACO still takes the unit to ensure good customer service. This policy has worked well for Avista
and JACO in managing customer satisfaction.
1.1.1.2 Program Challenges
Avista's Appliance Recycling program ceased to be cost-effective, which prompted Avista to
discontinue the program in June 2015. The JACO representative with whom we spoke provided
several suggestions on what Avista could have done to improve program cost-effectiveness: 1)
reducing or eliminating the incentive; 2) relying more on in-house marketing such as bill inserts
to manage marketing costs; and 3) processing, not destroying, CFC11 foam (destroying is
costly).
The JACO representative also noted that the program was not been able to achieve its goals. In
the last 3 to 4 years, JACO had a target of recycling about 1,500 units. JACO recycled around
1,100 units in 2014 and expected to recycle close to 1,100 units in 2015. There was not enough
budget to commit to the recycling volume Avista wanted to achieve. The JACO representative
further noted that Avista committed about 60% of the marketing budget that was needed to
achieve the established goals. JACO stated that they optimized this budget by identifying the
areas with likely higher participation rates, while ensuring that other areas were still being
served. Simple Steps, Smart Savings Midstream Program
This section presents results from the program implementer (CLEAResult) and Avista manager
of the Simple Steps, Smart Savings Program. CLEAResult and Avista manager were asked
about the program efficiencies, challenges they face during program implementation, and
recommendations for future program opportunities.
6.1.2.1 Program Efficiencies
The Simple Steps, Smart Savings program is Bonneville Power Administration's (BPA's)
regional promotion designed to increase adoption of various energy efficient technologies, such
as compact fluorescent lamps (CFLs), light emitting diode bulbs (LEDs), light fixtures, and
energy-saving showerheads. The program includes four delivery components: retail, direct
t-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 77
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 85 of 151
6 RESIDENTIAL PROCESS RES UL TS
install, direct mail, and bulk purchase. Avista participates only in the retail component of the
program and CLEAResult implements this program for Avista. Avista's staff explained that
Avista allocates funds for this program because it is easy to administer and achieves energy
savings. Staff explained:
Generally, we run this program because of the savings. It is a low touch with the
implementer, not a lot of time on our end to implement. Easy to get those savings.
Avista's staff did not report any communication issues with CLEAResult or BPA related to the
program. Likewise, CLEAResult also did not report any communication challenges with Avista.
Avista's staff communicates with CLEAResult once a month, when CLEAResult sends Avista a
monthly invoice. The invoice includes sales data, savings associated with sold products, and a
report noting services rendered by CLEAResult (for example, the number of store visits).35
Additional communication occurs during contract renewal phase, special product promotions,
and when CLEAResult forecasts sales by product category once a year. Avista's
communication with BPA is infrequent. There is a monthly conference call with BPA's program
manager, who provides program updates and facilities discussion about the program.
Avista's and CLEAResult's experience with Simple Steps, Smart Savings program indicates that
the program is delivered efficiently to the market. CLEAResult 1) recruits and negotiates
contracts with retailers and/or manufactures; 2) interacts with retailers to communicate program
updates and requirements as well as provide point-of-sale (POS) materials; and 3) conducts
quality control (QC) checks to verify pricing, POS materials (if present), and products (if on the
shelf). Avista conducts QC checks every quarter. The CLEAResult representative reports that
nearly all major retailers participate in the program, and the program is helping retailers sell
more efficient products. Both Avista staff and the CLEAResult representative note that
discounted products are found on store shelves, and the pricing has nearly always been correct.
6.1.2.2 Program Challenges
The challenges identified through the interviews relate to sales data reporting and POS
materials. The CLEAResult representative reported that smaller retailers have difficulty
providing sales data to CLEAResult because they lack a sophisticated reporting system that
larger retailers typically have. Avista's staff noted that different retailers have different rules on
what they will display on the shelf. When no POS materials are found on the shelf (it is unclear
how often this occurs), customers will not be able to learn of Avista's discount, which can
translate into higher free-ridership. Avista staff noted working with CLEAResult to ensure POS
materials are displayed in all the stores.
35 A · · d . h h vista may also receive a ocument noting any c anges tot e measures.
t-'1 Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 78
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 86 of 151
6 RESIDENTIAL PROCESS RES UL TS
6.1.2.3 Program Evolution and Future Opportunities
Although the Simple Steps, Smart Savings program functions well, it has changed recently to
meet the needs of BPA and the participating utilities. Specifically, in 2015, BPA no longer pays
for non-participating utility savings. Instead, non-participating utility savings are distributed
proportionally to participating utilities based on their share of the savings from purchases during
that fiscal year.36 Stated differently, most of the savings from the stores in Avista territory are
shared between Avista and other nearby public utilities. For example, if Avista wanted to
support a store whose Avista-related sales account for 60% of the store's total qualifying sales,
then someone else would have to pay for the remaining 40% of the sales. Before, BPA would
step in and pay for the 40% if no other utility wanted to cover the 40%. Now, BPA no longer
pays for the 40%. Participating utilities buy savings from Simple Steps, Smart Savings at a cost
that covers both their participation savings and a proportionate amount of non-participant
savings. In addition, because incentives are no longer fixed, CLEAResult, as explained by their
representative, is authorized to reduce the incentives for a product to mitigate the cost of non­
participating utility savings in a store.
The CLEAResult representative listed several technologies that Avista should consider if they
wanted to add measures to the program: advanced power strips, new lighting controls, water
heaters, and ductless heat pumps. The representative also emphasized that Avista should
continue with special promotions where higher incentives are promoted for a limited period.
Retailers like the limited-term promotions, and these promotions can drive sales. The
representative also commented on CFLs. He noted CFLs have not saturated the market and are
still an opportunity for utilities because they are cheaper than LEDs. CLEAResult, through their
direct install program, has observed three CFLs, on average, in the homes with typically 20-30
sockets.
6.1.3 Home Energy Reports Behavior Program
This section presents results from the program implementer (Opower) and Avista manager of
the HER Program. Opower and Avista program staff were asked about the program
performance, customization opportunities, challenges they face during program implementation,
and recommendations for future program opportunities.
6.1.3.1 Program Administration and Performance
Avista has contracted with Opower to deliver Home Energy Reports (HERs) for about three
years, starting in 2013. As part of the agreement, Opower is expected to mail the HERs to
participating Avista customers once per month for three months, and then once every two
months after that. This is an opt-out program; customers who have been randomly assigned into
36 BPA allocates savings to Avista by using the Regional Sales Allocation Tool (RSA T). RSAT identifies the amount of savings
that Avista and other utilities can expect to receive from stores that are in their territories, and that participate in the program.
L-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 79
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 87 of 151
6 RESIDENTIAL PROCESS RES UL TS
the group receiving HERs (the treatment group) and have not opted out participate in this
program. Avista provides Opower with contact information for participating customers, and
Opower manages the program data and analytics; Avista conducts follow-up quality control
checks on the customer data provided to Opower.
Avista staff reported expecting 1 % to 3% savings per year from this program, and the program
achieved -2% savings across 2014 and 2015. Further, there is evidence that the Avista
promotions described in the HERs have engaged customers. Avista staff reported that there are
typically 5-6 reports per year, and two of these reports include an Avista promotion for electric to
gas conversions or active rebate programs. Due to issues with the transition from one customer
database in 2014 to a new one in 2015, only two reports were sent out in 2015. A prior
evaluation documented an increased rate of participation in fuel conversion programs among
those in the HER treatment group compared to the control group.37 The current evaluation
showed that HERs plus rebate combination appears to act as the Multiplier Effect38, amplifying
savings, perhaps because HERs are influencing the type and number of rebate programs that
customers participate in or additional energy saving behaviors customers are undertaking in
their homes (for more detail on this analysis, see Section 7.2).
6.1.3.2 HER Customization
Presently, there is limited ability to customize the HERs, according to Avista staff, but that will
change since Opower is re-designing their reports at this time to make them more customizable.
The old 2015 reports are customizable, but the new report design, which Opower is working on,
will open more space in the report for customized content. An Opower representative noted that
the new re-designed reports will incorporate old non-customizable components (some of those
elements will be shortened) and allow for more space in the report for utility rebranding and
promotional offers. The old or 2015 report design includes four main components:
• Neighbor comparisons (Not customizable; comparing 100 similar-sized homes or homes
with similar attributes)
• Personal comparison (Not customizable; compares customer usage to the usage in the
same period last year)
• Tips (Customizable; Avista can add tips to the library, populate tips with rebate
information, or add a rebate graphic or a website address)
• Optional marketing module (Customizable; Opower can design this module in any way
for Avista to promote an offer)
37 Cadmus (2014). Avista 2013 Idaho Electric Impact Evaluation Report and Avista 2012-2013 Washington Electric Impact
Evaluation Report.
38 Multiplier effect occurs when a change in one variable leads to a much larger change in another variable.
L-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 80
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 88 of 151
6 RESIDENTIAL PROCESS RES UL TS
6.1.3.3 Program Challenges
The program faced a major delivery challenge when Avista changed its customer billing system
in January/February 2015. For about six months after the change, Opower did not receive the
necessary customer data from Avista that it needed to mail the HERs. Avista resolved the data
issue by June 2015, after which Opower continued sending HERs to participating customers.
Other challenges experienced relate to the eligibility criteria for this program. Initially, Avista
wanted to target high energy users. However, Avista did not have enough of these types of
customers because Opower needed about 100 homes within 100 miles radius with similar load
curves for each target customer to set up a comparison group. Avista also had to exclude
homes where usage was seasonal such as vacation homes. Thus, Avista staff decided to use a
lower minimum energy usage threshold for this program than they initially expected. The final
criterion that was established was 12,000 kWh/year or more in Washington and 8,000 kWh/year
or more in Idaho.
6.1.3.4 Future Program Opportunities
The Opower representative provided several suggestions on ways to expand the program:
• Opower suggested adding a monthly email report option for customers. If an email is
sent, there are live links that could link to promotions for the rebate programs on Avista's
web-based customer porta l. In another utility territory, Opower saw a 45% open rate on
an email HER and 8% click-through rate. Opower also reported seeing an increase in
savings with the email-based HER option.
• The Opower representative also suggested a high bill alert option, in which Opower can
send customers high bill alert notices to customers whose bills are projected to be higher
than expected. In other territories, Opower saw a 61 % open rate for these types of alerts
and a click-through rate of 21%.
• A third suggestion relates to Opower's "points and rewards" option. With this option,
customers can collect points based on how much energy they save. The points can be
redeemed for an Amazon gift card, for example. Opower suggested this could nudge
customers to change their behavior.
• A fourth suggestion offered by Opower was to target small and medium business with
the reports. Like the consumer facing program, targeting small and medium businesses
requires a minimum number of eligible customers to implement this option effectively.
• The last suggestion offered relates to low-income customers. Opower has developed
HERs suitable for low-income households which contain tips and suggestion that are
appropriate for this group of customers.
Avista staff informed the evaluation team that they are already considering ways to broaden
participation in their consumer behavior change programs. For example, Avista staff reported
planning installations of AMI meters in Washington in 2017. AMI meters will allow Avista to
design many different types of customer engagement and/or smart-grid programs. For example,
t.-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 81
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 89 of 151
6 RESIDENTIAL PROCESS RES UL TS
Avista could use the data from these meters to send real-time usage feedback or bill alerts to
customers to their mobile devices.
6.1.4 Low-income Program
This section reports the results from interviews with CAP agencies and Avista program staff who
work on the low-income program. Overall, the CAPs have an efficient method of delivering
services to low income customers, and customers are generally satisfied with the services they
received from CAP agencies. Nevertheless, CAPs struggle to serve the low-income market
because of limited budgets and high demand for their services.
6.1.4.1 Program Administration
Avista relies on CAP agencies to deliver this program. Figure 6-5 shows the process of how
CAP agencies deliver services to low-income customers
Figure 6-5: CAPs Delivery Process to Low-Income Customers
To date, recruiting customers into the low-income program has not been difficult for the CAPs.
Most participating low-income customers come from the bill assistance programs. The CAPs
also conduct some marketing and outreach, such as bulk mailings, advertising at community
fairs, posting flyers in the libraries or food banks, or including flyers in the Avista bills. Larger
CAPs, in particular, conduct more marketing than smaller CAPs.
Verifying program eligibility goes beyond documenting the customer's income. Some CAPs will
look at the condition of the customer's home; if it is in a bad shape (a roof needs to be replaced,
for example), then the CAPs may reject the applicant because the program funding generally
cannot cover non-weatherization repairs that exceed the amount of budget allowed for such
repairs. 39 Some CAPs also prioritize applicants based on their energy usage or if there are
elderly or children in the home. This prioritization enables the CAPs the flexibility to serve
customers with bigger electricity bills or other needs.
The pre-installation audit determines whether a customer is eligible for services . The auditors
examine energy usage in a home and identify any major repair issues as well as measures that
the program could install. For example, auditors at one CAP agency use modeling software on a
subset of homes to identify the most cost-effective measures to install, whereas another CAP
uses the audit to identify and assess which measures can be installed and subsidized by Avista
39 Avista allocates only 15% of its funds for non-weatherization measures, typically safety or health measures.
t.-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 82
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 90 of 151
6 RESIDENTIAL PROCESS RESULTS
or other funders. All CAPs use the audit information to assess customer eligibility, as discussed
previously.
The CAPs rarely outsource the installation of weatherization measures. Most (three of four) of
the interviewed CAPs have their own internal installation crews. One CAP outsources the
installation work to various contractors. The CAPs with internal crews may work with other
contractors, if there are health and safety issues to remedy or if there is no expertise to install a
certain measure. Ordinarily, the CAPs typically approve the installation of the following
measures:
• Shell upgrades (Insulation, air-sealing, etc.)
• Duct sealing
• Refrigerator replacements
• Fuel conversions
• Low-cost measures (window plastic or lighting measures)
• Health and safety measures (CO2 detector installation, asbestos, or rodent abetment,
etc.)
Some of these measures (for example, insulation) are priority measures for Avista because they
provide more energy savings and are more cost-effective. Priority measures are 100%
reimbursed, while non-priority measures are partially reimbursed.
Lastly, every project goes through a quality control (QC) inspection. QC is an important step. It
ensures CAPs catch any mistakes in the installation. CAPs use their internal staff for the QC
inspection , but they also rely on the city, county, or the state officials to inspect the work for
which contractors had to obtain the permits.
6.1.4.2 CAP Agency Interactions with Avista Staff
The CAPs communicate with Avista staff, when needed, and have reported no communication
challenges to date. All CAPs except one reported having no invoicing issues as well. (CAPs
send monthly invoices or reimbursement form to Avista, which Avista uses to track the progress
of this program.) The one CAP contact that noted an invoicing issue stated the invoicing was
complex and time-consuming. The representative explained that program staff and not agency's
accounting department had to complete the invoicing because of the dollar limitations Avista
places on measures.
6.1.4.3 CAP Agency Interactions with Participants
CAPs communicate with low-income customers from start to finish throughout the entire
participation process. CAPs also conduct surveys with their customers to gauge customer
satisfaction. Generally, customers reported being satisfied with the work done on their home,
according to the CAPs. The only negative comment CAPs have received relates to window
installations. All of the interviewed CAPs mentioned that participants want window
t-1Nexanr Process Evaluation of Avista 's 2014-2015 Energy Efficiency Programs 83
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 91 of 151
6 RESIDENTIAL PROCESS RES UL TS
replacements, but windows are not a cost-effective measure. CAPs will replace a limited
number of windows and try to explain to customers that other measures such as insulation or air
sealing will yield more energy savings than windows. However, customers have difficulty
understanding this concept.
6.1.4.4 Program Challenges
To CAPs, the main challenge is having sufficient funds to more effectively serve the low-income
market. Two CAPs noted that there is a bigger need in the market than what they can provide
with their services. The same two reported having waiting lists. One CAP noted that the working
class segment of the low-income population is underserved.40 Additionally, all CAPs report
some struggle in serving customers because budgets are limited. CAPs would like more
funding, and they are always looking to prioritize what they can afford. This is especially the
case with funding allocated for safety and health measures. One CAP mentioned constantly
fighting over those funds because they cannot weatherize a home without doing at least some
repairs. They also reported being cautious not to repair anything for which they will not be
reimbursed. Avista staff noted that previously federal funds (especially funding from the
American Recovery and Reinvestment Act or ARRA) outweighed utility funds for these
programs; today, utility-provided funds outweigh the funds from the federal sources.
CAPs noted a few additional challenges:
• Scheduling an inspection: Two CAPs noted that at times it is difficult to reach customers
to schedule an inspection. Inspection is necessary for CAPs to finalize their paperwork
and receive reimbursement.
• Difficulty in serving the low-income renter population : One CAP explained that benefits
of the weatherization work have to go to the low-income renter only. To ensure this, the
program would require landlords to not raise the rent for about 3 years or sell the
property for a certain period after work completion. If they sold, then they would have to
return some money to CAPs. Landlords are reluctant to sign-off on such requirements.
• Not covering gas measures in Idaho: One CAP has difficulty in identifying enough
qualifying customers in Idaho because Idaho funding covers electric measures only.
The main challenge noted by Avista program staff is to make this program cost-effective. Avista
staff explained that low-income projects are expensive and Avista tries to make this program as
cost-effective as possible. Additionally, Avista has found that over time the savings may be
overestimated for some homes that do not use much energy. This also affects cost­
effectiveness. Avista might have used a deemed energy savings value for a home when
40 The working class families often believe they do not qualify for CAP services because they work. Yet, a CAP can consider
helping families up to 200% above the federal poverty line with some funding streams. CAPs typically receive funding from: 1)
federal agencies (U.S. Department of Energy and U.S. Department of Health and Human Services); 2) regional organizations
(Bonneville Power Administration and Avista); and 3) state agencies (Washington Department of Commerce).
t-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 84
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 92 of 151
6 RESIDENTIAL PROCESS RES UL TS
estimating savings, but when they examined the annual usage, they have found claimed
savings to be more than the usage in certain homes. Avista caps those savings at 20% or 25%
of the usage.
6.1.4.5 Suggestions for Improvement
The CAPs provided several suggestions on how to improve the program:
• Avista could help low-income customers offer more in-depth education about saving
energy such as offering a class.
• Although acknowledging that only 15% of Avista funds are used for safety or health
measures, one CAP suggested Avista could cover more non-weatherization measures
such as plumbing leaks.
• Avista could consider funding for newer technologies, especially renewables such as
solar.
Avista staff also noted a couple of options they are considering to reach low-income customers,
such as working with tribal weatherization agencies to reach additional customers that are
typically hard to reach.
6.2 Customer Experience with Rebate Programs
To assess the residential customer experience with Avista's rebate programs during 2014 and
2015, the evaluation team compared survey results between program participants and
nonparticipants as well as between customers in Idaho and Washington. Statistically significant
differences between the states or years have been highlighted.41
This section documents the key findings from participant and nonparticipant surveys as related
to Avista's rebate programs (i.e., Shell, HVAC, Fuel Efficiency, Water Heat, and ENERGY
STAR Homes). The team also discusses findings related to Appliance Recycling program in this
section because Appliance Recycling participants received an incentive for the unit they
recycled. Topics covered include awareness and familiarity with Avista rebate programs,
motivation and barriers to participation, program experience, and attitudes toward energy use
and conservation.
Overall, the survey results suggest that Avista's marketing has been effective in increasing
customer awareness of the Avista rebate programs. For participants, in particular, contractors
were the main source of awareness of rebate offers and were influential in participants'
41 Statistical significance was determined based on differences between proportions or means at a 5% level of significance.
c.-"1 Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 85
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 93 of 151
6 RESIDENTIAL PROCESS RESULTS
decisions to participate. Both participants and nonparticipants expressed interest in learning
more about Avista's programs. Direct mail (bill inserts, for example) were identified as good
means of providing this information to customers. Additionally, participants were largely satisfied
with the programs, although this varied by program type. Furthermore, program participants did
not report any major challenges with the programs, although they expressed a desire for more
marketing and outreach about rebate offers and for clarifying program-related information about
quality assurance (QA) inspections. Aging or broken equipment and wanting to save energy or
money typically motivated participants to make energy efficient upgrades to their homes,
whereas the most commonly cited barrier to making efficient upgrades for nonparticipants was
the up-front cost of efficient upgrades or repairs. Subsections below document these findings.
6.2.1 Awareness and Familiarity with Avista Programs
The evaluation team reviewed program-related marketing materials and responses from
participant and nonparticipant surveys regarding awareness and familiarity with Avista's
programs to determine whether customers are learning of Avista's offerings through the
marketing channels used by Avista. Survey findings indicate Avista's marketing activities appear
to be effective at increasing customer awareness.
The evaluation team's review of Avista's marketing and outreach documents indicates that
Avista conducted the following marketing activities in 2014 and 2015:
• Direct mail and bill inserts;
• Print advertisements in newspapers;
• Television advertisements and newscast spots;
• Energy fairs at malls and community centers; and,
• Online digital advertisements.
The source of program awareness among customers is consistent with Avista's marketing
activities. Of the 29 nonparticipants who were aware of Avista incentives (41 % of the sample),
about half (45%) reported learning about Avista's rebate programs through channels Avista
used for outreach, such as newsletters, bill inserts, representatives, and events (Figure 6-6).
Please note that the nonparticipant sample is representative of the Avista 's residential customer
population.
'-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 86
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 94 of 151
6 RESIDENTIAL PROCESS RES UL TS
Figure 6-6: Source of Program Awareness (2015 Nonparticipants)
Direct mail (bill inserts, newsletters) 41%
Online (Avista or other website, email) -24%
Past experience with Avista programs 1111 14%
Avista events or representatives II 10%
Mass media (TV, radio, newspaper) 1 7%
Contractor 1 3%
Other 11 10%
Don't know I 3%
0% 50% 100%
Participants highlighted the importance of contractors in advertising Avista's programs.
Contractors were the main source of awareness for participants (Figure 6-7). Nearly half of the
surveyed participants indicated they first heard about Avista's programs from contractors,
whereas less than one-fifth (14%) reported first learning about the program they participated in
via channels Avista used for outreach.42
42 Participants and nonparticipants received slightly different questions. The evaluation team asked participants how participants
first heard about the Avista incentive they received (respondents provided a single response). The evaluation team asked
nonparticipants who were aware of Avista rebates, how they heard about the rebate (respondents were allowed to provide
multiple responses).
t.-'1 Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 87
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 95 of 151
6 RESIDENTIAL PROCESS RES UL TS
Figure 6-7: Source of Program Awareness (2014 and 2015 Participants)
Word-of-mouth -18%
Direct mail (bill inserts, newsletters) • 12%
Online (Avista or other website, email) 11 11 %
Mass media (TV, radio, newspaper) I 5%
Avista events or representatives I 2%
Other I 5%
Don't know I 5%
0% 50% 100%
It is difficult to gauge the relative impact of each source of program awareness just by
comparing the percentages of participants and nonparticipants that reported a source. A much
higher percentage of participants than nonparticipants cited a contractor as a source of program
awareness, but what exactly does that tell us about the relative impact of having a contractor
make someone aware of the program? How much does that increase the likelihood that
someone will become a participant?
The evaluation team developed a coefficient that better illustrates how strong the association
was between each source of awareness and program participation. For each awareness
source, the coefficient was the ratio between two percentages: 1) the percentage of participants
among those who cited a source of program awareness; and 2) the overall percentage of
participants in the population. For any given coefficient, the greater the value, the more strongly
that source of awareness predicts program participation.
Figure 6-8 shows the coefficient for each source of awareness for program participants. This
shows that awareness through a contractor was by far the greatest predictor of program
participation.43 Compared to the overall population, those who learned about the program
through a contractor are 11 times more likely to be a participant.
43 The evaluation team defined program nonparticipants as those who did not participate in 2014 or 2015, but some nonparticipants
so defined could have participated in 2013 or earlier. This likely explains why some nonparticipants identified past program
experience as their source of program awareness.
'-"Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 88
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 96 of 151
6 RESIDENTIAL PROCESS RES UL TS
Figure 6-8: Relative Association of Residential Participant Awareness with Participant
Population
Contractor/vendor 10.90
Q) 11 0.12 ~ Mass media :::,
0 U) 1 o.s1 (/) Other (/)
Q)
C
e! Online (website, email) I o.46 ~ <( I o.30 E Direct mail Ill Ol e Avista events or I 0.20 a.. representatives
Past experience 0.00
0.00 3.00 6.00 9.00 12.00 15.00
Coefficient of Assocation with Program Participation:
Participant% of Awareness Source/ Participant% of Population
Consistent with the finding that contractors are the single largest source of awareness and
information regarding Avista's programs among participants, it is not surprising that fewer than
half (46%) of participants reported being familiar with other energy efficiency rebate
opportunities from Avista (besides the program in which they had participated).
Awareness of other Avista energy efficiency rebate opportunities was highest among Water
Heat and Fuel Efficiency program participants and lowest among ENERGY STAR Homes
participants (Figure 6-9), which further suggests there may be some knowledge "gaps" among
the various contractors supporting Avista's programs regarding their awareness and familiarity
with Avista's full range of program offerings.
i-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 89
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 97 of 151
6 RESIDENTIAL PROCESS RES UL TS
Figure 6-9: Percentage of 2014 and 2015 Participants Familiar with Avista Rebates for
Other Programs
100%
"' E ~ Cl) e 75% a.
<ii .c 0 50% .c .. i
j
25% .E
"' LL
"#-
0%
Water Heater
(n=24) • Shell (n=75) Fuel Efficiency HVAC (n=116) Appliance ENERGY STAR
(n=25) Recycling (n=72) Homes (n=27)
Among the twenty-nine nonparticipants (41 % of the sample) that reported being familiar with
Avista incentives, between one-third and one-half reported being familiar with the Shell, HVAC,
Appliance Recycling, and Fuel Efficiency incentives (Table 6-2). Two surveyed nonparticipants
reported being familiar with CFL and LED store discounts offered by Avista. None reported
being familiar with Water Heater or ENERGY ST AR Homes incentives programs.
Table 6-2: Nonparticipant Awareness of Avista Incentives, (n=29; Multiple Responses
Allowed)
Incentives Familiar With Count Percent
Shell (insulation and windows) 13 45%
HVAC 11 38%
Appliance Recycling 10 35%
Fuel Efficiency (electric to gas furnace or water heater conversions) 10 35%
CFL and LED store discounts 2 7%
Other 4 14%
Don't know 3%
Interest in receiving additional information regarding Avista's energy efficiency offerings is high
among both participants and nonparticipants. About three-quarters (77%) of participants
reported being interested in receiving energy-saving information from Avista (Table 6-3).
Although still a majority, significantly fewer nonparticipants reported wanting information from
Avista. Information on energy efficiency programs and energy savings opportunities were the
most common types of information requested by respondents. However, significantly fewer
nonparticipants reported that they would like information on energy efficiency programs
compared to participants.
t.-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 90
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 98 of 151
6 RESIDENTIAL PROCESS RES UL TS
Table 6-3: Additional Energy Saving Information Requested (2014 and 2015 Participants
and 2015 Nonparticipants; Multiple Responses Allowed)
Participants (n=339) Nonparticipants (n=70)
Information regarding ...
Count Percent Count Percent
Energy efficiency programs 226 67% 37 53%
Energy savings opportunities 222 65% 41 59%
Workshops or events on energy efficiency 103 30% 22 31%
Nothing 76 22% 28 40%
Don't know 0% 0 0%
• Differences between participants and nonparticipants are statistically significant (Chi-square Test at p<0.05).
Participants and nonparticipants indicated they wanted to receive additional information from
Avista regarding energy efficiency by mail -which suggests that direct mail approaches are
good avenues to market programs. Both participants and nonparticipants who reported wanting
additional information from Avista indicated they would prefer to receive the information by mail
(78% and 90%, respectively) -primarily via a bill insert (Table 6-4). The evaluation team found
that nearly three-quarters of participants and nonparticipants reported receiving their bills in the
mail (71 % and 70%, respectively).
Table 6-4: Preferred Method of Receiving Information from Avista (2014 and 2015
Participants and 2015 Nonparticipants; Multiple Responses Allowed)
How first heard
Participants (n=262) Nonparticipants (n=42)
Count Percent Count Percent
By US mail 204 78% 38 90%
By US mail via bill insert 162 62% 26 62%
By US mail separate from bill insert 96 37% 19 45%
By e-mail 81 31% 14 33%
Avista website 28 11% 2 5%
Other 17 6% 2 5%
Nonparticipant survey findings, which are representative of the overall residential customer
base, also suggested Avista's marketing efforts were having an influence on customers. Of the
25 nonparticipants who reported making efficient upgrades to their home, over half (14
respondents) reported that Avista marketing was "very influential" in their selection of the
equipment (a rating of 4 or 5 on a five-point scale, from "no influence" to "great influence").
t..1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 91
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 99 of 151
6 RESIDENTIAL PROCESS RESULTS
6.2.2 Motivation and Barriers to Participation
Participants reported increased home comfort, saving energy, and saving money as the top
three motivations for participating in a rebate program, and they reported ease of participation
as a close fourth (Table 6-5).44 The evaluation team found that significantly more participants in
Idaho reported being motivated by a recommendation from a contractor, builder, or vendor
compared to Washington participants (70% in ID; 55% in WA; Chi-square Tests at p<0 .05).
Table 6-5: Motivations for Participating in a Rebate Program (2014 and 2015 Participants;
Multiple Responses Allowed)
Motivation Count Percent
Increase comfort of home (n=267) 235 88%
Save energy (n=339) 291 86%
Save money (n=339) 281 83%
Seemed easy to use program (n=339) 265 78%
Increase value of home (n=267) 166 62%
Reliability of equipment and service offered by Avista (n=305) 182 60%
Contractor, builder, or vendor recommended (n=267) 159 60%
Had a good experience with another Avista program (n=339) 94 28%
Other (n=339) 35 10%
Avista leverages the contractor channel to promote rebate programs. The overall participation in
the rebate programs has increased by 43% from 2014 to 2015 (see Section Table 7-1). This
increase in participation may indicate that contractors have been engaged in promotion of
Avista's rebates more so in 2015 than 2014. There is some evidence of this supposition.
Compared to 2014, there was an increase in the proportion of participants reporting being
motivated by a recommendation from a contractor, builder, or vendor to participate in a rebate
program in 2015 (53%, up from 40% in 2014).
Figure 6-10 shows that participant motivations for completing efficient upgrades to their home
vary by program type. For example, significantly more Shell participants reported participating in
the program to save energy compared to ENERYG STAR Homes, Water Heater, and Appliance
Recycling participants (Z-Test of Proportions at p<0.05). These differences suggest that
customers are participating in the various programs for different reasons, which speaks to the
importance of tailoring the marketing messages for each program.
44 This includes all rebate programs, including Appliance Recycling. The evaluation team included Appliance Recycling participants
because they also received a rebate for recycling their refrigerators or freezers .
'-'"Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 92
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 100 of 151
6 RESIDENTIAL PROCESS RES UL TS
Figure 6-10: Motivations for Participating in a Rebate Program, by Program (2014 and
2015 Participants; Multiple Responses Allowed) a
Motivation
Shell
(n=75)
Fuel Conversion
(n=25)
HVAC
(n=116)
ENERGY STAR
Homes (n=27)
Water Heater
(n=24)
Appliance
Recycling (n=72)
Sa1.e energy 97%-196% -89% -70% • 79% -74%
I
Increase comfort of home 93%-92% -91% -74% • 71 % • Sa1.e money 92%-88% -84% -74% • 83% -74%
I
Seemed easy to use program 85%-80% -66% • 74% • 67% • 94%
Increase 1alue of home 59% • 76% • 66% • 59% • 42% • Contractor, builder, or 1.endor 37%1 56% • 72% • 63% • 67% • recommended
Reliability of equipment and ser\ice 40% • 64% • 49% • 52% • 54% • 72% offered by Alista
Had a good experience with another 32% I 20% I 27% I 11% I• 50% • 26% Alista program
Other 9% I 12% I 6% I 19% I 0% 18%
• Arrows in figure represent significant differences between program types. Green, upward arrows indicate the value is
significantly higher than the values with red, downward arrows (Z-Test of Proportions at p<0.05).
b Appliance Recycling program participants were not provided with this option.
Twenty-five (36% of the sample) of nonparticipants reported completing an upgrade at their
home in the past two years. Nonparticipants reported completing a variety of upgrades,
including windows (eight mentions), insulation (seven mentions), and lighting upgrades (five
mentions). Eighteen (82%) of nonparticipants who completed an upgrade reported that at least
one of the upgrades they have made in the past two years were installations of equipment
labeled as ENERGY STAR certified or otherwise being highly energy efficient.
• NIA ' • -NIA '
NIA ' • I
I
Aging equipment was the primary motivation for replacing or upgrading equipment reported by
nonparticipants, followed by broken equipment (10 and 5 mentions, respectively). A minority
(four mentions) also noted wanting to save energy as a reason for completing efficient upgrades
to their home. Please note that the evaluation team asked nonparticipants about their reasons
for making upgrades to their home, whereas participants reported only about their motivations
for participating in a rebate program.
About one-quarter (24%) of nonparticipants reported they planned to make an efficient upgrade
to their home within the next two years. Among those respondents planning an upgrade,
window replacement was most commonly mentioned (eight mentions), followed by HVAC
equipment (four mentions) and lighting upgrades (three mentions). (Table 6-6).
t-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 93
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 101 of 151
-------------------------------------~-------····---
6 RESIDENTIAL PROCESS RESULTS
Table 6-6: Future Upgrades Planned (2015 Nonparticipants; n=70; Multiple Responses
Allowed)
Upgrades Planned Count Percent
Windows 8 11%
HVAC 4 6%
Lighting 3 4%
Insulation 2 3%
Refrigerator or freezer recycling 2 3%
Water heater 1%
Other 5 7%
Nothing 53 76%
About half (54%) of nonparticipants, reported facing at least one barrier to saving energy in their
home. The most frequently cited barrier was the up-front cost of efficient equipment or repairs
(Table 6-7), which indicates an importance of offering an incentive to customers for home
improvement projects. Nonparticipants also reported that living in a rental property prohibits
them from making improvements to their home. Further, demographic analysis revealed that
nonparticipants were significantly more likely to report being renters than participants (27% vs.
3%, respectively; Chi-square Test at p<0.05).
Table 6-7: Barriers to Making Energy Efficiency Improvements (2015 Nonparticipants;
n=38; Multiple Responses Allowed)
Barriers Count Percent
Up-front cost of equipment or repairs 16 42%
Renter -unable to make improvements 9 24%
Unspecific issues related to older/inefficient home 4 11%
Other occupants of home/ Occupant behavior 3 8%
Lack of time 2 5%
Payback period of equipment or repairs 2 5%
Other 6 16%
6.2.3 Program Experience
The following section provides a summary of participant survey findings related to satisfaction
with program elements, satisfaction with contractor interactions, and the clarity of program
information.
t.-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 94
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 102 of 151
6 RESIDENTIAL PROCESS RES UL TS
6.2.3.1 Program Satisfaction
More than four-fifths (84%) of program participants reported their overall satisfaction with their
Avista rebate program experience as being either "very" or "completely" satisfied (Figure 6-11).
The evaluation team found that overall program satisfaction has decreased for Washington
participants from 2014 to 2015 (80% "very" or "completely" satisfied, down from 89% in 2014;
marginally significant Chi-square Tests at p<0.1).
Additionally, participants reported the lowest satisfaction with the rebate amount they received
(Figure 6-11 ). Similarly, contractors reported the lowest satisfaction with the amount of
incentives provided by Avista when they rated various elements of Avista's rebate programs
(see Section 6.1 .1.2).45
Figure 6-11: Satisfaction with Program Elements (2014 and 2015 Participants)
Program
Rebate Turnaround Time
Rebate Amount
0% Percent of Respondents
• Don't know • Not at all • Slightly • Moderately • Very • Completely satisfied
Figure 6-12 shows that Shell, HVAC, and Fuel Conversion participants are generally more
satisfied with their program experience than Appliance Recycling, Water Heater, and ENERGY
STAR Homes participants. For example, Shell, HVAC, and Fuel Conversion participants
reported significantly higher satisfaction ratings compared to Water Heater participants (Z-Test
of Proportions at p<0.05).
45 The evaluation team has seen across many evaluations that program participants and contractors often report wanting higher
incentives. Higher incentives allow participants to offset more of the incremental cost and contractors to sell more jobs.
100%
t.-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 95
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 103 of 151
6 RESIDENTIAL PROCESS RES UL TS
Figure 6-12: Satisfaction Rating, by Program (2014 and 2015 Participants) a,b
"Very" or "Completely Shell (n=41) HVAC(n=116) Fuel Conversion Appliance Water Heater ENERGY STAR
Satisfied" with the ... (n=25) Recycling (n=72) (n=24) Homes (n=27)
Program 93% 92% 88%. 76% 59%
Rebate Pmount 71% 69% 80% 72% 63%
Rebate Turnaround Time 80% I • 86% 80% 61% 63%
I
• Percent reporting "Very" or "Completely Satisfied" on a 5-pt. scale (not at all, slightly, moderately, very, and completely
satisfied).
b Arrows in the figure represent significant differences between program types. Green, upward arrows indicate the value is
significantly higher than the values with red, downward arrows (Z-Test of Proportions at p<0.05).
c Only significantly higher than Water Heating and ENERGY STAR Homes participants (Z-Test of Proportions at p<0.05).
d Only significantly higher than Appliance Recycling participants (Z-Test of Proportions at p<0.05).
Based on interviews with JACO staff, Avista's appliance recycling implementation contractor,
most complaints regarding the Appliance Recycling program relate to appliance pick-up
difficulties during inclement weather, delays in the customer verification process, and incentive
check delays. The evaluation team had no additional information on complaints by participants
in the Water Heater or ENERGY STAR Homes programs, two other groups that exhibited lower
satisfaction.
One hundred respondents offered suggestions for improving the Avista rebate programs (Table
6-8).46 About two-fifths (39%) of these respondents felt that more or better program information
through marketing and program materials would improve the programs. However, respondents
did not provide more specifics regarding the types of materials or messaging that they would
like to see.
46 One-hundred and fifty-seven respondents said "Do not know" when asked to provide suggestions for improving the rebate
program.
'-"Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 96
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 104 of 151
6 RESIDENTIAL PROCESS RESULTS
Table 6-8: Suggestions for Improving the Rebate Program (2014 and 2015 Participants;
n=100; Multiple Responses Allowed) a
Suggestion Count Percent
More program outreach and advertising 37 37%
Higher rebate 18 18%
Communication improvements/Confusion with program requirements 11 11%
Process is too slow -increase speed 11 11%
Improvements to application process 10 10%
Offer additional incentives/Financial assistance 6 6%
Other 12 12%
• Includes all 2014 and 2015 respondents saying they have suggestions on how to improve the rebate program; 157 said
they did not know and 48 said the program is working well with no need for improvement. Thirty-four participants in the
direct install duct-sealing program were excluded as they did not receive a rebate.
6.2.3.2 Participant's Satisfaction with Contractors
Nearly all (91%) of the surveyed Fuel Efficiency, HVAC, Shell, and Water Heater participants
used a contractor to install the measure. About four-fifths (83%) of Water Heater participants
reported using a contractor, compared to 89% of Shell, 92% of HVAC, and all of Fuel Efficiency
program participants. The majority (88%) of these participants reported being satisfied with their
contractors (rating of "Very" or "Completely Satisfied" on a 5-pt. scale). The evaluation team
found participation satisfaction with their contractor increased significantly between 2014 and
2015 (92%, up from 83% in 2014; Chi-square Tests at p<0.05). Almost all (93%) of those who
used a contractor reported they would recommend their contractor to other people.
6.2.3.3 Clarity of the Program Information
A majority of participants reported that program-related information (e.g., website or rebate
form) was clear on how to apply for a rebate, which equipment qualified for a rebate, expected
energy savings of program eligible equipment, and who to contact if any issues arose (Figure
6-13). Significantly fewer Washington participants reported the expected energy savings claims
were clear in program collateral compared to Idaho participants (59% vs. 72%, respectively;
Chi-square Tests at p<0.05), although it is unclear whether the program materials, in fact, differ
by state. Figure 6-13 also shows that for Shell program participants, the program materials were
less clear about the quality assurance (QA) process. Additionally, the evaluation team found
that the clarity of information regarding which equipment or items qualified for rebates was less
clear for Shell participants than for other program participants (70%, compared to 90% for Water
Heater, 83% for ENERGY STAR Homes and Fuel Efficiency, and 80% for HVAC and Appliance
Recycling participants; Chi-square Tests at p<0.05).
t-'1 Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 97
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 105 of 151
6 RESIDENTIAL PROCESS RESULTS
Figure 6-13: Clarity of the Program Information by State across 2014 and 2015 (2014 and
2015 Participants) a
How to apply for Avista rebates (ID n=82, WA n=224)
Which eq. or items qualify for rebates (ID n=BO, WA n=223)
Expected energy savings from eligible eq. or items (ID n=78, WA n=208) b
How to follow up with program staff if there are questions (ID n=79, WA n=219)
That there may be an inspection prior to receiving a rebate (ID n=7, WA n=50) c
Idaho
I 79% ---'
II
Washington
78% ----
• Percent saying "4" or "5" on a 5-pt scale where 1 meant "the information was not at all clear" 5 meant "the information
was very clear." The evaluation team excluded "not applicable" from this analysis.
b Difference between Idaho and Washington statistical significant (Chi-square Tests at p<0.05).
c Only Shell participants were asked this question.
6.2.4 Attitudes toward Energy Use and Conservation
Participants and nonparticipants rated their agreement with eight statements designed to
measure respondents' attitudes towards adopting energy efficient behaviors. The statements
asked about intention to conserve, concern about environment or cost of energy, among others.
The evaluation team relied on the previous research, specifically the Awareness-Knowledge­
Attitude-Behavior (akAB) model of change, to develop these statements. The akAB model is
grounded in years of social science research on how individuals make energy conservation and
efficiency choices, as well as "green" choices more generally. It includes five stages of energy­
efficient behavior change: awareness/knowledge, concern, ascription of responsibility, intention
to conserve, and maintaining the behavior.47 The participant and nonparticipant surveys only
included statements on intention to conserve, ascription of responsibility, and concern .
Overall, respondents reported highest agreement (providing a 4 or 5 on a scale 1 "not at all
agree" to 5 "completely agree") that they intend to conserve electricity in their home and that it is
their responsibility to use less energy to help the environment (Figure 6-14). Although
participants and nonparticipants differed in responses on several metrics, differences were not
statistically significant, suggesting that participants do not differ from nonparticipants in relation
to how they think about the energy saving concepts noted in the figure below.
47 For more information, see the following study: PG&E and SCE. 2011 -2012 General Households Population Study in California,
http:llwww.calmac.org/publicationslGPS_Report_OB302012_FINALES.pdf
'-1Nexanr Process Evaluation of Avista 's 2014-2015 Energy Efficiency Programs 98
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 106 of 151
6 RESIDENTIAL PROCESS RESULTS
Figure 6-14: Agreement with Eight Statements Associated with Energy Usage and
Conservation a
79% 79% 77%
69%
60% 60%
52% 53% 59%
47%
27%
I intend to It is my I am very If my utility bill
conserve on responsibility to concerned about goes up, I feel
electricity use as little how energy use like I must do
consumption in energy as affects the something to
my home possible to help environment reduce it
the environment
I often worry that Conserving
the costs of electricity will
energy for my help reduce
home will global warming
increase
I feel guilty if I
use too much
energy
I sometimes
worry whether
there is enough
money to pay
my energy bill
• Participants (n=339) • Nonparticipants (n=70)
a Respondents rated their agreement with each statement on a five-point scale with 1 being "not at all agree" and 5
being "completely agree."
6.3 Customer Experience with Simple Steps, Smart
Savings Program
This section provides findings regarding customers' experience with the Simple Steps, Smart
Savings midstream program. Simple Steps, Smart Savings is BPA's regional promotion
designed to increase adoption of various energy efficient products, including CFLs, LEDs, light
fixtures, and energy-saving showerheads. The program discounts the following measures at
retail locations: standard and specialty CFLs, LED bulbs and fixtures, and low-flow
showerheads.
The evaluation team asked both rebate program participants and nonparticipants a series of
questions to determine: 1) the incidence rate of purchasing a CFL, LED, or a showerhead; 2)
the usefulness of in store point-of-purchase (POP) materials to buyers; and 3) their awareness
of the Simple Steps, Smart Savings program. By design, the nonparticipant sample is more
representative of customers in Avista's territory than the participant sample and thus provides a
more accurate representation of customer experience with the Simple Steps, Smart Savings
program. The participant sample consists only of a subset of Avista's customers (those who
participated in Avista's rebate programs), whereas the nonparticipant sample was drawn from
the entire Avista customer database and was designed to be representative of the state and
t..1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 99
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 107 of 151
6 RESIDENTIAL PROCESS RES UL TS
urban/rural population.48 To provide results that are more representative of Avista's customer
population, the evaluation team only presents findings from the nonparticipant survey in this
section.
Most (71 %) nonparticipants reported purchasing at least one product referenced above in 2015.
Among respondents who purchased CFLs, LEDs, or showerheads, most (78%) reported
purchasing standard CFL bulbs, followed by LED fixtures (34%), and low-flow shower heads
(26%; Table 6-9).
Table 6-9: Purchases of CFLs, LEDs, or Showerheads in 2015 (2015 Nonparticipants;
n=50; Multiple Response Allowed) a
Measure Count Percent Average number purchased
Standard CFL bulbs 39 78% 12
LED fixtures 17 34% 6
Low-flow showerheads 13 26%
Specialty CFL bulbs 7 14% 2
a The evaluation team did not ask nonparticipants about LED bulbs because they were not added to the Simple
Steps, Smart Savings program until July of 2015.
Figure 6-15 shows that a large majority of nonparticipants reported they were easily able to find
CFLs, LEDs, and low-flow showerheads at the stores where they commonly buy these products
(providing a rating of 4 or 5 on a five-point scale with 1 being "not at all easy" and 5 being "very
easy").
48 Participants are more likely to be urban dwellers than nonparticipants. Additionally, participants were more likely to be
homeowners and have higher incomes.
t-'1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 100
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 108 of 151
6 RESIDENTIAL PROCESS RES UL TS
Figure 6-15: Ease of Finding Lighting and Low-flow Showerheads (2015 Nonparticipants)a
Standard or Specialty CFLs (n=40) 90%
Low-flow showerheads (n=13) 85%
LED fixtures (n=17) 82%
0% 100%
Percent Reporting Easily Found (4 or 5)
a Respondents rated the ease of find ing the products on a five-point scale with 1 being "not at all easy" and 5 being
"very easy."
Findings also suggest that some of the products purchased by nonparticipants were program­
discounted measures. Nonparticipants who purchased CLFs, LED fixtures, or showerheads
reported whether they recalled seeing the Simple Steps, Smart Savings point-of-purchase
(POP) materials where they were shopping for these products. About one-quarter (12 of 50)
reported seeing the POP materials, of these, five reported recalling the product they purchased
was part of the Simple Steps, Smart Savings program (i.e., the product was discounted). In
comparison, more than two-fifths (44%) of rebate participants who purchased a CFL, LED, or
showerhead reported recalling the product they purchased was part of the Simple Steps, Smart
Savings program. This finding suggests that rebate participants may pay greater attention to
POP materials (either due to greater brand awareness or they are more likely to be looking for
discounts) when making these purchases than the general customer population.
6.4 Customer Experience with the Behavior Program
The evaluation team asked participants and nonparticipants a series of questions regarding the
Home Energy Reports they receive from Opower to determine their usefulness and impact. The
evaluation team found that there is some confusion among respondents as to whether they
received a HER. Slightly less than one-third of participants and nonparticipants reported
receiving a HER from Avista in 2014 (28% and 30%, respectively). However, after reviewing
program data, the evaluation team determined that fewer than one in ten (9% of participants
and 6% of nonparticipants) respondents surveyed actually received a HER from Avista in 2014
or 2015.49 It is possible that the respondents who incorrectly reported receiving a HER were
49 To determine who received a HER, the evaluation team matched participant and nonparticipant IDs with those in the HER
treatment group.
t-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 101
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 109 of 151
6 RESIDENTIAL PROCESS RES UL TS
referring to the energy saving information they received through their monthly online or paper
bill rather than the HER.
The overall recall rate among those who the evaluation team confirmed received a HER (n=29)
was high and consistent with findings from other data sources. About four-fifths of the 36
participants and nonparticipants who received a HER, correctly reported receiving a HER (78%
and 100%, respectively). The remaining respondents reported either they did not know if they
received (four mentions) or that they did not receive (two mentions) a HER. The recall rate is
consistent with a 2014 study conducted by MDC Research, which found about four-fifths (78%
unaided and 81% aided) of Avista customers who received a HER recalled receiving it.50
There is evidence that HERs are engaging customers to save energy in their homes. Among
those 29 participants and nonparticipants who reported and who actually received a HER, all
but two reported they "usually" or "always" read the report. Of the remaining respondents, one
reported reading the HER once or twice and one reported never reading the HER. Additionally,
about two-thirds (64%) of respondents who actually received and read their HER reported
taking action to save energy in response to the reports. Participants reported taking various
energy-saving actions, including: making unspecific energy saving modifications to their home,
adjusting how or when they use energy (eight mentions each), purchasing energy saving
products and receiving Avista rebates (six mentions), purchasing energy saving products and
not receiving Avista rebates, and looking for additional information on how to save energy (two
mentions each).
Participants and nonparticipants who correctly reported receiving and who read their HERs
reported varying levels of satisfaction and usefulness of the reports. Of the 28 participants who
confirmed they received and read their HERs, over half (58%) reported they were "very" or
"completely" satisfied with the report (Figure 6-16). Similarly, about two-fifths (40%) reported
finding the HER to be "very" or "completely" useful in helping them to better understand their
home's energy use.
50 MDC Research. Avista Energy Usage Communications Research Presentation. June 2014.
t-'1 Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 102
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 11 O of 151
6 RESIDENTIAL PROCESS RESULTS
Figure 6-16: Usefulness and Satisfaction with HER (2014 and 2015 Participants and 2015
Nonparticipants; n=28) a
Satisfaction 11% 29% 29%
. .
Usefulness 11% . '.: ·. · 29% . 11%
=} •
0%
•Not at all •Slightly • Moderately •Very • Completely
• Note: this analysis excludes one respondent who reported never reading the HER the received.
Twelve respondents provided additional comments regarding the HERs they received.
Comments included: being concerned with the accuracy of the HER (four mentions), wanting
more information and tips (four mentions), not understanding the comparisons between their
home's energy use and others (three mentions), and finding the HER interesting and easy to
understand (one mention).
6.5 Freeridership and Spillover
This section summarizes results about freeridership and spillover, two key aspects of energy
efficiency programs. Freeridership represents an estimate of the energy savings that the
program participants would have achieved without the program's assistance, and spillover is
what additional energy saving actions occurred outside the program but as a result of program
influence. This section begins with a discussion of freeridership and concludes with a discussion
of spillover. For a discussion of the methods used to calculate freeridership and spillover values,
see the 2014-2015 impact report discussion about net-to-gross calculations. Additionally, the
impact report covers how freeridership and spillover rates effect savings.
6.5.1 Freeridership
The evaluation team examined freeridership for five program types: appliances, HVAC and
Water Heat, Fuel Conversion, Weatherization and Shell, and ENERGY STAR homes. To see
how freeridership changed over time, the evaluation team plotted freeridership results for
PY2014 and PY2015 next to results from the previous evaluation dating back as far as 2010.
Fuel conversion freeridership scores were available back to 2012 and there were no reported
freeridership values for ENERGY STAR Homes in the previous evaluation.
t..1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 103
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 111 of 151
100%
6 RESIDENTIAL PROCESS RESULTS
Figure 6-17 shows freeridership values for active programs51 and shows, on average, slightly
lower rates for HVAC/Water Heat and Weatherization/Shell measures and a considerably lower
freeridership rate for Fuel Conversion, compared to the 2013 evaluation.
100%
80%
60%
40%
20%
0%
;::::-
<D
II .s
0 ,-<
0 N
;:;;-
LI'> M
II
M M 0 N
Figure 6-17: Freeridership Over Time*
67%
" ,..,
M
II
V1 (1) (cl
N M
0 N
.s u ·.:
ti a,
UJ
N M 0 N
68%
N
00
,..,
M 0 N
72%
.s u ·.: ti ~ u.J ,.., .....
0 N
HVAC and Water Heat
58%
LI'> M 0
N
oil
,;j'
M 0 N
54%
.-I
00
.s u E u ~ u.J
LI'> M 0 N
oil
,;j'
.-I 0 N
63% 62%
N .....
0 N
;::::-,..,
II .s u ·5 u a,
UJ ,.., .....
0 N
LI'>
N
II
C
u E u QJ
UJ
LI'> ..... 0 N
oil
,;j'
M 0 N
Conversion
45%
" <D
.s
0 M 0 N
;::::-,..,
.s
,-<
M 0 N
50%
" 00
V1 (1)
(cl
N M 0 N
37%
;:;;-.....
II
N ..... 0 N
56% 55%
,..,
M 0 N
rl .....
II
,.., ..... 0 N
Wx and Shell
49%
LI'> .-I 0 N
oil
,;j' .....
0 N
44%
00 N
II
C
u E u a,
UJ
LI'> M 0 N
oil
,;j'
,-<
0 N
53%
V1 (1)
(cl
LI'> M 0 N
oil
,;j' .-I
0 N
67%
w M
II .s
u E u ~
u.J
LI'> .-I 0 N
oil
,;j'
,-<
0 N
ENERGY
STAR
Homes
*Orange bars reflect values calculated by the evaluation team. Blue bars are values reported in previous
process evaluation (Avista 2012-2013 Process Evaluation Report, May 15, 2014, prepared by Cadmus
Inc.)
The previous evaluation attributed the general upward trend in freeridership, seen from 2010 to
2013 across HVAC/Water Heat and Weatherization/Shell, to the influence the program is having
on the market. The evaluation team agrees that the program could have influenced the market
51 Appliance recycling is not included here because it was discontinued in 2015.
L-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 104
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 112 of 151
6 RESIDENTIAL PROCESS RES UL TS
and that influence could have affected freeridership rates. Some of the differences seen in
freeridership scores between 2014 & 2015 values and prior analyses may be a result of different
methodologies used to calculate freeridership.
The Fuel Conversion program values noticeably differ from the other programs, however. The
evaluation team hypothesizes that the sharp drop in freeridership from 62% to 30% for the Fuel
Conversion program from 2013 to 2014 & 2015 may be a result of the distribution of low-income
participants in each program year. If in 2014 & 2015 there was a high participation rate among
low-income customers, that may drive freeridership values lower as low-income participants are
likely to be low free-riders. Conversely, if there were relatively few low-income participants in
2012 and 2013 that could increase freeridership values.
Another hypothesis related to the decline in freeridership in the Fuel Conversion program
relates to price of natural gas over the last six years. The decline of natural gas prices from
2008 to 201552 may have driven participants to convert to gas during the years the prices
decreased most notably, 2009-2013. As the price of gas plateaued in 2014 to 2015,
customers may feel less inclined to convert to gas, thus lowering freeridership.
6.5.2 Participant Spillover
Participant spillover occurs when program participants elect to conduct energy saving activities
outside of the program as a result of program influence. Because the actions took place outside
of the program, the program has no mechanism to capture these actions other than during
process surveys. The analysis below shows that 3% of weatherization/shell participants and 1 %
of HVAC/Water Heat participants reported they took a spillover action (Table 6-10). Other
program participants reported no spillover.
Table 6-10: Number of Participants Reporting a Spillover Action
Program Total Participants in Participants Who Did Percent of Participants
Sample Spillover Project Who Did Spillover Project
Weatherization and Shell 75 2 3%
HVAC and Water Heat 140 2 1%
All other programs53 52 0 0%
TOTAL 267 4 1%
For an analysis and discussion of what effect these actions had on savings, see the impact
report.
52 Energy Information Administration, Natural Gas Prices. https://www.eia.gov/dnav/ng/hist/n3010us3a.htm (Accessed on April 22,
2016)
53 Appliance recycling participants were excluded from the table because that program was discontinued in 2015.
t.-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 105
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 113 of 151
6 RESIDENTIAL PROCESS RESULTS
7 Special Studies
7.1 Declining Program Participation Rates
The 2012-2013 process evaluation report54 noted that program participation rates, based on the
number of rebated measures, have declined since 2010. The 2012-2013 process evaluation
report also suggested that explanations for the decline in participation included a decrease in
the list of rebated measures and a reduction in the incentive amounts that Avista offered in
response to declining measurable gross savings or higher freeridership. To investigate this
issue further, the evaluation team examined the list of rebated measures in both the
nonresidential and residential 2010-2015 program databases to assess the potential impact that
the reduction in the rebated measures list and the reduced incentive amounts had on
participation. The evaluation team also examined whether a decrease in repeat participation
may have partly contributed to the decline in participation. Finally, specifically for the residential
sector, the evaluation team examined whether evidence exists that the availability of qualifying
measures may have changed from 2010 to 2015, possibly contributing to the decline in
participation.
7.1.1 Nonresidential Participation Trends
7.1.1.1 Discontinued Measures and Reduced Rebate Incentives
For the analysis of discontinued program measures and reduced incentives the evaluation team
combined information from the 2010 to 2015 program databases. The combined 2010-2015
nonresidential program database contained 13,845 rebated measures. The evaluation team
focused on analyzing the prescriptive and Energy Smart Grocer rebated measures only
because these measures combined accounted for 91% of all the measures in the combined
2010-2015 database, respectively. The evaluation team excluded the Site Specific measures
from this analysis because this program provides custom incentives based on measured energy
savings, and so there is no standardized unit of analysis. The evaluation team excluded Oregon
measures because this evaluation focuses on Idaho and Washington and excluded measures
classified under "UCON MF" because of limited data.
54 Cadmus (2014). Avista 2012-2013 Process Evaluation Report.
t.-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 106
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 114 of 151
6 RESIDENTIAL PROCESS RES UL TS
Also note that we combined prescriptive and Energy Smart Grocer rebated measures, when
reporting findings. 55
The overall number of rebates declined in 2013, 2014, and 2015. The rate of decline slowed
down in 2015 (Figure 7-1). Lighting rebates, in particular, abruptly increased in 2012 and
declined substantially in subsequent years. Industrial process rebates started to decline in 2012
and continued declining in subsequent years. These lighting and industrial process measures
accounted for 73% of all the rebated measures examined in the 2010-2015 data.
Figure 7-1: Reported Number of Nonresidential Rebates, 2010-2015 Nonresidential Program
Data
67% 80% L. ro
60% QJ >-6
40% ! ro QJ 20% >-
QJ
tlO 0% C ro ..c
-20% u ... C
-25% -40% QJ u L. QJ
VI 4500
QJ ... 4000 ro ..c QJ 3500 a:
'O QJ 3000 t:
0 2500 C.
QJ a: .... 2000 0
L. 1500 QJ ..c
E 1000 ::i
-60% a.. z 500
0 -80%
2010 2011 2012 2013 2014 2015
-Lighting Interior -Lighting Exterior -Industrial Process
-Other -Percent Change Year-to-Year
The quantity of interior and exterior lighting rebates peaked in 2012 and declined by 55% and
37%, respectively, from 2012 to 2013 (Figure 7-2).
55 The Energy Smart Grocer measures accounted for a small proportion of all the measures in the database -less than 15%.
Additionally. the team struggled in separating Energy Smart Grocer measures from the same measures in the prescriptive program
in the database.
t-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 107
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 115 of 151
6 RESIDENTIAL PROCESS RES UL TS
Figure 7-2: Percent Change Year-to-Year by Measure Rebate Type, 2010-2015
Nonresidential Program Data*
500% ... 0
cii 400% ..0 E
:J z
QJ 300% ..r:. ..... V) C: QJ ..... ... C1l C1l ..0 200% QJ QJ >-c,:: 0 "'C +;' QJ ... t 100% C1l 0 QJ >-C. QJ
QJ c,:: t:ll)
C: 0% C1l ..r:. u ..... C: -100% QJ u ... QJ c..
-200%
2010 2011
250%
--26%
2012
463%
2013
-7%
2014
-83%
• Lighting Interior • Lighting Exterior • Industrial Process • Other
2015
10% --15%
-15%
* The percentage shown above or below each column represents the percentage change in rebates that year
relative to the previous year.
The abrupt increase in lighting rebates in 2012 was most likely related to changes in linear
fluorescent lamp standards. Effective July 14, 2012, all linear florescent lamps manufactured or
imported for sale in the U.S. had to meet more stringent lighting standards as stipulated by the
Energy Policy Act (EPACT) of 2005 and Independence and Security Act (EISA) of 2007. This
resulted in the cessation of U.S. production and importation of T12 fluorescent lamps after July,
2012. Likely in response to this new standard, which effectively shifted the baseline for
commercial lighting technologies, Avista changed the rebate amounts for lighting measures.
Avista's average rebate amounts per BTU56 saved decreased from 2012 to 2015 for lighting
measures (Table 7-1 ; this data comes from the 2010-2015 program database). Nonresidential
customers and contractors may have anticipated this reduction in rebate amounts by Avista
after 2012, which could explain the abrupt increase in the quantity of the lighting upgrades
through the Avista's programs in 2012.
56 BTU= British Thermal Unit. Many records in the database included both electric (kWh) and gas (Therm) savings. To estimate
total (electric+gas) savings, the evaluation team converted kWh and Therm savings for each record to BTUs, a traditional unit of
energy.
'-'1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 108
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 116 of 151
6 RESIDENTIAL PROCESS RES UL TS
Table 7-1: Lighting Rebate Amounts By Energy Savings By Measure Type, 2010-2015
Nonresidential Program Data
Average Rebate Amount Per 1000 BTUs Saved
($/1000 BTUs)
Measures 2010 2011 2012 2013 2014 2015
Lighting Interior 0.07 0.05 0.29 1.29 0.04 0.05
Lighting Exterior 0.03 0.04 0.25 0.08 0.11 0.06
To further assess changes in participation, the evaluation team examined the rebated measures
in the 2010-2015 program data and DSM business plans to determine which nonresidential
measures were discontinued since 2010. To quantify the effect of the discontinued measures on
overall participation, the evaluation team looked at the difference between two quantities: 1) the
quantity of measures that would have been incented in 2015 if the non-discontinued measures
had the same participation as in 2013, when the lighting standards shifted (the "2015
theoretical" quantity); and 2) the quantity of measures that were actually incented in 2015 (the
"2015 actual" quantity).
Table 7-2 shows each rebated measure, whether the measure was available ("Y") or not
available ("N") each year from 2010 to 2015, the number of 2013 rebates for that measure, and
the "2015 theoretical" and "2015 actual" quantities described above. Comparison of the 2015
theoretical and actual quantities shows that most of the overall decline in the number of rebates
was not attributable to the discontinued measures. Discontinued measures accounted for a
reduction of 27 measures, representing 2% of the total decline of 1,356 measures (Table 7-2).
t.-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 109
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 117 of 151
6 RESIDENTIAL PROCESS RES UL TS
Table 7-2: Theoretical Versus Actual Participation, Accounting for Discontinued
Measures, 2010-2015 Nonresidential Program Data
b Availability of Rebates• Baseline: 2015. 2015 Measures # of 2013 theoretic t 1 ac ua
2010 2011 2012 2013 2014 2015 rebates al. c quantity quantity
Top 3 measures, accounting for 73% of all 2010-2015 rebates
Lighting Interior y y y y y y 1,164 1,164 330
Lighting Exterior y y y y y y 398 398 169
Industrial Process y y y y y y 210 210 25
Other measures, accounting for 27% of all 2010-2015 rebates
Case lighting y y y y y y 128 128 36
Food service equipment y y y y y y 83 83 72
Windows and insulation N N y y y y 73 73 21
HVAC y y y y y y 41 41 34
Green motors y y y y y y 15 15 5
Motor controls, HVAC y y y y y y 12 12 14
Appliances y y y y y y 11 11 0
Commercial water heater y N N N y y 0 5d
Compressed air y y N y N y 0
PC network controls y y y y y y 0 0 0
Shell y y y y y y 0 0 0
Motors y y y y N N 0 0 0
Renewables y y y y N N 3 0 0
Generator block heater N y y y y N 24 0 0
TOTAL 2,163 2,136 707
Number of rebates 27 1,356
decline from baseline
•y means "yes, available that year" and N means "no, not available that year."
b Excludes steam trap replacement, vending machine, side-stream filtration, refrigerated warehousing, LED traffic signals,
demand controlled ventilation, LEED certification, motor control (industrial), and multifamily measures, as those were not
available any year from 2013 to 2015 and so, by definition, do not contribute to any of the counts.
c Assumes the non-discontinued measures would have had the same number of rebates as in 2013.
d Used 2014 rather than 2013 rebate number since this measure was not available in 2013.
Next, the evaluation team examined changes to the rebate amounts from 2013 to 2015 to
assess whether reduced incentives may have affected participation. The average rebate
amounts per BTU saved declined for each measure from 2013 to 2015, except for the HVAC
measure (Table 7-3). As the rebate amounts declined so did the quantity of rebated measures
(Table 7-3; Correlation=0.5), indicating that the reduced rebates could have affected
participation rates.
'-'1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 110
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 118 of 151
6 RESIDENTIAL PROCESS RESULTS
Table 7-3: Percent Change in Rebate Amounts and Counts, 2010-2015 Nonresidential
Program Data
Average Rebate Amount Per 1000 BTUs % change in % change
Measures avg. rebate, $ in rebate
($11000 BTUs) per 1000 BTUs quantity
2010 2011 2012 2013 2014 2015 2013-2015 2013-2015
Lighting Interior 0.07 0.05 0.29 1.29 0.04 0.05 i -96% -85%
Lighting Exterior 0.03 0.04 0.25 0.08 0.11 0.06 -25% -17%
Industrial Process 0.04 0.04 0.04 0.04 0.04 0.03 -15% -88%
Case lighting 0.06 0.05 0.06 0.06 0.05 0.04 -33% -72%
Food service eq. 0.03 0.02 0.02 0.03 0.03 0.02 -31% -13%
Windows and insulation 0.05 0.05 0.06 0.03 -43% -71%
HVAC 0.04 0.03 0.02 0.02 0.02 0.02 30% -17%
Green motors 0.03 0.03 0.03 0.04 0.02 I 0.02 -50% -67%
Motor controls, HVAC 0.02 0.03 0.02 0.04 0.02 I 0.03 -32% 17% I I
Note: The evaluation team excluded discontinued measures from this analysis. The evaluation team also excluded
compressed air, PC network controls, shell (not windows and insulation), appliances, and water heater measures
from this analysis because no rebates were recorded in 2015 for these measures (even though 2015 DSM business
plan notes rebates were offered) or rebates were offered recently (not much data to assess percent change).
7.1.1.2 Analysis of Repeat Participation Among Customers
The evaluation team conducted another analysis to assess patterns of repeat participation
among nonresidential customers over rolling three-year periods, using the combined 2010-2015
program database. For each three-year period (2010-2012 , 2011-2013, 2012-2014, and 2013-
2015), the evaluation team identified the number of unique customers that either. 1) participated
in more than one program ; or 2) participated in the same program more than one time within
that three-year period. (The team refers to these customers as repeat participants). For each of
those three-year periods, dividing the number of repeat participants by the total number of
unique customers that participated in any program within that three-year period produced the
repeat participation rate for that period . For example, the formula for calculating repeat
participation within the period from 2010 to 2012 was:
Repeat participation rate =
The total number of unique customers that either participated in multiple programs within
2010-2012 or participated in the same program multiple times within 2010-2012
The total number of unique customers that participated in any programs within 2010-2012
t-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 111
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 119 of 151
6 RESIDENTIAL PROCESS RES UL TS
Repeat participation rates declined slightly from 201 Oto 2015 (Figure 7-3). Repeat participation
also appears to be an important driver of participation since more than one-tenth of
nonresidential participants participated in multiple programs or multiple times since 2010.
Figure 7-3: Percent of Nonresidential Customers Participating in Multiple Programs or
Same Program Multiple Times, 2010-2015 Nonresidential Program Data
25%
20%
17% 17%
15% 15%
15%
10%
5%
0%
2010-2012 2011-2013 2012-2014 2013-2015
Three-year analysis period
7 .1.2 Residential Participation Trends
7.1.2.1 Discontinued Measures and Reduced Rebate Incentives
For the analysis of discontinued residential program measures and reduced incentives, the
evaluation team combined information from the 2010-2013 program database with program
data from 2014 and 2015. The combined 2010-2015 residential program database contained
100,796 measures, of which the evaluation team analyzed 96,343 measures ( or 96% of all the
measures in the database).57 The evaluation team binned these measures into six categories:
1) ENERGY STAR appliances, 2) shell, 3) HVAC, 4) fuel conversions (or Fuel Efficiency
program), 5) water heater, and 6) ENERGY STAR Homes measures. The 2010-2013 program
database lacked the information necessary to identify low-income program participants, who
also couldn't be uniquely identified based on the measure. Thus, the subsequent analyses and
findings document overall participation trends because the evaluation team was not able to
separate low-income program participants from other rebate program participants. 58
57 The evaluation team did not have a complete set of data for all the measures. For example, the program data extracts contained
no information on 2010-2013 Appliance Recycling and UCON duct sealing measures.
58 2014 and 2015 program data included the information on low-income participants. About 10% of all 2014 and 2015 measures
were installed in low-income residences.
'-"Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 112
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 120 of 151
6 RESIDENTIAL PROCESS RESULTS
The overall number of rebated measures declined from 2010 to 2013 (Figure 7-4). In 2014 and
2015, the number of rebates increased but were well below the levels reported in 2010.
Figure 7-4: Reported Number of Residential Rebates, 2010-2015 Program Data
40000 51% 60%
43% 35000 40%
V, 30000 <IJ 20% ... <1l
..0 25000 <IJ 0:: 0% -20000 0
L. -20% <IJ 15000 ..0 E -40% :::l 10000 z
5000 -60%
0 -80%
2010 2011 2012 2013 2014 2015
-ES Appliances -shell -HVAC
-Fuel Conversion -Water Heater -ENERGY STAR Homes
-% Change Year-to-Year
According to the prior evaluation , Avista staff believed that the decline in the number of rebates
was due to the expiration of tax credits for energy efficient upgrades and high-efficiency home
appliances offered under the American Recovery and Reinvestment Act (ARRA) of 2009. 59 Staff
reported that these tax credits likely prompted an increase in rebate program participation in
2009 and 201 O, followed by a decrease in participation by 2011 when ARRA incentives started
to wane.
Further analysis revealed that ENERGY ST AR appliances, in particular, accounted for 40% of
all the rebated measures examined in the 2010-2015 data. Avista ceased offering rebates for
ENERGY STAR appliances in 2013, except to low-income customers.60 This likely explains the
abrupt drop in appliance measures in 2013 and thereafter, as rebates were not discontinued for
any other measures.61
To quantify the effect of the discontinued appliance measures on overall participation, the
evaluation team looked at the difference between two quantities: 1) the quantity of measures
59 Cadmus (2014). Avista 2012-2013 Process Evaluation Report.
60 There is no incentive budget in 2013-2015 Avista's DSM plans for appliance measures.
61 The one exception is that the Avista 2013 DSM plan did not include water heater rebates in 2013, but did include them in all other
years of this analysis.
t-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 113
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 121 of 151
6 RESIDENTIAL PROCESS RES UL TS
that would have been incented in 2015 if the non-discontinued measures had the same
participation as in 2010,62 when the decline in rebate quantity began (the "2015 theoretical"
quantity); and 2) the quantity of measures that were actually incented in 2015 (the "2015 actual"
quantity). Note that the appliance measure was not discontinued for low-income customers and
there were 26 low-income appliance rebates in 2015. Therefore, the first of the above quantities
also assumes there would have been 26 low-income appliance rebates in 2015.
Table 7-2 shows each rebated measure, whether the measure was available ("Y") or not
available ("N") each year from 2010 to 2015, the number of 2010 rebates for that measure, and
the "2015 theoretical" and "2015 actual" quantities described above. Comparison of the 2015
theoretical and actual quantities shows that most of the overall decline in the number of rebates
was attributed to the discontinued appliance measures, which accounted for 17,332 of the total
decline of 23,453 measures, or 74% of the total (Table 7-2).
Table 7-4: Theoretical Versus Actual Participation, Accounting for Discontinued
Measures, 2010-2015 Residential Program Data
Availability or t<eoa1es Baseline 2015
Measures 2010 2011 2012 2013 2014 2015
2010, # of Projected
rebates # of
rebates**
' . . . y y y N* N* 1 N* 17358 26
Shell y y y y y y 7728 7728
HVAC y y y y y y 7562 7562
Fuel Conversion y y y y y y 256 256
Water Heater y y y N* y y 1345 1345
ENERGY STAR Homes y y y y y y 220 220
TOTAL 34,469 17,137
Number of rebates 17,332 decline from baseline
* Low-income customers still received Avista's rebates for appliance or water heaters.
** The number of rebates is the same as in 2010, except for discontinued measures.
Appliances were the most common measures in the 2010-2015 program data, followed by shell,
and HVAC measures. With regard to shell and HVAC measures, these measures declined from
2010-2013, but not in 2014 and 2015. As shown in Figure 7-5, shell rebates increased by 507%
and 68% from 2013 to 2014 and 2014 to 2015, respectively. HVAC rebates increased by 34%
from 2013 to 2014 and decreased by 7% from 2014 to 2015.
62 In contrast with the nonresidential analysis, which used 2013 as the baseline, the evaluation team selected 2010 as the baseline
because the team wanted to include the period when ARRA funding was available for residential energy efficiency upgrades, which
the previous evaluation identified as one reason for increased participation in Avista's rebate programs.
L-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 114
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 122 of 151
2015
Actual#
of
rebates
4295
4181
1742
688
84
11,011
23,458
6 RESIDENTIAL PROCESS RES UL TS
Figure 7-5: Percent Change Year-to-Year by Measure Rebate Type, 2010-2015 Residential
Program Data
2010 2011 2012 2013 2014 2015
600% 507% Qi ..c ..., V) 500% C Qi ·-..., .... Cl) 400% Cl) .0 Qi Qi >-er: 300% Cl "O ..., Qi
.'.. t:'. 200% Cl) 0
Qi C. 68% >-Qi 100% Qi er:
bO '+-2% -C 0 0% Cl) .... -..c Q) -7% u .0 -42% -32% ..., E -100% -54% -36% C -80% Q) ::i ~ Z -200%
Qi a..
•Shell •HVAC
Because the shell and HVAC measures accounted for 52% of all the measures in the 2010-
2015 program data, the evaluation team examined rebate amounts associated with these
measures to assess whether changes in incentive amounts affected shell and HVAC program
participation. To compare changes to the rebate amounts across the various shell and HVAC
measures, the evaluation team divided rebate amounts with estimated energy savings for each
record in the database. Many records in the database included both electric (kWh) and gas
(therm) savings. To estimate total (electric+gas) savings, the evaluation team converted kWh
and Therm savings for each record to British Thermal Units or BTUs.
Among the four shell measures examined and listed in Table 7-3, one measure in particular,
windows, accounted for nearly two-thirds of the total number of shell measures contained in the
2010-2015 program database. The average rebate amount per BTU saved for windows
declined from 2010 to 2013 and then increased from 2013 to 2015 (Table 7-3). This change
could explain why participation in the shell program declined from 2010 to 2013 and then
increased in 2014 and 2015.
Among the five HVAC measures listed in Table 7-3, three accounted for nearly all the HVAC
measures in the 2010-2015 program data: high efficiency furnace or boiler, high efficiency air­
source heat pump, and variable speed motor. The average rebate amount per BTU saved for air
source heat pumps and variable speed motors increased from 2010 to 2015, while the quantity
of rebated measures for air source heat pumps and variable speed motors decreased (Table
7-5). This indicates that the higher incentives per BTU saved in 2015 compared to 2010 did not
halt the decline in incented air source heat pump and variable speed motor installations. The
average rebate amount per BTU saved for the natural gas furnace or boiler measure decreased
from 201 O to 2015. This decrease in rebate amount may be associated with the decrease in the
quantity of natural gas furnace or boiler measures from 2010 to 2015 (Table 7-5).
t-'1 Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 115
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 123 of 151
6 RESIDENTIAL PROCESS RES UL TS
Table 7-5: Percent Change in Rebate Amounts and Counts, 2010-2015 Residential
Program Data
Windows* 0.036 i 0.034 0.031 0.000 0.047 0.054 48% -44%
Attic Insulation 0.035 i 0.031 0.044 0.060 0.035 0.030 -14% -64%
Floor Insulation 0.014 i 0.015 0.017 0.023 0.036 0.027 90% -33%
Wall Insulation 0.013 i 0.016 0.016 0.025 0.046 0.034 161% -65%
HVAC
Nat. Gas 0.033 0.033 0.038 0.038 0.028 0.024 -26% -52% Boiler/Furnace*
Air Source Heat Pump* 0.038 i 0.037 0.140 0.097 0.056 0.054 43% -80%
Variable Speed Motor* 0.052 i 0.052 0.065 0.067 0.067 0.067 28% -24%
Ductless Heat Pump 0.071 I 0.013 0.073 0.073 0.000 0.000 -100% -100%
A/C Replacement 0.054 i 0.000 0.000 0.000 0.000 0.000 -100% -100%
Note: Program data included a few additional shell and HVAC measures in 2014 and 2015. Because these measures
were not listed in 2010-2013 data extract, the evaluation team excluded these measures from this analysis.
* These are the most frequent measures and they constitute the majority of the measures in the shell or HVAC programs.
7.1.2.2 Analysis of Repeat Participation Among Customers
The evaluation team conducted another analysis to assess patterns of repeat participation
among residential customers over rolling three-year periods, using data from the program
databases from 2010 through 2014.63 For each three-year period (2010-2012, 2011-2013, and
2012-2014), the evaluation team identified the number of unique customers that either. 1)
participated in more than one program; or 2) participated in the same program more than one
time within that three-year period. Then, for each of those three-year periods, the above quantity
was divided by the total number of unique customers that participated in any program within that
three-year period. For example, the formula for calculating repeat participation within the period
from 201 Oto 2012 was:
63 The evaluation team had difficulty in matching participant ID variable with records in 2015 data.
t.-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 116
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 124 of 151
6 RESIDENTIAL PROCESS RES UL TS
Repeat participation rate =
The total number of unique customers that either participated in multiple programs within
2010-2012 or participated in the same program multiple times within 2010-2012
The total number of unique customers that participated in any programs within 2010-2012
Repeat participation rates declined threefold from 2010 to 2015, but this decline had little effect
on overall participation rates since less than one-tenth of residential participants participated in
multiple programs or multiple times since 2010 (Figure 7-6).
Figure 7-6: Percent of Residential Customers Participating in Multiple Programs or Same
Program Multiple Times, 2010-2014 Residential Program Data
10% 9%
8%
6% 5%
4% 3%
2%
0%
2010-2012 2011-2013 2012-2014
7.1.2.3 Analysis of Availability of Qualifying Measures at Lower Price Points
Ihe evaluation team conducted a third analysis using available 2010-2015 program data to
determine whether limited product availability may have affected program participation. Previous
research conducted by the evaluation team for the Energy Trust of Oregon revealed that the
proportion of rebated refrigerators at lower price points declined sharply over several years in
the Pacific Northwest. A single brand dominated the lower-priced refrigerator models that
qualified for rebates, suggesting that consumers had relatively few models to choose from at the
lower end of the market. The evaluation team did not have actual market data on model
availability, as it did for the Energy Trust analysis, but the evaluation team examined unit cost of
the rebated measure to determine whether evidence exists of a change in model availability.
Data on price paid were examined for these two measures: natural gas furnace/boiler and water
heater.
Customers participating in Avista's HVAC program are buying furnaces or boilers at lower cost,
on average, in 2015 than in the prior years. The average price of incented furnaces or boilers
peaked in 2012 and then declined in subsequent years (this trend was significant; ANOVA at
p<0.05). The average price in 2012 ($4,084), in particular, was significantly lower than the
average price in 2015 ($3,756) (Tukey post-hoc test at p<0.05), indicating that in recent years
participating customers have bought more incented units at lower price points. On the other
'-"Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 117
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 125 of 151
6 RESIDENTIAL PROCESS RES UL TS
hand, customers participating in Avista's Water Heater program are buying water heaters at
higher cost, on average, in 2014 and 2015 than in the prior years. The average price of incented
water heater units increased since 2010. The lack of a consistent relationship between average
price paid and participation rate does not support the hypothesis that the decline in participation
resulted from a change in model availability at different price points.
7 .2 Participation Rates Among Opower Behavioral
Program Participants and Nonparticipants
The evaluation team analyzed participation data from Avista's residential Behavioral Program ,
which is administered by Opower (Opower program), to gather insight into the effectiveness of
Opower's home energy reports at encouraging customers to do more energy savings activities
and/or participate in Avista's rebate programs. Th is analysis specifically investigates the
effectiveness of one particular combination: Opower plus Avista rebates.
The evaluation team used randomized-control trial participation data from Opower combined
with Avista rebate participation data to analyze differences in energy savings across four groups
of customers in a quasi-experimental study. The team performed this analysis to determine
whether participation in both the Opower program and one or more Avista rebate programs
resulted in more electricity savings than the combined savings associated with programs
individually. That is, the evaluation team wanted to determine whether there was a "multiplier
effect" associated with customer participation in both the Opower program and the rebate
programs.
The four customer groups the team analyzed were:
• Opower+Rebate participants, who participated in both the Opower program and one or
more Avista rebate programs
• Opower-only participants, who participated in only the Opower program but not in an
Avista rebate program
• Rebate-only participants who participated only in one or more Avista rebate programs
but not in the Opower program
• Nonparticipants who did not participate in either the Opower program or in one of the
Avista rebate programs
7 .2.1 Data and Methods
7.2.1.1 Data Preparation
A sample of over 86,000 Avista customers in Washington and Idaho were randomly assigned by
Opower to two groups: a treatment group that received home energy reports from Opower
(Opower participants) and a control group that did not receive the reports (Opower
nonparticipants) (Table 7-6). The evaluation team prepared the participation data for
t-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 118
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 126 of 151
6 RESIDENTIAL PROCESS RESULTS
Washington and Idaho customers (see impact evaluation reports for more details on data
preparation) as follows:
• Calendarized customer monthly billing data into calendar months, and
• Removed customers with duplicate billing data, customers with no billing data after the
month when the Opower reports began , and customers with no billing data for at least
12 months before the Opower reports began.
The evaluation team combined data from the two states into a single dataset for this analysis
(Table 7-6). For this analysis, the evaluation team also required a data set in which the
proportions of participants and nonparticipants in Idaho matched the proportions of participants
and nonparticipants in Washington. In the original data, the percentage of Opower participants
and nonparticipants in Idaho was 66% and 34% respectively, and the proportions for
Washington customers was 79% and 21%, respectively. To achieve proportionality between the
states, the team excluded a random sample of 5,380 Opower nonparticipant customers in Idaho
(Table 7-6).
Table 7-6: Number of Opower Participants and Nonparticipants Before and After
Removing Random Sample from Idaho Control Group
Total Washington Idaho
N % N % N %
Original Sample Sizes
Opower nonparticipants 22,579 26.2% 11 ,292 21 .3% 11 ,287 34.1%
Opower participants 63,502 73.8% 41 ,695 78.7% 21,807 65.9%
TOTAL 86,081 100% 52,987 100% 33,094 100%
Sample sizes after removing random sample of Idaho nonparticipant customers
Opower nonparticipants 17,199 21 .3% 11 ,292 21.3% 5,907 21.3%
Opower participants 63,502 78.7% 41 ,695 78.7% 21 ,807 78.7%
TOTAL 80,701 100% 52,987 100% 33,094 100%
In accordance with the program, Opower participants began receiving the home energy reports
in June and July of 2013, and continued receiving reports through December 2015 (treatment
period).64 However, due to a change to Avista's customer billing system during the first half of
2015, none of the Opower participants received Opower reports between February and July of
64 Opower participants received eight home energy reports in a year, or two per quarter of a year.
t-'1 Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 119
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 127 of 151
6 RESIDENTIAL PROCESS RESULTS
2015 (pause period). Opower participants began receiving reports again from August 2015
through December 2015, the end of the evaluation period. 65
During the treatment period between July 2013 and December 2015, about four percent of the
Opower participants and nonparticipants participated in one or more Avista rebate programs
(Table 7-7).66 The evaluation team merged the rebate program participation data with the
Opower program participation data.
Table 7-7: Number of Opower and Avista Rebate Participants and Nonparticipants
Opower Participant Opower Total
Nonparticipant
N % N % N %
Avista Rebate Participant I 2,s31
'
j 4.0% 656 i 3.8%
I
i 3,187 l 3.9%
Avista Rebate Nonparticipant I 60,911 ! J 96.0% 16,543 I 96.2%
I
I 11 ,s14 I 96.1%
TOTAL j 63,502 I 100% 1 11,199 I 100% i ao,101 I 100%
Calendarized monthly electricity usage data from billing records, including total monthly kWhs
and average daily kWhs, were available for all customers in the dataset for 16 months
preceding July 2013 (the pre-treatment period, March 2012 to June 2013). These data were
also available for up to 30 months during the treatment period (July 2013 to December 2015).
The data were structured such that each row represented a calendar month of customer billing
data, in which each unique customer could have up to 46 rows, or months, of billing data.
About 22% of customers opted-out of the Opower program or moved residences at some point
during the treatment period such that 63,283 customers remained in the dataset through the
entire treatment period. The evaluation team included the customers that opted out or moved
residences in its analyses to maintain the quasi-experimental design of the study and to avoid
reducing the relatively small number of Avista rebate participants in the dataset.67
For the analysis, the team used the following variables:
• Opower_lD: unique identifier for each customer.
65 Opower participants continued to receive Opower reports after December 2015 but all subsequent months fall outside the current
evaluation period and are not included in analyses.
66 Avista's "rebate" programs include rebates for high efficiency heating, ventilation, and air conditioning equipment upgrades, high
efficiency water heating equipment upgrades, conversions from electric to natural gas space and water heating equipment,
insulation and windows, and high efficiency equipment for ENERGY STAR® homes; the team also included UCONS direct install
duct sealing and incentives for appliance recycling.
67 Nonparticipants could not "opt out'' since they were not receiving Opower reports, and the team had no way to identify which
nonparticipants would have opted out if they had been receiving the Opower reports.
t..1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 120
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 128 of 151
6 RESIDENTIAL PROCESS RESULTS
• Daily_Average_kWh: measure of average daily kWh usage for each customer and
month.
• Daily_Average_kWh_Logged: logarithmic measure of average daily kWh usage for each
customer and month.
• Daily_Average_kWh_Preusage: measure of average daily kWh usage for each customer
and month in the pre-treatment period, coded to respective months in the treatment
period (e.g. daily average kWh usage for each customer in May 2013 is coded for the
customer in May 2014 and in May 2015).
• Year_Month: measure of time specifying the year and month of each electric bill.
• Pre_Post: indicator of the pre-treatment period (coded 'O' for each month , March 2012 to
June 2013) and treatment period (coded '1' for each month, July 2013 to December
2015).
• Opower_Participant: indicator of whether the customer is an Opower participant (coded
'1' for all months) or Opower nonparticipant (coded 'O' for all months).
• Avista_Rebate_Participant: indicator of whether the customer is an Avista rebate
participant (coded '1' for the month in which they participated and all subsequent months
and coded 'O' for all months prior to participation) or nonparticipant (coded 'O' for all
months).
7.2.1.2 Analysis Methods
The evaluation team analyzed the prepared data set to determine whether participation in one
or more Avista rebate programs and the Opower program results in more electricity savings
than the sum of the electricity savings attributed to participation in each program separately.
That is, the evaluation team wanted to determine whether there was a "multiplier effect"
associated with customer participation in both the Opower program and the rebate programs.
To do this, the evaluation team constructed cumulative and monthly lagged dependent variable
(LDV) regression models that estimate electricity savings of Opower-only, Avista Rebate-only,
and Opower+Avista Rebate program participation, compared to nonparticipants, using daily
average kWh usage as the dependent variable.
The team used two different statistical regression methods to estimate the differences in
electricity savings among the different customer groups. With the first method, the evaluation
team included binary (yes/no) indicator variables to denote participation in the Opower and
Avista rebate programs along with another indicator variable (an interaction term) that indicated
whether the customer was a participant in both programs.68 In the second method, the team
68 LDV Cumulative interaction model: Daily_average_kWh_usage = Opower_participant(l3) + Avista_Rebate_participant (13) +
Opower_participant (13)*Avista_Rebate_participant (13) + year_month+ daily_average_kWh_preusage + E
LDV Monthly interaction model: Daily_average_kWh_usage = ([HER_participant_group(l3) + Rebate_participant_group(l3) +
HER_participant_group(l3)*Rebate_participant_group(l3)] by year_month) + year_month + daily_average_kWh_preusage + E
t..'1 Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 121
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 129 of 151
6 RESIDENTIAL PROCESS RES UL TS
conducted separate regression models for each of the following six group comparisons.69 The
group comparison models do not control for the excluded groups like the interaction models do
but the team performed these group comparison models as verification that results from the
interaction models are robust.
• Nonparticipants (0) vs. Opower-only participants (1)
• Nonparticipants (0) vs. Avista Rebate-only participants (1)
• Nonparticipants (0) vs. Opower+Avista rebate participants (1)
• Opower-only (0) vs. Avista Rebate-only participants (1)
• Opower-only (0) vs . Opower+Avista Rebate participants (1)
• Avista Rebate-only (0) vs. Opower+Avista Rebate participants (1)
Electricity savings were measured in these models by comparing the actual daily average kWh
usage (from monthly billing data) in the treatment period across the four groups, controlling for
average daily kWh usage during the months in the pre-treatment period. The percent electricity
savings were measured by replacing actual daily average kWh usage with the logarithmic
measure of daily average kWh usage.
Due to the quasi-experimental design of the study, in which customers participated in Avista
rebate programs in different months of the treatment period, there were too few Avista Rebate­
only participants in the first three months of the treatment period (n < 45) to have the statistical
power needed to include these data in the analyses. In addition, the team excluded from
analyses data from customers using 500 daily kWhs or more in a month (n=48).
7.2.2 Findings
The evaluation team estimated the average daily electricity usage differences and percent
electricity savings across the four customer groups: nonparticipants, Opower-only participants,
Avista Rebate-only participants, and Opower+Avista Rebate participants. This section first
describes differences between these groups and then answers the question about whether the
combined Opower+Avista Rebate results in more electricity savings than the sum of the savings
attributed to each program separately.
During the pre-treatment period, nonparticipants and Opower-only participants had the lowest
average daily kWh usage, followed by the Opower+Avista Rebate participants and Avista
69 LDV Cumulative comparison models: Daily_average_kWh_usage = group1vsgroup2(13) + year_month +
daily_average_kWh_preusage + E
LDV Cumulative comparison models: Daily_average_kWh_usage = group1vsgroup2(13) by year_month + year_month +
daily_average_kWh_preusage + E
'-'1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 122
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 130 of 151
6 RESIDENTIAL PROCESS RES UL TS
Rebate-only participants. However, during the treatment period, these trends changed such that
Opower+Avista Rebate participants had the lowest average daily kWh usage, followed by Avista
Rebate-only participants, Opower-only participants, and, lastly, nonparticipants (Table 7-8).
These trends are illustrated across each month of the pre-treatment and treatment periods in
Figure 7-7.
Table 7-8: Average Daily kWh Usage Before and During the Treatment Period by Group
Nonparticipant Opower-only Avista Rebate-Opower+Avista
only Rebate
Pre-treatment period I 44.8 I 44.9 i 46.4 I 46.2
Treatment period ! 46.9 ! 46.0 i 44.9 i 43.6
Figure 7-7: Monthly Average Daily Energy Usage by Group
70
65
60
55
50
45
40
35
30
N ....
~ "' ~
N N N N rn .... .... .... .... .... >-' Cl. > C. "' :i QJ 0 ~ ~ ~ L/) z
--Nonparticipant
Treatment Period Begins
rn rn rn rn rn st st st st st .... .... 1 .... .... .... .... .... ";' ....
.'.. >-Cl. > C. .'.. >-Cl. "' "' :::, QJ 0 ~ "' "' :i QJ ~ ~ ~ L/) z ~ ~ ~ L/)
--Avista Rebate-only --Opower-only
7.2.2.1 Cumulative LDV Model Results
Pause Period
st LI) LI) LI) LI) LI) LI) "' .... .... .... .... ";' .... .... .... > C. .'.. >-Cl. > C. 0 ~ "' "' :i QJ 0 ~ z ~ ~ ~ L/) z
--Opower+Avista Rebate
The combination of the Opower home energy reports and Avista rebates appears to amplify
electricity savings. Opower+Avista Rebate participants used significantly less electricity during
the entire treatment period, on average, than the other groups (Figure 7-8; Table 1 in Appendix
A). Opower+Avista Rebate participants, compared with nonparticipants (or the baseline), used
5.7% less electricity (or 2.82 kWh/day less).
These savings in electricity usage were significantly greater than the sum of the average
savings attributed to the rebate programs alone (1.7%, or 1.35 kWh/day; Avista Rebate-only
group versus baseline) plus the Opower program alone (1 . 7%, or 0.90 kWh/day; Opower-only
group versus baseline). The sum of the savings from the two groups of customers individually
resulted in 3.4% savings, or 2.25 kWh/day.
t--1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 123
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 131 of 151
6 RESIDENTIAL PROCESS RES UL TS
These results were determined using the LDV cumulative regression model with the interaction
term (see equation in footnote 3 and full results in Table 1 in Appendix A). The model results are
similar to but more conservative than the results from using the group comparison LDV
cumulative regression models; these more conservative results were expected since the group
comparison models do not include all customer groups in the same model (see Table 2 in
Appendix A).
Figure 7-8: Average Cumulative Percent Electricity Usage Compared to Nonparticipants
-6.0%
vi -5.0% >
QJ bJl re =, ~ -4.0%
..C C
3 ~
.:::it. ·u c ·-e -3.0% w ro u 0..
aj g
~ z -2.0%
bJl ~ QJ ~ -1.0%
0.0%
-1.7%
Opower-only
* statistically significant at ps.05
-5.7%
-1.7%
Avista Rebate-only Opower+Avista Rebate
Note: These findings only take into account electric (kWh) savings. About 14% of Avista's rebate participants
in the Opower dataset participated in Avista's Fuel Efficiency program, which means they converted from
electric to natural gas space and/or water heating. These customers had an increase in natural gas
consumption (therms) that is not accounted for in this and subsequent analyses.
7.2.2.2 Monthly LDV Model Results
Although the energy usage difference between the Opower plus Avista rebate group and the
other customer groups is significant, further analyses revealed that Opower plus Avista rebate
participation significantly affected electricity usage only during the early months of the treatment
period . Figure 7-8 shows the average daily percent electricity usage for each group compared
with nonparticipants and for each month in the treatment period from October 2013 to
December 2015.70 The Opower+Avista Rebate participants, compared with Nonparticipants,
saved significantly more electricity per day, on average, during three months of the heating
70 The team excluded the months of July 2013 to September 2013 due to the small number of Avista rebate participants in the
dataset for these months; the number of rebate participants is too small (n<45) to have the statistical power to perform the analysis.
,1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 124
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 132 of 151
6 RESIDENTIAL PROCESS RES UL TS
season early in the treatment period of the Opower program: November 2013, January 2014,
and February 2014.
As shown in Figure 7-8 although the average daily electricity usage was not significantly
different during the following 2014-2015 heating season, these months coincide with the pause
period for distributing the home energy reports to participating customers. The evaluation team
lacked the data to extend its analysis through the 2015-2016 heating season; Figure 7-8
however, does show some evidence that Opower+Avista Rebate participants may have been
saving more energy during these months.
The results from the LDV monthly regression model with the interaction term are similar to but
more conservative than the results from the group comparison LDV monthly regression models
(see equation in footnote 3 and full results in Table 1 in Appendix A); the more conservative
results were expected since the group comparison models do not include all groups in the same
model (see Table 2 in Appendix A).
Figure 7-8: Average Daily Percent Electricity Usage for Each Month Compared to
Nonparticipants*
"' .... C: .,, a. ·;:;
t .,, a. C:
0 z
QJ tll) .,,
"' ::,
.s:: :i=: -"" .... C:
10.0%
5.0%
0.0%
-5.0%
~ -10.0%
QJ
0.
> ::c ~ -15.0%
0 ~
-20.0%
Pause Period
m m m "'" "'" "'" "'" "'" "'" "'" "'" "'" "'" "'" "'" Ll'l Ll'l Ll'l Ll'l Ll'l Ll'l Ll'l Ll'l Ll'l Ll'l Ll'l Ll'l '( .... .... .... .... .... .... .... .... '( .... .... '( .... '( .... .... .... '( .... .... '( .... .... '( .... '( .... > 6 c 1:, ~ C. ;.. c ~ bl) C. .... > u c 1:, ~ C. ;.. c "S bl) C. t, > u u 0 QJ ~ QJ .,, .,, ~ :::, QJ u 0 QJ ~ QJ "' "' :::, :, QJ 0 QJ 0 z 0 u.. ~ <l: ~ <l: Vl 0 z 0 u.. ~ <l: ~ ~ ~ <l: Vl 0 z 0
Month-Year
-• -Opower-only -Avista Rebate-only
--Opower-only + Avista Rebate-only --Opower+Avista Rebate
* Red asterisks ( ::+::) indicate statistically significant average daily percent savings at p:5.10
7.2.3 Discussion
It appears that there is a multiplier effect when rebate participants receive home energy reports.
The amplified Opower+Avista Rebate savings could be the result of additional electricity saving
'-"'Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 125
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 133 of 151
6 RESIDENTIAL PROCESS RES UL TS
actions these customers undertook in their homes. Furthermore, the Opower home energy
reports could be influencing the type and number of rebate programs in which these customers
are participating . For example, a significantly higher percentage of Opower+Avista Rebate
participants participated in the Fuel Efficiency rebate program to convert from electric to natural
gas space and/or water heating compared with Avista Rebate-only participants (14% vs. 12%,
respectively; ps.10). In addition, Opower+Avista Rebate participants participated in significantly
more rebate programs, on average, compared with Avista Rebate-only participants (1.55 vs.
1.46 rebate programs, respectively; pS.05). However, Opower+Avista Rebate participants did
not participate in Avista rebate programs at a higher rate compared with Avista Rebate-only
participants (4% vs. 3.8%, respectively; not significantly different).
Collectively, these findings suggest that home energy reports can be effective at engaging
customers and motivating them to take actions such as participating in Avista's rebate
programs, such as the Fuel Efficiency program. These findings validate Avista's strategy to
promote the rebate programs via the home energy reports .
These findings also suggest that customers who receive both home energy reports and rebates
are saving even more energy than would be expected based on the average per-customer
savings associated with each program. However, based on the current analysis, it is unclear
whether the additional savings are only realized seasonally, or if the additional savings are a
temporary phenomenon and lack persistence. Nevertheless, the possibility of a multiplier effect
could have important implications for future program planning.
Future research should continue exploring the question of whether a combination of the home
energy reports and rebate program participation results in more electric savings compared with
participation in each program alone. For example, it is important to try and replicate these
findings to ensure they are not an isolated outcome. It is also important to further analyze the
savings to determine whether the savings are persistent and/or whether they are only realized
during certain portions of the year (e.g., the heating season). Future research also should
investigate further the type and number of rebate programs in which customers are participating
and explore whether other program combinations could also amplify savings. Lastly, future
research should further examine attribution of electricity savings from the combination of
Opower participation and utility program participation to determine to what extent the Opower
reports are influencing customers to participate in other programs.
7.3 Commercial Uptake of Simple Steps Lighting
The Simple Steps, Smart Savings program promotes the sales of CFL and LEDs to residential
customers. Avista currently only reports savings for this offering through their residential lighting
program. However, due to the delivery mechanism of the program (in-store buy down
promotions), the evaluation team sought to understand if nonresidential customers were
purchasing bulbs discounted through the program and if so, what percent of Simple Steps bulbs
are 'leaking' into the nonresidential sector. The evaluation team estimated this "leakage" into the
commercial sector using the responses of customers (participants and nonparticipants), as well
t-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 126
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 134 of 151
6 RESIDENTIAL PROCESS RE SUL TS
as by conducting a survey of large retailers that sell Simple Steps items. The following section
describes this special study's objective, and results.
7.3.1 Objective
The objective of this study aimed to determine the distribution of Simple Steps, Smart Savings
CFL and LED items across the residential and commercial sectors. A second purpose was to
determine when retailers joined the Simple Steps program and identify future opportunities for
savings and participation in the Simple Steps program.
7.3.2 Results
The evaluation team describes the results of each method below, beginning with the customer
results.
7.3.2.1 Customer Results (Participant and Nonparticipant Surveys)
Of 375 surveyed nonresidential customers (participants and nonparticipants), 25 reported
purchasing 2,685 Simple Steps items for their businesses. About half of the items were CF Ls
and half were LED items (Table 7-9).
Table 7-9: Summary Items in the Commercial Sector Attributable to Simple Steps
Participants (n=305) Nonparticipants (n=70) Total (n=375)
Respondents Items Respondents Items Respondents Items
Standard CFLs 11 1,030 3 60 14 1,090
Specialty CFLs 8 274 12 9 286
LEDs* 11 736 0 0 11 736
LED Fixtures 4 517 2 56 6 573
TOTAL 21 2,557 4 128 25 2,685
* lncented in 2014 and second half of 2015
Multiplying each sample total by the inverse of the respective sampling ratio produced estimates
of 47,452 CFLs and 37,338 LEDs sold to nonresidential customers.71 Those estimates
represent 5.3% of the 896,485 of Simple Steps CFL items and 12.6% of the 295,870 of Simple
Steps LED items sold in Avista territory that were sold to nonresidential customers, thus
equating to the leakage percent of the program into this sector. The sample size of 375 provided
5% precision at 95% confidence.
71 The "sampling ratio," also known as the "sampling fragment," is the ratio of the sample size to the population size
(https://en.wikipedia.org/wiki/Sampling fraction). Thus, the total numbers of Simple Steps CFLs and of LEDs reported by
participants were multiplied by the inverse of the participant survey sampling ratio and the total numbers of Simple Steps CF Ls and
of LEDs reported by nonparticipants were multiplied by the inverse of the nonparticipant survey sampling ratio.
'-"Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 127
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 135 of 151
6 RESIDENTIAL PROCESS RES UL TS
7.3.2.2 Retail Manager Surveys
Retail respondents were typically lighting or electrical department managers and had held their
position from three months to 20 years, for an average of four years. Overall, the 27
respondents represented stores that sold 75% of all Simple Steps CFLs and 85% of all Simple
Steps LEDs. Of the 27 retailers surveyed, 17 could provide an estimate of the number of CFLs
sold to nonresidential customers, representing 51% of all Simple Steps CFL sales, and 14 could
provide an estimate of the number of LEDs sold in that sector, representing 53% of all Simple
Steps LED sales.
The evaluation team calculated the number of Simple Steps items sold to the commercial sector
by calculating the mean percentage of Simple Steps items sold to nonresidential customers,
weighted by the total number of Simple Steps items sold per respondent. Using the above
methods, the evaluation team estimated that 11.6% of Simple Steps CFLs (or 104,019 bulbs)
and 12% of LEDs (or 35,476 bulbs) were sold to nonresidential customers.
7.3.2.3 Comparison of Participant/Nonparticipant and Retail Manager Results
Figure 7-9 shows the estimated percentage of Simple Steps lighting sold to nonresidential
customers that each data source (customer surveys and retailer survey) produced. The two
data sources produced similar values: 12.6% and 12% of LED leakage for the customer and
retailer surveys, respectively. The estimates are less similar for CF Ls, with values of 5.3% and
11.6% for the customer and retailer surveys, respectively.
Figure 7-9: Estimates of Percent of Products in Commercial Sector
14% 12.6%
12% 11.6% 12.0%
10%
8%
6% 5.3%
4%
2%
0%
CFL LED
• Customer • Retailers
7.3.3 Retailers Experience with Simple Steps
Respondents reported promoting CFLs for longer time periods than LEDs. Fourteen of 27
respondents could estimate how long they had promoted Simple Steps CFLs; responses ranged
from three months to six years, averaging 2.2 years. Twelve of the 27 respondents could
t-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 128
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 136 of 151
6 RESIDENTIAL PROCESS RESULTS
estimate how long they had been promoting LEDs and reported promoting Simple Steps LEDs
from three months to two years, averaging slightly less than one year.
7.3.4 Other Opportunities for Simple Steps
Retailer respondents did not report many opportunities to improve the Simple Steps program for
residential or nonresidential customers going forward. Five of the 27 suggested maintaining or
expanding the program's LED offerings. None reported participating in the recent Simple Steps
washing machine offering.72
72 Simple Steps, Smart Savings, Appliance Frequently Asked Questions,
http://www.simplestepsnw.com/consumer/How%2520to%2520Choose/Appliance%20FAQ
t.-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 129
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 137 of 151
8 Conclusions and Recommendations
The 2014-2015 evaluation shows high levels of program awareness among all of Avista's
customers and shows high levels of satisfaction among program participants and contractors.
Program participants and contractors were complementary of Avista staff and generally
appreciated the opportunities to save money, save energy, and improve their properties that the
programs provide. The evaluation also shows that there are areas the programs could enhance
to make them better able to respond to the ever changing market conditions in which these
programs operate.
The evaluation team concluded the following and provides several suggestions for Avista's
programs. This section begins with conclusions and recommendations pertinent across all
programs (cross-cutting), followed by nonresidential and small business, and ending with
residential specific conclusions and recommendations.
8.1 Cross-cutting
Conclusion 1: Contractors are key program partners.
Contractors are the driving force of Avista's rebate programs, as they inform both nonresidential
and residential consumers about Avista's rebate opportunities and convince them to purchase
qualifying equipment. The nonresidential contractors also initiate a notable portion of work in
comparison to customer-initiated jobs and appear to be playing a larger role in application
preparation than in years past. Both nonresidential and residential customers report being highly
satisfied with contractors and are taking into account contractor's recommendations on what to
install. Although developing a trade ally network is not a priority, there are several things that
can be done short of an official network that could result in increased participation and savings.
Recommendations: Increase support for contractors.
Consider the following suggestions to continue strengthening relationships with contractors and
to improve their effectiveness in generating program savings:
1. Offer an opt-in mailing list to contractors. Contractors subscribed to this mailing list
would receive regular information on program offers, changes, trainings, and other
program supporting information. This list would be open to any interested contractor.
2. Promote outreach to contractors: Encourage program staff and account executives to
engage further with contractors by continuing and perhaps increasing their involvement
with contractor-related resources such as the Northwest Lighting Network. This work can
further educate contractors and nudge them to cross-promote the rebate programs to
their customers. Additionally, training may help contractors up-sell high efficiency
equipment through the program by improving their understanding of and ability to sell
high efficiency solutions. Therefore, Avista should continue to support contractors
t-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 130
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 138 of 151
8 CONCLUSIONS AND RECOMMENDATIONS
attending NEEA's training sessions including their recently launched comprehensive
training for lighting contractors and distributors.
3. Share effective messaging or marketing collateral with contractors. Contractors could
support program and marketing staff by providing insights into how to best target certain
customer types, learn from Avista on how to better target certain customer segments,
and possibly promote cross-program referrals and participation. As findings from the
evaluation show that most contractors specialize in the nonresidential or residential
sectors, even if they serve both, developing sector-specific messaging may be
particularly effective.
4. Investigate offering cooperative (co-op) marketing. Co-op marketing can help contractors
effectively market the program consistent with Avista's objectives and increase customer
perceptions of contractor's credibility and cross-promote other programs.
Conclusion 2: Although Avista and its implementation contractors deliver rebate
programs efficiently, promoting the programs further could help maintain or even
increase participation.
Several indicators suggest program promotions could be optimized. First, participants and
nonparticipants expressed high interest in learning more about Avista's rebate programs,
indicating that although they may be aware of Avista's offers, their knowledge is limited.
Second, a majority of residential participants who indicated learning primarily about Avista's
offers through contractors were not aware of other program opportunities outside the program
they participated in.
Recommendation: Develop more abilities to target marketing. For example, cross­
promote programs to recent participants by acknowledging their recent participation and
informing them of other program opportunities applicable to their home or business.
Recommendation: For residential customers, continue improving messaging in direct
mail promotions to better communicate program information since residential customers
prefer to receive this information via mail.
8.2 Nonresidential, Including Small Business
Conclusion 3: Although declining participation rates could threaten Avista's ability to
achieve long-term goals, evaluation results point to opportunities to drive additional
savings.
Developing new strategies to encourage deeper savings or increased participation will be
paramount to reversing the decline in participation and achieving long-term savings goals.
Almost one-third of nonparticipants reported they will make a building upgrade in the next two
years, indicating a continued potential for program participation. In particular, evidence suggests
that much opportunity remains for converting lighting from T12s.
t-'1 Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 131
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 139 of 151
8 CONCLUSIONS AND RECOMMENDATIONS
Recommendation: Develop a marketing approach specifically targeting replacement of
T12 lamps.
The switch to a TB baseline in 2012 had a dramatic effect on participation because the rebates
became far less attractive to customers to upgrade from T12s.73 While it may not be feasible for
Avista to alter the baseline for T12 change-outs, Avista should look into developing targeted
marketing strategies for convincing nonresidential customers with T12s to replace them with
more efficient lighting, focusing not only on savings but improved lighting quality and
performance. Avista could begin by targeting businesses that the Small Business Program has
identified as still having T12s.
Recommendation: Work with nonresidential lighting contractors to promote replacement
of T12 lamps.
Contractors make their living by selling equipment. Avista should work with nonresidential
lighting contractors to make sure they are fully aware of the advantages that more efficient
lighting (including the reduced wattage tube lighting that NEEA is targeting through its Reduced
Wattage Lamp Replacement Initiative) offer their customers.
Recommendation: Consider claiming Simple Steps savings for bulbs purchased for the
nonresiC.:ential sector.
The evaluation found that about 12% of Simple Steps LED sales and somewhere from 5% to
12% of Simple Steps CFL sales go to nonresidential customers. The mean hours of use for
such lighting is much higher in a nonresidential than residential settings, meaning that the total
Simple Steps savings is potentially higher than currently estimated, and at a minimum, Avista
should consider claiming the additional savings for these purchases.
8.3 Residential
Conclusion 4: Participation in the Avista rebate programs has rebounded since 2013
driven by a fivefold increase in shell program participation.
Rebate program participation reached a low point in 2013, after which participation increased
year over year by 51 % from 2013 to 2014 and by 43% from 2014 to 2015. This is a positive
sign; however, maintaining or increasing program participation requires cost effective savings
opportunities for residential customers. Avista's residential programs operate in a fast-changing
market. Consumers are adopting LEDs rapidly,74 retailers are transitioning away from CFLs to
73 A very similar thing happened to another program administrator in Missouri. See Ameren Missouri BizSavers Process Evaluation
Report 2015.
74 1 of 20 A-line bulbs sold nationally was an LED in third quarter of 2014, whereas in the quarter prior to that, it was 1 in 30. This
statistic comes from the 2015 LED Market Intelligence report by Bonneville Power Administration.
https://www.bpa.gov/ee/utility/research-archive/documents/momentum-savings-resources/led_market_intelligence_report.pdf
t.-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 132
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 140 of 151
8 CONCLUSIONS AND RECOMMENDATIONS
LEDs,75 and the federal government and regulators are mandating higher efficiency standards
for bulbs and other energy efficient technologies.76 The convergence of these forces has
implications for the cost effectiveness of Avista's downstream rebate programs. Program
administrators throughout the United States are exploring and testing alternative program
designs such as upstream and midstream designs in response to the evolving market. Although
Avista is currently participating in the Simple Steps, Smart Savings program (a midstream
program), when asked about future opportunities, program staff did not mention any upcoming
pilots or programs that apply these types of designs.
Recommendation: Continue regularly reviewing the expected savings and cost­
effectiveness of the measures in residential portfolio and exploring the benefits and
costs of other program designs including upstream and/or midstream designs.
Consider these suggestions:
1. Continue monitoring the technological advances and availability of ductless heat pumps
and water heating equipment. Surveyed contractors recommended both of these
categories as candidates for inclusion in Avista's programs. NEEA, for example, has
been working to promote the savings potential of heat pump water heaters in the
Northwest via the Northern Climate Heat Pump Water Heater Specification,77 and The
Northwest Power and Conservation Council has identified both of these measure types
as promising technologies in the recently adopted Seventh Power Plan.78
2. Explore upstream program opportunities outside of the lighting market. Upstream
incentive programs offer the potential to increase the adoption of energy efficient
technologies at a lower cost compared to downstream incentive programs. Program
administrators in California and elsewhere have successfully tested or used upstream
program designs for technologies that Avista currently incents, including HVAC
equipment and water heaters.79
Conclusion 5: Residential customers who rent their home are underserved.
75 Souza, Kim, 2016. Walmart to transition lighting products away from compact fluorescent to LED. Retrieved from
http://talkbusiness.net/2016/02/walmart-to-transition-lighting-products-away-from-compact-fluorescent-to-led/
76 The lighting standard, established by the Energy Independence and Security Act of 2007, requires that light bulbs use about 25%
less energy by 2014. New efficiency heating and cooling standards from the U.S. Department of Energy, which have gone into effect
Jan. 1, 2015, will increase the efficiency of heating, ventilation, and air-conditioning (HVAC) equipment in certain regions.
77 1. I http://neea.org/northernc 1matespec
78 http://www.nwcouncil.org/energy/powerplan/7/plan/
79 Quaid, M. and H. Geller (2014). Upstream Incentive Utility Programs: Experience and Lessons Learned. Retrieved April 14, 2016.
http://www.swenergy.org.
t.-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 133
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 141 of 151
8 CONCLUSIONS AND RECOMMENDATIONS
Nonparticipants say living in a rental property prohibits them from making improvements. This
was the second most commonly cited barrier to making energy efficient upgrades among
nonparticipants (after the up-front cost barrier). More than a quarter (27%) of nonparticipant
survey respondents were renters, whereas only 3% of the participant survey respondents were
renters. Renters account for about one-third of the population in Avista territory.80
Currently, Avista serves renters via the low-income program. The CAP agencies reported
having difficulty serving the low-income renter population because it is difficult to convince
landlords to participate. Additionally, there appears to be no multifamily program in the Avista
portfolio that could serve this market, although Avista does offer an incentive for a natural gas
space and water heating measures to multifamily property owners.
Recommendation: Investigate energy savings opportunities in the rental market.
Consider the following suggestions:
1. Estimate the number and distribution of rental units in the single family, manufactured
home, and among multifamily buildings. Analyzing these data geographically and by
vintage would likely yield insights regarding the energy saving potential in these markets.
2. Conduct needs assessment research with landlords to understand their needs and
concerns and explore ways to bolster their willingness to make energy efficiency
upgrades on their properties. This research should consider the needs landlords serving
low-income renters as well as renters not eligible for the low income program.
3. Conduct needs assessment research with renters to understand their needs and the
barriers to participation they face. For example, although some energy savings activities
may not be appropriate for renters (for example, HVAC system replacement), other
activities such as installing energy efficient lighting and/or advanced power strips could
be appropriate.
80 US Census Bureau. "825003 : Tenure." 2010 -2014 American Community Survey 5-Year Estimates. Web. 13 April 2016.
t-'1 Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs 134
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 142 of 151
Appendix A Opower
Table 1: Average Daily kWh Savings (P) Compared to Nonparticipants from Cumulative
and Monthly Lagged Dependent Variable Interaction Models
Opower Group1 Avista Rebate Opower Group X
Group 1 Avista Rebate
Group1'2
13 % 13 % 13 %
Cumulative Model3: i -0.90 ! i -1 .7% I -1.35 i -1 .7% ! -0.56
!
I -2.3%
Monthly Model4:
October 2013 I -0.85 l -1 .8% I -6.94 -17.8% I 2.89 I 9.0%
November 2013 I -1.16 ! -2.0% I -1.65 j -6.5% I -6.09 -9.0%
December 201 3 I -1.31 j -2.0% l -3.64 I -7.7% I -3.20 ! -4.5%
January 2014 I -1.13 1 -1.7% i -3.43 I -5.6% I -3.38 ! -6.4% I
February 2014 I -1.14 -1.7% I -0.85 i -2.1% I -8.33 -13.5%
March 2014 I -1.00 ! -1.7% I -2.88 I -4.2% I -1 . 71 ' -5.0% '
April 2014 i -0.79 j -1.6% I -1.86 I -2.8% I -1 .11 ! -3.8% ' i ' May 2014 I -0.58 ! -1.5% I -0.86 i -1.3% I -1 .45 j -4.4%
June 2014 I I i
I l -0.58 -1.3% -1.30 -1.4% I 0.20 -0.4%
July 2014 I -0.75 -1.3% I -0.39 ! 2.2% I 0.48 I 0.6% !
August 2014 I -0.58 I -0.8% I 1.26 [ 6.6%
1
-0.03 I -1 .5%
September 2014 I -0.66 ' -1.2% I -0.78 1 0.0% I -0.19 -1 .3%
October 2014 I -0.85 I -1 .8% I -2.95 ' -5.5% ' ! 0.46 I 0.3%
November 2014 I -1 .20 ! -2.3% I -1 .20 I -1.9% I -0.61 I -1 .5%
I -1 .60 ' -2.8% I 1.5% ' 1 0.0% December 2014 l 1.08 1 0.48
January 2015 I -1.56 i -2.8% I 0.12 0.1% f 0.49 ! -0.1% I
February 2015 i -1.24 -2.5% I -2.98 -5.1% I -0.92 I -2.1 % I
March 2015 I -1.16 i -2.4% 1 -1.83 -3.1% I 0.57 I 0.2% I
April 2015 I -0.97 I -2.3% -1.90 -3.3% I 0.37 ! -1.1% !
May 2015 I -0.69 i -1.6% i -1.27 1 -2.0% I -0.29 ! -2.5%
June 2015 -0.67 -1 .3% -0.67 I 0.6% i -0.06 ' -0.9% i ' i
July 2015 I -0.72 ! -1 .1% I -0.70 2.8% I -0.39 ! -3.2% I
August 2015 I -0.53 I -0.7% I 0.51 4.7% -1.07 ] -4.8%
September 2015 I -0.67 -1.5% I -1.88 ' -2.2% I -0.12 -3.3% ' '
October 2015 I -0.81 -2.1% I -3.72 I -8.0% I 0.26 -1 .3%
November 2015 l -1.15 ' -2.4% I -3.12 I -6.6% l -1.01 -3.6%
December 2015 I -1.81
! I -3.0% I -1.49 -3.7% I -1.21 i -3.1%
Observations ' 2,114,861 I
t-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs A-135
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 143 of 151
APPENDIX A OPOWER
Opower Group1 Avista Rebate Opower Group X
Group 1 Avista Rebate
Group1'2
R-squared I o.37
1 All bolded ps are significant at p'.S 0. I 0.
2 ps & percentages are for the interaction term, and the actual values for the Opower+Rebate group are the sum
of columns 2, 4, and 6 for ps and the sum of columns 3, 5, & 7 for percentages.
3 Cumulative lagged dependent variable regression model: Daily_average_kWh_usage =
Opower _participant(/J) + Avista_Rebate_participant (/J) + Opower_participant (/J) *Avista_Rebate_participant
(/J) +year_ month+ daily_ average_ kWh _preusage + e
4 Monthly lagged dependent variable regression model: Daily_average_kWh_usage =
([HER_participant_group(/J) + Rebate_participant_group(/J) +
HER _participant_group(/J) * Rebate _participant_group(/J)} by year_month) + year_ month +
daily_average_kWh_preusage + e
t-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs A-136
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 144 of 151
Cumulative
Model2:
Oct. 2013
Nov. 2013
Dec. 2013
Jan.2014
Feb. 2014
March 2014
April 2014
May 2014
June 2014
July 2014
August 2014
Sept. 2014
Oct. 2014
Nov. 2014
Dec.2014
Jan.2015
Feb. 2015
Feb. 2015
March 2015
April 2015
May 2015
June 2015
July2015
August 2015
APPENDIX A OPOWER
Table 2: Average Daily kWh Savings (13) from Cumulative and Monthly Lagged Dependent
Variable Group Comparison Models
Nonparticipants Nonparticipants Nonparticipants Opower-only Opower-only vs. Avista Rebate-
vs. Opower-only vs. Avista Rebate-vs. Opower+ vs. Avista Opower+Avista only vs. Opower+
Participants 1 only Participants 1 Avista Rebate Rebate-only Rebate Avista Rebate
Participants 1 Participants 1 Participants 1 Participants 1
p % p % p % p % p % p %
;
-0.90 -1 .7% -1.60 -2.2% -3.30 ! -6.5% -0.48 -0.1% ; -1.90 -4.0% -1 .53 -4.0%
Monthly Model3:
-0.85 1.8% -7.03 -17.9% -4.92 -10.7% -6.06 I -16.0% I -4.04 -8.9% 0.35 4.5%
-1 .16 2.0% -1.48 -6.3% -9.06 -17.8% I -0.54 -4.5% -7.70 -1 5.5% -6.45 -1 0.7%
-1.31 2.0% 0.01 -2.8% -10.51 -17.4% -2.37 -5.7% -6.82 -1 2.2% -8.26 -11 .9%
-1.13 1.7% -0.83 -2.0% -8.40 -14.3% -2.41 -4.0% -6.81 -12.0% -6.29 -11 .0%
-1.14 1.7% -0.82 -2.1% -10.44 -17.4% 0.28 -0.4% -9.14 -15.5% -8.12 -13.4%
-1 .00 1.7% -3.09 -4.6% -7.12 -1 3.6% -1 .88 -2.5% -4.59 -9.2% -3.68 -8.5%
-0.79 1.6% -1 .70 -2.4% -3.72 -8.2% -1 .08 -1 .2% -2.98 -6.6% -2.40 -6.6%
-0.58 1.5% -0.99 -1 .6% -2.81 -6.9% -0.22 0.3% -2.35 -5.7% -1 .83 -5.4%
-0.58 1.3% -1 .34 -1.5% -1.72 -3.3% -0.71 0.0% -1.08 -1.8% -0.38 -1.8%
-0.75 1.3% -0.37 2.3% -0.65 1.6% 0.36 3.5% 0.09 2.9% -0.26 -0.6%
-0.58 0.8% 0.80 5.6% 0.24 3.3% 2.03 7.9% 1.40 5.5% -0.62 -2.4%
-0.66 1.2% -0.69 0.2% -1.60 -2.4% -0.83 -0.7% -1.40 -2.4% -0.57 -1 .7%
-0.85 1.8% -3.03 -5.5% -3.43 -7.1% -2.07 -3.6% -2.48 -5.2% -0.40 -1.5%
-1.21 2.4% -1.41 -2.2% -3.26 -6.1% 0.05 0.5% -1.75 -3.4% -1.84 -3.9%
-1.60 2.8% -0.40 -0.6% -2.63 -4.9% 2.69 4.3% 1.59 1.6% -2.00 -4.0%
-1.55 2.8% 1.37 2.0% -1.05 -3.0% 1.63 2.9% 0.62 0.0% -1 .85 -4.2%
-1.24 2.5% -3.12 -5.3% -5.20 -9.8% -1.73 -2.6% -3.85 -7.2% -1.83 -4.1%
-1.24 2.5% -3.12 -5.3% -5.20 -9.8% -1 .73 -2.6% -3.85 -7.2% -1.83 -4.1 %
-1.16 2.4% -3.65 -6.6% -4.21 -8.7% -0.67 -0.7% -1.26 -2.9% -0.59 -2.2%
-0.97 .3% -2.04 -3.6% -2.51 -6.7% -0.93 -1 .1 % -1.53 -4.4% -0.08 -2.1%
-0.69 1.6% -1.34 -2.1% -2.25 -6.1% -0.57 -0.3% -1.58 -4.5% -0.98 -4.2%
-0.67 1.3% -0.79 0.3% -1.35 -1.6% 0.04 1.9% -0.74 -0.4% -0.62 -1 .9%
-0.72 1.1% -0.67 2.8% -1 .81 -1.5% 0.01 3.8% -1.09 -0.4% -1 .08 -4.1%
-0.53 0.7% 0.12 3.7% -1.46 -1 .7% 1.20 5.8% -0.40 0.3% -1.60 -5.5%
t-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs A-137
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 145 of 151
Sept. 2015
Oct. 2015
Nov. 2015
Dec.2015
Observations
R-squared
APPENDIX A OPOWER
Nonparticipants Nonparticipants Nonparticipants Opower-only Opower-only vs. Avista Rebate-
vs. Opower-only vs. Avista Rebate-vs. Opower+ vs. Avista Opower+Avista only vs. Opower+
Participants 1 only Participants 1 Avista Rebate Rebate-only Rebate Avista Rebate
Participants 1 Participants 1 Participants 1 Participants 1
13 % 13 % 13 % 13 % 13 % 13 %
-0.68 1.5% -1.79 -1 .9% -2.68 -7.0% -1.98 -2.6% -2.25 -6.1% -0.32 -3.5%
-0.81 2.1% -3.72 -7.9% -4.28 -11.4% -2.91 I -6.0% -3.46 -9.3% -0.58 -3.4%
-1.15 2.4% -3.25 -6.7% -5.39 -12.7% -1.94 -4.2% -4.09 -10.1 % -2.14 -5.9%
-1.81 3.0% -3.40 -6.4% -6.32 -12.5% 0.34 -0.6% -2.69 -6.8% -3.01 -6.1 %
2,067,403 450,317 478,045 1,636,816 1,664,544 47,458
0.37 0.50 0.49 0.38 0.38 0.42
1 All bolded ps are significant at p:S 0.10.
2 Cumulative lagged dependent variable regression model: Daily_average_kWh_usage = grouplvsgroup2(/J)
+year_ month + daily _average_ kWh _preusage + e
3 Monthly lagged dependent variable regression model: Daily_ average_ kWh_ usage = grouplvsgroup2(/J) by
year_ month +year_ month + daily_ average_ kWh_preusage + 8
t-'1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs A-138
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 146 of 151
Appendix B Program Logic Models
t-'1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs B-1
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 147 of 151
APPENDIX B PROGRAM LOGIC MODELS
Avista Nonresidential Natural Gas and Electric Program Logic Model
Data sources: Logi c model from the prior evaluation, program documentation, Avista staff
Program inputs: Prescriptive lighting, prescriptive non-lighting, and site-specific programs
Included in all Non-ResidentialPrograms Prescriptive Energy Smart Grocer
\,Veb site traffic;
newsletters;
ach·ertising
Established
target markets;
increased
program
awareness, EE
education
Regional and
national
market
transformation
advancing EE
equipment
(:::) I
Increased c ustoIIEr
program interelt &
established
customer
relationships
Increased c ustoIIEr
participation
Increased
program
penetration
Increased market
supply of EE
equipment
Output or
outcome
Planning,
reporting, and
Yeri fie at ion
Transparent
EM& V policies,
confirmation of
effective
operational
systems
Verified
program
sa,·ings and
optimum
program
performance
Outreachevente,
rebate
applica lions &
rebate paymen111
kWh and therm
savings; QA/ QC
accuracy of
records
Persistent
energy savings
CLEAReaults
trade ally
training;
Approval and
regiatntion;
Outreach
throughregioml
&industry
•
Established tl'1lde
ally relationahi~
increased
awareness l!t EE
education
•
Regional cross­
marketing of
EE programs;
regional&
market
transformation
.,.
Analysisof
energy use data
&customer
saving&; Rebate
applications &
payment«
incentives
i
Meallllre
installation l!t
energy saving&;
QA/QC accuracy
ofrecorda
•
Persistent
energy savings
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 148 of 151
APPENDIX B PROGRAM LOGIC MODELS
Avista Residential Natural Gas and Elctric and Electic-Only Program Logic Model
Data sources: Logic model from the prior evaluation, program documentation, Avista staff, Opower, JACO, and CLEAResult staff
Program inputs: Rebate programs (weatherization and shell, HVAC, conversions, etc.), Simple Steps Smart Savings, Behavior Home
Energy Reports, and Appliance Recycling*
Outreach
Community
partnerships,
partnerships with BPA
& NEEA, contracts
with implementers, &
leveraging contractors
Relationships
develop with
implementation
partners
Increased stocking &
promoti on of EE
technologies and
increased adoption of
EE building
techniques
C::) Output or
outcome
Marketing materials
& promotions,
Opower messaging,
Avista website,
customer service, &
outreach events
Customers
participate in rebate
programs, buy Avista
discounted EE
products, and adopt
EE behaviors
Increased program
penetrati on
•
Process Flow
• Appliance Recycling program was discontinued in June, 2015 .
Measure incentives
Increased customer
interest in EE eq. &
energy-sav mg
technologies
Immediate kW h and
therm savings
Persistent energy
savings
Evaluation,
meas urement, and
verification
Independent impact
& process evaluations
of residential
programs
Tracking of rebates,
quality assurance (not
all programs) to ensure
accuracy of records,
accounting, &
adherence to program
rules
Effecti veness of
program operati ons
confirmed
Program energy
savings verified
Optimum program
performance
maintained
8-3
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 149 of 151
Appendix C Survey Instruments
t-1Nexanr Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs C-1
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 150 of 151
APPENDIX C
.._...,Nexanr
SURVEY INSTRUMENTS
Process Evaluation of Avista's 2014-2015 Energy Efficiency Programs C-2
Exhibit No. 2
L. Roy, Avista
Schedule 2, Page 151 of 151