Loading...
HomeMy WebLinkAbout20201125Exhibit 1 - Avista 2018-2019 NG Impact Evaluations.pdfExhibit No. 1: Avista 2018 and 2019 Natural Gas Impact Evaluations Exhibit No. 1: Avista 2018 and 2019 Natural Gas Impact Evaluations PY 2018 Idaho Natural Gas Impact Evaluation Report June 1, 2019 Prepared for: Avista 1411 East Mission Avenue Spokane, WA 99202 Prepared by: Jeff Cropp Mitt Jones Christie Amero Rachel Fernandez Jon Lee i Table of Contents Portfolio Executive Summary ................................................................................................................ 1 Evaluation Methodology and Activities ................................................................................................. 1 Summary of Impact Evaluation Results ................................................................................................. 1 Conclusions and Recommendations ...................................................................................................... 2 Nonresidential Impact Evaluation ......................................................................................................... 4 Program Summary ................................................................................................................................. 4 Program Participation Summary ............................................................................................................ 4 Evaluation Goals and Objectives ............................................................................................................ 5 Nonresidential Impact Evaluation Methodology ................................................................................... 5 Nonresidential Evaluation Results ......................................................................................................... 7 Nonresidential Conclusions and Recommendations ............................................................................. 8 Residential Impact Evaluation ............................................................................................................. 10 Program Summary ............................................................................................................................... 10 Program Participation Summary .......................................................................................................... 10 Evaluation Goals and Objectives .......................................................................................................... 11 Residential Impact Evaluation Methodology ....................................................................................... 11 Residential Impact Evaluation Results ................................................................................................. 12 Residential Conclusions and Recommendations ................................................................................. 14 Low Income Impact Evaluation ........................................................................................................... 15 Program Summary ............................................................................................................................... 15 Program Participation Summary .......................................................................................................... 15 Evaluation Goals and Objectives .......................................................................................................... 15 Low Income Impact Evaluation Methodology ..................................................................................... 15 Low Income Impact Evaluation Results ............................................................................................... 16 Low Income Conclusions and Recommendations ................................................................................ 16 Fuel Efficiency Impact Evaluation ........................................................................................................ 17 Program Summary ............................................................................................................................... 17 Program Participation Summary .......................................................................................................... 17 Evaluation Goals and Objectives .......................................................................................................... 18 Fuel Efficiency Impact Evaluation Methodology .................................................................................. 18 ii Fuel Efficiency Impact Evaluation Results ............................................................................................ 19 Fuel Efficiency Conclusions and Recommendations ............................................................................ 20 Tables Table 1. PY 2018 Natural Gas Program Evaluation Activities ....................................................................... 1 Table 2. PY 2018 Reported and Gross Verified Natural Gas Savings ............................................................ 2 Table 3. Nonresidential Prescriptive Natural Gas Savings (PY 2018) ............................................................ 4 Table 4. Nonresidential Prescriptive Participation Goals by Equipment Rebated ....................................... 5 Table 5. Nonresidential Prescriptive Participation by Project (PY 2018) ...................................................... 5 Table 6. Nonresidential Site Specific Natural Gas Savings (PY 2018) ........................................................... 5 Table 7. Idaho Nonresidential Prescriptive Natural Gas Evaluation Sample ................................................ 7 Table 8. Idaho Nonresidential Site Specific Natural Gas Evaluation Sample ................................................ 7 Table 9. Nonresidential Prescriptive Natural Gas Impact Findings .............................................................. 8 Table 10. Nonresidential Prescriptive Evaluation Summary of Discrepancies ............................................. 8 Table 11. Nonresidential Site Specific Natural Gas Impact Findings (PY 2018) ............................................ 8 Table 12. Residential Prescriptive Reported Natural Gas Savings (PY 2018) .............................................. 10 Table 13. Residential Prescriptive Participation (PY 2018) ......................................................................... 11 Table 14. Multifamily Direct Install Program Reported Natural Gas Savings ............................................. 11 Table 15. Residential Prescriptive Database Review Natural Gas Impact Findings .................................... 13 Table 16. Residential Prescriptive Natural Gas Impact Document Review Realization Rates .................... 13 Table 17. Residential Prescriptive Natural Gas Impact Findings ................................................................ 14 Table 18. Low Income Reported Savings (PY 2018) .................................................................................... 15 Table 19. Low Income Participation (PY 2018) ........................................................................................... 15 Table 20. Low Income Natural Gas Impact Findings .................................................................................. 16 Table 21. Residential Prescriptive Fuel Efficiency Reported Natural Gas Savings (PY 2018) ...................... 17 Table 22. Residential Prescriptive Fuel Efficiency Reported Participation (PY 2018) ................................. 18 Table 23. Low Income Fuel Efficiency Reported Natural Gas Savings (PY 2018) ........................................ 18 Table 24. Low Income Fuel Efficiency Participation (PY 2018) ................................................................... 18 Table 25. Nonresidential Fuel Efficiency Natural Gas Impact Findings ...................................................... 19 Table 26. Residential Prescriptive Fuel Efficiency Natural Gas Impact Findings ........................................ 20 Table 27. Residential Prescriptive Fuel Efficiency Natural Gas Document Review Realization Rates ........ 20 Table 28. Low Income Fuel Efficiency Program Natural Gas Impact Findings ............................................ 20 1 Portfolio Executive Summary For several decades, Avista Corporation has been administering demand-side management programs to reduce electricity and natural gas energy use for its portfolio of customers. Most of these programs have been implemented in-house, but a few have external implementers. Avista contracted with Cadmus to complete process and impact evaluations of its PY 2018 and PY 2019 natural gas demand-side management programs in Idaho. This report presents our natural gas impact evaluation findings for PY 2018. Cadmus did not apply net-to-gross adjustments to savings values, except in cases where deemed energy savings values already incorporate net-to-gross as a function of the market baseline. Evaluation Methodology and Activities Cadmus conducted the Idaho portfolio evaluation using a variety of methods and activities, shown in Table 1. Table 1. PY 2018 Natural Gas Program Evaluation Activities Sector Program Document/ Database Review Verification/ Metering Site Visit Nonresidential Prescriptive (Multiple) ü ü Site Specific ü ü Residential Simple Steps, Smart Savings™ ü -- HVAC ü -- Shell ü -- ENERGY STAR® Homes ü -- Multifamily Direct Install ü -- Low Income Low Income ü -- Fuel Efficiency Site Specific (nonresidential) ü -- Prescriptive (Residential) ü -- Low Income ü -- Summary of Impact Evaluation Results Overall, the Idaho portfolio achieved a 100% realization rate and acquired 247,756 therms in annual gross savings (Table 2) Cadmus calculated the Avista reported savings through database extracts from Avista’s Customer Care and Billing (residential) and InforCRM (nonresidential) databases and from data provided by third-party implementers. We used the label verified savings for our findings. Following the end of the two-year evaluation cycle, Cadmus will conduct utility billing regression analyses to evaluate the most accurate energy savings for most residential programs. 2 Table 2. PY 2018 Reported and Gross Verified Natural Gas Savings Sector Reported Savings (therms) Gross Verified Savings (therms) Realization Rate Nonresidential 38,613 34,992 91% Residential 205,001 207,992 101% Low Income 5,185 4,772 92% Total 248,799 247,756 100% Conclusions and Recommendations During the course of the PY 2018 evaluation, Cadmus identified the following areas for improvement by sector. Nonresidential Conclusions and Recommendations The nonresidential sector achieved total verified natural gas energy savings of 34,992 therms in PY 2018 with a combined realization rate of 91%. The nonresidential sector fell short of the combined Prescriptive and Site Specific program paths’ natural gas savings goal of 79,605 therms by 56%. Cadmus has one recommendation for improving the nonresidential sector natural gas savings: • Revisit the Prescriptive ENERGY STAR food service equipment calculator workbook and review the default assumptions for hours of use and pounds of food cooked per day. During three food service project verifications, the feedback provided by site contacts for these calculator inputs differed significantly from the calculator default values. We also recommend adjusting future rebate application forms to ask for site-specific hours of use and load estimates. Cadmus will review the RTF calculation methods to determine whether the deemed RTF values are more appropriate for these measures. RTF savings values will be more consistent with regional savings estimates. Residential Conclusions and Recommendations Verified natural gas savings show a realization rate of 101% on savings of 205,978 therms for residential Prescriptive programs, which is 125% of the savings goal for the year. Reported savings for the Multifamily Direct Install (MFDI) program add 2,014 therms of savings, for a total of 207,992 therms in acquired savings. The HVAC program accounts for most verified residential natural gas savings—79%—followed by the Shell program with 19% of natural gas savings. Simple Steps, Smart Savings; MFDI; and ENERGY STAR Homes account for a combined 2% of savings, primarily through water-saving measures. Avista confirmed during evaluation that natural gas unit energy savings (UES) values for several measures throughout the portfolio mistakenly had not been updated to 2018 TRM values. Initially, the Shell natural gas program grossly unreported savings, which were based on 2017 TRM values. Under Avista direction, Cadmus adjusted reported savings for the Shell windows measures to use 2018 TRM values. 3 Cadmus offers three recommendations regarding Avista’s residential natural gas programs: • Ensure that reported savings on Prescriptive measures are calculated using current TRM UES values or RTF methods. For Simple Steps, Smart Savings showerhead measures, Avista has moved to an RTF methodology for PY 2019, which Cadmus will also adopt for its evaluation. • Continue to encourage installations of high-efficiency natural gas equipment through the HVAC program, which provides nearly three-quarters of natural gas savings for residential programs. The Northwest Energy Efficiency Alliance Residential Building Stock Analysis II estimates that roughly 70% of natural gas furnaces in Washington single-family homes and 50% in Idaho single- family homes have an annual fuel utilization efficiency (AFUE) rating under 90%, indicating plenty of remaining opportunity for savings. • Continue to emphasize windows measures through the Shell program, given their contribution of 19% of residential program path natural gas savings. Fuel Efficiency Conclusions and Recommendations Nonresidential Site Specific and Multifamily Market Transformation Fuel Efficiency measures achieved verified natural gas penalties of 10,441 therms, yielding an 87% realization rate. Residential Prescriptive Fuel Efficiency measures achieved verified natural gas penalties of 71,430 therms, yielding a 116% realization rate. Low Income Fuel Efficiency measures contributed natural gas penalties of 4,668 therms, with a realization rate of 115%. Residential Prescriptive natural gas measures more than offset the natural gas penalty of Residential Prescriptive Fuel Efficiency measures, with verified natural gas savings of 205,978 therms. Similarly, Low Income natural gas measures also more than offset the Low Income Fuel Efficiency natural gas penalties, with verified savings of 4,772 therms. Cadmus recommends that Avista adjust reported natural gas penalties on all residential Prescriptive and Low Income Fuel Efficiency measures to match current TRM values. 4 Nonresidential Impact Evaluation Through its nonresidential portfolio of programs, Avista promotes the purchase of high-efficiency equipment for commercial and industrial utility customers. Avista provides rebates to partially offset the difference in cost between high-efficiency equipment and standard equipment. Program Summary Avista completed and incented 40 nonresidential natural gas measures in Idaho in PY 2018 and reported total natural gas energy savings of 38,613 therms. Through the nonresidential sector, Avista offers incentives for high-efficiency equipment and controls through three program paths: Prescriptive, Site Specific, and Fuel Efficiency. The Prescriptive program path is selected for smaller, straightforward equipment installations that generally have similar operating characteristics (such as simple HVAC systems, food service equipment, and envelope upgrades). The Site Specific program path is reserved for more unique projects that require custom savings calculations and technical assistance from Avista’s account executives (such as compressed air, process equipment and controls, and comprehensive HVAC retrofits). Fuel Efficiency measures are part of the Site Specific program path, but they involve a combination of electric savings and natural gas penalties. These measures typically involve replacing electric space heating or water heating systems with natural gas equipment. Please refer to the Fuel Efficiency Impact Evaluation section for evaluation methodology and results discussion of the nonresidential Fuel Efficiency measures. Program Participation Summary This section summarizes nonresidential sector participation and progress toward PY 2018 goals through the Prescriptive and Site Specific program paths. Nonresidential Prescriptive Programs Table 3 shows natural gas energy savings goals assigned to Avista’s nonresidential Prescriptive programs for PY 2018 as well as reported savings and a comparison between reported savings and goals. Table 3. Nonresidential Prescriptive Natural Gas Savings (PY 2018) Program Type Savings Goals (therms) Savings Reported (therms) Percentage of Goal HVAC 8,054 3,956 49% Shell 9,400 1,149 12% Food Service Equipment 14,903 12,492 84% Energy Smart Grocera 6,248 0 0% Total 38,605 17,597 46% a The Energy Smart Grocer savings goal includes Site Specific Energy Smart Grocer measures. The Site Specific portion constitutes approximately 10% of the overall goal. 5 Table 4 shows participation goals by rebated equipment quantity, as provided by Avista. The PY 2018 nonresidential tracking database extract listed individual projects, but it did not include rebated equipment quantity. For reference, Table 5 provides participation by unique application numbers. Table 4. Nonresidential Prescriptive Participation Goals by Equipment Rebated Program Type Participation Goal HVAC 2,700 Shell 50,000 Food Service Equipment 23 Energy Smart Grocer N/A Table 5. Nonresidential Prescriptive Participation by Project (PY 2018) Program Type Participation Reporteda HVAC 11 Shell 4 Food Service Equipment 20 Energy Smart Grocer 0 Total 35 a A participant is defined as a unique application number. Nonresidential Site Specific Program Table 6 shows natural gas savings goals assigned to the Site Specific program path in Avista’s nonresidential sector for PY 2018, as well as reported savings. Note that the table does not include reported natural gas penalties for the Fuel Efficiency sector, such as those associated with the Multifamily Market Transformation program. Table 6. Nonresidential Site Specific Natural Gas Savings (PY 2018) Program Savings Goals (therms) Savings Reported (therms) Percentage of Goal Site Specific 41,000 21,016 51% Evaluation Goals and Objectives For the PY 2018 quarterly, semiannual, and annual reports, Cadmus conducted nonresidential impact activities to determine verified savings for most programs. Nonresidential Impact Evaluation Methodology To evaluate impact evaluation savings for the PY 2018 nonresidential sector, Cadmus performed several activities in two waves: • Selected an evaluation sample and requested project documentation from Avista • Performed project documentation review • Prepared on-site M&V plans 6 • Performed site visits and on-site data collection (such as trend data, photos, and operating schedules) • Used site visit findings to calculate verified savings by measure • Applied realization rates to total reported savings population to determine overall verified savings The program context, along with Cadmus’ sample design, document review, and on-site verification activities, is described in more detail below. Program Context As the first step of our evaluation activities, we gained an understanding of the programs and measures being evaluated. Specifically, Cadmus explored documents and data records: • Avista’s annual business plans, which detail processes and energy savings justifications • Project documents from external sources (customers, program consultants, and implementation contractors) Based on the initial review, Cadmus outlined the distribution of program contributions to the overall portfolio of programs. In addition, the review allowed us to understand the sources for UES for each measure offered through the programs, along with the sources for energy-savings algorithms and the internal quality assurance and quality control processes for large nonresidential program projects. Following this review, Cadmus designed the sample strategy for the impact evaluation activities, as discussed in the following section. Sample Design We based the first evaluation sample on program data from January 2018 to April 2018, and we based the second evaluation sample on program data from May 2018 through December 2018. As a guideline, Cadmus used the proposed, overall PY 2018 and PY 2019 nonresidential sample sizes by subprogram in the M&V plan, seeking to complete approximately one-quarter of the sample during the first wave and another one-quarter during the second wave. For each activity wave, we broke down submitted program applications by path and measure (such as Site Specific shell measure, Prescriptive HVAC), allowing us to select the highest-savings applications in each category with certainty. For applications with reported savings greater than 1% of total savings by category, Cadmus assigned random numbers and sampled randomly. We removed applications with less than 1% of total savings by category from the sample consideration, except where another application at the same location or facility was previously selected (and where we could assess both applications with one site visit, which is a cost-effective verification strategy even if the second application represents minimal claimed savings). Cadmus sampled randomly selected sites across both Washington and Idaho since Avista’s programs are implemented similarly in both states. We pooled the results from the randomly selected sites to calculate a realization rate by stratum that we applied to projects in both states. We applied verified savings for sites selected with certainty only to the state in which they had been implemented. 7 Table 7 summarizes the Idaho nonresidential Prescriptive program path natural gas evaluation sample. Across both states, Cadmus sampled 21 Prescriptive applications at 19 unique sites. Of the sampled applications, we selected five for certainty review based on the scale of savings, measure type, or location, and selected the remaining 16 applications randomly. Table 7. Idaho Nonresidential Prescriptive Natural Gas Evaluation Sample Program Type Applications Sampled Sampled Savings (therms) Percentage of Reported Savings HVAC 2 646 16% Shell 1 198 17% Food Service Equipment 4 3,761 30% Nonresidential Prescriptive 7 4,605 26% Table 8 summarizes the Idaho nonresidential Site Specific program path natural gas evaluation sample. Across both states, Cadmus sampled five Site Specific applications at five unique sites. Of the sampled applications, we selected four for certainty review based on scale of savings, measure type, or location, and selected the remaining application randomly. Table 8. Idaho Nonresidential Site Specific Natural Gas Evaluation Sample Program Applications Sampled Sampled Savings (therms) Percentage of Reported Savings Site Specific 3 19,750 94% Document Review Cadmus requested and reviewed project documentation for each sampled application and prepared M&V plans to guide the site visits. Project documentation typically included incentive applications, calculation tools (usually based on the 2017 Regional Technical Forum [RTF]),1 invoices, equipment specification sheets, and post-inspection reports. On-Site Verification Cadmus performed site visits at 23 unique nonresidential locations to assess natural gas energy savings for 26 unique Prescriptive and Site Specific measures (not including Fuel Efficiency measures). Site visits involved verifying the installed equipment type, make and model numbers, operating schedules, and setpoints, as applicable. Cadmus used the project documentation review and on-site findings to adjust the reported savings calculations where necessary. Nonresidential Evaluation Results This section summarizes the nonresidential sector Prescriptive and Site Specific program paths’ natural gas impact evaluation results for PY 2018. 1 Regional Technical Forum. 2017. Standard Protocols. https://rtf.nwcouncil.org/standard-protocols 8 Nonresidential Prescriptive Programs Table 9 shows reported and verified natural gas energy savings for Avista’s nonresidential sector Prescriptive program path and the realization rates between verified and reported savings for PY 2018. The overall nonresidential sector Prescriptive program path natural gas realization rate was 79%. Table 9. Nonresidential Prescriptive Natural Gas Impact Findings Program Type Reported Savings (therms) Verified Savings (therms) Realization Rate HVAC 3,956 3,956 100% Shell 1,149 1,149 100% Food Service Equipment 12,492 8,871 71% Nonresidential Prescriptive 17,597 13,976 79% Of the evaluated applications, Cadmus identified discrepancies for four based on the site visit and project documentation review (with one application having two discrepancies). Table 10 summarizes the reasons for discrepancies between reported and verified savings. Table 10. Nonresidential Prescriptive Evaluation Summary of Discrepancies Project Type Number of Occurrences Savings Impact Reason(s) for Discrepancy Food Service Equipment 3 ê • Cadmus reduced the pounds of food cooked per day for three fryer measures from the value in the savings calculator based on the site manager interview. 1 ê • Cadmus decreased operating hours for a fryer measure from the value in the savings calculator based on the site manager interview. 1 ê • Cadmus reduced the pounds of food cooked per day and operating hours for a steam cooker measure from the value in the savings calculator based on the site manager interview. Nonresidential Site Specific Program Table 11 shows reported and verified natural gas energy savings for Avista’s PY 2018 nonresidential sector Site Specific program path, as well as a comparison between verified and reported savings for PY 2018. The overall Site Specific program path natural gas realization rate was 100%. Note that the table does not include reported and verified natural gas penalties for measures in the Fuel Efficiency path. Cadmus did not identify discrepancies in any of the three evaluated applications. Table 11. Nonresidential Site Specific Natural Gas Impact Findings (PY 2018) Program Reported Savings (therms) Verified Savings (therms) Realization Rate Site Specific 21,016 21,016 100% Nonresidential Conclusions and Recommendations The nonresidential sector achieved total verified natural gas energy savings of 34,992 therms in PY 2018 with a combined realization rate of 91%. The nonresidential sector fell short of the combined Prescriptive and Site Specific program paths’ natural gas savings goal of 79,605 therms by 56%. 9 Cadmus has one recommendation for improving the nonresidential sector natural gas savings: • Revisit the Prescriptive ENERGY STAR food service equipment calculator workbook and review the default assumptions for hours of use and pounds of food cooked per day. During three food service project verifications, the feedback provided by site contacts for these calculator inputs differed significantly from the calculator default values. We also recommend adjusting future rebate application forms to ask for site-specific hours of use and load estimates. Cadmus will review the RTF calculation methods to determine whether the deemed RTF values are more appropriate for these measures. RTF savings values will be more consistent with regional savings estimates. 10 Residential Impact Evaluation Cadmus designed the residential sector impact evaluation to verify reported program participation and energy savings. We used data collected and reported in the tracking database, online application forms, Avista TRM and RTF savings review, and applicable updated deemed savings values. Program Summary Avista completed and incented 23,974 residential natural gas measures in Idaho in PY 2018 and reported total natural gas energy savings of 205,001 therms, not including participation and savings from Fuel Efficiency measures, which are included below in the Fuel Efficiency Impact Evaluation section. The residential programs comprise two primary paths—Prescriptive and MFDI. The Prescriptive path includes Simple Steps, Smart Savings, which encourages consumers to purchase and install high- efficiency showerheads and other equipment, such as LEDs and clothes washers; the residential HVAC program, which incents high-efficiency heating and cooling equipment; the residential Shell program, which provides rebates to encourage customers to install high-efficiency windows and storm windows; and the ENERGY STAR Homes program, which offers 15% to 25% energy savings relative to state energy code. Through the MFDI program, Avista provides free direct-install measures to multifamily residences (of five units or more) and common areas. Program Participation Summary This section summarizes residential sector participation and progress toward PY 2018 goals for the residential Prescriptive and residential MFDI programs. Residential Prescriptive Programs Table 12 shows savings goals assigned to Avista’s residential sector Prescriptive programs for PY 2018, as well as reported savings and the goal portion achieved in PY 2018. Reported savings for the Simple Steps, Smart Savings program achieved only 19% of goal, but an extremely high realization rate (see Table 17) brought verified savings much closer to goal. Table 12. Residential Prescriptive Reported Natural Gas Savings (PY 2018) Program Savings Goals (therms) Savings Reported (therms) Percentage of Goal Simple Steps, Smart Savings 2,328 445 19% HVAC 145,850 164,165 113% Shell 16,687 37,567 225% ENERGY STAR Homes 406 811 200% Residential Prescriptive Total 165,271 202,987 123% Table 13 summarizes participation goals and reported participation in Avista’s residential sector Prescriptive programs for PY 2018, along with the percentage of goal achieved. 11 Table 13. Residential Prescriptive Participation (PY 2018) Program Participation Goals Participation Reported Portion Achieved Simple Steps, Smart Savingsa 864 897 104% HVACb 1,825 2,080 114% Shellc 11,400 19,665 173% ENERGY STAR Homesb 2 2 100% Residential Prescriptive Total 14,091 22,644 161% a Participation is defined as the number of purchased units. b Participation is defined as the number of rebates. c Participation is defined as square feet of installed windows or storm windows. Multifamily Direct Install Program Table 14 shows reported savings and participation for the MFDI program in PY 2018. Avista launched this program as a pilot in PY 2018 and did not set annual program goals, then transitioned this from a pilot to an ongoing study in September 2018. Table 14. Multifamily Direct Install Reported Natural Gas Savings Program Savings Reported (therms) Participation Reported Multifamily Direct Install 2,014 1,330 Evaluation Goals and Objectives For the PY 2018 quarterly, semiannual, and annual reports, Cadmus verified savings for most programs through a combination of database review and document review, which are described below. Residential Impact Evaluation Methodology To determine the residential sector verified savings for PY 2018, Cadmus employed two impact evaluation methods for most residential programs:2 • Database review • Document review Similar to previous practice, Cadmus calculated adjusted savings based on results of the database review and applied realization rates for document reviews. Verified savings represented adjusted savings multiplied by the document review realization rates, as shown in Figure 1. 2 With approval from Avista, Cadmus ceased performing a third impact activity—verification surveys—in Q3 PY 2018 to eliminate redundancy between verification surveys and document review. 12 Figure 1. Residential Impact Process Database Review For the impact evaluation database review, Cadmus used UES values, as provided in the TRM, to calculate savings for measures reported in the measure tracking database. This impact activity may help identify incorrect UES values used to calculate reported savings. Savings calculated during the database review are defined as adjusted savings. Document Review For the document review, Cadmus compared information from rebate forms and other supporting documents to measure tracking data for a random sample of projects. This impact activity may identify installed measures that did not meet eligibility requirements, quantities that did not match the measure tracking database, and other discrepancies. Following the review of all projects, Cadmus calculated a realization rate for document review by dividing savings calculated for the sample (using the revised information) by reported savings for the sample. We then multiplied this realization rate by adjusted savings for the entire program to determine verified savings. Cadmus conducted 34 document reviews for the HVAC and Shell programs, drawing roughly equal samples from participants in each quarter. Residential Impact Evaluation Results The following sections summarize findings and provide verified savings for both of Cadmus’ impact evaluation methodologies. The database review resulted in the largest number of adjustments to reported savings. Database Review Table 15 shows database review findings, with adjusted savings being higher than reported savings for some programs and lower for others. Adjusted savings differed from reported savings because reported UES values differed from TRM values for several measures. In most cases, Avista determined that the reported savings for these measures used values from an older customer database that did not align with those in the current TRM. For measures with reported savings based on measure-specific 13 parameters, Cadmus could not confirm the reported savings calculations, which depended on inputs that were not included in the tracking data (such as air infiltration and duct sealing). Table 15. Residential Prescriptive Database Review Natural Gas Impact Findings Program Reported Savings (therms) Adjusted Savings (therms) Percentage Change Simple Steps, Smart Savings 445 2,202 395% HVAC 164,165 163,356 0% Shell 37,567 37,502 0% ENERGY STAR Homes 811 406 -50% Residential Prescriptive Total 202,987 203,466 0% Document Review Table 16 summarizes document review findings to date. The HVAC program had a 100% natural gas document review realization rate and the Shell program had a 107% natural gas document review realization rate. Table 16. Residential Prescriptive Natural Gas Impact Document Review Realization Rates Program PY 2018-PY 2019 Target Document Audit Count Document Audit Count Achieved to Date Sample Reported Savings (therms) Sample Verified Savings (therms) Document Audit Realization Rate HVAC 68 34 5,791 5,791 100% Shell 68 34 1,928 2,057 107% Cadmus identified several discrepancies during the document review through Q4 PY 2018: • For two window measures, documentation showed a square footage for installed windows that differed from that reported. In one case the documented square footage was higher than the reported, and in the other case it was lower. Cadmus adjusted savings based on the corrected area for both measures. • For two window measures reported at sites with electric heating, project documents identified the heating fuel as natural gas. Cadmus added natural gas savings and removed electricity savings at the sites. Table 17 shows verified savings, which apply the realization rates shown in Table 16 to the adjusted savings calculated based on the database review. The verified savings represent Cadmus’ best estimate of savings to date. With its high realization rate, the Simple Steps, Smart Savings program achieved 95% of goal based on verified savings, despite achieving reported savings of only 19% of goal. 14 Table 17. Residential Prescriptive Natural Gas Impact Findings Program Reported Savings (therms) Adjusted Savings (therms) Verified Savings (therms)a Realization Rates Simple Steps, Smart Savings 445 2,202 2,202 495% HVAC 164,165 163,356 163,356 100% Shell 37,567 37,502 40,014 107% ENERGY STAR Homes 811 406 406 50% Residential Prescriptive Total 202,987 203,466 205,978 101% a Verified savings represents adjusted savings only for Simple Steps, Smart Savings and ENERGY STAR Homes. Residential Conclusions and Recommendations Verified natural gas savings show a realization rate of 101% on savings of 205,978 therms for residential Prescriptive programs, which is 124% of the savings goal for the year. Reported savings for the MFDI program add 2,014 therms of savings, for a total of 207,992 therms in acquired savings. The HVAC program accounts for most verified residential natural gas savings—79%—followed by the Shell program with 19% of natural gas savings. Simple Steps, Smart Savings; MFDI; and ENERGY STAR Homes account for a combined 2% of savings, primarily through water-saving measures. Avista confirmed during evaluation that natural gas UES values for several measures throughout the portfolio mistakenly had not been updated to 2018 TRM values. Initially, the Shell natural gas program grossly unreported savings, which were based on 2017 TRM values. Under Avista direction, Cadmus adjusted reported savings for the Shell windows measures to use 2018 TRM UES values. Cadmus offers three recommendations regarding Avista’s residential natural gas programs: • Ensure that reported savings on Prescriptive measures are calculated using current TRM UES values or RTF methods. For Simple Steps, Smart Savings showerhead measures, Avista has moved to an RTF methodology for PY 2019, which Cadmus will also adopt for its evaluation. • Continue to encourage installations of high-efficiency natural gas equipment through the HVAC program, which provides nearly three-quarters of natural gas savings for residential programs. The Northwest Energy Efficiency Alliance Residential Building Stock Analysis II estimates that roughly 70% of natural gas furnaces in Washington single-family homes and 50% in Idaho single- family homes have an AFUE under 90%, indicating plenty of remaining opportunity for savings. • Continue to emphasize windows measures through the Shell program, given their contribution of 19% of residential program path natural gas savings. 15 Low Income Impact Evaluation Cadmus designed the Low Income programs’ impact evaluation to verify reported program participation and energy savings. We used data collected and reported in the tracking database and conducted a TRM savings review. Program Summary Avista leverages the infrastructure of a single Community Action Partnership agency to deliver energy effiicency programs for the company’s low-income residential customers in the Idaho service territory. The program is designed to serve Avista residential customers in Idaho whose income falls between 175 percent and 250 percent of federal poverty level. For PY 2018, the program achieved 5,185 therms reported natural gas savings in Idaho. Program Participation Summary Table 18 shows Avista savings goals for the Low Income sector for PY 2018 as well as reported savings and goal portions achieved in PY 2018. Table 18. Low Income Reported Savings (PY 2018) Program Savings Goals (therms) Reported Savings (therms)a Portion Reported Low Income 7,837 5,185 66% a Reported savings do not include Low Income Fuel Efficiency savings, shown in the Fuel Efficiency Impact Evaluation section. Table 19 summarizes participation goals for the Low Income programs, along with participation reported and achieved in PY 2018. Table 19. Low Income Participation (PY 2018) Program Participation Goalsa Participation Reporteda Portion Achieved Low Income 56,784 12,635 22% a Participation numbers do not include Low Income Fuel Efficiency participation, shown in the Fuel Efficiency Impact Evaluation section. Participation is defined as the number of installed units or square feet of installed insulation or windows. Evaluation Goals and Objectives For quarterly and semiannual reports in PY 2018 and PY 2019, Cadmus will determine verified savings for the Low Income programs through database review (described above in the Database Review section). This approach will provide a strong estimate of achieved savings until Cadmus can perform billing analysis at the end of the two-year evaluation cycle. Low Income Impact Evaluation Methodology Cadmus’ impact evaluation for the Low Income programs’ measures included a database review (described above in the Database Review section). We used UES values provided in the TRM to calculate savings for measures reported in the measure tracking database. Cadmus labeled savings calculated during the database review as adjusted savings. 16 Low Income Impact Evaluation Results Table 20 shows reported and adjusted natural gas savings for Low Income conservation measures. The table does not include savings for Low Income programs Fuel Efficiency path measures (shown in the Low Income Fuel Efficiency Impact Findings section below). Table 20. Low Income Natural Gas Impact Findings Program Reported Savings (therms) Adjusted Savings (therms) Verified Savings (therms) Realization Rate Low Income 5,185 4,772 4,772 92% Low Income Conclusions and Recommendations With a realization rate of 92% for natural gas savings, the Low Income programs achieved savings of 4,772 therms in PY 2018, or about 61% of the goal. Verified savings were less than reported savings because reported savings did not match the UES values listed in the Avista TRM. The 39% gap between verified savings and the goal results largely from relatively low program participation: Reported program participation reached 22% of the participation goal. Cadmus understands that Avista relies on Community Action Program agencies and tribal weatherization organization to deliver Low Income savings. Cadmus’ PY 2019 evaluation activities will include a process review of the Low Income programs, which may help identify opportunities to improve program performance. 17 Fuel Efficiency Impact Evaluation Cadmus designed the Fuel Efficiency sector impact evaluation to verify reported program participation and energy savings. We used data collected and reported in the tracking database and details from online application forms, as well as reviewed TRM and RTF savings and applicable updated deemed savings values. Program Summary Fuel Efficiency measures replace electric space heating or water heating systems with equipment using natural gas. These measures are offered within the nonresidential Site Specific path, residential Prescriptive programs, and Low Income programs. Across these programs, the Fuel Efficiency measures achieved reported participation of 190 in PY 2018 and a natural gas energy penalty of 77,852 therms. Fuel Efficiency measures provide positive electricity savings and negative natural gas savings, reflecting negative avoided costs. We report the electric energy savings in the PY 2018 Idaho Electric Impact Evaluation Report. Program Participation Summary This section summarizes Fuel Efficiency sector participation and progress toward PY 2018 goals for the nonresidential Site Specific path, residential Prescriptive programs, and Low Income programs. Nonresidential Site Specific Path The nonresidential sector Site Specific program path includes Fuel Efficiency measures that replace electric space heating or water heating systems with natural gas equipment. Fuel Efficiency measures provide positive electricity savings and negative natural gas savings, reflecting negative avoided costs. Three types of measures are considered Fuel Efficiency in the PY 2018 nonresidential sector database: • Site Specific HVAC combined • Energy Smart Grocer Site Specific case doors • Site Specific multifamily Only five Fuel Efficiency measures were incentivized in Idaho in PY 2018. Avista confirmed that it did not set natural gas participation goals for nonresidential Fuel Efficiency measures. Residential Prescriptive Programs Table 21 shows Avista PY 2018 natural gas savings goals for residential Prescriptive Fuel Efficiency measures as well as reported savings and percentage of goal through PY 2018. Table 21. Residential Prescriptive Fuel Efficiency Reported Natural Gas Savings (PY 2018) Program Savings Goals (therms) Reported Savings (therms) Percentage to Goal Residential Prescriptive Fuel Efficiency N/A -61,755 N/A 18 Table 22 shows the Avista PY 2018 participation goal and reported participation for residential Prescriptive Fuel Efficiency measures, as well as the participation percentage of goal through Q4 PY 2018. Table 22. Residential Prescriptive Fuel Efficiency Reported Participation (PY 2018) Program Participation Goalsa Participation Reporteda Percentage to Goal Residential Prescriptive Fuel Efficiency 271 170 63% a Participation is defined as the number of rebates. Low Income Programs Table 23 shows Avista PY 2018 natural gas savings goals for Low Income Fuel Efficiency measures, as well as reported savings and percentage of goal through PY 2018. Table 23. Low Income Fuel Efficiency Reported Natural Gas Savings (PY 2018) Program Savings Goals (therms) Reported Savings (therms) Percentage to Goal Low Income Fuel Efficiency N/A -4,042 N/A Table 24 summarizes participation goals for Low Income Fuel Efficiency measures, as well as participation reported and achieved through PY 2018. Table 24. Low Income Fuel Efficiency Participation (PY 2018) Program Participation Goalsa Participation Reporteda Percentage to Goal Low Income Fuel Efficiency 46 15 33% a Participation is defined as the number of rebates. Evaluation Goals and Objectives For quarterly and semiannual reports in PY 2018 and PY 2019, Cadmus will determine verified savings for nonresidential Site Specific and residential Prescriptive Fuel Efficiency measures through database review (described above in the Database Review section) and document review (described above in the Document Review section). For Low Income Fuel Efficiency measures, Cadmus will determine adjusted savings through database review. These approaches will provide strong estimates of achieved savings until Cadmus can perform billing analysis at the end of the two-year evaluation cycle. Fuel Efficiency Impact Evaluation Methodology The impact methodology for Fuel Efficiency measures is outlined below for the nonresidential Site Specific path, residential Prescriptive programs, and Low Income programs. Nonresidential Site Specific Fuel Efficiency Impact Methodology Cadmus followed the same impact evaluation methodology for Fuel Efficiency measures as outlined in the For the PY 2018 quarterly, semiannual, and annual reports, Cadmus conducted nonresidential impact activities to determine verified savings for most programs. 19 Nonresidential Impact Evaluation Methodology section. We sampled six Multifamily Market Transformation program projects for our evaluation of the nonresidential sector Fuel Efficiency measures, all of which were in Washington. Of the sampled applications, we selected five for certainty review based on scale of savings, measure type, or location, and selected the remaining application randomly. Cadmus performed site visits at five unique nonresidential locations to assess natural gas penalties for the six unique Multifamily Market Transformation program measures. Site visits involved verifying installed equipment type, make and model numbers, operating schedules, and set points, as applicable. Residential Prescriptive Fuel Efficiency Impact Methodology For our impact evaluation of residential Prescriptive Fuel Efficiency measures, we followed the methodology described in the Residential Impact Evaluation Methodology section and conducted database review and document review. We completed document reviews for 34 Fuel Efficiency participants in PY 2018. Low Income Fuel Efficiency Impact Methodology For our impact evaluation of Low Income Fuel Efficiency measures, we focused on a database review (described above in the Database Review section). We used unit savings values provided in the TRM to calculate savings for measures reported in the measure tracking database. Savings calculated during the database review are adjusted savings. For Low Income programs’ measures in general (including Low Income Fuel Efficiency measures), these savings are also considered verified savings. Fuel Efficiency Impact Evaluation Results The following sections summarize findings for the nonresidential Site Specific path, residential Prescriptive programs, and Low Income programs Fuel Efficiency measures. All Fuel Efficiency measures provide positive electricity savings and negative natural gas savings because these measures replace electric space heating or water heating systems with equipment that uses natural gas. Negative savings, reflecting negative avoided costs, are incorporated in the electric cost-effectiveness calculations. We report the positive electric savings in the PY 2018 Idaho Electric Impact Evaluation Report. Nonresidential Site Specific Fuel Efficiency Impact Findings Table 25 shows reported and verified natural gas penalties for Avista’s nonresidential sector Fuel Efficiency measures—along with realization rates—through PY 2018. Table 25. Nonresidential Fuel Efficiency Natural Gas Impact Findings Fuel Efficiency Measure Reported Savings (therms) Verified Savings (therms) Realization Rate Nonresidential Site Specific -2,701 -2,701 100% Multifamily Market Transformation -9,354 -7,740 83% Total -12,055 -10,441 87% Cadmus identified discrepancies in the randomly-sampled application based on the evaluation site visit and project documentation review. The site installed more efficient furnaces than reported, resulting in 20 lower natural gas energy consumption of the installed units versus baseline efficiency units and a reduced natural gas energy penalty. Residential Prescriptive Fuel Efficiency Impact Findings Table 26 shows reported, adjusted, and verified natural gas energy savings for the residential Prescriptive Fuel Efficiency measures. Table 26. Residential Prescriptive Fuel Efficiency Natural Gas Impact Findings Fuel Efficiency Measure Reported Savings (therms) Adjusted Savings (therms) Verified Savings (therms) Realization Rate Residential Prescriptive Fuel Efficiency -61,755 -71,430 -71,430 116% In reviewing documentation for 34 residential Fuel Efficiency measures, Cadmus found no issues that affected natural gas savings. This led to a document review realization rate of 100% for natural gas energy savings. Table 27 shows the natural gas results of our impact document review for residential Prescriptive Fuel Efficiency measures. Table 27. Residential Prescriptive Fuel Efficiency Natural Gas Document Review Realization Rates Fuel Efficiency Measure PY 2018-PY 2019 Target Document Audit Count Document Audit Count Achieved to Date Sample Reported Savings (therms) Sample Verified Savings (therms) Document Audit Realization Rate Residential Prescriptive Fuel Efficiency 68 34 -14,630 -14,630 100% Low Income Fuel Efficiency Impact Findings Table 28 shows reported and adjusted natural gas energy savings for Low Income Fuel Efficiency measures. Table 28. Low Income Fuel Efficiency Program Natural Gas Impact Findings Fuel Efficiency Measure Reported Savings (therms) Adjusted Savings (therms) Verified Savings (therms) Realization Rate Low Income Fuel Efficiency -4,042 -4,668 -4,668 115% Fuel Efficiency Conclusions and Recommendations Nonresidential Site Specific and Multifamily Market Transformation Fuel Efficiency measures achieved verified natural gas penalties of 10,441 therms, yielding an 87% realization rate. Residential Prescriptive Fuel Efficiency measures achieved verified natural gas penalties of -71,430 therms, yielding a 116% realization rate. Low Income Fuel Efficiency measures contributed natural gas penalties of 4,668 therms, with a realization rate of 115%. 21 Residential Prescriptive natural gas measures more than offset the natural gas penalty of residential Prescriptive Fuel Efficiency measures, with verified natural gas savings of 205,978 therms. Similarly, Low Income natural gas measures also offset of Low Income Fuel Efficiency natural gas penalties, with verified savings of 4,772 therms. PY 2019 Idaho Natural Gas Impact Evaluation Report May 29, 2020 Prepared for: Avista Corporation 1411 East Mission Avenue Spokane, WA 99202 i Table of Contents Portfolio Executive Summary ................................................................................................................ 1 Evaluation Methodology and Activities ................................................................................................. 1 Summary of Impact Evaluation Results ................................................................................................. 1 Conclusions and Recommendations ...................................................................................................... 2 Nonresidential Impact Evaluation ......................................................................................................... 4 Program Summary ................................................................................................................................. 4 Program Participation Summary ............................................................................................................ 4 Evaluation Goals and Objectives ............................................................................................................ 6 Nonresidential Impact Evaluation Methodology ................................................................................... 6 Nonresidential Evaluation Results ......................................................................................................... 8 Nonresidential Conclusions and Recommendations ............................................................................. 9 Residential Impact Evaluation ............................................................................................................. 10 Program Summary ............................................................................................................................... 10 Program Participation Summary .......................................................................................................... 10 Residential Impact Evaluation Methodology ....................................................................................... 11 Residential Impact Evaluation Results ................................................................................................. 14 Residential Conclusions and Recommendations ................................................................................. 17 Low-Income Impact Evaluation ........................................................................................................... 19 Program Summary ............................................................................................................................... 19 Program Participation Summary .......................................................................................................... 19 Low-Income Impact Evaluation Methodology ..................................................................................... 19 Low-Income Impact Evaluation Results ............................................................................................... 20 Low-Income Conclusions and Recommendations ............................................................................... 20 Fuel Efficiency Impact Evaluation ........................................................................................................ 22 Program Summary ............................................................................................................................... 22 Program Participation Summary .......................................................................................................... 22 Fuel Efficiency Impact Evaluation Methodology .................................................................................. 22 Fuel Efficiency Impact Evaluation Results ............................................................................................ 23 Fuel Efficiency Conclusions and Recommendations ............................................................................ 25 ii Tables Table 1. PY 2019 Natural Gas Program Evaluation Activities ....................................................................... 1 Table 2. PY 2019 Reported and Gross Evaluated Natural Gas Savings ......................................................... 1 Table 3. Nonresidential Prescriptive Natural Gas Savings ............................................................................ 5 Table 4. Nonresidential Prescriptive Participation Goals by Equipment Rebated ....................................... 5 Table 5. Nonresidential Prescriptive Participation by Project ...................................................................... 5 Table 6. Nonresidential Site Specific Natural Gas Savings ............................................................................ 5 Table 7. Idaho Nonresidential Prescriptive Natural Gas Evaluation Sample ................................................ 7 Table 8. Idaho Nonresidential Site Specific Natural Gas Evaluation Sample ................................................ 7 Table 9. Nonresidential Prescriptive Natural Gas Impact Findings .............................................................. 8 Table 10. Nonresidential Prescriptive Evaluation Summary of Discrepancies ............................................. 8 Table 11. Nonresidential Site Specific Natural Gas Impact Findings ............................................................ 9 Table 12. Residential Programs Reported Natural Gas Savings ................................................................. 11 Table 13. Residential Programs Participation ............................................................................................ 11 Table 14. Residential Programs Database Review Natural Gas Impact Findings ....................................... 15 Table 15. Residential Natural Gas Impact Document Review Realization Rates ........................................ 15 Table 16. Residential Programs Billing Analysis Results ............................................................................. 16 Table 17. Residential Programs Natural Gas Impact Findings .................................................................... 17 Table 18. Low-Income Reported Savings ................................................................................................... 19 Table 19. Low-Income Participationa .......................................................................................................... 19 Table 20. Low-Income Natural Gas Impact Findings .................................................................................. 20 Table 21. Avista Portfolio Fuel Efficiency Participationa ............................................................................. 22 Table 22. Nonresidential Fuel Efficiency Natural Gas Impact Findings ...................................................... 23 Table 23. Residential Fuel Efficiency Analysis Results ................................................................................ 24 Table 24. Residential Fuel Efficiency Natural Gas Impact Findings ............................................................ 24 Table 25. Low-Income Fuel Efficiency Program Natural Gas Impact Findings ........................................... 25 Figure Figure 1. Residential Impact Process .......................................................................................................... 12 1 Portfolio Executive Summary For several decades, Avista Corporation has administered demand-side management programs to reduce the electricity and natural gas energy use of its portfolio of customers. Avista contracted with Cadmus to complete process and impact evaluations of its program year (PY) 2018 and PY 2019 natural gas demand- side management programs in Idaho. This report presents Cadmus’ natural gas impact evaluation findings for PY 2019. Cadmus did not apply net-to-gross adjustments to savings values, except in cases where deemed energy savings values already incorporated net-to-gross as a function of the market baseline. Evaluation Methodology and Activities Cadmus conducted the Idaho portfolio evaluation using a variety of methods and activities, shown in Table 1. Table 1. PY 2019 Natural Gas Program Evaluation Activities Sector Program Document/ Database Review Verification/ Metering Site Visit Billing Analysis Nonresidential Prescriptive (Multiple) ü ü -- Site Specific ü ü ü Residential Simple Steps, Smart Savings™ ü -- -- HVAC ü -- ü Shell ü -- ü ENERGY STAR® Homes ü -- -- Multifamily Direct Install ü -- ü Multifamily Direct Install Supplemental Lighting ü -- -- Low-Income Low-Income ü -- ü Fuel Efficiency Site Specific (Nonresidential) ü ü -- Residential ü -- ü Low-Income ü -- ü Summary of Impact Evaluation Results The Idaho portfolio achieved a 78% realization rate and acquired 216,962 therms in annual gross savings (Table 2). Cadmus calculated Avista’s reported savings through database extracts from its Customer Care and Billing (Residential) and InforCRM (Nonresidential) databases and from data provided by third-party implementers. Table 2. PY 2019 Reported and Gross Evaluated Natural Gas Savings Sector Reported Savings (therms) Gross Evaluated Savings (therms) Realization Rate Nonresidential 36,965 33,271 90% Residential 238,129 179,759 75% Low-Income 3,828 3,932 103% Total 278,922 216,962 78% 2 Conclusions and Recommendations During the course of the PY 2019 evaluation, Cadmus identified several areas for improvement, outlined below by sector. Nonresidential Conclusions and Recommendations The Nonresidential sector achieved total evaluated natural gas energy savings of 33,271 therms in PY 2019, with a realization rate of 90%. The Nonresidential sector fell short of the combined Prescriptive and Site Specific program paths’ natural gas savings goal of 76,944 therms by 43%. Cadmus has one recommendation for improving the Nonresidential sector natural gas savings: • Revisit the Prescriptive ENERGY STAR food service equipment calculator workbook and review the default assumptions for hours of use and pounds of food cooked per day. During three food service project verifications, the feedback from site contacts for these calculator inputs differed significantly from the calculator default values. The team also recommend adjusting future rebate application forms to ask for site-specific hours of use and load estimates. Cadmus will review the Regional Technical Forum’s (RTF’s) calculation methods to determine whether the deemed RTF values are more appropriate for these measures. RTF savings values will be more consistent with regional savings estimates. Residential Conclusions and Recommendations Evaluated natural gas savings show a realization rate of 75% on savings of 179,759 therms for Residential programs, which is 82% of the savings goal for the year. The HVAC program accounts for most evaluated Residential natural gas savings, 88%, followed by the Shell program with 10% of natural gas savings. The Simple Steps, Smart Savings; Multifamily Direct Install; and ENERGY STAR Homes programs account for a combined 2% of savings, primarily through water-saving measures. Billing analysis results for natural gas furnace measures served as the biggest driver of the 75% realization rate for Residential savings, providing a measure-level realization rate of 69%. The Avista Technical Resource Manual (TRM) unit savings value of 102 therms appears to be based on a 2011 billing analysis of natural gas upgrades, which showed higher natural gas savings largely because roughly 10% of participants in the treatment group installed heat pumps along with a more efficient natural gas furnace; participants who installed a heat pump along with a furnace upgrade showed a sharp reduction natural gas usage, indicating that some heating load shifted to the heat pump. For PY 2019, Cadmus did not identify any participants who installed both a high-efficiency natural gas furnace and a heat pump. Billing analysis also found lower natural gas savings for storm windows and replacement windows than estimated by 2019 TRM values. Cadmus offers three recommendations for Avista’s Residential natural gas programs: • Adjust the Avista TRM to provide lower savings values for natural gas furnaces, replacement windows, and storm windows, based on the billing analysis conducted for this evaluation. The 3 billing analysis unit energy savings of 71 therms for the G Natural Gas Furnace measure and 0.37 therms per square foot for G Storm Windows with Natural Gas Heat and G Window Replc with Natural Gas Heat appear to provide more accurate estimates of savings than the current TRM values. • Continue to encourage installations of high-efficiency natural gas furnaces, which provided 65% of evaluated natural gas savings for Residential programs. The Northwest Energy Efficiency Alliance’s Residential Building Stock Analysis II estimated that roughly 50% of natural gas furnaces in Idaho single-family homes have an annual fuel utilization efficiency under 90%, indicating substantial savings opportunities remain. • Continue to emphasize installation of smart thermostats, which accounted for 12% of PY 2019 Residential natural gas savings. Billing analysis showed smart thermostats have a 104% realization rate with natural gas heating equipment. Fuel Efficiency Conclusions and Recommendations Nonresidential Site Specific Multifamily Market Transformation Fuel Efficiency measures achieved evaluated natural gas penalties of 16,813 therms, yielding an 99% realization rate. Residential Fuel Efficiency measures achieved evaluated natural gas penalties of 70,331 therms, yielding a 141% realization rate. Low-Income Fuel Efficiency measures contributed natural gas penalties of 1,535 therms, with a realization rate of 97%. Residential natural gas measures more than offset the natural gas penalty of Residential Fuel Efficiency measures, with evaluated natural gas savings of 179,759 therms. Similarly, Low-Income natural gas measures also more than offset the Low-Income Fuel Efficiency natural gas penalties, with evaluated savings of 3,932 therms. Cadmus recommends that Avista adjust reported natural gas penalties on all Residential Fuel Efficiency measures to match values determined through the billing analysis conducted for this evaluation, which appear to provide a more accurate estimate of savings than the 2019 TRM values. Based on billing analysis results for the Low-Income Fuel Efficiency measures as a whole, Cadmus also recommends adjusting reported natural gas penalties for those measures. 4 Nonresidential Impact Evaluation Through its Nonresidential program portfolio, Avista promotes purchases of high-efficiency equipment for commercial and industrial utility customers. By providing rebates, Avista partially offsets cost differences between high-efficiency and standard equipment. Cadmus conducted Nonresidential impact evaluation activities to determine program year (PY) 2019 evaluated savings for most programs; the team also conducted measurement and verification of Prescriptive and Site Specific projects across the full PY 2019 sample. Program Summary Avista completed and rebated 45 nonresidential natural gas projects in Idaho in PY 2019 and reported total natural gas energy savings of 36,965 therms. Through the Nonresidential sector, Avista offers incentives for high-efficiency equipment and controls through three program paths: Prescriptive, Site Specific, and Fuel Efficiency. The Prescriptive program path serves smaller, straightforward equipment installations that generally include similar operating characteristics (such as simple HVAC systems, food service equipment, and envelope upgrades). The Site Specific program path serves more unique projects, requiring custom savings calculations and technical assistance from Avista’s account executives (such as compressed air, process equipment and controls, and comprehensive HVAC retrofits). Multifamily Market Transformation measures involve a combination of electric savings and natural gas penalties. Typically, these measures include replacing electric space-heating or water-heating systems with natural gas equipment. The Fuel Efficiency Impact Evaluation section provides a discussion of the evaluation methodology and the results for Multifamily Market Transformation measures. Program Participation Summary This section summarizes Nonresidential sector participation and progress toward PY 2019 goals through the Prescriptive and Site Specific program paths. Nonresidential Prescriptive Programs Table 3 shows natural gas energy savings goals assigned to Avista’s Nonresidential Prescriptive programs for PY 2019, as well as reported savings and a comparison between reported savings and goals. 5 Table 3. Nonresidential Prescriptive Natural Gas Savings Program Type Savings Goals (therms) Savings Reported (therms) Percentage of Goal HVAC 26,221 11,257 43% Shell 1,826 5,830 319% Food Service Equipment 24,119 12,728 53% Energy Smart Grocera 8,134 0 0% Total 60,300 29,815 49% a The Energy Smart Grocer savings goal includes Site Specific Energy Smart Grocer measures. The Site Specific portion constitutes approximately 10% of the overall goal. Table 4 shows participation goals by rebated equipment quantity, as provided by Avista. The PY 2019 Nonresidential tracking database extract listed individual projects, but it did not include rebated equipment quantity. For reference, Table 5 provides participation by unique application numbers. Table 4. Nonresidential Prescriptive Participation Goals by Equipment Rebated Program Type Participation Goal HVAC 8,250 Shell 8,880 Food Service Equipment 45 Energy Smart Grocer N/A Table 5. Nonresidential Prescriptive Participation by Project Program Type Participation Reporteda HVAC 21 Shell 5 Food Service Equipment 17 Energy Smart Grocer 0 Total 43 a Participant is defined as a unique application number. Nonresidential Site Specific Program Table 6 shows natural gas savings goals assigned to the Site Specific program path in Avista’s Nonresidential sector for PY 2019, as well as reported savings. The table does not include reported natural gas penalties for the Fuel Efficiency sector, such as those associated with the Multifamily Market Transformation program. Table 6. Nonresidential Site Specific Natural Gas Savings Program Savings Goals (therms) Savings Reported (therms) Percentage of Goal Site Specific 16,644 7,150 43% 6 Evaluation Goals and Objectives For the PY 2019 quarterly, semiannual, and annual reports, Cadmus conducted Nonresidential impact activities to determine evaluated savings for most programs. Nonresidential Impact Evaluation Methodology As the first step in evaluating PY 2019 savings for the Nonresidential sector, Cadmus explored the following documents and data records to gain an understanding of programs and measures slated for evaluation: • Avista’s annual business plans, detailing processes and energy savings justifications • Project documents from external sources (such as customers, program consultants, or implementation contractors) Based on the initial review, Cadmus checked the distribution of program contributions with the overall program portfolio. The review provided insight into the sources for unit energy savings (UES) claimed for each measure offered in the programs, along with sources for energy-savings algorithms, internal quality assurance, and quality control processes for large Nonresidential sector projects. Following this review, Cadmus designed a sample strategy for impact evaluation activities, with Cadmus performing the following evaluation activities in two waves: • Selected evaluation sample and requested project documentation from Avista • Reviewed project documentation • Prepared on-site measurement and verification plans • Performed site visits and collected on-site data (such as trend data, photos, and operating schedules) • Used site visit findings to calculate evaluated savings by measure • Applied realization rates to total reported savings population to determine overall evaluated savings Sample Design Cadmus created two sample waves for PY 2019. Sample 1 included program data from January 2019 through June 2019, and sample 2 included program data from July 2019 through December 2019. As a guideline, Cadmus used the proposed, overall PY 2019 Nonresidential sample sizes by subprogram in the measurement and verification plan, seeking to complete approximately half of the sample in each wave. For each activity wave, Cadmus broke down submitted program applications by path and measure (such as Site Specific shell measure, Prescriptive HVAC), allowing the team to select the highest-savings applications in each category with certainty. For non-certainty applications, Cadmus assigned random numbers and developed a random sample. In some cases, the team sampled another application at the same location or facility previously selected (and where Cadmus could assess both applications with one 7 site visit). This was a cost-effective verification strategy even if the second application represented minimal claimed savings. As Avista similarly implements its programs in both states, Cadmus sampled randomly selected sites across both Washington and Idaho. The team pooled results from the randomly selected sites to calculate a realization rate by stratum and applied that realization rate to projects in both states. The team applied evaluated savings for sites selected with certainty only to the state in which they had been implemented. Cadmus applied evaluated savings for sites selected with certainty only to the state in which they had been implemented. Table 7 summarizes the Idaho Nonresidential Prescriptive program path natural gas evaluation sample. Cadmus sampled seven Prescriptive applications at seven unique sites in Idaho. Of the sampled applications, the team selected two for certainty review based on the scale of savings, measure type, or location, and selected the remaining five applications randomly. Table 7. Idaho Nonresidential Prescriptive Natural Gas Evaluation Sample Program Type Applications Sampled Sampled Savings (therms) Percentage of Reported Savings HVAC 3 2,528 22% Shell 1 3,920 67% Food Service Equipment 3 3,030 24% Nonresidential Prescriptive 7 9,478 32% Note: totals may not sum due to rounding. Table 8 summarizes the Idaho Nonresidential Site Specific program path natural gas evaluation sample. Cadmus sampled one Site Specific application at one unique site in Idaho. The sampled application was selected randomly. Table 8. Idaho Nonresidential Site Specific Natural Gas Evaluation Sample Program Applications Sampled Sampled Savings (therms) Percentage of Reported Savings Site Specific 1 6,724 94% Document Review Cadmus requested and reviewed project documentation for each sampled application and prepared measurement and verification plans to guide the site visits. Project documentation typically included incentive applications, calculation tools (usually based on the 2017 Regional Technical Forum [RTF]),1 invoices, equipment specification sheets, and post-inspection reports. On-Site Verification Cadmus performed site visits at eight unique nonresidential locations in Idaho and Washington to assess natural gas energy savings for eight unique Prescriptive and Site Specific measures (not including Fuel 1 Regional Technical Forum. 2017. Standard Protocols. https://rtf.nwcouncil.org/standard-protocols 8 Efficiency measures). Site visits involved verifying the installed equipment type, make and model numbers, operating schedules, and setpoints, as applicable. Cadmus used the project documentation review and on-site findings to adjust the reported savings calculations where necessary. Nonresidential Evaluation Results This section summarizes the Nonresidential Prescriptive and Site Specific program paths’ natural gas impact evaluation results for PY 2019. Nonresidential Prescriptive Programs Table 9 shows reported and evaluated natural gas energy savings for Avista’s Nonresidential Prescriptive program path and the realization rates between evaluated and reported savings for PY 2019. The overall Nonresidential Prescriptive program path natural gas realization rate was 88%. Table 9. Nonresidential Prescriptive Natural Gas Impact Findings Program Type Reported Savings (therms) Evaluated Savings (therms) Realization Rate HVAC 11,257 11,483 102% Shell 5,830 1,910 33% Food Service Equipment 12,728 12,728 100% Nonresidential Prescriptive 29,815 26,120 88% Note: totals may not sum due to rounding. Of the evaluated applications, Cadmus identified discrepancies for three based on the site visit and project documentation review. Table 10 summarizes the reasons for discrepancies between reported and evaluated savings. Table 10. Nonresidential Prescriptive Evaluation Summary of Discrepancies Project Type Number of Occurrences Savings Impact Reason(s) for Discrepancy Commercial HVAC 1 ↑ • Cadmus determined from on-site inspection that a furnace reported as 80 kBtu/hr on the application was actually a 100 kBtu/hr unit. The installation verification (IV) report only contained a distant photo of the unit and did not show the nameplate or confirm the capacity. Shell Measure 2 ↓ • Avista reported incorrect savings values for a shell insulation project due to an error in its new database software. Cadmus reviewed all prescriptive shell measures to confirm that only one project was affected by the bug. The team treated the affected project as a certainty project and evaluated savings using the typical savings calculator methodology. Nonresidential Site Specific Program Table 11 shows reported and evaluated natural gas energy savings for Avista’s PY 2019 Nonresidential Site Specific program path, as well as a comparison between evaluated and reported savings for PY 2019. The overall Site Specific program path natural gas realization rate was 100%. The table does not include reported and evaluated natural gas penalties for measures in the Fuel Efficiency path. Cadmus did not identify discrepancies in the evaluated application. 9 Table 11. Nonresidential Site Specific Natural Gas Impact Findings Program Reported Savings (therms) Evaluated Savings (therms) Realization Rate Site Specific 7,150 7,150 100% Nonresidential Conclusions and Recommendations The Nonresidential sector achieved total evaluated natural gas energy savings of 33,271 therms in PY 2019 with a combined realization rate of 90%. The Nonresidential sector fell short of the combined Prescriptive and Site Specific program paths’ natural gas savings goal of 76,944 therms by 43%. Cadmus has one recommendation for improving the Nonresidential sector natural gas savings: • Provide more thorough documentation with Avista IV reports. Cadmus recommends that all IV reports include basic information explicitly stating the quantity and type of equipment found and include clear photos of equipment nameplates. 10 Residential Impact Evaluation Cadmus designed the Residential sector impact evaluation to verify reported program participation and energy savings. The team used data collected and reported in the tracking database, online application forms, the Avista Technical Reference Manual (TRM) and RTF savings review, and applicable updated deemed savings values. Program Summary In PY 2019, Avista reported participation of 85,858 for the Residential natural gas program in Idaho, which resulted in reported natural gas savings of 238,129 therms. This did not include participation and savings from Fuel Efficiency measures. Participation was defined as installed pieces of equipment (such as a furnace or showerhead) for some measures and square feet of surface for others (such as wall insulation and replacement windows). The Residential program path includes several programs: • Simple Steps, Smart Savings, which encourages consumers to purchase and install high- efficiency lighting and showerheads. • Residential HVAC, which offers incentives for high-efficiency heating and cooling equipment. • Residential Shell, which provides rebates to encourage customers to install insulation and high- efficiency windows and storm windows. • ENERGY STAR Homes, which offers 15% to 25% in energy savings relative to the state energy code. • The Multifamily Direct Install (MFDI) program, which provides free direct-install measures to multifamily residences (five units or more) and common areas. • MFDI Supplemental Lighting, which revisited multifamily properties served by the MFDI program to install additional common area lighting. Program Participation Summary This section summarizes Residential sector participation and progress toward PY 2019 goals for the Residential programs. Residential Programs Table 12 shows savings goals assigned to Avista’s Residential sector programs for PY 2019, as well as reported savings and the goal portion achieved in PY 2019. All programs except Simple Steps, Smart Savings exceeded savings goals, based on reported savings, leading to an overall achievement of 109% for Residential programs. 11 Table 12. Residential Programs Reported Natural Gas Savings Program Savings Goals (therms) Savings Reported (therms) Percentage of Goal Simple Steps, Smart Savings 6,273 44 1% HVAC 199,183 208,904 105% Shell 9,911 23,095 233% ENERGY STAR Homes 67 471 703% Multifamily Direct Install 3,480 5,615 161% Multifamily Direct Install Supplemental Lighting N/A N/A N/A Residential Total 218,914 238,129 109% Table 13 summarizes participation goals and reported participation in Avista’s Residential programs for PY 2019, along with the percentage of goal achieved. Table 13. Residential Programs Participation Program Participation Goals Participation Reported Percentage of Goal Simple Steps, Smart Savingsa 907 164 18% HVACb 2,066 2,700 131% Shellc 66,934 81,850 122% ENERGY STAR Homesb 1 4 400% Multifamily Direct Installd 57 1,140 2000% Multifamily Direct Install Supplemental Lighting N/A N/A N/A Residential Total 69,965 85,858 123% a Participation is defined as the number of purchased units. b Participation is defined as the number of rebates. c Participation is defined as square feet of installed windows or storm windows. d Participation is defined as number of living units that received measures. Residential Impact Evaluation Methodology To determine the Residential sector evaluated savings for PY 2019, Cadmus employed a combination of three impact evaluation methods:2 • Database review • Document review • Billing analysis First, Cadmus calculated adjusted savings for each program, based on results of a database review. For the HVAC, Shell, and Fuel Efficiency programs, Cadmus also applied realization rates for the document 2 With approval from Avista, Cadmus ceased performing a third impact activity—verification surveys—in Q3 PY 2018 to eliminate redundancy between verification surveys and document review. 12 reviews. For these programs, Cadmus calculated prescriptive evaluated savings by multiplying adjusted savings by the document review realization rate, as shown in Figure 1. Figure 1. Residential Impact Process To provide, where practical, the most rigorous evaluation method, Cadmus analyzed consumption data for all available participants of the HVAC, Shell, and Fuel Efficiency programs. As described in more detail in the Billing Analysis section, the team applied billing analysis results to determine evaluated savings only for measures where savings could be isolated (that is, where a sufficient number of participants could be identified who installed only that measure) and where confidence and precision met specific targets. Program-level realization rates for the HVAC, Shell, and Fuel Efficiency programs incorporate billing analysis results for some measures. Database Review For the impact evaluation database review, Cadmus used UES values provided in the TRM to calculate savings from measures reported in the measure tracking database. Such impact activity may help identify incorrect UES values used to calculate reported savings. Cadmus defined savings calculated during the database review as adjusted savings. Document Review To conduct the document review, Cadmus compared information from rebate forms and other supporting documents to measure tracking data for a random sample of projects. This impact activity may identify installed measures that did not meet eligibility requirements, quantities not matching the measure tracking database, and other discrepancies. Following the review of all projects, Cadmus calculated a realization rate for the document review by dividing savings calculated for the sample (using the revised information) by reported savings for the sample. The team multiplied this realization rate by adjusted savings for the entire program to determine prescriptive evaluated savings for PY 2019. Cadmus conducted 51 document reviews for the HVAC and Shell programs, drawing roughly equal samples from participants in each quarter. Based on the low variation in document review results, these sample sizes easily met the target of ±10% relative precision at 90% confidence established for this evaluation activity. 13 Billing Analysis For the Residential sector, Cadmus conducted billing analysis using available natural gas and electricity consumption data from Avista for the HVAC, Shell, and Fuel Efficiency programs. Evaluating Simple Steps, Smart Savings program savings through billing analysis was not practical because participants of the midstream retail program were largely unknown. The ENERGY STAR homes program had too few participants to produce meaningful billing analysis results. With MFDI, Cadmus did not analyze natural gas consumption because it would have been impossible to separate lighting interactive effects from savings that resulted from installations of aerators and efficient showerheads. HVAC, Shell, and Fuel Efficiency Savings Estimates With the HVAC, Shell, and Fuel Efficiency programs, Cadmus eliminated the effects of multiple energy efficiency measures by including in the analysis only participants who installed one measure. With these programs, the goal was to provide average unit savings values at the measure level to ensure the most accurate values possible were used for evaluated savings and cost-effectiveness. Cadmus used the unit savings value provided by the billing analysis for a given measure when results for that measure met two requirements: the number of sites in the participant group was at least five, and the relative precision achieved was no greater than ±40% at the 90% confidence level. If results calculated using only Idaho participants met these requirements, the team used those results. If results based only on Idaho participants failed to meet the requirements, Cadmus used combined results for Idaho and Washington if those results passed. If no billing analysis results passed for a given measure, Cadmus applied the results of database review and document review to determine evaluated savings. Data Sources To conduct the consumption analysis, Cadmus used program measure tracking data provided by Avista, monthly electric and gas consumption data provided by Avista, and weather data (which included actual average daily temperatures for 13 weather stations in Idaho and Washington from the National Oceanic and Atmospheric Administration) for the billing analysis period). The team used zip codes to match daily heating and cooling degree days to respective monthly bill read dates. Additionally, Cadmus used typical meteorological year (TMY3) 15-year normal weather values from 1991–2005, obtained from the National Oceanic and Atmospheric Administration for the same weather stations, in assessing energy use under normal weather conditions. Participant and Comparison Group Designation Cadmus gathered data for a participant (treatment) group, composed of all HVAC, Shell, and Fuel Efficiency program participants with measures installed in 2018. This allowed enough pre- and post- consumption data to analyze the various measures’ effects. To isolate the impact of exogenous factors (such as energy rate changes, economic condition changes, and non-programmatic effects) on energy use, Cadmus used a quasi-experimental design that involved selection of a comparison group, composed of participants with installation dates in late PY 2019. Through this approach, the team compared the treatment group’s pre- and post-change energy use (assumed to capture the program treatment) to the comparison group’s change in energy use (reflecting 14 what would have happened absent the program). To ensure similarity between treatment and control groups, the team chose to use future participants as the comparison group because they would have similar qualifications and could be assumed to have not participated in energy efficiency prior to program treatment. Data Screening Starting with all HVAC, Shell, and Fuel Efficiency participants and the comparison group, Cadmus cleaned the data and screened for several criteria to identify final analysis samples. Data cleaning included performing account-level reviews of the pre- and post-period monthly consumption of all individual participants to identify anomalies (such as periods of unoccupied units) that could bias the results. Cadmus conducted the consumption analysis using participants who had not moved since participating and who had at least 10 months of pre- and post-period billing data. Cadmus applied several screens to remove anomalies, incomplete records, and outlier accounts. The following are examples of accounts excluded from the analyses: • Accounts missing records, prohibiting the team from merging participant program tracking data with consumption data • Accounts with low annual use in the pre- or post-period, such as less than 1,240 kWh annually • Customers with incorrect signs on Princeton Scorekeeping Method (PRISM) parameter estimates • Accounts with other extreme values, including vacancies in billing data (outliers), non-program- related heating or cooling system changes (such as added or removed heating or cooling loads), baseload equipment changes, or changes in occupancy. This included screening for accounts with large gaps in interval data, such as having zero consumption across multiple months. Analysis To estimate measure-level impacts, Cadmus employed a pre- and post-installation savings analysis using household-level PRISM models that accounted for differences in pre- and post-installation weather conditions. The team estimated the heating PRISM model using variable 45°F to 65°F heating bases in both the pre- and post-period for each customer. Residential Impact Evaluation Results The following sections summarize findings for each impact evaluation methodology and provide PY 2019 evaluated savings. Database Review Table 14 shows database review findings. Adjusted savings are slightly higher than reported savings for some programs and considerably lower for others. Adjusted savings differed from reported savings with ENERGY STAR Homes because the Avista TRM provides a value of zero therm savings for dual-fuel ENERGY STAR manufactured homes and because the tracking data used the higher 2018 TRM savings value for natural gas homes instead of the 2019 TRM value. For the MFDI program, Cadmus applied RTF 15 UES values for multifamily direct-install aerators that were much lower than the older values used by the implementer. Table 14. Residential Programs Database Review Natural Gas Impact Findings Program Reported Savings (therms) Adjusted Savings (therms) Percentage Change Simple Steps, Smart Savings 44 44 0% HVAC 208,904 207,889 0% Shell 23,095 23,103 0% ENERGY STAR Homes 471 67 -86% Multifamily Direct Install 5,615 4,296 -23% Multifamily Direct Install Supplemental Lighting 0 0 N/A Residential Total 238,129 235,398 -1% Note: totals may not sum due to rounding. Cadmus noted additional discrepancies between the measure-tracking data and TRM values, although these generally balanced each other out or had only a small effect on program-level adjusted savings. The following list shows a few examples of these discrepancies: • Measure-tracking data reported various unit savings values for smart thermostats other than the 2019 TRM value of 26 therms. • Some window replacement measures reported much lower savings values than provided in the 2018 and 2019 TRMs. • All PY 2019 high-efficiency natural gas water heater measures reported savings of 25 therms per unit rather than the 20.9 therms value provided in the 2019 TRM. • For several instances of installed Shell measures, apparent typos for square feet of windows or insulation or incorrect UES values caused discrepancies between reported and adjusted savings. Document Review Table 15 summarizes document review findings for measures installed from Q1 PY 2018 through Q2 PY 2019, which Cadmus applied to estimate PY 2019 savings. The HVAC program had a 100% natural gas document review realization rate, and the Shell program had a 105% natural gas document review realization rate. Table 15. Residential Natural Gas Impact Document Review Realization Rates Program Document Audit Count Sample Reported Savings (therms) Sample Evaluated Savings (therms) Document Audit Realization Rate HVAC 51 7,306 7,306 100% Shell 51 3,195 3,360 105% Cadmus’ document review (through Q2 PY 2019) identified several discrepancies: • For two window measures, documentation showed square footage for installed windows that differed from the reported window area. In one case, the documented area was higher than 16 reported; in the other case, it was lower. Cadmus adjusted savings based on the corrected window area for both measures. • For three window measures reported for sites with electric heating, project documents identified natural gas as the heating fuel. Cadmus added natural gas savings and removed electricity savings at these sites. Billing Analysis Table 16 shows measure-level billing analysis results, used when calculating natural gas savings. The participant count and relative precision for each measure easily met requirements established to ensure meaningful results, which required a participant count of at least five and a relative precision no greater than ±40% at the 90% confidence level. Table 16. Residential Programs Billing Analysis Results Measure 2019 Avista TRM UES (therms) na Pre-Installation Weather Normalized Usage (therms) Annual UES (therms) Realization Rate Relative Precision at 90% Confidence Participant State G Natural Gas Furnace 102.0 348 824 70.542 69% ±16% Idaho G Storm Windows with Natural Gas Heatb 0.6 11,901 732.323 0.367 61% ±34% Idaho G Window Replc With Natural Gas Heatb 0.6 11,901 732.323 0.367 61% ±34% Idaho G Smart thermostat Do-It-Yourself with Natural Gas Heatc 26.0 607 848 27.024 104% ±26% Idaho and Washington G Smart thermostat Paid Install with Natural Gas Heatc 26.0 607 848 27.024 104% ±26% Idaho and Washington a To provide unit savings values that align with TRM units (sq. ft.), this table presents participant count in sq. ft. of window surface for storm widow and replacement window measures. b Results shown represent combined analysis of storm window and window-replacement measures, to maximize relative precision. Separate results for each measure appeared similar c Results shown represent combined analysis of smart thermostats for do-it-yourself and contractor installation to maximize relative precision. Separate results for each delivery method appeared similar. Billing analysis results for natural gas furnace measures had a large impact on the realization rate for the HVAC program, and the Residential sector as a whole. The Avista TRM unit savings value of 102 therms appears to be based on a 2011 billing analysis of natural gas furnace upgrades across Avista programs in both states. Cadmus identified four reasons for the reduction in billing analysis results from 102 therms to 71 therms. The difference with the largest impact was that roughly 10% of participants included in the 2011 study installed an air source heat pump along with a more efficient natural gas furnace. Installation of a heat pump appeared to result in greater natural gas savings by shifting some of the heating load to the heat pump, based on the observed sharp reduction in natural gas consumption for these participants. 17 Other factors included a shift from normalizing temperature based on TMY2 weather data in 2011 to TMY3 weather data for the current study and using a floating degree day base temperature for the current study rather than the fixed base temperature of 65°F in 2011. In addition, in contrast to the 2011 value of 102 therms, which represented combined results for Idaho and Washington, the 71 therm unit savings value from the current billing analysis is specific to Idaho participants. The 2011 value for Idaho participants only was 100 therms. Billing analysis also provided relatively low natural gas savings for replacement windows relative to the 2019 TRM value of 0.6 therms per square foot of window area. For Idaho participants, the billing analysis estimated savings of 0.37 therms per square foot. Smart thermostats achieved strong savings, as estimated by the billing analysis. The billing analysis UES value of 27 therms provided a realization of rate of 104% relative to the TRM UES value of 26 therms. To provide participant counts high enough to support statistically significant estimates, Cadmus combined participants for the two measures, which distinguished between do-it-yourself installation and contractor installation. Because billing analysis results for only Idaho failed to meet the ±40% precision requirement, Cadmus based evaluated Idaho savings on the combined results for Idaho and Washington participants. Evaluated Savings To calculate evaluated savings, Cadmus used unit savings values determined through the billing analysis for the measures shown in Table 16. For the remaining measures, Cadmus applied the results of database review and, where applicable, the document review to evaluate savings for each measure. The analysis then rolled up measure-level evaluated savings to calculate evaluated savings and a realization Table 17. Residential Programs Natural Gas Impact Findings Program Reported Savings (therms) Evaluated Savings (therms) Realization Rates Simple Steps, Smart Savings 44 44 100% HVAC 208,904 157,894 76% Shell 23,095 17,458 76% ENERGY STAR Homes 471 67 14% Multifamily Direct Install 5,615 4,296 77% Multifamily Direct Install Supplemental Lighting 0 0 N/A Residential Total 238,129 179,759 75% Residential Conclusions and Recommendations Evaluated natural gas savings show a realization rate of 75% on savings of 179,759 therms for Residential programs, which is 82% of the savings goal for the year. The HVAC program accounts for most evaluated Residential natural gas savings, 88%, followed by the Shell program with 10% of natural gas savings. Simple Steps, Smart Savings; MFDI; and ENERGY STAR Homes account for a combined 2% of savings, primarily through water-saving measures. 18 Billing analysis results for natural gas furnace measures served as the biggest driver of the 75% realization rate for Residential savings, providing a measure-level realization rate of 69%. The Avista TRM unit savings value of 102 therms is based on a 2011 billing analysis of natural gas upgrades, which showed higher natural gas savings largely because roughly 10% of participants in the treatment group installed heat pumps along with a more efficient natural gas furnace; participants who installed a heat pump along with a furnace upgrade showed a sharp reduction natural gas usage, indicating that some heating load shifted to the heat pump. For PY 2019, Cadmus did not identify any participants who installed both a high-efficiency natural gas furnace and a heat pump. Billing analysis also found lower natural gas savings for storm windows and replacement windows than estimated by 2019 TRM values. Cadmus offers three recommendations regarding Avista’s Residential natural gas programs: • Adjust the Avista TRM to provide lower savings values for natural gas furnaces, replacement windows, and storm windows, based on the billing analysis conducted for this evaluation. The billing analysis unit energy savings of 71 therms for the G Natural Gas Furnace measure and 0.37 therms per square foot for G Storm Windows with Natural Gas Heat and G Window Replc with Natural Gas Heat appear to provide more accurate estimates of savings than the current TRM values. • Continue to encourage installations of high-efficiency natural gas furnaces, which provided 65% of evaluated natural gas savings for Residential programs. The Northwest Energy Efficiency Alliance’s Residential Building Stock Analysis II estimated that roughly 50% of natural gas furnaces in Idaho single-family homes have an annual fuel utilization efficiency under 90%, indicating substantial savings opportunities remain. • Continue to emphasize installation of smart thermostats, which accounted for 12% of PY 2019 Residential natural gas savings. Billing analysis showed smart thermostats have a 104% realization rate with natural gas heating equipment. 19 Low-Income Impact Evaluation Cadmus designed the Low-Income program impact evaluation to verify reported program participation and energy savings. Evaluation methods included a database review and billing analysis. Program Summary Avista leverages the infrastructure of a single Community Action Partnership agency to deliver energy effiicency programs for the company’s low-income residential customers in the Idaho service territory. The program is designed to serve Avista residential customers in Idaho whose income falls between 175% and 250% of federal poverty level. For PY 2019, the program achieved 3,828 therms reported natural gas savings in Idaho. Program Participation Summary Table 18 shows Avista savings goals for the Low-Income sector for PY 2019 as well as reported savings and goal portions achieved in PY 2019. Table 18. Low-Income Reported Savings Program Savings Goals (therms) Reported Savings (therms)a Percentage of Goal Low-Income 25,262 3,828 15% a Reported savings do not include Low-Income Fuel Efficiency savings, shown in the Fuel Efficiency Impact Evaluation section. Table 19 summarizes participation goals for the Low-Income programs, along with participation reported and achieved in PY 2019. Table 19. Low-Income Participationa Program Participation Goalsa Participation Reported Percentage of Goal Low-Income 154,647 3,303 2% a Participation numbers do not include Low-Income Fuel Efficiency participation, shown in the Fuel Efficiency Impact Evaluation section. Participation is defined as the number of installed units or square feet of installed insulation or windows. Low-Income Impact Evaluation Methodology Cadmus’ impact evaluation of the Low-Income programs’ measures included a database review (described above in the Database Review section). The team used UES values provided in the TRM to calculate savings for measures reported in the measure-tracking database and labeled savings calculated during the database review as adjusted savings. For many measures reported in the tracking database, notes indicated that savings were capped at 20% of consumption. When duplicating savings calculations using TRM values, Cadmus used the newly calculated value if it was less than the capped value, but used the capped value if the TRM value indicated greater savings. 20 Cadmus also conducted billing analysis for the Low-Income program, using all available natural gas consumption data for PY 2018 and PY 2019 program participants. Because of the relatively small number of Low-Income program participants, Cadmus was unable to isolate measure-level savings for the program (which are necessary for cost-effectiveness calculations). However, the billing analysis did provide savings estimates for the program as a whole that produced a point of comparison for evaluated savings, estimated using prescriptive methods. Low-Income Impact Evaluation Results Table 20 shows reported, adjusted, and evaluated natural gas savings for Low-Income measures. The table does not include savings for Low-Income Fuel Efficiency measures (shown in the Low-Income Fuel Efficiency Impact Findings section below). Table 20. Low-Income Natural Gas Impact Findings Program Reported Savings (therms) Adjusted Savings (therms) Evaluated Savings (therms) Realization Rate Low-Income 3,828 3,932 3,932 103% During the database and TRM review, Cadmus noted errors in the measure-tracking data, such as measures with little or no reported savings and some unit savings values that did not match TRM values. Overall, however, the errors largely canceled one another out, leading to the overall realization rate of 103%. The billing analysis estimated a realization rate of 112% for Low-Income natural gas savings, excluding homes that installed Fuel Efficiency measures, but participation was not high enough to allow for isolation from effects of other installed measures. Such isolation is necessary to provide valid measure- level savings, which are necessary to support cost-effectiveness calculations for each measure. Additionally, with relative precision of ±39% at the 90% confidence level, the billing analysis estimate has relatively large error bounds. Accordingly, while the 112% realization rate suggests that natural gas savings may be understated for the Low-Income program, Cadmus recommends adopting the more conservative interpretation that the result provides strong support for the 103% realization rate calculated for the Low-Income program. Low-Income Conclusions and Recommendations With a realization rate of 103% for natural gas savings, the Low-Income programs achieved savings of 3,932 therms in PY 2019, or about 15% of the goal. The 85% gap between evaluated savings and the goal results largely from relatively low program participation: reported program participation reached 2% of the participation goal, though some of the shortfall likely results from Avista using square feet to set participation goals for some measures that the TRM addresses on a per-project basis, such as air infiltration and duct sealing in gas-heated homes. 21 The Low-Income program measure-tracking data did not include adequate information to determine when savings values were appropriately capped. Cadmus recommends providing annual consumption for each measure in the tracking data, so that evaluation can include verifying that savings were capped at 20% of consumption for applicable measures. 22 Fuel Efficiency Impact Evaluation Cadmus designed the Fuel Efficiency sector impact evaluation to verify reported program participation and natural gas consumption impacts. Evaluation methods included a database review, document review, and billing analysis. Program Summary Fuel Efficiency measures replace electric space heating or water heating systems with equipment that uses natural gas. These measures are offered within the Nonresidential Site Specific path, Residential programs, and Low-Income programs. Across these programs, the Fuel Efficiency measures achieved reported participation of 160 projects in PY 2019 and a natural gas energy penalty of 88,679 therms. Fuel Efficiency measures provide positive electricity savings and negative natural gas consumption impacts, reflecting negative avoided costs. Cadmus reported the electric energy savings in the PY 2019 Idaho Electric Impact Evaluation Report. Program Participation Summary This section summarizes Fuel Efficiency sector impact in PY 2019 for the Nonresidential Site Specific path, Residential programs, and Low-Income programs. Table 21 shows Avista’s PY 2019 participation estimate and reported participation for Multifamily Market Transformation, Residential, and Low-Income Fuel Efficiency measures as well as achieved percentages of the estimate. Avista did not estimate Nonresidential sector participation outside of the Multifamily Market Transformation program. There were four Multifamily Market Transformation program participants and no Nonresidential Site Specific participants in PY 2019. Table 21. Avista Portfolio Fuel Efficiency Participationa Program Participation Estimate Participation Reported Percentage of Estimate Multifamily Market Transformation N/A 4 N/A Residential Fuel Efficiency 141 143 101% Low-Income Fuel Efficiency 30 13 43% a Participation is defined as the number of rebates. Fuel Efficiency Impact Evaluation Methodology This section presents the impact methodology for Fuel Efficiency measures included in the Nonresidential Site Specific path, Residential programs, and Low-Income programs. Nonresidential Site Specific Fuel Efficiency Impact Methodology Cadmus followed the same impact evaluation methodology for Fuel Efficiency measures as outlined in the Nonresidential Impact Evaluation Methodology section. The team randomly sampled one Multifamily Market Transformation program project in Washington for the evaluation of the Nonresidential sector Fuel Efficiency measures. Cadmus did not evaluate the single Nonresidential Site 23 Specific Combined HVAC application in the Idaho Fuel Efficiency program, but did evaluate several applications with the same measure category in the electric and gas Site Specific programs, and found realization rates of 100% on those projects. Verification site visits involved verifying installed equipment type, make and model numbers, operating schedules, and set points. Residential Fuel Efficiency Impact Methodology Cadmus applied billing analysis results to evaluate natural gas consumption impacts for all Residential Fuel Efficiency measures, using the methodology described previously in Billing Analysis. Cadmus also completed database review of all PY 2019 reported Residential Fuel Efficiency impacts as well as document reviews for 50 Fuel Efficiency participants from Q1 PY 2018 through Q2 PY 2019. Low-Income Fuel Efficiency Impact Methodology To evaluate natural gas consumption impacts for the Low-Income Fuel Efficiency measures, Cadmus conducted a database review (described above in the Database Review section) and billing analysis. The relatively low number of participants for the Low-Income program made it impractical for the billing analysis to isolate consumption impacts for specific measures. Using unit savings values provided in the TRM, Cadmus calculated natural gas consumption impacts for measures reported in the measure- tracking database. For Low-Income program measures in general (including Low-Income Fuel Efficiency measures), the evaluation relied on results from the database review to determine evaluated natural gas consumption impacts. Fuel Efficiency Impact Evaluation Results This section summarizes findings for Fuel Efficiency measures in the Nonresidential Site Specific path, Residential program, and Low-Income program. All Fuel Efficiency measures provide positive electricity savings and negative natural gas consumption impacts because the measures replace electric space- heating or water-heating systems with equipment that uses natural gas. Negative natural gas consumption impacts reflect negative avoided costs and are incorporated in the electric cost- effectiveness calculations. Cadmus reported positive electric savings in the PY 2019 Idaho Electric Impact Evaluation Report. Nonresidential Site Specific Fuel Efficiency Impact Findings Table 22 shows reported and evaluated natural gas penalties for Avista’s Nonresidential Fuel Efficiency measures, along with realization rates, through PY 2019. Table 22. Nonresidential Fuel Efficiency Natural Gas Impact Findings Fuel Efficiency Measure Reported Consumption Impacts (therms) Evaluated Consumption Impacts (therms) Realization Rate Multifamily Market Transformation (16,944) (16,813) 99% Total (16,944) (16,813) 99% Cadmus identified a minor discrepancy for the one randomly sampled application based on the evaluation site visit and project documentation review. The site installed more efficient furnaces than 24 reported, resulting in lower natural gas energy consumption of the installed units versus baseline efficiency units and a reduced natural gas energy penalty. Residential Fuel Efficiency Impact Findings Table 23 shows measure-level billing analysis results used when calculating PY 2019 natural gas consumption impacts. The participant count and relative precision for each measure easily met requirements established to ensure meaningful results, which required a participant count of at least five and relative precision no greater than ±40% at the 90% confidence level. The billing analysis found the additional natural gas usage to be notably higher than predicted by the 2019 Avista TRM values. Realization rates relative to 2019 TRM values ranged from a low of 112% to a high of 194%, indicating that 112% to 194% more natural gas was consumed than predicted by the TRM values. Table 23. Residential Fuel Efficiency Analysis Results Measure 2019 Avista TRM UES (therms) na Pre- Installation Weather Normalized Usage (therms) Annual UES (therms) Realization Rate Relative Precision at 90% Confidence Participant State E Electric To Natural Gas Furnace (275) 43 78 (449) 163% 13% Idaho E Electric To Natural Gas Furnace & Water Heat (420) 21 110 (565) 135% 20% Idaho E Electric To Natural Gas Wall Heatera (466) 10 - (520) 112% 21% Idaho E Multifamily Electric to Natural Gas Furnace and Water Heat (199) 20 - (386) 194% 10% Idaho and Washington a The 2019 Avista TRM does not include the E Electric to Natural Gas Wall Heater measure. The TRM value shown is taken from the 2018 Avista TRM. Table 24 shows reported, adjusted, and evaluated natural gas impact results for the Residential Fuel Efficiency measures. Based on the measure-level billing analysis results listed in Table 23, the evaluation calculated a 141% realization rate for evaluated natural gas consumption impacts for the Residential Fuel Efficiency path, meaning that the measures resulted in 141% of the natural gas usage reported in the measure tracking data. Table 24. Residential Fuel Efficiency Natural Gas Impact Findings Fuel Efficiency Measure Reported Consumption Impacts (therms) Adjusted Consumption Impacts (therms) Evaluated Consumption Impacts (therms) Realization Rate Residential Fuel Efficiency (50,028) (47,482) (70,331) 141% Adjusted consumption impacts, which reflects findings of database review, were different than reported impacts because of discrepancies between reported unit savings values and those in the 2019 TRM. 25 These adjustments largely offset one another. Because billing analysis produced valid estimates for all Residential Fuel Efficiency measures, adjusted impacts had no effect on evaluated consumption impacts. In reviewing documentation for 50 Residential Fuel Efficiency measures, Cadmus found no discrepancies affecting natural gas consumption impacts, resulting in a document review realization rate of 100%. Cadmus did not apply document review results to estimate evaluated impacts because billing analysis produced valid estimates for all Residential Fuel Efficiency measures; the 100% realization rate appears to indicate strong compliance with program requirements among contractors and participants. Low-Income Fuel Efficiency Impact Findings Table 25 shows reported and adjusted natural gas energy consumption impacts for Low-Income Fuel Efficiency measures. Table 25. Low-Income Fuel Efficiency Program Natural Gas Impact Findings Fuel Efficiency Measure Reported Consumption Impacts (therms) Adjusted Consumption Impacts (therms) Evaluated Consumption Impacts (therms) Realization Rate Low-Income Fuel Efficiency (1,585) (1,535) (1,535) 97% Adjusted and evaluated consumption impacts differed slight from reported impacts because of discrepancies between reported UES values and 2019 TRM UES values for some projects. The billing analysis estimated a realization rate of 200% for Low-Income Fuel Efficiency natural gas consumption impacts, with a relative precision of ±22% at the 90% confidence level. Participation was not high enough to support isolating consumption impacts at the measure level, which are necessary for calculating cost-effectiveness, but the results do indicate much greater natural gas fuel penalties for Low-Income Fuel Efficiency measures as a whole than indicated by 2019 Avista TRM values. This finding also supports the electric billing analysis finding that electric savings for Low-Income Fuel Efficiency measures are much higher than estimated by the 2019 Avista TRM (see PY 2019 Idaho Electric Impact Evaluation Report). Together, the electric and natural gas billing analysis results suggest a much greater heating load than indicated by TRM values, which is evident as the heating load shifts from electricity to natural gas. Fuel Efficiency Conclusions and Recommendations Nonresidential Site Specific and Multifamily Market Transformation Fuel Efficiency measures achieved evaluated natural gas penalties of 16,813 therms, yielding a 99% realization rate. Residential Fuel Efficiency measures achieved evaluated natural gas penalties of 70,331 therms, yielding a 141% realization rate. Low-Income Fuel Efficiency measures contributed natural gas penalties of 1,535 therms, with a realization rate of 97%. Residential natural gas measures more than offset the natural gas penalty of Residential Fuel Efficiency measures, with evaluated natural gas consumption impacts of 179,759 therms. Similarly, Low-Income 26 natural gas measures also more than offset the Low-Income Fuel Efficiency natural gas penalties, with evaluated consumption impacts of 3,932 therms. Cadmus recommends that Avista adjust reported natural gas penalties on all Residential Fuel Efficiency measures to match values determined through the billing analysis conducted for this evaluation, which appear to provide a more accurate estimate of consumption impacts than the 2019 TRM values. Based on billing analysis results for the Low-Income Fuel Efficiency measures as a whole, Cadmus also recommends adjusting reported natural gas penalties for those measures.