20240801Exhibit No. 4 - NEEA Response to ADM Evaluation.pdf

May 4, 2023 >>
neea
To: Theresa Drake, Customer Relations and Energy Efficiency Senior Manager, Idaho Power
Nicole Hydzik, Director of Energy Efficiency,Avista Utilities
From: Becca Yates, Executive Director and NEEA Executive Team
CC: Executive Committee of the NEEA Board of Directors
Thank you for the opportunity to respond to the data request dated April 7, 2023. We appreciate the
importance of the Idaho Public Utility Commission's questions and ensuring the Northwest Energy
Efficiency Alliance (NEEA) is delivering value back to Idaho ratepayers, commensurate with their investment
in NEEA and in alignment with NEEA's purpose and regional role.
NEEA's executive team carefully reviewed the evaluation findings and recommendations and included a
detailed response below. We shared the evaluation report and data request from Avista and Idaho Power,
along with our response, with the Executive Committee of the NEEA Board of Directors. We respectfully
request this response letter be submitted in full to the Idaho Public Utility Commission.
This memo is structured in two parts.The first, addresses the recommendations put forth in the letter from
Avista Utilities and Idaho Power dated April 7, 2023, and the second, is NEEA's detailed response to the
Evaluation of NEEA Impacts Allocated to Idaho Power Company and Avista Utilities Within the State of
Idaho, dated April 6, 2023, and submitted by:ADM ASSOCIATES, INC.
NEEA staff is pleased to see the evaluator found NEEA's overall savings for the five-year reporting period to
have a utility benefit cost-test (UCT) ratio of 5.68 for Idaho Power, 8.55 for Avista electric and 6.24 Avista
gas. While we agree with this overall finding, we feel that it is important to acknowledge the methodology
used in the evaluation is not consistent with how market transformation programs are evaluated.The
evaluation used conventional impact protocols intended for assessing conventional resource acquisition
programs. Evaluation of market transformation programs requires a different paradigm providing a
comprehensive framework to assess the long-term strategy, program design, implementation and
corresponding market response over the entire lifecycle of an efficiency innovation. NEEA uses the market
transformation measurement and evaluation framework to guide independent,third-party evaluations of
all our market transformation programs. Other regions, including California and Illinois, have also
recognized this essential difference and have established frameworks specifically for this purpose as well.
To make wise investment decisions, it is critical the appropriate measurement and evaluation protocols are
utilized.This issue is discussed in more detail below.
NEEA also has questions regarding several findings outlined in the report that do not appear consistent with
the information we provided directly to the evaluator or the significant amount of publicly available
documentation.The second part of this memo outlines these questions along with NEEA staff responses.
With that said, NEEA staff are confident we can meet the general requirements of the seven
recommendations articulated in your letter. However, Recommendation#9 will warrant further
investigation and consulting with a NEEA Advisory Committee. Our responses to your questions are
detailed in the table below.
Northwest Energy Efficiency Alliance
700 NE Multnomah Street, Suite 1300, Portland, OR 97232
503.688.5400 1 Fax [phone]
neea.org I [e-mail]

Recommendation Can NEEA Impact/Timeline
Comply?
1. Request NEEA to report annual Yes. Additional analytical labor costs associated with service
savings for Idaho via service territory territory reporting. NEEA's current service territory
methodology for each measure' approach will be used. Full implementation to start with
claimed by NEEA. reporting 2025 energy savings.
2. Request that annual savings reports Yes. Additional labor costs associated with Idaho-specific
include estimates of administrative reporting.This report will use the same service territory
costs, incentive costs, and non- allocation used in reporting of savings.The report will
incentive costs by service territory. provide an allocation of program administrative, incentive,
and non-incentive costs. Develop report format in 2024
with implementation in 2025.
4. Track progress for each code change Yes. Additional labor costs associated with Idaho-specific
relative to administrative dollars reporting. Develop report format in 2024 with
spent towards state level codes and implementation in 2025.
associated energy savings accrued by
each state-level code.
5. Detail measure-level values as Yes. None; matches current practice. In the savings workbook
accurately as possible, and that each provided annually to each funder, NEEA already enters all
field is completed in the workbook to data fields for active programs and measures within each
allow for year-over-year tracking of program for which we are reporting savings. NEEA
regional units, baseline units, references all the data sources in the workbook.
retirement units and unit energy
savings over time.
6. NEEA to distribute naturally occurring Yes. None. Per our current practice, NEEA distributes naturally
baseline units more equitably occurring baseline units equitably between local programs
between local program units and and total regional units today, and no further adjustment
total regional units. is recommended. NEEA staff would be happy to have a
discussion to further clarify and understand the concern.
8. Complete third-party influence Yes. None.This is part of NEEA's current practice today.
evaluations for all federal standards
claimed by NEEA, as well as any
future standards in which NEEA
hopes to claim savings for the future.
Using quantitative estimate of
influence calculate a naturally
occurring baseline for each standard.
9. Complete influence evaluations for Possibly, NEEA already conducts third-party evaluation of its energy
each code update to estimate NEEA's pending codes work. Since the current influence approach's
qualitative and quantitative influence further application to reported energy savings was recommended
towards the code update. discussion by CEAC, any changes would need to be discussed by that
with the Cost committee. Possible implementation in 2025 with cost
Effectiveness implications to be determined.
and Evaluation
Advisory
Committee
(CEAC).
Northwest Energy Efficiency Alliance -2 -

Detailed NEEA Responses to Findings and Recommendations
Following are NEEA questions and responses to the findings and recommendations of the Evaluation of
NEEA Impacts Allocated to Idaho Power Company and Avista Utilities Within the State of Idaho report;April
6, 2023;ADM Associates San Ramos, California.
The following section of this memo is organized to match "1.4 Findings and Recommendations" of the
Evaluation, highlighting where a misunderstanding of the Market Transformation approach may lead to
misleading or incorrect assumptions.
General Comments
Overall,the Evaluators found that that there were substantial energy savings resulting from the utilities
investments in NEEA over the five-year period 2017-2021 and that in aggregate investment in NEEA over
multiple years resulted in highly positive utility benefit-cost test ratio, shown in Tables 1-4, 1-5 and 1-6.
Going deeper, the evaluation found that the contribution to NEEA efforts for standards, and codes was
highly cost-effective across program years 2017 through 2021,with cost-benefit ratios ranging between
11.92 to 167.66, except for one code program in Avista Gas.
However,the Evaluators found that all efficiency measure efforts were not cost effective for all program
years,with cost-benefit ratios ranging between 0.0 and 0.7.
The evaluation also looked at the methodology used to allocate and report energy savings specific to Idaho
utilities participating in the alliance. Compared to an allocation methodology based on Idaho-specific
service territory,the evaluation determined that using funding share allocation methodology,
overestimated savings accrued out-of-state and underestimated savings accrued within Idaho.The
Evaluators used the ratio of estimated savings using service territory allocation to funder share allocation to
develop "realization rates"for individual measures. The evaluation found realization rates ranging from
under 100%to over 100%. However,the overall effect of this change revealed the collective utility
investment in NEEA remained cost-effective for Idaho Power electric, Avista electric, and Avista gas due to
codes and standards savings.
NEEA Response: NEEA staff are in general agreement with the overall finding that there were substantial
energy savings resulting from the utilities' investments and participation in the alliance over the five-year
period 2017-2021 and that, in aggregate, investment in NEEA over multiple years resulted in a highly
positive utility benefit-cost test score, shown in Tables 1-4, 1-5 and 1-6.
That said, NEEA notes that the methodology used in the report is inconsistent with the accepted practice of
evaluation for long-term strategic market transformation programs. The report used an approach that
isolates a single component of the market transformation implementation strategies deployed, or isolates a
truncated time horizon, during the market transformation program life cycle.The full life cycle
encompasses activities that range from emerging technology demonstration, market barrier removal, as
well as activities to advance codes and standards. NEEA documents these activities in an overall market
transformation logic model that is evaluated annually through market progress indicators. For any
efficiency opportunity that has an applicable code or standard, these elements are targeted from the
earliest strategy and implementation of the program.Conducting a cost-effectiveness analysis of a market
transformation initiative for only a portion of the program's life, or at an individual measure level,will lead
Northwest Energy Efficiency Alliance - 3 -

to results that may drive incorrect decisions. See NEEA's response to Recommendation#2 for further
explanation.
For example, the alliance (NEEA staff and regional utilities and efficiency program administrators) has been
advancing electric heat pump water heater technology through targeted interventions and incentive
programs since 2008. From the very beginning of this work, NEEA developed a market transformation
program that included elements of emerging technology development for cold-climates, retail and
distributor engagement, contractor training, consumer marketing, ENERGY STAR®specification
development, as well as deliberate collection and reporting of Northwest-specific cost and performance
data into the federal standards process to ensure that Northwest regional needs are incorporated into a
national Heat Pump Water Heather(HPWH) standard that is currently under consideration.
During the early years of HPWH emerging technology development, a cost-benefit analysis for any one year
would have looked infinitely non-cost-effective. Conversely, an annual view of the cost-benefit ratio
following federal standard implementation will likely look highly cost-effective;yet the standard could not
have been adopted without all the emerging technology and market development work that preceded it.
To address this challenge, NEEA uses a 20-year net-present value cost-benefit analysis that accumulates
costs and savings over the full span of market intervention activities that result in increases in market
adoption that, in turn, result in measurable energy savings. We believe this is the appropriate way to
represent the long-term nature of the benefits and costs of market transformation programs which seek to
achieve the full potential of the efficiency innovations targeted. This is illustrated below in our response to
Recommendation#2.
The segregation and separate cost-benefit analysis of the integrated market transformation is a
representation of a broader problem with the overall evaluation framework. The ADM team appears to
have used a standard resource acquisition program evaluation framework. Market Transformation
programs require a different framework for evaluation and energy savings assessment as they are
fundamentally different in their approach, implementation, and long-term nature. NEEA uses an evaluation
approach and savings assessment methodology that has been developed over the last 25 years, in
collaboration with Northwest stakeholders,that include regulatory commissions of all four states.This
comprehensive approach is consistent with other states that have engaged in market transformation
programs including California'and Illinois.'
NEEA staff also have questions regarding the approach and data ADM used to both determine realization
rates, as well as their service territory allocation method. See NEEA's response to Finding#9 below for
more information.
Section 1.4.1. Specific Comments by Findings:
• Finding#1: Utilities that fund NEEA can choose whether savings are reported by allocation share
methodology or service territory methodology.The allocation shares methodology overrepresents
'Building a Policy Framework to Support Energy Efficiency Market Transformation in California, Prahl,and Keating,
California Public Utilities Commission, October 2014. https://pda.energydataweb.com/#!/documents/1207/view
'2022 Illinois Statewide Technical Reference Manual for Energy Efficiency Version 10.0 Volume 4:Cross-Cutting
Measures and Attachments FINAL;Attachment C Framework for Counting Market Transformation Savings in Illinois,
September 24, 2021 https://icc.illinois.gov/downloads/public/2022%201L-
TRM%20Version%2010.0%20Volume%204%20Cross-Cutting%20Measures%20and%20Attachments%20(Final).pdf
Northwest Energy Efficiency Alliance -4 -

out-of-state and out-of-service territory savings across measures, codes, and standards while
simultaneously underrepresenting in-state and in-service-territory savings across measures, codes,
and standards. However,the service territory methodology accurately represents benefits directed
to Avista and Idaho Power customers within the state of Idaho.
NEEA Response: NEEA staff was unable to replicate all of the service territory allocations used to
compute the realization rates that ADM reported, presumably from the data NEEA provided. Not all
NEEA data has resolution that readily supports individual service territory reporting. Accordingly,
NEEA staff apply specific methodologies to analyze and allocate data when approximating service
territory level allocation. Our uncertainty of ADM's full understanding and application of the
appropriate methodology, or data, leaves NEEA staff with significant concerns with this finding.
We were not able to replicate the results reported by ADM using our own data and therefore
would need more discussion with ADM to understand their analysis and data used before we could
fully support this finding. Please see the NEEA response to Finding#9, regarding NEEAs concern
that the evaluator utilized incomplete and unverified data. For these reasons, as well as the
regional nature of the work and how technologies diffuse across markets, most alliance utilities
have chosen to report savings by funder share. However, NEEA staff will continue to offer and
provide either option upon request.
• Finding#2:The data NEEA utilizes to estimate net market savings is available at resolutions that
allow NEEA to estimate precise savings for each utility service territory.
NEEA Response: It is incorrect to state that NEEA has data at resolutions to estimate precise
savings by utility. We have data at varying levels of granularity for the programs in our portfolio,
and this data can have levels of variability each year, as we rely on market relationships to obtain
data and that can change over time for any given program.
"Net market effects" are a calculated category of savings and are dependent on the total market
adoption for a given year, a regionally forecasted estimate of the naturally occurring baseline, as
well as the information provided by utilities for their local program, any of which can vary at any
time, as far as granularity.
• Finding#3:The Evaluators found that the methodology in which savings were estimated across
measures were inconsistent. For some measures, service territory methodology was used, and for
others,funder share allocation methodology was used.
NEEA Response: As noted by NEEA staff and in the report, allocation method for reporting savings
to Avista changed, per their decision, in 2019 from service territory methodology to funder share
allocation methodology. NEEA would like to understand the specific areas of inconsistency ADM
found.
• Finding#4: NEEA prioritizes cost-effective savings in terms of regional benefit.Therefore, savings
and cost-effectiveness are distributed across the region evenly, despite observed distribution of
savings across states.Although this philosophy has merit, more precise estimates of utility-level and
program-level savings help NEEA's stakeholders relay relevant savings and cost-effectiveness
results to their respective regulatory commissions.This remains critical, due to some state-level
commission orders to pursue all cost-effective energy efficiency efforts.
Northwest Energy Efficiency Alliance - 5-

NEEA Response:This is an incorrect finding as NEEA does not distribute cost effectiveness across
the region, rather NEEA employs an assessment methodology at the regional level. NEEA does not
then distribute the outputs of this assessment across the region. NEEA's practice is to work
individually with each utility, per their request,to help relay savings and cost-effectiveness results
to their respective regulatory commissions.
• Finding#5:The interviews revealed that although the three parties fundamentally want to improve
energy efficiency and increase market adoption of emerging technologies,their preferred
approaches to this shared goal vary. Unlike the utilities, who strive to demonstrate the cost-
effectiveness of their initiatives and investments on an annual or bi-annual cycle, NEEA operates on
a five-year funding cycle, which is different than the typical annual or biannual utility planning
cycle.
NEEA Response: It is correct that NEEA operates on a five-year funding cycle. This funding
structure is designed intentionally to recognize the long-term nature of market transformation and
allow for consistency and continuity in the investments required to develop and execute on such
programs in our portfolio. Consistent with NEEA's regional portfolio management approach and the
long-term nature of market transformation, NEEA regularly reviews the benefits and costs of its
programs and portfolio to ensure a benefit-cost ratio greater than 1.The benefit-cost ratio
represents a long-term forecast across the full development lifecycle of transformation. The
assumptions used in the forecast for each market transformation program are reviewed annually
and relevant updates made as needed; this updated result is then reported annually. Given the
difference pointed out in the recommendation, we are unclear how ADM's finding is reflected in
the methodology used in this report and in the recommendations.
• Finding#6: NEEA's programs are designed with a broader constituency in mind than that of its
member utilities. While the Idaho utilities' programs are targeted to produce benefits for their
ratepayers,—NEEA is tasked with developing programs that need to consider what is best for the
entire four-state region.At its core, NEEA's ethos assumes that changes made in one state will
eventually spillover into another state and that eventually, regional change will be realized.
NEEA Response: NEEA agrees in general with this finding. However,we disagree with the final
statement of an assumption that changes in one state will "eventually spillover" into another state.
NEEA's purpose is to pool resources and share risks to transform the market for energy efficiency to
the benefit of all consumers in the Northwest, including customers of Idaho utilities.To accomplish
this goal, NEEA focuses its efforts on engaging decision-makers (e.g., manufacturers, retailers, and
distributors) in competitive markets that often cross utility service territories and state boundaries.
As a result, NEEA's market transformation work is designed to work and result in changes that
affect all four states in the region. Sometimes this happens in all four states simultaneously(e.g.,
federal appliance standards) and in other cases,the transformation happens over time across the
region. Regardless, NEEA works to ensure that market actors are engaged in all four states and that
the entire four-state region is transformed.There are examples (e.g., Building Operator
Certification, Heat Pump Water Heaters) where NEEA has continued to fund transformation
activities in Idaho and Montana to ensure that transformation that started in Washington and
Oregon continues in the eastern part of the region. This active engagement is a commitment to
ensuring that NEEA's purpose is fulfilled to all utilities and fundamentally differs from the
assumption of the spillover statement in this finding.
Northwest Energy Efficiency Alliance -6 -

• Finding#7: NEEA currently allocates code savings via funder share methodology, which estimates a
proportion of total NEEA funding to each utility based on number of electric retail customers and
overall load.Therefore, savings from code adoption in other states are in-part assigned to Idaho.
The Evaluators found that out-of-state code building savings are currently being attributed to Idaho
utilities.The Evaluators are skeptical that spillover from out-of-state code changes result in energy
savings within the state of Idaho.Although the barriers to code adoption from one state to the next
may be similar,there is no evidence to suggest that these learnings transfer to observable and
measurable savings. NEEA has stated that starting in 2022, code savings will be allocated via service
territory allocation.
NEEA Response: NEEA would like to understand this finding and its basis, especially the statement
by ADM that they are skeptical about spillover. NEEA has never claimed that building savings from a
code change would spill over into another state. Rather, until recently, code adoption in one state
influenced code adoption in an adjacent state, and/or the International Energy Conservation Code,
which is the basis for many state energy codes. Using the funder share allocation method, we saw
savings being allocated proportionate to the individual state's code over time. Prior to this
evaluation, NEEA decided to change its allocation method for codes, due to the significant changes
we were seeing across the states relative to policy and associated impact on codes timing and
stringency. NEEA no longer allocates energy savings from codes using a funder share allocation
method, and instead will use a state allocation methodology.
• Finding#8:The NEEA Cost Effectiveness Advisory Committee (CEAC) meets quarterly with the NEEA
objectives to provide space for discussion around results of recently completed evaluation,
progress of field studies, relevant updates to programs, and acceptance or questioning of NEEA
methodology towards calculation of savings.
NEEA Response: CEAC are public meetings, and NEEA encourages all stakeholders and interested
parties to attend.
Efficiency Measure Findings
• Finding#9:The Evaluators estimated verified Ex Post aMW for the efficiency measures to display
39%, 52%, and 0% realization rates for Idaho Power electric,Avista electric, and Avista gas savings
within the state of Idaho, respectively.The difference in claimed savings and verified savings is due
to the change to using service territory allocation rather than funder share allocation.The efficiency
measures category Ex Ante savings included savings for measures completed in Washington,
Oregon, and Montana—therefore,for some measures,the funder share allocation methodology
underestimated Idaho-specific savings while others overestimated out-of-state savings.The overall
effect of this change resulted in a lower than 100% realization rate.
NEEA Response: It appears that ADM used preliminary, incomplete, and unverified data in their
attempt to generate a service territory view of NEEA savings. NEEA has a standard practice of
ensuring that all data is verified and complete before making a final assessment and reporting of
savings by the allocation selected by the utility. To be responsive to ADMs request in early January,
NEEA staff provided access to data that was preliminary, incomplete, and unverified. Staff informed
ADM of this fact and asked that the data not be used because it was not fully updated or verified
and was not intended to be used in any reporting to the utilities.
Northwest Energy Efficiency Alliance - 7 -

Additionally, as stated in the NEEA Response for Finding#1, NEEA has concerns that ADM did not
properly employ the correct methodology or analysis required to estimate service territory level
savings.
• Finding#10:The database review revealed that a variety of fields (measure life, UES)were empty
across measure types due to lack of savings claimed for the measure,which made verification of
values difficult and complicates tracking of a measure progress overtime.
NEEA Response:ADM did not have access to any NEEA database; however,they did have access to
the data sets that populated savings reports for the funders and years requested. These data sets
come from a database that includes measures for which NEEA did not report energy savings for
these funders, and hence those cells were empty.
• Finding#11:The database review revealed that NEEA's current method for distribution of modeled
naturally occurring baseline units between local program and NEEA efforts is not reasonable.A
portion of energy-efficient technology sales are due to naturally occurring baseline. NEEA nets out
modeled naturally occurring baseline to avoid claiming savings for units that would have been sold
had no program or NEEA-effort been provided within the market. However,the method in which
these baseline units are netted out is not distributed equitably. For some measures, NEEA estimates
that a large proportion of local program units are baseline, and therefore a larger proportion of the
remaining net market effects is assigned to NEEA efforts.The Evaluators raise concern for this
assumption, as it is unlikely locally incentivized, rebated measures display the same free ridership
as non-incentivized measures in the region.
NEEA Response: As stated in response to Finding#10, it is important to clarify that ADM did not
have access to any NEEA database. NEEA disagrees with ADM's assertion that the distribution
method of baseline units NEEA uses is not reasonable or equitable. A foundational principle of
NEEA, as an alliance, is that local programs support market adoption and are therefore a part of the
market transformation taking place. NEEA's baseline market share estimate represents the
adoption that would have occurred without the intervention of NEEA or its funders' local programs,
so NEEA's approach assumes that a portion of the baseline market share applies to the local
incentive units.
Regarding the question of equitability, NEEA does not assign savings to NEEA efforts separately
from funder programs; rather, it measures the full market transformation savings from our
collective efforts (Co-created Savings) and reports that to funders.To help our funders avoid
double-counting, in our savings reports we net out local program savings.The remaining savings,
called net market effects savings are not representative of distinct NEEA efforts, nor do they reflect
attribution.
• Finding#12:The Evaluators reviewed the utilized UES via the Regional Technical Forum (RTF)
workbooks,field study data, and simulation analysis findings and note no large concerns with NEEA
UES methodology or market baseline assumptions.
NEEA Response: It is NEEA's practice to align with the RTF, unless we have newer data. NEEA
works closely with RTF and Council members to discuss any new information or methodology that
is pertinent to analyses done by both NEEA and the Council.
Northwest Energy Efficiency Alliance - 8 -

• Finding#13:The Evaluators found that NEEA calculates cost-effectiveness of its portfolio using the
total regional savings rather than the net market effects.The Evaluators determined that this
methodology raises concern, and the NEEA cost-effectiveness tests currently account for all
measure, standard, and code completions across the entire region, effectively double counting local
program savings and simultaneously claiming naturally occurring baseline savings. Because Avista
and Idaho Power calculate their own internal cost effectiveness tests, this finding does not impact
Idaho Power or Avista reporting. However, the Evaluators highlight this finding, as NEEA savings
allocation and cost allocation methods are not currently consistent with regulatory requirements.
NEEA Response: NEEA uses a Total Resource Cost (TRC) test which uses total regional costs and
total regional savings to assess the cost-effectiveness of both individual market transformation
programs as well as the portfolio as a whole.The TRC methodology is consistent with that used by
the Northwest Power and Conservation Council'and the principles embedded in the Northwest
Power Act.4 This approach looks at cost-effectiveness from a societal perspective inclusive of all
costs, including NEEA, consumer and local utility program administration costs, and benefits over a
long-term, 20-year planning horizon. As a regional organization, NEEA has used TRC as an
appropriate tool for making decisions regarding the pursuit of new market transformation
opportunities as well as managing the overall portfolio. The TRC methodology and application to
individual programs is reviewed by both the CEAC as well as by third-party evaluation contractors
who independently review each program.
NEEA staff refutes the statement in the finding regarding double counting of local program savings.
In fact, NEEA collects annual data on energy savings units claimed through local utility programs and
subtracts them from total market adoption units to avoid double counting of energy savings
reported to utilities and regulators. NEEA also deducts savings associated with the naturally
occurring baseline units.
It is also incorrect that NEEA includes all standard and code completions across the region in the
cost effectiveness assessment of the portfolio. Each program is assessed separately and may
include a code or standard associated with the program strategy. However not all savings
recognized in code or standard results are included in the portfolio cost effectiveness assessment.
NEEA recognizes that units adopted into code will have a different, quite often lower, unit energy
savings, since the baseline replacement unit in new construction code is typically higher performing
than a baseline replacement unit in retrofit practice, and so this higher baseline condition is netted
out.
Additionally,to be conservative in the portfolio-level benefit cost calculation, NEEA includes all of
the regional investment dollars in NEEA, including early lifecycle costs invested in programs, new
construction codes and federal and state standards work in development, regional building stock
assessments and all general administration,while only recognizing the benefits of a portion of the
programs in our portfolio that are currently funded, fully developed,vetted and in the market
development phase of their lifecycle.
'Cost-Effective Methodology(nwcouncil.org)
4 Northwest Power Act(USC numbered)(nwcouncil.org)
Northwest Energy Efficiency Alliance - 9 -

Given this, NEEA is unclear about the specific concern mentioned by ADM. NEEA has always been,
and remains, committed to providing necessary data to all NEEA funders in their individual
regulatory filings. NEEA recognizes that Idaho utilities are subject to a Utility Cost Test metric that
is distinct from the TRC metric that NEEA uses. NEEA is happy to address any concerns or
clarifications needed to support Idaho Power and Avista in their regulatory filings in Idaho.
Standards Findings
• Finding#14:The Evaluators estimated verified Ex Post aMW for the standards efforts to display 34%
and 50% realization rates for Idaho Power electric and Avista electric within the state of Idaho,
respectively. Avista gas did not claim any savings for standards.The difference between claimed
savings and verified savings is due to the change to using service territory allocation rather than
funder share allocation. A minor cause of discrepancy is due to corrected baseline units using
influence evaluation values.
NEEA Response: Please see response to Finding#9, as it applies to this finding as well. In addition, it
appears that savings from one standard,which was previously evaluated by a third-party evaluator
for influence and impacts across the states within the region,were removed. All standards in which
NEEA is reporting co-created savings have been evaluated and are available.
• Finding#15: NEEA contracts third-party evaluators to conduct "influence evaluations"for each
standard, which summarizes NEEA's overall qualitative and quantitative influence towards federal
standards updates. NEEA uses the quantitative assessment as an estimate of federal standards
naturally occurring baseline.The Evaluators found that some of these influence scores were not
integrated properly to estimate baseline units.The Evaluators also found more than half(13 of 25)
federal standard measures lack influence evaluations.
NEEA Response: NEEA does not agree with these two assertions:that influence scores were not
integrated properly and that reported energy savings lacked evaluations.The variance was either
due to rounding or a reduction of the percentage based on NEEA's participation (Fluorescent Lamp
Ballasts). There is one case (Pumps)where the assumptions contained a preliminary value and is to
be updated based on the final evaluation value. In cases where NEEA reported co-created energy
savings and ADM is stating no evaluation was conducted, they are mistaken. NEEA reviewed all
appliance standards for which we claimed co-created savings and confirmed evaluations are
available in each instance.
Lastly, it appears that the evaluator did not include any energy savings associated with a state
appliance standard that NEEA influenced named "Residential/Commercial Battery Chargers
Standards effective in Oregon in 2014.They claimed there was no evaluation conducted. A third-
party evaluation was conducted, and results reviewed by the Cost Effectiveness and Advisory
Committee in 2015.The report found that NEEA did influence the standard, and the majority of
battery chargers sold in Idaho, Oregon and Washington met this new standard. A copy of the
evaluation is posted on NEEA's website and can be found at https://neea.org/resources/2015-
battery-charger-standard-evaluation-for-neeas-non-adoptive-states.
Northwest Energy Efficiency Alliance - 10-

Code Findings
• Finding#16:The Evaluators estimated verified Ex Post aMW for the code efforts to display 137%,
125%, and 87% realization rates for Idaho Power electric,Avista electric, and Avista gas savings
within the state of Idaho, respectively.The difference between claimed savings and verified savings
is due to the change to using service territory allocation rather than funder share allocation.
Overall,the funder share allocation underestimated Idaho-specific code savings using the current
NEEA policy of claiming 100%code after code is implemented.
NEEA Response: NEEA staff was unable to replicate the realization rates ADM computed from the
data NEEA provided. NEEA staff agree that State Energy codes were understated during the
reporting time frame of this evaluation when using the funder share allocation methodology.
• Finding#17: Currently, NEEA does not complete third-party evaluations of NEEA"influence"
towards codes updates as is currently done for federal standards updates.Therefore, NEEA
currently claims 100%savings for code-built homes.As summarized in the standards influence
evaluations summarized in Table 3-35 of the report,NEEA influence towards standards ranges
between 2.6%and 61%. If codes are evaluated similarly, and portray a similar range of influence,
NEEA code savings could be significantly overrepresenting savings. NEEA's current policy is to
report 100%of code-built residential and commercial building savings (while integrating
compliance rates)for 10 years after the effective code update date. Currently, NEEA does not
maintain a model to estimate naturally occurring baseline over time, as it does for its energy
efficiency measures.The current NEEA methodology assumes that there would be a 10-year lag in
current residential and commercial building code if NEEA did not participate in code update efforts.
NEEA Response: NEEA would like to understand the basis for this finding. Specifically,what
rationale ADM found that state energy building codes should be treated the same as Federal
Appliance Standards. NEEA has provided a significant portion of the overall code support funding in
all four states over the last 25 years of NEEA's existence. NEEA's funding has also provided
representation for the four states in the model code development process (IECC) and affiliated
buildings standards processes (DOE,ASHRAE,AHRI). NEEA also conducts code compliance and new
construction practice assessments for each state,this provides the foundation for future code
development within the region and nationally. Given the continuous and substantive nature of
NEEA's work in the state and national model building codes,the CEAC developed and
recommended the use of a standardized influence factor of 10 years, essentially three normal code-
cycles, of acceleration of code provisions that NEEA has worked directly on. In addition, NEEA
evaluates code compliance in each of the four states and applies this to the savings it reports.
Finding#18:The Evaluators reviewed simulation model methodology used by NEEA to estimate
code savings and found that UES methodology for code savings do not present any concerns.
NEEA Response: No response.
Northwest Energy Efficiency Alliance - 11 -

1.4.2 Recommendations. Specific Comments by Recommendation.
• Recommendation#1:The Evaluators recommend Avista and Idaho Power request NEEA to report
annual savings via the service territory methodology for each measure claimed by NEEA for each
Idaho Power electric,Avista electric, and Avista gas. (Based on Finding#1,#2,#3)
NEEA Response: NEEA can report annual savings and an estimate of costs to each utility using
NEEA's current service territory approach. NEEA seeks to have data for each of its market
transformation initiatives (MTI)to achieve this level of reporting.There are times, where either the
market data is not available or cost prohibitive at that level of granularity, and in these
circumstances, NEEA will report at the level it has the best data available. On page 53, ADM agrees
with this by caveating—"if source data permits."Annual reports containing energy savings and
estimated costs are available in April,for the prior year, and forecasting at a service territory level is
not possible. NEEA has already committed to Idaho Power staff,to discuss the level of detail
available in a given year so that the content and final reported values for this methodology are
understood.There is additional NEEA analytical and administrative labor costs associated with this
recommendation.
• Recommendation#2:The Evaluators recommend that Avista and Idaho Power request annual
savings reports to include estimates of administrative costs, incentive costs, and non-incentive costs
by service territory.This will allow each utility to calculate more accurate cost-effectiveness tests
for each initiative to determine whether extension of funding is a viable option within each utility's
regulatory environment. (Based on Finding#4)
NEEA Response: See NEEA's response to Recommendation#1 regarding inclusion of cost data.
NEEA does not agree that calculating cost effectiveness on an annual basis for a long-term multi-
phase market transformation initiative (MTI) is accurate. Doing so could hinder advancement of any
emerging efficient technology and,therefore, not be used to influence a code or standards in later
years, once the market barriers were removed and market adoption was occurring.
Critical to managing MTIs is acknowledging the long-term horizon (10-20 years) of the program
investment and return on investment in the market. A resource acquisition (RA) approach, which is
a one-to-one transactional approach over a short time horizon (1-3 years), is limited in its value
beyond the immediate efficiency gain the purchase creates. A typical cost-effectiveness construct
for a RA program is a 1-to 2-year comparison of dollars in vs energy savings produced. In contrast,
an MTI is designed and implemented to make sustained changes in markets that ultimately result in
energy savings. For this reason,the MTI cost and benefit period should span a 20-year horizon.This
covers the full-term of market investment, response, and value realized (similar to an R&D model).
The early years of a MTI are high investment years requiring resources to identify, address, and
remove barriers in the market so that the system provides the energy efficiency options to the
consumer. In later years,when the barriers are reduced/removed and the MTI investment can
cease,the market has established consistent practices, the efficient option becomes widely
available in the supply chain and known to the consumer, and the data and market information can
be used to influence a change in an energy code or appliance standard, and the full value of the
investment is realized.The figure below illustrates the diffusion of innovation and how an MTI
investment inversely relates to the value delivered, in terms of sustained market adoption and
energy efficiency.
Northwest Energy Efficiency Alliance - 12-

Choosing a narrow time slice of an MTI for assessing cost-effectiveness will yield misleading results
as it will not capture the full scope of the MTI investment and value return phases. It will also not
support proper decision-making.
• Recommendation#3:The Evaluators recommend that NEEA work with utilities to accurately
produce service territory-level savings and to best serve each state's current regulatory
environment and utility's localized concerns. (Based on Finding#5)
NEEA Response: NEEA works closely with its funders to understand their concerns and their
regulatory environment. NEEA is happy to work with Idaho Power Company and Avista Idaho to
produce service territory reporting.
• Recommendation#4:The Evaluators recommend that NEEA track progress for each code change
relative to administrative dollars spent towards state-level codes and associated energy savings
accrued by each state-level code. With the 20-year market transformation in mind,the service-
territory-level savings will still accrue over the 20-year horizon, however, using this methodology,
actual market transformation effects of co-created savings will be more accurately tracked. (Based
on Finding#6,#7)
NEEA Response: NEEA can provide this information.There is additional NEEA administrative labor
costs associated with this recommendation.
• Recommendation#5:The Evaluators recommend that measure-level values are detailed as
accurately as possible, and that each field is completed in the workbook to allow for year-over-year
tracking of regional units, baseline units, retirement units, and unit energy savings values over time.
(Based on Finding#10)
NEEA Response: NEEA provides all detailed data annually as called out in recommendation#5. The
only missing data, as finding#10 states,were fields intentionally left empty due to NEEA not
reporting savings for that measure.
Recommendation#6 The Evaluators recommend that NEEA distribute naturally occurring baseline
units more equitable between local program units and total regional units. (Based on Finding#11)
NEEA Response: NEEA staff use a well-vetted, equitable approach to allocating baseline unit for all
funder reporting calculations, applying a weighted average across the region based on total
regional results and local unit acquisition by each funder in the alliance. This approach has been
reviewed extensively by CEAC. NEEA staff is unable to discern from the report exactly what
elements of this approach ADM finds inequitable. NEEA staff would be happy to discuss this further
ADM or Idaho power and Avista staff as appropriate.
• Recommendation#7: In the case that cost effectiveness tests are completed using NEEA- reported
savings,the Evaluators recommend that Avista and Idaho Power calculate cost-effectiveness using
net market effects rather than total regional savings,as is consistent with current regulatory
requirements to report gross savings that would not have occurred without program intervention.
(Based on Finding#13)
Northwest Energy Efficiency Alliance - 13-

NEEA Response: Assuming the cost effectiveness test referenced in this recommendation is a
utility cost test, NEEA recommends co-created savings be used in the analysis.As shared with ADM,
NEEA employs an accounting approach to segmenting energy savings for reporting. Co-created
savings represent what was achieved by both the local program effort and the regional market
transformation effort. "Net market effects" is a calculated category of savings, and it is the savings
that are remaining after both baseline and local program units are removed. The sole purpose of
this category is to not double count savings when reporting; it is not intended to assign attribution.
Additionally, NEEA recommends using a window of time that is representative of the full cycle of
costs and benefits, as benefits lag the cost outlay, and a narrowly defined time horizon can yield
misleading results. Please see NEEA's response to the Overall Finding and recommendation#2 for
more information.
• Recommendation#8:The Evaluators recommend that third-party evaluations are completed for
the federal standards claimed by NEEA, as well as any federal standards in which NEEA hopes to
claim savings for in the future. Using the quantitative estimate of NEEA influence,the Evaluators
recommend that NEEA calculate a naturally occurring baseline for each standard. (Based on Finding
#15)
NEEA Response: NEEA has been operating in full compliance of this recommendation. NEEA
conducts third-party evaluations of all federal standards in which NEEA sought to influence.These
evaluations are conducted after the standard has been adopted, and prior to reporting any energy
savings.
• Recommendation#9:The Evaluators recommend an evaluation is completed for each code update
to estimate NEEA's qualitative and quantitative influence towards the code update. (Based on
Finding#17)
NEEA Response:As outlined in finding# 17, NEEA's significant influence in the state building energy
code process is distinct from federal appliance standards and therefore we would like to
understand ADM's basis for this recommendation before suggesting any changes to process or
reporting. If there were a change, their most likely would be cost implications to NEEA.
Thank you again for the opportunity to respond.
Northwest Energy Efficiency Alliance - 14-