20200731Compliance Report.pdf

Avista Corp. 1411 East Mission P.O. Box 3727 Spokane, Washington 99220-0500 Telephone [phone] Toll Free 800-727-9170
July 31, 2020
Jan Noriyuki, Secretary
Idaho Public Utilities Commission 11331 W. Chinden Blvd. Bldg. 8, Ste. 201-A Boise, Idaho 83714
Re: Case Nos. AVU-E-18-12 and AVU-G-18-08 – Compliance Reports
Dear Ms. Noriyuki:
Avista Corporation, dba Avista Utilities (Avista or Company), submits the following
documents (collectively “Reports”), in compliance with Order No. 34647 in Case Nos. AVU-E-
18-12 and AVU-G-18-08:
1) Annual Conservation Review BPI Report Out, inclusive of Appendices A-D
2)Internal Audit DSM Review Report
3) EM&V Engagement Modification Report
4)Idaho Commission Staff Comments and Recommendations Implementation Report
These Reports are intended to deliver upon the eight action items addressed in Order No.
34647, and the associated Settlement Stipulation, in the above-referenced Cases.
If you have any questions regarding this filing, please contact Ryan Finesilver, Manager of
Energy Efficiency, at [phone].
Sincerely,
/s/Shawn Bonfield
Shawn Bonfield Sr. Manager of Regulatory Policy & Strategy
Avista Utilities [e-mail]
RECEIVED
2020 July 31, PM 4:27
IDAHO PUBLIC
UTILITIES COMMISSION
Annual Conservation ReviewBusiness Process Improvement (BPI) Report Out
Ryan Finesilver, Planning and Analytics Manager 7/31/2020
2 | P a g e
Contents Idaho Settlement Action Item ............................................................................................................................................................. 3 Overview ..................................................................................................................................................................................................... 3 Goals of the BPI workshop ................................................................................................................................................................... 4 BPI Process ................................................................................................................................................................................................. 4 Review BPI Purpose & Settlement Stipulation ....................................................................................................................... 4 Identify SIPOC – Suppliers, Inputs, Process, Outputs and Customers .......................................................................... 4 Process Overview – Current Process and Future Process ................................................................................................. 5 Opportunities Identified .................................................................................................................................................................. 6 Additional Steps Taken: Settlement Compliance .................................................................................................................. 6 Outcomes and Rolling Action Item List (RAIL) ...................................................................................................................... 7 Conclusion .................................................................................................................................................................................................. 9 Questions .................................................................................................................................................................................................... 9 Appendices ................................................................................................................................................................................................. 9
3 | P a g e
Idaho Settlement Action Item The “Annual Conservation Review BPI Report Out” is provided in response to Settlement Action Item #3 of the Stipulation and Settlement for Avista’s 2016‐2017 Electric and 2014‐2017 Natural Gas Energy Efficiency1 prudence application which states: “Hold one or more business process improvement (BPI) workshops, to be facilitated by Avista’s internal BPI experts, focused on Avista’s internal processes and staffing roles for compiling and verifying annual EM&V and information contained in Avista’s annual conservation reports by the second quarter of CY 2020. The recommendations of this workshop and a plan for realizing them, will be provided to Staff by August 1, 2020.” Through its investigation into the prudence of Avista’s 2016‐2017 electric and 2014‐2017 natural gas energy efficiency expenditures, the Idaho Public Utilities Commission’s (IPUC) Staff determined that Avista’s conservation reporting and supporting impact evaluation processes were inadequate. The resulting Settlement in this case2, served as a catalyst for the Company to initiate its own investigation into the overall efficacy and quality control of its energy efficiency reporting practices. To further guide Avista’s examination of its processes, IPUC Staff issued a set of written comments with eleven items for Avista to consider implementing and/or improving upon. These recommendations were incorporated into the final Settlement Stipulation, along with eight specified “Settlement Action Items” to be addressed by the Company. One such action item, facilitation of one or more BPI workshops, is intended to lay bare any inefficiencies or opportunities for development that exist in current procedures, and to provide a better path forward for Avista’s energy efficiency reporting activities.
Overview In March 2020, Avista’s Business Process Improvement (BPI) group facilitated a three‐day workshop to improve the annual conservation review process. Twelve individuals from eight different teams participated in the workshop, bringing forth a wide and diverse variety of perspectives and backgrounds. Participants in the workshop included the following individuals:
Ryan Finesilver, Planning & Analytics Manager
Leona Haley, Efficiency Program Manager
Greta Zink, Efficiency Program Manager
Matt Iris, Efficiency Engineer
Carlos Limon, Efficiency Engineer
Jaime Majure, Regulatory Policy Analyst
Mike Gump, Efficiency Planning Analyst
Meghan Pinch, Efficiency Planning Analyst
Angela Koker, Account Executive
Jen Pearson, HPI Program Manager
Bryan Powers, Business Process Improvement 1 Case Nos. AVU‐E‐18‐12 and AVU‐G‐18‐08 2 Approved via Order No. 34647
Photo 1 – Energy Efficiency BPI Working Group
4 | P a g e
Lisa Garrett, Renewable Energy Coordinator/Business Process Improvement
Goals of the BPI workshop
 To develop a new reporting format that addresses the specific comments found in Idaho Public Commission Staff’s “Issues and Staff Recommendations Regarding Avista’s DSM Programs” 3, while also being usable for reporting to other regulators
 Formalize processes and procedures for implementing recommendations and results from future third‐party Impact Evaluations and Process Evaluations
 Establish adequate employee involvement for the preparation, review and delivery of Annual Conservation Reports
 Develop a formal oversight structure for annual conservation review process
 Develop report validations and quality controls
BPI Process The BPI process focuses on understanding the “why” of a project’s purpose and breaks down process flows for identifying opportunities.
Figure 1: BPI Process

Review BPI Purpose & Settlement Stipulation The first step of the BPI process was to review the purpose of the workshop and identify and key areas that need to be addressed. For this step, the BPI team reviewed the 2016‐2017 Electric and 2014‐2017 Natural Gas Energy Efficiency Prudence Settlement documents including IPUC Staff’s comments and recommendations and began to establish the framework that would help to facilitate the BPI process. For reference, Avista has included the BPI project charter as Appendix A to this report.
Identify SIPOC – Suppliers, Inputs, Process, Outputs and Customers The second step of the BPI process is to identify the Suppliers, Inputs, Process, Outputs and Customers that are included or would potentially be included in the Energy Efficiency Reporting process. This helps to set 3 Case Nos. AVU‐E‐18‐12 and AVU‐G‐18‐08, Stipulation and Settlement, Attachment A
Review BPI Purpose & Settlement Create SIPOC Review Current Process Identify Opportunities Establish Future State Rolling Action Item List
5 | P a g e
the boundaries and the scope of the reporting process along with providing more clarity around the multiple parties to the project’s development. The BPI team identified 10 key customers of the Annual Conservation Report outside of its primary audience, the Idaho Public Utility Commission (Commission).
Figure 2 ‐ SIPOC

Process Overview – Current Process and Future Process Following the SIPOC, the BPI team mapped out the Current Process used by the Energy Efficiency team for preparing the Annual Conservation Report (ACR). The current process utilized inputs from Energy Efficiency Engineering, Program Managers, the Leadership Team, and the third‐party EM&V vendor in order to provide key information and to ensure accuracy in its reporting. Each functional area or department involved in the ACR process was identified by the BPI team (represented by yellow vertical sticky‐notes in Photo 2), then given a
swim lane (represented by color‐coded horizontal rows in Photo 2). This layout was then utilized to map out each identifiable step or action taken by each group, to further determine what contributions were being made by which participants along the ACR planning and reporting process. A key take‐away from having a visual representation of the current process was that it became evident that much of the report development and review was performed within the Energy Efficiency team (as illustrated by the pink sticky notes in Photo 2 above). It was noted that there are several resources available within the Company that could also contribute to the design of the report that would add value. The team then developed a Future process that includes more cross‐functional team utilization. In the future process, additional functional areas were identified as contributors to the ACP along with new process
Suppliers•Program Managers•Vendors•Engineers•Customers•Marketing•Idaho Commission•Washington Commission
Inputs•CCB•InforCRM•DSMc•Conservation Potential Assessment•Integrated Resource Plan•Evaluations•T&D Team
Process•Review Previous Reports/Orders•Gather all Data and Information•Compile New Report
Outputs•Report•Verified Savings•Tariff Rider Balances•Cost‐Effectiveness•Marketing Effects
Customers•IPUC•WUTC•Advisory Group•Shareholders•Customers•Internal Teams•Leadership•Vendors•Regulatory•Power and Gas Supply
Photo 2 – Avista’s ACR Process
6 | P a g e
tasks occurring in more functional areas. Account Executives, Engineers, Marketing and Program Managers will have increased roles as suppliers and will provide additional inputs into the reporting process. The future review component established also includes contributions finance and economics leads within the company. The future state process flow diagram has been included as Appendix B of this report. 
Opportunities Identified By laying out the steps in the current process and identifying the teams, functional groups and tasks, the BPI team identified 25 opportunities that will help to improve the reporting process. These opportunities were then prioritized based on the BPI methodology. Although the BPI methodology for ranking tasks traditionally focuses on the trade‐offs between importance and effort, for the purposes of this particular workshop, the team modified that process in order to ensure that the settlement stipulation requirements were given a higher priority.
Figure 3 ‐ Opportunities
Priority Opportunity 
1  Develop formal pre and post review process for orders & comments  
1  Track changes to be sure to include in report 
1  Share takeaways with DSM staff for internal audits, 3rd party impact eval, IPUC orders and comments 
1  Develop process for documenting exceptions or differences in #'s (evaluator vs. Avista) 
1  Validate vendor methodology‐ standardize agreed method if applicable 
1  Formalize Avista work papers development process (AVA) 
1  Develop RACI or other ownership/ role clarity tools 
1  Establish communication channel for sharing information w/ all levels 
1  Explore internal audit process 
2  Holistic program reporting 
2  Documentation of decisions (Captain's Log) 
2  Develop an annual program summary report template w/ content types 
2  Re‐vamp overall report format ‐ Avista 
2  Identify new internal report review team & review process / expectations 
2  Formalize process for implementing 3rd party program recommendations 
2  Formalize work papers review process 
2  Better share customer stories and our stories 
2  Update calendar w/ key dates 
3  Evaluate more robust inclusion of information in report 
3  Discuss report value w/ WUTC & evaluate opportunity to align report information for both states. 
3  Evaluate opportunities to create acceptable levels of oversight with vendors 
3  Program Evaluation Training (Stats) 
3  Share Best Practices w/I team 
3  Story of the processes 
3  Report Cycle Kickoff meeting 
Additional Steps Taken: Settlement Compliance Where the traditional BPI methodology aims only to prioritize opportunities, the BPI team took an additional step to cross‐reference the current approach with these identified opportunities to address every item within the IPUC comments. In order to accomplish this, the opportunities went through a secondary process to link
7 | P a g e
each opportunity to a relevant IPUC Staff comment or recommendation. The mapping also helped the team further refine priority opportunities. Below is an example of this cross‐referencing:

Figure 4 – Settlement Compliance Example
Priority IPUC Settlement Opportunity 
1  2A,2B,4A,4B,8C,5A 
Develop process for documenting exceptions or differences in #'s (vendor vs 
Avista) 
1  1B,2A,2B,4A,5A,6A,8B  Validate vendor methodology‐ standardize agreed method if applicable 
1  6A,7A,8A,8C  Formalize Avista work papers development process (AVA) 
Outcomes and Rolling Action Item List (RAIL) The last step in the BPI process is to use a tool that will help track the various opportunities and actions identified in the BPI and to set timelines of when those changes will be made. In order to accomplish this, the BPI team developed its RAIL (Rolling Action Item List), which creates a list of action items for the various opportunities identified in the BPI process. Moreover, the RAIL provides a template for tracking the team’s progress towards accomplishing the action items and identifies key participants or responsible parties to guarantee accountability for each item. As a template, the RAIL consists of the following separate components:
Figure 5: RAIL
As the Energy Efficiency team progresses through its revised process, the Planning and Analytics Team will track the progress towards accomplishing the items identified in the RAIL. Figure 6 below identifies several of the RAIL action items. With the emphasis of the RAIL to meet the requirements set forth by the settlement documentation, each opportunity includes a reference to any applicable items from Staff’s comments. The below figure identifies several of the items included in the RAIL.

Opportunity Action Item Who and By When?Update Complete
8 | P a g e
Figure 6 – RAIL Items
A notable process change that had resulted from the BPI is a formal structure for review of IPUC comments and recommendations along with Impact and Process Evaluation recommendations from the third party EM&V vendor. Included as Appendix C and Appendix D to this report are examples of two of the lead sheets that the Energy Efficiency team has implemented to better track and implement recommendations and comments.
•Re‐draft 2018 Report•Implement vendor management plan•Create protocol for triggering events•Establish Captain's Log•Formalization of documentation for EM&V changes•Create inventory of alternative tools for reporting•Draft template with content types•Create Review and ACR content teams•Develop clear message for any related change to vendor scope / deliverables•Develop report out on BPI to share with internal audiences•Review current EM&V plan and align with vendor for consistency•Initiate discussion with IPUC/WUTC staff to get high level input
Q1 2020 Action Items
•Meet with team for settlement item tracking•Work with program managers and marketing to craft program specific "stories" of program's performance•Include narrative of exceptions and variances as part of overall approach to impact evaluation.•Also include in specific program sections. •Consider engagement with Helvetica for proofreading•Develop QC checklist for accuracy review•Develop structure for storing inputs from efficiency team. Define work papers in this context•Include narrative on BPI in annual report•Engage economist to review current methodology / findings•Develop QC checklist for accuracy review•Review existing TRM for accuracy against unverified savings; •Clean up undated comments in TRM per IPUC comment 7a•Develop protocol for engaging teams to address changes to programs related to recommendations•Review current contract and identify current oversight elements
Q2 2020 Action Items
•Engineers share best project documentation practices with efficiency team•Review BPI outcomes from previous site specific project BPI•Evaluate what elements of site specific best practices can be transferred to project documentation for other programs•Implement periodic check in with vendor to ensure methodology is staying consistent‐ consider mid‐engagement check‐in
Q3 and Q4 2020 Action Items
9 | P a g e
The “Order and Comment Review” lead sheet is used to verify acknowledgement of Commission Orders and Staff Comments/Recommendations and to outline any actions that need to take place. The “EM&V Recommendation Review” lead sheet is a routing sheet that follows each Impact and Process Evaluation recommendation and identifies the steps taken in response. The BPI team and its sponsors are confident that these changes, as well as the continued improvements and closer attention to detail resulting from this BPI, will result in a stronger and more structured report creation process. At the same time, it will fulfill the requirements set forth by the Idaho Commission and incorporate the valuable recommendations made by Commission Staff.
Conclusion The Business Process Improvement workshop helped to identify several opportunities to improve processes and structure for compiling and reporting Energy Efficiency data. The BPI team was intentional in the workshop design to focus on gaps and identify potential opportunities directly related to Idaho Staff’s comments and recommendations. As a result of the workshop, the Company identified process changes that will lead to continuous refinement in the work related to Avista’s Energy Efficiency reporting. By leveraging more teams within Avista, we are able to involve more expertise in the process to improve the overall program reporting. The work identified within the BPI is done on an ongoing basis, and as Avista continues to adapt its Energy Efficiency programs, the Company will adjust the application of changes as appropriate. The Energy Efficiency team will continue to focus on program improvements and seek out additional ways to strengthen its structure, design, and procedures.
Questions If you have any questions please contact: Ryan Finesilver Energy Efficiency Planning and Analytics Manager (509) 495‐4873 [e-mail]
Appendices
Appendix A – BPI Charter
Appendix B – Future State Process Flow Diagram
Appendix C – Order and Comment Review
Appendix D – EM&V Recommendation Review
Annual Conservation Review Process Improvement
Initiative Project Background Goal Statement / Project Objective 3 Yr Value
Target
Project Sponsor:
Anna Scarlett
BPI Mentor:
Brian Powers
Lisa Garrett
Project Champion:
Ryan Finesilver
Meghan Pinch
Avista recently reached a settlement with the state of Idaho
requiring us to return dollars to our tariff associated with the
cost of the Annual Conservation Report. The Idaho Public
Utilities Commission found that the conservation report and
related third party impact evaluations lack adequate
supporting documentation, suffer from insufficient reporting
methodologies, and insufficiently describe program activities.
To develop a new reporting format that
emphasizes Idaho settlement items while also
being usable for Washington reporting.
Formalize processes and procedures for
implementing recommendations and results from
the Impact Evaluation and Process Evaluations
Establish adequate employee involvement for the
preparation, review and delivery of Annual
Conservation Reports
Develop a formal oversight structure for annual
conservation review process
Develop report validations and controls
Low :
High :
Problem Statement
IPUC has determined that Avista’s annual conservation
report and supporting impact evaluation processes are
inadequate. This determination has identified the existence of
a lack of trust and communication between Avista and the
IPUC Staff.
Team Members Project Scope Estimated Benefits (Quantitative and Qualitative)
and Metrics
BPI Leader:
Brian Powers
Process Owner:
Ryan Finesilver
Core Team Members:
Ryan Finesilver
Leona Haley
Greta Zink
Matt Iris
Carlos Limon
Jaime Majure
Mike Gump
Meghan Pinch
Colette Bottinelli
Angela Koker
Jen Pearson
Support Team Members:
Grant Forsyth
Jeff Cropp - Cadmus
Key Stakeholders:
Commission Staff
Pat Ehrbar
IN SCOPE: Data gathering, compiling, and analysis, all
elements that go into report, including structure of report;
framework for implementing evaluation recommendations
OUT OF SCOPE: Prudence review processes, requests from
ELM vendor, report design and format
PROCESS START: Identifying key participants and data
collection/tracking tools
PROCESS STOP: Annual report is submitted to the PUC.
Avista’s great work is communicated more clearly and
accurately to the IPUC
The relationship between Avista and the IPUC is improved by
increased trust and better communication
Future prudence reviews will not result in financial penalties
More alignment between evaluation processes and planned
program changes/ updates
Key Risks Milestones Est. Timeline
Plan Phase Complete
Analyze Phase Complete
Design Phase Complete
Implement Phase Complete
Sustain Phase Complete
Future State Idaho Annual Review Process
Pl
a
n
n
i
n
g

&

A
n
a
l
y
t
i
c
s

Te
a
m
Ev
e
n
t
3rd
 P
a
r
t
y

E
v
a
l
u
a
t
o
r
En
g
i
n
e
e
r
s
Pr
o
g
r
a
m

M
a
n
a
g
e
r
s
Re
g
u
l
a
t
o
r
y
Ad
v
i
s
o
r
y

G
r
o
u
p
Le
a
d
e
r
s
h
i
p
Ec
o
n
/
S
t
a
t
s
Re
v
i
e
w

T
e
a
m
Ma
r
k
e
t
i
n
g
Ac
c
o
u
n
t
Ex
e
c
u
t
i
v
e
s
Phase
Develop Work 
Papers
Internal Audit 
Results
Review 
Methodology
Review 
Methodology w/
Econ/Stats Review 
Team
Review prior year’s 
annual report (flip)
Review 
methodology with 
P&A Team
Lead kick off 
meeting
Lead kick off 
meeting
Participate in Kick 
off Meeting
Participate in Kick 
off Meeting
Participate in Kick 
off Meeting
Participate in Kick 
off Meeting
Participate in Kick 
off Meeting
Kick Off 
Meeting
Pull data and 
reports 
Receive raw 
data from 
Avista
Request report 
content & 
Narratives
Provides 
Program 
Narratives in 
reporting 
template
Send data to 
3rd party 
evaluator
Pull data and 
give to P&A 
Team
Pull data and 
give to P&A 
Team
Provides sector 
marketing 
information 
and narratives
Provides 
customer 
stories and 
narratives
Provides initial 
review findings
Receives initial 
review findings
Review initial 3rd 
party evaluator 
findings
Lead initial 3rd party 
review findings
Lead initial 3rd party 
review findings
Review and 
comment on initial 
findings
Review and 
comment on initial 
findings
Review and 
comment on initial 
findings
Review and 
comment on initial 
findings
Review and 
comment on initial 
findings
Incorporate 
feedback into 
draft findings
Deliver verified 
savings and work 
papers
Deliver impact 
evaluation  and 
work papers
Review 
responses 
from 3rd party 
evaluator and 
consider 
implementing
Review 
responses 
from 3rd party 
evaluator and 
consider 
implementing
Review 
responses 
from 3rd party 
evaluator and 
consider 
implementing
Review Impact 
Evaluation
Receive impact 
evaluation
Lead Review of 
Impact Evaluation
Review Impact 
Evaluation
Review Impact 
Evaluation
Review Impact 
Evaluation
Content 
Review
Content 
Review
Content 
Review
Content 
Review
Content 
Review
Final 
Review
Final 
Review
Final 
Review
Final 
Review
Engage 
professional to 
design report
Submit
Final
Report
Post Process Review 
& Evaluation
Submit
Final
Report
Post 
Review 
Process
Post 
Review 
Process
Post 
Review 
Process
Post 
Review 
Process
Post 
Review 
Process
Post 
Review 
Process
Post 
Review 
Process
Post 
Review 
Process
Support the 
development of 
work papers
w/P&A team
Check what 
groups/players 
involved should 
provide svgs & 
Narratives
Receive & 
review verified 
savings
Incorporate 
any updates 
from content 
review
Submit
Final
Report
P&A Team begins 
draft of Report
Update draft 
report & 
finalize for 
content review
Appendix B - Future State Process Flow Diagram
Energy Efficiency
Document Review 
Item:Idaho Settlemen
Docket:AVU‐G‐18‐08 and AVU‐E‐18‐12
Process Owner Ryan Finesilve
Date Received:3/1/2020 Reply Comments Due:4/1/2020
Initial Review Meeting Date 3/1/2020
Initial Review Team Ryan Finesilve Mike Gump Tom Lienhard Chris Drake
Anna Scarlett Meghan Pinch Jaime Majure
Action Items
Team Member Date Assigned Due Date Task  Completion Date Task Completed
Jaime Majure 3/2/2020 3/2/2020 Read Order 3/2/2020 Yes
Mike Gump 3/2/2020 3/2/2020 Read Order 3/2/2020 Yes
Meghan Pinch 3/2/2020 3/2/2020 Read Order 3/2/2020 Yes
Tom Lienhard 3/2/2020 3/2/2020 Read Order 3/2/2020 Yes
Review Checklis
Has communicaiton been made to each party involved Yes
Have all action items/to do items been assigned NA
Initial Review Meeting Notes: Avista received the Order approving its settelement with Idaho Staff for Docket AVU‐G‐18‐08 and AVU‐E‐18‐12 on March 1, 2020. The 
Order was circulated to the Initial Review Team and it was determined that no additional aciton was required.

Appendix C - Order and Comment Review
DATE ARTIFACT NUMBER
[insert date] [x ]
PROGRAM TO BE CHANGED
[Program name]
TITLE
[Title of the change request]
AUTHOR
[Person writing this change request]
RECOMMENDATION
[Short summary of recommended change or description of why a change is not recommended.]
PEOPLE CONSULTED OR WHO PROVIDED INPUT
[Functional managers/ others that were consulted during the writing of this change request]
DECISION MAKER
[Functional Manager]
Approve Approve with modifications Do not approve
Comments:___________________________________________________________________
____________________________________________________________________________
____________________________________________________________________________
____________________________________________________________________________
HIGH LEVEL SUMMARY
[Provide key elements of the modification and the situational and contextual perspective of this
change. Include perceived impact on time, resources, and any funding of the work. Offer a
recommendation along with any supplemental details (ie timeline, urgency, etc.). Note: this
section should be less than one page]
TYPE OF CHANGE AND DESCRIPTION OF CHANGE
[Categorize the opportunity/change(s)]
[ ] Incentive [ ] Internal Process [ ] External Process
[ ] Specification [ ] Savings [ ] Documentation requirements
[ ] Code Update [ ] New Measure [ ] Measure Modification
[ ] M&V Requirements [ ] Other: _______________________
BENEFIT OF THE CHANGE

Appendix D - EM&V Recommendation Review
[Qualitative and quantitative description of the benefit to Avista, Avista’s program goals,
reporting, staff capacity; benefits for customers, contractors, community]
IMPACT
[Estimated qualitative description of the impact to other programs, utility, other business
divisions, customers, community; What is the impact of NOT doing the change? Is there an
opportunity to align change with other programs?]
ALTERNATIVES CONSIDERED
[Provide a short explanation of any alternatives to the chosen approach and any impacts]
DOES THIS CHANGE REQUIRE INTERNAL OR EXTERNAL STAKEHOLDER CONSULATION?
[Describe any related stakeholder engagement activities that must take place before this
change is implemented].
URGENCY
[Is the change bound to time restrictions? If so, what is the bound of time?]
COST
[Estimate of internal person-hours and to realize the change; estimate of financial costs of the
change. Use a chart or table if possible.]
BARRIERS
[Describe any barriers to implementing this change- legal or regulatory barriers, financial
hurdles, etc. Do we currently have legal authority? Are there any regulatory considerations that
may impact the implementation of this change? ]
RISK
[List and detail any known issues or possible risks with this change as well as the risk of not
making the change. Use the following prompts for each risk.
Risk:
Type of Risk (financial, legal, compliance, savings, reputational, customer service, technical,
management, other):
What might cause the risk:
What are the potential impacts of the risk:
What is the probability of the risk:
What is the risk response strategy? Does it need an action plan/ owner? ]
NEXT STEPS AND TIMELINE FOR EXECUTING THE CHANGE
[What are the next steps necessary to realize this change? Who will execute these? When will
they be executed, QC’d, introduced back into the program, etc.? What’s the communication
plan, training, etc.?]

Appendix D - EM&V Recommendation Review
Responsible Month
1
Month
2
Month
3
Month
4
Month
5
Month
6
Month
7
Month
8
Activity
1
Activity
2
Activity
3
Activity
4
Activity
5

Appendix D - EM&V Recommendation Review

Internal Audit Department

DATE: July 28, 2020
TO: Ryan Finesilver, Nicole Hydzik, Tom Lienhard
CC: Kevin Christie, Anna Scarlett, Dennis Vermillion, Tracy Van Orden
FROM: Janice Gibler, Carly Guillory
SUBJECT: Demand Side Management (DSM) Review Report
Focus Area

Background
On February 11, 2020, Avista Corporation signed a Stipulation and Settlement with the Idaho
Public Utilities Commission (IPUC) regarding the Company’s electric and natural gas energy
efficiency Case Nos. AVU-E-18-12 and AVU-G-18-08. On April 28, 2020, the final Settlement
Stipulation was approved by all parties (Order No. 34647), and one of the Settlement’s eight
action items is required to be completed by Internal Audit:
“Direct Avista Internal Audit to perform an audit of energy efficiency processes for
adequacy of controls and adherence to industry best practices. The Company will provide
audit finding, recommendations, and a plan for realizing them to IPUC staff by August 1,
2020.”

Purpose and Scope
The objective of this review is to ensure that the DSM department has appropriate policies and
procedures in place to accurately process qualified customer rebates and to compare those
policies and procedures to industry best practices. Residential and non-residential rebates issued
between January 1, 2019, and April 30, 2020, were included in the scope of this review.

Procedures
Internal Audit performed the following procedures:
• Interviewed DSM department management in order to determine areas of risk/concern
within the department.
✓ Our Customers
We must hold our customers’ interests at the forefront of all our decisions, operating our business by showing that we
are transparent, genuinely care, and are easy to do business with.
• Obtained an understanding of the various reviews performed by third parties (Cadmus,
WA/ID/OR commissions, etc.) and inquired of any material exceptions/findings from
recent reviews.
• Reviewed the DSM Standard Operating Procedures (SOPs) to understand the rebate
qualifications and the procedures for processing rebates. These procedures were
compared to industry best practices.
• Selected 25 residential rebates (including three low-income) and 25 non-residential
rebates (including five site-specific) and performed the following procedures:
o Verified the rebate qualifications were met and the rebate amount paid to the
customer appeared accurate.
o Confirmed all required paperwork, signatures, approvals, etc. were obtained prior
to the customer receiving payment.
o Used ACL (data analytics) to identity any duplicate rebate payments.

This review was conducted in conformance with the International Standards for the Professional
Practice of Internal Auditing.

Observations and Recommendations
Observations:
1. While testing for potential duplicate rebate payments made to customers, it was noted that
one rebate selected for testing appears to be a duplicate payment. Currently, residential
rebates are tracked and processed in Oracle’s Customer Care & Billing system (CC&B) and
once the rebate is submitted and approved within CC&B, changes and/or adjustments to the
rebate are not shown. The Energy Efficiency team inquired with the Remittance Services
department to verify whether a credit memo was created for the customer, but because
there is no tracking system for it and because the dollar amount was $75 (a common
amount), a credit memo for the customer was not discovered.

Recommendation:
We recommend that the interface within CC&B be updated so that rebate duplicates and
reversals/reimbursements can be updated or changed when they are discovered after the
initial creation date. If this is not feasible within CC&B, we recommend that this be done in
iExpense or another DSM program. If a credit memo is created for a customer due to a
duplicate payment, this should be posted to the customer’s account to provide assurance
that a duplicate rebate was not paid.

Management Response:
The Energy Efficiency team will address this issue with the CC&B Architect to explore the
program’s capabilities for capturing duplicates and reversals and work on a solution. As
Avista works to implement iEnergy and other customer technologies, we will work with the
program vendor to identify controls around duplicate incentives. Depending on system
capabilities, the Energy Efficiency team will investigate potential monthly reports that will
help to recognize duplicates in a timely manner.

2. Four non-residential prescriptive rebates selected for testing did not have the rebate form
signed by the customer. The SOPs state that the rebate form should be completed, and this
includes the customer’s signature.

Recommendation:
All rebates should be approved by the customer; therefore, a signed or approved form
should be obtained for each rebate. The current rebate form, or an electronic version of the
form, should be retained with the customer’s rebate request.

Management Response:
The Energy Efficiency team will take this under review as we further incorporate technology
platforms and e-products such as DocuSign, will establish new procedures to ensure that
either the customer’s signature is present or that other acknowledgement takes place.

Best Practices Recommendations:
• When prescriptive rebates are issued, a rebate calculation showing the qualified and paid
amounts should be attached as supporting documentation. When a customer qualifies for
the same type of rebate over multiple years and different amounts are paid because the
incentive amounts were updated, supporting documentation explaining why each amount
was paid should be maintained.
• For site-specific rebates, a Technical Review Top Sheet (Top Sheet) is completed for each
rebate approved. The date of the peer or supervisor review of the Top Sheet is not currently
documented. We recommend that this be tracked in iEnergy after the review is completed or
a date be added to the form to document when the review was completed.
Management Response:
The Energy Efficiency team will look for ways to make incentive calculations and Top Sheet
information more visible. As we transition to iEnergy and other electronic resources, we will
explore the program(s) capabilities and resources.

Conclusion:
Other than the observations noted above, it appears the DSM department has appropriate
internal controls in place to accurately process qualified customer rebates.

Evaluation Measurement & Verification Engagement Modification Report
Ryan Finesilver, Energy Efficiency Planning and Analytics Manager
7/31/2020

1 | P a g e

Contents Idaho Settlement Action Item ............................................................................................................................................. 2 Background ................................................................................................................................................................................ 2 EM&V Modifications & Enhancements Summary ...................................................................................................... 2 Review of EM&V contract ................................................................................................................................................ 2 EM&V Engagement Changes for 2020‐2021 ...................................................................................................... 3 Vendor Performance Management Plan ................................................................................................................... 4 Subjective Scoring.......................................................................................................................................................... 4 Objective Scoring ........................................................................................................................................................... 5 2022‐2023 EM&V Vendor Selection ................................................................................................................................ 5 IPUC Staff Involvement..................................................................................................................................................... 5 Conclusion .................................................................................................................................................................................. 5 Questions .................................................................................................................................................................................... 6

2 | P a g e

Idaho Settlement Action Item The “EM&V Engagement Modification Report” is provided in response to Settlement Action Item #2 which states: “Review the contract and the statement of work with the current third‐party Evaluation, Measurement and Verification (EM&V) vendor to ensure that future work conforms with the terms of this Stipulation. The EM&V vendor will partner with Avista Supply Chain Contract Management to develop a vendor performance management plan and to add clarity and process around roles, relationships, and internal controls by the second quarter of CY 2020. Avista will provide this plan to Staff by August 1, 2020.” In addition, this report also addresses Settlement Action Item #7 which states: “Engage with Commission Staff when selecting a third‐party EM&V vendor. Avista will work with Staff to identify a schedule for evaluations that more closely matches the prudence filing dates so that results can be evaluated, discussed, and programmatic changes can be implemented within a reasonable timeframe.”
Background During 2020, Avista’s Energy Efficiency program team made several modifications and enhancements to its Evaluation, Measurement & Verification (EM&V) engagement approach. These changes impact both the current 2020‐2021 period in addition to providing guidance for the upcoming 2022‐2023 biennium. The updates made to the engagement are intended to provide more rigor, support and transparency to the EM&V process while also being responsive to the requests and concerns of Idaho Public Utilities Commission (IPUC) Staff and other stakeholders. Per the Settlement Stipulation in Case Nos. AVU‐E‐18‐12 and AVU‐G‐18‐08, approved in Order No. 34647 on April 28, 2020, the Company agreed to the following: “Review the contract and the statement of work with the current third‐party Evaluation, Measurement and Verification (EM&V) vendor to ensure that future work conforms with the terms of this Stipulation. The EM&V vendor will partner with Avista Supply Chain Contract Management to develop a vendor performance management plan and to add clarity and process around roles, relationships, and internal controls by the second quarter of CY 2020. Avista will provide this plan to Staff by August 1, 2020.” This Engagement Modification Report is intended to fulfill the reporting requirement as identified in the agreement above.
EM&V Modifications & Enhancements Summary
Review of EM&V contract In March 2020, Avista conducted a Business Process Improvement (BPI) for its Annual Conservation Reporting process, which identified several opportunities for potential improvement. Through the BPI process, these opportunities for refining the annual reporting process were collected, prioritized

3 | P a g e

by importance and impact, and transferred to a Rolling Action Item List (RAIL). For the action items that were applicable to the EM&V engagement, those items were added to the 2020‐2021 Scope of Work (SOW) to ensure that future third‐party work is in alignment with the agreed upon items from the settlement and IPUC Staff’s comments.
EM&V Engagement Changes for 2020-2021
Annual Conservation Report A primary change for the 2020‐2021 evaluation period is that for the Idaho jurisdiction, the scope of work for the EM&V vendor does not include direct contributions to the Annual Conservation Report (ACR). While historically Avista and the EM&V vendor would prepare this document jointly, Avista has transitioned to a process where the ACR is developed and overseen by the Energy Efficiency group. The new process leverages internal and external experts as needed. As a result of this modification, the third‐party vendor will instead focus their EM&V efforts on the impact and process evaluations.
Jurisdictional Separation of EM&V Efforts Another modification from prior engagements is the requirement that the evaluation of Idaho Energy Efficiency programs be separately conducted from Washington Energy Efficiency programs. For 2020‐2021, our vendor will employ two separate evaluation teams that will focus on Idaho and Washington independently. The evaluation for Idaho will contain only projects from the Idaho jurisdiction to the extent possible. Any EM&V work that contains both Washington and Idaho information must be pre‐authorized by Avista Energy Efficiency Staff.
Use of Multiple EM&V Vendors For 2020 and 2021, Avista seeks to work with multiple vendors for the Evaluation, Measurement and Verification of its Energy Efficiency program. For its Residential and Low‐Income programs, Avista will provide a Request for Proposal (RFP) to potential vendors for the upcoming 2020‐2021 evaluation work. The current vendor will retain the evaluation work for Avista’s Commercial and Industrial sector for the 2020‐2021 period. Avista’s Energy Efficiency Team will work closely with IPUC Staff in the selection of the vendor as agreed to in the settlement stipulation.
Schedule for Evaluations As identified in Settlement Item #7, Avista will work with Commission Staff to identify a schedule for evaluations that more closely match the prudence filing dates. Avista has discussed opportunities with their current EM&V vendor in order to expedite the evaluation work and will continue to seek alternatives that would result in timelier reporting. Avista has noted that historically their ACR has included evaluated savings and that other utilities provide annual savings achievements based on reported or unverified savings. Avista will look to discuss with Idaho Commission Staff the possibility of changing its ACR to include reported savings amounts in an effort to expedite the reporting process.
Evaluation Work Plan Several enhancements were also made to Avista’s requirements for the Evaluation Work Plan including the provisions that the consultant must: 1. Outline in the work plan their quality assurance and quality control process;

4 | P a g e

2. Describe its approach to be used for projects with small population sizes to ensure there is sufficient confidence to make decisions about individual programs and measures; 3. Include an adjustment of energy savings results for weather sensitive programs, and; 4. Meet with Avista to develop accurate equations that represent the Consultant’s methodology for evaluating projects.
Impact Evaluations  In an effort to provide support for, and more transparency in, the evaluation of programs and associated calculation of results, Avista has requested that the vendor provide supporting documentation for the data contained in the Impact Evaluation reports. In addition, Avista has required that the consultant notify the Company if any deviations from evaluation occur or are planned to occur. Below are the specific items included in the SOW for the Electric and Natural Gas Impact Evaluations for the 2020‐2021 engagement: 1. Provide all supporting workpapers for the calculations, tables, graphs, and other illustrations contained in the deliverable. The supporting workpapers must be complete and well prepared, free of hardcoded numbers to the degree possible, and include supporting calculations and data. Workpapers must be preserved by the Consultant until the conclusion of the Company’s prudence filing for the 2020‐2021 program years. 2. Clearly communicate any deviation from historical methodology for calculating cost effectiveness. This communication shall be included in the final Impact Evaluation report and presented to Avista’s Energy Efficiency Advisory Group.
Vendor Performance Management Plan The Energy Efficiency Planning and Analytics Team engaged with Avista’s Supply Chain Management team in May 2020 to develop a Vendor Performance Management Plan, also called a “Vendor Scorecard”. This process, which is considered an industry best practice, focuses on the relationship between the Company and its contractor to provide feedback on the vendor’s performance in meeting performance expectations, and has proven highly effective in other areas of the Company. The Vendor Scorecard process considers the performance of vendors from both objective and subjective scoring and communicates the importance of accuracy, communication, and improves correspondence.
Subjective Scoring The subjective score focuses on the overall perception of how effectively the vendor met the needs of the Company. The scoring focuses on a 1‐5 score with 5 being the highest attainable level of performance. The table below illustrates the ratings along with the attributes that lead to those scores.

5 | P a g e

Objective Scoring The Objective score is intended to quantify the performance of the vendor based on several categories with varying weight given to each. While the process allows flexibility to tailor the scoring matrix to fit the engagement, several factors are universal. For the EM&V engagement, the following areas were considered.
 Cost‐Reductions – 5%
 Risk Mitigation – 5%
 Quality of Service – 30%
 Responsiveness/Adaptability – 15%
 Organization – 15%
 Completeness – 25%
 Innovation – 5%
2022-2023 EM&V Vendor Selection
IPUC Staff Involvement Avista will issue a RFP in 2021 for selection of its 2022‐2023 EM&V engagement. Throughout that process, Avista is committed to working with IPUC to ensure that an appropriate level of involvement is made available to Staff.
Conclusion Avista’s modified approach to evaluation of Idaho’s Energy Efficiency program addresses several of the issues raised by staff and others. Avista made significant efforts to revise its ACR process so that data contained in the report is developed and overseen by the Energy Efficiency group and will work with Idaho Commission Staff to determine if efficiencies can be gained by transitioning to a “reported savings” reporting format for its ACR.

6 | P a g e

For its current and future EM&V engagements, Avista has added additional requirements to its Scope of Work and Evaluation Work Plans for its current and future EM&V engagements which are directly linked to Idaho Staff’s recommendations. Current and future EM&V engagements will be jurisdictionally specific with separate EM&V teams for Idaho and Washington. Avista has implemented a Vendor Performance Management Plan to rate, track and provide feedback to EM&V vendors. For continued improvements of its process, Avista will look to include separate evaluators for its Residential, Low‐Income, and Non‐Residential Segments. Lastly, Avista will work closely with Idaho Commission Staff when selecting a new EM&V vendor for future engagements.
Questions Please Contact: Ryan Finesilver Energy Efficiency Planning and Analytics Manager (509) 495‐4873 [e-mail]

Idaho Commission Staff Comments and Recommendations Implementation Report Ryan Finesilver, Energy Efficiency Planning and Analytics Manager 7/31/2020

1 | P a g e

Contents Idaho Settlement Action Item ............................................................................................................................................. 3 Staff Settlement Comments and Recommendations ................................................................................................. 4 Item 1 – Annual Conservation Reports ...................................................................................................................... 4 Staff Comment and Recommendation ................................................................................................................... 4 Avista Response and Update ..................................................................................................................................... 4 Item 2 – Use of Realization Rates from Impact Evaluations ............................................................................. 5 Staff Comment and Recommendation ................................................................................................................... 5 Avista Response and Update ..................................................................................................................................... 6 Item 3 – Formal Process for Implementation of Impact Evaluation Results ............................................. 6 Staff Comment and Recommendation ................................................................................................................... 6 Avista Response and Update ..................................................................................................................................... 7 Item 4 – Project Sampling Methodologies ................................................................................................................ 8 Staff Comment and Recommendation ................................................................................................................... 8 Avista Response and Update ..................................................................................................................................... 8 Item 5 – Statistical Modeling .......................................................................................................................................... 9 Staff Comment and Recommendation ................................................................................................................... 9 Avista Response and Update ..................................................................................................................................... 9 Item 6 – Weather Normalized Data ........................................................................................................................... 10 Staff Comment and Recommendation ................................................................................................................. 10 Avista Response and Update ................................................................................................................................... 10 Item 7 – Technical Reference Manual ...................................................................................................................... 10 Staff Comment and Recommendation ................................................................................................................. 10 Avista Response and Update ................................................................................................................................... 11 Item 8 – Third Party Data Verification ..................................................................................................................... 11 Staff Comment and Recommendation ................................................................................................................. 11 Avista Response and Update ................................................................................................................................... 12

2 | P a g e

Item 9 – Separation of Duties and Quality of Deliverables .............................................................................. 13 Staff Comment and Recommendation ................................................................................................................. 13 Avista Response and Update ................................................................................................................................... 13 Item 10 – Internal Evaluation and Controls .......................................................................................................... 14 Staff Comment and Recommendation ................................................................................................................. 14 Avista Response and Update ................................................................................................................................... 14 Item 11 – Leadership Experience .............................................................................................................................. 15 Staff Comment and Recommendation ................................................................................................................. 15 Avista Response and Update ................................................................................................................................... 15 Conclusion ................................................................................................................................................................................ 16 Questions .................................................................................................................................................................................. 16

3 | P a g e

Idaho Settlement Action Item The “Idaho Commission Staff Comments and Recommendations Implementation Report” is provided in response to Settlement Action #8 of the Stipulation and Settlement for Avista’s 2016‐2017 Electric and 2014‐2017 Natural Gas Energy Efficiency1 prudence application which states that Avista will: “Address the issues and implement the recommendations identified in Attachment A: Issues and Staff Recommendations Regarding Avista’s DSM Programs, November 2019. Avista will submit a report to Staff on the status of each of these items by July 31, 2020.” As part of Avista’s application for a determination of 2016‐2017 electric and 2014‐2017 natural gas energy efficiency expenses as prudently incurred2, Avista entered into a settlement stipulation agreement with the Staff of the Idaho Public Utilities Commission (Staff) to address several items within its Energy Efficiency program. On Tuesday, November 9, 2019, Avista and Commission Staff met in Boise to discuss concerns about Avista’s 2016 and 2017 DSM Annual Conservation Report & Cost Effectiveness Analysis. During this meeting, Staff presented 11 comments and recommendations for discussion. This report is intended to fulfill the reporting requirement as identified in the agreement above.
1 Case Nos. AVU‐E‐18‐12 and AVU‐G‐18‐08 2 Case Nos. AVU‐E‐18‐12 and AVU‐G‐18‐08, approved in Order No. 34647 on April 28, 2020

4 | P a g e

Staff Settlement Comments and
Recommendations
Item 1 – Annual Conservation Reports
Staff Comment and Recommendation Avista's Conservation Reports and the third‐party impact evaluation lack proper documentation, suffer from inaccurate and inconsistent reporting methodologies, and insufficiently describe how programs are documented, especially in the portrayal of expenses and the cost‐ effectiveness of Idaho programs. a. Staff recommends the Company closely examine the accuracy of its reports, tables, and figures, and more diligently proofread these documents. b. Staff recommends the Company work closely with its third‐party evaluator to develop accurate equations that represent its methodology. c. Staff recommends the Company describe how programs are operated through more in‐depth program descriptions, while organizing Conservation Reports to focus on programs holistically, perhaps on a chapter by chapter basis. d. Staff recommends Conservation Reports include program details that have historically only been contained in the Annual Conservation Plan, including: i. Program changes made during the year ii. The latest evaluation, measurement, and verification plan iii. Frequently used terms, and iv. A Unit Energy Savings (UES) list for measures, including any updates.
Avista Response and Update Avista has made major revisions to its approach for conservation reporting for both its 2018 and 2019 program years. As part of this restructure, the Energy Efficiency team engaged in a Business Process Improvement (BPI) workshop, a 3‐day endeavor to improve the annual conservation review process. Restructuring the reporting format was one major area of focus in the BPI workshop, as was developing a more robust and cross functional quality control process for the report. Specific progress related to each action item is described below.
Item a: The BPI team developed and implemented a broad, cross functional approach to report and supporting data review, with multiple individuals participating in some element of the QC process. More details on the BPI project are included in the concurrently filed report entitled “Annual Conservation Review BPI Report Out.”

5 | P a g e

Item b: Avista also had several conversations with its Evaluation, Measurement, and Verification (EM&V) vendor to ensure that the methodologies used in its evaluation of Avista programs were adhered to, and that their internal equations were readily available, visible and transparent. Looking ahead to the 2020‐2021 engagement, Avista has stipulated in the EM&V work plan that equations related to methodologies be provided and discussed between Avista and its EM&V vendor. More information on changes to its EM&V modifications are included in the “EM&V Engagement Modification Report” submitted concurrently with this report.
Item c: The new reporting format consists of a program‐by program approach to organizing and presenting data. This program‐by‐program organizational structure allows reviewers to see holistic program information all in one place, including program changes made during the year, program‐level impact evaluation results, program‐level recommendations from the third party evaluator; marketing efforts for each program, program plans for the next year, and key takeaways for the program year.
Item d: The new format also includes information that was historically only contained in the company’s Annual Conservation Plan, the latest evaluation, measurement and verification plan; frequently used terms; and a list of current Unit Energy Savings (UES) values.
Item 2 – Use of Realization Rates from Impact Evaluations
Staff Comment and Recommendation The Company does not consistently use impact evaluation results to evaluate program effectiveness or measure cost effectiveness. a. When the value of the realization rate is statistically significant (outside sampling level bounds), the Company should investigate to determine the reason for the discrepancy between the values determined by the Company and its third‐party evaluator. If, after investigation, the discrepancy is determined to be valid, the Company should adjust program and measure savings used to determine cost test ratios (e.g. Utility Cost Test (UCT), Total Resource Cost (TRC), etc.) and to adjust savings reported to the Commission in annual reports. b. Staff recommends that the Company apply realization rates from the most recently evaluated program savings to subsequent years of unverified program savings. This should continue until realization rates are updated in a future evaluation.

6 | P a g e

Avista Response and Update Avista has made changes to its process around formalizing the use of impact evaluation results and incorporating those changes into its program. Avista’s 2018 and 2019 reporting is based on verified and evaluated saving amounts that are the result of the analysis done by the EM&V vendor.
Item a: Avista’s process for making programmatic changes due to evaluation results involves the investigation of any program that has a statistically significant variance in realization rate, outside of sampling level bounds. If it is determined that a UES value change is appropriate, Avista makes those changes to its UES library and all related program implementation elements, including cost test ratios. If the realization rate variance is not related to a UES value, Avista investigates the cause of the discrepancy and determines what improvements can be made going forward. In response to IPUC’s comments and recommendations, Avista has been more proactive about making changes to UES values. As soon as it is determined that a UES adjustment is appropriate, the Energy Efficiency team works directly with its technicians to make the adjustments within our system. For example, several residential measures were adjusted as a result of the 2019 Impact Evaluation and the adjusted UES values went into effect in the following month. The TRM has been adjusted to reflect these changes, and the cost benefit ratios were recalculated.
Item b: Inherent to Avista’s revised EM&V process, when a realization rate suggest that a UES value would need to be adjusted, Avista will make the adjustment to all subsequent years until the next evaluation of the program takes place.
Item 3 – Formal Process for Implementation of Impact Evaluation
Results
Staff Comment and Recommendation The Company has no formal processes for using third‐party evaluation results to identify problem areas or improvement opportunities in its own programs, and Staff found little evidence that results are used informally. a. In order for the impact evaluation to be useful, its results should be used by the Company to evaluate the cost effectiveness of programs and measures, to identify opportunities for process improvement, and to provide more accurate estimates of savings (verified savings) to the Commission. b. Staff strongly recommends that the Company develop a formal procedure for communicating, evaluating, and using feedback from its third‐party evaluator to assess and improve its programs, while still maintaining the independence of the third‐party evaluator.

7 | P a g e

Avista Response and Update
Item a: Avista has long depended on impact evaluation results to drive company decisions to launch, improve or sunset programs as a part of the annual business planning process. Historically, recommendations from impact evaluations have been noted and included in the company’s annual conservation plans and then implemented through the normal annual business process. However, in response to Staff’s comment, the Energy Efficiency team has instituted a formal process for the documenting, tracking and implementation of third‐party evaluation results and recommendations. That process is described in detail in item b. Cost effectiveness has historically been calculated by Avista’s third‐party evaluator at the portfolio level rather than at the program level. For the 2018 and 2019 conservation reports, Avista’s evaluation consultants continued to use a portfolio‐level methodology for calculating cost effectiveness. However, for the 2020 engagement and beyond, Avista will require third‐party evaluators to provide cost effectiveness assessments at the program level in addition to the overall portfolio cost effectiveness.
Item b: Avista has developed and implemented a more formal process for reviewing and implementing recommendations from third‐party evaluators. First, Avista’s energy planning team generally performs an initial review of impact evaluation results, noting the status of each fuel portfolio’s cost effectiveness metrics; reviewing the realization rates and verified savings of each program; and inventorying recommendations from impact and process evaluations. The planning team then compiles these recommendations into a matrix and convenes a meeting with the Energy Efficiency Leadership team to review the recommendations. Potential changes to programs are discussed and prioritized and decisions are made as to which changes will be made (and on what timeline). If questions arise about specific details of the recommendations, the planning and analytics team serves as liaison with the consultant, passing along questions as necessary. The planning and analytics team may convene a separate meeting with the consultant team to provide further clarification on recommendations; however, the consultant is not included in conversations where recommendations are reviewed, prioritized, and ultimately decided on. The team then meets with a broader cross section of staff, including program managers, engineers, the planning and analytics team, and other departments where necessary, to review recommendations and discuss steps needed to implement changes. Responsible parties are identified, and work is assigned to implement the recommendation. As part of the formal process, an individual “change request” is created for each recommendation. This report is intended to provide a higher level of detail around the change that was made, the decision makers involved in the change, the person responsible for implementing the change, the approving manager, a high level summary and the modification type.

8 | P a g e

In addition to recording these items, programmatic changes that are more systematic require additional information to be collected including the overall benefit of the change, the impact to the program, alternatives considered, internal and external stakeholder consultation, any urgency around the change, costs, barriers, risks, and if necessary, a schedule for implementing the changes. Ultimately, the planning and analytics team works with the Energy Efficiency Leadership team to track changes and ensure they’ve been implemented.
Item 4 – Project Sampling Methodologies
Staff Comment and Recommendation The sample sizes used to determine Verified Savings, in some cases, were inadequate for determining cos‐effectiveness with sufficient confidence to make decisions about individual programs or measures. a. Staff believes that the Company should work with their third‐party evaluator to assure that the Company has sufficient confidence in the results of the impact evaluation to make decisions about programs and individual measures. This may require that the Company and third party evaluator adapt their approach to sampling: For example, Staff believes that the Company and its third‐party evaluator should consider analyzing small programs and measures, like the Commercial Insulation program, on a biennial, or triennial basis so that sufficient data is available to provide reliable estimates of measure/ program cost effectiveness. b. Staff believes it is necessary for some of Avista's team members to have sufficient statistical knowledge to be able to formulate requirements such as sampling levels and sampling plans, to proofread and interpret the statistical results of evaluations, and to have some knowledge of statistical survey methodology, experimental design, and multiple regression.
Avista Response and Update
Item a: The Company and its third‐party evaluator have worked together in development of its 2020 EM&V Work plan to identify projects that have historically had lower participation rates in order to modify its evaluation cycle. For the 2020 engagement, Avista and its third‐party evaluator will mutually agree on an evaluation cycle that considers a biennial basis for smaller programs. While a triennial basis is a consideration, the current evaluation engagement will end in 2022 leaving one year of evaluation for the next evaluator. To avoid confusion, a biennial basis is being explored.
Item b: Avista has brought its chief economist into an advisory role for the entire impact evaluation process. For the 2018‐2019 engagement, the chief economist and other employees with a high degree of statistics experience, reviewed impact evaluations (including work papers) to better understand the vendor’s methodologies. Members of the economics team attended multiple meetings with the

9 | P a g e

third‐party evaluator to understand the evaluation process and ask the evaluator about methodologies used in the engagement. For the 2020 engagement, the Chief Economist and team reviewed the entire EM&V scope, including evaluation methodologies, sampling plans and sampling levels. Any recommendations made by Avista’s economics team were incorporated into the 2020 consultant scope.
Item 5 – Statistical Modeling
Staff Comment and Recommendation Staff found some models used by Nexant3 to be inappropriate, applied incorrectly, or to be of questionable value. For example, given the randomized design used to select participants in the Oracle Home Energy Reports program, a t‐Test would have been more appropriate than the overly complicated model used by Nexant. Furthermore, the Nexant model included interaction terms, but no Main Effects terms for some of the terms used in the Model. a. Staff believes that the team used to evaluate its third‐party evaluator's performance should include one or more members with skill in statistical survey/experimental design and multiple regression analysis. Staff recognizes an appropriate level of knowledge and skill ls is necessary to manage and understand third‐party work products and identify problems prior to submitting reports and analyses for prudency filings.
Avista Response and Update
Item a: The Company has modified its process to include companywide resources in the review of third‐party reports so that adequate statistical skillsets are part of the impact evaluation review. Avista team members with skillsets in statistics and economics have contributed to the review of the impact evaluations prepared by the third‐party evaluator and has made inquiries into the underlying methodologies and calculations for the 2019 annual conservation report. In addition, the Energy Efficiency team has also collaborated with others within the Company that have experience in statistical analysis to assist in the review. 3 Nexant is a third‐party evaluator used by Avista in this case.

10 | P a g e

A copy of the Impact Evaluation, EM&V work plan, and third‐party workpapers for sampling during the 2019 program year were provided to the economics team for review. The team compiled a list of questions and a meeting was held between Avista and the third‐party evaluator. In addition, the economics team made recommendations in the impact evaluation for readability and to avoid unnecessary need for clarification. As part of the 2020 EM&V engagement, the economics team will have a continued role as part of the evaluation review team and will participate in EM&V discussions on an ongoing basis. In addition to bringing in the economics team, Avista is working to build the statistical skillsets of the planning and analytics team. The Chief Economist and team have had several meetings with Avista planning and analytics staff to give an overview of relevant statistical concepts and principles. Avista plans to do more of these trainings in 2020 and 2021. Avista had also planned to send staff to the Association of Energy Service Professionals Foundations of Impact Evaluation training (a three day intensive training that results in a Certificate of Excellence in Foundations of Impact Evaluation). However, due to COVID‐19, this training has been postponed until further notice. Avista will consider sending staff to this training when it resumes.
Item 6 – Weather Normalized Data
Staff Comment and Recommendation Nexant did not adjust energy savings results for weather sensitive programs (Fuel Efficiency, Energy Star, HVAC, and low income). a. Staff recommends that the Company correct energy savings from weather sensitive program s for the effects of weather.
Avista Response and Update
Item a: Avista has met with its current third‐party evaluator to ensure that weather sensitive programs are weather normalized as part of their evaluation. In addition, Avista modified its SOW to require that future EM&V work plans adjust for weather sensitive programs.
Item 7 – Technical Reference Manual
Staff Comment and Recommendation During its review of Company documentation, Staff noted that the Company did not have a method in the Technical Reference Manual (TRM) to identify who is making a change, when a change will go into effect, or what the new value will be. In particular, the Company's updates to its TRM often consist of undated marginal notes to values within an EXCEL spreadsheet.

11 | P a g e

a. The Company's procedures for updating data and improving processes should be formalized. Staff suggests it should be maintained and updated using a formal document management system.
Avista Response and Update
Item a: Avista has created a process for updating the Company’s TRM which includes an explanation of how the TRM is used, the information required to be populated, the allowable sources of UES updates, the applications for which the TRM is used, where system updates will need to occur as a result of a UES change, and the required details of each item in the TRM. In addition, the TRM process will track inactive UES values within the same document so that historic savings values can be referenced and considered for potential reactivation in the future. Avista is considering several different approaches to enhancing its TRM. As a software solution, Avista’s DSM enterprise software, iEnergy, contains a measure library that individually tracks, identifies, and provides details around each measure found in the Company’s portfolio. Using the measure library as a TRM has the potential to add structure and a document management system. Avista is currently implementing the iEnergy platform and has moved its non‐residential program fully into the software and hopes to have the entirety of its program fully integrated in 2021. In addition, Avista also begun to develop a narrative TRM format to provide measure specific information in a readable format. The strengths of this approach is that each measure is outlined in its own section with an update log, descriptions of source data, detail around UES value changes and references to outside sources that contribute to the measure’s development. For the 2018 and 2019 period, Avista has provided references in its TRM tracking spreadsheet so it is clear when changes are made and who initiated the UES change.
Item 8 – Third Party Data Verification
Staff Comment and Recommendation Workpapers provided to Staff by the Company were incomplete, and often consisted of hard‐coded numbers with no supporting calculations or data. As a result, Staff was unable to verify either the savings claimed by the Company, or the savings and realization rates determined by its third‐party evaluator, Nexant. a. The Company should preserve workpapers and source data for Staff's review, both for the Company's calculations and for the calculations performed by its third‐party evaluator. The third‐party evaluator should be retained for a period of time sufficient to answer questions that arise during Staff's review of the third party's work products.

12 | P a g e

b. Staff recommends Avista retain employees with adequate knowledge, skills, and abilities to oversee third‐party contracts, products, and outputs‐providing strong contractual direction from the beginning, including distinct criteria for product success and robust internal controls. c. Staff believes that the method for documenting site‐specific projects and computations can serve as a model for retaining other computations performed by either the Company or its third‐party evaluator.
Avista Response and Update Avista has made several improvements to its internal process for the preparation of supporting workpapers for its annual reports.
Item a: To support the figures included in the annual report, Avista has created an Excel workbook to provide the source of savings, cost‐effectiveness and expenses. One member of the planning and analytics team is responsible for authoring and maintaining the workbook throughout the annual review process, although multiple people review it for quality and offer any error corrections or suggested changes to the workbook owner. In addition, Avista has modified its Statement of Work for the current and future EM&V engagements to require that third‐party evaluators retain workpapers to support their impact evaluations.
Item b: Avista’s planning and analytics team has benefitted from a new manager in late 2019 who has experience overseeing third‐party, contracts, products and outputs. He’s been effective in providing strong contractual direction and has provided clear criteria for product success. He is also very familiar with Avista’s regulatory processes and requirements. Avista also hired an efficiency analyst in early 2020 who brings 8 years of utility efficiency program planning and implementation experience, as well as extensive experience managing third party consultant contracts, scopes and products related to utility energy efficiency programs and evaluation. The planning and analytics team is rounded out by an efficiency analyst with a strong background in data management. He has been able to partner with the consultant to ensure that the consultant team is provided with robust, accurate data sets. The Energy Efficiency team has also modified its approach in managing contracts to include the expertise of its Supply Chain/Contracts department which has extensive experience with managing vendors and contracts. In addition, the Director of Energy Efficiency had oversaw Supply Chain and Contracts in a previous role. These additional resources will add to the overall knowledge and capabilities applicable to managing third party vendors. Details on how Avista has modified its current and future engagements have been detailed the concurrently submitted “EM&V Engagement Modification Report.”

13 | P a g e

Item c: Avista has reviewed the site‐specific project documentation method and is considering whether and how to use this model more broadly. As Avista transitions its program to iEnergy, more opportunities for housing workpapers will be explored.
Item 9 – Separation of Duties and Quality of Deliverables
Staff Comment and Recommendation Staff is concerned that the Company has been delegating fundamental tasks to its third‐party contractor, while providing little or insufficient oversight. a. Staff found little evidence that the Energy Efficiency Planning group had proofread either the Annual Conservation Report, the Impact Evaluation, or the workpapers supporting these documents. Staff believes that the Company ultimately bears responsibility for the quality, accuracy, and usefulness of both reports. Staff is uncertain that the Energy Efficiency Planning and Analytics group, as currently constituted, has the skill set requisite for properly evaluating both reports. b. Delegating responsibility for both its Annual Conservation Report and its Impact Evaluation to the same contractor creates a situation in which the contractor is evaluating its own work. c. Staff believes internal production of the Annual Conservation Report, using Idaho Power's Annual DSM Report as a model, and working closely with Commission Staff, has the potential to resolve some of the issues identified herein.
Avista Response and Update As stated in Item 1, Avista’s Annual Conservation Report has been redesigned with Idaho Staff’s comments at the forefront of those revisions. The structure of the report emphasizes clarity around each program and provides more detailed information.
Item a: Avista has also improved its review process for both the Impact Evaluation and the Annual Conservation Reports with more structure around review teams including subject matter experts for each section of the report. As part of Avista’s BPI process, the Energy Efficiency Team developed a RACI (Responsible, Accountable, Consulted, and Informed) matrix for the various components of the ACR and Impact Evaluations. The RACI provided clarification around roles and responsibilities for reviewing and proofing the reports.
Item b: For the 2018 and 2019 reports, Avista has taken ownership of the entire reporting process by internally producing the Annual Conservation Report, eliminating the potential conflict of interest identified by Staff. For data that has been provided by its third‐party evaluator that is present in the ACR, Avista took additional steps to ensure that the data is valid and supported by workpapers.

14 | P a g e

Item c: Avista’s Planning and Analytics team has been leading the process of producing the 2018 and 2019 Idaho Conservation Reports. The newest planning analyst has led report authoring and production and has modeled the new format after Idaho Power’s Annual DSM report. The planning analyst has worked closely with program managers, engineers and others across the utility to delegate authoring of specific parts of the report to relevant SMEs. The analyst has then pulled everything together and edited for consistency of tone, then worked with Avista’s marketing team to produce final report format, layout and graphics. Avista’s Planning and Analytics Manager has been the point person for work paper oversite on the Cadmus side and work paper creation on the Avista side, relying heavily on the team’s data specialist to ensure data included is accurate. The Planning and Analytics Manager has also overseen the final report QC process, ensuring that a thorough QC review has taken place. As part of Avista’s ongoing process for improving its Energy Efficiency program, consideration around preparing future ACRs based on “reported” savings rather than “evaluated” savings are being considered, as they would allow annual reporting to be issued earlier in the year. Avista will work with Staff to explore the potential of submitting reports on this basis.
Item 10 – Internal Evaluation and Controls
Staff Comment and Recommendation Staff believes Avista's Energy Efficiency Program requires internal controls and oversight, which used to be provided by the Planning and Analytics staff. Staff's analysis of Avista's DSM program revealed a lack of internal controls and insufficient use of quality assurance procedures not only in reporting, but also in record keeping. Though the Company has created Standard Operating Procedures (SOPs) and other management strategies to improve controls, deficiencies were apparent during the audit and through an examination of the report, which should have been caught if quality assurance protocols were being implemented. a. An organizational structure needs to be in place where recommendations and findings of the internal evaluators can be implemented.
Avista Response and Update
Item a: As noted in Item 3 of this report, Avista’s Energy Efficiency team has adopted a formal process for implementing recommendations made by its third‐party evaluator. In addition to these program enhancements, the Company has also directed its internal audit department to review its rebate process and make recommendations for improvements while observing Energy Efficiency best practices throughout the region. Findings from the internal audit are included in the “DSM Audit Report” filed along with this report.

15 | P a g e

Item 11 – Leadership Experience
Staff Comment and Recommendation Staff is concerned that inexperienced leadership and frequent turnover in key management positions with a limited knowledge surrounding demand‐side management, program development, measurement, and administration may be contributing to the functionality of the energy efficiency group. The Avista energy efficiency program lacks a cohesive process management focus and coordination throughout its teams. a. Staff believes the existing structure requires the Director of Energy Efficiency maintain sufficient authority to make required changes to plan, coordinate, staff, and achieve key process objectives. In order to do so, Staff believes the Director must sustain sufficient understanding of DSM, the energy efficiency group, and its programs to identify opportunities for continual improvement of processes and set organizational goals.
Avista Response and Update Avista’s current Director of Energy Efficiency began the role in Oct. 2020 after having served broadly in leadership roles throughout the company, including in customer‐facing, communications, and operational roles. The Energy Efficiency director reports directly to the Senior Vice President of External Affairs & Chief Customer Officer and has oversight of the Energy Efficiency Programming, Engineering, and Planning & Analytics teams, along with Avista’s Account Executive team, each of which is led by a manager. The director has organizational authority and accountability for these teams and has made organizational and work changes in 2020 to align skillsets and ensure proficiency across the teams and meet energy efficiency objectives. In addition, the director has broad reach throughout the organization to leverage competencies outside the scope of Energy Efficiency, such as data science and contract management. The Avista Energy Efficiency leadership team has a diverse mix of individuals with 53 years of experience in the utility industry and 48 in Energy Efficiency specifically. In addition, the wider Energy Efficiency team has experience in engineering, project management, customer service, corporate communication, rates and regulatory, accounting, and process implementation. The diversity of its leadership team brings many strengths and capabilities that are constant with Avista’s “how might we” approach to conservation. In addition, the Avista Energy Efficiency Leadership team participates in regional efforts for the advancement of Energy Efficiency. The Director of Energy Efficiency is a board member of Northwest Energy Efficiency Alliance (NEEA) and other members of the leadership team participate in NEEA committees and working groups.

16 | P a g e

Avista follows industry best practices for hiring talented and qualified employees for all areas within the company. While the company strives to mitigate turnover in all departments, especially those in key management positions, Avista is also very supportive of its employee’s desires to broaden their work experience, explore differing roles, or to advance internally. Avista is also is committed to developing employees to meet the needs of the company and specific departments through rotations and succession planning. As part of these efforts, Avista routinely rotates leaders within the company to build on their experience and add to their expertise.
Conclusion In addition to the above items, Avista has implemented other formal processes that add structure and role clarity to its program. In order to ensure that current and future comments and recommendations are captured and addressed, Avista developed and implemented a process for formally reviewing Staff comments and Commission orders for items pertaining to Energy Efficiency. The process includes a response task force that will review all incoming Energy Efficiency comments, orders or other applicable items. The review is documented and tasks are formally assigned to ensure that acknowledgment of and response to Staff comments occurs in a timely manner. Avista’s Energy Efficiency team is committed to an ongoing process of improving its programs. Avista feels that many of these changes help to strengthen the Energy Efficiency program and to safeguard the prudent spend of customer dollars. While many of the program changes resulting from Staff’s recommendations and comments affect 2020, Avista will continue to be responsive to future communications from Idaho Staff.
Questions Please Contact: Ryan Finesilver Energy Efficiency Planning and Analytics Manager (509) 495‐4873 [e-mail]