20140113AVU to Staff 39,46,48,51,52,55.pdf

Avista Corp.
1411 East Mission P.O. Box3727
Spokane. Washington 99220-0500
Telephone [phone]
TollFree 800-727-9170
Paul Kimball
Regulatory Analyst
Enclosures
CC (Email):
#rrrtsrfr
4=-4
IPUC (Klein, English, Donohue, Karpavich)
Corp,
!]ili ,..!.',': i.i ',-1 ir, 32
Idaho Public Utilities Commission
472W. Washington St.
Boise,lD 83720-0074
Attn: Karl T. Klein
Deputy Attorney General
Re: Production Request of the Commission Staff in Case No. AVU-E-13 -091G-13-02
Dear Mr. Klein,
Enclosed are an original and three copies of Avista's responses to IPUC Staffs production
requests in the above referenced docket. Included in this mailing are Avista's responses to
production requests 39, 461 48, 51, 52, and 55. The electronic versions of the responses were
emailed on 0l/10/14 and are also being provided in electronic format on the CD included in this
mailing.
Also included is Avista's CONFIDENTIAL response to PR 55C. This response contains
TRADE SECRET, PROPRIETARY or CONFIDENTIAL information and is separately filed
under IDAPA 31.01.01, Rule 067 and 233, and Section 9-340D, Idaho Code. It is being
provided under a sealed separate envelope, marked CONFIDENTIAL.
If there are any questions regarding the enclosed information, please contact Paul Kimball at
[phone] or via e-mail at [e-mail]
Sincerely,
AVISTA CORPORATION
RESPONSE TO REQITEST FOR INFORMATION
ruRISDICTION: IDAHO
CASE NO: AVU-E-I3-09/AVU-G-13-02
REQUESTER: IPUC StAffTYPE: Production Request
REQUEST NO.: Staff:39
DATE PREPARED: 0110812014
WITNESS:
RESPONDER:
DEPARTMENT:
TELEPHONE:
Lori Hermanson
DSM Policy, Planning & Analysis
(s09) 49s-46s8
REQUEST:
Please provide the source and calculations for non-energy benefits associated with the Nonres-Site
Specific Electric program in 2010, 2011, and20l2.
RESPONSE:
Please see Staff PR_39 Attachment A and Staff_PR_39 Attachment B. Attachment A includes
145 non-residential electric site-specific projects which had non-energy benefits associated with
them. Columns 4 and 5 of the attachment are non-energy benefit inputs into the database.
Columns 8-l I provide the non-energy benefits distributed on a BTU-basis. Columns 12 and 13
are the present value ofnon-energy benefits and values used for cost-effectiveness. The bulk of
these projects were lighting projects.
Attachment B includes a SMath Studio model which references Avista's site-specific lighting
calculator and lamp data table as well as Avista's site specific lighting calculator. Non-lighting
projects are addressed separately within the excel spreadsheet mentioned above.
AVISTA CORPORATION
RESPONSE TO REQUEST FOR INFORMATTON
JURISDICTION: IDAHO
CASE NO: AVU-E-13-09/AVU-G-13-02
REQUESTER: IPUC StAffTYPE: Production Request
REQUEST NO.: Staff-46
REQUEST:
DATE PREPARED: 0110812013
WITNESS:
RESPONDER:
DEPARTMENT:
TELEPHONE:
Lori Hermanson
DSM Policy, Planning & Analysis
(so9) 49s-46s8
Please provide all drafts of the 2012Process Evaluation Memorandum.
RESPONSE:
Please see Staff PR_46 Attachments A.
To:
From:
Subject:
Date:
MEMORANDU M
Lori Hermanson, Avista
Danielle Kolp and Hope Lobkowicz, Cadmus
2012 Process Evaluation Memorandum
+t+ty+SACsus'LZ, 2013
Cadmus' 2012 process evaluation activities for the Avista nonresidential portfolio included the following:
o A Eest Practice Comporotive Review (memo delivered in February 2013)
r ln-person interviews with program stakeholders
o Database and realization rate review
Because Cadmus is not developing a formal process evaluation report for Avista until 2014, this memo
presents the findings of the staff interviews and database and realization rate review conducted for the
2012 program year. Our objective is to provide key personnel at Avista with findings now to assist them
in improving program processes in real-time.
Key Findings
lnterview Findings: Large Project Review Challenges and Changes
ln August 2011, Avista instated a new internal system to independently review site-specific projects with
incentives greater than 550,000. This review stemmed from a recommendation in the 2010 Moss Adams
process report, pursuant to the 2010 Washington Utilities and Transportation Commission (UTC) rate
case settlement terms. The objective of the independent review was to examine project evaluation
reports prior to entering into contract with the customer, to ensure that:
o All supporting documentation was in place,
o Savings calculations were reasonable and well supported, and
r The project complied with tariff rules.
Avista staff who participated in the review process experienced multiple challenges, which are discussed
in more detail below. By the end of 2OL2, staff concluded that the review process was not functioning
efficiently, nor did it align with the intention of the Moss Adams report recommendation. Avista
suspended the review process on January 7,2OL3.ln 2013, Avista intends to implement a new approach
for reviewing site-specific projects, with the goal of balancing customer service and expediency with a
sound review. ln June 2013, Avista demand-side management (DSM) staff were finalizing this new
approach.
720 SW Washington Street
Suite 40o
Portland,0R 97205
Voice: 503.467.7100
Fax:[phone]
Staff PR 46 Attachment A
A n Emp I oyee -Owned Catn pa ny
wwwcadmusgro$p.com
Corporate Headquarters:
1 00 5th Avenue, Suite I 00
Waltham, MA 02451
Voice: 617.673.7000
Fax: 6l 7.673.7001
Page 1 of 18
Review Process Challenges ldentified by Avista
Cadmus interviewed five Avista DSM staff who were involved in the review process. During the
interviews, we discussed several core areas of concern with the process and determined that the
intended protocol was not being followed. The process dictated that the Planning, Policy, and Analysis
(PPA) team independently review the energy savings and proposed incentive levels of all site-specific
projects with incentives greater than $50,000, to ensure these impacts were calculated reasonably. ln
2072, only one-third of projects that met the criterion were sent to PPA for review.
When Cadmus asked staff about the challenges with this review process, the following four main issues
surfaced:
1. Diflerentfocusgg!a'ttention*ee*Wivsqcrossteoms. One sStaffry reported that the
key personnel within the DSM department involved in the review efte+had different focuseg[s
attention,_-which in some cases translated to varying objectives for reviewing and approving
projects. This is a problem across manv orsanizations and is. bv no means. limited to Avista.
While implementation teams are most conc€rerned with customer satisfaction and speedv and
efficient delivery, planning and evaluation teams are most concerned with comoliance. At
Avista,Ser+m#q the lmplementation team was focused heavily on the customer relationship,
while PPA was focused on ensuring compliance with the tariff, minimizing the risk of uncertainty
associated with claimed savings, and navigating relationships with regulatory bodies and
stakeholders. This is not to sav that neither team was uneenee+enedunconcerned with the
other's obiectives. While staffreoe*edasreed that their roles support the comprehensive
functions and o// overarchine goals of Avista's DSM programs-nsti,n€on€flren+, t$es'e<*nique
@fu€le+added to-misunderstandings about the value anC
Bu+Beseof the review and, in some cases, differing opinions on how and when to resolve issues"
ie+t'
Tronsparency. Some staff who were heavily involved in Avista's site-specific projects reported
not understanding the purpose, actions, or outcomes of the review. Without program-
stakeholder buy-in at all levels of the process, successful implementation was challenging. One
particular concern was a lack of information regarding how long the review would take to
complete for each project; this made it difficult to communicate accurate information to
customers on the status of their projects and the expected timeline.
Time lag ond time commitment. A common obstacle cited bv all@aff
interviewed bv Cadmus ag+eedwas that the review process took too long to complete for each
project. Often, the issues identified during the review required further discussion to understand
the assumptions behind the savings estimation, new data or information requests from the
customer, or new analysis, which caused delays. Another challenge was the volume of the
projects and limited staff resources. Having only one engineer dedicated to reviewing the large
projects was problematic and often caused bottlenecks.
Linking review with conuete octions. The review process lacked a formal follow through
procedure for problems uncovered during the review. This caused frustration -as, at times,
findingsandrecommendationswerenotimplemented'lnterviews@
2.
3.
4.
Staf PR 46 Attachment A Page 2 of 18
@documentationofthereviewproCess7indicatedthattheextentto
which the issues were resolved varied. For enhanced deliverv of DSM services. there needs to be
an agreement regarding the best
path forward for calculating savings.
lssues ldentified Through the Large Project Review
@ofthemajorfindingsofthereviewwastheoverallrelianceoncustomer-
supplied data and the need for a reliable and replicable approach to source that data. +mptemen+a+ien
tearR-membersAvista staff were in agreement a6ree4that increasing the clarity and transparency about
where engineering assumptions and inputs were coming from was a needed improvement and a
successful outcome of the review process.
Cadmus reviewed the communication logs for 22 projects that undenryent the internal review. ln
addition to the above issue of reliance on customer-supplied data or assumptions (which was inaccurate
in some cases), the following issues were documented for these projects:
o lnteractive effects were accounted for incorrectly
o Projects had missing documentation, such as invoices
o EnBineering errors resulted in incorrect claimed savings and incentive amounts (the significance
of these errors varied in size)
Planned Process lmprovements
ln2013,workedtogethertodesignanewsystemto
address the challenges cited and issues discovered with the 2012 review process. The€SM staff is
currently implementing a two-step review process for all site-specific projects that entails a technical
review by the engineering team and an administrative review by program staff.
o Technicol Review: Ensures that savings and incentive calculations in a project's Evoluation
Report are well-supported, and calculated according to tariff terms and Dual Fuel lncentive
Calculator policy. The new system includes a checklist with questions that guide the review,
along with instructions and policy guidelines. The Technical Review will be completed before the
evaluation report is sent to the customer, which contains estimated energy savings and the
corresponding incentive level.
o Administrotive Review: Ensures that minimum requirements are met before a contract is issued
with a customer and before an incentive is paid.
ln the new process, PPA conducts random spot-checks to QA/QC projects, and ensures that the review
process is smooth and effective. A main distinction between the 2012 and 2013 process is that this
random spot-check is intended to happen after the project has entered contract, or, in some cases, after
the incentive has been paid. According to implementation staff, this will help overcome bottleneck
challenges.
Both checklists (the Technical Review and Administrative Review) will be formalized documents known
as Top Sheets, which will be attached to project documentation through the life of the project. Avista
Staff_PR_46 Attachment A Page 3 of 18
intends to synchronize the Top Sheet information with Tracker, the engineering database, and with
SalesLogix, the customer information system that houses nonresidential rebate and incentive
information. ln June 2013, the lmplementation team began using Top Sheets for all projects.
2077-2072 Databose €rnd Realization Rote Review
As part of the 2012 process evaluation, Cadmus reviewed Avista's 2012 nonresidential project database
and the 2011 and 2012 realization rates for the nonresidential portfolio. The documents that were part
of each effort and our associated research questions are listed in Table lTa.ble-l.
Database Review
Realization Rate
Review
2012 SalesLogix i
Database Extract i
:20tland20t2
i
lmpact Evaluation
Sample
Are daia bejne tragked agculSlgly and consislently?
Are contracts issued in accordance with Avista policy?
oo inieniivts Comply wittr tariff rules ior Wisf ineton .nU iarfrol
. wny oo io-1e p'ot;;j;hil ;;'y lil ;l;;tyir' 'lali.Ition 'rt.?, Are there opportunities for Avista to improve the process of
, calculatinB reported savings to improve the realization rates?
Database Review
Tariff Schedules 90 and 190 govern how Avista can spend funds from the Energy Efficiency Rider
Adjustment paid by Washington and ldaho ratepayers.l To assess compliance with these Tariff
Schedules, we examined two main indicators:
Project incentive amount: electric and natural gas project incentives should not exceed 50% of
the incremental cost ofthe project (pp.3 ofSchedule 90; pp. 2 ofSchedule 190).
Project simple payback
For lighting measures, the simple payback period must be a minimum of one year and
should not exceed eight years. (pp. 2 of Schedule 90)
For non-lighting electric and natural gas measures, the simple payback period must be a
minimum of one year and should not exceed 13 years. (pp. 2 of Schedule 90; pp. 2 of
Schedule 190)
Schedule 90: Electric Energy Efficiency Programs, Washington. Available at:
http://www.avistautilities.com/services/enerevoricine/wa/elect/Documents/WA 090.pdf; Schedule 190:
Natural Gas Energy Efficiency Programs, Washington. Available at:
http://www.avistautilities.com/services/enersvoricins/wa/eas/Documents/WA 190.pdf; and Schedule 90:
Electric Energy Efficiency Programs, ldaho. Available at:
htto://www.avistautilities.com/services/enerevpricine/idlelect/Documents/lD 090.pdf
1.
2.
a.
b.
Table 1. Database and Realization Rate Review Activities
Staff_PR_46 Attachment A Page 4 of 18
The tariff rules make exceptions for the following programs or projects (pp. 3 of Schedule 90; pp. 2 of
Schedule 190):
o DSM programs delivered by community action agencies contracted by Avista to serve limited
income or vulnerable customer segments, including agency administrative fees and health and
human safety measures;
o Low-cost electric/natural gas efficiency measures with demonstrable energy savings (e.g.,
compact fluorescent lamps); and
o Programs or services supporting or enhancing local, regional, or national electric/natural gas
efficiency market transformation efforts. (ln 20L2, Avista considered new construction fuel
conversions in multifamily building projects and T12 to T8 commercial lighting conversion
projects as market transformation efforts.)
Applicability of Tariff to Prescriptive Prajects
ing
t the time of this memo. Avista's tariff was undereoins revisions
and a new tariff was filed on June 26, 2013.
Avista uses the tariff orovisions to: 1) design prescriptive measure offerinps and incentive amounts and
2) evaluate the elisibilitv of site-specific proiects on a proiect-bv-proiect basis to ensure compliance
before approving them. Cadmus does not believe the tariff language was clear enoueh on the topic of
compliance to conclude whether individual prescnptive proiects should be subiect to the simple pavback
period and incentive cap restrictions at the time of rebate application approval. -lnternallv. Avista staff
also expressed disagreement on this matter.
For purposes of this review, Cadmus evaluated both prescriptive and site-specific projects against the
provisions of the tariff described above, to allow Avista to review the findings and incorporate them into
their planning. -lt should be clear that bv presenting the prescriptive findinss below, Cadmus is simplv
suegesting that better claritv is needed and not necessarily tha-t -no+s'uge€stin*+halthese proiects
were out of compliance.
ive
weuld inevitably varyi allewinB fer flexibility in the pregram delivery and the ability te pay preseriptive
p+e
ief,'
Avista's new tariff, prepesed oroposed tariff' clarifies that moving forward, €'n+site-
specific projects are subject to the incentive cap and simple payback periods at the time of proiect
approval, while these parameters will be used in the
planning process for prescriptive measure offerings and incentive amounts.
Staff_PR_46 Attachment A Page 5 of 18
Simple Payback Findings
The majority of projects were in compliance with simple payback rules. Cadmus found that all site-
specific projects met the 13-year and eight-year payback periods, with the exception of some legacy
projects that were initiated before the new tariff rules took effect on January !,zOL1..
Less than tO% of prescriptive projects exceeded tariff simple payback periods. Table 2T+&l+2
summarizes our findings.
Site-Specific Projects
Prescriptive Lighting
(includes market
transformation and T12
gloj_ect1)*
1
Prescriptive N on-Lighting
(excludes multifamily)
Total
n/a
4,438,942 kwh
n/a
S8ss,s3s
$72,L3L
5927,665
n/a
9%
6%
8%
28t to%
LO%
Upon reviewing a sample of 10 prescriptive lighting projects that exceeded the eight-year simple
payback period, Avista found that five projects involved a T12 to T8 conversion and three projects
contained database errors that inflated the simple payback period. ln these cases, what should have
been entered as months were assumed to be years, and multiplied by 12.
The sample size for this manual review was not large enough to extrapolate findings to the full
population. However, based on the review findings, it is probable that a large proportion of the projects
included in lab.!g2+e${e+ involved T12 to T8 conversions and/or experienced database errors, thus
significantly lowering the impact on energy savings and incentive costs.
Project I ncentive Fi ndi ngs
Site Specific
The vast majority of site-specific projects had incentive costs that were compliant with the tariff rule not
to exceed 50% of the incremental project cost. lnitially, Cadmus found 74 site-specific projects (19%)
that exceeded this cap. Upon reviewing these projects, however, we found that nearly half experienced
a rounding error from Avista's Dual Fuel lncentive Calculator that put them over the 50% limit by just
$0.25 (see Fieure 1Fi$+re4). Avista staff reviewed the remaining projects to understand why they
exceeded the incentive cap, and found that the majority were incorrectly entered in SalesLogix. Avista
reported that these projects had been calculated and processed as prescriptive projects, but incorrectly
entered into the database as site-specific.
Table 2. 2012 Projects Exceeding Simple Payback Periods
Staff PR 46 Attachment A Page 6 of 18
Figure 1. Range of lncentive Amounts Exceeding 50% of lncrernental Costs, 2012 Site-Specific Proiects
60
50c
e40(J
830o'd 20
G 10
0
tu$o*utu**S*"un"..".u$
(includesmarket | ) I I I| 2,574 ; 80% ', 26,747,g65kwh i s1% i 5z,2go,o3ttransformationandTl2 , i i i | '
t:ii
Prescriptive Non-Lighting , ^.^ | -^^, I 3,220,704kWh | 58% ', a,-- .^-
""te,t-
Prescriptive
Significantly more prescriptive projects (74o/ol exceeded the 50% cap. As noted above, this finding was
expected because Avista's program design and delivery strategy did not consider prescriptive payments
as being subject to the tariff rules, and the lighting market transformation effort exceeded 50%by
design. Table 3Table3 outlines the incentive payment and energy savings impacts from projects that
exceeded the 50% incentive cap.
349 , 50% |. .'.. : . ..... " "i-""" i 5475,437 45%(excludes multifamily) rar I Jwla io,oaath;n:ii -: li,q^ i
I
I
29,968,659 kwh i 77% iTotalPrescriptive | 2,923 | 74yo i ..-;;.:^"-- ,- lo* : $z,tis,ase
* Cost impact represents the aggregate amount exceeding 50% of the incremental cost.
** Avista's database extract does not denote which projects involved T12-T8 lighting conversions,
Again, Avista manually reviewed 10 lighting projects that were over the 50% cap, and found that eight
were T12 to T8 conversion projects, considered market transformation. Based on these findings, it is
probable that a large proportion of the lighting projects listed in Table 3 involved T12 to T8 conversions,
which would greatly reduce the cost impacts and energy saving impacts of from lighting projects over
the 50% cap.
3A%
Table 3. 2012 Prescriptive Projects Exceeding 50% lncentive Cap
Staff PR 46 Attachment A Page 7 of 18
Data Entry end Dota Tracking
ln addition to assessing policy conformance, Cadmus reviewed the 2012 database for data accuracy and
completeness. We found that:
o 8 projects were recorded as paid before construction was completed (most of these were entry
errors)
o 12% of all proiects were missing Construction Complete dates
o 44 proiects (1% of all proiects) were missins incremental cost data
o L8% of site-specific projects were missing contract date fields in SalesLogix
r 1'96 ef all prejeets were missing €enstrsetien Cemplete dates
r I I prejeets (1%) were missing ineremental eest data
o 44o/o of site-specific projects were missing post-verification dates (and it is Avista's policy to
co n d uct post-i nsta llatio n i ns pectio ns of a I I site-s pecific p rojects)
Avista reviewed 20 prescriptive lighting projects to determine whether they were market-
transformation projects (as noted above). They also uncovered several data errors with these specific
projects. ln all 20 projects, at least one of the following issues was found:
o Simple payback periods were entered in the database in years instead of months,
o Simple payback periods were entered incorrectly (Saleslogix data fields were not consistent
with calculations),
r Prescriptive projects were entered as site-specific projects,
r lnformation from invoices regarding quantity and type of light fixtures was not transferred to
prescriptive incentive forms and SalesLogix correctly,
o lneligible measures were rebated, and
o lncentives were calculated incorrectly.
Realization Rate Review
Cadmus' impact evaluation methodology consisted of validating the reported savings for a sample of
projects by conducting independent metering, simulation, or regression analysis and by visiting the
project sites to verify that equipment was installed and operating as intended. The result of our project-
level measurement and verification tasks is a verified, ot ex post, savings value for each project in the
sample. The ratio of verified savings to reported savings is the project's reolization rate. A realization
rate of 100% indicates that no adjustments were made to the reported savings value,
ln 2011, Cadmus' nonresidential impact evaluation sample consisted of L79 electric and gas projects.2 Of
those , the majority (n=112) required a saving adjustment by more than 10%. That is, 63% of projects
had realization rates of either LlO% or greater, or 90% or lower. Specifically, just 35% of electric projects
' This number reflects projects with gas savings and electric savings. We actually evaluated 157 unique projects,
some of which achieved dual-fuel savings. For the purpose of the realization rate review, we treated gas
savings separately from electric savings.
Staff_PR_46 Attachment A Page I of 18
and 42% of gas project realization rates ranged between 90o/o and 110%. This changed in 2OL2, when the
majority of projects (64 of 101)3 experienced realization rates between 90% and 110% (see Figures 4 and
5 below).
Cadmus analyzed how frequently the evaluation resulted in an upward or downward adjustment of
reported savings, by how much, and the reasons behind the discrepancy between reported and
evaluated savings. The purpose of this review is to provide Avista with information to assist in improving
the reliability of the reported savings in the future, thereby improving realization rates for the
non residentia I portfolio.
Direction, Frequency, and Magnitude af Veritied Sovings Adjustments
Cadmus determined that when savings needed to be adjusted by more than tO%, they were more likely
to decrease than increase. ln other words, most reported savings for projects in this group were being
overestimated, and the verification process resulted in a downward adjustment. This was true for all
2011 projects, and for all20L2 electric projects. ln2Ot2, gas projects required more upward
adjustments.
2011 Projects
tjggre_ZFign{€4 illustrates the distribution of realization rates in increments for 2011 projects. ln 2011,
51 electric projects had a realization rate below 90% (42o/ol, while 27 electric projects had a realization
rate above LLO% (23%\ Gas projects exhibited a similar pattern, with 26 projects having a realization
rate below 9Oo/o (44%l and eight having a realization rate above LL}% (1.4o/ol.
Figure ?4. Distribution of 2011 Realization Rates by lncrements for Electric and Gas Projects*
*Note: Percentages may not match above text exactly due to rounding
' The full2OL2 impact evaluation sample contained 109 projects. We excluded eight projects from our analysis
that still had measurement and verification activities occurring at the time of writing this report.
r Below 50%
',: lLOYoto L25Yo
J50to75%
wL25%to L50%
w75to90%
r Over 150%
Gas Projects (n=59)
Proportion of Reported
Therms
Staff PR 46AftachmentA Page I of 18
For electric projects, the relative proportion of reported kWh savings in each increment was relatively
consistent with the number of projects in that increment. However, for gas projects, the relative
proportion of reported therm savings in each increment did not accurately represent the corresponding
number of projects. For example, while just L9% of gas projects experienced a realization rate of below
50% (but more than 0%), these projects represented 47% of reported savings.
Dividing the projects by increments revealed that a large portion of the projects with realization rates
below 90% were in fact below 50%, and most of the projects with realization rates over 110% were
actually over LSO%. This indicates that not only was the range of realization rates large, but a significant
portion of reported savings values were substantially different from verified savings, requiring an
adjustment of 5Oo/o or greater.
2012 Projects
ln2OL2, realization rates improved. Rates were less variable, and projects required smaller reported
savings adjustments than those in 2011. For example ,6Lo/o of electric projects and 67% of gas projects
had a realization rate between 90% and 110%, leaving only approximately one-third of projects that
required an adjustment over 10% (see Fisure 3F+gu++.5).
Of the 2012 electric projects that required an adjustment over 10%, most required a downward
adjustment (18 projects; 31%). This is consistent with 2011 results. Of those 2072 gas projects that
required an adjustment over LO%,the direction was upward (eight projects; 19%).
Figure 35. Distribution of 2012 Realization Rates by lncrements for Electric and Gas Projects
*Note: Percentages may not match above tefi exactly due to rounding
Cotaloging Projects with High and Law Reolizatian Rotes
To understand more about the projects that had severe adjustment factors (very high or very low
realization rates), we conducted a desk review of the project files and engineering analyses for a sample
r0
ffi 90% to 110%
Electric Projects (n=59)
Proportion of Reported kwh
Gas Projects (n=42)
Proportion of Reported Therms
I Below 50%
.',.tL0Yoto 125%
a50%to75% w75%ro9O%
#t25%to 150% : Over 150%
Staff_PR_46 Attachment A
10
Page 10 of 18
of projects from 2011 and 20L2. Specifically, this sample entailed projects with electric savings that had
been adjusted by over 25o/oin either direction (a realization rate below 75% or above t2\o/ol.
The original sample size was 75 projects; 57 from 2011 and just 18 from 2072. Upon reviewing the 2011
project files, we found that seven projects did not have sufficient reported savings documentation to
accurately conclude the reason for the savings adjustment. Therefore, the final 2011 sample size was 50,
leading to an overall sample size of 58.
Based on our review, Cadmus concluded that there were nine main reasons for the savings adjustments;
these are outlined in Table 4Ta$le-4.
Table 4. Reason Categories for Variable Realization Rates
1. Participant Operator Error
I
i. C"rcrirti"" E;;;;i; C6;n;J -
,
Savings required adjustment due to customer actions, such as installing or
operating equipment incorrectly
Reported savings calculations or assumptions were incorrect
Cadmus used updated deemed savings values for ENERGY STAR clothes
washers, dishwashers, freezers, and refrigerators to verify savings,
requiring an adjustment from the reported values, which relied on older
Cadmus used metering results to inform verified savings, while Avista used
other tools to genera-te reported savings estimatgs
Both Cadmus and Avista used metering results to inform savings values;
.fr oweve r, th e 99m gan j e1'- p: ra m
-ete J: ol_.!! m j 1c oire-59
Some values in the database extract were erroneous due to a database
error, not a human error, and savings needed adjustment to reflect the
accurate value
I Cadmus and Avista used different methodologies to calculate savings (i.e.,
I, regression analysis versus simulation), creating different results
:I The reported savings for some lighting projects were based on incorrect
HOU assumptions
The on-site equipment parameters (size and efficiency) differed from the
: assumptions used in the original savings estimate
3. ENERGY STAR@ Appliances
Deemed Savings Update
4. Cadmus Metering Results vs,
Avista Simulation or Analysis
5. Cadmus Metering Results vs.
nv!1q Metering Results
6. Database Error
7. Cadmus Calculation
Methodology vs. Avista
Calculation Methodology
8. lnaccurate Lighting Hours-of-Use
(HOU)Estimates
9. Equipment Verification
ln 201.1, the most frequent reasons for savings adjustments of 25% or greater were due to metering
results being over the original estimates formed using simulation or analysis (n=10) and calculation or
assumption errors in the reported savings values (n=10). Other top reasons included ENERGY STAR
deemed savings updates (n=9) and differences in Cadmus' and Avista's calculation methodology (n=8).
ln 2072, there were far fewer projects with adjustment factors of 25% or greater. The top reason
categories in 2Ot2 stayed relatively consistent with those in 2011, excluding the ENERGY STAR deemed
savings updates.
Staff_PR_46 Attachment A
11
Page 1 't of 18
Fiaure 4EigN illustrates the number of projects in each of the reason
categories outlined ip Table tllable4, scross both yesrs. Appendix A
Table 8 catalosues the oroiects requiring a savines adiustment of 25% or greater.
Table 8Table8, at the end of the memo, lists the specific projects included in the review and a
description of each project's specific savings adjustment.
Figure ![6. Number of Projects with Savings Adjustments of 25o/o or Greater by Category,2Otl-2A12
lmpact on Gross Sovrngs
While the majority of savings adjustments in 2011 resulted in decreased savings, certain reason
categories experienced realization rates higher than 100%, on average. For example, three reason
categories (Cadmus Metering Results vs. Avista Simulation or Analysis, ENERGY STAR Appliances
Deemed Savings Update, and Equipment Verification) resulted in increased savings. ln other words, the
projects in these groups experienced realization rates higher than 100%, on average.
ln 2012, just one reason category (Cadmus Metering Results vs. Avista Simulation or Analysis) resulted in
increased savings. Projects in the other 2012 reason categories (Calculation Error in Reported Savings,
Cadmus Calculation Methodology vs. Avista Calculation Methodology, and Participant Operator Error)
resulted in decreased savings.
The aggregate kWh impact for each 2011 reason category is listed in Table STabls4. The aggregate kWh
impact for each 2012 reason category is listed in Table 6Tab{e€.
L2
10
0
E8'a
4$oo
o€4tr
=z
2
.20L2
.2OLL
'.""-" ".f.{.s1""."s-i1f$c
$J"un
Staff_PR_46 Attachment A
1,2
Page 12 of 18
@
o(o
oct)(E(L
\t
o
E!oC'
(oor
dIL*t
Ga
xxo<t
ssot\
S:)l xFt
do\dO,O O
<fln(noro
lri, -. r-
iFl;,ro i1N:ltn
i
ii(tlid
:0orO
AJ
o-
Et!
co,F
(o
(!
o
u(!o-
.E
(l)
-c
.g
bI)c't
(s
EC]E
c.,
fi
o
o
o
c
OJI
OJo-
(g
(E
0.,ucc,L
QJ
€
OJc
0)s
,9
,qEt---*
N:Nsf .mN:N
sli rYt'otFi
xrsodN
_^-, _
| .lnor,o,ut
: trlIn
.N
,,,,,,.',,1,
IF-l I lYlrr l@Fl IFl
:otro
i<l
I
o)l(D(o l<fCO rO]Nlo: t-{
,ln.:.
.-.r i olln
1
l
I
I
:
,
c'..9 1orUiE]L]o, I):eiclo:Eio_-'=lgE, OLLI F
=6q:OJJd,fr,ood_.gbn
OoJ,=
.! 6,' O)
^,zoJa6>(E:--oF(!s€Eou<
I
l
:
l
I
Ij
1:
ttt
\O \O \O
N ir{ F{
(n Fl slF{ lt.o l,}\ lo{ c1
6llft: O00.r\ (or'\ I r{
r{Or\@ulr .1Fl I 1..r{ l'\rtr $
Nsf(n (n6: rnrd. ra6: rOFl: (o
Ni F{
fn
Flr\
@cor\
ro
oL
UJ
o(,
o)o.o
c(oo.'o,F
(o
CL
\o
Nxo
"- ' *---t-*--*
N:CONlNui]lnr"-:dr.t{,iN
i;R'Xnoo:
Orsf$:+f- : Orlorsf
>R .;QOl\
NiN(oif\c{:sr\iNlJ, : tO
O)iolO:str.. \l
fni@stlNFllr{
Fl l\(oNG,l : otrtIstl/t otFl:(n
a(,\o
r{iOtrr{(\r i o'r 1 tlFtio-r.1(Drot{N:o1rot6i
t::i
x>RS(oNFl:d
r'{ lOr.{ OriN
Or FlN
>eodt.
ol@O)
NLr)oI
Fl
sm
ChiFl ,moo:(Y',):siOlit-rlOOriNrr\[e-mail].
,
ooroGl(o,(oorNiN,0O(ntr\l(.oiNlrnirnj
r{lFlt
,: o,E ii'og ,!6h 3sr .6i=a5 il PE gg6
-*b.s =o-=o5{Ep5 E E oo & E 9 €i=si' *;dg;;"
: ! E s p E p: E:5 e
E E fi M H E E g 6iE E
ooOLr}(oFlrnF.:Nri(oio)N:lnG1:G}l:
o
(!o(,
o\o
tnN
o
I
0,
E
t!o
u!',(!ltt
g
3
a{,'5'
o.
o
q,l
!ounc,
t!U
o
t!o
,t
3
!o
G'o
o
b!c
o(n
!o
'io
-c,c(!
'oo
oo.0,u
oal
'd(,3(0
otoriirl
@
o*
ocn(gIL
o,EE(,)s
@ol
EtL
ul(E
U)
..-'**-'i^**'i^_*
S ilR iXel 1O iOli;ii
>R i "SF{ : :trt
;l'-; oio-r{,<t<tOtD6,1 ; an <l o.Ei
lrl- : 6) gt ,d sflrl ! (f F iN irA
i I | | i'ii,
;::,:;'i - :. i :'!i:NN'osfNl,)FN:r\\o(oor<rroNor r! Ot F{st'ilD
:
>Ro\o
st
o
(ooL(9
o
o\lnN
oq
o
E
t!
u0c
(tvt
&'t
IJq,'5'
o.
o
(,
ou!4J
ou
o
ooE
.:
3
'1Oo
(!
(Jog
n0tr'iot1To
'io
!
Itt1'q,tonod,
N()N
ut
C)3(!F
x >q >R ssN st Fl O:tXriAj
,!:iE:(o : : i ,qi;i:ci : : :O: : , .Ei5(r1 O N O,n!*N N Ol :O mi5O) . i-{ q O_ .t: orrYi d r'i ioi+:st (n rilN F i YN tn !i :':q,H-.'tr, .0,m sl 6 ' :Ll.t !NNto'loPn q \ idl ,.E@ 00 0 :lr -(o ro st :t\ hb\ or m io.. .s!-{ :m ;,, , ' : , ',io
Fl rrt 6 Oi<i.PJ , Ln O O'l\:.Y(\t : d) r\l o.t\ i:=S -i d jdib
'4: q st, :<.,=r{ : ri : iti;E, ; ,o
I -lP:i(o , ro , rn ,;ioq'9.t.I: i ;-o, : tE
: ' i 1g
uoio:9.:rg;.e € T i :3q- O tr gi iort Z'E : oi OiF,6 ,E E Ei€i;= ,&* ti H: b,E.q c 9i tsI ci'E E '; E, E: o:ir3 = o oJ. iliE=dto-Ei*!rcQ: 9 q,:)roUOcEi:E(Jo6.o=:Et'6;2;!HoE,.9: ,gitr > i = E,.9'-.ei .U € < * 6'=,E g rI >,o ui; (! fo:G O -t <iLJ > Z. l) (^.d Fi+::ii
Fisure 5$igure-7 illustrates 2011 projects in each reason category as a percentage of the total sample
compared to the percentage of each categories' net kWh impact. While the ENERGY STAR Appliances
Deemed Savings Update category contained nine projects (representing about 8% of the total sample),
the net difference in ex onte and ex post savings was actually minimal: a gain of 1,151 kwh (see Table
lTabJeS), less than O.O7o/o of savings in the impact evaluation sample. The Cadmus Calculation
Methodology vs. Avista Calculation Methodology category had similarly minimal savings despite
containing a relatively large number of projects (eight). On the other hand, the Cadmus Metering Results
vs. Avista Simulation or Analysis and Participant Operator Error categories represented 8% and 3o/o of
projects, respectively, but the net differences in ex onte and ex post savings represented L3% andTo/o of
the total verified savings in the impact sample, respectively.
Figure $. Relative Proportions of Projects and Savings lmpacts by Reason Category, 2011
r Metering vs. Simulation
' ES Appliances Update
ir lnaccurate HOU
,ir Database Error
I Calculation Error, Rprt'd Savingsr Diff. MethodologyI Participant Error
x* Diff. Metering Results
Net Difference as
% of Verified
Savings in Sample
% of Total Projects
in Sample
ln 20t2, the percentage of projects in each category was higher than the respective percentage of kWh
savings in each category (see erfor! Not a valid bookmark self-reference.Figr++e8). For example, the
Cadmus Metering Results vs. Avista Simulation or Analysis and the Cadmus Calculation Methodology vs.
Avista Calculation Methodology categories both represented 10% of all projects in the evaluation
sample, but their net differences in ex ante and ex post savings were relatively small, representing only
2% and 4% of the total verified savings in the sample, respectively.
Staff_PR_46 Attachment A
16
Page 15 of 18
Figure Q8. Relative Proportions of Projects and Savings lmpacts by Reason Category, 2012
r Metering vs. Simulation lCalculation Error, Rprt'd Savings Diff. Methodology r Participant Error
Net Difference as % of Verified Savings in
SamPle
%of Total Projects in Sample
Co ncl u sion s a n d Recom me n dati ons
Based on the above findings, we offer the following conclusions and encourage Avista consider the
recommendations listed below to improve their internal processes.
Large Project Review Process
Conclusion: Avista's 2011 Large Project Review process was not implemented successfully due to a
series of communication issues and the absence of a mechanism to address concerns about project
parameters and correct mistakes. ln the first half of 2013, Avista has been designing a new process for
all site-specific projects. While this process is underway, we have several recommendations may assist
Avista with successful implementation and an effective process.
Recommendations:
o Elfectively communicate the new projed review process to all key teom members. Many of the
issues identified through Avista staff interviews regarding the prior review process centered on
communication challenges. When implementing the new process, ensure that all stakeholders
have a clear understanding of the review goals and correct protocol.
c Ensure there ore clear protocols in place for oddressing issues identilied during the review and
the spot-check To ensure that Avista and its customers are benefiting from the time and
resources dedicated to this process, consider implementing some check-points and policies to
clarify how and when to alter project savings and incentive levels if issues arise during the
review. This may include designating a senior-level point person to serve as the decision-maker
for questions or disagreements regarding a project or its calculation methodology. Consider
identifying methods to ensure that all issues are discussed and resolved before incentive
amounts are communicated to the customer.
c Establish a gool for the number or percentoge of projeds that should undergo o random spot-
check. Avista's new process dictates that the PPA team will independently review a sample of
Staff_PR_46 Attachment A
17
Page 16 of 18
projects, in addition to the peer review process. We suggest establishing a clear metric for the
number or percentage of projects this sample will include, such as five projects or 10% of all
projects.
Establish a reosonoble goal for how long the review process should toke, A core challenge with
the prior review process was the time lag. Keeping in mind that any process aimed at improving
the quality and accuracy of incentive payments and claimed savings will add time to existing
procedures, Avista should internally discuss the amount of delay that is reasonable. lt may be
beneficial to create objectives for how long various steps of the process should reasonably take.
For example, Avista could establish one goal to complete the first Top Sheet review within a
certain timeframe, then establish another goalto guide how long it should take to resolve any
issues, if identified.
Consider odopting a tiered approach to the review so that larger, high-risk projeds receive
more scrutiny before controcts ore issued and incentives are poid. Under the planned
approach, all site-specific projects will undergo peer review. Often, utilities employ a risk-
mitigation approach to ensure that the largest and most expensive projects receive the most
rigorous review before they are approved. Avista might explore adjusting their review process to
focus the most time and resources on larger projects. An example of this type of approach is
provided in Table 7Ta${e+.
Peer Review
secona engineeiini neview Projects above $50,000
Third Engineerine Review
PPA Review
Pre-tnstaltition visiii
nanOom arOit (rpoi-cft".i)
Projects above $75,000
Projects above $1OO,OOO
Projects above S1OO,OOO, ftus others as needed
5 projects or t0% of all projects
Consider struduring rondom spot-checks, or "oudits,' to occur ot vorious times ol the process.
The current review structure plans to have some projects receive independent review after the
project evaluation report is complete or after the project is paid, so that any rnistakes can be
corrected for future projects. However, it may be beneficial to stagger projects so that a
random portion also receives independent audits before incentive information is communicated
to the customer.
Database and Realization Rate Review
Conclusion: The accuracy of Avista's claimed savings, measured by realization rates, improved
significantly from 2011 lo 2072. Three of the four main reasons for large savings adjustments in 2Ot2
are largely outside Avista's control. However, Avista can still improve the reliability of claimed savings
estimates falling into the reason category of Calculation Error in Reported Savings.
Table 7. Example of Tiered Approach to Large Project Review
Staff_PR_46 Attachment A Page 17 of 18
Recommendation:
o Continue to move forward implementing the new review process to identify and resolve savings
calculation errors.
Conclusion: Most of the nonresidentialprojects were compliant with the 2012 tariff rules, but
disagreement among DSM staff on tariff interpretation makes it difficult to draw conclusions about
prescriptive projects. Avista has already begun updating the tariffto address this concern and create a
more coherent policy. There are several improvements Avista can make to data tracking activities to
clarify policy compliance on a project-by-project basis and improve data collection overall.
Recommendations:
o Clearly document legocy projeds or morket transformotion projects in Saleslogix. Avista's
tracking system specifies measure type, but lacks detailed information such as whether the
project involved a T12 to T8 lighting conversion. This makes it challenging to understand which
projects are considered market transformation. Further, legary projects are not specified. To
streamline internaltracking, auditing, and evaluation, consider adding a field to denote which
projects are eligible for transition policy (legacy projects) and which projects are considered
market transformation, as well as any other project characteristics that warrant exception to
tariff rules under Avista's new policy.
c Continue to improve doto entry in Saleslogix to reduce missing or incorrect fields and enhance
the comprehensive dataset.
Staff_PR_46 Attachment A
79
Page18of18
AVISTA CORPORATION
RESPONSE TO REQUEST FOR INFORMATION
JURISDICTION: IDAHO DATE PREPARED: 0110612014
CASE NO: AVU-E-I3-09/AVU-G-13-02 WITNESS:
REQUESTER: IPUC Staff RESPONDER: David ThompsonTYPE: Production Request DEPARTMENT: DSM Policy, Planning & Analysis
REQUEST NO.: Staff-48 TELEPHONE: [phone]r
REQUEST:
Please explain why Avista sent only one-third of projects with incentives over $50,000 to the
Planning, Policy, and Analysis (PPA) team for review in20l2, even though Avista did not suspend
the review process until January 1,2013.
RESPONSE:
The understanding of the engineering team in sending projects to the PPA team for review was that
the review process was designed for site-specific projects with individual measures exceeding
$50,000. Therefore, prescriptive projects and projects with multiple measures that in aggregate
were more than $50,000 were not consistently forwarded to the PPA team for review.
AVISTA CORPORATION
RESPONSE TO REQUEST FOR INFORMATION
ruRISDICTION: IDAHO
CASE NO: AVU-E-13-09/AVU-G-13-02
REQUESTER: IPUC StaffTYPE: Production Request
REQUEST NO.: Staff-S1
DATE PREPARED: 0l/07 12014
WITNESS:
RESPONDER:
DEPARTMENT:
TELEPHONE:
David Thompson
DSM Policy, Planning & Analysis
(s09) 49s-282t
REQUEST:
While the PPA independent review of large projects was in place (August 2011- January 2013),
please provide the percentage of reviewed projects that were found to: l) rely too heavily on
customer data,2) use incorrect interactive effects,3) have missing documentation (such as
invoices), or 4) contain engineering errors that resulted in incorrect claimed savings and incorrect
incentive amounts.
RESPONSE:
From August 2011 through January 2013, seven (7) Idaho non-residential projects were reviewed
by the PPA team. It is important to note that this review was conducted at a "single point in time"
of the project life cycle (i.e. project not yet completed). Issues identified at the time were reported
to the Implementation team for their review, response, and opportunity to address. Some items
were subsequently classified as non-issues based on an interpretation of subjective engineering
judgment.
For the seven projects reviewed during this time period:o Three final reports incorporated some adjustments to the energy acquisition and incentive
calculations associated with the energy effrcient measure;o One project missing documentation, that when collected from the Implementation team,
substantiated the project costs along with the claimed energy efficiency acquisition and
incentive calculation;o One project did not complete and, therefore, did not claim savings or go to payment;o One project's primary recommendation was for a regulatory review prior to payment that
was then issued; ando One project had multiple recommendations that were addressed but for a remaining
question about the applicability of a baseline assumption. Post-installation measurement
and verification of the installed unit showed the actual savings were in alignment with the
estimated savings of the original report and updated both the claimed energy acquisition
and incentive amounts.
AYISTA CORPORATION
RESPONSE TO REQUEST FOR INFORMATION
ruRISDICTION: IDAHO
CASE NO: AVU-E-13-09/AVU-G-13-02
REQUESTER: IPUC StaffTYPE: Production Request
REQUESTNO.: Staff-52
DATE PREPARED: ll2l20l4
WITNESS:
RESPONDER:
DEPARTMENT:
TELEPHONE:
Lori Hermanson
DSM Policy, Planning & Analysis
(s09) 49s-46s8
REQUEST:
Please provide a trend chart of all non-incentive utility pro$am costs as a percentage of total DSM
program costs from 2002to2012.
RESPONSE:
The chart below provides a trend of all non-incentive utility program costs as a percentage of total
DSM program costs from 2005 to 2012. Non-incentive utility costs include labor and loadings,
evaluation, measurement and verification, stakeholder meeting facilitation, customer outreach as
well as third party program adminishation such as Energy Smart Grocer, Simple Step Smart
Savings bulb and showerhead buy-down, appliance recycling, mobile home duct repair and a
behavioral progrulm.
Non-incentive utility costs in 2011 increased mostly attributable to the launch of Avista's 'CFL
contingency' program which enabled the Company to reach its energy efficiency targets. In2012,
non-incentive utility costs were up from earlier years due to increased contribution from the
Northwest Energy Efficiency Alliance coupled with increases in labor and labor loadings costs.
In 2005, the Company converted its financial systems so information prior to 2005 is not
consistent with the current tracking of non-incentive utility costs.
Non-incentive Utility Costs as
percentage of DSM Budget
4oo/o
35olo
3OYo
2sYo
ZOYo
lSYo -Ntuc%
LOYo
5o/o
OYo
200s 2006 2007 2008 2009 20Lo zolL 20L2
AVISTA CORPORATION
RESPONSE TO REQUEST FOR INFORMATION
JURISDICTION: IDAHO
CASE NO: AVU-E-I3-09/AVU-G-13-02
REQUESTER: IPUC StaffTYPE: Production Request
REQUEST NO.: Staff-55
DATE PREPARED: 0110612014
WITNESS:
RESPONDER:
DEPARTMENT:
TELEPHONE:
Chris Drake
DSM
(s09) 49s-8624
REQUEST:
Please provide a detailed list of measures that were not compliant with DSM tariff rules. For each
measure, please provide the simple payback period, the incented amount and whether or not the
measure exceeded the 50% cap. Please provide the specific cost-effectiveness calculations that
demonstrate the measure was still cost-effective.
RESPONSE:
Please see Avista's response StafLPR_55C, which contains TRADE SECRET,
PROPRIETARY or CONFIDENTIAL information and exempt from public view and is
separately filed under IDAPA 31.01.01, Rule 067 and233, and Section 9-340D, Idaho Code.
The Company followed processes and procedures as set forth in its DSM tariff Schedule 90.
Schedule 90 previously did not explicitly mention prescriptive (or 'standard offer') programs.
Prescriptive programs allow the Company to offer a fixed amount for the installation of measures
without the need for individualized calculation of the rebate or the signature of a pre-project
agreement. This approach is important to the efficient marketing and administration of programs
promoting small measures. Prescriptive programs have been a critical element to the Avista DSM
portfolio for many years. The Company proposed and received approval of additional language to
provide greater clarity of how these progrirms are offered and designed in compliance with tariff
rulesl. The clarification was as follows:
Prescriptive progftrms are guided by the typical application of that measure in
accordance with the previously defined incentive structure. Incentive levels for
these progams are based on market conditions at the time of program design and
are not dependent on actual project cost relative to incentive caps. Incentives shall
not exceed project costs.
Please see Staff PR_55 Attachment A. The attached list includes 48 measures along with their
simple payback period, incentive amount, percentage of measure cost and cost-effectiveness
calculation. Notably:
o Of the 48 measures that went over 50ol0, 45 of them were by only about $0.50.
' Advice No. l3-04-E