20111114Reply Comments.pdf

"~.ROCKY MOUNTAINPOER
A DIVISION OF PAClACRP
Q¡:cr Ql \. ~_ i....
November 11, 2011 2011 NOV 14 AM 9: 22
201 South Main, Suite 2300
Salt Lake City, Utah 84111
VI ELECTRONIC MAL
AND OVERNIGHT DELIVERY
Jean D. Jewell
Commission Secreta
Idaho Public Utilities Commssion
472 W. Washington
Boise,ID 83702
RE: PAC-E-II-13 - In the Matter of the Application of Rocky Mountain Power Seeking
Authoriation to Suspend Future Program Evaluations of Schedule 21, Low Income
Weatheriation Servces for Income Qualifying Customers
Dear Ms. Jewell:
Please fid enclosed for fiing in the above-captioned case an origial and seven (7) copies of
Rocky Mountan Power's reply comments to the Idao Public Utilties Commssion Sta and
Communty Action Parership Association of Idaho's comments fied in the above referenced
mattr.
Sincerely,~KK~/~
Jeffey K. Laren
Vice President, Reguation
cc: Brad Purdy/CAP AI
Enclosures
Mark C. Moench
Daniel E. Solander
Rocky Mountain Power
201 S. Main St., Suite 2300
Salt Lake City, UT 841 i 1
Telephone: [phone]
Fax: [phone]
Attorneysfor Rocky Mountain Power
BEFORE THE IDAHO PUBLIC UTILITIES COMMISSION
IN THE MATTER OF THE APPLICATION OF )
ROCKY MOUNTAIN POWER FOR AN )
APPROVAL OF AN EVALUATION )
OF SCHEDULE 21, LOW INCOME )
WEATHERIZATION SERVICES OPTIONAL )
FOR INCOME QUALIFYING CUSTOMERS )
CASE NO. PAC-E-II-13
COMES NOW PacifiCorp dba Rocky Mountain Power ("RMP" or the "Company") and
pursuant to Rules 56 and 121 of the rules of Procedure of the Idaho Public Utility Commission
(the "Commission"), submits its reply comments in the above referenced case.
Background
On April 29, 2011 the Company fied an application seeking an Order to remove future
obligation of the Company to perform program evaluations of its Low Income Weatherization
Service Optional for Income Qualifying Customers program offered under Schedule 21.
On October 28, 2011 the Idaho Public Utilties Commission Staff ("Staff') and
Community Action Partnership Association of Idaho ("CAP AI") fied comments concerning the
Application of Rocky Mountain Power in Case No. PAC-E-l1-13. The Company believes that
the comments contained several inaccurate statements or misunderstandings that it respectfully
requested an opportunity to clarify.
1
Response to Staff's Comments
Staff raised concern that "...a biling analysis was applied five years apart (one year of
biling data on either end of the three year period selected) to low income customers who move
much more frequently than regular-income residential customers."i
The Company would like to clarify that the billng analysis consists of comparing the
usage prior to program participation with the usage after program paricipation. In an analysis of
a single year this would result in three years of data in the analysis, one year prior to
paricipation, the year of paricipation and one year subsequent to participation. Due to the
limited number of paricipants, the entire three year evaluation was treated as a single cohort.
The pre-period, therefore, was the year prior to the first program year and the post-period was the
year subsequent to the final program year. This results in a five-year total analysis period. In its
analysis, Cadmus addressed the effect of movers by including them in the savings regression
model, thus controllng for their impacts.
Staff also comments that "The biling analysis captured 1 00% of the savings, even though
Rocky Mountain only paid for 75% of the energy efficiency measures installed during the life of
this program. Correcting this over-estimation by claiming 75% of the savings would further hurt
the cost effectiveness of this program.,,2 The Company claims 100% of the savings because it
believes that unless the paricipating customers are free riders, i.e. they would pursue the savings
without the availabilty of the incentives, the energy savings associated with the project would
not otherwise have been achieved. Consequently the energy savings in total and any attributes
associated with the energy savings are a benefit of the investment.
i Staff Comments page 6, third paragraph.
2 Staff Comments page 6 and 7.
2
Staff states "When a CAP installs a suite of measures in a residence, it records and
reports to Rocky Mountain the number of measures installed and the cost of each measure, but
Rocky Mountain does not capture this data in its database.,,3 The Company captured and stored
the CAP reports in its legacy REST database. In 2010 the Company implemented a new tracking
system (TrakSmar) which went live in 2011. The following information is stored in the legacy
REST system:
1. Type of dwellng - single family/manufactured home/aparment
2. Primary heat type
3. All measures installed
4. Total estimated kWh savings per measure
5. Date services provided
6. Agency providing services
7. Total cost per measure
8. RMP rebate per measure
9. RMP payment towards agency admin.
As noted, Cadmus utilzed a billng analysis approach to evaluate the LIW A program
which did not use some of the information stored in REST. The Company would also note that
biling analysis is the industry-standard method for verification of savings in residential
weatherization programs, especially for low income programs. The large number of paricipants
and non-paricipants allows for the generation of statistically significant savings estimates.
Comparison groups are created by usage characteristics to account for differences in housing
size, occupancy, and other demographics which may affect savings. Using deemed savings
numbers or engineering algorithms for programs offering home weatherization may overestimate
savings due to the interaction of the measures.
On page 9, Staff recommends; "that Rocky Mountain facilitate implementation of
electronic fie transfer capabilty so that information can be transmitted electronically from the
agencies to Rocky Mountain and retained for future analysis." As noted above Rocky Mountain
3 Staff Comments page 8, first paragraph.
3
Power implemented a new database system effective January 1, 2011, TrakSmart. In the
development of this system the Company considered the electronic transfer of the low income
weatherization information into the tracking system. Given the cost associated with the
implementation of this functionality and the number of anual completed homes in Idaho, the
decision was made to delay the implementation of this fuctionality until it proved cost effective.
Response to CAP AI's Comments
CAP AI's comments seem to reflect a misinterpretation of the findings of the April 20,
2011 Cadmus evaluation of Rocky Mountain Power's low income weatherization program. The
comments question the credibilty of the findings of the report, indicating the report has "fatal
flaws". CAP AI's comments are based on a review of the report by Mr. Roger Colton. The
Company wil address each ofMr. Colton's observations below.
Before addressing Mr. Colton's review of the evaluation, the findings of Cadmus'
evaluation should be clarified. CAP AI asserts that "The CADMUS study proclaims that Rocky
Mountain Power's curent program is not cost-effective based on traditional evaluation
methodologies and criteria.,,4 This statement, which appears to be the fudamental basis for the
remainder of CAP AI's comments, is factually incorrect. Cadmus' report clearly states that "the
program is cost effective when all benefits are considered."s This is consistent with the
Company's Application:
"Rocky Mountain Power's portfolio of energy efficiency programs are cost-
effective, but the cost-effectiveness calculations included in the attached
evaluation indicate that Schedule 21 is not cost-effective from the Total Resource
Cost (TRC), Utilty Cost (UCT) or Ratepayer Impact (RIM) perspectives unless
non-energy benefits are included.,,6 (Emphasis added)
4 CAPAI's comments, page 2.
5 Page ES-4 of the Cadmus report states "When all benefits (energy and non-energy) are included, the program is
cost-effective from both the total resource cost (TRC) and PacifiCorp total resource cost (PTRC) perspectives, at
1.5 and 1.23 respectively, as shown below in Table 4."
6 Section 6 of
Rocky Mountain Power's April 29, 2011 Application PAC-E-I 1-13.
4
CAPAI admitted that LIWA is a unique DSM program and "Consequently, many of the
program benefits fall outside the traditional DSM cost-benefit evaluation methodologies.,,7
CAP AI may have mistakenly relied upon the portion of the analysis that indicated the cost-
effectiveness based only on the direct energy savings. CAP AI acknowledges that it was not a
pary to the "Memorandum of Understanding for Prudency Determination of DSM
Expenditures" entered into by the Commission staff and the three investor owned electric
utilities in Idaho on December 21,2009, and may not be aware that this MOU clearly recognizes
the value of non-energy benefits in determining cost effectiveness and directs the utilties to
include them explicitly as benefits rather than reductions to cost.8 Cadmus' study adhered to the
methodology of the MOU and found that the Rocky Mountain Power's low income
weatherization program is cost effective when non-energy benefits are included. CAP AI states
that conclusions derived from the study are "highly inaccurate and misleading." This seems to
contradict CAP AI's assertion that it believes the program to be cost-effective, which is the
conclusion of Cadmus' evaluation.
CAP AI correctly notes that Cadmus chose not to attempt to quantify certain non-energy
benefits. The quantification of non-energy benefits is often subject to considerable debate.
Certain non-energy benefits, such as the impact on arearages, may be determined with
reasonable accuracy through an analysis of utilty records. Others, such as health improvements,
are much more subjective and stil other potential benefits such as "reductions in crime,
homelessness, and improved living conditions," cited by CAPAI are unquantifiable. All
evaluation studies require professional judgment as to what information to quantify and
explicitly include, what information to include qualitatively, and what information is superfluous
7 CAP AI Comments page 13.
8 MOD Attachment 1, paragraph 4.
5
or insignificant and should be excluded. These decisions are driven by availability of data, the
cost of analysis and the significance of the information to the overall findings. In the present
case, Cadmus determined that inclusion of the selected non-energy benefits was sufficient to
demonstrate that the program was cost-effective. As Mr. Colton acknowledged, "the basic
approach used by Cadmus in the evaluation of the cost-effectiveness of the Rocky Mountain
Power low-income weatherization program is, fudamentally, a sound approach.,,9
The Company disagrees with CAP AI's assertion that the evaluation of the LIW A
program is "long-overdue". To the contrary, Order No. 32196 states; "The Commission
recognizes the Company's DSM Memorandum of Understanding commitments and its
compliance efforts; accepts Staffs analysis of the Company's 2008 and 2009 DSM programs and
related expenditures; finds the expenditures to be just, reasonable and in the public interest; and
finds the costs to be prudently incured and appropriate for recovery in the Company's Schedule
191 (Customer Effciency Services Rate Adjustment) tariff." At the time the MOU was entered
into, Rocky Mountain Power was already engaged in overseeing the work of 17 different
evaluations that were completed by the end of 2010 and first quarer of 2011. The evaluation of
the Company's Idaho LIWA program for the 2007 through 2009 was expected early in 2011
consistent with Company testimony at the general rate case hearings. The LIW A final evaluation
report was issued on April 20, 2011 and fied with the Commission April 29, 2011.
CAP AI noted that the Cadmus' evaluation of the Company's 2007 through 2009 LIW A
program did not include the costs the Company paid to Cadmus. Costs incurred for the
evaluation of programs are recognized in the program year the activity takes place. The costs of
the LIW A evaluations were included in 2010 and 2011 program costs. These costs can represent
9 Refer to page 25, "Assessing the Cost-Effectiveness of Low-Income Weatherization in Idaho" prepared by Roger
Colton.
6
ten to twenty percent of the annual program expenditures and wil have a significant impact on
the cost effectiveness of the programs in those years. If the Commission rules that LIW A
program evaluations should continue, the Company respectfully requests that evaluation costs
should be considered at the DSM portfolio level not at individual program levels due to the
impact these costs can have on smaller programs and the inabilty to smooth the costs between
evaluations. The Company disagrees with CAP AI's assertion that the cost of evaluations should
be less in subsequent years. A third-party program evaluation requires basic review and analysis,
even if you only account for inflation, future evaluations wil cost more than they do today.
Mr. Colton's review of the report suggests that quantification of additional non-energy
benefits, using a different methodology, adoption of quantifications determined in other studies
or the use of a non-energy benefit "adder" may increase the reported cost-effectiveness of the
program. The Company agrees, but the point is moot as noted in Table 4 from Cadmus'
evaluation that the program passes cost-effectiveness tests without the benefits suggested by Mr.
Colton.
Table 1 P C t Efi t s i I d N E B fit 2007 2009*I ~--
Benefit
Levelized Net / Cost
Cost Effectiveness Test $/ kWh Costs Benefits Benefits Ratio
Total Resource + Conservation Adder (PTRC)$0.099 $426,022 $525,295 $99,273 1.23
Total Resource No Adder (TRC)$0.099 $426,022 $491,475 $65,453 1.15
Utility (UCT)$0.099 $426,022 $346,529 .$79,493 0.81
Ratepayer Impact (RIM)$0.189 $815,476 $346,529 -$468,947 0.42
Participant (PCT)$0.083 $355,470 $744,924 $389,454 2.10
Lifecycle Revenue Impact $0.00001028.The calculations are based on the program components in place during the period 2007.2009 when Rocky Mountain Power provided a rebate
covering 75% of the cost of approved measures. As of 12/28/10, they are covering 85% of these costs, which will reduce the net benefits from
the utilty and rate impact perspectives.
Mr. Colton's report "Assessing the Cost-Effectiveness of Low-Income Weatherization in
Idaho" specifically challenges Cadmus' methodology or findings in 11 areas. The Company wil
address each of these findings in the following paragraphs.
7
Account for the disproportionate impact of price hikes and economic recession on non-
participants.
Mr. Colton states that, "Cadmus notes that while 'these factors would likely also affect the
paricipant population...the impact of these on their household would have been less than for the
(non-)participating population.' (Cadmus, at 18)10." Cadmus' report highlights two potential
factors that may have contributed to the observed reductions in consumption in nonparticipant
homes (e.g., the rate increase and the economic recession). The Cadmus report notes that
weatherization paricipants' overall decline in usage would have mitigated some of the impacts
associated with these two factors. Because they did not receive the benefit of reduced
consumption through weatherization, nonparicipants may have pursued other energy-saving
behavioral changes.
In its report, Cadmus recognized that there are a large number of factors that affect
energy use. Some of these factors may have different effects on paricipants and the
nonparticipant group. Cadmus controlled for these differences by carefully matching participant
and nonparticipant pre-program usages to the maximum extent possible.
Account for difference in persistence of savings between participants and non-partcipants.
Mr. Colton states that "The Cadmus study failed to account for the lower persistence rate of
usage reduction actions taken by non-participants in response to price increases and economic
recession."1I Mr. Colton contends that "lifetime net savings attributable to the weatherization
program are understated.,,12 Mr. Colton does not quantify these differential impacts nor does he
offer a method for how they might be quantified to support his contention. Uncertainties
10 Refer to page 4, "Assessing the Cost-Effectiveness of Low-Income Weatherization in Idaho", prepared by Roger
Colton. While the Cadmus report states that the impacts would have been greater for the participating population, it
subsequently acknowledged a tyographic error and the intent to state that the impact of the price increases and
economic recession on the savings for the non-participant population would have been greater than on the
tiarticipant population. (CAPAI-I-23, CAPAI-I-24).1 Page 22, "Assessing the Cost-Effectiveness of Low-Income Weatherization in Idaho", prepared by Roger Colton.12 Page 7, this problem, too, would affect all calculations of changes in arrearages and/or payments.
8
surounding the persistence of energy savings exist for both paricipants and non-paricipants.
Consequently, Cadmus did not apply a degradation factor to either population.
Distinguish between heating and non-heating participants and Identify non-heating non-
participants.
Mr. Colton claims that the nonparicipant group would have been comprised primarily of
electrically-heated customers (based on identification using energy assistace payments, which
are likely provided for primary heating service) and, for this reason, he asserts the study's
approach is fundamentally flawed due to this "critical analytic mistake.,,13 Mr.Colton
misinterprets the quotation in the report that indicates between 35 and 48 percent of homes
served are electrically heated. This figure represents the proportion of total homes served by the
interviewed community action program agency that are electrically heated, not the proportion of
electrically heated homes in the analysis sample.
Of the 166 paricipants included in the analysis sample, 164 were electrically heated
homes. In order to determine the impact of the two non-electrically heated homes, Cadmus re-
estimated savings excluding these two participants. This analysis indicated savings that were
nearly identical to the original gross savings estimate, see Table 1 below.
Table 1. Gross Participant Model Savings Estimates
(with and without the non-electrically heated I!roup)
Gross Savings
Group n (kWh)
Participants Only (All)166 1,987
Participants Only (Removing non-electrically heated homes)164 1,985
Moreover, Mr. Colton asserts that the nonparticipant sample selected as a comparison
group was inappropriate for this analysis due to potentially higher income and that Cadmus did
not account for this.
13 Page 6, "Assessing the Cost-Effectiveness of Low-Income Weatherization in Idaho", prepared by Roger Colton.
9
The nonparicipant group was identified based on those Rocky Mountain Power
customers that received energy assistance payments based on LIHEAP income qualifications (60
percent Idaho median income). The program paricipants are based on federal poverty level
income qualification (150 percent of Federal Poverty Level). Table 2 below shows a comparison
of the two income eligibility criteria by family size.
Table 2. Comparisons of2011 Eligibilty Crite
150%
Idaho 60%Federal
Persons in Est.Median Poverty
Household Income Level
1 $19,854 $16,335
2 $25,963 $22,065
3 $32,072 $27,795
4 $38,180 $33,525
5 $44,289 $39,255
6 $50,398 $44,985
ria
Normalized consumption for disproportionate change in days off system due to
disconnection for nonpayment.
Mr. Colton asserts that "The Cadmus study failed to account for the fact that non-paricipants
had a higher rate of disconnections for nonpayment, as well as a higher proportion of customers
experiencing any disconnections." According to Mr. Colton, this omission "would not result in a
. .. reduction in sales/usage attributable to days on which service was disconnected would
inappropriately appear as a 'usage reduction' in the non-paricipant population."
It is important to note that, as described in the report, the payment analysis (including
disconnections) and savings analysis were conducted on separate samples.14 To investigate the
possible effect of the differential impacts of disconnects on savings, Cadmus conducted
additional analysis excluding all of the participants and nonparicipants with any disconnections
14 Rocky Mountain Power's response to CAPAI Data Request No. 30
10
in either the pre- or post-periods. As shown in Table 3, excluding homes that had disconnections
during this period from both the paricipant and nonparticipant populations would result in lower
net program savings.
Table 3. Gross Participant and Nonparticipant Model Savings Estimates
(with and without disconnections)
Gross Savings
Group n (kWh)
Paricipants Only (All)166 1,987
Paricipants Only (No Disconnections)150 1,825
Nonparticipants Only (All)664 932
Nonparticipants Only (No Disconnections)589 885
Exclude previously-weatherized homes from non-participant population.
Mr. Colton states "The Cadmus study failed to account for the fact that a significant number of
non-participant homes would have received federally-fuded weatherization services prior to the
study period. As a result, the savings for these homes would have been increased relative to
previously not weatherized program participants, due to the increased ability of households to
control their energy consumption,"ls
Cadmus conducted a sensitivity analysis by assuming the same level of federally-funded
weatherized homes in 2006 as in 2007. This increased the annual per home net savings by about
5 percent (from 1,308 to 1,379 kWh).
Calculate arrearage reductions over the life of the weatherization measures.
Mr. Colton states "The Cadmus study inappropriately found that arrearage reductions were a
one-time, first year, impact rather than an anual impact. The impact of reduced arearages
should be considered over time, not merely in the first year.,,16 Cadmus is not aware of any
research indicating persistence of arearage reductions beyond 1 year. In TecMarket Works 2004
15 Page 23, "Assessing the Cost-Effectiveness of Low-Income Weatherization in Idaho", prepared by Roger Colton.
16 Page 23, "Assessing the Cost-Effectiveness of Low-Income Weatherization in Idaho", prepared by Roger Colton.
11
Study "Assessment of WHEAP and W AP Program Participation and the Effects on Wisconsin's
Low-income Population" the authors find that weatherized paricipants, after receiving energy
assistance to reduce their arearage balance, accrue 60% of that balance back within six months.
Assign cost savings to improvements in payment amounts, increased number of payment
numbers, and reduced number of disconnections.
Mr. Colton states "The Cadmus study failed to assign any utilty cost savings to reduced
arearages, reduced numbers of service disconnections for non-payments, and an increased
number and dollar value of paricipant payments. Cadmus failed to account for fudamental cost
savings such as reduced working capital, decreased bad debt, reduced and redeployed credit and
collection activities, and reduced lost sales.,,17 The Cadmus study did not attempt to
quantify all non-energy benefits. The Company agrees with Mr. Colton that the quantification of
additional benefits would increase the benefit cost ratio above its current level of a TRC of 1.15.
Account for differences in payments toward bils for other than current usage (e.g., late
payment charges, reconnection fees, etc.).
Mr. Colton states "The Cadmus study failed to account for the fact that an increased proportion
of non-paricipant payments were directed toward bils other than for curent usage. Non-
paricipant payments were disproportionately devoted to customer service fees such as late
payment charges, reconnect fees, field collection fees, and the like.,,18
Mr. Colton only mentions the possible impact of additional charges incured on the
payment analysis, without also looking at the additional utilty credits. Cadmus looked at both
the charges and credits in combination early in the analysis. The additional charges and the
additional credits substantially cancelled each other out. The net effect between average anual
charges and credits ranged $8 for paricipants to $16 for nonparticipants. These represent only 1
17 Page 23, "Assessing the Cost-Effectiveness of Low-Income Weatherization in Idaho", prepared by Roger Colton.
18 Page 23, "Assessing the Cost-Effectiveness of Low-Income Weatherization in Idaho", prepared by Roger Colton.
12
percent of the biled amount, and hence Cadmus believed that their inclusion would be
insignificant to the payment analysis.
Include pre-installation months in "post-installation" data.
Mr. Colton states "The Cadmus study failed to ensure that all "post-installation" data actually
represented time subsequent to the installation of weatherization measures. Including pre-
installation data on usage and payments/arears has the impact of reducing the calculated impact
of the weatherization measures."
19 The date field included in the paricipant database
corresponded to the date in which the project information was entered into the database, rather
than the installation or project completion dates. For this reason, some projects with dates listed
after October 2009 actually had installations prior to October 2009 and did not conflct with the
selected post-period.
Cadmus repeated the analysis, excluding all participants with date field values of October
2009 or later. 41 participants (twenty-five percent) were eliminated when this screen was
applied. As shown in Table 4, the gross paricipant savings increased by nineteen percent. Given
this potential increase in savings, additional research on the project completion dates may be
waranted.
Table 4. Gross Participant Savings Estimates
(with and without overlapping dates in post-period)
Gross Savings
Group n (kWh)
Participants Only (All)166 1,987
Paricipants Only (No dates after Oct 201 1)125 2,344
Counts bil credits as a "negative arrears" rather than as a $0 arrears.
Mr. Colton states "The Cadmus study inappropriately included monthly bil credits as a
reduction to arears. An account with a bil credit in any given month has a $0 arears, not a
19 Page 23, "Assessing the Cost-Effectiveness of Low-Income Weatherization in Idaho", prepared by Roger Colton.
13
negative arears. ,,20 Cadmus conducted additional analysis of payments, setting negative arears
to zero. The average net arrearage reduction increased from $31 per customer to $34 per
customer, resulting in an increase in the anual arearage benefit from $8,302 to $9,031.
Conclusion
Rocky Mountain Power is concerned about the significant burden program evaluation
costs place on the smaller DSM programs where these costs can represent ten to twenty percent
of the annual expenditures for the program. For the evaluation period, 2007 through 2009, there
were no evaluation costs included because they were not incurred until 2010 and 2011. That
expense wil be included in the evaluation of the program results for those years. Staff, CAPAI,
and the Company all acknowledge that there are societal non-energy benefits from the LIW A
program. Rocky Mountain Power's portfolio of energy effciency programs is cost-effective, but
the cost-effectiveness calculations included in the evaluation indicate that Schedule 21 is not
cost-effective from the Total Resource Cost (TRC), Utility Cost (UCT) or Ratepayer Impact
(RIM) perspectives unless non-energy benefits are included. Staff and CAP AI raised questions
about what non-energy benefits should be included in LIW A evaluations. The Company is open
to Staffs workshop recommendation to determine a standard evaluation approach for low-
income programs. If the Commission determines that evaluation of LIW A should continue, the
Company respectfully requests that evaluation costs be included at the DSM portfolio level
rather than at the individual program leveL.
20 Page 24, "Assessing the Cost-Effectiveness of Low-Income Weatherization in Idaho", prepared by Roger Colton.
14
DATED this 11 th day of November 2011.
15
~c~/~
Mark C. Moench
Daniel E. Solander
Rocky Mountain Power
201 S. Main St., Suite 2300
Salt Lake City, UT 84111
Telephone: [phone]
Fax: [phone]
Attorneys for Rocky Mountain Power