20080929_2367.pdf

DECISION MEMORANDUM
TO:CO MMISSI 0 NER RED FORD
CO MMISSI 0 NER SMITH
COMMISSIONER KEMPTON
COMMISSION SECRETARY
COMMISSION STAFF
LEGAL
FROM:CAROLEE HALL
DATE:SEPTEMBER 22, 2008
RE:COMMISSION APPROVAL FOR STAFF TO JOIN THE REGIONAL
OVERSIGHT COMMITTEE (ROC) MULTI-STATE REVIEW PROCESS
FOR QWEST'S PERFORMANCE ASSURANCE PLAN (QP AP) AND ITS
PERFORMANCE INDICA TOR DEFINITIONS (PIDS).
BACKGROUND AND DISCUSSION
The Qwest's Performance Assurance Plan (QP AP or PAP) was developed in a regional
workshop process during 2001-02 in which Qwest, numerous competitive local exchange
carriers (CLECs), and nine state commissions, including Idaho, participated. The Idaho QPAP
was allowed to go into effect pursuant to 47 USC g252(f)(3)(B), and it became an essential
component of Qwest's successful application to the Federal Communications Commission for
authority to provide interLA T A toll services in Idaho pursuant to Section 271 of the federal
Telecommunications Act. The Performance Indicator Definitions (PIDs) comprise Exhibit B of
Qwest's Statement of Generally Available Terms and Conditions (SGA T). The QP AP is Exhibit
K of the SGAT. Qwest recently filed Case No. QWE-08-04 requesting Commission approval
to eliminate its SGA T including the QP AP and rIDs.
The QP AP's purpose is to provide meaningful and significant incentives to ensure Qwest
provides service to its competitors at parity with the service it provides to itself and to ensure
Qwest's local service market continues to remain open to competing carriers. The QPAP
requires Qwest to comply with agreed-upon performance standards (PIDs) related to the
wholesale service Qwest provides to CLECs. Like the QPAP, the rIDs were collaboratively
developed in a multi-state process. Under the terms of the QPAP, Qwest's failure to comply
DECISION MEMORANDUM SEPTEMBER 22, 2008
with the rIDs results in automatic payments from Qwest to affected CLECs (Tier 1 payments)
and, in certain instances, to the state special QP AP account (Tier 2 payments).
Audits ofthe PAP were to be conducted in two-year cycles under the auspices of the
participating Commissions with a detailed audit plan developed by an independent auditor. The
audit plan was designed to coordinate with other state Commissions to avoid duplication, and not
impede Qwest's ability to comply with the other provisions ofthe PAP 1 . For the audit
participating Commissions were to select an independent auditor with input from Qwest and the
CLECs. Liberty Consulting Group was the independent auditor selected and on November 30
2007, a very extensive audit report using 2005 data was submitted to the Commission. (Please
see Attachment 1 for the Executive Summary of the audit report.) As required by the QPAP
Idaho s portion of the cost of the audit was paid for using Tier 2 payments from the Idaho
Special Funds account.
Because the last audit report used 2005 data, Staff believes it is time for a review of the
current measurements. The Regional Oversight Committee (ROC) staff members have been
meeting with Liberty Consulting to prepare an expedited review process. The ROC believes that
because Liberty Consulting performed the previous audit this review can be completed
economically and expeditiously. In accordance with the QPAP Section 16.1 Qwest, CLECs, or
the Idaho Public Utilities Commission may initiate a review ofthe performance measurements to
determine whether measurements should be added, deleted, or modified. Section 16.3 of the
QP AP provides that "the Commission and Qwest shall review the appropriateness of the PAP
and whether its continuation is necessary.3 In light ofthese two sections of Qwest's
QP AP and
Qwest's recent filing to withdraw its SGAT that would eliminate the QPAPIPIDs all together , it
is imperative that a review and assessment of the measurements be performed to determine the
appropriateness of the measures on a going forward basis.
1 Qwest's Performance Assurance Plan, Exhibit K 15.2 Qwest makes Tier 2 payments to the state commission or a special state fund based on the quality of service
provided to CLECs in aggregate. Tier 1 payments are made directly to the CLEC based on the quality of service
provided to that individual CLEC. To calculate the Tier 1 and Tier 2 payments, Qwest uses the QWEST
PERFORMANCE ASSURANCE REPORTING SYSTEM (QPAPS).3 Idaho PAP, Section 16.4 QWE- T -08-04 Petition of Qwest Corporation Requesting Authorization to Withdraw its Statement of Generally
A vailable Terms and Condition.
DECISION MEMORANDUM SEPTEMBER 22, 2008
On July 31 , 2008 the ROC Chair, Commissioner Phil Jones (W A), sent a letter and
scoping document to Qwest informing the Company that the ROC will be conducting a multi-
state collaborative review of the QPAP and encouraged the Company s cooperation and
participation in the process. The memo requested a response from Qwest by August 15, 2008
which the Company complied with.
Qwest responded and declined to participate in the review, citing a number of concerns
the Company has regarding a ROC review process, its efficiency and the relevance of continuing
the QPAP. Attachment 2. Essentially, Qwest stated many reasons why the multi-state process
will not benefit the Company. Contrary to Qwest's arguments , the FCC stated in its Order
granting Qwest's 271 approval that
, "
... the Commission has always held that an applicant's
performance towards competing carriers in an actual commercial environment is the best
evidence of nondiscriminatory access to OSS and other network elements.s Moreover, in the
same FCC Order granting 271 , Commissioner Copps stated that, "I commend the Regional
Oversight Committee (ROC) for its unique collaborative process to share the limited resources of
the State Commissions in the region. This extensive undertaking should serve as a model for
future Federal-State cooperative efforts as we seek to ensure that Qwest complies with its
statutory obligations . Commissioner Copps continued on to state that
, "
I repeat today what I
have advocated previously - the need for sustained and systematic follow-up to our approval of
Section 271 applications in all states where they have been granted.
Qwest also objected to the multi-state process because it felt that the collaborative review
would not be cost effective. Staff disagrees with the Company and is confident that there are
adequate Tier 2 funds available to pay for the expenses of a review as set forth in the QP AP.
Moreover, a collaborative process will help minimize the need for Staff resources and expedite
the discovery process for Qwest. This process will also provide an Idaho specific report from
Liberty that will assist the Commission in determining which parts of the Q P AP IPIDs are still
relevant and which ones may be modified or eliminated.
Liberty prepared a proposed work plan estimating the total cost of the review to be
$250 450 and Idaho s portion to be $6 073.00. Currently the Tier 2 fund account balance is
approximately $21 000. Liberty anticipates that it will take approximately 29 weeks to complete
5 FCC02-332 Para. 13 , K- 76 FCC 02-332 Statement of Commissioner Michael 1. Copps
DECISION MEMORANDUM SEPTEMBER 22, 2008
the review and prepare state specific reports. This time line, however, depends upon Qwest'
cooperation and participation. Liberty's proposed work plan is attached for the Commission
review as Attachment 3.
RECOMMENDATIONS
Because the previous audit is based upon 2005 data and Qwest has recently petitioned the
Commission to withdraw its SGAT, Staff believes it is time for a review of Qwest's QPAP and
rIDs to determine their relevance. The multi-state review process will minimize the need for
Staff resources and provide the Commission with a detailed State specific report, which will be
beneficial for the Commission s determination in the pending Qwest Case No. QWE-08-04.
The expense for the multi-state review will be adequately provided for using the state special
QP AP (Tier 2) payments already paid by Qwest as a result of PID violations.
Staff recommends the following for the Commission s approval:
1. Open a docket to review Qwest's QPAP/PIDs.
2. Approve Staffs participation in the ROC multi-state process.
3. Hire Liberty Consulting to perform the review and recommend that Qwest
cooperatively participate in all data requests Liberty deems necessary and appropriate
to expeditiously complete its review, as outlined in Attachment 3.
4. Approve the disbursement from Qwest's Tier 2 Idaho Special Funds for payments to
Liberty for its review and final Idaho report.
COMMISSION DECISION
Does the Commission Agree with Staff s recommendations?
u/chall/decision memo/staff memo for Roc collaborative PAP PID review
DECISION MEMORANDUM SEPTEMBER 22, 2008
Final Report for the Audit of
Qwest's Performance Assurance Plans
Executive Summary
The Qwest Regional Oversight Committee (ROC) chose The Liberty Consulting Group (Liberty)
to conduct an audit of Qwest wholesale performance remedy payments and the results of certain
Qwest wholesale performance measures for the year 2005 (2005 ROC Audit). All 14 public
utility commissions agreed to participate in the 2005 ROC Audit, which was intended as a
follow-up to the audits of 2004 Qwest Performance Assurance Plan (QP AP 1 ) results. There were
two 2004 QP AP audits, one (2004 ROC Audit) conducted by Liberty for all Commissions except
the Colorado Public Utilities Commission (PUC) and the other (2004 CP AP Audit) conducted by
NorthStarNantage for the Colorado PUC.
The scope of the 2005 ROC Audit included
An audit of QP AP payments for all 14 states
A full audit of five measures:
OP-
MR-
BI-
BI-
NI-
A partial audit of specific items associated with 29 other measures, related to
The status ofQwest's response to findings from the 2004 ROC Audit
Requirements of the Colorado PUC for the 2005 ROC Audit based on
recommendations from the 2004 CP AP Audit
Additional changes that Qwest introduced in its wholesale measures
QP APs, and systems between the time of the 2004 ROC and CP AP Audits
and the end of 2005
Specific audit requirements of the Colorado PAP.
An audit of certain other items not specific to individual measures.
The audit commenced in late April 2007 with initial requests for data from Qwest and a meeting
and initial interviews with Qwest and representatives from the ROC Steering Committee held in
Denver on May 7 and 8 , 2007. Based on this initial information, Liberty developed a work plan
for the audit. Liberty revised this work plan based on comments from Qwest and the ROC
Steering Committee. The ROC Executive Committee approved the final version of the work plan
on August 21 , 2007. Data gathering, interviews, and analysis continued throughout the summer
of 2007. Representatives of the ROC Steering Committee held regular meetings by
teleconference with Qwest and Liberty during the audit (generally weekly at the beginning of the
audit and bi-weekly toward the end) to monitor and review the conduct of the audit. Liberty
1 Qwest calls its performance assurance plans for Colorado and Minnesota "CPAP" and "MPAP " respectively. It
calls its plan for the other states the "QPAP," For convenience, when referring to all the performance assurance
plans, Liberty will use the tem1 "QPAP.
- "",_""'."""'--_.__..."_..",,-..-""-, .""""'" """---
....,-,--"u...,""- ,
-. -."..-
November 30, 2007
~,~
The Liberty ColISulting Group
Page 1
Attachment 1
Decision Memorandum
09/22/08 Page 1 of 4
Final Report for the Audit of
Qwest's Performance Assurance Plans
closed field work and issued a draft final report for comments by the ROC and Qwest on October
, 2007. Qwest and representatives from the ROC Steering Committee provided comments to
Liberty; Liberty updated the draft to address the comments and issued this fmal report on
November 30 2007.
Liberty obtained information from Qwest through 425 data requests and ten separate interviews
with Qwest personnel. Liberty also analyzed performance measurement and QP AP payment data
provided by Qwest. As part of this audit, Liberty reviewed Qwest's business processes and
systems, and reviewed Qwest's public and internal documentation. Liberty examined the
integrity of the data used in the calculations of the five measures subject to the full audit and
replicated reported results for these and many of the 29 other measures subject to a partial audit.
For the 2005 QPAP payments , Liberty reviewed the QPAP process, replicated the calculated
payments, reviewed the payment process, and verified payments made.
During the audit, Liberty notified Qwest and the ROC Steering Committee of possible fmdings.
Liberty issued 15 such preliminary findings during the course of the audit and provided an
opportunity for Qwest to respond to them. Based on Qwest's response , Liberty withdrew one of
the preliminary findings. Liberty classified the findings according to the nature and severity of
the issues involved and used four classification levels defined in Section II.
Overall, Liberty found that Qwest produced accurate performance results and penalty payments
for the items covered by this audit during 2005. Most of the 14 findings that Liberty identified
during the audit were relatively minor. For those that are likely to cause changes in reports or
payments, the size and scope of the changes are likely to be relatively small. Qwest agreed with
most of the findings and has ei~en action or plans to take action to address them.Liberty
discusses the findings in detail in Section XII. Not all fmdings apply to every state participating
in the audit. Appendix A lists by state those findings that apply to the state and the status of
Qwest's response to the findings.
Liberty identified five findings as Classification 1 , findings for which Liberty believes a
correction could cause a change in Qwest's reported results or QPAP payments or for which
Qwest's practices or methods are clearly inconsistent with the Performance Indicator Definitions
(PIDs) or QPAPs. These findings are:
Qwest did not calculate the interval correctly for MR- to identify all
lines/circuits that had a repeat trouble report within 30 days. (Finding 8)
Qwest's implementation of the permutation test Z-score calculations for
proportion measures that require a permutation test is not the method specified in
the QPAP. (Finding 10)
Qwest recorded the incorrect Tier 2 payment level for the Enhanced Extended
Link- Digital Signaling Level l (EEL-DSI) product for MR-7 and MR-8 in
Arizona. (Finding 11)
The data Qwest used to calculate BI-l results contained duplicate records.
(Finding 13)
"., . . ...-. ...."""~--.
M'_
"'""-",--"-
P""
"""'-'""",,"---,,-
Novemher 30, 2007
~,~
The Liberty Consulting Group
Page
Attachment 1
Decision Memorandum
09/22/08 Page 2 of 4
Final Report for the Audit of
Qwest's Performance Assurance Plans
The BI-3A results included revenue associated with products and services that
Qwest should not include in its measure results. (Finding 14)
Qwest agreed with all five of these findings and has taken steps to address them. Some of the
findings affected QP AP payments, sometimes resulting in overpayments and sometimes
underpayments, but the overall effect was relatively small. The largest net underpayment was a
net $5 000 underpayment to Arizona associated with Finding 11.
In addition, Liberty identified five findings as Classification 2, findings for which Liberty
believes a correction may not change Qwest s reported results or QPAP payments, the magnitude
of the change is unknown, or Qwest's practices or methods may be in error or inconsistent with
the PID or QPAPs. Liberty identified three findings as Classification 3 , findings for which
Liberty found a gap or potential flaw in Qwest's methods, procedures, or documentation for
which a change could lead to an improvement in the reliability of reported results or QP
payment. Liberty identified one finding as Classification 4, findings for which there is not a clear
inconsistency with Qwest's interpretation of the PID or QPAP, but for which clarification is
necessary; or for which Qwest has adopted conventions that are not documented in the PIDor
QP APs or has interpreted these documents in ways that Liberty agrees are consistent with the
wording but for which other reasonable interpretations are possible. Qwest has taken or has
agreed to take action to address all but one of the Classification 2, 3 , and 4 findings.
In addition to the findings , Liberty makes one additional recommendation for Qwest and the
Commissions' consideration in Section XlI. Liberty believes that Qwest's approach is reasonable
in this case but believes that further action by Qwest would be helpful. This recommendation is:
Regarding PO-20 reporting, Qwest still improperly overrides error codes for
manual service orders on occasion; however, Liberty found Qwest's performance
much improved since the 2004 ROC Audit. Liberty believes that Qwest should
continue to self-monitor and evaluate the error code override process and continue
to provide feedback to personnel performing overrides. The override activity will
continue to be prone to human error and each improper override artificially
improves Qwest's PO-20 results.
In its reexamination of the 2004 ROC Audit findings, Liberty determined that Qwest
successfully resolved the 17 of the 26 findings that it had addressed by the end of2005.
In its investigations related to the Colorado CPUC requirements based on the 2004 CP AP Audit
recommendations, Liberty uncovered no remaining problems and considers the issues raised in
these recommendations to be resolved.
The remainder of this Final Report is organized as follows. Section II provides background
information and a description of the audit approach. Section III describes the systems that Qwest
2 The scope of the 2005 ROC Audit was confined to changes made through the end of 2005. Therefore, this audit did
not address the current status of the remaining nine findings, The table in Section XII.C below shows the status of
these findings as of the end of the 2004 ROC Audit.
" ",, ""
", H_'"
" ,.."
'HH"
"""
'--_m
""_"'_"'. ","",
November 30, 2007
~,~
The Liberty Consulting Group
Page
Attachment 1
Decision Memorandum
09/22/08 Page 3 of 4
Final Report for the Audit of
Qwest's Performance Assurance Plans
uses for performance reporting and QP AP payment calculation. Section IV provides an overview
of Liberty's analyses. Sections V through IX provide details of Liberty's analysis of specific
measures. Section X provides details of Liberty's analysis of the 2004 QP AP payments. Section
XI provides a description of Liberty's analysis of some changes in systems and processes not
specific to a single measure or group of measures. Section Xli provides details about Liberty'
audit findings and recommendations. Finally, the report contains two appendices. Appendix A
summarizes by state the applicability and status of each of Liberty's fmdings. Appendix B
provides a glossary of acronyms used in the report.
Liberty appreciated the graciousness and cooperation of Qwest during this audit and found the
Qwest personnel to be very knowledgeable and responsive to Liberty's requests. Liberty also
appreciated the interest and active involvement of the ROC Steering Committee throughout the
audit process.
-", ".."~._-"."....""""-"".,, """'-"--.'-""_"_~""_"'-"_"_'_'-_'"-,"_..-,...,"".~",....".._"",,_.. .",,-......-,-'-... -." ,."., .",",-
November 30. 2007
~,~
The Liberty Consulting Group
Page
Attachment 1
Decision Memorandum
09/22/08 Page 4 of 4
Qwe st.
Spirit of Service
Qwest
1801 Ca1~omia Street, 52'" floor
Denver. Colorado 80202
Phone [phone]
Facsimile 303 244-0958
PCS 720-203-4200
R. Steven Davis
Senior Vice President
Public Polley
August 15 2008
Commissioner Philip B. Jones
Washington Utilities and Transportation Commission
Regional Oversight Committee Chair
1300 S. Evergreen Park Dr., S.
O. Box 47250
Olympia, Washington 98504-7250
Re:Letter of July 31 2008 regarding the Qwest Performance Assurance Plan (QPAP) and Request
for Response by August 15 2008
Dear Commissioner Jones:
Thank you for your letter sent on behalf of the Regional Oversight Committee (ROC) Executive
Committee requesting Qwest's participation in a QP AP Analysis Collaborative. Qwest respectfully
declines to participate in this process, as currently structured.
Qwest participated in a prior "collaborative effort" upon establishment of the QPAPs, That effort was not
successful in reducing time, resources or in reaching consensus on critical issues. Because some
interested parties did not participate in that process and because neither commissions nor their staffs were
in any way bound by the findings and conclusions reached, those findings and conclusions were of little
or no value in the subsequent legal proceedings brought in all 14 participating states,
We believe that the currently proposed collaborative process is destined to follow the same path. We
have major concerns about the process, the scope of the collaborative, and what can realistically be
achieved by use of a collaborative.
The purpose of the QP AP sunset review is to examine "whether the QP AP's continuation is necessary?"
This type of sunset review is separate and distinct from six-month or annual reviews and from audits that
are also set forth in the QPAP. This is not a review of measurement or payment definitions or accuracy,
as is done in six-month reviews or audits. Rather, it is a consideration from a policy and legal perspective
as to what extent, if at all, Qwest is or should be required to continue to volunteer the QP AP as part of
interconnection agreements. Moreover, this issue must be examined in light of the FCC's determination
that Qwest is no longer required to manage its business through the use of a separate 272 affiliate,
The QPAPs do not call for a multi-state collaborative review to determine their continued usefulness. To
the contrary, the QP APs in effect in eight Qwest states explicitly call for this review to be undertaken by
the Commission and Qwest," Therefore, Qwest does not see benefits in its participation in a
collaborative review that is not likely to add value to the fundamental question identified above, and
which is not likely to reduce the time and cost of necessary state-by-state proceedings on this matter.
Attachment 2
Decision Memorandum
09/22/08 Page 1 of2
Commissioner Philip B. Jones
Washington Utilities and Transportation Commission
Regional Oversight Committee Chair
August 15 2008
Page 2
As noted above, past experience has shown that a collaborative review is neither cost effective nor
efficient. Specifically, toward the end of the 271 OSS Test, the parties (including commission staffs)
attempted to collaborate in developing the QP AP (in an effort called the Post-entry Enforcement Plan, or
PEPP"). That process was unsuccessfuL Later, a similar review was attempted for "long-term pm
administration." After many months without meaningful results, Qwest opted for a business-to-business
approach to PID administration. This approach yielded a successful stipulation , containing agreement on
numerous changes to PIDs and PAP payments, all within the existing foundational structure of the QPAP,
We respectfully encourage the ROC Executive Committee to consider the points outlined herein and
support completion of the required sunset reviews of the QP AP in each state. Thank you for your
consideration.
Sincerely,
Steve Davis
Senior Vice President
Public Policy
Cc:Regional Oversight Committee Executive Committee
Commissioner Ray Baum, Oregon Public Utilities Commission
Chairman Ted Boyer, Utah Public Service Commission
Commissioner Anne Boyle, Nebraska Public Service Commission
Chairman Greg Jergeson, Montana Public Service Commission
Attachment 2
Decision Memorandum
09/22/08 Page 2 of2
Proposed Work Plan ROC Collaborative Review of the QPAPs
ROC Collaborative Review of the
Qwest Performance Assurance Plans (QP APs)
Proposed Work Plan
Introduction
The Qwest Regional Operating Committee (ROC) has requested the Liberty Consulting
Group (Liberty) to conduct a review of the Qwest Performance Assurance Plan (QPAp
for the regulatory commissions (Commissions) of 12 of the 14 ROC states. These 12
Commissions include the Arizona Corporation Commission, the Colorado Public Utilities
Commission, the Idaho Public Utilities Commission, the Iowa Utilities Board, the
Montana Public Service Commission, the Nebraska Public Service Commission, the New
Mexico Public Regulation Commission, the North Dakota Public Service Commission
the South Dakota Public Utilities Commission, the Utah Public Service Commission, the
Washington Utilities and Transportation Commission, and the Wyoming Public Service
ommlsslOn.
The QP AP is a mechanism through which Qwest makes payments to the states and/or to
competitive local exchange providers (CLECs) if its performance in providing wholesale
services to the CLECs fails to meet the defined standards of certain performance
measures that are documented in the Qwest Performance Indicator Descriptions (PIDs).
Qwest has filed a QPAP in each of the 14 ROC states. The QPAPs include provisions for
their review after certain events have occurred to determine whether they should remain
in operation and, if so, whether they should be modified. The triggering event for this
review varies from state to state, and the various triggers include Qwest's filing for
section 272 relief and a specific point in time (five and one-half years after the QP AP'
commencement or six months prior to the QPAP's proposed end). Because these triggers
have occurred or are about to occur, the Commissions have elected to begin the QP
reviews and have asked Liberty to conduct an analysis of the QP APs to facilitate the
reviews. The Commission Staffs have defined the scope ofthis review to include:
A detailed review and analysis of both the performance plan and rIDs, which
would include draft recommendations concerning a) the current effectiveness
value, and usefulness of the performance plan and PID measures in relation to
their intended purpose and function; b) whether some of all of the performance
plan or PID measures may no longer be necessary; and c) possible modifications
to the performance plan and PID measures. The review, analysis and draft
recommendations should be provided in a baseline document, and the baseline
document may be used for collaborative discussions between the various
Commission Staffs, Qwest and the CLECs and/or for use by individual
Commissions in their separate state six-month, six-year, or other appropriate
dockets.
1 This term is taken in this document to include the Colorado Performance Assurance Plan (CPAP),2 The Oregon Public Utility Commission has elected to participate in the review as an observer and the
Minnesota Public Utilities Commission plans to conduct its own analysis,
" - -.. .."'" .
August 2008
~,~
The Liberty Consulting Group
Page
Attachment 3
Decision Memorandum
09/22/08 Page 1 of 7
Proposed Work Plan ROC Collaborative Review of the QPAPs
The review will include the participation of and consultation with the QP
stakeholders: Qwest, Competitive Local Exchange Carriers with business in the
relevant fourteen-state region, and the appropriate participating state public
commission regulatory bodies.
Each state would receive a copy of the analysis and report and then use the data
and findings in whatever capacity it sees fit.
The Commission Staffs also recommended that the cost be apportioned to the states in the
same manner as the Qwest QP AP audits.
The following provides Liberty s proposed work plan to address the scope of this review
together with an estimate of the cost of the work. Any work associated with collaborative
discussions that might take place in any of the states after Liberty produces draft
recommendations is not included in this proposal. Liberty and the ROC will assess what
work may remain to be performed associated with such collaborative discussions after the
draft recommendations are released.
Proposed Work Plan
Liberty believes that in order to best determine the effectiveness, value, and usefulness of
the QP APs, it is important not only to explore the trends in the performance results
during the period in which the QP APs have been operating and the trends in the industry
during the same period but also to reexamine the structure of the QP APs and PID
measures in light of these trends to determine whether they have met the objectives they
were originally intended to meet. In addition, Liberty believes it is important to review
recommendations and experiences from the stakeholders most affected by the QP APs:
the Commissions, the CLECs, and Qwest. Thus, Liberty proposes that the analysis
consist of five separate but related investigations:
1. Analysis of QP AP payments and PID results
2. Analysis of the structural components ofthe QPAPs
3. Analysis of the structure of the PID measures
4. Analysis of recommendations and experiences of stakeholders
5. Analysis of the competitive environment.
Liberty plans to perform these five streams of analysis in parallel, using insights gained
from anyone analysis stream to inform the other investigations.
Analysis 1 - Historical QP AP Payments and PID Results
One method of assessing how well the QP APs and PID measures are working is to
examine trends in the payments, transactions volumes, and PID measure results over the
life of the QP APs. The purpose of this examination is to identify measures that either
might be consistently generating payments or consistently meeting the standards. These
would be noted for further investigation. The analysis would also examine PID measures
that are reported but are not currently part of the QP APs to determine to see whether
there is still any value in Qwest's reporting them and if they should be considered for
" ""
August 11, 2008
~,~
The Liberty Consulting Group
Page
Attachment 3
Decision Memorandum
09/22/08 Page 2 of 7
Proposed Work Plan ROC Collaborative Review of the QPAPs
future inclusion in the QP APs. In addition, Liberty will examine trends in transaction
volumes to determine whether some PID measures or measure reporting dimensions
contain so few transactions that they may no longer have value.
This analysis will include the following steps:
Obtain QP AP reports for the 12 participating states showing payments since the
inception of the QPAP (or for a minimum of five years) and the triggers for the
payments
Obtain reports of all PID results for the last five years in the 12 participating
states.
Determine whether there are PID measures showing consistently low activity
volumes and low or minimal QP AP payments.
Determine whether there are PID measures that are consistently out of compliance
with the benchmark and/or leading to significant QP AP payments.
Determine whether there are PID measures whose associated volumes or
payments are increasing (even if small).
Determine whether there are PID measure product disaggregations that are
experiencing low activity volumes.
Liberty estimates that this work will require 16 weeks: six weeks to collect the data and
10 weeks for analysis.
Analysis 2 - QPAP Structure
The purpose of this analysis is to determine whether the current structure of the QP AP is
meeting its original objectives. The analysis performed of the historical QP AP payments
will be used to help direct this investigation. The analysis will include the following
steps:
Obtain the latest QP AP (Exhibit K) for each of the 12 states participating in the
reVIew
Review the structural components of the QP APs (e., statistical methods
payment levels, payment triggering mechanisms)
Identify any components that do not appear to be meeting their apparent
objectives or are no longer relevant.
Liberty estimates that this work will require eight weeks: two weeks to obtain the
documentation and six weeks for analysis.
Analysis 3 - PID Measure Structure
The purpose of this analysis is to determine whether the current structure of the
performance measures is meeting its original objectives. The analysis of the historical
PID measure results will be used to help direct this investigation. The analysis will
include the following steps:
Obtain the latest version ofthe 14-state PID documentation
- "~,- -
August 2008 ~I~
The Liberty Consulting Group
Page
Attachment 3
Decision Memorandum
09/22/08 Page 3 of?
Proposed Work Plan ROC Collaborative Review of the QPAPs
Review the structure of the PID measures (e., formula, exclusions, reporting
disaggregations)
Identify any PID measures or components of the measures that either do not
appear to be meeting their apparent objectives or are no longer relevant.
Liberty estimates that this work will require 10 weeks: two weeks to obtain the
documentation and eight weeks for analysis.
Analysis 4 - Stakeholder Input
In addition to examining the historical record and the structures of the QP APs and PID
measures, Liberty will seek input from the principal stakeholders of the QP APs: the
Commissions, the CLECs, and Qwest. Liberty will ask the Staffs from each of the
participating Commissions to provide their own experiences, concerns
recommendations, and objectives for the QP APs. In addition, Liberty will ask them to
provide lists of CLECs to contact regarding their experiences. Then, Liberty will contact
the CLECs and Qwest for their input. This analysis will include the following steps:
Liberty will ask the Commission Staffs for the 12 participating states for their
input on
What components of the QPAPs (including the PID measures involved) are
working well and are believed to be necessary to preserve going forward?
What components of the QPAPs (including the PID measures involved) are
not working well and should be changed going forward? If any, how should
they be changed?
What components of the QPAPs (including the PID measures involved) are
unnecessary and should be dropped going forward?
Other comments or input.
Liberty will ask Qwest the same questions as those asked of the Staffs
Liberty will ask the Staffs to provide lists of CLECs to contact
Liberty will ask the CLECs the same questions as those asked of the Staffs
As necessary, Liberty will seek clarification of the responses.
Liberty estimates that this work will require 10 weeks: six weeks to obtain the input from
the parties and four weeks for follow-up questions and analysis.
Analysis 5 - Industry Trends
The purpose of this analysis is to determine trends in the competitive local
telecommunications industry in the 12 states that might affect the continued applicability
of the QPAPs. In particular
Liberty will try to obtain information on the number of active CLECs and other
competitors in each of the 12 participating states
If needed as a supplement to the PID reports, Liberty will ask Qwest (through the
12 participating states) to provide the volume of transaction activity for each of
the CLECs if there is missing information regarding this in the PID reports
August 2008
~,~
The Liberty Consulting Group
Page
Attachment 3
Decision Memorandum
09/22/08 Page 4 of 7
Proposed Work Plan ROC Collaborative Review of the QPAPs
To the extent that the information is available, Liberty will seek information on
the market shares the carriers have in each of the 12 participating states
To the extent that the information is available, Liberty will analyze the financial
viability of the competitors in each of the 12 participating states
Liberty will evaluate the nature of the market, investigating whether there are key
submarkets for which the nature of competition is different from the general
market and how this might affect the requirements for the QP APs.
Liberty will use this information to determine to what extent the QP APs still address the
needs of the current telecommunications marketplace. This analysis would explore
whether either some portions of or the entirety of the QP APs should be eliminated
because they no longer are needed to facilitate competition or they should be modified to
addressed the changed nature of the competition.
Liberty estimates that this work will require eight weeks: four weeks to gather and four
weeks to analyze the data.
Draw Conclusions and Develop Recommendations
Based on the five streams of analysis described above, Liberty will:
Evaluate the continued need for QP APs. Are they necessary or helpful in
maintaining a competitive market?
Identify PID measures that should be eliminated if the QPAPs are to be continued.
Identify modification in PID measures that should be introduced if the QP APs are
to be continued.
Identify QPAP structural components that should be eliminated if the QPAPs are
to be continued.
Identify QP AP structural components that should be modified ifthe QP APs are to
be continued.
Identify PID measure structural components that should be eliminated if the
QP APs are to be continued.
Identify PID measure structural components that should be modified if the QP APs
are to be continued.
If the CLECs mainly compete in certain submarkets or there are a large number of
CLECs that compete in these submarkets, evaluate whether the QP APs should be
reconfigured to address the special nature of those submarkets.
If the competitive market has changed substantially from the initiation of the
QP APs through the addition of new types of service providers , consider possible
additions to the QP APs to address the needs ofthese new competitors.
Liberty estimates that this work will require four weeks after the conclusion of the five
analysis streams.
- ~-
August 2008 ~I~
The Liberty Consulting Group
Page
Attachment 3
Decision Memorandum
09/22/08 Page 5 of7
Proposed Work Plan ROC Collaborative Review of the QP APs
Draft Report on Conclusions and Recommendations
Liberty will draft a report describing its conclusions and recommendations and then
release it to the Commission Staffs for review and comments. To the extent that the
Commission Staffs decide that other stakeholders should review and comment on the
draft report, Liberty will also release the report to those stakeholders. Liberty estimates
that it will take three weeks to draft the report after formulating the conclusions and
recommendations.
Produce Final Report
After receiving comments from the stakeholders, Liberty will modify the draft report as
necessary and produce a final report. Liberty proposes that the stakeholders be given four
weeks to comment on the draft report. Liberty estimates that it will take two more weeks
after receiving these comments to produce a final report.
Estimated Cost and Timeline
The work on this plan will begin as soon as the contract or contracts for the work are
complete and signed, although some of the initial requests for data could begin prior to
the execution of the contracts. Liberty estimates that it will take 29 weeks to complete
this work plan after commencement of the project. This includes
Analysis streams I to 5 (running concurrently): 16 weeks
Conclusion and recommendation formulation: 4 weeks
Produce and distribute draft report: 3 weeks
Stakeholders comment on the draft report: 4 weeks
Produce final report: 2 weeks.
Liberty will provide a seasoned team to work on this project, all of whom have extensive
experience with wholesale performance plans and measures and, in particular, with those
that Qwest uses. This team includes:
John Antonuk - Engagement Director
Charles King - Project Manager
Robert Falcone - Measure and Industry Analysis
Alan Salzberg - Statistician and Performance Plan Analysis
Liberty estimates that the cost of this work will be as follows:
Consultant Hours
Travel Expense
Administrative Ex ense
Total
$220,450
000
10,000
$250,450
August 2008
~,~
The Liberty Consulting Group
Page
Attachment 3
Decision Memorandum
09/22/08 Page 6 of 7
Proposed Work Plan ROC Collaborative Review of the QPAPs
Assuming the same allocation method among the states as was used in the 2005 QP AP
audit (proportional to the total QP AP payments during 2005), this cost would be allocated
as follows:
State
Arizona
Colorado
.._,------
Idaho
Iowa._---m'_____
--.
Montana ...m__....... mm
Nebraska
New Mexico
.__
~~EQ1.,~...!I_~~!I!...
South Dak!'~JUtah
.....~g!~.~......
Womin
Jotal
___
--.J
2005 PAP
Payments
424 251
613 467
138,433
329 663
164,480
151 000
223 593
154 579
197 086
474 255
595 748
243 103
709 658
Fraction
0.2494
2826
0242
0577
0288
0264
0392
0271
0345
0831
1043
0426
0000
Cost
$62 473.
$70 773,
072,
$14,460.43
214,
623,
807,
780,
645.
$20 802,
$26 132.
$10 663,
$250 450.
August 2008 ~I~
The Liberty Consulting Group
Page
Attachment 3
Decision Memorandum
09/22/08 Page 7 of 7