20090817Liberty Final Report.pdf

Qwc-'t-ó9-o'l
Analysis
of
Qwests Performance Assurance Plans
Final Report
Prepared for:
The Qwest Regional Oversight Committee
By:
CDTING
GROUP
'llr1i
65 Main Street
Quentin, Pennsylvania 17083
[phone] (voice)
[phone] (facsimile)
Admin(fLibertyConsultingGroup.com (e-mail)
June 30, 2009
RECEIVED
2009 AUG r 7 AM ll: 29
IDAHO PUBliC
UTILITIES COMMISSIOÌ'~
i.
II.
II.
iv.
V.
Table of Contents
A.
B.
C.
Executive Summar ..................................................................................... ........... 2
Introduction.............................................................................................................8
Background and Purpose ofthe Review......................................... .........................8
Overview of Qwests Performance Assurance Plans and Performance Measures..9
Overview of Liberty's Analysis.............................................................................15
Analysis. .... .... ..... ....... .... .... .... ........ .... .... .... ........ .... .............. .... .............. ................ 24
Performance Assurance Plan Payment Trends .............. ....................................... .24
Wholesale Volumes ...............................................................................................31
Qwests Performance.............................................................................................39
Historical Analysis of Key Payment Drivers.........................................................46
Industry Trends......................................................................................................4 7
Summary and Conclusions ....................................................................................54
Proposals......................................... ............................... ........ ............................... 57
Stakeholder Input ...................................................................................................57
Mechanisms to Address Low Volume Issues ........................................................62
Proposed Performance Measure Changes Affecting the PAPs..............................65
Proposed Product Changes..... ...... ...... ........................ ...........................................78
Proposed Performance Indicator Definition Changes............................................ 8 1
Other PAP Changes ...............................................................................................83
Summary of Conclusions and Recommendations ................................................ 85
A.
B.
C.
D.
E.
F.
A.
B.
C.
D.
E.
F.
June 30,2009 ~,~
The Liberty Consulting Group
Page i
Final Report Analysis of Qwests
Performance Assurance Plans
I. Executive Summary
The Performance Assurance Plans (pAPs) in effect in the states in which Qwest Corporation
(Qwest) is the incumbent local service provider include provisions for their review and
modification. In addition to a regular six-month review to consider potential modifications to the
performance measurements, standards, and performance measurement classifications, most PAPs
also call for longer-term reviews of the effectiveness of the PAP and whether its continuation is
necessary. The triggering event for these longer-term reviews varies from state to state, and the
various triggers include Qwests fiing to eliminate its 272 affliate and a specific point in time
(five and one-half years after the PAP's commencement or six months prior to the PAP's
proposed end). Because these triggers had occurred or were about to occur, 1 1 of the 14 state
commissions (Commissions) that are members of the Qwest Regional Oversight Committee
(ROC) elected to authorize ajoint analysis of their PAPs to faciltate the review processes. These
1 1 participating Commissions engaged The Libert Consulting Group (Libert) to conduct this
analysis.
The Commission Staff members forming the QPAP/CPAP Collaborative Committee
(Collaborative Committee) defined the scope of this work to include a detailed review and
analysis of the PAPs and the Performance Indicator Definitions (PID) measures, which are used
to assess Qwests performance. The Collaborative Committee specified that the work would
result in draft recommendations concerning:
. The current effectiveness, value, and usefulness of the PAPs and PID measures in
relation to their intended purpose and function
. Whether some or all of the PAP or PID measures may no longer be necessary
. Possible modifications to the PAP and PID measures.
The Collaborative Committee intended that the review, analysis, and draft recommendations be
provided in a baseline document to be used for collaborative discussions between the various
Commission Staffs, Qwest, and the Competitive Local Exchange Carriers (CLECs), and by
individual Commissions in appropriate state proceedings. However, each state Commission
would use the data and findings in whatever manner it deems appropriate. The present report is
meant to provide the baseline documentation of Liberty's review, analysis, and draft
recommendations contemplated in the Collaborative Committee's scope definition.
The Collaborative Committee intended this investigation to include consultation with Qwest and
the CLECs, in addition to the Commission Staffs. The Commission Staffs and CLECs responded
to Liberty's request for input and suggestions, which Liberty used in the analysis and in
formulating the recommendations. Qwest elected not to actively participate in the review and
declined to provide its positions on or any proposals for changing the PAPs. However, Qwest
agreed to provide Liberty with extensive historical data on PAP payments and PID measure
results, which were invaluable in supporting the analysis.
June 30, 2009 ~~
The Liberty Consulting Group
Page 2
Final Report Analysis of Qwests
Penormance Assurance Plans
Libert began conducting this analysis in December 2008, focusing on five separate but related
lines of inquiry:
1. Analysis of PAP payments and PID measure results
2. Analysis of the structural components of the PAPs
3. Analysis of the structure of the PID measures
4. Analysis of recommendations and experiences of stakeholders
5. Analysis of industry trends.
In evaluating the continuing effectiveness, value, and usefulness of the PAPs, Libert reviewed:
· The number of active CLECs that have a significant total subscriber base and are
dependent on Qwests wholesale products and services to serve their end users
. The level of Qwest s penalty payments
· The extent of Qwest s performance that is out of compliance with standards
· The burden on Qwest of maintaining the PAPs and whether this burden outweighs
the advantage of protecting competitors.
Libert analyzed trends in PAP payments, PID performance measurement results, transaction
volumes, and lines in service since January 2004. Based on this analysis, Libert determined that
the PAP penalty payments have declined overall in all the participating states since the beginning
of 2004. A significant source of this general i decline has been an improvement in the quality of
Qwests wholesale service performance as measured by the PID measurements. However,
another significant source of the payment decreases has been a decline in the number of active
CLECs. Nevertheless, the volume of CLEC activity remains significant in all the paricipating
states, and Qwest continues to make payments based on inadequate performance for some
functional areas, with the largest number of recent payments coming from sub-standard
performance on Maintenance & Repair transactions.
Liberty found that CLEC order volumes and lines in service have declined markedly. Major
contributors to this decline were the Federal Communication Commission's (FCC's) Triennial
Review Order (TRO) and the Triennial Review Remand Order (TRRO) decisions, which
eliminated a number of unbundled services, including Unbundled Network Element - Platform
(UNE-P). There has also been a significant decline in Resale transactions and a smaller decline
in ONs unaffected by the TRO and TRRO decisions. Despite these declines, the volume of
number porting orders has remained high, indicating the increasing importance of facilties-based
competitors like cable companies. The wireless carriers are also major and growing competitors
of Qwest, but this source of competition is not reflected in the volumes reported in the PAPs and
PID measures, because these carriers rarely, if ever, use the wholesale services monitored in this
way.
i In addition to the factors mentioned here that apply to alI states, special factors contrbuted to the declines in some
of the states. For example, there was a significant decrease in Tier 2 payments in Colorado after 2006, which
resulted primarily from Colorado PAP changes introduced after the Colorado three-year review that reduced the
number and types ofPID measures eligible for Tier 2 payments.
June 30, 2009 ~,~
The Liberty Consulting Group
Page 3
Final Report Analysis of Qwests
Performance Assurance Plans
Based on analysis presented in this report, Libert concludes that the PAPs are stil serving a
useful purpose in all the participating states. Although Qwests largest competitors are the
wireless and cable companies, which are less dependent on Qwests wholesale services, there
continues to be a significant group of CLECs that rely heavily on Qwests wholesale services to
conduct their business, and there are limited readily available alternatives to Qwests wholesale
service for these CLECs. These CLECs stil provide significant competition for Qwest,
particularly in such important pars of the market as broadband and business services. As noted,
Libert found that Qwests performance in providing wholesale services continues to improve,
contributing to a decline in PAP payments. Although it is diffcult to verifY from historical data,
the incentive provided by the PAPs has likely contributed to this performance improvement.
Despite the improvement in Qwests performance and reduction in PAP payments, the PAP
incentives continue to be important in helping to ensure that Qwests performance level does not
deteriorate, because Qwests wholesale services remain critical for the CLECs stil relying on
them. Recent experiences in Hawaii and northern New England demonstrate the severe impact
on competitors when an incumbent local company fails to provide adequate wholesale
performance, despite the best intentions and preparations.2 The circumstances of those cases are
very different from what the CLECs face in Qwests operating territory. However, they ilustrate
conditions that can arise in extreme cases without adequate protections. The Qwest PAPs help
ensure that the correct incentives are in place to prevent such conditions from occurring.
Although concluding that the PAPs should continue to be maintained, Libert believes some
changes should be made in the PAPs to simplifY them and make them more targeted to the
continuing needs of the competitive marketplace. Liberty used the results of its analysis as well
as input from stakeholders, including the CLECs, in identifYing potential proposals. In
evaluating potential proposals, Liberty considered:
. Whether changes in the marketplace have made elements of the PAPs obsolete
. Whether paricular types of transactions are no longer relevant
. Whether the volumes of transactions for sub-measures and products are too small
to warrant their continued inclusion in the PAPs
. Whether the PAPs and PID can be simplified
. Whether there are any biases and distortions in the PAPs that need to be corrected
. Whether there are important transactions types that are currently not monitored in
the PAPs and PID
. Whether the effort to secure support for and cost of making the changes
outweighs the advantage of making them.
Liberty offers several recommendations for the participating Commissions as follows. Many of
these recommendations continue a process of evolving the PAPs to tailor them to current needs,
which has occurred since their inception. Most notably, major changes were made in the
See, for example, Liberty's report on the FairPoint Communications, Inc. cutover:
http://www.puc.state.nh.us/T eleco m/Filin gs/F airPoint/ost-Cutover/F airPoint%20 Post-
Cutover%20Status%20 Report%2004-0 i -09. pd f
June 30, 2009 ~,~
The Liberty Consulting Group
Page 4
Final Report Analysis of Qwest's
Performance Assurance Plans
Colorado PAP during 2006 after the three-year review in that state, and in most other states at
various times since 2006 in response to recommendations from a joint stipulation between Qwest
and some CLECs signed in 2007.
The following recommendations apply to all the paricipating state PAPs.
Recommendation 1. The Commissions should introduce a new aggregation mechanism to
minimize low-volume tests in determining payments. Specifcally, transactions for CLECs with
low volumes should be aggregated with those of other CLECs, and, as necessary, aggregated
over up to a three month period, for the purpose of determining non-conformance and
calculating payments.
Recommendation 2. The Commissions should eliminate the following PID measures (in additon
to those included in the 2007 Stipulation recommendations) from consideration for PAP
payments for those states that use them, and place them on the list of measures subject to the
Reinstatement/Removal Process:
. PO-9 Timely Jeopardy Notices
. PO-19 Stand Alone Test Environment (SATE) Accuracy
. PO-20 Manual Service Order Accuracy
· CP-1 Collocation Completion interval
· CP-2 Collocations Completed within Scheduled Intervals
· CP-4 Collocation Feasibilty Study Commitments Met.
Recommendation 3. The Commissions should make the following additional changes to certain
PID measures in the PAPs:
· For OP-5 (New Service Quality), use sub-measure OP-5T instead of sub.
measures OP-5A and OP-5B.
· Replace the current retail analog of "retail Integrated Services Digital Network
Basic Rate Interface (ISDN-BRJ) designed" with some other retail product or
with a benchmark.
Recommendation 4. The Commissions should eliminate the following low-volume products from
the OP and MR measures in the PAPs:
. Unbundled Digital Signaling Level 3 (DS-3) Loops
. Unbundled Dedicated Interoffce Transport (UDIT) - Above DS1
. Unbundled 4-Wire Non-Loaded Loops
. Loops with Conditioning (applies only to OP measures)
· Unbundled ISDN Capable Loops (applies to all states and measures except for
MR measures in Arizona and Colorado)
. Line Sharing (already removed in Colorado).
June 30, 2009 ~~
The Liberty Consulting Group
Page 5
Final Report Analysis of Qwests
Penormance Assurance Plans
Recommendation 5. The Commissions should make the following additional changes to certain
PID measures:
. Limit MR-4 (All Troubles Cleared within 48 Hours) to service-afecting troubles
· Add a diagnostic sub-measure to OP-4 (Installation Interval) to measure
performance on expedited orders
. Add a diagnostic sub-measure to MR-7 (Installation Interval) to measure chronic
troubles
. Add a diagnostic sub-measure to OP-3 (Installation Appointments Met) to
measure the percentage of coordinated appointments met.
Recommendation 6. The Commissions should adopt provisions to assess Qwest for the cost of
PAP administration functions, including independent auditor and audit costs and payment of
other expenses incurred by the participating Commissions in the regional administration of the
PAP, if the Special Funds created by the Tier 2 payments are insuffcient for fund these
functions.
The following recommendation applies to all participating states except Colorado and Uta.
Recommendation 7. The Commissions should adopt changes in the PAPs and P ID to recognize
Qwest's replacement of the Electronic Data Interchange (EDI) interface by the Extensible Mark-
up Language (XML) interface.
The following two recommendations apply only to Colorado.
Recommendation 8. The Colorado Public Utilties Commission should restore the Tier 1 B, Tier
1 C, and Tier 2 mechanisms to the CP AP, subject to the changes required by Liberty's other
recommendations.
Recommendation 9. The Colorado Public Utilties .Commission should make the following
additonal changes to the CP AP:
. Restore the Unbundled Asynchronous Digital Subscriber Line (ADSL)-Capable
Loop product
. Eliminate the UNE-P products.
The following recommendation applies to Montana only.
Recommendation 10. The Montana Public Service Commission should adopt the
recommendations of the 2007 Stipulation.
June 30.2009 ~~
The Liberty Consulting Group
Page 6
Final Report Analysis of Qwests
Performance Assurance Plans
The next chapter of this report (Chapter II) details the background and purpose of Liberty's
review, describes Qwests PAPs and PID measures including a high-level description of recent
changes, and outlnes Liberty's analysis approach. Chapter II describes Liberty's data analysis.
Chapter iv discusses proposals for PAP and PID modifications. Chapter V summarizes Liberty's
conclusions and recommendations.
Attached to the proposal are four appendices. Appendix A summarizes the key features of the
PAPs, indicating those areas where the PAPs differ among the states. Appendix B provides
details of Liberty's data analysis for each of the 11 paricipating states. Appendix C describes the
detailed applicability of Liberty's recommendations for each of the 11 paricipating states.
Appendix 0 provides a glossary of terms used in the report.
June 30, 2009 ~~
The Liberty Consulting Group
Page 7
Final Report Analysis of Qwests
Penormance Assurance Plans
II. Introduction
A. Background and Purpose of the Review
Eleven member state commissions of the Qwest ROC, an organization of the 14 Commissions of
the states in which Qwest provides local exchange service, chose Liberty to conduct a review of
Qwest PAPs3 in effect in the 1 1 participating states. These 1 1 Commissions are the Arizona
Corporation Commission, the Colorado Public Utilties Commission, the Idaho Public Utilties
Commission, the Iowa Utilities Board, the Montana Public Service Commission, the Nebraska
Public Service Commission, the New Mexico Public Regulation Commission, the North Dakota
Public Service Commission, the South Dakota Public Utilities Commission, the Utah Public
Service Commission, and the Wyoming Public Service Commission.4
The PAP is a mechanism through which Qwest makes payments to the states and/or to CLECs if
its performance in providing wholesale services to the CLECs fails to meet the defined standards
of certain performance measures that are documented in the Qwest PID. Qwest has filed a PAP
in each of the 14 ROC states. The PAPs include provisions for their review and modification;5 in
addition to a regular six-month review to consider potential modifications to the performance
measurements, standards, and performance measurement classifications, most PAPs also call for
longer-term reviews. In particular, most PAPs call for reviews several years after the initiation of
the PAP to assess the PAP's effectiveness and whether its continuation is necessary. The
triggering event for these longer-term reviews varies from state to state. In most states, the
trigger is when Qwest fies to eliminate its Section 272 affiiate. A few states specifY a specific
point in time (five and one-half years after the PAP's commencement or six months prior to the
PAP's proposed end).6 Because these triggers had occurred or were about to occur, the 11
paricipating Commissions elected to authorize a joint analysis of their PAPs to faciltate the
review processes and engaged Liberty to conduct the analysis. The Commission Staff members
forming the Collaborative Committee defined the scope of this review to include:
. A detailed review and analysis of both the performance plan and PID measures,
which would include draft recommendations concerning a) the current
effectiveness, value, and usefulness of the performance plan and PID measures in
relation to their intended purpose and function; b) whether some or all of the
performance plan or PID measures may no longer be necessary; and c) possible
modifications to the performance plan and PID measures. The review, analysis
3 In this report, the term "PAP" wil be used to designate all the Qwest Performance Assurance Plans. The term
"CPAP" wil be used to refer to the Colorado Performance Assurance Plan and the term "QPAP" wil be used to
refer to the PAPs in the other ten paricipating states.4 The Oregon Public Utilty Commission and the Washington UtiÍities and Transporttion Commission have elected
to paricipate in the review as an observer. The Minnesota Public Utilties Commission chose not to participate in
the study.
5 These provisions are contained in Section 16.0 of
the QPAPs and Section 18.0 of the CPAP.6 The Arizona PAP provides for the six-month reviews but has no specific provisions for a longer-term review. The
Colorado and New Mexico PAPs call for reviews to begin five and one-hal f years after the inception of the PAPs.
The Idaho, Iowa, Montana, Nebraska, North Dakota, South Dakota, Uta, and Wyoming PAPs call for a review
after Qwest eliminates its Section 272 affiiate.
June 30, 2009 ~~
The Liberty Consulting Group
Page 8
Final Report Analysis of Qwest's
Penormance Assurance Plans
and draft recommendations should be provided in a baseline document, and the
baseline document may be used for collaborative discussions between the various
Commission Staffs, Qwest and the CLECs and/or for use by individual
Commissions in their separate state six-month, six-year, or other appropriate
dockets.
· Paricipation of and consultation with the PAP stakeholders: Qwest, CLECs with
business in the relevant fourteen-state region, and the appropriate participating
state public commission regulatory bodies.
. Provision to each state ofa copy of the analysis and report; each state would then
use the data and findings in whatever capacity it sees fit.
Contrary to what was originally contemplated, Qwest elected not to actively participate in the
review, although the Commission Staffs and CLECs responded to Liberty's request for input.
Nevertheless, Qwest did voluntarily provide Liberty with extensive historical data on PAP
payments and PID measure results and answered questions about the data provided, and this
input was invaluable in supporting the analysis.
This report provides the baseline documentation of Liberty's review, analysis, and draft
recommendations contemplated in the Collaborative Committee's scope definition. Libert
began conducting the analysis in December 2008.
B. Overview of Qwest's Penormance Assurance Plans and
Performance Measures
The Qwest PAPs and PID are incorporated as exhibits in the Statement of Generally Available
Terms and Conditions (SGAT) for Qwests wholesale local exchange services in each state. The
PID is Exhibit B in the SGA T and the PAP is Exhibit K. Each of the PAPs has unique features,
but there are two basic versions, one used by Colorado (CPAP) and Minnesota and the other
(QPAP) used by the remaining 12 ROC states.
Appendix A of this report lists the most common provisions of the PAPs and the differences
from these common provisions applicable to each of the PAPs for the 1 1 states participating in
this review. The PAPs are generally two-tiered, with Tier 1 used for payments to CLECs and
Tier 2 for payments to the states.7 Payments for each tier are based on Qwest's performance on
specific PID sub-measures (or sub-measure/product combinations for the CPAP) applicable to
that tier. For the QPAPs, the Tier 1 and Tier 2 sub-measures are classified as High, Medium, or
7 The current version of the CPAP (Ninth Revision, Sixteenth Amended), which has been in effect since January 2,
2009, has eliminated the Tier 2 payments and all Tier I payments except Tier IA. The Colorado Public Utilties
Commission adopted these changes in Decision No. COS-1345 by allowing the implementation of Section IS.I I of
the CPAP, which provides for such a change after six years but also contemplated the completion of the Colorado
Six-Year Review by that time. As noted below, Libert's analysis provided in this report corresponds to the Six-
Year Review for Colorado. In adopting the CPAP change, the Colorado Commission noted, "By the conclusion
reached in this Order, we make no predetermination as to the status of any CPAP submeasures following our
completion of the Six-Year Review."
June 30,2009 ~,~
The Liberty Consulting Group
Page 9
Final Report Analysis of Qwests
Performance Assurance Plans
Low, depending on their importance, with dollars at risk declining from High to Low sub-
measures. For the CPAP, there is no importnce distinction for the Tier 2 sub-measures; the
CPAP designates sub-measures and product combinations as Tier lA, IB, and LC, which
correspond roughly to the High, Medium, and Low classifications for the QPAPs.
PAP payments are based on tests of the extent of Qwest s conformance with defined standards
for the sub-measures and the number of consecutive months of non-conformance. Payments for
most sub-measurements (Per Occurrence measures) are based on the number of "occurrences,"
which are measures of i) the volume of transactions, and ii) the extent to which Qwest has
missed the standard. Payments for some sub-measurements (Per Measurement measures), which
are generally associated with gateway systems and call center performance, are made on a "per
measurement" basis, with specific payments determined by the level of performance relative to
certin benchmarks independent of the volume oftransactions.
The PID contains the definitions and business rules for the measures and sub-measures that
Qwest reports, including those used in the PAPs. Some of the PID measures and sub-measures
are only diagnostic and are not incorporated in any of the PAPs. There are three basic types of
PID measures: i) means, such as mean time to restore; ii) percentages, such as percent report
troubles met; or ii) ratios or proportions, such as trouble report rate. The PID provides the
descriptions, calculation formulae, product reporting and other disaggregations, and exclusions
for the m~asures and sub-measures, as well as the standards against which the performance is
measured. The standards are either parity with a Qwest retail analogue or benchmarks. The
measures are classified into ten different domains:
. Electronic Gateway Availabilty (GA)
. Pre-order/Order (PO)
· Ordering and Provisioning (OP)
. Maintenance & Repair (MR)
. Biling (BI)
· Database Updates (DB)
. Directory Assistance (DA)
. Operator Services (OS)
. Network Performance (NI and NP)
. Collocation (CP).
For example, MR-6 (Mean Time to Restore) is a Maintenance & Repair measure.
The Qwest PAPs generally went into effect at the time of Qwests Section 271 approval by the
Federal Communications Commission (FCC) in each state, which occurred during 2002 and
2003 depending on the state. The specific provisions of the PAPs have changed since their
inception, with some, like the CPAP, changing more significantly than the rest. Liberty's
analysis covered the period from January 2004 through October 2008 (Study Period). At the end
June 30, 2009 ~~
The Liberty Consulting Group
Page 10
Final Report Analysis of Qwests
Penormance Assurance Plans
of this Study Period, the version of the PID referenced in most state SGATs (Exhibit B) was
Verizon 9.0,8 and PAPs (Exhibit K) in effect in each of the paricipating states were:
. Arizona: SGAT Fourteenth Revision, Fourth Amended Exhibit K, dated June 22,
2007
. Colorado: SGA T Ninth Revision, Fifteenth Amended Exhibit K, dated August 13,
2008
. Idaho: SGAT Third Revised, Sixth Amended Exhibit K, dated June 26, 2007
. Iowa: SGAT Sixth Revision, Fifth Amended Exhibit K, dated June 26, 2007
. Montaa: SGAT Fifth Revision, Fourth Amended Exhibit K, dated November 30,
2004
· Nebraska: SGAT Sixth Revision, Fifth Amended Exhibit K, dated June 26, 2007
. New Mexico: SGA T Eleventh Revision, Fourth Amended Exhibit K, dated
November 24, 2004
· North Dakota: SGAT Exhibit K, dated June 22,2007
· South Dakota: SGA T Exhibit K, dated June 22, 2007
. Utah: SGAT Seventh Revision, Fifth Amended Exhibit K, dated June 26,20079
. Wyoming: SGAT Sixth Revision, Fifth Amended Exhibit K, dated June 26, 2007.
Since October 2008, the Colorado and New Mexico PAPs have been revised again. The latest
CPAP (Ninth Revision, Sixteen Amended) became effective on January 2, 2009 and the latest
New Mexico PAP (Eleventh Revision, Fifth Amended) became effective on May 1,2009.
1. Past PAP Changes
Although most PAPs have had a number of changes since their inception, the most significant
changes have happened at two times in the past:
. For the CPAP, in 2006 after the completion of the three-year review
. For the QPAPs, beginning in 2007 in response to a Qwest-CLEC Stipulation
agreement.
8 As of the date of this report, the Commissions in two of the paricipating states, Colorado and Uta, have adopted
changes to recognize Qwests replacement of its EDI interface by an XML interface. These changes are captured in
an updated PID, version 9.1. Liberty understands that the Washington Utilties and Transportation Commission has
also adopted these changes.9 At the end of the Study Period, this version of the Utah PAP was only applicable to the paries to the 2007
Stipulation. On February 4, 2009, the Utah Commission issued an order to extend the applicability to all CLECs.
That order also adopted changes to reflect the replacement of EDI by XML. These changes are not yet reflected in
the Utah SGAT Seventh Revision, Fifth Amended Exhibit K.
June 30, 2009 ~~
The Liberty Consulting Group
Page 11
Final Report Analysis of Qwests
Performance Assurance Plans
CPAP Three-Year Review
Sections 18.7, 18.1 0, and 18.11 of the CPAP require three and six-year reviews to consider
fundamental changes in its strcture and operations. Section 18.10 specifies that the three-year
review was to begin 30 months (two and one-half years) after the effective date of the CPAP and
to be performed with the assistance of an outside, independent expert. The Colorado Public
Utilities Commission engaged the Barrington-Wellesley Group, Inc. (BWG) as the independent
expert. BWG conducted the review during 2005 and produced a final report on December 7,
2005. During the review, BWG solicited input and proposals from Qwest and the CLEC
community, and worked with the parties to faciltate an agreement. Several of the interested
parties reached agreement on proposed CP AP changes in a stipulation (Three-Year Review
Stipulation),
10 which they presented to the Commission for adoption on February 17, 2006. The
Commission adopted the Three-Year Review Stipulation on March 15, 2006.
The CPAP in Section 18.10 specifically required the Three-Year Review to analyze:
1. Payment amounts, determining whether there was any harm associated with
paricular non-conforming wholesale performance and recommending
adjustments in the payment amounts accordingly.
2. Economic alternatives, evaluating whether there were such available alternatives
to Qwests wholesale service offerings and whether these alternatives provided
competitors with a meaningful opportunity to compete. This analysis was to
consider the rationale for removing measures based on the evidence of Qwests
ability to deliver reliable wholesale performance and/or reduction in Qwests
critical role in the market as a provider of key wholesale inputs.
3. Removal of measure dimensions, determining whether some product
disaggregations or geographic areas no longer needed to be measured and/or
subjected to payments for non-conforming performance.
4. The revision process, evaluating whether these should take place semi-annually,
annually, or otherwise.
BWG drew conclusions and made recommendations in each of these four areas. In particular,
they concluded that:
. The CP AP was not a source of financial harm for any party.
. Qwest performance had improved in many areas, but should service deteriorate,
the CLECs and the competitive environment could be harmed; therefore, penalties
should continue as an incentive to Qwest to maintain and improve performance.
. Qwest appeared to be making payments in Colorado out of proportion to those in
the other Qwest states and analysis provided by Qwest indicated that if the QP AP
used in other states had been in place in Colorado, penalty payments would have
been only 38 percent of the CPAP payments.
10 The parties to the Three-Year Review Stipulation were Qwest; DIECA Communications, Inc. d/b/a Covad
Communications Company; Eschelon Telecom Inc., and MCI Metro Access Transmission Services LLC d//a
Verizon Access Transmission Services.
June 30, 2009 ~~
The Liberty Consulting Group
Page 12
Final Report Analysis of Qwest's
Performance Assurance Plans
· The CLECs continued to rely heavily on Qwests network to reach end-user
customers and few adequate alternatives to Qwest s network exist.
· The volume of certin products was too low for continued tracking and should be
removed from the CPAP, but there was not enough information to conclude that
any geographic disaggregation should be removed.
· The six-month reviews should be changed to annual reviews.
· Several fundamental changes should be made in the CPAP, including both
structural and measure/sub-measure changes.
BWG's report!! recommended a number of specific changes in addition to those just noted
which were subsequently adopted by the Commission. The ones that are most relevant to the
current analysis are the following:
· A Reinstatement/emoval Process was introduced into the CPAP, designating
certain measures to be removed from payment determinations but providing for
automatic reinstatement of the measures based on three consecutive months of
non-conforming performance. The measures subject to this process are: 12
o GA-3 Gateway Availabilty Electronic Bonding-Trouble
Administration (EB-TA)
o GA-4 System Availabilty Exchange Access Control & Tracking
(EXACT)
o GA-7 Timely Outage Resolution Following Softare Releases
o PO-2B Electronic Flow-through 13
o PO-3 Local Service Request (LSR) Rejection Notice Interval
o PO-5D Firm Order Confirmations (FOCs) On Time (Access Service
Requests (ASR) for Local Interconnection Service (LIS) Trunks)
o PO-7 Biling Completion Notification Timeliness
o PO-8 Jeopardy Notice Interval
o PO- 1 6 Timely Release Notifications
o OP-7 Coordinated "Hot Cut" Interval- Unbundled Loop (UN-L)
o OP- 1 7 Timeliness of Disconnects Associated with Local Number
Portabilty (LNP) Orders
o MR- 11 LNP Trouble Reports Cleared within 24 Hours
o 81-4 Billng Completeness
11 Independent Expert Report for the Three-Year Review of the Qwest Corporation Colorado Performance
Assurance Plan. Barrington-Wellesley Group, Inc., December 7, 2005.12 BWG also recommended including PO-I9 - "Stad-Alone Test Environment (SATE) Accuracy" on the list.
However, the Colorado Commission had already determined that PO-19 should be treated as diagnostic and
removed from measures that can generate payments. This change was made effective in the CP AP version dated
May 6, 2005 (Colorado SGAT Ninth Revision, Ninth Amended Exhibit K, dated May 6, 2005).
13 PO-2B is evaluated on a quarerly basis and thus reinstatement is based on two consecutive quarters rather than
three consecutive months. PO-2A had originally been par of the CPAP, but BWG recommended that it be dropped
entirely, because it measures all orders, not just those eligible for flow-through.
June 30, 2009 ~~
The Liberty Consulting Group
Page 13
Final Report Analysis of Qwests
Penormance Assurance Plans
o NI-l Trunk Blocking
o NP- 1 NXX Code Activation
o CP-3 Collocation Feasibility Study Interval
o QX- 1 Timely and Complete Notifications of Product/rocess
Change. 14
. Implementation of the One-Allowable Miss rule for low-volume benchmark or
non-interval parity sub-measures that otherwise would require perfect
performance to meet the standard.
. Elimination ofthe Per Occurrence measurements from Tier 2.
. Elimination of the following product disaggregations with little activity (the
criterion for inclusion on the list was a Colorado volume less than 130 from
February 2003 through June 2005):
o Resale Centrex
o Resale Centrex 21
o Resale Frame Relay
o Unbundled ADSL
o Resale PBX (non-designed and designed)
o Resale ISDN BRI (non-designed and designed)
o Resale ISDN PRI (non-designed and designed)
o Resale DSO (non-designed and designed)
o Resale Digital Subscriber Line (DSL) (designed)
o 91 lÆ91 1 Trunks.
. Remove Line Sharing as a product disaggregation because of the FCC Triennial
Review Order, which eliminates the requirement for Qwest to provide this
product.
2007 Qwest-CLEC Stipulation Changes
Apparently motivated in part by some of the changes to the CP AP resulting from the Three-Year
Review, Qwest invited CLECs to join it in discussins recommendations for changes to the
QP APs. As a result of these discussions, three CLECs! signed a stipulation with Qwest (2007
Stipulation) proposing QPAP changes, which the stipulating parties subsequently fied for
approval in the 14 ROC states. The recommended changes in the QPAPs were similar to many of
those made to the CPAP after the Three-Year Review, but some additional recommendations
applied both to the CPAP and QPAPs. Specifically, the proposed changes included:
14 QX- I is not a PID measurement but is defined specifically for the CP AP.
15 The CLECs who are parties to the 2007 Stipulation are Eschelon Telecom Inc.; DIECA Communications, Inc.
d//a Covad Communications Company; US Link, Inc. d/b/a TDS Metrocom; and McLeodUSA Communications
Services, Inc. (McLeodUSA now does business as PAETEC Business Services.)
June 30,2009 ~~
The Liberty Consulting Group
Page 14
Final Report Analysis of Qwest's
Performance Assurance Plans
. Introduction of the Reinstatement/emoval Process for certain measures. The list
of measures is the same as in the CPAP Three-Year Review Stipulation except for
the removal ofPO-19 and QX-l from the list of excluded measures.
· Elimination of low-volume product disaggregations. The list is the same as for the
CPAP Three-Year Review Stipulation except for the addition of Sub-Loop
Unbundling, UNE-P plain old telephone service (POTS), UN-P Centrex, and
UNE.P Centrex 21 to the list of excluded products, and the removal of Unbundled
ADSL from the list of excluded products.
. Elimination of Resale DSL from PIDs and modification of PID and PAP
references to Qwest DSL, because of the FCC Broadband Order classifying these
services as information services.
. Introduction of the One-Allowable Miss rule.
. Changing the minimum payment provision from flat to tiered payments.
. Changing the provisions for Tier 2 payment candidacy to be triggered by three
consecutive months of non-conformance unless there are two out of three
consecutive months of non-conformance. The payments are triggered by two
consecutive month's missed for measures with Tier 1 counterparts and the current
month's miss for the rest of the measures.
. Other retail analogue and PID changes, many specific to individual states.
. Other specific PAP provisions, applicable to all or subsets of the states.
Libert understands that all or most of the recommendations of the 2007 Stipulation have now
been adopted by the Commissions of all 1 1 paricipating states except Montana. The Utah
amendments were originally applicable only to the paries to the Stipulation, but on Februar 4,
2009, after the Study Period for Libert's analysis, the Utah Commission extended their
applicabilty to all CLECs.!6 The New Mexico amendments became effective on May 1,2009.
c. Overview of Liberty's Analysis
1. Guidance from the PAPs
The appropriate context for the current analysis differs among the eleven participating states
because the requirements of the PAPs vary. The following lists the PAP guidance associated
with the most appropriate context in each of these states.
16 The Public Service Commission of Uta orders in Docket 07-049-31, issued June 30, 2008, approved the changes
for the paries to the 2007 Stipulation only. The Commission's order in Docket 08-049-50, issued February 4, 2009,
extended the applicability to aU CLECs. The February 4 order also approved changes to reflect the replacement of
EDI by XML.
June 30,2009 .~~
The Liberty Consulting Group
Page 15
Final Report Analysis of Qwests
Performance Assurance Plans
Arizona
The Arizona PAP does not specifY the need for any longer-term PAP reviews, but requires
regular six-month reviews. 17 Liberty understands from the Arizona Staff that the current analysis
is meant for use in such a six-month review. Section 16.1 specifies that these six-month reviews
shall review the performance measurements to determine:
. Whether PID measures should be added, deleted, or modified
. Whether the applicable benchmark standards should be modified or replaced by
parity standards
. Whether to move a classification of a measure to High, Medium, or Low or Tier- 1
to Tier-2.
The criteria for review of the measurements, other than for possible reclassification, shall be:
. Whether there exists an omission or failure to capture intended performance
. Whether there is duplication of another measurement.
However, the PAP also notes in Section 16.1 that
... the Commission reserves the right to modif the PAP including, but not limited
to performance measurements, penalty amounts, escalation factors, audit
procedures and reevaluation of confidence levels, at any time as it sees fit and
deems necessary upon Commission Order after notice and hearing.
Furthermore from Section 16.2,
Notwithstanding section 16.1, any party may submit a root cause analysis to the
Commission requesting removal of a PID or sub-measure from the PAP or
requesting exemption of a P ID or sub-measure from the application of the trigger
mechanism for reinstatement or subsequent removal. In the analysis and
recommendations concerning the root cause analysis, the Commission is to
consider, at a minimum, whether the root cause analysis provides evidence of no
harm, the same harm as covered by other P ID measures, non-Qwest related
causes, or other factors which directly relate to the harm or circumstances
specifc to the P ID or sub-measure being analyzed.
Colorado
In Colorado, the timing of Liberty's analysis is most consistent with that of a six-year review.
The CP AP states in Section 18.i 1 :
17 All the participating QPAPs (in Section 16.1) call for six-month reviews with terms similar to Arizona's. The
CPAP originally required six-month reviews, but amendments introduced after the Three-Year Review changed
these to annual reviews (Sections 18.2 - 18.6).
June 30,2009 ~~
The Liberty Consulting Group
Page /6
Final Report Analysis of Qwests
Performance Assurance Plans
Except as provided in this Section, this CPAP wil expire six years from its
effective date. Only Tier 1A submeasures and payments wil continue beyond six
years, and these Tier 1A submeasures and payments shall continue until the
Commission orders otherwise. Five and one-half years after the CPAP's effective
date, a review shall be conducted with the objective of phasing-out the CP AP
entirely. This review shall focus on ensuring that phase-out of the CP AP is indeed
appropriate at that time, and on identifing any sub measures in addition to the
Tier 1A submeasures that should continue as part of the CPAP.
In fact, the six-year point has now passed, and the latest version of the CP AP has implemented
the changes required in Section 18.1 1 (removal of all but the Tier 1 A sub-measures). Although
the circumstances involved in obtaining commitment for a joint ROC analysis prevented
Libert's analysis from meeting the specific timing for the six-year review, Liberty understands
from the Colorado Staff that the analysis wil nevertheless be used as part of that review process.
The CPAP specifies in Section i 8.7 that the following areas are also eligible for change in the
three-year and six-year reviews:
. The statistical methodology (Sections 4.0, 5.0 and 6.0) except for additions to the
variance tables for new Tier lA measures
. The payment caps (Sections 1 1.0 and 18.8)
· The duration of the CPAP (Section 18.1 1)
. The payment regime strcture (Sections 2.0, 7.0, 8.0, 9.0, 10.1, and 10.4) except
for the addition of payment amounts for new Tier 2 measures and of payment
amounts for violations of change management requirements
. The legal operation ofthe CPAP (Sections 15.0 and 16.0)
. The Independent Monitor (Section 17.0) with the exception of assignment of the
Independent Monitor function to an Administrative Law Judge
. Any proposal that does not relate directly to measuring and/or providing
payments for non-discriminatory wholesale performance.
Idaho, Iowa, Montana, Nebraska, North Dakota, South Dakota, Utah, and Wyoming PAPs
The Idaho, Iowa, Montana, Nebraska, North Dakota, South Dakota, Utah, and Wyoming PAPs
call for six-month reviews with language and scope similar to Arizona's. In addition, the PAPs
call for two other longer-term reviews with similar language. For example, from the Idaho PAP:
16.2 Two years after the effective date of the first FCC 271 approval of the PAP,
the participating Commissions may conduct a joint review by a independent third
party to examine the continuing effectiveness of the PAP as a means of inducing
compliant performance. This review shall not be used to open the PAP generally
to amendment, but would serve to assist Commissions in determining existing
conditons and reporting to the FCC on the continuing adequacy of the PAP to
serve its intended functions. The expense of the reviews shall be paid from the
Special Fund.
June 30,2009 ~~
The Liberty Consulting Group
Page 17
Final Report Analysis of Qwests
Penormance Assurance Plans
16.3 Qwest wil make the PAP available for CLEC interconnection agreements
until such time as Qwest eliminates its Section 272 affliate. At that time, the
Commission and Qwest shall review the appropriateness of the PAP and whether
its continuation is necessary. However, in the event Qwest exits the interLATA
market, that State PAP shall be rescinded immediately.
Either the six-month review or the second of these two longer-term reviews (Section 16.3) is the
most appropriate context for the current analysis. The PAP language indicates that the key
objective of the Section 16.3 review is the appropriateness of the PAP and whether it should
continue. In the South Dakota PAP, Qwest must petition the Commission for the elimination of
the PAP as a precondition for that review.
New Mexico
The New Mexico PAP has provisions for regular six-month review and a two-year review with
language similar to the other QPAPs. In addition, like the CPAP, the New Mexico PAP in
Section 16.3 has a sunset provision to occur six years after the PAP's effective date, eliminating
all sub-measurements except those listed in Attachment 3 and callng for a review with language
similar to the QPAP:
This QP AP wil expire six years from its effective date. Only the sub measurements
identifed in Attachment 3 and payments wil continue beyond six years, and these
sub measurements and payments shall continue until the Commission orders
otherwise. Five and one-half years after the QPAP's effective date, a review shall
be conducted with the objective of phasing-out the QPAP entirely. This review
shall focus on ensuring that phase-out of the QP AP is indeed appropriate at that
time, and on identifing any submeasurements in additon that should continue as
part of the QP AP.
In addition (Section 16.4),
The Commission may, at its discretion, join a multi-state effort to conduct QPAP
reviews and develop a process whereby the multi-state group would have the
authority to act on the Commission's behalf consistent with its authority under
law.
2. Approach of the CP AP Three-Year Review
In the CPAP Three-Year Review, BWG noted that it was guided by the requirements in CPAP
Sections 18.7 and 18.10 and the following general objectives designed to balance the needs of
the Commission, the CLECs, and Qwest:
. Simplification (reduction in complexities) ofthe CPAP
June 30.2009 ~~
The Liberty Consulting Group
Page 18
Final Report Analysis of Qwesl's
Performance Assurance Plans
. A voidance of significant incremental financial, administrative, or operational
harm to CLECs or Qwest
. Continuation of incentives for growth in the competitive marketplace
. Balancing the implications for end-user customers, CLECs, and the competitive
environment
· Level of historical penalties and implications for future payments
. Flexibilty for interested parties to request appropriate modification and provide
feedback on any such request.
BWG used the following factors in determining which specific measures and sub-measures
should be removed from the CP AP:
. Importance in measuring aspects of service which impact end-user customers, as
well as CLECs' ability to communicate with end-users, make customer
commitments, operate effciently, and compete on a level-playing field
. Simplicity of tracking, measuring, and reporting for Qwest, the Commission, and
CLECs
. Limited administrative burdens for Qwest and other paries
. Balancing financial and other harm for Qwest and CLECs
. Implications for future penalty payments given past performance and recent
trends (conforming performance in the past does not ensure good performance in
the future, while strong performance leading to no penalty payments is not a
burden for Qwest)
· Flexibility to address changes in regulatory requirements, business realities, and
the competitive landscape.
By design, the current analysis was conducted in a different mode from CPAP Thee-Year
Review. In particular,
. The analysis was designed to meet the separate needs of the eleven participating
states rather than a single state.
. The analysis was not intended to be part of any specific on-going reviews or
dockets in any of the participating states, but was intended as input to such
proceedings.
. There was no collaborative process between the CLECs and Qwest, since Qwest
elected not to provide proposals and recommendations and the CLECs provided
input through a single Liberty questionnaire.
. There were specific requirements and objectives for the Three-Year Review that
do not apply to the Six-Year Review in Colorado.
Thus, the objectives of Libert's analysis were necessarily somewhat different from BWG's.
Nevertheless, Liberty approached aspects ofthe analysis in ways similar to BWG's.
June 30, 2009 ~,~
The Liberty Consulting Group
Page 19
Final Report Analysis of Qwests
Penormance Assurance Plans
3.Guidance from the Collaborative Committee
As noted, the ROC Collaborative Committee specifically noted the following areas for this
analysis:
. The PAPs and PID measures, with draft recommendations concerning:
o The current effectiveness, value, and usefulness of the PAP and PID
measures in relation to their intended purpose and function
o Whether some or all of the PAP or PID measures may no longer be
necessary
o Possible modifications to the performance plan and PID measures.
. Paricipation of and consultation with the PAP stakeholders: Qwest, CLECs, and
Commissions.
The Collaborative Committee specifically noted that because of the different contexts of the
reviews in the different states, this analysis and the resulting recommendations should be
presented in a document to use for discussions and proceedings as each state deems appropriate.
4. Liberty's Approach
Given varied contexts for this review in the different participating states, Libert relied mainly
on the specific requirements of the review outlined by the ROC Collaborative Committee. Based
on these requirements, Liberty developed a work plan focused on five separate but related
investigations:
1. Analysis of PAP payments and PID measure results
2. Analysis of the structural components of the PAPs
3. Analysis of the structure of the PID measures
4. Analysis of recommendations and experiences of stakeholders
5. Analysis of industr trends.
During the course of the analysis Libert held project calls with members of the Collaborative
Committee and provided monthly status reports of the review to the Committee. Liberty also met
with the ROC Commission Staffs on April 23, 2009 in Denver to review the analysis, provide
initial results and conclusions, and seek input from the Staffs. Because of the lack of the abilty
to seek equal input from Qwest and the CLECs, Liberty's analysis was unable to benefit from the
give and take that such a process can provide.
Analysis 1- Historical PAP Payments and PID Measure Results
One method of assessing how well the PAPs and PID measures are working was to examine
trends in the payments, transactions volumes, and PID measure results over the life of the PAPs.
The purpose of this examination was to identifY measures that either might be consistently
generating payments or consistently meeting the standards. These were noted for further
June 30,2009 ~~
The Liberty Consulting Group
Page 20
Final Report Analysis of Qwests
Performance Assurance Plans
investigation. The analysis also examined PID measures that are reported but are not currently
part of the QPAPs to determine whether there was stil any value in Qwest reporting them or if
they should be considered for future inclusion in the PAPs. In addition, Liberty examined trends
in transaction volumes to determine whether some PID measures or measure reporting
dimensions contain so few transactions that they may no longer have value. Liberty also
examined any cases where PID measures have been removed from a QP AP and assessed whether
there was any evidence that this has influenced Qwest performance.
This analysis included the following steps:
· Qwest provided PAP payment results to Liberty for all 14 states for each month
beginning January 2004 and continuing through October 2008 (Study Period)
· Qwest provided PID measure results to Liberty for the 1 1 participating states for
each month from January 2004 through October 2008
· Libert analyzed trends in the PAP payments at the region-wide (14-state) level
. Libert analyzed trends in the PAP payments and measure results individually for
each of the 1 1 participating states
. Libert looked for cases where measures showed low or minimal PAP payments
. Liberty looked for product disaggregations with low activity volumes.
Analysis 2 - QPAP Structure
The . purpose of this analysis was to determine whether the current structure of the PAP is
meeting its original objectives. The analysis included the following steps:
. Libert obtained and reviewed the PAP documentation for each of the 1 1
participating states.
. Liberty reviewed the PAP strctural components (e.g., statistical methods,
payment levels, payment triggering mechanisms).
. Liberty examined whether the components appeared to be meeting their apparent
objectives or were no longer relevant based on the observed historical trends.
Analysis 3 - PID Measure Structure
The purpose of this analysis was to determine whether the current structure of the performance
measures is meeting its original objectives. The analysis included the following steps:
. Liberty obtained and reviewed the 14-state PID documentation
. Liberty reviewed the structure of the PID measures (e.g., formula, exclusions,
reporting disaggregations)
. Liberty examined whether there are any PID measures or components of the
measures that either do not appear to be meeting their apparent objectives, or are
no longer relevant based on the observed historical trends.
June 30, 2009 ~~
The Liberty Consulting Group
Page 21
Final Report Analysis of Qwests
Performance Assurance Plans
Analysis 4 - Stakeholder Input
In addition to examining the historical record and the structures of the PAPs and PID measures,
Libert sought input from the principal stakeholders of the PAPs: the Commissions, the CLECs,
and Qwest. Liberty asked the Staffs from each of the participating Commissions to provide their
own experiences, concerns, recommendations, and objectives related to the PAPs. In addition,
Liberty asked the Staff members to provide lists of CLECs to contact regarding their
experiences. Then, Liberty contacted the CLECs and Qwest for their input. This analysis
included the following steps:
. Libert asked the Collaborative Committee Commission Staff members for the 1 1
paricipating states for their information on:
o What historical information the states maintain on PAP payment and
Qwest performance results
o Lists of active CLECs in the state and contact information
o What information the states maintain about the CLECs and other
competitors
o Any input the Staffs had on concerns about or issues with the PAPs
. Libert developed a questionnaire and reviewed it with the Collaborative
Committee.
. Liberty sent the questionnaire to the CLECs and Qwest addressing:
o What components of the PAPs (including the PID measures involved) are
working well and are believed to be necessary to preserve going forward?
o What components of the PAPs (including the PID measures involved) are
not working well and should be changed going forward? If any, how
should they be changed?
o What components of the PAPs (including the PID measures involved) are
unnecessary and should be dropped going forward?
o Other comments or input.
Qwest declined to participate in providing this input.
Analysis 5 - Industry Trends
The purpose of this analysis was to determine trends in the competitive local telecommunications
industry in the 11 states that might affect the continued applicabilty of the PAPs. In paricular,
. Libert used FCC industry analysis reports to examine trends in competition in
each of the 1 1 participating states.
. Libert examined trends in transaction volumes by product type and function
based on the PID measure data provided by Qwest.
June 30. 2009 ~~
The Liberty Consulting Group
Page 22
Final Report Analysis of Qwests
Performance Assurance Plans
Libert used this information to determine to what extent the PAPs (including the included PID
measures, products, and specific strctural provisions) stil address the needs of the current
telecommunications marketplace.
Draw Conclusions and Develop Recommendations
Based on the five streams of analysis described above, Liberty drew conclusions and developed
recommendations for the three basic objectives of the study outlned by the Collaborative
Committee:
· Evaluation of the current effectiveness, value, and usefulness of the performance
plan and PID measures in relation to their intended purpose and function
· Determination whether some or all of the performance plan or PID measures may
no longer be necessary
· Consideration of possible modifications to the performance plan and PID
measures.
In particular, Liberty evaluated the continued need for PAPs, including whether they are
necessary or helpful in maintaining a competitive market. In addition, if the PAPs are to continue
Libert identified and developed recommendations for:
. PID measures that should be eliminated
. Product disaggregations that should be eliminated
. Revisions to and additions of PID measures
. Modifications to PAP strctural components.
In this evaluation, Liberty relied principally on the data from the Study Period. However, late in
the analysis, Liberty obtained from Qwest additional information about payments from
November 2008 through March 2009 for the 1 1 paricipating states, and used this additional
information in assessing whether to recommend the elimination ofPID measures.
June 30, 2009 ~~
The Liberty Consulting Group
Page 23
Final Report Analysis of Qwests
PenormaDce Assurance Plans
III. Analysis
A. Performance Assurance Plan Payment Trends
Based on data provided by Qwest, Liberty analyzed trends in PAP payments during the Study
Period. Through this analysis, Liberty examined which PID measures and product
disaggregations contributed most to the payments and how this varies from state to state and
during the Study Period.
During the course of its review, Liberty found that both the Tier 1 and Tier 2 payments decreased
dramatically during the Study Period. In 2004, monthly payments across all 14 states were more
than $600,000 on average (about $500,000 for the 11 states considered in this review). By 2008,
monthly payments were typically below $200,000 (below $100,000 in total for the 11 states
reviewed). Figure II-A- 1 below shows the combined Tier 1 and Tier 2 payments from January
2004 through October 2008 across the Qwest footprint.
Figure III-A-l
Total Tier 1 and Tier 2 Payments
All 14 States
. Total lier 2
_Total TIer 1
$1.400,000
$800,000
-
I I 111111111111
$1.200,000
$1.000.000
$600.000
$400,000
i
I $200,000
I $-I il ;"
L .."1- .l
il...f i' I'.. ./../.? ,/.... ..~../,.~ ,/.. ..../../ ../¡g...f
June 30,2009 ~,~
The Liberty Consulting Group
Page 24
Final Report Analysis of Qwests
Penormance Assurance Plans
1.Differences in Payments by State
Analyzing the Tier 1 and Tier 2 payments by state, Libert found that Arizona and Colorado
received about two-thirds of the total payments made during the Study Period for the 1 1 states
that participated in this review. However, each of the 11 states had significant payments, with a
total of at least $250,000 in every state. Figure II-A-2 below shows the Tier 1 and Tier 2
payments by state during the Study Period.
Figure III-A-2
Tier 1 and Tier 2
Payments by State
. Total Tier 2
. Total Tier 1
$7,000,000
$6,000,000
$5,000,000
$4,000,000
$3,000,000
$2,000,000
$1,000,000
$0
AZ co IA 10 MT NO NE NM SO UT wv
Liberty also reviewed how the payments changed over time for each state participating in this
review. Appendix B contains figures with Tier 1 and Tier 2 payments over time for each of the
11 states. For the most part, the Tier 1 payments went down uniformly. Tier 2 payments by state
declined in a similar fashion, with one major exception. Colorado payments for Tier 2
dramatically fell after April 2006.
Liberty found that the dramatic reduction in Colorado's Tier 2 payments resulted from changes
to the structure of the CPAP after the Three-Year Review. The August 15,2005 version of the
CPAP included two requirements for Tier 2 payments that were eliminated with the May 1, 2006
version:
. That Qwest make Tier 2 payments for Tier lA or Tier IB failures that were
missed by at least 50 percent of the applicable standard for two or more
June 30,2009 ~~
The Liberty Consulting Group
Page 25
Final Report Analysis of Qwests
Penormance Assurance Plans
consecutive months. This could result in a Tier 2 payment of $25,000 for each
Tier 1 A sub-measure missed and/or $8,000 for each Tier 1 B sub-measure missed.
. That Qwest make Tier 2 payments for "Tier I Y" failures. Fifty percent of the
penalties for Tier 1 Y failures were paid to the CLEC that received substandard
service and 50 percent was paid to Tier 2 Special Fund. The Tier 1 Y measures
calculated penalties on a Per Occurrence basis.
With the elimination of these two requirements, the remaining Tier 2 measures were calculated
on a Per Measurement instead of a Per Occurrence basis. The May 1, 2006 version of the
Colorado PAP maintained Tier 2 payments for the following region-wide Wholesale Support
Systems measures, which are also found in the other PAPs:
· GA-l, GA-2, GA-3, GA-4,and GA-6
. PO-l
. OP-2
. MR-2
The versions of the CPAP before and after May 1, 2006 allow for Tier 2 payments for PO-6,
which is not a Tier 2 measure in the other states. This measure, which evaluates the aggregate
performance to all CLECs, is also subject to Per Measurement instead of Per Occurrence
payments.
18
Table II-A-l below shows that before the May 1, 2006 revision, the highest Tier 2 payments
made in Colorado were similar to, though higher than, the payments per unit (i.e., the payments
made per transaction) made in other states. Beginning in May 2006, Colorado's Tier 2 payments
for these Per Occurrence measures went to zero while Qwest continued to make payments to
other states as before.
e ore an er e ay ,evision
Percent of Total
Percent of CO AverageTotal CO All Other States All Other States
CO Tier 2
Tier 2 Tier 2 Average Tier 2 Average Tier 2MeasurePaymentsPaymentsPayment per Payment per Unit -Payment per Unit -Before May Beginning Unit - Before Before May 2006 Beginning May 20062006
May 2006
May 2006
MR-5 9.9 0 LIS 49 48
MR-6 6.0 0 244 S8 69
MR-8 37.2 0 187 137 21S
PO-2 36.1 0 33 18 48
Table III-A-l
Tier 2 Payments for All States
B ti d Aft th M 1 2006 CP AP R
18 The revised May I, 2006 Colorado PAP did not contain Per Occurrence Tier 2 penalties like those found in other
PAPs for the following measures: GA.7, PO-S, PO-16, PO-19, OP-3, OP-4, OP-S, OP-6, OP-8, OP-I3, OP.I 7, MR-
6, MR-7, MR.8, MR-I I, BI-I, BI-4, NI-I, NP-I, and CP.2.
June 30,2009 ~~
The Liberty Consulting Group
Page 26
Final Report Analysis of Qwests
Penormance Assurance Plans
Additionally, Qwest noted that Tier 2 payments made for MR-8 Unbundled OS 1 and Enhanced
Extended Loop (EEL) DSI were suspended from May 2005 through June 2007 in Colorado
"pending a collaborative investigation by the Independent Monitor, Qwest, and CLECs.,,19
Qwest continued to pay Tier 2 penalties for MR-8 Unbundled OS 1 and/or EEL OS 1 in all other
states.
2. Differences in Payments by Domain
The vast majority of Tier 1 payments during the period under review were for performance in
Pre-Ordering/Ordering (PO), Ordering/Provisioning (OP), Maintenance & Repair (MR), and
Biling (81). In 2004 and 2005 Maintenance & Repair and Biling comprised the majority of Tier
1 payments. Since 2005, Maintenance & Repair alone comprised the majority of Tier 1
payments. Appendix B contains figures with Tier 1 payments by domain over time for each of
the 11 states. Figure II-A-3 below shows these Tier 1 payments by domain for all 14 states
combined.
Figure III-A-3
Total Tier 1 Payments by PID Domain
All 14 States
$100,000
. PO TotalTer 1
. OPTotal Tier 1
"MRTotaiTier i
. BlTotal Tier i
ä OtherTotal Tier 1
$600.000
$500.000
$400,000
$300.000
$-~ ~ ~ ,? ~ ¡:' "," ¡:" ¡:" ~ "," ¡:" tl sP,/" sP sP ¡:" ~ ~ ¡:~ ¡:~ "'~ ~ r! ¡:'" ¡:'" ¡:'" ~,." ~~ _.... ,~ ,¡q #i ,." .., .... ,s ,¡"I #' ,4'- _.~ #,,¡q #i ,:" ,l .,¡ ~ ,¡"I #i ,4' ~., ~ç. ,s ,¡'Í'\..-.~-'\.... ~.."'..¥'~.... "\..~-'\ "'.... 'I....~...... ..........
~nJjI1L ll~
$200,000
$100,000
19 Response to Qwest Data Req uest # I 3.
June 30, 2009 ~,~
The Liberty Consulting Group
Page 27
Final Report Analysis of Qwests
Performance Assurance Plans
Most Tier 2 payments were for Maintenance & Repair, but in 2004 and early 2005, Qwest paid
significant amounts of Tier 2 payments for Pre-Ordering/Ordering and Biling performance. The
large Tier 2 payments for Maintenance & Repair occurred frequently until 2008 when Tier 2
payments in general were minor due to improved performance for MR-2 and MR-8. Figure II-
A-4 below shows these Tier 2 payments by domain.
Figure III-A-4
Total Tier 2 Payments by PID Domain
All 14 States
. POTotol Tier 2
. OPTotolTier 2
$800,000
.81 Totol Tier 2
$700,000
"OtherTotolTier 2
$600,000
$500,000
$400,000
$300,000
$-
_" _" _" _" .. .. ~ ~ ~ ~ ~ ~ .. _10 .. .. .. b ~ ~ ~ ~ ~ ~ .. .. .. .. ..
,ç'" ,'" ,l "" rl ~ J' ,~ l' ,J l' )$ ". .~". ,r :1" rl"..J ,f'''.'' ,? ;i .5' J' ? J' ,J rl"..' ..~~.l ..'~ ..'" .... .."' ..~. ~~ ..'~ ..'" .... .."''' ...j .. ..'l or .."' o,"".. o,'~..'" .. .."' .j o,oN ..0/..",
1 il ~~$200,00
$100,000
Only 15. measures were responsible for the overwhelming majority (97 percent) of Tier 1 and
Tier 2 payments in the 11 states in the study. The MR-8 measure was associated with over $5
millon of the approximately $16 milion in Tier 1 and Tier 2 payments during the Study Period.
Payments for BI-3A accounted for more than $2 milion of Tier 1 and Tier 2 combined payments
during the Study Period. Figure II-A-5 below shows the Tier 1 and Tier 2 payments by measure.
The top three measures (MR-8, BI-3A, and OP-4) generated 57 percent of the payments during
the Study Period.
June 30,2009 ~,~
The Liberty Consulting Group
Page 28
Final Report Analysis of Qwests
Performance Assurance Plans
)y -ta es
. Total Tier 2
8 Total Tier
S6.00.oø L
$5,000,000 .
$4,000,000 I
$3,000,000 __________m________"',,___"'''_,,__,______~_~_,~_____,__._______"'__._______.____.__________,__._____,________.._..______.w_
$2,000,000 -
$1.000,000 .........,lJ,lm~-;-~...:~=~~-:-,..$0
9:'b ~",,,
~l)","'\~9:~9:~~",9:"9:""~"~~'\'l r, ", t)"
.N (j ~ç,O .N .N (j .N .N Oç,(j ~""
~Q3 ~'l ç,~
(5 (5 ~
~L'.,
iii~(l
Figure III-A-5
Total Tier 1 and Tier 2 Payments
b PID 11 S t
Table II-A-2 below provides the detailed data in the figure above.
)y -tates
PID Code Title Total Tier 1 Total Tier 2 Total Tier 1 and
Tier 2 Payment
MR-8 Trouble Rate $2,656,552 $2,556,198 $5,212,750
BI -3A Biling Accuracy - Adjustments $2,011,695 $114,308 $2,126,003for Errors (UNEs and Resale)
OP-4 Installation Interval $1,031,604 $548,242 $1,579,846
BI-IA Time to Provide Recorded Usage $964,463 $398,182 $1,362,645Records (UNEs and Resale)
PO-2B Electronic Flow-through (all flow-$4,543 $1,065,000 $1,069,543through-eligible LSRs)
MR-5 All Troubles Cleared within 4 $679,753 $268,349 $948,102hours
MR-6 Mean Time to Restore $748,702 $161,616 $910,318
OP-3 Installation Commitments Met $460,897 $I78,711 $639,608
MR-7 Repair Repeat Report Rate $335,064 $95,313 $430,377
MR-2 Calls Answered within 20 Seconds N/A $344,000 $344,000- Interconnect Reoair Center
Table III-A-2
Total Tier 1 and Tier 2 Payments
b PID 11 S
June 30,2009 ~~
The Liberty Consulting Group
Page 29
Final Report Analysis of Qwests
Penormance Assurance Plans
OP-5A New Service Installation Quality $162,118 $14,976 $177,094Reported to Repair
OP-6 Delayed Days $105,720 $20,813 $126,533
Timeliness of Disconnects
OP-17A associated with LNP Orders $II 1,190 $0 $1 I 1,190
(timely CLEC requests)
OP-8C Number Portability Timeliness $104,657 $4,500 $109,157(without Loop Coordination)
MR-3 Out of Service Cleared within 24 $90,770 $446 $91,216Hours
Remainder 0 f $349,996 $182,788 $532,784PIDs---
The list and order of the top 15 PIDs with regard to total Tier 1 and Tier 2 payments changes
slightly if only more recent data after May 2006 is used. Nevertheless, MR-8, OP-4, and BI-3A
remain the top three contributors, generating 60 percent of the payments; the top 15 PID
measures stil represent 98 percent of the Tier 1 and Tier 2 payments in the 1 1 states during this
more recent period. Table II-A-3 below shows these detailed data.
)y a er ay -a es
Total Tier i and
PID Code Title Tier i Payment Tier 2 Payment Tier 2 Payment
MR-8 Trouble Rate $675,634 $284,453 $960,087
OP-4 Installation Interval $503,606 $380,666 $884,272
Biling Accuracy -
Adjustments for Errors
BI-3A (UNEs and Resale)$447,717 $0 $447,717
Calls Answered within 20
Seconds - Interconnect
MR-2 Repair Center N/A $342,000 $342,000
MR-6 Mean Time to Restore $275,799 $0 $275,799
Installation Commitments
OP-3 Met $181,432 $75,675 $257,107
All Troubles Cleared within
MR-5 4 hours $207,662 $0 $207,662
Time to Provide Recorded
Usage Records (UNEs and
BI-IA Resale)$90,099 $0 $90,099
MR-7 Repair Repeat Report Rate $82,829 $4,000 $86,829
OP-6 Delayed Days $47,143 $3,900 $51,043
Calls Answered within
Twenty Seconds -
Interconnect Provisioning
OP-2 Center $0 $45,833 $45,833
New Service Installation
OP-5A Quality Reported to Repair $39,915 $2,450 $42,365
Table III-A-3
Total Tier 1 and Tier 2 Payments
b PID ft M 2006 11 St t
June 30, 2009 ~~
The Liberty Consulting Group
Page 30
Final Report Analysis of Qwest's
Performance Assurance Plans
Biling Completion
PO-7A Notification Timeliness $27,317 $0 $27,3 I 7
Work Completion
PO-6A Notification Timeliness $26,949 $0 $26,949
Out of Service Cleared
MR-3 within 24 Hours $25,485 $0 $25,485
Remainder 0 f
PIDs ---
$57,193 $18,417 $75,610
B. Wholesale Volumes
As part of its review, Libert analyzed the ordering volumes and total lines in service for the
wholesale products and services that Qwest records in the PID measures, and examined how
these volumes have changed over the Study Period.B ecause some products, such as those Qwest
provides through commercial agreements, are not included in the PID measurements, this
analysis is not meant to depict the full scope of wholesale products available. Nevertheless, it
does show the volumes of transactions and lines relevant for determining the PAP payments.
During its examination of ordering volumes captured in the PID measurements, Liberty used the
number of service orders Qwest recorded in its PID measurements for inward services20 and for
standalone number ports. As a measure of service order volumes for inward services, Liberty
used the denominator of OP-3 (Installation Commitment Met) which provides data on the total
number of service orders for inward services completed in the reporting period.21 As a measure
of LSR volumes for standalone number port orders, Liberty used the denominator of OP-8C
(LNP Timeliness without Loop Coordination).
Liberty also examined trends in the number of CLEC lines in service which depend on Qwest
wholesale services measured by the PID measurements. For this quantity, Liberty used the
denominator ofthe MR-8 (Trouble Rate measure), which provides data on the total number of
lines in service for each product disaggregation during the reporting period.
This analysis revealed that overall wholesale service order transaction volumes measured in the
PID measures have decreased significantly over the course of the Study Period as shown in
Table II-B-l.
20 Inward services are defined as order types for change of service, new service, and transfer of service.
21 In its analysis, Liberty did not include disconnect and record change service orders which are excluded from the
OP-3 service order volumes. Liberty also excluded any other service orders subject to the OP-3 exclusions, such as
service orders for which the due date was missed for non-Qwest reasons and service orders with invalid due dates or
application dates. Despite these exclusions, Libert was able to obtain suffcient information to determine the order
volume trends for the purposes of this study, using the total inward order volumes obtained from the Op-3
denominator.
June 30,2009 ~~
The Liberty Consulting Group
Page 31
Final Report Analysis of Qwests
Penormance Assurance Plans
o a -a e e210n
Year Annual Order Difference Monthly
Volume from Previous Average
Year
2004 1,022,208 N/A 85,184
2005 377,738 -644,470 31,478
2006 245,067 -132,671 20,422
2007 162,966 -82,101 13,581
2008 (through 113,643 *II,364
October)
Table III-B-l
Inward Service Order Volumes
T t 114 St t R .
* The difference \\ll not be indicative of the full year experience as the 2008 data only ex1ends though October.
Liberty found that the biggest factors in this decrease in order volumes measured by the PID
measures were the TRO and the TRRO decisions, which removed the requirement for Qwest to
offer UNE-P and Line Sharing services as unbundled network elements under Section 251 of the
1996 Telecommunications Act. ii As a result of these orders, Qwest offers services equivalent to
UNE-P and Line Sharing under commercial agreements with the CLECs, which are not tracked
or reported by the PID measures.23 In 2004, these services accounted for 745,490 (72.9 percent)
of the 1,022,208 total inward orders issued to Qwest by the CLECs. The vast majority (722,988)
of these 745,490 orders were for UNE-P POTS. However, after the effective date of the TRRO
in 2005, the order volume for these services dropped to 8,928 orders, a 98.9 percent decrease
from the previous year's order volumes. The order volume for Line Sharing and UNE-P service
has continued to decline to the 2008 region-wide monthly average of only 31 orders per month
for these services.24
Libert found that Qwest has experienced declining ordering volume for all wholesale services
and products with the exception of the five products shown in Table II-B-2. For example, in
2004 Qwest averaged 5,260 orders per month for all Resale services, while in 2008 the average
dropped to 814 resale orders per month. For all other inward service orders (e.g., unbundled
loops, unbundled dedicated interoffice transport, Enhanced Extended Loops, etc.), Qwest
averaged 17,800 orders per month in 2004. The average number of monthly orders Qwest
received for these services in 2008 was 8,763 (through October). Two factors that may have
contributed to the drop in Qwests overall wholesale service order volumes are i) CLECs
entering into commercial agreements with Qwest, thereby removing their ordering activity from
the PID calculations, and ii) the general trends in the telecommunications industry of customers
22 FCC 03-36, Triennial Review Order, August 21, 2003 and FCC 04-290 Triennial Review Remand Order,
February 4, 2005.23 In response to Data Request # 1 8, Qwest explained that it replaced its UNE-P offer with its commercial products
offering of Qwest Platform Plus (QPP). This offering was later replaced by Qwest Local Services Platform (QLSP).24 In response to Data Request # 18, Qwest indicated that it stil reports a small number of UNE-P and Line Sharing
services in the PID because not every CLEC completed the interconnection/commercial agreements with Qwest that
would have moved these services to the non-reportable comparable commercial agreement service (e.g., the Qwest
Local Service Platform replacement service for UNE-P service).
June 30, 2009 ~~
The Liberty Consulting Group
Page 32
Final Report Analysis of Qwests
Penormance Assurance Plans
migrating away from the wireline service business to competitors such as cable and wireless
service providers.
25
P d h E
Table Iß-B-2ed I . 0 d V Iro ucts t at xpenenc an ncrease in r er o urnes
Product Percent Increase (2004 - 2008)
2-Wire Non-Loaded Loops 8.6
ADSL Caoable Looos 22.0
EEL-DS-I 34.3
DS-I Loops 64.0
xDSL Capable Loops 90.7
Consistent with industry trends of reduced reliance on wireline telecommunications service,
Qwest experienced a steady increase in the number of service orders it received from its
competitors for stand-alone LNP. Stand-alone LNP orders are typically issued by competitors
such as wireless carriers and cable companies that are able to gain access to the end-user without
relying on Qwests wireline facilities to do so. These cariers typically only order number ports
(to move the existing customer's telephone number to their network), directory listings, and
interconnection trunks from the Incumbent Local Exchange Carriers (lLECs). Table II-B-3
shows the trend in Qwests stand-alone LNP service order volumes.
Year Total LNP Orders Difference Monthly A vera2e
2004 466,374 N/A 38,865
2005 553,943 87,569 46,162
2006 588,090 34,147 49,008
2007 597,430 9,340 49,786
2008 (through October)523,221 *52,322
Table ßI-B-3
Stand-alone LNP Service Order Volurnes26
* The difference will not be indicative of the full year expenence as the 2008 dat only extends though October.
As can be expected from the results on Qwests order volume trends, the wholesale lines in
service followed the same downward trend in volume as shown in Table II-B-4.
YearEnd Lines In Service Difference
Volume
December 2004 2,712,891 N/A
December 2005 1,890,999 (821,892)
December 2006 1,726,161 (164,838)
December 2007 1,691,505 (34,656)
October 2008 1,624,765 *
Table III-B-4
Wholesale Lines In Service
* The diffrence will not be indicative of the full year expenence as the 2008 data only extends though October.
25 See Section E, "Industry Trends" for additional details.
26 The data for this table was obtained from the denominator of the OP-8C "Number Portbilty Timeliness"
measure, which provides data on the total number of standalone number ports.
June 30, 2009 ~~
The Liberty Consulting Group
Page 33
Final Report Analysis of Qwests
Performance Assurance Plans
The very large reduction in the number of lines in service between 2004 and 2005 is also a result
of the TRO and TRRO decisions. In December 2004, Qwest reported 876,122 UNE-P and Line
Sharing lines in service. By December of2005 this number dropped by 727,778 lines to a total of
148,344 in-service UNE-P and Line Sharing lines, accounting for 88.5 percent of the total loss in
wholesale lines between 2004 and 2005. This number was further reduced by 87,612 lines to a
total of 60,732 UNE-P and Line Sharing lines in service at the end of 2006, accounting for 53.2
percent of the wholesale line loss in 2006. Qwest also experienced a significant reduction in its
Resale service lines during the Study Period. Qwests year-end-2004 resold lines in service
totaled 143,895 lines. Since that time the number of in-service Resale lines has decreased by 60.8
percent to 56,389 lines as of October 2008. Additionally, many of the wholesale lines that are
reported in the PID measures may have also been replaced by such commercial-agreement
services as Qwest Local Service Platform (QLSP) which would not be reflected in the numbers
reported by Qwest in the PID measures.
For other wholesale services such as UN-L, the reduction in in-service lines has not been as
significant as it has been for UNE-P and for Resale lines. Qwest ended 2004 with 1,692,874 lines
in service for these products, as counted by the PID measures, and since that time there has only
been a 10.6 percent reduction in these in-service line counts to an October 2008 total of
1,513,970 lines in service. As shown in Table II-B-5, Qwest has actually experienced very
significant growth from 2004 to October 2008 in some products in this product category.
Table III-B-5
U b dl d Pdt th tEd S' 'ti t L' G thn un e ro uc s a xpenence 1201 ican me row
Product Percent Increase (2004 - October 2008)
2-Wire Non-Loaded Loops 20.7
DS-I Loops 7I.
EEL-DS-I 117.7
Sub-Loops 346.1
xDSL Loops 642.5
ADSL Capable Loops 787.9
The following region-wide graphs provide a summary view of the trends discussed above in
Qwests wholesale order transactions and wholesale Iines-in-service volumes.
June 30. 2009 ~,~
The Liberty Consulting Group
Page 34
Final Report Analysis of Qwests
Penormance Assurance Plans
Figure III-B-l
Orderin Volumes: All States
50,000
. UNE LOOPS & OTHER
. UNE-P & LINE SHARING
. RESALE
fl lNP
200,000
150,000
!I
IE:iÕ::100,000..!I
5
0
""""""""""Ll Ll Ll Ll (Ø (Ø (Ø (Ø r-r-r-r-eo eo eo000000000000000000000000000000000000000~(\(\(\(\(\(\(\(\(\(\(\(\(\(\(\(\(\(\(\i:..i:lí (,..i:0.(,..i:0.(,..i:0.(,..i:0-m c:::(\m ::(\(\c:::(\(\c:::Q)Q)m ::(J--::--(f 0 ::--(f 0 ::--(f 0 ::--(f 0 ::--C/
June 30, 2009 ~,~
The Liberty Consulting Group
Page 35
Final Report Analysis of Qwests
Performance Assurance Plans
Figure III-B-2
Lines in Service: All States
3.000.000
~.~
lI 2.000.000If
.5
mi:::
1,000,000
. UNE LOOPS 8. OTHER
. UNE-P 8. LINE SHARING
. RESALE
0 ..'O ......i(i(i(i(ID ID ID ID t-t-t-t-eo eo eo00000000000000000000000000000000000~0 0 0 0NNNNNNNNNNNNNNNNNNNi:..i:Q.(...i:Q.(...i:Q.(...i:Q.(.'-i:Q.ti ti ::l1 l1 ti ::l1 l1 ti ::l1 l1 ti ::l1 l1 ti ::l1-.~-.(j 0 ~-.(J 0 ~-.(J 0 ~-.(j 0 ~-.(j
The same downward trends in both overall wholesale order volumes and wholesale lines in
service can be seen at the state leveL. Although there are some state-specific variations, such as
an increase in the order volumes in Montana, Libert found that the general trend has been a
decrease in the number of inward wholesale orders and in wholesale lines in service as reflected
in Tables II-B-6 and II-B-7 respectively.
Table III-B-6
State-Specifc Wholesale Order Volumes
Total Inward Orders
2004-2008
State 2004 2005 2006 2007 2008 (through
October)
Arizona 305,039 168,613 140,204 117,784 87,890
Colorado 179,329 147,464 120,413 I I 1,23 I 106,503
Iowa 106,795 53,169 48,830 45,061 29,882
Idaho 29,502 15,761 9,128 16,566 12,367
Montana 11,226 10,327 17,815 16,718 12,253
North Dakota 30,351 17,045 11,549 26,873 9,400
Nebraska 58,881 25,376 18,635 16,169 14,401
New Mexico 42,743 20,514 17,827 17,446 18,295
June 30, 2009 Page 36~,~
The Liberty Consulting Group
Final Report Analysis of Qwests
Performance Assurance Plans
South Dakota 36,222 11,553 23,032 16,010 8,930
Utah 128,612 83,712 57,623 53,558 53,160
WyominlZ 15,352 8,624 13,698 9,415 7,698
State Year End 2004 Year End 2005 Year End 2006 Year End 2007 October 2008
Arizona 398,595 245,566 230,897 240,901 229,799
Colorado 367,218 289,842 249,935 265,888 250,443
Iowa 175,631 114,844 94,127 82,266 78,682
Idaho 53,886 34,702 31,401 32,573 31,959
Montana 29,603 26,643 28,026 26,677 25,406
North Dakota 42,360 39,981 38,319 34,334 34,006
Nebraska 98,962 50,609 43,549 41,712 4 1,238
New Mexico 71,298 47,525 45,576 41,913 38,107
South Dakota 54,748 20,715 18,462 16,714 16,702
Utah 208,053 148,250 130,027 129,725 127,288
Wyominl!30,147 12,973 13,256 11,826 11,976
Table Ill-B-7
State-Specific Wholesale Lines In Service
Total Lines
2004-2008
As displayed in Table II-B-8, the trend for standalone LNP orders, which is currently the most
prevalent wholesale ordering type, shows quite a bit of variation across the states with many
states showing significant growth in the annual volume of these orders while the order volumes
in other states (such as North Dakota and South Dakota) seem to fluctuate from year to year.
Arizona and Nebraska have experienced a declining volume in these orders.27 Graphs of the
state-specific order volume trends can be found in Appendix B.
State 2004 2005 2006 2007 2008 (through
October)
Arizona 146,894 135,191 119,841 100,451 77,460
Colorado 53,754 92,245 89,871 91,405 91,000
Iowa 40,041 24,230 32,762 36,438 25,720
Idaho 10,189 9,218 6,517 14,262 9,548
Montana 2,964 6,836 15,087 14,654 10,716
North Dakota 3,460 7,380 4,640 22,645 6,683
Nebraska 19,406 18,886 15,471 13,390 12,986
New Mexico 5,865 12,787 11,643 14,858 16,819
South Dakota 13,033 7,786 19,903 15,431 8,652
Utah 31,110 37,305 38,947 40,383 41,892
WyominlZ 990 3,792 11,072 8,487 7,262
Table Ill-B-8
State-Specifc Standalone LNP Order Volumes
Total Orders
2004-2008
27 Because the data for 2008 is incomplete, it is possible that Nebraska's 2008 volumes wil match or exceed its
2007 volumes which wil represent the first time the downward trend in Nebraska LNP order is reversed.
June 30, 2009 ~,~
The Liberty Consulting Group
Page 37
Final Report Analysis of Qwests
Penormance Assurance Plans
Libert also found that most of the in-service line losses were in the UNE-P and Resale products.
The line loss for all other products, such as UNE-L, has not been as large and in some states has
even grown. The trends in UN-P, Resale, and UNE-L are depicted in Tables II-B-9, II-B-lO
and II-B-ll respectively. Additionally, graphs of the state-specific trends in line loss can be
found in Appendix B.
State Year-End 2004 Year-End 2005 Year-End 2006 Year-End 2007 October 2008
Ariona 149,279 12,066 7,594 4,895 4,057
Colorado 95,748 32,726 17,3 I 9 10,420 8,879
Iowa 56,171 6,779 2,140 1,228 1,899
Idaho 15,862 588 6 4 16
Montana 7,264 365 214 83 65
North Dakota 19,085 1,055 24 30 343
Nebraska 38,918 2,743 764 429 479
New Mexico 24,895 6,261 4,484 2,985 3,064
South Dakota 31,151 581 6 5 0
Utah 74,109 16,854 4,514 3,028 2,663
Wyoming 19,142 2,326 2,134 1,430 1,491
Table III-B-9
State-Specific Lines In Service
UNE_p28
2004-2008
State Year-End 2004 Year-End 2005 Year-End 2006 Year-End 2007 October 2008
Arizona 6,188 5,121 4,332 2,464 1,979
Colorado 11,842 8,506 5,147 4,960 4,494
Iowa 9,958 9,398 5,239 3,841 3,605
Idaho 1,339 1,045 714 551 554
Montana 9,652 6,800 5,067 3,843 .3,362
North Dakota 4,818 3,222 2,536 1,506 1,040
Nebraska 2,925 2,179 1,069 846 778
New Mexico 2,812 1,855 1,554 1,165 1,022
South Dakota 5,936 4,263 3,357 2,788 2,399
Utah 2,973 3,688 2,586 1,646 2,140
Wyoming 3,129 2,848 1,681 1,017 839
Table III-B-IO
State-Specific Lines In Service
Resale Lines
2004-2008
28 In response to Data Request # 18, Qwest indicated that not all CLECs have completed the commercial agreements
that require Qwest to continue reporting in-service UNE-P lines as UNE-P rather than as the QLSP commercial
agreement replacement product for UNE-P.
June 30,2009 ~~
The Liberty Consulting Group
Page 38
Final Report Analysis of Qwests
Performance Assurance Plans
State Year-End 2004 Year-End 2005 Year-End 2006 Year-End 2007 October 2008
Arizona 243,128 228,379 218,971 233,542 223,763
Colorado 259,628 248,610 227,469 250,508 237,070
Iowa 109,502 98,667 86,748 77,197 73,178
Idaho 36,685 33,069 30,681 32,018 31,389
Montana 19,951 19,478 22,745 22,751 21,979
North Dakota 37,542 35,704 35,759 32,798 32,623
Nebraska 57,119 45,687 41,716 40,437 39,981
New Mexico 43,591 39,409 39,538 37,763 34,021
South Dakota 17,661 15,871 15,099 13,921 14,303
Utah 130,971 127,708 122,927 125,05 I 122,485
Wvominl!7,876 7,799 9,441 9,379 9,646
Table III-B-11
State-Specific Lines In Service
"UN-L and Other" Lines29
2004-2008
Based on this analysis of volume trends, Liberty concludes:
. The TRO and TRRO had a significant impact on the volume of service orders and
lines in service that Qwest receives and reports on in the PID measures.
. With the exception of orders for a small number of UNE-L products, EELs, and
stadalone number ports, the trend in Qwests overall LSR order volumes
continues to show declining volumes.
· Lines in service dropped dramatically for Resale and UN-P, and to a much lesser
extent for unbundled network element products.
. Competition is increasing from service providers that provide their own facilties
to the end-users such as wireless and cable companies, as evidenced by the
generally increasing volume of stadalone number port orders.
. There are some state level variations to the regional level trends, such as the
fluctuating or declining volumes in standalone number port orders in some states.
c. Qwests Performance
Qwest s overall performance across the states during the course of the Study Period showed
improvement. As detailed in Section III. A, failures for specific measures and their product
disaggregations were the underlying cause of the majority of Tier 1 and Tier 2 payments.
To better understand the results relating to failure rate, the statistical design of the PAPs needs to
be considered. That design allows for random variation in month-to-month performance. This is
because a process that is producing parity results wil, by chance, be below parity some months
and above parity some months. The statistical tests that are par of the PAP only produce a
failng result five percent of the time when Qwest is operating at parity. The percentage is lower
29 The "other" category includes all lines that are not provided by Qwest resold service or by UNE-P service. It does
not include lines that are self-provided by the CLEC such as cable company lines.
June 30, 2009 ~~
The Liberty Consulting Group
Page 39
Final Report Analysis of Qwests
Performance Assurance Plans
for small sample sizes. Thus, failure rates of below five percent can be considered to be artifacts
ofthe statistical framework and not a true indication that Qwest is providing substandard service.
When sample sizes are small, there is simply not enough information in most circumstances to
make a clear determination whether Qwest has met the standard (at or above parity for parity
measures or at or above the benchmark for benchmark measures), and the tests applied to
determine penalties can lead to biases. For most states, the Z-score cutoff for failure declines
when sample sizes are below ten, making it more likely that Qwest wil pay penalties even when
they are operating at parity. (Such a condition is known in statistics as a "Type I error.") In
contrast, most states have now added the "One Allowable Miss Rule," which applies to
benchmark and non-interval parity measures. This rule prevents a single miss from causing
payments, and means that for small sample sizes (tyically below 20) service below benchmark
may not lead to payments. (Such a condition is known in statistics as a "Type II error.") The end
result is that when sample sizes are small, Qwest wil pay penalties in more circumstances when
they are operating at parity (for parity measures), and Qwest wil not pay penalties in more
circumstances when they are operating below the benchmark (for benchmark measures and non-
interval parity measures).
Liberty reviewed the average and median sample sizes per test over the Study Period. Because
most measures are broken down by product, state, and CLEC, the median sample size was only
about four for the entire period. The average sample size began at about 70 and dropped to 50
over the course of the Study Period. As a result, the small sample size rules used in the PAPs are
becoming more important in determination of penalties, and the Type I and Type II error issues
mentioned above are more prevalent.
The following graphs show the percentage of state-level measure failures over time by
measurement domain: Pre-Ordering/Ordering (PO), Ordering/Provisioning (OP), Maintenance &
Repair (MR), Biling (81), and all other measures.
For Pre-Ordering/Ordering, measure failures began at about seven percent in 2004 and slowly
declined to approximately three percent by 2008. As noted above, a failure rate near five percent
would be expected even when Qwest is providing service at, or slightly above, the standard.
June 30,2009 ~~
The Liberty Consultng Group
Page 40
Final Report Analysis of Qwests
Performance Assurance Plans
Figure III-C-L
Percent of Failed PO PIDs
15
12
9
6
3
0
'i 'i 'i 'i 'i 'i 1I 1I 1I 1I 1I 1I \!\!\!\!\!\!""""""co co co co co0000000000000000000000000000000000000000000000000000000000NNNNNNNNNNNNNNNNNNNNNNNNNNNNNi:..;:~0-;:i:..;:"S 0-;:i:..;:"S 0-;:i:..;:"S 0-;:i:..;:"S 0-.!tt tt Q)0 tt tt tt ..Q)0 tt tt tt ..Q)0 tt tt tt ..Q)0 tt tt tt ..Q)~~VI Z ..~~VI Z ..~~VI Z ..~~VI Z ..~~VI
Table ii-C- 1 table shows the average failure rate by year for PO measures.
Table III-C-LA F" P tb Yvera2eai ure ercen I)y ear-
Year Averaiie Percent Failure
2004 7.3
2005 4.3
2006 2.8
2007 3.2
2008 3.2
PO
June 30, 2009 ~~
The Liberty Consulting Group
Page 41
Final Report Analysis of Qwests
Penormance Assurance Plans
For Ordering/Provisioning, the percent of PID measure failures was similarly low. The monthly
average decreased from six percent in 2004 to two percent in 2008. Figure II-C-2 shows the
monthly failure rate for OP measures over time.
Figure III-C-2
Percent of Failed OP PIDs
10
9
8
7
6
5
3
4
2
1
o
'O 'O 'O 'O 'O 'O i.i.i.i.i.i.U)U)U)U)U)U)""""""00 00 00 00 000000000000000000000000000000000000000000000000000000000000NNNNNNNNNNNNNNNNNNNNNNNNNNNNN
c:..;.a Co ;;c:..;.a Co ;;c:..;.a Co ;;c:..;.a Co ;;c:..;."S Co.!ro ro Qj 0 .!ro ro Qj 0 ro ro ro Qj 0 .!ro ro Qj 0 .!ro ro ..Qj~~VI Z ~~VI Z ..~~VI Z ~~VI Z ~~VI
Table II-C-2 show the average failure rate by year for OP measures.
Table III-C-2Av F'I P b Yeral!e ai ure ercent )y ear-
Year Avera2e Failure Percent
2004 5.7
2005 3.7
2006 4.2
2007 3.0
2008 2.2
OP
June 30, 2009 ~~
The Liberty Consulting Group
Page 42
Final Report Analysis of Qwests
Performance Assurance Plans
For Maintenance & Repair the percent ofPID failures also remained at or below ten percent. The
monthly averages decreased from seven percent in 2004 to four percent in 2008. The following
figure shows the monthly failure rate for MR measures over time.
Figure III-C-3
Percent of Failed MR PIDs
12
10
8
4
2
.H.I0
""""""""""""i.i.i.i.i.i.lD lD lD lD lD lD """"""co 00 00 co 000000000000000000000000000000000000000000000000000000000000NNNNNNNNNNNNNNNNNNNNNNNNNNNNNi:..;:~a.;:i:..;:~a.;:i:..;:~a.;:i:..;:~a.;:i:..;:"'a.
E¡l'l'Ql 0 E¡l'l'Ql 0 l'l'l'Ql 0 E¡
l'l'Ql 0 E¡
l'l'Ql:;:;in Z :;:;in Z ..:;:;in Z :;:;in Z :;:;..in
The following table show the average failure rate by year for MR measures.
A
Table III-C-3F" P b Y MRvera!!e ai ure ercent )y ear-
Year Average Failure Percent
2004 7.3
2005 5.2
2006 5.8
2007 5.3
2008 4.0
For Biling measures, the percent of PID measure failures varied more over time than the other
domains. In 11 out of the 58 months under review (approximately 19 percent), there were no
failures, while in November 2005 the percentage peaked at 36 percent. For six of the last 12
months, failure rates were at or above ten percent. The figure below shows monthly failure rates
for BI measures.
June 30, 2009 ~~
The Liberty Consulting Group
Page 43
Final Report Analysis of Qwests
Performance Assurance Plans
Figure III-C-4
Percent of Failed BI PIDs
40
35
30
25
20
15
5
10
o 11.......... .........
oc oc oc oc oc oc in in in in in in Ul Ul Ul Ul Ul Ul i-i-i-i-i-i-co co co co co0000000000000000000000000000000000000000000000000000000000NNNNNNNNNNNNNNNNNNNNNNNNNNNNNi:..;::;c.::i:..;::;c.::i:..;:~c.::i:..;:~c.::i:..;::;c...11 11 Ql 0 ..11 11 Ql 0 ..11 11 Ql 0 ..11 11 Ql 0 11 11 11 Ql~~-.VI Z ~~-.in Z ~~in Z ~~VI Z -.~~-.VI
The following table show the average failure rate by year for BI measures.
A
Table III-C-4F" P tb Y BIvera2;e ai ure ercen )y ear-
Year Averal!e Failure Percent
2004 6.5
2005 io
2006 I I.
2007 io
2008 4.8
For all other measures, there were no PID failures in 40 out of 58 months under review. In
January and August 2004, however, the PID failure rate peaked at 12 percent. In July 2008, the
PID measure failure rate was ten percent. In general, the failure rate for the other measures was
low with a few anomalies.
June 30,2009 ~~
The Liberty Consulting Group
Page 44
Final Report Analysis of Qwest's
Performance Assurance Plans
Figure III-C-5
Percent of Failed PIDs for Other Measures
15
12
9
6
3
I i ........................................................................................................1..
o:o:o:o:o:o:i.i.i.i.i.i.\0 \0 \0 \0 \0 \0 """"""00 00 00 00 000000000000000000000000000000000000000000000000000000000000NNNNNNNNNNNNNNNNNNNNNNNNNNNNNc:..;::i c.~c:..;::i c.~c:..;:c.~c:..;::i c.~c:..;::i c.ni ni ni Q)0 ni ni ni Q)0 ni ni ni .a Q)0 .!ni ni Q)ni ni ni Q)~~..~..~~..~....Vl Z ..~Vl Z ..~Vl Z ~Vl Z ..~Vl
The following table show the average failure rate by year for other measures.
Av
Table III-C-5F" P b Y o herera!!e ai ure ercent iy ear-t
Year Avera2e Failure Percent
2004 5.0
2005 0.7
2006 0.2
2007 0.2
2008 1.0
In summary, Qwests failure rate by measurement area across states showed improvement during
the Study Period. For all domains, average failure rates in 2008 are below five percent, indicating
that Qwest performance overall is at or above the standard, according to the statistical framework
inherent in the QP APs. However, for certain measurements and products the failure rates have
been consistently high. In particular MR-5 EEL_OS 1 and UBL_DS1, MR-6 (multiple products),
MR-8 UBL_DS1, OP-4 EEL_OS 1 and UBL_DSI are consistently above five percent, indicating
continued substandard performance. Also, as detailed in Section III.A, failures for specific
measures and their product disaggregations caused the majority of Tier 1 and Tier 2 payments.
These payments were not caused by poor performance for an entire measurement area.
June 30,2009 ~~
The Liberty Consulting Group
Page 45
Final Report Analysis of Qwests
Penormance Assurance Plans
D. Historical Analysis of Key Payment Drivers
The extreme complexity of the PAP payment mechanisms makes it diffcult to summarize the
key drivers of payments without some sort of modeling. Thus, Liberty performed statistical
analyses, including regression modeling, of Tier 1 and Tier 2 payments to determine the major
factors driving the payments in the 1 1 states reviewed during the Study Period. These analyses
adjusted for measure and product, and thus the specific results given are averages over measure
and product.
Libert found four major items that led to increases or declines in payments: i) transaction
volume,30 ii) PID failure rate, iii) Number of CLECs with activity, and iv) severity of failure.
Transaction volume is expected to impact payment amounts because most payments are Per
Occurrence, meaning that the amount paid is based on the number of transactions that fell below
the standard or benchmark. However, with lower volume and the same failure rate, there wil be
fewer such transactions driving payments. PID failures lead directly to payments in most cases,
so PID failure rate should be highly associated with payment amount. The number of CLECs
with activity should not obviously affect payments once total transaction volume has been
considered if the performance for each CLEC is approximately the same. The reason why the
number of CLECs might have an effect could be that there are differences among CLECs or that
a higher number of CLECs leads to more Type I errors, which is the error that occurs when
Qwest is required to make a payment despite service that is at or above standard in general. This
error can happen as a result of the statistical testing that is performed to determine that parity was
met. Finally, severity should have some impact on payment amounts, since the number of
occurrences increase with severity of the failure and because some states have penalties for more
severe failures.
The effects of these four factors were largely consistent across states, product, and measure, with
the exception of Billing. Biling is discussed separately below. For non-Biling measures, the
biggest determinant of payments made was the number of CLECs, while for Billng, the biggest
driver was transaction volume (typically the total dollar amount biled).
Tier 1 Measure Payments
With the exception of Biling-related payments, transaction volume had little or no effect on Tier
1 payment amounts, either at a regional level or by individual state. However, the other three
factors considered (i.e., failure rate, number of CLECs, and severity) did affect payments. For
example, a doubling of the failure rate typically gave rise to an increased payment of $ 1 5 for the
related measure and disaggregation. An additional CLEC meant that, on average, $35 more in
payments were made. A doubling of the severity of the miss typically resulted in an increased
payment of $6. These results varied some by state, but the overall conclusion was the same.
30 Transaction volume is defined as the CLEC denominator used in the measure calculations. For Ordering and
Provisioning, volume is typically number of orders while for Maintenance & Repair, it is typically the number of
troubles reported. For Billing, it is typically total dollar amount billed.
June 30,2009 ~,~
The Liberty Consulting Group
Page 46
Final Report Analysis of Qwests
Performance Assurance Plans
Results for Biling measures were both higher overall and substantively different in that
transaction volume had a very significant and substantial effect on total payments. The
relationship between failure rate and severity remained similar to that in non-Biling measures;
i.e., failure rate had far more effect than severity. On the other hand, the addition of a CLEC was
less important than a doubling of overall failure rate. This differs from the non-Biling measures,
for which change in the number of CLECs was more importnt than either failure rate or
severity.
Payments for Tier 2 Measures
For its regression analysis of Tier 2 measures, Liberty found that it was not possible to measure
the effects by state, because for many products and measures, there were no Tier 2 payments in a
particular state. Thus, Liberty looked at the effects for all 1 1 states in a single model for non-
Biling measures and a second model for Billing measures.31 Because most Tier 2 payments are
for the aggregate CLEC, Liberty did not attempt to consider the effect of adding an individual
CLEC. Liberty found that volume did not have a strong effect on overall Tier 2 payments,
though for non-Biling measures (except Colorado) there was a statistically significant effect.
Similarly, failure rates were generally not statistically significantly related to Tier 2 payment
amounts. Severity had a clear and strong effect on Tier 2 payments.
E. Industry Trends
In order to understand how changes in the telecommunications industry might affect the
continued applicabilty of the PAPs or the relevance of aspects of the PAP structure, Liberty
examined recent industry trends in the Qwest local operating territory. Liberty reviewed the most
current data available on the FCC's website to investigate industry trends.32 One of the most
significant trends is the continuing decline of in-service access lines for traditional ILEC wireline
carriers. Since its nationwide peak of 188.5 milion access lines in service at year-end 2000, the
number of traditional wireline access lines has decreased by 22.1 percent to a year-end 2006 total
of 146.9 millon access lines in service.33 These figures include end-user access lines, access lines
resold to other carriers, and UNE-P lines. They do not include CLEC lines provided over their
own facilties, such as cable company lines. Table II-E- 1 reflects the 10-year trend in ILEC
wire line lines in service.
31 Liberty reviewed ~Colorado separately for non-Biling measures, but there was not suffcient data to consider
Colorado separately for Biling measures. Liberty handled Colorado differently because of the structural change in
the CP AP that resulted in almost no Tier 2 payments after May 2006.32 FCC documents referenced by Libert include "Local Telephone Competition: Status as of December, 2007" and
"Trends in Telephone Service," August 2008 report. Both documents were created by the Industry Analysis and
Technology Division of the Wireline Competition Bureau and are dated September 2008.33 "Trends in Telecommunications Service," August 2008; p. 7-1 and Table 7.1. Unlike other data in these reports
which typically extend to year-end 2007, this data was only provided through year-end 2006. Additionally, this
reduction is not entirely indicative of the actual reduction in access lines during this timeframe and understates the
number of lines that were lost. Prior to 2005 only LECs with at least 10,000 lines in a state were required to report to
the FCC. Beginning with the June 2005 report, all LECs were required to report their Iines-in-service counts
regardless of the number of lines they had. Therefore all access line counts from 2000 to year-end 2004 have been
understated by not including the lines for these smaller carrierS.
June 30,2009 ~,~
The Liberty Consulting Group
Page 47
Final Report Analysis of Qwests
Performance Assurance Plans
Year-end Wireline access lines in service Annual Growth
1997 173,866,799 N/A
1998 179,849,045 3.4
1999 185,002,911 2.9
2000 188,499,486 1.9
2001 185,587,160 -1.5
2002 180,095,333 -3.0
2003 173,147,710 -3.9
2004 165,979,938 -4.1
2005 157,037,503 -5.4
2006 146,848,926 -6.5
Table III-E-l
Trends in ILEC Wireline Access Lines
1997-2006
For total access lines in service the FCC reports include all access lines, including those that are
self-provided by the CLECs. Table II-E-2 reflects the three-year trend from year-end 2005
through year-end 2007 of total ILEC and CLEC in-service access lines for each state
participating in this study. 34
2005 2006 Percent 2007 Percent
State Lines In Service Lines In change 2005-Lines In Service change 2006-
Service 2006 2007
Arina 3,273,829 3,193,105 .2.5 3,104,872 -2.8
Colorado 2,928,554 2,658,781 -9.2 2,451,394 -7.8
Idaho 748,393 740,226 -1.703,396 -5.0
Iowa 1,546,333 1,511,339 -2.3 1,468,712 -2.8
Montana 524,610 517,114 -1.4 509,566 .1.5
Nebraska 918,609 892,697 -2.8 888,691 -0.5
New Mexico 957,838 934,816 -2.4 888,496 -5.0
North Dakota 345,786 337,370 -2.4 324,159 -3.9
South Dakota 415,243 397,441 -4.3 387,330 -2.5
Utah 1,184,901 1,139,235 -3.9 1,055,368 -7.4
Wyoming 285,637 281,587 -1.4 273,091 -3.0
I I-State Total 13,129,733 12,603,71 I -4.0 12,055,075 -4.4
Table III-E-2
Total ILEC and CLEC Access Lines In Servicé5
i I-State View
According to its 2008 Annual Report, Qwests line loss has been more severe than for the
industr as a whole. Qwest indicated that between 2006 and 2007 it lost 1,006,000 access lines, a
7.3 percent decrease in access lines from the previous year, and between 2007 and 2008 it lost an
additional 1,224,000 access lines, a decrease of 9.6 percent from the previous year. However,
Liberty notes that although the nationwide figures shown in Table II-E-l and the state-wide
numbers shown on Table II-E-2 represent a total reduction of access lines in service for the
34 Because of the data issue referenced in the previous footnote, this table does not reflect data prior to 2005.
35 "Local Telephone Competition Status as of December, 2007;" Tables 10 and II.
June 30, 2009 ~,~
The Liberty Consulting Group
Page 48
Final Report Analysis of Qwests
Penormance Assurance Plans
entire industry, some of the access-line loss reported by Qwest in its annual report was from
losses to competitors using their own facilities and not a complete disconnect of the access line.
In its report, the FCC suggests that this reduction in wire line access lines is likely due to some
consumers substituting wireless service for wireline service, and some households eliminating
second lines when they move from dial-up internet service to broadband service.36 This
assumption is supported by the data in the FCC's report which shows that from 2005 through
2007, there was a 13.3 percent reduction in residential lines nationwide whereas the nationwide
reduction in business lines was only 2.1 percent. 37 Residential customers are more likely than
business customers to disconnect their wireline service and replace it with a wireless alternative.
They are also the most likely users of dial-up internet access.
Unlike the wireline industry, the FCC report indicates that the wireless industry is experiencing
robust growth.J8 From year-end 2005 to year-end 2007 the number of wireless subscribers
increased by 45,568,000 subscribers, a 22.4 percent increase in wireless phone users over the
two-year period. Total nationwide wireless subscribers at year end 2007 were 249,235,715. This
means that given the 2007 population estimate of 302 milion people in the United States, cell
phone penetration has reached 82.8 percent of the total population.39 Table II-E-3 reflects the
three-year trend in wireless growth in the II-states, providing the number of wireless subscribers
in each state at the end of each year.
State 2005 2006 Subscribers Percent change 2007 Subscribers Percent change
subscribers 2005-2006 2006-2007
Arizona 3,844,357 4,405,032 14.6 4,799,648 9.0
Colorado 3,246,994 3,608,209 11.3,967,902 10.0
Idaho 834,219 972,825 16.6 1,078,387 10.9
Iowa 1,8 I 1,400 2,009,826 11.0 2,165,772 7.8
Montana 525,003 619,620 18.0 693,507 11.9
Nebraska 1,160,062 1,272,067 9.7 1,387,022 9.0
New 1,170,186 1,333,210 13.9 1,489,120 11.7
Mexico
North 431,675 472,799 9.5 513,238 8.6
Dakota
South 481,404 547,812 13.8 596,562 8.9
Dakota
Utah 1,529,501 1,774,755 16.0 1,970,501 11.0
Wyoming 342,008 387,164 13.2 441,161 14.0
Total 15,376,809 17,403,319 13.2 19,102,820 9.8
Table III-E-3
Total Wireless Subscribers
ll-State View
36 "Trends in Telecommunications Service," August 2008; p. 7-1.
37 "Local Telephone Competition Status as of December, 2007;" Table 2.
38 "Local Telephone Competition Status as of December, 2007;" Table 14.39 Population estimate obtained from the Population Reference Bureau at www.prb.org.
June 30, 2009 ~,~
The Liberty Consulting Group
Page 49
Final Report Analysis of Qwests
Performance Assurance Plans
Nationwide, the CLECs experienced a one percent decline in their market share for access lines
in service between 2005 and 2007, going from a market share of 19.1 percent of the total access
lines in service to a share of 18.1 percent. During this period the CLECs' portion of the total
access-line market dropped from 31.4 millon access lines in service at year-end 2005 to 28.7
access lines in service at year-end 2007.40 As shown in Table II-E-4, this loss was driven mainly
by service provided over UNEs, a category which the FCC's TRO and TRRO orders
significantly changed by delisting some products, including UNE-P.
ccess lDes)y ervice l ype
Year Total Resold4~Percent UNE Percent CLEC Percent
CLEC Lines (000)Resale Lines UNE Owned Facilties
Lines (000)Facilties Based
(000)(000)
2005 31,388 6,704 21.4 14,521 46.3 10,163 32.4
2006 28,626 5,819 20.3 11,663 40.7 11,144 38.9
2007 28,717 6,41 I 22.3 10,582 36.8 11,724 40.8
Table III-E-4CLECA L' b S T 41
Of the CLECs providing service using their own facilties, the FCC's report indicates that there
has been significant growth in cable companies providing telephone service over their cable
facilities. According to the FCC data, access lines over cable facilties between 2005 and 2007
grew by 64.7 percent going from 5.1 milion lines at year-end 2005 to 8.4 milion lines at year-
end 2007. By year-end 2007, cable facilties accounted for 29.2 percent of all CLEC Iines.43
Table II-E-5 shows the year-end CLEC lines in service for the 1 I-states for 2005 through 2007.
According to the FCC report, these figures reflect all CLECs doing business in each of the states
and include access lines for cable and other companies that provide their own access-line
facilties to the end users. These figures do not, however, include mobile wireless users. As
shown in Table II-E-5, there are some very significant variations in the CLEC line growth
trends with most states showing growth in overall CLEC access lines, while states such as
Colorado and Utah have lost CLEC lines over the three-year period.
Libert notes that the state-specific access line data reported in the FCC's report are very
different from the in-service quantities that Libert derived from the Qwest reported access lines
in service using the denominator of the MR-8 "Trouble Rate" measure. For the total CLEC
access lines shown on Table II-E-5, in all cases the FCC's line counts are substantially higher
than the total CLEC lines reported by Qwest.44 A number of factors contribute to this difference.
These factors include line counts in the FCC report from companies that self-provide access line
facilities and thus wil not be reflected in Qwests reported numbers,C LECs doing business with
40 "Local Telephone Competition Status as of December, 2007;" Table i.
41 "Local Telephone Competition Status as of December, 2007;" Table 3.
42 The FCCs report defines resold lines as including lines that the CLEC provides by using special access lines or
other facilities that it obtains from unaffliated ILECs or CLECs as tariffed services or under commercial agreements
43 "Local Telephone Competition Status as of December, 2007;" Table 5.
44 Total CLEC lines reported by Qwest can be found in Table II-B-7 in section II.B of this report.
June 30,2009 ~~
The Liberty Consulting Group
Page 50
Final Report Analysis of Qwests
Performance Assurance Plans
Qwest under commercial agreements that are shown as resold lines in the FCC report but are not
included in Qwests reported MR-8 figures, use of access line equivalents for reporting lines in
the FCC report, and possible reporting errors. To the best of Libert's knowledge there are no
audits of the data reported by the telecommunications carriers to the FCC for the creation of its
report.
2005 Total 2006 Total Percent 2007 Total Percent
State CLEC Lines CLEC Lines change 2005-CLEC Lines change 2006-
2006 2007
Arizona 978,582 1,017,866 4.0 1,070,963 5.2
Colorado 590,821 452,270 -23.5 394,574 -12.8
Idaho 75,951 76,063 0.2 74,962 -1.
Iowa 221,758 238,161 7.4 268,858 12.9
Montana 52,014 71,746 37.9 93,177 29.9
Nebraska 237,496 248,839 4.8 265,020 6.5
New Mexico 65,123 75,169 15.4 72,932 -3.0
North Dakota 66,830 70,031 4.8 70,767 i.
South Dakota 136,073 I 19,025 -12.5 119,051 0.0
Utah 260,478 244,772 -6.0 211,583 -13.6
Wyoming 34,004 43,552 28.1 48,391 i i.
Total 2,719,130 2,657,494 -2.3 2,690,278 1.2
Table III-E-5
CLEC Access Lines
1l-State45
The FCC's August 2008 "Trends in Telephone Service" report, which provides data through
June of2007, shows that the availabilty of high-speed service lines has more than doubled in the
two-year period from June 2005 through June 2007.46 According to the data reported by the FCC
in June 2005 there were 42,517,810 high-speed lines available to end users and by June of 2007
the number of these available lines grew to 100,921,647, an increase of 137 percent over the two
year period.47 Table II-E-6 provides information on the number of lines available by technology
type during this period. As can be seen from this table, during this period there was explosive
growth in mobile wireless high-speed data access which overtook both cable modem and ADSL
as the most prevalent technology available to subscribers.
4S "Local Telephone Competition Status as of December, 2007;" Table 9.
46 For FCC reporting purposes, high-speed service "lines" are considered both wired and wireless connections to end
users that are faster than 200kbps in at least one direction. For FCC reporting puroses, high-speed service "lines"
are defined as connections, both wired and wireless, to end users that are faster than 200kbps in at least one
direction.47 For ILECs and cable system operators, reporting is based on the availability of high-speed service to their
respective end users whether or not the household actually subscribes to the service.
June 30, 2009 ~~
The Liberty Consulting Group
Page 51
Final Report Analysis of Qwests
Performance Assurance Plans
sy ec no og;:ype
Percent Percent
Technology June 2005 (000)June 2006 (000)change 2005-June 2007 (000)change 2006-
2006 2007
ADSL~~16,316 22,584 38.4 27,516 21.8
SDSL~u 412 337 -18.2 320 -5.0
Traditional 487 61 I 25.5 709 16.0Wireline
Cable Modem 24,017 29,174 21.5 34,409 17.9
Fiber 316 686 I 17.5 1,403 104.5
Satellte 377 495 31.669 35.2
Fixed Wireless 209 361 72.7 586 62.3
Mobile Wireless 380 II,OI7 2799.2 35,305 220.5
Power Lines and 5 5 0.00 5 0.00Other
Table III-E-6
High-Speed Lines Over 200kbps in One Direction
B T hiT 48
Table II-E-7 reflects the growth in high-speed access lines in the 1 I-states, while the type of
technology used to provide these lines is provided in Table II-E-8.
Percent Percent
State June 2005 (000)June 2006 (000)change 2005-June 2007 (000)change 2006-
2006 2007
Ariona 809,819 1,392,71 I 720 2,192,644 57.4
Colorado 688,189 1,165,853 69.4 1,827,860 56.8
Idaho 149,023 202,926 36.2 483,049 138.0
Iowa 325,710 446,187 37.0 826,096 85.2
Montana 90,583 139,946 54.5 346,230 147.4
Nebraska 253,968 355,013 39.8 537,693 51.
New Mexico 174,534 252,361 44.6 544,706 115.8
North Dakota 86,274 108,476 25.7 144,994 33.7
South Dakota 112,506 138,621 23.2 164,627 18.8
Utah 259,150 471,137 81.8 818,665 73.8
Wyomin2 55,905 70,574 26.2 205,711 191.5
Total 3,005,661 4,743,805 57.8 8,092,275 70.6
Table III-E-7
High-Speed Lines Over 200kbps in One Direction
ll-StatesSI
48 "Trends in Telecommunications Service," August 2008; Table 2. I.
49 Asymmetric Digital Subscriber Line
so Symmetrical Digital Subscriber Line
51 "Trends in Telecommunications Service," August 2008; Table 2.7.
June 30,2009 ~,~
The Liberty Consulting Group
Page 52
Final Report Analysis of Qwests
Performance Assurance Plans
State ADSL SDSL Tra ~ition al Cable Fiber Fixed TotaiS3WirelineModemWireless
Arizona 405,724 1,491 12,630 850,307 1,996 17,122 2,192,644
Colorado 529,504 2,810 16,060 560,557 1,285 21,864 1,827,860
Idaho 129,188 340 1,507 116,273 635 34,905 483,049
Iowa 270,101 4,244 3,151 267,712 5,633 14,802 826,096
Montana 95,790 2,549 876 74,246 286 7,653 346,230
Nebraska 124,126 3,135 1,081 238,019 527 10,866 537,693
New 179,856 401 1,867 117,336 424 2,518 544,706Mexico
North 51,096 3,288 382 76,353 5,508 4,873 144,994Dakota
South 45,772 3,895 252 100,903 2,724 4,878 164,627Dakota
Uta 249,683 5,454 N/A 3,947 1,907 21,252 818,665
Wyoming 49,933 1,657 190 N/A 294 3,445 205,71 I
Total 2,130,773 29,264 37,996 2,405,653 21,219 144,178 8,092,275
Table III-E-8
High-Speed Lines by Technology Type
June 2007 - ll-States52
In summary, based on review of the FCC's reports, as well as the data derived from Qwests PID
measurement reports and from Qwests annual report, Libert notes the following trends:
. Subscribers to telephone service using traditional wireline facilties (i.e., twisted
pair) have been continually declining since 2000.
. Wireless services are experiencing considerable growth and are contributing to
the loss of wire line services as customers give up their traditional phone service in
favor of a wireless option.
. Facilities-based CLECs such as cable companies are growing while CLEC
services provided via Resale and UNE are generally on the decline.
54
. Though CLECs are becoming less dependent on the Qwest for Resale and, to a
lesser extent, UNE products, many CLECs are stil dependent on these services
and products to serve their end users.
. Access to high-speed data lines has grown significantly over the period of 2005-
2007, especially through mobile wireless facilities. Cable facilties and ADSL
service are also widely used technologies for access to broadband services. Many
CLECs that provide a DSL service alternative to Qwest are dependent on Qwests
wholesale products, such as UN-L to serve their customers.
52 "Trends in Telecommunications Service," August 2008; Table 2.6.
53 Total exceeds the sum of the pars because, although specific data for Satellte, Mobile Wireless and Power Lines
are not available at the state level, these technologies are included in the total line count.54 As shown on Tables II-B-9 and Il-B-IO, found in section Il-B of this report, between 2004 and 2008 Qwest
Resale lines have shown a continuous decline in CLEC lines purchased from Qwest; however, the trend in UNE
lines varies by state with some states showing stable to slightly growing UNE line counts.
June 30,2009 ~~
The Liberty Consulting Group
Page 53
Final Report Analysis of Qwests
Performance Assurance Plans
Trends such as the shift away from wire line services and the decline in the use of Resale and, to
a lesser extent, UNEs by the CLECs suggest the need for some revisions to the PAP. However,
because many CLECs continue to depend on Qwest to serve their customers through Resale and
UN services and products, the PAP remains a critical tool to ensure parity of service
performance by Qwest.
F. Summary and Conclusions
From the analysis of the historical data on PAP payments, Liberty determined that both Tier 1
and Tier 2 payments have decreased overall across the Qwest operating region and in each of the
participating states during the Study Period (Januar 2004 through October 2008). The principal
reasons for this reduction applicable across the region are: i) a decline in the number of active
CLECs and ii) improvement in Qwest s performance. Liberty found that the Qwest failure rate
(number of times Qwest has been out of conformance with the standards as determined by the
PAP tests) has generally decreased in all measure domains across this time period. Although the
general trend in payments was evident in all the participating states, there was some variation in
the detail trends from state to state. For example, there was a dramatic decrease in Tier 2
payments in Colorado, after implementation of changes in 2006 after the CPAP Three-Year
Review. Only a few PID measures generate the majority of the PAP payments; 15 measures have
been the source of 97 percent of the payments, with the Maintenance and Repair (MR) domain
dominating the payments, particularly recently. Only three measures have been responsible for
approximately 60 percent of the payments: MR-8 (Trouble Rate), BI-3A (Biling Accuracy -
Adjustments for Errors), and OP-4 (Installation Intervals).
Liberty found that CLEC order volumes and lines in service reported in the PID measures have
also declined markedly during the Study Period. A major contributor to this decline was the
adoption by the FCC of the TRO and TRRO orders, which delisted a number ofUNs including
those key to UNE-P (particularly unbundled switching). Lines in service dropped dramatically
for Resale and to a much lesser extent for UNE products (with the exception of those delisted by
the TRO and TRRO). Despite the decline in Resale and UN orders, orders for number porting
have remained high, indicating the increasing importance of facilties-based competitors. There
are some state-level variations to these regional trends, with a large share of the market in Resale
in some states, for example.
Liberty used data reported by the FCC to examine trends in the telecommunications industr as a
whole in the Qwest operating territory, including trends in competition. Such data provide
information on the full set of competitors and competitive services, not just those captured in the
Qwest PID measures. This data reveals that there has been an increasing decline in subscribers to
services using traditional wireline facilties, but growth in the wireless services. Fully facilties-
based competitors, such as cable companies, have a growing subscriber base, while services
provided via Resale and, to a lesser extent, UNE are in decline. Nevertheless, there is stil a
significant number of CLECs with a considerable subscriber base that stil depend on Qwests
services and products to serve their end users. There has also been a considerable growth in
June 30,2009 ~~
The Liberty Consulting Group
Page 54
Final Report Analysis of Qwests
Performance Assurance Plans
broadband services, and many CLECs use Qwests UN-L to provide these services. In addition,
Liberty understands that a number of CLECs rely on wholesale services, such as QLSP, that
Qwest provides through commercial agreements rather than as UNEs, particularly after the
FCC's TRO and TRRO orders eliminated UN-P and other network elements from the list of
UNEs. However, Qwest does not include these services in its PAP payment calculations and PID
reports, and Liberty has no data to quantifY the volume of products and services provided
through these means.
Based on this analysis, Liberty considered the continuing effectiveness, value, and usefulness of
the PAP. In making this evaluation, Liberty considered how importnt the PAPs are in
continuing to maintain competition. In particular, this involved considering:
. The number of active CLECs with a significant total subscriber base and
dependent on Qwests wholesale products and services to serve their end users
. The level of Qwests penalty payments
· The extent of Qwest s performance that is out of compliance with standards
. The burden on Qwest of maintaining the PAPs and whether this burden outweighs
the advantage of protecting competitors.
Although recent changes in the industry have resulted in significant reductions in volumes of a
number of services, there are stil a number of CLECs with significant a subscriber base and
transaction volumes. These CLECs depend on Qwests wholesale services and products in
various ways to provide service. There are few realistic alternatives available to Qwests
wholesale products and services for essential components these CLECs use, such as UN-L.
Although these CLECs' share of the market has declined overall since 2004, they continue to
provide significant competition for Qwest, paricularly in such important pars of the market as
broadband and business services, as shown in Tables II-B-2 and II-B-5. These tables show the
products that have experienced growth in CLEC orders and lines in service growth between 2004
and 2008. The CLEC growth products are those that a CLEC would typically be ordering to
service either a business customer (e.g., DS-l Loop) or a broadband customer (e.g., xDSL
Loops, 2-Wire Non-Loaded Loop).
Although there is evidence that Qwests wholesale performance has been improving, the PAPs
continue to provide incentives to help ensure that Qwest s performance level does not
deteriorate. Despite the decline in PAP penalty payments, the level of payments is stil
significant. In addition, Liberty is aware that there have been recent cases, in Hawaii and
northern New England, where the inability of an incumbent local exchange company to provide
reliable and high quality wholesale services to CLECs has significantly affected the abilty of
those CLECs to serve their own end-user customers. Although the causes of this poor wholesale
performance was related to a change of ownership and operation of the local exchange
businesses in these cases, and thus they are unrelated to the current situation in the Qwest
territory, the examples do demonstrate the harm to competitors that can result from poor
wholesale performance by an incumbent. The Qwest PAPs help assure that the correct incentives
are in place to help prevent such conditions occurring.
June 30, 2009 ~~
The Liberty Consultng Group
Page 55
Final Report Analysis of Qwests
Penormance Assurance Plans
Because Qwests participation in this study was limited to providing data on PAP payments and
PID results and answering questions about them, Liberty has no data on the costs and other
burdens on Qwest of maintaining the PAPs. However, because the infrastructure to maintain the
PAPs has been in place for some time, the principal cost to Qwest is likely to be in processing
the underlying data and running the PAP systems. Although this certinly imposes a cost on
Qwest, it appears to be acceptable given the significant CLEC community relying on Qwests
wholesale services. Liberty recognizes that Qwest has significant competitors, such as the
wireless and cable providers, that do not rely on Qwests wholesale services or only rely on them
to a limited extent. These competitors appear to be gaining in market share. Nevertheless,
Libert does not believe this is justification for abandoning the PAP and thereby potentially
placing those competitors relying on Qwests wholesale services at a potential disadvantage.
Although Liberty concludes that the PAPs continue to serve a useful purpose and should be
maintained, the industry trends do support the need for some continued fine tuning of the PAP
structures. Trends such as the shift away from wire line services and the decline in the use of
Resale and, to a lesser extent, UNEs by the CLECs suggest the need for some revisions to the
PAPs to ensure that they are focused on those products, services, and transactions that are stil
importnt for the CLEC community. Section iv examines which changes would best serve this
purpose.
June 30, 2009 ~~
The Liberty Consulting Group
Page 56
Final Report Analysis of Qwests
Penormance Assurance Plans
iv. Proposals
Libert considered a number of changes to improve the PAPs by eliminating unnecessary aspects
and increasing their focus on the tyes of service, products, and transactions that continue to be
importt in maintaining a healthy CLEC community in the Qwest territory. Liberty also looked
at the underlying PID measures to see what changes might be appropriate to achieve the same
ends. In evaluating these various options, Liberty used the following considerations:
. Whether changes in the marketplace have made elements of the PAPs obsolete
· Whether paricular types of transactions are no longer relevant
. Whether the volumes of transactions for sub-measures and products are too small
to warrant their continued inclusion in the PAPs
. Whether the PAPs and PID can be simplified
· Whether there are any biases and distortions in the PAPs that need to be corrected
· Whether there are important transactions types that are currently not monitored in
the PAPs and PID
. Whether the effort to secure support for and cost of making the changes
outweighs the advantage of making them.
Libert notes that it is diffcult to address all of these considerations simultaneously. For
example, the PAPs could be significantly simplified by decreasing the number of sub-
measurements and product disaggregations. However, doing so would cause the measurements
to lose resolution and potentially introduce biases and distortions.
In identifYing possible changes, Liberty used both input from stakeholders and the results of the
analysis outlined in Section II. The following sections provide a discussion of the input and
analysis Liberty used to develop the PAP change proposals.
A. Stakeholder Input
At the beginning of this study, Liberty contacted the Commission Staff members the
Collaborative Committee and requested input on any concerns or issues they have with the
PAPs. The responses indicated no specific concerns. Liberty also drafted a stakeholder
questionnaire, and shared it with the Staffs for edits and other input. Liberty developed a list of
CLECs based on input from the Collaborative Committee, including email and U.S. mail
addresses. Where possible, Liberty sent the questionnaire to the recipients by email; when no
email addresses were available, Libert sent the questionnaire to the CLECs by U.S. maiL. In
some cases, Liberty had multiple contacts for the same company or for different subsidiaries of a
company. In those cases, Liberty requested that the recipients coordinate responses so as to
obtain a single response for a company. Liberty sent the questionnaire to 92 different CLEC
recipients. Although Qwest had indicated it did not plan to paricipate in this study, Liberty also
sent a questionnaire to Qwest. Qwest did not respond to the questionnaire.
June 30,2009 ~~
The Liberty Consulting Group
Page 57
Final Report Analysis of Qwests
Performance Assurance Plans
Libert's questionnaire contained the following questions:
1. In which states of the i 4-state Qwest operating territory do you do business? For
which affliates and legal entities and under what names do you do business in
those states?
2. What Qwest wholesale services (e.g., resold services, specific unbundled network
elements, Local Number Porting) do you currently use? What Qwest wholesale
services have you used in the past but no longer use? For which states in the
Qwest operating region do you use these services?
3. Have you "opted in" to the QPAP (or CPAP) for any of the states in the Qwest
operating territory in which you do business? That is, have you adopted the QP AP
or CP AP as par of your interconnection agreement with Qwest? If so, in which
states?
4. If you are not currently "opted in" to the QPAP or CPAP, have you done so in the
past? If so, in which states and for what time periods?
5. If you have "opted in" to a QPAP (CPAP), have you ever received "Tier 1"
payments from Qwest?
6. If you have never "opted in" to a QP AP (CP AP), what experience with or
knowledge do you have of these plans?
7. Please specify which QPAPs (CPAP) components (e.g., the sizes ofthe payments,
how payments are assessed, focus on individual CLEC vs. aggregate CLEC
results) you believe are working well and those you believe are not working well.
If relevant, please also include in this response your opinions about the specific
PID measures, products tracked, standards (benchmark and parity), and reporting
levels (e.g., state, MSA, Zone Type) in the measures.
8. If there were to be changes in the QPAPs (CPAP) in the future, which current
components or PID measures (including products tracked, standards, and
reporting levels) do you believe are necessary to preserve and/or are particularly
important for your company? To the extent that this response might vary by state,
please indicate how.
9. What QPAP (CPAP) components or PID measures (including products tracked,
standards, and reporting levels) do you believe are unnecessary and can be
dropped? To the extent that this response might vary by state, please indicate
how.
10. What QPAP (CPAP) components or PID measures (including products tracked,
standards, and reporting levels) do you believe should be added? Would you
recommend changing any PID measures that are now diagnostic (without
standards) to ones with standards and including them in the QPAPs (CPAP), or
vice versa? To the extent that this response might vary by state, please indicate
how.
11. Please specify the interface used by your company (i.e., IMA-GUI or IMA-XML)
for submitting LSRs and ASRs to Qwest. If you use both interfaces, please
provide an estimate of your percent usage for each (e.g., GUI - 35%, XML -
65%).
June 30, 2009 ~~
The Liberty Consulting Group
Page 58
Final Report Analysis of Qwesl's
Performance Assurance Plans
12. Please provide any other comments and input that you believe Liberty and the
Commission Staffs should have in conducting this review and analysis.
Liberty received 14 replies, including responses from
. 360networks (USA), Inc.
. American Fiber Network, Inc.
. Blackfoot Communications, Inc. (f/a Montana Wireless, Inc.)
· Bullseye Telecom, Inc.
. Cbeyond Communications, LLC
. DIECA Communications, Inc. d/b/a Covad Communications Company
. Integra Telecom, Inc.
55
· Level 3 Communications, LLC
. LISCO f/a L TDS
. McLeodUSA Telecommunications Services, Inc. d//a PAETEC Business
Services
. Unity Business Networks
. Talk America d//a Cavalier Telephone
. TW Telecom LLC
. XO Communications Services, Inc.
One of the responding CLECs, Cavalier, indicated that it does not do business in Qwests
territory. Three other CLECs (Level 3, LISCO, and Unity Business Networks) responded that
they had limited experience with the PAPs or were unable to provide much input about the PAPs
for other reasons. The other eleven CLECs provided a number of comments and suggestions.
Generally, the CLECs indicated satisfaction with the PAPs and PID measures and believed they
were important in helping to maintain telecommunications competition in the Qwest region.
Integra pointed out that traditionally AT&T and MCI had the most resources to advocate for the
CLECs' position. Now that they have merged with Regional Bell Operating Companies, their
advocacy for CLECs has significantly diminished, which increases the need for a strong PAP to
protect the CLECS.
The CLECs generally believed that no PAP components or PID measures, including product
disaggregations, should be dropped because they were no longer necessary. However, several
CLECs made specific suggestions for additions to the PAPs and PID measures, as follows.
UNE Facility Assignment
Integra, PAETEC, and Blackfoot Communications expressed concerns about the abilty to obtain
DSL-capable loops that support their bandwidth requirements because Qwest does not provision
available pairs to meet their requirements, substituting lower capabilty pairs instead. Integra and
55 Integra subsidiaries and affliated operating in the Qwest operating territory include Electric Lightwave, Eschelon,
Mountain Telecom, InfoTel Communications, One Eighty Communications, and Advanced TelCom,
June 30, 2009 ~,~
The Liberty Consulting Group
Page 59
Final Report Analysis of Qwests
Penormance Assurance Plans
PAETEC advocate the development of a PID measure to ensure the appropriate and
nondiscriminatory assignment of facilties. Blackfoot Communications indicated interest in
supporting this proposaL.
While Liberty believes the CLECs have raised a valid concern, it is not clear that it can be
addressed within the context of Liberty's analysis. It appears that a commitment from Qwest to
provide a UNE-L product offering with the bandwidth requirements to meet the CLECs' needs is
the most appropriate way to address the concern. Whether Qwest is required to provide such a
product and have its performance measured through the PAPs is a question that goes beyond the
scope of Liberty's study.
Expedited Ordering
Integra observed that there currently are no requirements in the PAPs that address the process for
expedited ordering. P AETEC joins Integra in the recommendation to develop such requirements.
Libert believes this is a relevant issue to address through PID changes, and offers a
recommendation for it in Section IV.E.
Chronic Troubles
Integra noted that there is no PID measure that measures chronic troubles, and indicated that
such troubles have serious end-user customer impacts. Integra defines these as cases where there
are more than two troubles for a given customer in a specified timeframe, where that timeframe
should extend beyond 30 days because the situation can occur over extended periods of time.
PAETEC and Cbeyond support Integra's recommendation to develop a measure of chronic
troubles and incorporate it in the PAPs. Liberty believes this is a relevant issue to address
through PID changes, and offers a recommendation for it in Section IV.E.
MR-6 (Mean Time to Restore) Modifcations
Integra indicated that the 2007 Stipulation included a modification of MR-6 to remove No
Trouble Found (NTF) and Test Okay (TOK) trouble reports when the ticket's duration is one
hour or less, because there is a greater percentage of NTF and TOK reports for retail than
wholesale circuits. Typically, such trouble reports are quickly resolved, and removing short
duration ones helped to resolve a bias against Qwest. However, Integra claims that, in retrospect,
this change has introduced another bias, because it fails to tae into account the fact CLECs
provide test results to Qwest before Qwest begins to repair a CLEC facilty, unlike the retail
case. PAETEC and Cbeyond support Integra's argument that this bias needs to be corrected.
It is not clear to Liberty that the bias the CLECs describe actually exists. To the best of Liberty's
knowledge, Qwest performs a mechanized loop test (ML T) on its retail lines before opening a
trouble report on a customer's line. The MLT results that the Qwest technician receives should
be the equivalent of the test results that the CLEC provides to Qwest. However, if the CLECs
have evidence that such a bias exists, the best way to resolve the issue would be through
collaborative discussions with Qwest.
Retirement of EDI and Replacement by XML
June 30, 2009 ~,~
The Liberty Consulting Group
Page 60
Final Report Analysis of Qwests
Performance Assurance Plans
Qwest has retired its Electronic Data Interchange (EDI) interface, which it provided as means for
CLECs to submit pre-order and order transactions, and has introduced an Extensible Mark-up
Language (XML) interface as a replacement for the retired EDI interface. Integra notes that
Colorado, Utah, and Washington have introduced changes to the PID and PAPs to account for
these changes. The changes are incorporated into PID Version 9.1. PAETEC supports Integra's
recommendation that these changes should be made throughout the Qwest operating territory.
Liberty concurs with this recommendation and addresses it Section IV.C.
Coordinated Appointments
Blackfoot Communications noted that there is a large charge associated with coordinated
installations and indicated that Qwest has not been reliably meeting these appointments. This
causes Blackfoot to be charged more for a service which is no better than the basic installation
service. Blackfoot recommends the development of a PID measure to monitor this issue. Liberty
believes this is a relevant issue to address through PID changes, and offers a recommendation for
it in Section IV.E.
Increasing Penalties Jor Chronically Failing Domains
Cbeyond suggested that penalties should increase in cases where Qwest is consistently making
PAP payments in a given category. This is an interesting suggestion. However, Liberty believes
that the mechanisms for penalty escalation for continuing non-conformance are adequate to
address the concern.
Better Ability to Understand Payments
360networks and American Fiber Network noted difficulty in understanding the PAP payments
they received; in particular, they find it diffcult to tie the payment to specific transaction failures
and are unable to receive useful information from Qwest in understanding the payments.
360networks indicates that it would be helpful for Qwest in its reports and the documents that
accompany the payments to provide information that enables 360networks to track the payments
with particular transactions and the particular service standards with which Qwest failed to
comply.
While Liberty agrees that the PAPs are complex, both 360networks and American Fiber Network
acknowledged that their experience with the PAPs has been limited. Because CLECs professing
more experience with the PAPs did not raise the same concerns, Liberty believes more
information would need to be gathered about from the CLEC community as a whole before the
need to address this concern could be assessed. Such additional investigation is beyond the scope
of this study, but might be appropriate to raise as a point of discussion in industry collaborative
sessions.
June 30, 2009 ~~
The Liberty Consulting Group
Page 61
Final Report Analysis of Qwests
Penormance Assurance Plans
B. Mechanisms to Address Low Volume Issues
As discussed in Section II.C, Liberty's analysis has found that a large number of the tests
conducted to determine penalty payments are based on small numbers of CLEC transactions.
Small sample sizes can introduce biases into the results. The basic PAP design anticipated that
small sub-measure volumes would occur to some extent, and all the PAPs introduced special
testing procedures to use in low-volume situations in an attempt to moderate these biases. These
include:
. Use of permutation tests for cases where the number oftransactions is less than 30
. Restrictions on Tier 2 tests to cases where the volumes are 10 or greater
. Adjustments in the critical Z-values, depending on CLEC volume, for both modified
Z-tests and permutation tests
. Variance factor adjustments for Tier lA measure tests in the CPAP.
For parity measures, small sample sizes create the possibility of failure even when performance
is at parity, because both the standard and the permutation test used in these circumstances are
typically designed to fail at least five percent of the time due to chance variation in performance.
The basic testing rules in the PAP were intended for tyical CLEC volumes of around 140, not
the low volumes that occurred during the Study Period. At the volume of 140, the chance of
generating payments for a process that was in parity (called Type I error chance) is about equal
to the chance of generating no payments for a process that is substantially out of parity (called
Type II error chance). In 2008, tyical CLEC volume is well below ten for the measurement
disaggregations as they currently exist, thus producing a high chance of Type I error for parity
measures and a high chance of Type II error for benchmark measures. This higher Type II error
is due to the One Allowable Miss Rule, which is explained below. Although the small sample
rules listed above were designed to increase the sensitivity to performance and reduce the Type I
and Type II error chances when CLEC volumes are small, they are imperfect. Therefore, the
probabilty of bias is significantly higher when these rules are invoked as frequently as they have
been during the Study Period.
Some of the changes introduced into the PAPs through the CP AP Three-Year Review and 2007
Stipulation recommendations have helped to alleviate aspects of this problem. Of particular note
are the One Allowable Miss Rule and the elimination of low-volume product sub-measures. The
One Allowable Miss Rule apples to sub-measure with a benchmark standard (or for non-interval
parity sub-measures),56 which is sometimes the case for Line Splitting and UBL-xDSLI Loops in
the OP measures. This rule implies, for example, that consistent performance of 90 percent (e.g.,
9 out of 10) for a benchmark measure with a standard of 95 percent wil not produce any
payments, because there wil be only a single miss.
56 That is, those measuring percent, ratios, or proportions
June 30, 2009 ~~
The Liberty Consulting Group
Page 62
Final Report Analysis of Qwests
Penormance Assurance Plans
Despite these changes, Liberty observed that there continue to be very frequent low-volume
situations that are not accounted for in these changes. To help alleviate this problem with small
volumes, Liberty proposes to additional changes to the PAP:
. Elimination of measures from payment calculations that have relatively low CLEC
volumes. This is described in Section IV .C.
. Additional elimination of low-volume product disaggregations of measures. This
proposal is described in Section IV.D.
. A change to the PAP's testing rules to aggregate low-volume transactions.
Libert considered three main approaches to achieve the aggregation of low-volume
transactions: i) aggregation across products, ii) aggregation across months, and iii) aggregation
across CLECs. After some analysis, Liberty rejected the first approach. Aggregation across
products increases the sample size. However, in order to retain reasonable retail analogues for
the aggregated products, it would be necessary to introduce a complex process of weighting both
the wholesale and retail transactions. In addition, some measures have product disaggregations
with retail analogues and others with benchmarks. The process of combining retail analogues and
benchmarks would also be complex. In addition, even with different standards for the different
products, such a combination would tend to mask poor performance and could mean that
sustained poor performance for one product could be compensated for by sustained good
performance for another product. Therefore, Liberty rejected the approach of combining product
disaggregations, and believes that the only way to make this approach workable without
unnecessary additional complexity would be for the industr to agree to new retail analogues for
the aggregated products.
Liberty recognizes there can also be concerns with aggregation across months and CLECs. In
particular, aggregation across time would delay the application of payments, and aggregation
across CLECs could potentially dilute the impact of an individual CLEC's transactions.
However, Libert notes that there already exists a similar type of CLEC aggregation in the Low
Volume, Developing Markets mechanism 57 in the PAPs. Liberty used this as a model for the
CLEC aggregation and to combine this with the monthly aggregation, if insufficient volumes are
attained.
Liberty's low-volume aggregation proposal would work as follows:
1. Aggregate transactions for all CLECs that have fewer than ten transactions in a
month for any given sub-measure disaggregation (e.g., OP-3A Resold Business
Service) before determining whether a payment is due.
2. If the outcome of this CLEC aggregation equals or exceeds ten transactions, use
the aggregate result for these CLECs to calculate whether penalty payments are
required.
57 The Low Volume Developing Markets mechanism is described in Section 10.0 of all the QPAPs, but is not
included in the CPAP.
June 30,2009 ~~
The Liberty Consulting Group
Page 63
Final Report Analysis of Qwests
Penormance Assurance Plans
3. Should the calculation determine that Qwest was out of compliance with the
standard for the sub-measure, payments wil be made to the aggregated CLECs
based on each CLEC's relative share ofthe total number of misses.
4. If the aggregate total does not exceed ten transactions, then carry forward the
aggregate result to the following two months until either the threshold of ten
aggregate transactions is met or three months of results data have been used in an
attempt to meet the minimum volume threshold.
58
5. When either of these criteria has been met, the process starts again the following
month for that sub-measure.
59
Any CLEC with ten or more transactions on the same sub-measure would not have its results
aggregated with the other CLECs; it wil be treated as a standalone CLEC consistent with the
current process. By way of a hypothetical example, assume that a state has four CLECs with
transaction volume for the OP-3A, Resale Business product. CLEC A has three transactions,
CLEC B has four transactions, CLEC C has four transactions and CLEC 0 has 12 transactions.
Under this proposal, CLECs A, B and C would have their respective results combined together.
This would bring the aggregate total to 1 1 transactions, which would be used for penalty
payment calculations assuming there were some failures. Because it has 12 transactions on its
own, CLEC D's results would not be combined with the others.
Liberty examined one other approach to minimize low volume situations: combining MSA and
Zone disaggregations. In order to assess the applicabiliy of this approach, Liberty considered
whether the MSA and Zone designations, which apply to the measures OP-3, OP-4, OP-6, MR-3,
MR-4, MR-5, MR-6, MR-7, and MR-9, make a difference in performance. Liberty found that
although these designations did not always make a difference in performance results, there were
frequently statistically significant differences, and thus these designations were stil important in
determining and comparing performance overalL.
Table IV-B- i below shows by measure the calculated p-values and a determination of whether
there exists a statistically significant difference between MSA and Zone disaggregation results.
A p-value of less than .05 indicates that there does exist a statistically significant difference in
the results for the measure. While these differences do not appear to exist for all measures,
Libert does not recommend combining Zone and MSA for any measures, because that would
make comparisons across measures diffcult.
58 Another issue that needs to be addressed is how to account for the consecutive month payment escalation when
such aggregation across months is required. There are multiple ways to address this issue, but Liberty recommends
treating aggregation across months as if it were a single month for payment escalation purposes. For example,
suppose a CLEC experienced a failure requiring payment in April, either because there was a single failure for that
CLEC in April or because April was the last month of an aggregation across months in which that CLEC
participated. Then, if the CLEC participates in an aggregation across May, June, and July that results in a failure,
that failure would be treated as the next consecutive month of failure for the purpose of determining the payment
leveL.59 This is superior to having a rollng three month window, because a single bad month wil not affect payments
more than one time.
June 30, 2009 ~,~
The Liberty Consulting Group
Page 64
Final Report Analysis of Qwests
Performance Assurance Plans
Table IV-B-l
Statistical AnalysisMSA d Z D. t R ItD.flanoneisa~2reRa ion esu i erences
Measure p-value StatiticaUy Significant
Difference?
OP3 0.742 No
ON 0.000 Yes
OP4-AB 0.415 No
ON. DE 0.000 Yes
OP6 0.003 Yes
OP6-12 0.102 No
OP6-45 0.010 Yes
OP6A 0.140 No
OP-6AI2 0.334 No
OP-6A45 0.188 No
OP6B 0.000 Yes
OP-6BI2 0.334 No
OP-6B45 0.001 Yes
MRJ 0.1 15 No
MR4 0.425 No
MRS 0.81 I No
MR6 0.003 Yes
MR6-AB 0.001 Yes
MR6-DE 0.584 No
MR7 0.037 Yes
MR9 0.000 Yes
Liberty also reviewed the other mechanisms involving low volume situations, including the
"Low Volume, Developing Markets" mechanism found in Section 10 of every participating state
PAP except Colorado's. Liberty found these mechanisms to stil be useful and does not
recommend changing them. In particular, the "Low Volume, Developing Markets" provisions
provide minimum payments for certain products (mainlyADSL) when ordering volumes are low
and Qwest does not meet the PID standard. Liberty concluded that these provisions appear to be
obsolete in most states for ADSL, where volumes are far above the minimums. However, Liberty
also believes that this provision is helpful in ensuring parity for developing markets and that new
products could be added as needed.
C. Proposed Penormance Measure Changes Affecting the
PAPs
Through the CPAP Three-Year Review and consideration of the 2007 Stipulation
recommendations, all participating states except Montana have already removed a number of
PID measures from automatic inclusion in the PAP payment mechanisms. In almost all cases, 60
60 The PO-2A sub-measures was completely eliminated from the CP AP in 2006.
June 30,2009 ~~
The Liberty Consulting Group
Page 65
Final Report Analysis of Qwests
Performance Assurance Plans
the eliminated measures are subject to a PID Reinstatement/Removal Process. This mechanism
reinstates the measures and allows them to generate payments after three months of non-
conforming performance, and removes them once conforming performance is restored.
Based on the analysis of historical payments and Qwest performance described in Section II,
Liberty reexamined PID measures excluded. Libert also used the following criteria to determine
whether any additional PID measures should be excluded from the PAP payment mechanism:
. Payment history and hence Qwest's performance for the wholesale process the
PID measurement measures
. Transaction volumes for the measured process
. Impact of the poor performance for the measured process on the CLECs' abilty
to conduct its business and on the CLECs' end-user customers.
After further analysis, Libert concurs with the list of measures identified in the 2007
Stipulation, but noted five measures on this list that nevertheless have generated modest to
substantial payments. Liberty also identified six additional measures that should be considered
for removal from PAP payment plans and included on the Reinstatement/emoval Process list.
In conducting its analysis, Liberty used the PAP payment data from Januar 2004 through
October 2008. To identifY additional candidates for PAP removal Liberty initially focused on all
measures that generated less than $10,000 in total payments for all states during the January
2007 through October 2008 time period. Liberty then examined the payment history on these
measures in prior years and in the November 2008 through March 2009 timeframe to determine
whether the low level of payments was consistent throughout the entire period. If the payment
amounts were relatively small, Liberty considered the other factors listed above. In addition to
the measures that Liberty proposes adding to the Reinstatement/emoval Process list, Libert
also proposes revisions to three other performance measures that would affect the PAP payment
calculations. These proposals are detailed below.
1. Review of Measures on the ReinstatementlRemoval List
One of Liberty's primary criteria for removal of PID measures from the PAP is the history of
payments during the Study Period. The size and consistency of these payments is a measure of
Qwest's performance for the process underlying the PID measure. Libert reexamined these
payments for the PID measures removed through the CP AP Three-Year Review or
recommendation from the 2007 Stipulation, taking into account the fact that these payments
would necessarily drop off significantly after their removal from a PAP. In almost all these
cases, the PAP payments were small or modest even before the measures were placed on the
Reinstatement/emoval I.st. However, five of the measures generated modest to substantial
payments: PO-2B (Electronic Flow-through), PO-3 (LSR Rejection Notice Interval), PO-7
(Billng Completion Notification Timeliness), OP- 17 (Timeliness of Disconnects Associated
with LNPOrders), and BI-4 (Billng Completeness). After further examination, Libert
June 30,2009 ~,~
The Liberty Consulting Group
Page 66
Final Report Analysis of Qwests
Penormance Assurance Plans
concludes that these measures are appropriate to remain on the Reinstatement/emoval list, as
discussed below.
PO-2B - "Electronic Flow-Throu.gh"
PO-2B is a measure that is only in the Colorado and the New Mexico PAPs among the
participating states.
61 It measures the percentage of orders expected to flow through without
manual handling that actually flow through. Order flow-through is important because manual
handling of orders can slow implementation and increase errors. PO-2B is a benchmark measure,
and only Resale, UN-L, LNP, and UNE-P have benchmarks and thus affect payments. The
benchmarks for these products are 95 percent, 85 percent, 95 percent, and 95 percent,
respectively. As noted in Section II.B, only UN-L and LNP continue to have substantial
ordering volumes in the Qwest region. Table iv -C- 1 shows the payments generated by PO-2B
during the Study Period.
-ec rome ow-rou2
2004 2005 2006 2007 2008 through November 2008
October to March 2009
$984,202 $79,434 $742 $1,898 $3,267 $2,062
Table iv -C-t
tt-State Penalty Payment History
PO 2B "El t . FI Th h"
As can be seen, the payments were very large before the removal PO-2B from the CPAP in
2006, and have been significant since then, despite the inclusion of the measure in only in the
New Mexico PAP.62 Despite the size of the payments, BWG supported the removal of this
measure during the CPAP Three-Year Review because Qwest's performance had recently
improved (as shown by the reduced payments in 2005 over 2004) and other PAP measures help
assure and timely installation (OP-3). Liberty believes that this logic is stil sound.63 In addition,
because the measure is on the Reinstatement/emoval list, consistent poor flow-through
performance for three months would stil cause payments to be assessed.
PO-3 - "LSR Rejection Notice Interval"
PO-3 is a measure in all the participating state PAPs which assesses the timeliness of Qwests
providing notices of rejection of CLECs' service requests. The payment history is shown in
Table iv -C-2, and shows modestly high payments continuing into the present, despite the
removal of this measure from most of the PAPs.
61 PO-2B is also in the Minnesota and Washington PAPs.
62 Of the $984,202 in total 2004 payments, $900,000 was Tier 2 payments made to Colorado. In 2005, of the
$79,434 paid in total payments, $75,000 was Colorado Tier 2 payments.63 Although Liberty recommends in Section IV.B.2 the removal of PO-20 also, Qwest's performance on this
measure has been reasonably good and it remains on the Reinstatementlemoval list.
June 30, 2009 ~~
The Liberty Consulting Group
Page 67
Final Report Analysis of Qwests
Performance Assurance Plans
-e.iec ion o ice n erva
2004 2005 2006 2007 2008 through November 2008
October to March 2009
$29,061 $10,314 $9,985 $534 " $2,914 $2
Table IV-C-2
II-State Penalty Payment History
PO 3 "LSR R' f Nfl t i"
In the CPAP Three-Year Review, BWG recommended removal of this measure because PO-5,
which measures FOC timeliness, provides an incentive for Qwest to provide FOCs on time.
Timely FOCs indirectly allow CLECs to determine whether their orders have been rejected.
Libert believes that this logic is stil sound.
PO-7 - "Billng Completion Notifcation Timeliness"
PO-7 is a measure in all the participating state PAPs which assesses the timeliness of Qwest's
providing notices of completion ofthe CLECs' orders in Qwests biling systems. The payment
history is shown in Table IV -C-3, and shows modestly high payments continuing into the
present, with particularly high payments during 2007, despite the removal of the measure from
the CPAP and several QPAPs.
-i 102 ompie ion o i ica ion ime mess
2004 2005 2006 2007 2008 through November 2008
October to March 2009
$18,148 $8,1 I3 $2,123 $25,752 $335 $61
Table IV -C-3
ll-State Penalty Payment History
PO 7 "Bll C i f N ffi f T' l "
In the CPAP Three-Year Review, BWG recommended removal of this measure because it
measures only notification timeliness not the actual completion of the order, and therefore does
not measure a process that has a direct impact on a CLECs' customers. Libert notes that there
can be some end-user customer impact from failure of a CLEC to receive a timely biling
completion notifcation. The biling completion notification is Qwest's notification that all parts
of an order are complete, including updating of the biling records. However, Libert supports
the continued removal of PO-7 from the PAPs. PO-6, which measures the timeliness of work
completion notifications, remains in the PAPs and provides an indication to the CLEC of
completion of all the provisioning work on the order. Furthermore, any consistent poor
performance on providing timely billng completion notices would stil produce penalties,
because PO-7 remains on the Reinstatement/emoval list.
OP-17 - "Timeliness of Disconnects Associated with LNP Orders"
OP- 1 7 is a measure in all the participating state PAPs which assesses whether Qwest completes
number ports without disconnecting the customer's line before the scheduled time and date. Only
OP- 1 7 A is non-diagnostic and par of the PAPs. The OP- 1 7 A payment history is shown in Table
IV -C-4, and shows high payments initially but relatively small payments since 2004. While a
premature disconnect of the customer's line by Qwest prior to the date and time of the number
port wil remove the customer from service, Liberty notes that the significant reduction in
June 30, 2009 ~~
The Liberty Consulting Group
Page 68
Final Report Analysis of Qwests
Penormance Assurance Plans
payments since 2004 indicates that Qwest is providing relatively good service for this function
and believes it is appropriate to keep the OP-17 measures on the Reinstatement Process List.
2004 2005 2006 2007 2008 through November 2008
October to March 2009
$108,940 $1,500 $0 $600 $150 $150
Table IV -C-4
II-State Penalty Payment History
OP-17 A "Timeliness of Disconnects Associated with LNP Orders"
BI-4 - "Billng Completeness"
81-4 is a measure in all the paricipating state PAPs which assesses the completeness of Qwest's
bills to the CLECs. 81-4 payment history is shown in Table IV-C-5, and shows a continuing
modest level of payments.
-i 102 ompieteness
2004 2005 2006 2007 2008 through November 2008
October to March 2009
$19,697 $5,462 $312 $2,239 $6,171 $1,262
Table IV-C-5
II-State Penalty Payment HistoryBI 4 "Bll C I "
In the CPAP Three-Year Review, BWG recommended removal of this measure, because BI-3
measure the extent of biling adjustments, and such adjustments would be required if the bils
were not complete. Furthermore, with the phase-out of UNE-P, the likelihood that CLECs wil
rely on Qwest's bils to invoice usage to end-users is reduced. Given the relatively modest
payments for BI-4 relative to BI-3, Liberty sees no reason to change that assessment.
2.Additional Measures Recommended
Reinstatement/Removal Process
for the
Through this analysis, Liberty identified six other measures to recommend as additions to the
Reinstatement/emoval list.
PO-9 - "Timely Jeopardy Notices"
PO-9 is a parity measure that can be found in the PAP for all states which assesses how well
Qw~st provides timely notices to the CLECs that the installation date and an order is in jeopardy.
Table IV-C-6 demonstrates that the PAP payments generated by this measure have been
relatively smalL. PO-9 is disaggregated into four sub-measures; PO-9A - "Non-Designed
Services," PO-9B - "Unbundled Loops." PO-9C - "LIS Trunks," and PO-9D - "UNE-P POTS,"
all of which experience low to moderate transaction volumes each month.
June 30, 2009 ~~
The Liberty Consulting Group
Page 69
Final Report Analysis of Qwests
Performance Assurance Plans
-imeiy eopar iy 0 ices
2004 2005 2006 2007 2008 through November 2008
October to March 2009
$9 $0 $28 $371 $39 $56
Table iv -C-6
II-State Penalty Payment History
PO 9 "T' I J d N t "
PO-9 was considered for removal during the CPAP Three-Year Review, but BWG decided not to
recommend removal because timely jeopardy notices are critical to the CLECs' abilty to provide
a realistic date to their end-user customers for service implementation. Liberty agrees that it is
important for a CLEC to be able to communicate with its customer when a due date wil be
missed, but notes that the payments have continued to be small, implying that Qwest's
compliance with the standard has been relatively good. Additionally, Libert found the volumes
of jeopardy notices associated with this measure to be very small for three of the four PO-9 sub-
measures, with the PO-9 A, PO-9C and PO-9D sub-measures averaging less than ten transactions
per month at the 14-state level between January 2007 and October 2008. The PO-9B sub-
measure experienced a moderate level of jeopardy notices averaging 403 transactions per month
across the 14-state region during the same time frame. By placing this measure on the
Reinstatement/emoval list, it can affect payments after three months of poor performance,
thereby continuing to provide an incentive to Qwest to provide timely jeopardy notices.
PO-19 - "Stand-Alone Test Environment (SATE) Accuracy"
PO- 1 9 is a benchmark measure that is currently used only in the Arizona and New Mexico
PAPS.64 This measure addresses Qwest's performance in providing an accurate test environment
for new softare releases. This measure has never experienced a penalty payment from January
2004 through March 2009 demonstrating that Qwest's performance has been good over the
entire Study Period. In addition, the process has limited immediate impact on end-user
customers. Therefore, Liberty recommends placing this measure on the Reinstatement/Removal
list.
PO-20 - "Manual Service Order Accuracy"
The PO-20 measure has a history of low penalty payments, as shown on Table IV-C-7. Liberty
found that although Qwest often fails to meet the 95 percent benchmark on this measure, this
failure frequently results from a single miss on a low volume of transactions (i.e., Qwest would
need to have 100 percent performance or it would fail as the result of a single miss). Of the total
payments that Qwest made between 2004 and October 2008, 29.7 percent were generated by
single miss failures to meet the 95 percent benchmark. Now that the One Allowable Miss Rule
has been implemented in most states, the number of such failures would be diminishing, as
shown in the recent payment history.
64 PO-19 is listed as a diagnostic measure in the CPAP. In early versions of the CPAP, PO-19 was eligible for
generating penalty payments. However, it was made diagnostic during 2005 prior to the completion of the Three-
Year Review in the CPAP version dated 5/6/05 (Colorado SGAT Ninth Revision, Ninth Amended Exhibit K, dated
May 6, 2005).
June 30, 2009 ~~
The Liberty Consulting Group
Page 70
Final Report Analysis of Qwest's
Performance Assurance Plans
-anua ervce r er ccuracy'
2004 2005 2006 2007 2008 through November 2008
October to March 2009
$2,846 $4,356 $2,542 $1,885 $212 $225
Table iv -C-7
11-State Penalty Payment History
PO 20 "M IS' 0 d A "
PO-20 was not considered for removal in the CPAP Three-Year Review because it was argued
that there could be significant end-user. consequences from manual service order errors.
However, given the relatively low volume of Qwest manual inward service orders evaluated each
month and recent payment history, Liberty believes that placing this measure on the
Reinstatement/emoval list wil provide adequate protection for the customers.
CP-l- "Collocation Completion Interval"
CP- 1, a benchmark measure only found in the Arizona and Colorado PAPs, has generated only a
minimal Tier 1 payment of $4 in Arizona in 2007. There were no other payments in Arizona and
no payments made in Colorado during the 2004 through March 2009 timeframe. The CP-l
measure is also a very low volume measure with an average of only 1.3, 0.8, and 3.0 transactions
per month in Colorado and an average of 1.0, 1.3 and 3.3 transactions per month in Arizona for
the CP-IA, CP-IB and CP-IC sub-measures respectively. Qwest did not miss the benchmark on
any of the CP-L sub-measures in any other state from January 2007 through October 2008.
CP-l was considered for removal during the CPAP Thee-Year Review, but BWG elected not to
recommend its removal because of the importce of collocation for CLEC market entry.
Because of the recommendation to remove CP-3, BWG felt it was important to maintain at least
one collocation measure in the PAP. At the time ofBWG's analysis, the FCC had recently issued
the TRO and TRRO orders, which eliminated UNE-P. BWG speculated that this would likely
increase the importance of collocation, because the elimination of UNE-P would force the
CLECs to rely on other UNEs. However, Liberty's analysis shows that overall CLEC entry and
the volume of collocation has in fact decreased since 2005. Thus, given the relatively low
volumes and the consequent limited impact of temporary poor performance on collocation
completion timeliness, Liberty recommends placing CP- 1 on the Reinstatement/emovallist,
CP-2 - "Collocations Completed within Scheduled Intervals"
CP-2 is a benchmark measure which is in the PAPs of all paricipating states, except Colorado. It
has generated no payments during the Study Period in any of the participating states. During the
Januar 2007 through October 2008 timeframe, the CP-2B sub-measure averaged only 20
collocation completions per month and the CP-2C sub-measure averaged only 23 monthly
collocation completions across the entire 14-state region. There were no collocation completions
measured by the CP-2A sub-measure during this time. The focus of CP-2 is similar to CP- 1,
timely collocation completion intervals. Therefore, the same considerations apply, and Liberty
recommends placing CP-2 on the Reinstatement/emovallist.
June 30, 2009 ~~
The Liberty Consulting Group
Page 71
Final Report Analysis of Qwests
Penormance Assurance Plans
CP-4 - ~~Collocation Feasibility Study Commitments Met"
The CP-4 measure appears in the PAPs for all participating states, except Colorado. This
measure did not result in any penalty payments from January 2004 through March 2009. It is
also a measure that typically has a relatively low volume of transactions with an average of 43
transactions per month across the entire 14-state region for the period of January 2007 through
October 2008. This measure is similar to CP-3, which was removed from the CPAP as part of the
Three-Year Review. In its analysis, BWG concluded that CP-3 was relatively unnecessary, given
the existence of CP- 1, which measures ultimate completion timeliness, rather than one step in the
process, like CP-3. Libert agrees with this assessment. Given the low collocation volumes and
lack of past penalty payments Liberty recommends placing CP-4 on the Reinstatement/emoval
list.
3. Other Measures Considered for the Reinstatement/Removal
Process
Liberty considered several other measures for inclusion in the Reinstatement/emoval process
because of low PAP payments. However, after considering other factors, Libert does not
recommend that these measures be removed from the PAPs. The reasons that Libert
recommends their continued inclusion in the PAPs follow.
GA-l- ~~Gateway Availability -IMA-GUI" and GA-6 - "Gateway Availability - GUI-Repair"
Both of these measures have had low PAP payments over the Study Period, as shown in Table
IV -C-8. However, the IMA-GUI is necessary for the many CLECs to electronically transmit
automated pre-ordering and ordering transactions, and the Repair GUI is similarly necessary for
transmitting electronic maintenance and repair transactions. The abilty to conduct these
transactions is critical for the CLECs and would have a significant impact on their end-user
customers if these systems were unavailable to the CLECs. All the parties during the CPAP
Three-Year Review concurred that these measures should not be removed at that time. The same
considerations that were raised at that time are stil valid. Sustained poor availabilty of these
interfaces would have a major impact on the CLECs' abilty to do business, and if they were to
be placed on the Reinstatement/Removal list, Qwest would only be assessed penalties after three
months of poor performance. Therefore, Liberty does not recommend that these two measures be
removed from the PAPs and put on the Reinstatement/emoval list.
Table IV-C-8
II-State Penalty Payment History
GA-I "Gateway Availabilty - lMA-GUI" & GA-6 "Gateway Availabilty - GUI-Repair"
2004 2005 2006 2007 2008 through November
October 2008 to March
2009
GA-I $9,167 $0 $0 $0 $9.167 $0
GA-6 $9,167 $0 $0 $0 $0 $8,167
June 30, 2009 ~~
The Liberty Consultng Group
Page 72
Final Report Analysis of Qwests
Penormance Assurance Plans
PO-L- "Pre-Order/Order Response Times"
The PO- 1 measure never experienced a penalty payment due to a failure during the 2004 to 2008
timeframe. However, similar to the GA-l and GA-6 measures, the PO-L measure monitors a high
volume activity critical to the entire CLEC community. As a result, Liberty does not recommend
removing it from the PAP. This is consistent with BWG's conclusion during the CPAP Three-
Year Review.
PO-5 - "Firm Order Confirmations On Time"
PO-5 measures the timeliness of FOCs, and contains four sub-measures. One sub-measure, PO-
50, which measures FOC timeliness for LIS trunks, was placed on the Reinstatement/Removal
Process list for the CPAP after the Three-Year Review and on the PAPs of all the other
paricipating states except Montana following a recommendation in the 2007 Stipulation. The
other three sub-measures remain active in the PAPs of all the participating states. Table iv -C-9
shows the history of payments generated by this measure during the Study Period. The payments
have not been large, particularly recently, but there have been payments for all three of the sub-
measures remaining in the PAP.
2004 2005 2006 2007 2008 through November 2008
October to March 2009
$14,748 $3,000 $13,050 $1,496 $1,169 $942
Table IV-C-9
ll-State Penalty Payment History
PO-5 "Firm Order Confirmations On Time"
Despite the relatively low recent payments from PO-5, Liberty does not recommend placement
of this measure on the Reinstatement/emoval list. This is consistent with the recommendations
of all parties during the CPAP Three-Year Review, and there is no evidence that the
circumstances since that time have changed to alter the considerations leading to those
recommendations. Timely FOCs are very important for CLECs, because they contain
information crucial to meeting their end-user customers' needs in service installations, such as
installation due dates and assigned telephone numbers, which the CLECs need for
communicating service delivery expectations with their customers. There also remains a high
volume for such transactions. FOCs are needed for every order issued by the CLEC, unlike
jeopardy notices, which are only required in cases where there is a delay in providing service.
Therefore, Liberty believes it is appropriate for PO-5 (FOCs on Time) to remain in the PAP
while PO-9 (Timely Jeopardy Notices) can move to the Reinstatement/Removallist.
OP-13A - "Coordinated Cuts On Time - Unbundled Loop"
OP-13A measures the timeliness of coordinated hot cuts for UN-L. The payment history is
shown in Table iv -C- 1 O. The payments have been relatively small, but consistent across the
Study Period. Despite, the relatively low payments, Liberty does not recommend removal of OP-
13A from the PAPs. Hot cuts continue to be an important transaction for a large number of
CLECs that provide service through UN-L; poor hot cut performance can have a significant
impact on such CLECs' customers. Therefore, Liberty does not recommend removal of this
June 30,2009 ~~
The Liberty Consultng Group
Page 73
Final Report Analysis of Qwests
Penormance Assurance Plans
measure from the PAPs. This is consistent with the BWG's conclusions during the CPAP Three-
Year Review, and the changes in the industry since then have not minimized the importance of
these arguments.
2004
OP-13 "C
2005
Table IV-C-IO
II-State Penalty Payment History
oordinated Cuts On Time - Unbundled Loo "2006 2007 2008 through
October
$4,172 $1,330 $1,247
November 2008
to March 2009
$1,608$2,733 $890
BI-l- "Time to Provide Recorded Usage Records"
BI-l is a measure in all participating state PAPs which assesses Qwest's timeliness in providing
usage records. CLECs use these records to bil their customers or other carriers and to verifY the
accuracy of their Qwest bils. There are two sub-measures BI-IA, which assesses the timeliness
of usage for Resale and UNE products, and BI-IB, which assesses the timeliness of usage for
jointly provided switched access. The payment history is shown in Table iv -C- 1 1. The payments
have been very high in the past, but have been dropping off significantly recently, parly because
the reduction in CLECs use of Resale and UNE products involving switched usage (like UNE-P)
has dropped dramatically over the Study Period. Nevertheless, the BI-IA payments remain well
above the threshold Libert used for identifying candidate measures to remove from the PAPs.
BI-I "
Table IV-C-ll
II-State Penalty Payment History
Time to Provide Recorded Usa e Records"2005 2006 2007 2008 through
October
2004
BI-1A
BI-IB
$443,324
$5,380
$804,073
$55,002
$66,315
$0
$38,866
$0
$10,067
$0
November
2008 to March
2009
$5,008
$0
During the CPAP Three-Year Review, BWG recommended keeping BI-l in the PAP largely
because of the impact on the CLECs and their end-user customers of untimely usage records.
Despite the drop in volume of these usage records resulting from changes in the CLEC service
mix, a number of CLECs continue to rely on them for the data the CLEC needs to bil its
costumers. Late usage records provided to the CLECs wil result in late biling to the CLECs'
customers. Therefore, Liberty believes BI-l should remain in the PAP.
BI-3 - iiBillng Accuracy -Adjustments/or E"ors"
BI-3 is a measure in all participating state PAPs which assesses the accuracy of Qwests bils
rendered to the CLECs for wholesale services by measuring the percentage of biled revenue that
has been adjusted because of errors. There are two sub-measures BI-3A, which measures errors
in UN and Resale bils, and BI-3B, which measures errors in reciprocal compensation bils. The
payment history is shown in Table IV-C-12. The payments have been very high for UN and
June 30,2009 ~~
The Liberty Consulting Group
Page 74
Final Report Analysis of Qwests
Performance Assurance Plans
Resale bils (BI-3A), but recently there have been no payments for reciprocal compensation bils
(BI-3B). Liberty therefore considered whether to eliminate BI-3B from the PAPs.
-i 102 ccuracy-I.IUS men s or rrors
2004 2005 2006 2007 2008 through November
October 2008 to March
2009
BI-3A $1,111,576 $403,809 $297,355 $253,935 $59,328 $36,999
BI-3B $21,074 $10,726 $0 $0 $0 $0
Table IV -C-12
ll-State Penalty Payment History
BI 3 "Bir A Ad' t t ti E "
During the CPAP Three-Year Review, BWG recommended keeping BI-3 in the PAP although
wholesale biling errors have relatively little impact on end-user customers. BWG noted that
biling errors nevertheless can absorb considerable CLEC resources. Liberty agrees. Despite the
lack of recent payments for BI-3B, the continued high payments for BI-3A indicate good reason
to keep both sub-measures ofBI-3 in the PAP.
4. Additional pin change proposals that would affect the PAP
In addition to proposing measures that should be considered for removal from the PAP and
placed on the Reinstatement/emoval Process list, Liberty has identified three additional PID
change proposals that would impact the PAP.
OP-5 - "New Service Quality"
This measure was designed to evaluate the quality of newly-installed service orders that are free
of CLEC/customer-initiated trouble reports during the provisioning process and within 30
calendar days following installation completion. Currently this measure is divided into four sub-
measures: OP-5A "New Service Installation Quality Reported to Repair," OP-5B "New Service
Provisioning Quality," OP-5T "New Service Installation Quality - Total," and OP-5R, "New
Service Quality Multiple Report Rate." OP-5A has a parity standard and OP-5B has a benchmark
standard for those product disaggregations that have a standard, and both OP-5A and OP-5B are
in all the state PAPs.65 Both the OP-5T and OP-5R sub-measures are currently diagnostic
measures. The OP-5A performance measure reports the percentage of inward line service orders
that are free of trouble repair reports within 30 calendar days of installation completion. The PID
defines repair trouble reports as CLEC or retail customer notifications to Qwest of an out-of-
service or other service affecting condition for which Qwest opens a repair ticket in its
maintenance and repair management and tracking operations support systems. The PID specifies
that OP-5A considers trouble reports created by Qwest's call center and stored in its call center
database provisioning trouble reports and includes these tickets in the OP-5B results
calculation.66
65 Each ofthese two sub-measures has a few products that are diagnostic.
66 14-State 271 PID, Version 9.0.
June 30,2009 ~~
The Liberty Consulting Group
Page 75
Final Report Analysis of Qwests
Performance Assurance Plans
OP-5B measures the percentage of inward line service orders free of provisioning trouble reports
during the provisioning process and within 30 calendar days of installation completion. The PID
defines provisioning trouble reports as CLEC notifications to Qwest of out-of-service or other
service affecting conditions that are attributable to provisioning activities, including but not
limited to LSR/service order mismatches and conversion outages. For provisioning trouble
reports, Qwest creates call center tickets in its call center database. Qwest captures call center
tickets closed in the reporting period or the following month for this measurement. Qwest does
not count call center tickets closed to network reasons in OP-5B when a repair trouble report for
that order is captured in OP_5A.67
Liberty believes that the manner in which this measure is currently split between repair center
trouble reports and call center trouble reports creates an unnecessary complexity to the reporting
structure. This split of trouble reports is not required to make a determination of the quality of
Qwest's new service installations, which is the overall purpose of OP-5. It also creates a low
volume problem for the calculation of OP-5B payments, because the number of call center
provisioning trouble reports created by Qwest that count toward this sub-measure is very smalL.
For example, for the time period of January 2007 through October 2008, with the exception of
two product disaggregations, every product reported under the OP-5B sub-measure averaged less
that one provisioning trouble report per month across the entire 14-state region. Additionally, the
two products that did average more than one provisioning trouble reports per month, unbundled
analog loops and resale residential service, also experienced extremely low volumes of 8.3 and
1.6 provisioning troubles per month,re spectively. Total Tier 1 payments for this sub-measure for
all of 2007 and 2008 were $681. In contrast, OP-5A had nine product disaggregations that
exceeded an average of 10 repair trouble reports per month across the 14-state region with five of
these nine product disaggregations averaging more than 30 trouble reports per month.68 Tier 1
payments for the OP-5A sub-measure during this same time period were $31,184.
To eliminate the low volume problem for OP-5B, that sub-measure could be removed from the
PAPs. However, Liberty believes that a better approach would be to use what is effectively a
combination of OP-5A and OP-5B. This can be accomplished by changing OP-5T from a
diagnostic to a parity measure and replacing OP-5A and OP-5B in the PAPs with OP-5T. Qwest
calculates the OP-5T based on both types of trouble reports (i.e;, repair trouble reports and
provisioning trouble reports) essentially combining the OP-5A and OP-5B sub-measures into a
single measure that can be used to determine the quality of Qwest's new service installations.
Because the OP-5T measure would be used to determine parity of new service installation
quality based on the total number of repair trouble reports referred to Qwest within 30 calendar
days of service installation, the same parity standards that are used for the OP-5A measure would
be used for the OP-5T measure.
67 Ibid, P 59.
68 The five products and the average monthly trouble report volume for each are: unbundled analog loops (122..2
troubles per month), unbundled DS-l loops (66.8 troubles per month), EEL DS-I (65.0 troubles per month),
Unbundled 2-wire non-loaded loops (56.5 troubles per month) and Resale residential service (48.5 troubles per
month).
June 30,2009 ~~
The Liberty Consulting Group
Page 76
Final Report Analysis of Qwests
Performance Assurance Plans
Electronic Gateway Availability (GA) and Pre-Order/Order (PO) Measures
Version 9.0 of the PID document (Exhibit B) and the QPAP (Exhibit K) for most states, stil
contains measures that involve reporting on the availabilty and the performance of the IMA-EDI
interface. However, Qwest retired this interface in November 2007 and replaced it with the IM-
XML interface, which was made available to the CLECs in October 2006. Currently most states
do not have measures in the PAP monitoring Qwest's performance on the XML interface.
Colorado was the only state among the II-states paricipating in this study for which the
reporting measures specific to the XML interface had been approved before the end of the Study
Period. The Utah Commission approved PAP changes to incorporate the XML interface on
February 4, 2009.69 Libert recommends that the remaining states adopt those changes in
Version 9.1 of the 14-State PID document and corresponding PAP changes which eliminate
reference to the EDI interface and replace the EDI performance results with those for the XML
interface. The specific measures affected by this proposal are:
. GA-2 - "Gateway Availabilty - IMA-EDI" - this PID should be replaced by
GA-8 in the PAPs
. GA-8 - "Gateway Availabilty - IMA-XML" - this PID should replace GA-2 in
the PAPs
. PO-l - "Pre-Order/Order Response Time"
. PO-2 - "Electronic Flow-through"
. PO-3 - "LSR Rejection Notice Interval"
. PO-4 - "LSRs Rejected"
. PO-5 - "FOCs On Time"
. PO-6 - "Work Completion Notification Timeliness"
. PO-7 - "Biling Completion Notification Timeliness"
. PO- 16 - "Timely Release Notifications"
. PO- 1 9 - "SA TE Accuracy"
. PO-20 - "Manual Service Order Accuracy"
Ordering and Provisioning Measures (OP)
Versions 9.0 and 9.1 of the PID document use retail ISDN-BRI designed service as the parity
stadard for a number of wholesale UNE-L products in the ordering and provisioning measures.
However, Qwest rarely has any order volumes for its retail ISDN-BRI designed service. As a
result, Qwest cannot fail the measure test for these wholesale products because there is no retail
analog result to measure against. Liberty recommends that the stadard for these wholesale
products be changed to either i) a retail product that experiences consistent volumes or ii) a
benchmark measure, if such a retail comparative does not exist. Liberty recommends that that a
collaborative process be used to determine the appropriate replacement standards for retail
ISDN-BRI. The measures and products affected by this recommendation are:
. OP-3 - "Installation Commitments Met"
69 The Washington Utilities and Transportation Commission has also approved these changes.
June 30, 2009 ~,~
The Liberty Consulting Group
Page 77
Final Report Analysis of Qwests
Penormance Assurance Plans
ISDN Capable Loop
. OP-4 - "Installation Interval"
ISDN Capable Loop
. OP-5 - "New Service Quality"
Non-Loaded 2-Wire Loop
ADSL Qualified Loop
ISDN Capable Loop
. OP-6 - "Delayed Days"
Non-Loaded 2-Wire Loop
XDSLI Capable Loop
ADSL Capable Loop
ISDN Capable Loop
. OP- 15 - "Interval for Pending Orders Delayed Past Due Date"
Non-Loaded 2-Wire Loop
ADSL Qualified Loop
ISDN Capable Loop
D. Proposed Product Changes
One of the recommendations in the 2007 Stipulation is to remove low volume products from
applicable OP and MR measures from the PAPs in all 14 states in the Qwest operating territory
except Colorado. Most of these products were eliminated from the CPAP in 2006 after the
Three-Year Review. The performance results for these low volume products continue to be
reported in the 271 performance plans. The products identified for removal from the PAPs in the
2007 Stipulation include:
. Resale Centrex
. Resale Centrex 21
· Resale DSO (Designed and Non-Designed)
. E91 1/911 Trunks
. Resale Frame Relay
. Resale Basic ISDN (Designed and Non-Designed
. Resale Primary ISDN (Designed and Non-Designed)
. Resale PBX (Designed and Non-Designed)
· Sub-Loop Unbundling
. UN-PPOTS
. UNE-P Centrex
. UNE-P Centrex 21
June 30.2009 ~,~
The Liberty Consulting Group
Page 78
Final Report Analysis of Qwests
Performance Assurance Plans
Liberty agrees that, with the exception of sub-loop unbundling in Colorado,70 the products
shown on the list above have a low level of transaction activity in all the participating states, and
that it is appropriate to remove them from the PAP. Because of the continued low order and
trouble report volumes for these products, Libert recommends that they be removed from the
PAP in all states that stil include them, with that one exception.71
Libert examined the historical ordering and trouble reporting volumes on the remaining
products to identifY other candidates for removal from the PAPs. As the basic criterion for
product removal, Libert used the condition that the ordering and trouble reporting volumes
never exceeded ten transactions per month in any state at the CLEC aggregate level from January
2007 through October 2008. Liberty also considered products that came close to meeting this
criterion and considered them on a case-by-case basis. Based on this analysis, Liberty
recommends that the states remove six additional low volume products from the PAPs for all OP
and MR measures in which they appear, with one exception noted below:
. Unbundled DS-3 Loops
. UDIT - Above DS-l
. Unbundled 4- Wire Non-Loaded Loops
. Loops with Conditioning 72
. Unbundled ISDN Capable Loops (Applies to all states and measures except for
MR measures in Arizona and Colorado)
. Line Sharing
Appendix B contains tables that show the state-by-state ordering and trouble report volumes for
these six products. As shown in these tables, in Arizona and Colorado there were low monthly
ordering volumes for the Unbundled ISDN Capable Loops but a significant number of trouble
reports each month. This is because of the substantial embedded base of such loops in Arizona
and Colorado (1,356 and 1,000, respectively, in October 2008). Thus, Liberty recommends
making the exception for these two states in removing this product from the MR measures. 73
70 Sub-loop unbundling averaged 14.4 orders per month in Colorado from January 2007 through October 2008, but
had little to no order activity in the other ten paricipating states during this same time period.71 Montana should remove all the products on the 2007 Stipulation list. Colorado should remove UNE-P POTS,
Centrex, and Centrex 21 .72 "Loops with Conditioning" is a product disaggregation for OP measures but not for MR measures. Loops ordered
in this way appear in other provisioned product categories in the MR measures, such as UBL 2-Wire Non-Loaded
Loops.73 Because calculation of MR-8 (Trouble Report Rate) includes not only trouble report volumes but also lines, an
argument could be made that both the trouble report and lines should be considered in determining whether
"volumes" are too small for product disaggregations of this measure. Thus, states with large quantities of lines in
service for the products Libert has identified for elimination might want to modify Libert's recommendations.
However, in the interest of enhancing simplicity, Libert chose not to introduce the extra complexity this would
entaiL.
June 30, 2009 ~~
The Liberty Consulting Group
Page 79
Final Report Analysis of Qwests
Penormance Assurance Plans
Table iv -0- 1 below provides a summary view of the ordering volumes on the products listed
above for January 2007 through October 2008. The information provided in this table is the 22-
month average monthly volume in the state that had the greatest level of transaction activity, the
highest transaction level for the state that had the greatest number of transactions in a single
month, the number of states that average more than three transactions per month during the 22-
month period, the number of states that exceeded ten transactions in any given month, and the
number of times ten transactions per month were exceeded across all states. The only product
that experienced more than ten transactions in a single state more than once was Loops with
Conditioning which occurred in Iowa in May 2008 (18 transactions) and again in September
2008 (12 transactions). However, Iowa's monthly average volume for this product was 1.8 orders
per month. Thus, these two months appear to be exceptional and do not affect Liberty's
recommendations to remove the product in all states.
Line Sharing requires special mention. Ordering volumes in Colorado exceeded Liberty's low-
volume threshold during the first six months of2007. However, the volumes have been very low
in Colorado since then, probably as a result of the TRO/TRRO phase-out provisions for this
product. Thus, it is stil appropriate for Line Sharing to be excluded from the CPAP, as occurred
in a CPAP in 2006 after the Three-Year Review. For alI the other states, this product meets the
low ordering volume criterion; only one other state received any orders for the product. All
states, including Colorado, met Liberty's criteria for Line Sharing trouble reports, supporting
Libert's recommendation to removing the Line Sharing product from all the MR measures.
-a e ummary iew
Product Greatest Greatest Number of Number of Number of
Average Single States that States that Times to
Monthly Monthly Averaged Exceeded 10 Transactions
Volume in Volume in Less than 3 Transactions Was
Any State Any State Transactions Exceeded
(Transactions (Transactions)Per Month
Per Month)
UBL-DS3 0.5 3 I I 0 0
UDIT Above 3.5 16 10 I I
DSt
UBL4-Wire I.II I I I I
Non-Loaded
Loop
Loops with 4.9 18 10 3 4
Conditionin2
UBLISDN 3.9 12 10 I I
Capable Loop
Line 6.8 40 10 I 3
Sharin274
Table IV-D-l
CLEC Aggregate Ordering Volumes for Selected Products
January 2007 - October 200811 St t S V'
74 All the Line Sharing statistics on this summar table are for Colorado. New Mexico was the only other state with
any Line Sharing order volumes during the 2007-2008 period, and it only received a single order for the service.
June 30,2009 ~~
The Liberty Consulting Group
Page 80
Final Report Analysis of Qwests
Performance Assurance Plans
There are a number of other product disaggregations with tyically low to moderate ordering and
trouble report volumes although larger than the products considered above. Although the
volumes are high enough for continued inclusion in the PAPs, they are small enough in many
states that the kind of low-volume test situations discussed in Section iv -C occur. As discussed
in that section, Liberty recommends an aggregation method for these low volume situations that
wil help alleviate this problem.
In addition to the product removals mentioned, Liberty has one recommendation for adding a
product. The Colorado Commission should add the Unbundled ADSL Capable Loop product to
the CPAP. This product was removed from the CPAP as part of the Three-Year Review decision
in 2006. Since that time, the ADSL Capable Loop product has been experiencing increasing
volumes in Colorado and appears to be an increasingly important competitive product for the
CLECs in that state. This product is currently included in the QPAP for the other 10 states
participating in this study. As shown in Table IV-D-2 below, for the period of January 2007
through October 2008, Colorado's order volumes for this product generally exceed those of the
other participating states. Additionally, for the most recent period of November 2008 through
March 2009, which is not included on the table below, Colorado averaged 73.8 ADSL capable
loop order per month, showing continuing growth in the volumes of these orders in the state.
r ervo urnes or apa e oops
Average
January monthly
State 2007 volumes through October volume for 10 Total Volumes
2008 volumes month period
in 2008
Ariona 16 337 33.7 353
Colorado 2 527 52.7 529
Idaho 0 0 0 0
Iowa 257 87 8.7 344
Montana 40 47 4.7 87
North Dakota 830 760 76.0 1,590
Nebraska 184 113 I I.297
New Mexico 363 181 18.1 544
South Dakota 0 0 0 0
Utah 0 41 4.1 41
Wvominl!82 67 6.7 149
Table IV-D-2
o d ti ADSL C bl L
E. Proposed Performance Indicator Definition Changes
Libert proposes that the states consider four changes to the PIDs that wil not have an impact on
the PAPs. These proposals are a result of both the measure analysis performed by Libert and
input that Liberty received from the CLECs. Some of these changes wil make the reported
results more meaningfuL. Others add sub-measures to monitor Qwest's service quality for
activities that CLECs indicate are importnt to their business and are not monitored today. These
June 30,2009 ~~
The Liberty Consulting Group
Page 81
Final Report Analysis of Qwests
Performance Assurance Plans
sub-measures would be diagnostic and allow evidence to be developed as to whether Qwest's
performance for these activities warrants inclusion of the sub-measures in the PAP. These
changes are described below.
1. MR-4 - "All Troubles Cleared within 48 Hours"
The purpose of the MR-4 measures is currently described as: "(e)valuates timeliness of repair for
specified services, focusing on trouble reports of all types (both out of service and service
affecting) and on the number of such trouble reports cleared within the standard estimate for
specified services (i.e., 48 hours for service-affecting conditions).,,75 Libert proposes that the
definition of this measure be modified so that it only reports service affecting trouble reports and
not all trouble reports, thereby eliminating the out-of-service troubles from the report. The
rationale for this is that Qwest's performance for out-of-service trouble reports, which have an
objective restoral time of 24 hours, is reported by the MR-3 "Out of Service Cleared with 24
Hours" measure. Because the product disaggregations for both the MR-3 and MR-4 measure are
identical, including out-of-service trouble reports in the MR-4 reported results could potentially
mask poor performance in the resolution of service affecting troubles on these products within 48
hours. The lower objective restoration time of 24 hours for out-of-service troubles wil
effectively lower the overall restoration time for all trouble reports. By limiting the MR-4
measure to service affecting troubles only, the users of the report wil receive more accurate data
on Qwest's ability to resolve these troubles within 48 hours.
2. OP-4 - "Installation Interval"
The OP-4 measure evaluates the timeliness ofQwest's abilty to install service for customers by
calculating the average time it takes Qwest to install inward service orders for various products.
One of the CLECs suggested that Qwest also report on its performance on expedited service
orders. Liberty believes that such a sub-measure may provide useful data to both the CLECs and
Qwest regarding Qwest's ability to install service on a reduced interval in circumstances that call
for it. As such, Liberty proposes that a diagnostic sub-measure be added to OP-4 to report on
Qwests results in meeting expedited due dates.
3. MR-7 - "Repair Repeat Trouble Rate"
The MR-7 measure evaluates the accuracy of Qwest's repair performance by calculating the
number of repeat trouble reports on the same line or circuit within 30 days of the initial trouble
report being closed. However, neither this, nor any other, measure provides data on the number
of chronic trouble reports being experienced by the CLECs. Chronic troubles would be defined
as lines or circuits that receive greater than two trouble reports over an extended period of time.
Liberty believes that it is possible that the repeat trouble report metric is missing an important
component of reporting on chronic troubles that may be indicative of faulty facilties, other
15 14-Siaie 211 PID Version 9.0
June 30,2009 ~,~
The Liberty Consulting Group
Page 82
Final Report Analysis of Qwests
Penormance Assurance Plans
network problems and/or Qwest repair process problems. Liberty proposes that Qwest include a
diagnostic sub-measure to the MR-7 measure or create an entirely new MR measure that wil
report the number of lines and circuits that receive more than two trouble reports over a rolling
six month period to provide the users of the PID reports data on the number of chronic trouble
reports that CLECs are experiencing.
4. OP-3 - "Installation Commitments Met"
The OP-3 measure is intended to evaluate Qwest's abilty to install services for customers by the
scheduled due date. One of the inputs that Libert received from the CLECs is that Qwest does
not reliably meet coordinated installation appointments that it sets with the CLEC. The CLECs
pay a greater non-recurring installation charge for such appointments. To provide Qwest, the
CLECs and the states with the ability to monitor Qwest's performance on these coordinated
appointments, Liberty proposes that Qwest add a diagnostic sub-measure to the OP-3 measure.
This sub-measure would report the percent of coordinated appointments that Qwest is able to
meet.
F. Other PAP Changes
As noted, the decline in Tier 2 payments during the Study Period has been particularly
significant, and larger than that for Tier 1 payments. Most states rely on the Tier 2 payments to
provide the funds for administration of the PAP, since as for audits and studies such as this one.
A continued decrease in Tier 2 payments could leave insuffcient funds for PAP administration.
Because the purpose of the PAP is to help incent wholesale performance rather than provide
funds to the states, Liberty believes that an alternative means should be considered for funding
PAP administration activities in addition to the Tier 2 payments. For example, the CPAP has
provisions that in certain cases if the Special Fund created to hold the Tier 2 payments are
insuffcient to pay for certain PAP administration activities, Qwest would be assessed for the
cost. Liberty believes that a more general provision of this sort would be advisable in all the
PAPs.
Liberty also examined other aspects of the PAP structure for possible changes. Liberty's analysis
and review confirmed that, with some exceptions in Colorado, performance has a similar impact
on payments throughout the 11 states as discussed in Section II.D. However, the 11 state QPAPs
have many differences, and there is some value in eliminating these differences because the
differences add to the complexity of Qwest's PAP administration and tend to make it diffcult for
a CLEC operating in several states to understand the different PAP rules. Nevertheless, Liberty
does not recommend moving to a single uniform PAP across the Qwest operating region. The
differences evolved through specific proceedings in each state and were justified by the evidence
provided in those proceedings. In addition, the changes would require work on Qwest's part. The
PAPs are working well as they are, and because moving to a uniform PAP would not have a
major impact on results, Libert believes the cost of making such a change outweigh its benefits.
June 30, 2009 ~~
The Liberty Consulting Group
Page 83
Final Report Analysis of Qwests
Penormance Assurance Plans
Liberty examined other methods to simplifY the PAPs, and also concluded that the costs of
making the changes outweighed the benefits.
June 30.2009 ~~
The Liberty Consulting Group
Page 84
Final Report Analysis of Qwests
Penormance Assurance Plans
v. Summary of Conclusions and Recommendations
Liberty concludes that the PAPs are stil serving a useful purpose in all the participating states.
There continues to be a significant group of CLECs in the states that rely heavily on Qwest s
wholesale services to conduct their business and with few realistic alternatives. These CLECs
continue to provide significant competition for Qwest, paricularly in such important parts of the
market as broadband and business services. In addition, as Integra has pointed out, with the
merger of AT&T and MCI with Regional Bell Operating Companies, their traditional strong
advocacy for the interests of the CLEC community has significantly diminished. This enhances
the need for strong PAPs to protect the interests of the CLECs.
Although Liberty concludes that the PAPs should be maintained, some changes should be made
in the existing PAPs to simplify them and make them more targeted to the continuing and
evolving needs of the competitive marketplace. Most of these changes continue a process of
evolution of the PAPs since their inception to continue to tailor them to current needs. Liberty
considered a number of different possible changes, including additional whys to simplifY the
PAPs. Some approaches, such as eliminating certin measure disaggregations could not be
justified because they would tend to mask poor performance or might have the unwanted results
of increasing the PAP complexity. Some simplification approaches were rejected because the
potential benefits were minimal and would not justifY the potential cost of their implementation.
After considering the alternatives, Liberty developed the recommendations for PAP changes
outlined below.The detai led applicability of these proposals in each of the 11 participating states
is provided in Appendix C.
The following recommendations apply to all the paricipating state PAPs.
Recommendation 1. The Commissions should introduce a new aggregation mechanism to
minimize low-volume tests in determining payments. Specifcally, transactions for CLECs with
low volumes should be aggregated with those of other CLECs, and, as necessary, aggregated
over up to a three month period, for the purpose of determining non-conformance and
calculating payments.
Liberty's analysis reveals that a large number of the tests performed to determine PAP penalty
payments are based on CLEC transaction sample sizes which are very smalL. Such low-volume
tests can introduce statistical errors, either biasing the results against Qwest or against the
CLECs depending on the circumstances. Furthermore, the relative biases are not likely to be
balanced. Libert considered several structural changes to the PAPs which could have reduced
the number of low-volume tests, but concluded that aggregation primarily over CLECs and
secondarily over time would be the best way to avoid unnecessary complexity in the PAP
mechanism.
In Liberty's proposal, payments with low-volume CLEC transactions would be determined
through the following steps:
1. Aggregate transactions for all CLECs that have less than ten transactions in a
month for any given sub-measure disaggregation.
June 30,2009 ~~
The Liberty Consulting Group
Page 85
Final Report Analysis of Qwest's
Penormance Assurance Plans
2. If the outcome of this CLEC aggregation equals or exceeds ten transactions, use
the aggregate result for these CLECs to calculate whether penalty payments are
required.
3. Distribute any penalty payments to the aggregated CLECs based on each CLEC's
relative share of the total number of misses.
4. If the aggregate total does not exceed ten transactions, then car forward the
aggregate result to the following two months until either the threshold of ten
aggregate transactions is met or three months of results data have been used in an
attempt to meet the minimum volume threshold. 76
5. Star the process again after either of these criteria has been met.
A more complete description of the analysis behind this recommendation is in Section N.B.
Recommendation 2. The Commissions should eliminate the following PID measures (in additon
to those included in the 2007 Stipulation recommendations) from consideration for PAP
payments for those states that use them, and place them on the list of measures subject to the
Reinstatement/Removal Process:
· PO-9 Timely Jeopardy Notices
· PO-19 SATE Accuracy
· PO-20 Manual Service Order Accuracy
· CP-1 Collocation Completion interval
· CP-2 Collocations Completed within Scheduled Intervals
· CP-4 Collocation Feasibilty Study Commitments Met.
This recommendation continues a process started with the CP AP Three-Year Review and 2007
Stipulation recommendations of simplifYing the PAPs and focusing them on the measures which
continue to assess those Qwest wholesale functions with the highest importance to a large class
of CLECs. A precondition for this recommendation is the introduction of the
Reinstatement/Removal Process into the PAP, as recommended in the 2007 Stipulation. Liberty
also reviewed the measures recommended in the 2007 Stipulation for removal from the PAP but
subject to the Reinstatement/emoval Process, and found that the rationale for this treatment is
stil valid. Because Montana has not yet adopted these recommendations, Libert believes
Montaa should adopt the 2007 Stipulation recommendations, as noted below in a separate
recommendation.
Libert chose the additional measures for PAP removal (PO-9, PO-19, PO-20, CP-l, CP-2, and
CP-4) based on the relatively small contribution to the PAP payments in all the states, the small
measured CLEC volumes, and the limited impact their removal would have on the CLECs'
abilty to serve their end-user customers. Liberty considered other measures for removal based
76 Liberty recommends treating aggregation across months as if it were a single month for payment escalation
purposes.
June 30, 2009 ~,~
The Liberty Consulting Group
Page 86
Final Report Analysis of Qwests
Penormance Assurance Plans
on relatively low recent payments but rejected their inclusion in the list of measure for removal,
largely because of the potential negative impact on the CLECs and their customers.
A more complete description of the analysis behind this recommendation is in Section IV.C.
Recommendation 3. The Commissions should make the following additional changes to certain
PID measures in the PAPs:
. For OP-5 (New Service Quality), use sub-measure OP-5T instead of sub-
measures OP-5A and OP-5B.
. Replace the current retail analog of "retail ISDN-BRJ designed" with some other
retail product or with a benchmark.
The change for OP-5 has the advantage of avoiding unnecessarily disaggregating the orders
examined for new service quality into two classifications, whether troubles were repair center
trouble reports (OP-5A) or provisioning trouble reports (OP-5B). Combining these two
classifications in OP-5T helps minimize the low-volume tests mentioned in Recommendation 1.
The standard for OP-5T would be the same parity standards that are used for the OP-5A
measures, as explained in Section iv -C-4.
Liberty observed that a number of wholesale products use retail ISDN-BRI designed as the retail
analogue. This occurs in the following measures:
. OP-3 - "Installation Commitments Met"
ISDN Capable Loop
. OP-4 - "Installation Interval"
ISDN Capable Loop
. OP-5A - "New Service Quality"
Non-Loaded 2-Wire Loop
ADSL Qualified Loop
ISDN Capable Loop
. OP-6 - "Delayed Days"
Non-Loaded 2- Wire Loop
XDSLI Capable Loop
ADSL Capable Loop
ISDN Capable Loop
. OP- 15 - "Interval for Pending Orders Delayed Past Due Date"
Non-Loaded 2-Wire Loop
ADSL Qualified Loop
ISDN Capable Loop
However, retail ISDN-BRI designed frequently has an insufficient number of order transactions
to use in the conformance tests to determine payments. As a result, for the wholesale products
using this retail analogue, it is often impossible for the tests to fail and a payment to be made.
June 30, 2009 ~~
The Liberty Consulting Group
Page 87
Final Report Analysis of Qwests
Penormance Assurance Plans
Liberty recommends that a Qwest-CLEC collaborative determine the appropriate alternative
standard to retail ISDN-BRI designed for the measures and wholesale products listed above.
A more complete description of the analysis behind this recommendation is in Section IV.C.
Recommendation 4. The Commissions should eliminate the following low-volume products from
the OP and MR measures in the PAPs:
· Unbundled Digital Signaling Level 3 (DS-3) Loops
· Unbundled Dedicated Interoffce Transport (UDIT) - Above DS1
. Unbundled 4- Wire Non-Loaded Loops
. Loops with Conditoning (applies only to OP measures)
· Unbundled ISDN Capable Loops (applies to all states and measures except for
MR measures in Arizona and Colorado)
. Line Sharing (already removed in Colorado).
In addition to the low-volume products eliminated in the CPAP Three-Year Review and through
the 2007 Stipulation recommendations, Libert has identified these other products with
transaction volumes that are too small to warrant continued inclusion in the PAP payments tests.
These products would stil continue to be monitored through the PID reports. As with
Recommendation 2, this recommendation assumes that the products recommended for removal
in the 2007 Stipulation have also been removed. Liberty reviewed the products in the 2007
Stipulation recommendation and agrees they should be removed. Because Montana has not yet
adopted these 2007 Stipulation recommendations, Liberty believes Montana should adopt the
2007 Stipulation recommendations, as noted below in a separate recommendation.
The analysis supporting this proposal is described more fully in Section IV.D.
Recommendation 5. The Commissions should make the following additional changes to certain
PID measures:
· Limit MR-4 (All Troubles Cleared within 48 Hours) to service-affecting troubles
· Add a diagnostic sub-measure to OP-4 (Installation Interval) to measure
performance on expedited orders
· Add a diagnostic sub-measure to MR-7 (Installation Interval) to measure chronic
troubles
. Add a diagnostic sub-measure to OP-3 (Installation Appointments Met) to the
percentage of coordinated appointments met.
Some of these changes wil make the reported results more meaningfuL. Others add sub-measures
to monitor Qwest's service quality for activities that CLECs indicate are important to their
June 30, 2009 ~~
The Liberty Consulting Group
Page 88
Final Report Analysis of Qwests
Performance Assurance Plans
business and are not monitored today. These new sub-measures would be diagnostic and allow
evidence to be developed as to whether Qwest's performance for these activities warants
inclusion ofthe sub-measures in the PAP.
This recommendation is described more fully in Section IV.E.
Recommendation 6. The Commissions should adopt provisions to assess Qwest for the cost of
PAP administration functions, including independent auditor and audit costs and payment of
other expenses incurred by the participating Commissions in the regional administration of the
PAP, if the Special Funds created by the Tier 2 payments are insuffcient for fund these
functions.
In order for the PAPs to be effective, the Commissions need to have resources for administering
them. This includes funds for such activities as audits and special studies to support the regular
reviews of the PAPs. Most, but not all, of the PAPs call for the Tier 2 Special Funds to be used
for this purpose.77 With the decline of Tier 2 payments, there is a possibilty that the Special
Funds established to fund these activities could soon be exhausted. The approach of assessing
Qwest directly for such costs is already par of the CP AP provisions in certain circumstances.
This approach should be applied more broadly. In cases where there are no provisions for PAP
administration funding, Liberty recommends adopting the necessar provisions.
The following recommendation applies to all participating states except Colorado and Utah.
Recommendation 7. The Commissions should adopt changes in the PAPs and PID to recognize
Qwest's replacement of the Electronic Data Interchange (EDI) interface by the Extensible Mark-
up Language (XML) interface.
Most state PAPs still involve monitoring and PAP payments based on use of the EDI interface
for ordering and pre-ordering. Qwest has now phased out use of this interface and replaced it
with an XML interface. This means that the PAPs no longer have the abilty to generate
payments based on failures of Qwest to provide ordering and pre-ordering through an e-bonded
interface. This involves replacement of the language in PID document Version 9.0 related to
these interfaces with the language introduced in Version 9.1. In PID Version 9.1, GA-2, which
measured the availabilty of ED I, has been dropped and replaced with GA-8, which measures the
availabilty of the XML database. Version 9.1 also replaces the EDI interface with the XML
interface in the following measures:
. PO- 1 - "Pre-Order/Order Response Time"
. PO-2 - "Electronic Flow-through"
. PO-3 - "LSR Rejection Notice Interval"
. PO-4 - "LSRs Rejected"
. PO-5 - "FOCs On Time"
77 This is usual1y speci fied in paragraph 11.3 of the PAP.
June 30,2009 ~,~
The Liberty Consulting Group
Page 89
Final Report Analysis of Qwest's
Penormance Assurance Plans
. PO-6 - "Work Completion Notification Timeliness"
· PO-7 - "Biling Completion Notification Timeliness"
. PO- 16 - "Timely Release Notifications"
. PO- 19 - "SA TE Accuracy"
· PO-20 - "Manual Service Order Accuracy"
In addition, references to GA-2 need to be replaced by GA-8 and other references to EDI need to
be changed in the PAP.
The Colorado and Utah Commissions have already adopted these changes.
The following two recommendations apply only to Colorado.
Recommendation 8. The Colorado Public Utilties Commission should restore the Tier 1 B, Tier
1 C, and Tier 2 mechanisms to the CP AP, subject to the changes required by Liberty's other
recommendations.
The current version of the CPAP has implemented a sunset provision which automatically
eliminates the Tier 1 B, Tier 1 C, and Tier 2 mechanisms after six years. Although there are a
number of aspects of this change which are consistent with Liberty's generally applicable
recommendations above, there are some products and measures eliminated through this change
which Liberty stil considers to be important for inclusion in the CPAP. In particular, many of
the Tier 1 Band 1 C measures have been removed from the Colorado Reinstatement Process list.
The list went from 16 measures that could be reinstated if Qwests performance was not in
conformance with the established standard for three consecutive months to only five remaining
measures. Eleven measures have essentially been removed from the PAP forever by this change.
Additionally, all biling measures and all regionally measured measurements (e.g., all GA
measures, PO- 1, etc) have been removed from the PAP.
Recommendation 9. The Colorado Public Utilties Commission should make the following
additional changes to the CP AP:
· Restore the Unbundled A DSL-Capable Loop product
· Eliminate the UNE-P products.
After the Three-Year Review, the Colorado Commission eliminated Unbundled ADSL-Capable
Loop as a product. At the time of the Three-Year Review, the order volume for this product was
very small. However, since that time, there has been a significant increase in the volumes for this
product. It has become an important product for certain CLECs to provide broadband service.
Therefore, Liberty recommends restoring this product to the CPAP.
UNE-P was delisted as a UNE by the FCC in the TRO and TRRO decisions. The UNE-P
products (UNE-P POTS, UNE-P Centrex, and UNE-P Centrex 21) were eliminated in those
QPAPs that have adopted the 2007 Stipulation recommendations. However, they were not
June 30,2009 ~,~
The Liberty Consulting Group
Page 90
Final Report Analysis of Qwests
Penormance Assurance Plans
eliminated in the CPAP. Because these products are now obsolete for PAP purposes, Liberty
recommends that they be removed from the CP AP.
The following recommendation applies to Montana only.
Recommendation 10. The Montana Public Service Commission should adopt the
recommendations of the 2007 Stipulation.
Liberty has reviewed the recommendations of the 2007 Stipulation and finds them to be
appropriate. All participating Commissions except Montana's have adopted most of these
recommendations. Of particular relevance are the following recommendations:
. Introduction of the Reinstatement!emoval Process with application to the
following measures (for Montana):
o GA-3 Gateway Availabilty EB-TA
o GA-4 System Availabilty EXACT
o GA-7 Timely Outage Resolution Following Softare Releases
o PO-3 LSR Rejection Notice Interval
o PO-50 FOCs On Time (ASRs for LIS Trunks)
o PO-7 Biling Completion Notification Timeliness
o PO-8 Jeopardy Notice Interval
o PO- 1 6 Timely Release Notifications
o OP- 17 Timeliness of Disconnects Associated with LNP Orders
o MR- 11 LNP Trouble Reports Cleared within25 Hours
o 81-4 Biling Completeness
o NI- 1 Trunk Blocking
o NP- 1 NXX Code Activation
. Elimination of the following low-volume products from consideration in
determining PAP payments but continue to report them in the PID reports:
o Resale Centrex
o Resale Centrex 21
o Resale Frame Relay
o Resale Private Branch eXchange (PBX) (non-designed and designed)
o Resale ISDN-BRI (non-designed and designed)
o Resale ISDN Primar Rate Interface (PRJ) (non-designed and designed)
o Resale Digital Signaling Level 0 (DSO) (non-designed and designed)
o Resale DSL (designed)
o Sub-Loop Unbundling (except in Colorado)
o UNE-PPOTS
o UN-P Centrex
June 30,2009 ~,~
The Liberty Consulting Group
Page 91
Final Report Analysis of Qwest's
Performance Assurance Plans
o UN-P Centrex 21
o E91 1/91 1 Trunks
· Introduction of the One Allowable Miss Mechanism for low-volume benchmark
and non-interval parity measures
June 30.2009 ~,~
The Liberty Consulting Group
Page 92