2680 journal of lightwave technology, vol. · pdf file · 2016-01-07paper addresses...

2680 JOURNAL OF LIGHTWAVE TECHNOLOGY, VOL. 22, NO. 11, NOVEMBER 2004

Metro Network Design Methodologies That Build aNext-Generation Network Infrastructure Based on

This Generation’s Services and DemandsRonald Skoog, Member, IEEE, Member, OSA, Ann Von Lehmen, Member, IEEE, Member, OSA,

George Clapp, Member, IEEE, Joel W. Gannett, Senior Member, IEEE, Haim Kobrinski, Member, IEEE, andV. Poudyal, Member, IEEE

Invited Paper

Abstract—This paper describes two key network architecturedesign concepts that relate to evolving existing transport networksinto economically viable next-generation optical networks. Today’smetropolitan transport networks largely consist of synchronousoptical network/synchronous digital hierarchy rings or switch-to-switch fiber connections for some form of optical Ethernet. The re-sult is an optical–electrical–optical infrastructure that has limiteduse in providing wavelength services. Wavelength-division multi-plexing (WDM) is the enabling technology for wavelength services,but it has limited penetration in the metropolitan area due to itscost justification being dependent primarily on fiber relief. Thefirst part of this paper shows how existing services, primarily usingtime-division-multiplexing (TDM) transport, can be used to eco-nomically justify a WDM infrastructure while achieving signifi-cantly lower costs than legacy design techniques would produce.Dynamic bandwidth-on-demand (BoD) service is another elusivegoal envisioned for next-generation metropolitan networks. Thispaper addresses how an economically viable BoD infrastructurecan be built based on revenues from existing enterprise servicesrather than relying on revenues from new and unproven services.Quantitative analyses, presented in the paper, show the key param-eters that determine when BoD services will be used, how band-width granularity affects BoD decisions, and how the customer’suse of BoD drives service provider network design considerations.

Index Terms—Bandwidth-on-demand (BoD), network design,next-generation networks, synchronous optical network/syn-chronous digital hierarchy (SONET/SDH) networks, wave-length-division-multiplexing (WDM) networks.

I. INTRODUCTION

FOR SOME TIME, a common assumption has been thatthe role of optical networking is to support extraordinary

growth of existing services and enable new large-bandwidthapplications. However, for this vision to become a reality, it isnecessary that both the envisioned new services are econom-ically viable and there is an economically feasible path fromthe present networks to those that enable the new services.

Manuscript received February 4, 2004; revised July 21, 2004. This work wassupported in part by the Laboratory for Telecommunications Sciences, Adelphi,MD.

The authors are with Telcordia Technologies, Inc., Red Bank, NJ 07701 USA.Digital Object Identifier 10.1109/JLT.2004.836748

Metropolitan networks play a critical role in the overall expan-sion of network services. They not only provide for serviceswithin individual metropolitan areas, but they also serve asthe gateways for wide-area national- and international-scalenetworks.

There is considerable activity today in the research andeducation (R&E) community in exploring the possibilitiesfor optical networking and the applications they can enable,and many metro, regional, national, and international R&Enetworks are being deployed (e.g., National LambdaRail,1

CA*net4,2 SURFnet,3 and TransLight [7]). These networks aresupported in large part by government funding and thereforedo not have the same financial constraints as commercialcarrier networks. Commercial carriers are currently reluctantto make major infrastructure investments for futuristic servicesand uncertain revenues. This creates a “chicken-and-egg”problem in the deployment of commercially viable next-gen-eration optical networking. This paper proposes to resolve this“chicken-and-egg” dilemma by using normal growth in existingservices to support the cost of building the next-generation op-tical network infrastructure. In particular, two critical areas formetropolitan networks are explored: deploying wavelength-di-vision-multiplexing (WDM) infrastructure in metropolitanareas based on growth in time-division-multiplexing (TDM)services and deploying a bandwidth-on-demand (BoD) infra-structure based on existing enterprise network services.

II. WDM DEPLOYMENT IN METRO NETWORKS

WDM is an enabling technology for our network de-sign method, in which normal growth in traditional TDMservices triggers deployment of a WDM infrastructure asthe most economic solution. The WDM infrastructure canalso be used to provide non-TDM and wavelength services.Traditional synchronous optical network/synchronous dig-ital hierarchy (SONET/SDH) metro network design hasbeen based on building either flat (noninterconnected) ring

1www.nationallambdarail.org/2www.canarie.ca/canet4/3www.surfnet.nl/en/

0733-8724/04$20.00 © 2004 IEEE

SKOOG et al.: METRO NETWORK DESIGN METHODOLOGIES 2681

Fig. 1. SONET/SDH transport architectures. (a) UPSR/BLSR. (b) PTP (PTP). (c) PTP with multiple hubs.

network architectures [4] or hierarchical (interconnected)ring network architectures. The SONET/SDH rings can beeither bidirectional-line-switched rings (BLSRs) or unidi-rectional-path-switched rings (UPSRs). The optimal choicebetween BLSR and UPSR depends on the point-to-point loadpattern, with BLSR being used when the loads are mesh-like,and UPSR being used when the loads are primarily to hubnodes. These traditional SONET/SDH ring network designs donot stimulate WDM deployment. This has been recognized forsome time, and extensive work has been done to address thisproblem by exploring a combined WDM and SONET/SDHring design [8]–[10].

The problem with most of these approaches is that the net-work models considered are as shown in Fig. 1, where a singlefiber ring is considered, and thus the SONET/SDH and WDMrings are very tightly coupled, i.e., they share the same fiber-ring structure. In real networks, the fiber network is a meshnetwork, and an important consideration is where to place theSONET and WDM rings on that fiber mesh infrastructure. Wepropose to decouple the SONET/SDH and WDM rings by notrequiring that they be placed on the same fiber ring. Note thatthe SONET/SDH ring needs a bidirectional lightpath betweeneach adjacent pair of add–drop multiplexers (ADMs), but thelightpaths for the different adjacent ADM pairs can come fromdifferent WDM rings.

The basic idea of our WDM-based design technique is toaggregate TDM loads so that well-utilized lightpaths are cre-ated edge-to-edge, edge-to-hub, and hub-to-hub (the use of hubnodes is shown hereafter to be very important). The PTP (PTP)lightpaths created from this aggregation process are used to de-sign the WDM network infrastructure.

A. Design Methodology Concepts

In our network design methodology, we consider the servicedemands to be TDM PTP loads, and we concentrate on thegrooming and aggregation of SONET STS-1s and higher PTPloads into OC-192/OC-48 channels that are transported directlyon WDM lightpaths. This assumes that lower level TDM trafficdemand, e.g., DS1/VT1.5, has already been aggregated and

channeled into STS-1s using an efficient edge grooming/hub-bing methodology.

PTP TDM traffic demand can be transported at theSONET/SDH layer using three alternative network architecturesolutions: direct PTP, distributed grooming using SONET/SDHring ADMs, or centralized grooming using SONET/SDHdigital cross-connect systems (DCSs). Any of these solutionsmay be the most efficient depending on variables such as de-mand distribution, demand level, transport system technology,transport system capacity, and geography.

Fig. 1 illustrates the three basic types of network architec-ture. For simplicity, all the nodes are shown physically con-nected on a fiber ring (solid black lines). The dashed lines in-dicate the bidirectional lightpath connections. The ADMs andDCSs are assumed to have STS-1 (DS3) grooming capabilities,i.e., broad-band DCSs (BDCSs) or grooming optical cross con-nects (OXCs).

The first stage of the design process is to aggregate sublight-path demand, e.g., STS-1s, between the same endpoints ontowell-filled4 lightpaths between those two endpoints. This corre-sponds to building the PTP lightpaths in the diagram in Fig. 1(b).

The second stage involves a process of establishing hubsand building well-filled lightpaths between edge nodes and thehubs and between hubs. This corresponds to the PTP lightpathsin the diagram in Fig. 1(c). The primary objective of this andthe next stage is to reduce the amount of electronic equipmentin the network (either cross connecting at a DCS or havingSONET/SDH channels that pass through an ADM’s electronicfabric). Modiano and Berry developed results in [5] that providethe main insights as to how this hub selection process shouldbe done. Their analysis assumed uniform PTP load betweenall nodes, and while that is not realistic, the results providean intuitive grounding as to how to build heuristics for thenonuniform case. The major points to extract from [5] are asfollows.

1) The basic design strategy is to groom traffic to maximizethe amount of single-hop and double-hop routing, i.e.,avoid three-hop edge-to-hub/hub-to-edge routing.

4What is considered “well filled” from an economic standpoint depends onfactors such as the growth rate and the interest rate reflecting the cost of capital.Typically, it is around 75% or higher.


Fig. 2. High-level view of metro design methodology.

2) The number of lightpaths out of an edge node should beno more than that required to carry the aggregate trafficfor that node.

3) The optimum number of hubs5 is equal to the number oflightpaths required to carry the aggregate load from anedge node; each edge node has one lightpath to each hub.

The general strategy in developing heuristics for the nonuni-form case is to select as hubs those nodes with the highest ag-gregate load. This will tend to maximize single-hop routing.Maximizing single-hop routing is also the reason to use mul-tiple hubs when aggregate loads out of edge nodes grow beyonda single lightpath. It is important to note that if aggregate loadsare sufficiently small, SONET/SDH rings (BLSR or UPSR) aremore efficient than a hubbed design. Therefore, we use hubswhen we can fill edge-to-hub lightpaths efficiently; if not, weuse SONET/SDH rings.

B. Metropolitan Network Design Methodology

Based on the intuitive insights outlined in the previous sub-section, we developed heuristic algorithms based on load ag-gregation and hubbing concepts to achieve network designs thatcould significantly reduce costs by utilizing WDM technology.The hubbing strategy has the advantage of reducing the cost ofelectronics, but it consumes more lightpaths than other typesof design. If WDM were not used, the fiber costs would makemost hub solutions uneconomical.6 Therefore, WDM becomesan enabling technology for the new network design method, inwhich traditional TDM demand growth triggers economic de-ployment of a WDM infrastructure that can also be used to pro-vide non-TDM and wavelength services.

The new design method is geared toward a target network ar-chitecture that provides an overall macro view of how the net-work should evolve and the primary strategies to be used for

5This result requires uniform load and the aggregate lightpaths required outof an edge node to be less thanN=2(N = umber of nodes). A recursive designmethod is used to handle the resulting hub-to-hub load.

6If a carrier has large amounts of unused fiber available, then the “cost” offiber might be viewed as very low and WDM would not be seen as cost effective.In that case. the hubbing strategy would be implemented directly over fiber.

Fig. 3. Node clustering and hub selection.

cost optimization. The structure of the target architecture con-sists of SONET rings (primarily at the network edges wheretraffic levels from a single node do not justify lightpaths), a fewoptimally selected nodes with grooming OXCs, and WDM PTPand ring systems. The target architecture is implemented usingnext-generation SONET (NG SONET) and optical networkingequipment such as NG ADMs, grooming OXCs, OTMs (op-tical terminal MUXes), and fixed/reconfigurable optical ADMs(OADMs). Fig. 2 is a high-level schematic representation of themethod that we developed to optimize a network design basedon this overall target network architecture.

The major steps illustrated in Fig. 2 for designing a specificnetwork and are explained in detail hereafter.

1) Aggregation and Clustering: This is a preprocessing stepdone to the demand data before submitting the data toa commercial WDM network design tool, thereby influ-encing the tool to produce a more economical solution.

a) Partition the entire set of nodes into geographicallydistinct “clusters” of nodes. Within each cluster,select a single “hub node” for STS-1 grooming(Fig. 3). The clustering can be determined by auto-matic graph partitioning techniques applied to thephysical link graph, the demand graph, or someweighted combination of the two. We found thatperforming clustering was more important than theexact details of the clustering. The hub nodes can


be chosen by a center-of-gravity technique. For ex-ample, for the physical link graph, one can choosethe node whose total distance to all other nodes inthe cluster is smallest.

b) Choose the desired “fill” thresholds for direct con-nections and for routing through the hub nodes. Inmost of our experiments, we chose a common valueof 75% for all thresholds.

c) Create a new database of demands based on thethresholds and the selected hubs to generate light-path (optical channel) requirements for the WDMnetwork:

• First, create the direct PTP wavelength pathsbetween any two nodes with more traffic thanthe direct threshold. These paths can be cre-ated both intra- and intercluster.

• Next, create the node-to-foreign hub wave-length paths. These are built when the sum ofthe traffic between a node in a cluster and allof the nodes in a foreign cluster is greater thanthe node-to-foreign hub threshold.

• Next, create the node-to-local hub wave-length paths. These are built when the sum ofthe traffic between a node and all of the othernodes in the same cluster is greater than thenode-to-local hub threshold.

• Recursively apply the previous three steps tobuild the interhub network.

• What is left over are the nonwavelength de-mands.

2) Design an Optical Network: To cost-effectively route boththe optical channel requirements of the electronic layer(obtained as a result of aggregation and clustering) andthe “wavelength” services required by clients:

a) determine optimal WDM ring and terminal systemlayout and capacity;

b) use metro level criteria—cost, survivability, percentdemand satisfied, future growth capacity, etc.

3) Design a SONET Network: To carry nonwavelength de-mands:

a) determine the optimal SONET system layout andcapacity;

b) use the metro level criteria: cost, survivability, per-cent demand satisfied, future growth capacity, etc.

4) Iteratively Evaluate Both the DWDM and SONET Sys-tems:

a) Determine if SONET systems would be a better fitthan any of the DWDM systems designed.

b) Determine if SONET systems integrated withDWDM can provide better efficiencies.

c) Use metro level criteria: cost, survivability, percentdemand satisfied, future growth capacity, etc.

C. Results for Metropolitan Design Case Study

A case study was done for a metropolitan network in thethe United States [6]. In this study, it was found that the new

Fig. 4. Sample comparison of PMO and new method.

design method consistently produced lower cost network plansover a wide variety of nextgeneration and legacy SONET/SDHequipment options. In comparing the optimal legacy design(SONET/SDH-ring-based designs) to the optimal design fromthe new design method, a savings of approximately 16% wasachieved from the new design method. Fig. 4 shows a relativecost comparison between the present mode of operation (PMO)and the new design methodology. Results are shown for twotypes of SONET/SDH equipment.

In the designs using the new method, the O/E/O costs werepredominantly incurred at the edge as terminal MUX equip-ment and at hub nodes for cross-connect capability (BDCSs orgrooming OXCs). The remainder of the network was mostlyWDM equipment. Thus, the new design method not only gen-erated significant cost savings, it also established a WDM infra-structure. If a flat SONET/SDH ring network design was used(i.e., no hubbing and limited ring interconnect), it was found thatthat solution had about 39% more cost than the optimal solutionusing the new design methods.

Sensitivity analyses were performed on many variables, andthe cost of WDM systems was found to be a significant contrib-utor to the overall cost. This will affect the decision of how muchWDM to use compared with more fiber and/or next-generationSONET/SDH systems. However, the cost variance and othersensitivity analyses performed did not alter the main results ofthe study, namely, that performing a metrowide optimization ofthe network using hubbing and lightpath aggregation results in abetter optimized design incorporating both WDM and next-gen-eration SONET/SDH.

III. BANDWIDTH-ON-DEMAND DEPLOYMENT

BoD has been an elusive goal for many years. Frame relayand asynchronous transfer mode (ATM) have both had the capa-bility to provide switched bandwidth services, but no such ser-vices have emerged from either technology. Multiprotocol labelswitching (MPLS) is establishing a firm footing in Internet pro-tocol (IP) networks, but it is not being used for dynamic band-width service capability. Rather, it is being used to establishstatic tunnels for virtual private networks (VPNs) and IP net-work traffic engineering.

Now we have generalized MPLS (GMPLS) from the In-ternet Engineering Task Force (IETF), automatically switched


Fig. 5. Enterprise network model.

transport network (ASTN), and automatically switched opticalnetwork (ASON) standards from the International Telecommu-nications Union–Telecommunication Standardization Sector(ITU-T), and optical User Network Interface (UNI 1.0) andNetwork Node Interface (NNI) implementation agreementsfrom the Optical Internetworking Forum (OIF). Many vendorshave implemented some combination or subset of these stan-dards, and virtually all vendors have such capabilities on theirproduct roadmaps. Therefore, the technology will likely onceagain exist to provide flexible BoD services, but what will ittake to make such services a reality?

The approach that we examine here is to establish the infra-structure for such services based on an existing service baserather than relying on new and unproven services. In line withthe view that is emerging from the R&E community [1], we en-vision a networking layer that will be a hybrid packet–opticalinfrastructure (HOPI). This means that the networking layer willprovide various levels of bandwidth granularity for bandwidthmanagement, ranging from the very fine granularity of best-ef-fort IP packet networking to a very coarse granularity of fullwavelengths and groups of wavelengths. In this paper, we di-vide the bandwidth granularity into fine and coarse. Fine gran-ularity relates to establishing connections with IP/MPLS pathsor SONET/SDH channels using virtual concatenation and allo-cating bandwidth increments in the range of 1–150 Mb/s. Coarsegranularity refers to full wavelengths, which would typicallysupport bandwidths in the range of 1–10 Gb/s in today’s net-works.

A. Fine Granularity Bandwidth Management

In this section, we examine the fundamental forces that willdrive the deployment decisions and network design methodsregarding the network resources used to provide fine granularBoD capability for enterprise networks. Specifically, we con-sider how enterprise user sites would use fine granular BoD ca-pabilities, and we identify the key parameters and issues thatshould be considered in network design to achieve efficient BoD

facility usage while meeting enterprise network performance re-quirements (e.g., latency, jitter, and loss).

We first look at the potential user sites for BoD capability andexamine how they would make decisions based on economictradeoffs between dedicated connections (e.g., traditional pri-vate line service) and BoD connections. We then examine theBoD service provider’s problem of providing efficiently utilizedtransport facilities for the traffic flows generated by the BoDusage.

The main results of this section are as follows.

1) In the majority of cases, the most cost-effective enterprisenetwork design is to use a mix of dedicated and BoD con-nections. The dedicated bandwidth connections are usedto provide a “base level of bandwidth” that is known to beneeded between user sites. BoD is used to provide band-width capacity that exceeds the “base level bandwidth.”

2) To achieve the efficiencies needed to make BoD a cost-ef-fective capability, the BoD service provider needs to ag-gregate user BoD traffic flows so each transmission sec-tion (link) supports many BoD traffic streams. Our studiesshow that the number of traffic streams, rather than thesize of the streams, is the critical parameter.

3) The cost and utilization of the dedicated facilities used toaccess BoD has a significant impact on the amount of BoDthat will be used. As the cost of dedicated BoD access be-comes a larger portion of the total BoD costs, the averageutilization of that dedicated access must increase for theuse of BoD to be cost effective.

4) The bandwidth granularity used for BoD has a significantimpact on the cost efficiency of facilities used to provideBoD, with finer granularity providing greater cost effi-ciency (i.e., higher average utilization).

1) Network and Traffic Model: Fig. 5 illustrates the networkmodel used. We consider an enterprise network with multiplesites. Each site supports a number of users that generate demandfor bandwidth between their site and each of the other enter-prise sites. Each site has dedicated access bandwidth capacity


Fig. 6. Model of site-to-site bandwidth demand.

to a carrier network that can provide both private-line and BoDservices.

The bandwidth requirements between sites vary stochsticallywith user activity. We approximate the varying bandwidthrequired by the enterprise users by assuming bandwidth requestscome in a fixed discrete bandwidth capacity unit (e.g., abandwidth capacity unit could be a SONET STS-1 or VT1.5or an IP/MPLS label-switched path (LSP) allocated 1 Mb/s).Each requested bandwidth unit is used for a random holding time(its average is denoted by ). The idea is to provide sufficientbandwidth over time to meet user performance objectives, andthe required bandwidth is measured in multiples of the discretebandwidth unit. Fig. 6 illustrates this bandwidth model. Thecontinuous curve represents the bandwidth needed to meetusers’ performance requirements. The stair-step curve showsthe number of bandwidth units needed to meet those userrequirements. Below the graph is shown the random arrivalsand departures of bandwidth unit requests that generate thestair-step curve.

The discrete bandwidth unit is called the granularity of theBoD capability. We assume the aggregate user behavior israndom (e.g., user behavior is not correlated) so the discretebandwidth unit requests arrive as a Poisson process7. We assumethat the average arrival rate of bandwidth requests betweentwo sites is the same for all site pairs, and we denote thisaverage arrival rate by . The offered load between each node

7Extensive work in recent years (e.g., [11]–[13]) has shown that datanetwork traffic is not Poisson and exhibits long-range dependence. Morerecent work [14] has shown that time-dependent (piecewise-linear) Poissontraffic characterization can be used for Internet traffic. We use the stationaryPoisson assumption here to keep the analysis tractable so we can identifythe key parameters and basic tradeoffs involved with this problem. Moresophisticated techniques would need to be developed to generate the kindof accurate quantitative results needed for detailed network engineering.

pair is then the quantity , which we denote by . isa dimensionless quantity expressed in units called Erlangs.

2) The Decision Regarding If and How Much BoD to UseBetween Enterprise Sites: Assume there is some amount ofdedicated bandwidth capacity between a pair of enterprisesites. All bandwidth requests are first offered to the dedicatedbandwidth and served by it if capacity is available. If alldedicated bandwidth is in use, the bandwidth request is offeredto the BoD service. The BoD service is designed to meeta specified blocking probability assumed . Fig. 7shows how the bandwidth unit requests shown in Fig. 6 areserved by the dedicated capacity and the BoD service. Thethick arrows show the bandwidth unit requests that are sentto the BoD service.

To determine how much dedicated capacity and how muchBoD service to use, we need a cost model. The cost of BoDservice between a node pair is charged on a per-unit-timebasis per bandwidth unit, and the cost-per-time unit is denotedby . The cost of dedicated bandwidth is a fixed costper month per bandwidth unit, and dividing that fixed costper month by the number of time units per month gives anequivalent cost per unit time for dedicated bandwidth; thisis denoted as . An important quantity is the cost ratio(CR), defined as . It is easily seen that CR .This follows from the fact that dedicated capacity would beobtained from long-term contracts and thus would have adiscount in price from BoD capacity.

To determine the optimal choice for the amount of dedicatedbandwidth that should be used between a pair of user sites, thetotal cost is derived for the number of dedicated bandwidth unitsand the BoD costs to handle the load that is blocked by the ded-icated bandwidth and is provided over the BoD channels. Themodel is shown in Fig. 8.


Fig. 7. Dedicated and BoD bandwidth unit requests.

Fig. 8. Determining the optimal amount of dedicated capacity.

The blocking probability , at the dedicated band-width, with bandwidth units and offered load , is given bythe Erlang formula [2]

The analysis to determine the optimal choice for the numberof dedicated bandwidth units follows the method for sizingdirect trunk groups in telephone networks using hierarchicalrouting [3]. It simply involves finding the integer value ofthat minimizes the cost function derived in Fig. 8, namely tofind the integer value of that minimizes

The result of this type of analysis is illustrated in Fig. 9. Fig. 9shows that the two parameters that determine if and how muchBoD service is used are and CR. When the CR is very low,BoD looks expensive compared with dedicated capacity, and soif the CR is small enough, no BoD is used. At the other ex-treme, if the CR is close to 1, the BoD cost is close to the cost ofdedicated bandwidth, and because BoD incurs costs only whenit is used, BoD is less expensive than using dedicated band-width, and no dedicated bandwidth is used. However, it is seen

Fig. 9. Customer decision regions for BoD.

that there is a fairly wide region where both BoD and dedicatedbandwidth are used.

3) The Problem of Providing Facilities for BoD: Fig. 10shows the situation that needs to be considered to provideadequate capacity for BoD. Consider a transmission section(link) on which BoD channels need to be provided. The figureillustrates that there are site pairs that would need BoDcapacity on this transmission section. Each site-pair load is firstoffered to the dedicated capacity, and if all dedicated capacity isbusy, it overflows to the BoD channels. The BoD provisioningproblem is to determine how much bandwidth is needed to meetthe BoD blocking objective of 1% blocking. The overflow loadfrom each site pair is a bursty non-Poisson arrival process. Tosize the required bandwidth to provide for a number of overflowstreams as shown in Fig. 7, we use the Wilkinson equivalentrandom method [2].

Fig. 11 shows the average utilization of the carrier’s BoD in-stalled bandwidth as a function of the number of customer pointpairs providing overflow load to the BoD facilities. We have as-


Fig. 10. Carrier BoD traffic model.

Fig. 11. Utilization of carrier’s BoD facilities.

sumed here all point pairs have the same offered load . Utiliza-tion curves are shown for the point-pair-offered loads of 10, 20,and 30 Erlangs. The main conclusion that can be drawn fromthis result is that there needs to be a large number of point pairsproviding overflow traffic if reasonable utilizations areto be achieved. In addition, it is seen that the value of the offeredload has little effect on the utilization. That is, the number ofpoint pairs overflowing is the critical parameter that determinesa BoD carrier’s bandwidth efficiency. This is because as isincreased, most of the increased offered load is carried by ded-icated facilities, and the overflow load grows much slower than

’s growth. Thus, the BoD network needs to be designed so thatmany BoD traffic flows share the same transmission sections.

4) The Effect of Dedicated BoD Access Costs: The previ-ously mentioned results assumed that the dedicated BoD access

cost was negligible. If the dedicated BoD access cost is con-sidered, it can be shown that the previous results can be usedwith the CR being suitably modified to reflect the access costs.Fig. 12 illustrates a single enterprise site and shows how loadto each remote site is first offered to the dedicated capacity tothat remote site, and if it is blocked, the load is sent to a commonpool of dedicated BoD access channels. Let denote the costper unit time per bandwidth unit for the dedicated BoD access(this is the monthly cost of the access equipment divided by thenumber of time units per month). If the average utilization ofthose dedicated BoD access facilities is , then the usagecost per unit time for BoD access is , and thetotal usage cost for BoD service becomes . There-fore, the cost ratio including the cost of dedicated BoD access,denoted by CR , is CR CR . Thus, aseither increases or decreases, increases, whichin turn causes CR to decrease, and thus less BoD services tobe used.

The basic conclusion that can be drawn from these results isthat as the BoD access costs become more significant, the higherthe utilization of those dedicated access facilities need to be forBoD to be economical. Thus, if access costs are significant, in-frequent use of BoD (e.g., for failure or overload conditions)will not be cost effective.

B. Bandwidth Granularity and Network Capacity Issues

The bandwidth granularity used to provide BoD channelsis an important consideration. If the bandwidth incrementsare chosen to be very small, then the bandwidth assigned to


Fig. 12. Effect of dedicated BoD access facilities.

a site–site path can closely match the actual load, and thereis little “stranded capacity”; however, frequent bandwidthadjustments will need to be made. If bandwidth increments arechosen to be large, the frequency of bandwidth assignmentswill be reduced, but the assigned bandwidth will exceed theamount of bandwidth actually needed. The excess assignedcapacity is called “stranded capacity” because it is unavailablefor use by other site–site loads, and as a result, there will behigher blocking of BoD channel requests.

But even if the bandwidth of the channels is well matched tothe BoD requests (so the channels are well utilized once theyare established), and even if the number of channels on eachlink is well matched to the time-average offered load, morefundamental considerations show that a small implies highblocking. This follows because 1 out of 1 channel utiliza-tion states is the blocked state (i.e., all channels in use); hence, if

is small, the blocked state simply becomes more probable sta-tistically because there are fewer unblocked states. The curvesin this section quantify this effect.

Another aspect that needs to be considered in BoD networkdesign is the network load-carrying capacity. As the results inFig. 11 show, BoD facility utilization on a link increases as thenumber of site–site flows using that link increases. Thus, net-work designs that allow many site–site loads to share link ca-pacity will yield high link utilization. However, to achieve thatsharing, longer site–site paths may be required, and this resultsin more total bandwidth capacity being installed per site–siteunit of load. Thus, an important property of a network design isthe additional site–site load that can be handled by an increaseof a unit of bandwidth capacity on each network link.

To examine these issues, we carried out simulations on threerealistic local access transport area (LATA)-like networks ofsizes 19, 71, and 200 nodes. We assumed each link in the net-work has the same number of channels (the channel size isthe BoD bandwidth granularity). The model for site–site band-width unit requests coming into the network used exponentiallydistributed (Poisson) interarrival times and exponentially dis-tributed holding times. As each bandwidth request arrives to the

network, its user site endpoints are assigned randomly such thatthe expected time-average number of connections terminatingat a node is proportional to the degree of that node.

The total load offered to the network is normalized to thenumber of channels per link; specifically, the network offeredload is expressed as Erlangs per channel. For a fixed amountof link capacity, increasing the number of channels per link isequivalent to decreasing the size of the BoD bandwidth granu-larity. Thus, a large number of channels per link represents veryfine bandwidth granularity and, thus, the ability to closely matchthe assigned capacity to the required bandwidth. We approxi-mated the fine-grained “infinitesimal” limit with a simulationof 2048 channels on each link.

Fig. 13 shows the BoD network blocking probability as afunction of the network offered load for different numbers ofchannels per link (i.e., different bandwidth granularity). Theindicated load is the limit value for the allowed offeredload (Erlangs per channel) that keeps the blocking probabilityat 0.001 (0.1%) as the number of channels goes to infinity (in-finitely fine granularity). This quantity measures the maximumnetwork throughput that can be achieved with 0.1% blocking.If the bandwidth of a link is b/s, then the maximum networkthroughput is b/s. This limiting throughput is related tothe connectivity of the network, as will be discussed hereafter.For a smaller number of channels per link (coarser granu-larity), the maximum offered network load for a specifiedblocking probability (e.g., 0.1%) decreases, as shown in Fig. 13;thus, as the granularity becomes coarser, the maximum networkthroughput b/s decreases.

If we plot the normalized maximum network offered loadas a function of the granularity , we obtain the “rel-

ative efficiency” curve shown in Fig. 14. From these results, wesee that it takes a granularity of 109 channels per link to attain90% of the maximum load limit . It is also seen that 79% ofthe limit can be obtained with a relatively coarse granularityof 32 channels per link. It is also seen that very coarse granu-larity (e.g., less than ten channels per link) leads to a significantdrop in relative efficiency.


Fig. 13. Blocking versus offered load for the 19-node network.

Fig. 14. Relative efficiency versus granularity for the 19-node network.

TABLE ISIMULATION RESULTS SUMMARY

TableIshowsoursimulationresultsforthethreeexampleLATAnetworks.Forcomparison,theresultsforasinglelinkareincludedin the first row. The 19- and 71-node networks are based on realLATAs, while the 200-node network was generated using a pro-prietary Telcordia statistical LATA network generator.

The first column in Table I gives the number of nodes andnumber of links in the network, while the second gives the av-erage nodal degree (i.e., the average number of links impingingon a node). The third gives the size of the minimal balancedcutset of the network graph. That is, for each possible parti-


tioning of the nodes into two equal-sized groups (or two groupswhose size differs by one when the total number of nodes isodd), the set of links connecting the two groups is called a bal-anced cutset. A balanced cutset that has no more links than anyother balanced cutset is called a minimal balanced cutset. Thelast three columns of Table I give, respectively, , the gran-ularity (number of channels per link) needed to attain an effi-ciency ratio of 90%, and the efficiency ratio at a granularity of32 channels per link.

It is notable in Table I that seems to be tracking the min-imal balanced cutset size. This is intuitively reasonable, sincethe minimal balanced cutset would tend to be the bottleneck ofthe network in the limit of infinitesimal granularity. On average,about 50% of the site–site channel requests offered to the net-work would have their and sites lying on opposite sides ofa given balanced cutset; this is a higher percentage than wouldbe expected for any unbalanced cutset. Hence, a minimal bal-anced cutset would tend to be the bottleneck of the network inthe limit of infinitesimal granularity.

While applies in the limit of infinitesimal granularity, aswe reduce the number of channels, the chance of any particularlink anywhere in the network becoming blocked increases. Localconnectivity then becomes more important, and the numberof channels available at a node becomes dominant. Hence,average nodal degree (a local topology metric) becomes moreimportant than minimal balanced cutset size (a global topologymetric) as the granularity becomes coarser. This observationis reflected in Table I, where all three LATA networks havesimilar local connectivity (average nodal degree between 3.3and 3.9), and we see that the finite-channel performance metricsin the fifth and sixth columns are similar from row to row,despite the differences in minimal balanced cutset size.

C. Coarse Granularity Bandwidth Management

As discussed previously, the networking layer requires arange of bandwidth granularity choices. The previous subsec-tion addresses the fine-to-medium range of granularity. In thissubsection, we address the coarse bandwidth management gran-ularity, namely when lightpaths are used to establish site–siteconnectivity. The applications requiring a lightpath (or groupof lightpaths) are those that involve very large file transfersthat need to be completed in a relatively short amount of time(e.g., so a collaborative group can exchange large volumes ofdata and have meaningful “near real-time” interactions). Theproblem is how to provide on demand lightpath capability in acost-effective manner.

In the early stages of developing BoD networking capabili-ties, it is expected that most of the connectivity requirementswill be in the fine to medium bandwidth granularity. The numberof users (applications) requiring lightpath connectivity will berelatively small, and the frequency that lightpath connectionswould need between a specific pair of sites will also be rela-tively small. Thus, in the context of Fig. 9, lightpath connectivitywould initially be well within the region where only BoD facil-ities are used. For lightpath connectivity, each enterprise net-work site would need dedicated access to a core network thatprovides on-demand lightpath connectivity. Initially, this core

network might be hub based with a few major interconnectedhubs. Individual enterprise sites would have dedicated lightpathaccess to one of the hub nodes.

The cost and blocking performance of an individual site’sdedicated lightpath access is an important consideration. If aBoD capability to serve random arrival connection requests isdesired, then the cost of the dedicated access can be quite high.For example, if a 10% (1%) blocking probability were desired,a single lightpath access connection could only be loaded to10% (1%) utilization before a second access lightpath wouldneed to be installed. Since these access facilities would need toreach a major hub node, they would be relatively expensive, andthus the access costs could become prohibitive for the randomarrival/blocking mode of operation. An alternative is to use ascheduled access to the network. With scheduling, it is possibleto achieve very high utilization of an access facility and stillmeet the needs of the user community. The impact on the usersis that they need to be willing to adjust their schedule to whenfacilities are available. Before significant demands for lightpathconnectivity emerge, the core network (hubs and their intercon-nection facilities) may also need to be scheduled in order toachieve cost-effective use of facilities.

When the aggregate lightpath connection load out of an enter-prise site becomes large enough, the scheduling discipline canbe changed to a queueing discipline and achieve nearly the samecost savings. An important parameter in this case is the delaytime the users would tolerate. An important distinction betweenscheduling and queueing is that, with queueing, the user doesnot have to plan far in advance; however, they need to be tol-erant of some delay before their random arrival connection re-quest can be met.

IV. CONCLUSION

This paper examines how new metro network design method-ologies can be used to satisfy existing services at lower costthan legacy design techniques would achieve and, at the sametime, establish next-generation network infrastructures that canbe used to provide new wavelength and BoD services.

Regarding the infrastructure for wavelength services, WDMis the enabling technology, and the WDM infrastructure is es-tablished based on legacy TDM service demands. The key con-cept used to achieve this is to aggregate loads and establish net-work hub designs so that well-filled lightpaths are created. Thisthen provides the needed demand base to establish a WDM in-frastructure that is financed from TDM service revenue. A keyelement of the design technique is the need to take a high-levelview of the network and develop a target architecture. This targetarchitecture is used to steer the design process to achieve the de-signer’s overall goals. There are many detailed network designtools available both commercially and from the open literature.However, these design tools tend to look for local optimizationconcepts and do not optimize the “big picture.” The design tech-nique presented here develops the big picture through a targetnetwork architecture and then uses detailed design tools to carryout specific network design tasks (e.g., designing WDM rings ordesigning SONET/SDH rings).


Regarding BoD and how a dynamic networking layer wouldbe established, we looked at existing enterprise network ser-vice demands and showed how BoD can improve network per-formance and cost. The results show that, for fine granularitybandwidth management, most enterprise network sites will useBoD in conjunction with dedicated bandwidth services. Thus,BoD will serve overflow traffic that exceeds dedicated capacity,and the BoD service provider needs to consider how to handleoverflow traffic, which is bursty. The network must properly ag-gregate traffic to achieve sharing of capacity and efficient fa-cility utilization. The bandwidth granularity used for a BoDservice is a critical design choice; it impacts both the carrier’stransport system efficiency and the efficiency of the BoD cus-tomer’s dedicated BoD access facilities. If the chosen gran-ularity is too coarse, the BoD facility utilization will be toolow, and BoD will not be economically viable. In the earlystages of dynamic BoD networking, coarse bandwidth granu-larity (e.g., wavelength services) appears to be best managedthrough scheduling or queueing disciplines rather than randomarrival/blocking disciplines.

ACKNOWLEDGMENT

The authors would like to thank M. Esfandiari, A. Zolfaghari,and S. Gloeckle for their support and input regarding the workdescribed in Section II. The authors would also like to thankI. Habib and S. Yun for their support and helpful discussionsregarding the work described in Section III.

REFERENCES

[1] J. Mambretti et al.. Hybrid Packet-Optical Infrastructure. [Online].Available: http://www.internet2.edu/presentations/fall-03/20 031 015-Networks-Mambretti.ppt

[2] R. B. Cooper, Introduction to Queueing Theory, 2nd ed. New York:North Holland, 1981.

[3] G. R. Ash, Dynamic Routing in Telecommunications Networks. NewYork: McGraw-Hill, 1998.

[4] A. Zolfaghari, M. El-Sayed, Y. Hu, and M. Mezhoudi, “Edge groomingin a nonhierarcical metro network architecture,” in Nat. Fiber OpticsEngineering Conf. (NFOEC), Orlando, FL, Sept. 7–11, 2003.

[5] E. Modiano and R. Berry, “The role of switching in reducing networkport counts,” presented at the 39th Allerton Conf. Communications,Control, Computing, Allerton, IL, Sept. 2001.

[6] M. Esfandiari, S. Gloeckle, A. Zolfanghari, G. Clapp, J. Gannett, H. Ko-brinsky, V. Poudyal, and R. Skoog, “Improved metro network design bygrooming and aggregation of STS-1 demands into OC-192/OC-48 light-paths,” in Nat. Fiber Optics Engineering Conf. (NFOEC 2003), Orlando,FL, Sept. 7–11, 2003.

[7] T. deFanti, C. de Laat, J. Mambretti, K. Neggers, and B. St. Arnaud,“TransLight: A global-scale lambdagrid for e-Science,” in Commun.ACM, vol. 46, Nov. 2003, pp. 35–41.

[8] O. Gerstel, R. Ramaswami, and G. Sasaki, “Cost-effective trafficgrooming in WDM rings,” IEEE/ACM Trans. Networking, vol. 8, pp.618–630, Oct. 2000.

[9] X. Zhang and C. Qiao, “An effective and comprehensive approach fortraffic grooming and wavelength assignment in SONET/ADM rings,”IEEE/ACM Trans. Networking, vol. 8, pp. 608–617, Oct. 2000.

[10] L. Liu, A. Li, P. Wan, and O. Frieder, “Wavelength assignment in WDMrings to minimize SONET ADMs,” presented at the IEEE INFOCOM,Tel Aviv, Israel, Mar. 26–30, 2000.

[11] W. E. Leland, M. S. Taqqu, W. Willinger, and D. V. Wilson, “On theself-similar nature of ethernet traffic,” IEEE/ACM Trans. Networking,vol. 2, pp. 1–15, Jan. 1994.

[12] V. Paxton and S. Floyd, “Wide-area traffic: The failure of poisson mod-eling,” IEEE/ACM Trans. Networking, vol. 3, pp. 226–244, June 1995.

[13] Comput. Netw. (Special Issue on Advances in Modeling and Engineeringof Long-Range Dependent Traffic), vol. 40, no. 1, 2002.

[14] T. Karagiannis, M. Molle, M. Faloutsos, and A. Broido, “A non-stationary poisson view of Internet traffic,” presented at the IEEEINFOCOM, Hong Kong, Mar. 7–11, 2004.

Ronald Skoog (M’89) received the B.S. degree fromOregon State University, Corvallis, and the M.S. andPh.D. degrees in EE (control and systems theory)from the Massachusetts Institute of Technology(MIT), Cambridge.

Prior to joining Telcordia, he spent 29 years at BellLaboratories/AT&T Bell Laboratories/AT&T Lab-oratories working in the areas of transport networkdesign; signaling network design, protocols, andperformance/reliability studies; and circuit-switchednetwork systems engineering and performance/reli-

ability studies. He has been a Senior Scientist at Telcordia Technologies, Inc.,Red Bank, NJ, since October 1998, and during that time he has worked in theareas of optical networking architectures, Internet protocol/wavelength-divi-sion-multiplexing network architectures and evolution studies, optical networkmanagement and control, emerging network technology studies (e.g., GigabitEthernet and next-generation synchronous optical networking, or SONET),and reliability studies for optical networks and optical network elements.As part of his work at Telcordia, he has managed a research program onoptical network management and control under a photonics research programTelcordia has with the Laboratory for Telecommunications Sciences (LTS).This work has explored a wide range of issues related to management andcontrol of emerging transport network technologies and transparent opticalnetworks. He has also been a Principle Investigator for the Defense AdvancedResearch Projects Agency (DARPA) contract Control Mechanisms to PreventMaliciously Induced Network Instability in the Fault Tolerant Networkingprogram in the Advanced Technology Office (ATO).

Dr. Skoog is a Member of the IEEE Communications Society (COMSoc), theIEEE Lasers & Electro-Optics Society (LEOS), the Optical Society of America(OSA), and Sigma Xi.

Ann Von Lehmen (M’03), photograph and biography not available at the timeof publication.

George Clapp (M’01) received the B.A. degree inpsychology and the M.S. degree in computer sciencefrom Swarthmore College, Swarthmore, PA, in 1972and 1983, respectively. He is currently working to-ward the Ph.D. degree in computer science at DrexelUniversity, Philadelphia, PA.

He previously worked at Ameritech AdvancedData Services on Ameritech’s Internet service andthe Chicago Network Access Point (NAP) for theNational Science Foundation (NSF). He currentlymanages a group at Telcordia Technologies, Inc.,

Red Bank, NJ, that focuses on the integration of the Internet and opticalnetworks, the design of optical transport networks, and the control and manage-ment of optical networks. He has also managed research programs at Telcordiaon Internet service quality, Internet-protocol-based virtual private networks(IPVPNs), Internet telephony, and the interworking of TCP/IP with a variety ofpublic data services.


Joel W. Gannett (S’77–M’80–SM’91) receivedthe B.S.E.E. degree (with High Distinction) fromthe University of Iowa, Iowa City, and the Ph.D.and M.S. degrees from the Department of ElectricalEngineering and Computer Science at the Universityof California, Berkeley.

He was formerly a Member of Technical Staff atAT&T Bell Laboratories (now Lucent Technologies),Murray Hill, NJ, and a Senior Computer-Aided De-sign Engineer at Advanced Micro Devices, Sunny-vale, CA. He currently works with Telcordia Tech-

nologies, Inc., Red Bank, NJ. His interests include network performance andnetwork design and optimization, and he has authored or coauthored publi-cations in these and other fields. His self-testing digital circuit invention waspatented in the United States and several other countries (U.S. Patent 4 551 838).

Dr. Gannett is a Member of Tau Beta Pi and Eta Kappa Nu.

Haim Kobrinski (M’84), photograph and biography not available at the timeof publication.

V. Poudyal (S’92–M’01) received the M.E. andPh.D. degrees in electrical engineering from theStevens Institute of Technology, Hoboken, NJ, in1994 and 1998, respectively.

He was formerly a Lecturer of electrical engi-neering at the Institute of Engineering, Kathmandu,Nepal. He is currently a Senior Systems Engineerat Telcordia Technologies, Inc., Red Bank, NJ.For the past seven years, he has been working onvarious aspects of optical networking, includingsynchronous optical networks (SONET) and

dense-wavelength-division-multiplexing (DWDM) network design, DWDMequipment specifications and standards, multilayer network optimization,multiperiod network design methods, and network availability analysis.

2680 journal of lightwave technology, vol. · pdf file · 2016-01-07paper addresses...

Documents