network engineering—control of dynamic link topology in user networks

12
Network Engineering—Control of Dynamic Link Topology in User Networks Lily Cheng, John Ellson, Admela Jukan, Patrice Lamy, and Eve Varma To date, considerable effort has been devoted to the study of dynamically switched optical networks. While much work has focused on provider issues of setting up and tearing down connections to satisfy user requests, this paper studies the user network issues of dynamically changing the link topology and free capacity via automatically choosing the optical transport connections. We introduce a new process to complement traffic engineering and network planning, which we call network engineering. Network engineering entails complement automatically rearranging a user network link topology in support of user traffic demands by increasing or decreasing the capacity of a network. We also give a rationale for network engineering and discuss where and when the link topology needs to be adjusted. Network engineering enables effective rearranging (connecting, modifying, and disconnecting) of aggregated traffic units at timescales larger than an individual traffic unit in a user-provider network relationship.© 2003 Lucent Technologies Inc. Introduction Optical technologies have recently made possible the design of automatically switched networks with bandwidths of multiple Gb/s. However, these tech- nologies are not yet capable of efficiently switching individual Internet protocol (IP) packets. For example, to individually switch a 10-kb packet at 10 Gb/s the optical switch would have to change state every 1 s. Therefore, the current generation of optical switches needs to switch aggregates of packets at timescales much larger than the individual packet. The key point is that the decisions to connect and disconnect a high- capacity circuit need to be based on aggregated traffic, not on individual packet routing decisions. The aggregation of traffic can be spatial or tem- poral. Spatial aggregation results naturally from a multiplexing function, which takes multiple spatially separated signals and carries them on a single com- posite signal, which is switched as a single entity. Temporal aggregation of traffic occurs without multi- plexing, i.e., when there are switched circuits whose switching decisions are long term and not based on the individual “calls.” An example of temporal aggre- gation is the slow-switch connectivity provided by cross connects in telephone (POTS) networks. These calls make temporal reuse of a common switched circuit. The important point being made here is that layering (spatial multiplexing) is not fundamental to the problem of how to control a slow-switched (e.g., optical) network. It is traffic aggregation that is fundamental. In order to generalize the principles, we have found the notion of a traffic unit very useful. A traffic Bell Labs Technical Journal 8(1), 207–218 (2003) © 2003 Lucent Technologies Inc. Published by Wiley Periodicals, Inc. Published online in Wiley InterScience (www.interscience.wiley.com). • DOI: 10.1002/bltj.10054

Upload: lily-cheng

Post on 06-Jun-2016

217 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Network engineering—Control of dynamic link topology in user networks

� Network Engineering—Control of DynamicLink Topology in User NetworksLily Cheng, John Ellson, Admela Jukan, Patrice Lamy,and Eve Varma

To date, considerable effort has been devoted to the study of dynamicallyswitched optical networks. While much work has focused on provider issues ofsetting up and tearing down connections to satisfy user requests, this paperstudies the user network issues of dynamically changing the link topology andfree capacity via automatically choosing the optical transport connections. Weintroduce a new process to complement traffic engineering and networkplanning, which we call network engineering. Network engineering entailscomplement automatically rearranging a user network link topology insupport of user traffic demands by increasing or decreasing the capacity of anetwork. We also give a rationale for network engineering and discuss whereand when the link topology needs to be adjusted. Network engineeringenables effective rearranging (connecting, modifying, and disconnecting) ofaggregated traffic units at timescales larger than an individual traffic unitin a user-provider network relationship.© 2003 Lucent Technologies Inc.

IntroductionOptical technologies have recently made possible

the design of automatically switched networks with

bandwidths of multiple Gb/s. However, these tech-

nologies are not yet capable of efficiently switching

individual Internet protocol (IP) packets. For example,

to individually switch a 10-kb packet at 10 Gb/s the

optical switch would have to change state every 1 �s.

Therefore, the current generation of optical switches

needs to switch aggregates of packets at timescales

much larger than the individual packet. The key point

is that the decisions to connect and disconnect a high-

capacity circuit need to be based on aggregated traffic,

not on individual packet routing decisions.

The aggregation of traffic can be spatial or tem-

poral. Spatial aggregation results naturally from a

multiplexing function, which takes multiple spatially

separated signals and carries them on a single com-

posite signal, which is switched as a single entity.

Temporal aggregation of traffic occurs without multi-

plexing, i.e., when there are switched circuits whose

switching decisions are long term and not based on

the individual “calls.” An example of temporal aggre-

gation is the slow-switch connectivity provided by

cross connects in telephone (POTS) networks. These

calls make temporal reuse of a common switched

circuit. The important point being made here is that

layering (spatial multiplexing) is not fundamental

to the problem of how to control a slow-switched

(e.g., optical) network. It is traffic aggregation that is

fundamental.

In order to generalize the principles, we have

found the notion of a traffic unit very useful. A traffic

Bell Labs Technical Journal 8(1), 207–218 (2003) © 2003 Lucent Technologies Inc. Published by Wiley Periodicals, Inc.Published online in Wiley InterScience (www.interscience.wiley.com). • DOI: 10.1002/bltj.10054

Page 2: Network engineering—Control of dynamic link topology in user networks

208 Bell Labs Technical Journal

Panel 1. Abbreviations, Acronyms, and Terms

ATM—asynchronous transfer modeBGP—border gateway protocolIP—Internet protocolIS-IS—intermediate system to intermediate

systemMPLS—multiprotocol label switchingOSPF—open shortest path firstPOTS—”plain old telephone service”SDH—synchronous digital hierarchySLA—service-level agreementWDM—wave division multiplexed

unit is the entity about which routing decisions are

made. For instance, in an IP network the traffic unit is

an individual packet. In a switched wave division mul-

tiplexed (WDM) network, the traffic unit is a connec-

tion, which could be 10 Gb/s “wide” by 1000 seconds

“long.” Recall that there are traditionally three

timescales of processes related to the switching of

traffic units (see Figure 1):

• Switching/routing, the selection of a route for indi-

vidual traffic units (where the traffic unit is a set

of packets forming a logical connection, or a set of

timeslots forming a time division multiplexed

connection) based on values available in the pre-

established routing tables.

• Traffic engineering, the processes of route planning

and establishing routing tables that make optimal

use of an existing topology of fixed capacity.

Typically automated in IP and POTS networks,

traffic engineering is a short-term process to opti-

mize network resource utilization, based on a

given topology, for near-term traffic fluctuation

[2–4, 7, 8].

• Network planning, the process of sizing the network

capacity to accommodate demand projections,

introduction of new technology, and changes in

network topology. The network planning process

optimizes network resources for traffic growth by

laying out physical topology. This network capac-

ity is fixed until the next network planning cycle

is executed. We note that the computation for net-

work planning is typically done off line and re-

sults in provisioning of new equipment and fibers.

Referring back to the discussion on the relation-

ship between two networks (IP and optical networks),

we can state that one deploys faster switching than

the other. We call the faster network the user network

Switching Trafficengineering

Networkplanning

Network state

Connection stateupdates

Routing tableupdates

Capacity &topology updates

Dig holes.

Install equipment.

Busy/idle TrafficTraffic

Connection requestand updates

∫ ∫

Figure 1.Conventional engineering processes.

Page 3: Network engineering—Control of dynamic link topology in user networks

Bell Labs Technical Journal 209

and the slower-switching network the provider net-

work. Note that the distinction between user and

provider networks is fundamentally one of timescale,

not of corporate relationship. When we use the terms

user and provider, we imply only this fundamental

switching timescale difference.

Based on this model, we focus on the user net-

work issues and, in particular, how a user network

can automatically identify and rank link modifications

to its proper topology. The issues in the provider

network of setting up a connection to satisfy user

requests have been well studied by others, and we

can assume that these issues have been resolved.

In order to capture the availability of an auto-

matically switched network to switch aggregates of

the user network’s traffic units, we introduce a fourth

process in between traffic engineering and network

planning (see Figure 2). We call this process network

engineering, and we define it as the automated process

of increasing or decreasing the capacity of a network

by changing the capacities of existing links, by adding

new links, or by dropping the existing ones.

Whereas traffic engineering is about tuning the

routes to put the traffic where the capacity is, net-

work engineering is about tuning the link capacity to

where the traffic is expected to be. We note in passing

TrafficengineeringSwitching Network

engineeringNetworkplanning

User network state

Routing tableupdates

Connection stateupdates

Capacity &topology updates

Capacity &topology updates

Dig holes.

Install equipment

Busy/idle Traffic

Traffic

TrafficTraffic

Connection requestand updates

∫∫

Connection requestand updates

TrafficengineeringSwitching Network

planning

Provider network state

Routing tableupdates

Connection stateupdates

Capacity &topology updates

Busy/idle Traffic ∫∫

Figure 2.Network engineering process in a user–provider network.

Page 4: Network engineering—Control of dynamic link topology in user networks

210 Bell Labs Technical Journal

that asynchronous transfer mode (ATM) or multipro-

tocol label switching (MPLS) virtual circuits, according

to this definition, would fall under the scope of traffic

engineering since they do not provide any change in

capacity of a network [8]. In contrast to network plan-

ning being a static process where the capacity, once

put in place, is rarely torn down, network engineering

automates and dynamically establishes, releases, or

rearranges (increases, decreases, restructures) a link.

The principles of network engineering are general:

they can be applied to any network technologies in a

user-provider relationship, e.g., switched SDH over

switched WDM, or POTS over switched SDH.

The remainder of this paper is organized as fol-

lows. In the next section, we describe and motivate

network engineering. We follow that section by pre-

senting basic elements of network engineering and

then giving an example of different link types and

basic link operations. We next present an example

of recursive user–provider network engineering pro-

cesses by studying an IP/SDH/WDM network. In the

last section, we summarize our work.

Network EngineeringWithin this section, we provide a more detailed defi-

nition of, and rationale for, network engineering.

MotivationFrom the perspective of an IP network, conven-

tional backbone networks are static. Traditionally,

transport links are provisioned for long periods of

time, as rapid reconfiguration of physical links is dif-

ficult. To optimize for mid-term traffic fluctuations in

the user network, such a non-configurable topology

with a fixed amount of resources does not offer the

flexibility to trade off the traffic characteristics with

the expensive transmission resources. Traffic engi-

neering, as deployed today, limits itself to achieving

higher network resource utilization assuming a fixed

topology.

To address these limitations, there is an opportu-

nity to use switched optical networks, which can be

considered as a “configurable backbone.” With this

capability, the user network topology can be dynam-

ically configured according to actual traffic character-

istics resulting in optimizing the overall network

performance. The question arises as to how traditional

IP traffic engineering, which was designed with static

topology in mind, handles a more dynamic transport

network topology. To answer this question, we intro-

duce the concept of network engineering, which

represents a new network operations process. This

process enables dynamic tuning of the user network

topology by allowing configuration of the user net-

work link topology in conjunction with traffic engi-

neering. To configure the user network link topology,

the network engineering function adds, modifies, or

removes links from the provider network and uses

them to modify capacity in the user network. Thus,

network engineering supports a superior and more

flexible network topology for the traditional traffic

engineering function to utilize without having to

change existing traffic engineering capabilities.

Finally, the network engineering function natu-

rally complements switched optical networking. For a

dynamically switched optical network to be of use, a

large-timescale steady state must be reached in which

demands for capacity are withdrawn as often as they

are presented. This is a function that is directly sup-

ported by network engineering. In this state, a degree of

resource sharing occurs in the optical network, which

results in an effective capacity that is larger than if the

same resources were permanently connected.

Tuning the User Network Link TopologyTo the network engineering function, a user net-

work topology consists of a fixed (non-configurable)

node and link resources obtained from network plan-

ning and flexible (configurable) link resources ob-

tained from the provider networks. In this example,

there are two layers—the IP layer and the underlying

circuit-switched layer. These two layers interact in a

user-provider relationship. The circuit-switched opti-

cal network layer provides dynamically configurable

links for the user network. Figure 3 depicts an ex-

ample of how a user link topology can be dynami-

cally tuned.

Network engineering allocates/de-allocates

provider network resources to be used by the aggre-

gated traffic units of the user network. It can be

thought of as an automatic network provisioning

procedure, providing topology changes that are more

Page 5: Network engineering—Control of dynamic link topology in user networks

Bell Labs Technical Journal 211

OXC—Optical cross connect

(b) User network view

(a) Provider network view

(Fixed links)

Router

Router

AZ

R2

R6

R3

R4

R1

Router

Link topology

Router

A

R1

R6

Z

R2

R4

R3

OXC

Figure 3.User network link topology.

responsive to actual traffic demands than can be

achieved by manual provisioning. In other words, the

intent of network engineering is to add capacity

where it is needed by the traffic or, to remove capac-

ity where the traffic has diminished. It supplements

the current traffic engineering function in better op-

timizing the network resource utilization via link

topology changes. As can be seen from Figure 3, the

views of the user network and the provider network

are different, and both dynamically change.

Where and WhenIn order to change the user network link topol-

ogy, network engineering has some basic operations.

It may add a link, delete a link, or modify a link (by

adding or deleting link-connections or “channels”) as

a result of predicting traffic demands and ranking the

most effective changes. The connection resources

available from the provider network are limited. Due

to the limited resource pool, the optimization aspect of

a network engineering function concerns choosing

when and where to add (or to delete or to modify) the

most beneficial links. For these reasons, the choice of

a basic operation to perform and its timely effectua-

tion are crucial.

To choose where and when a link needs to be

added, dropped, or modified, the network engineering

function monitors the bandwidth demands either

from a source/destination relationship or across a link,

Page 6: Network engineering—Control of dynamic link topology in user networks

212 Bell Labs Technical Journal

in order to identify the traffic patterns in the network.

Where necessary, a user network’s link topology can

be optimized, e.g., by adding direct links between two

heavily overloaded nodes. Decisions are taken based

on the service-level agreement (SLA) between the

user and the provider and the policies set by the

provider network operators.

Network Engineering and Traffic EngineeringWe will now describe in more detail how traffic

engineering and network engineering work in a com-

plementary fashion. As previously mentioned, traffic

engineering controls the traffic flow, while network

engineering controls the link topology in a user

network.

We assert that traffic engineering controls the

routing tables in the nodes of the user network, so it

controls the flow of the network. We imply that

processes, which are related to routing table updates,

are part of traffic engineering. Basically traffic engi-

neering controls the routing tables in the nodes,

which decide that all calls should be routed over a

predefined set of links. Network engineering controls

the links of the user network. It modifies the link

topology and, hence, the capacity distribution of the

network. Therefore, network engineering provides

the node-to-node capacity that can be used by traffic

engineering.

Adding a flow to a network cannot improve the

total capacity of a network. It can only simplify the

per-traffic-unit routing decisions that are made at each

node. In fact, adding flows that do not correspond to

shortest-path routes can significantly reduce the avail-

able instantaneous capacity of a network by occupy-

ing more [bandwidth � distance] than is optimally

necessary.

Flows do not reduce the number of routing deci-

sions; they just simplify the decisions and make the

route lookup easier to perform. For example, in label

switching, consecutive packets can have different la-

bels, so there are still packet-by-packet routing deci-

sions being made in the nodes. The labels have simply

reduced the complexity of routing lookup. (If an in-

termediate node has only four routes, then the rout-

ing lookup can only have one of four results. Labels

permit a four-deep routing table lookup instead of

repeating at each node the entire lookup based on

the final destination of the packet.)

Adding links to or modifying links on the link

topology increases the capacity of a network. It can

also result in the availability of shorter paths, so after

a link has been added, the routing table must be up-

dated or traffic will not use the new capacity.

Traffic engineering is about providing routing

decisions. Network engineering is about providing

capacity. Note that the network engineering function

is complementary to the use of effective traffic engi-

neering schemes. Indeed, it is necessary to use an

effective traffic engineering scheme to optimize the

traffic distribution via the routing table over the ex-

isting network resources between dynamically chang-

ing the link topology of the user network.

Elements of Network EngineeringThe input to the network engineering function is

traffic and network topology information. This infor-

mation can be predictive based upon external de-

mands or cyclical variations (e.g., busy hour), or

reactive based upon current traffic patterns. The out-

put of this function is a set of IP links to be added,

deleted, or modified. The result of the network engi-

neering function is thus an adjusted logical network

topology that must be dealt with by the traffic engi-

neering function.

Traffic InformationAn IP network has network internal information

and network external information that can be applied

to proactively or reactively trigger the optimization

function for network engineering. The network in-

ternal information is obtained based on the measure-

ments within the user network, and the network

external information can be estimated based on the

service contract information related to the new traffic.

Both external and internal information may result in

configuration or reconfiguration requests.

The network internal information deals with the

traffic statistics collected over a certain period of time,

based on which particular traffic flows can be estab-

lished according to the predictable traffic patterns. The

internal information can also trigger the traffic engi-

neering function to reconfigure the existing traffic

Page 7: Network engineering—Control of dynamic link topology in user networks

Bell Labs Technical Journal 213

patterns for more efficient service-level guarantees or

network throughput. If traffic engineering cannot

reconfigure traffic patterns, then network engineering

is attempted. This information is reactive, but it can

also be used predictively by assuming cyclic traffic

patterns.

The network external information deals with

edge-to-edge traffic requests. If such a request identi-

fies an amount of resources needed, that is used to

assess the need for new IP links. This information can

be neither estimated nor measured with complete ac-

curacy. An example of such information might be a

new contract or a new traffic request. In this case, for

example, the predictable traffic patterns as defined by

the long-term traffic measurements have to be

revised.

Traffic statistics may be stationary or non-stationary.

For stationary traffic statistics data, using periodically

sampled data to construct a probability distribution

function may be useful. This probability distribution

function is then a good input to the network engineer-

ing function. For non-stationary traffic, more data

(e.g., bandwidth utilization) is needed to perform the

network engineering function.

Examples of internal information are packet loss

[1], router overload, and port overload [9]. Examples

of external information are SLA violations and new

SLAs. Such information triggers the network design to

automatically configure an IP link. Depending on the

triggering information, configuration can consist of

adding or deleting circuit links or modifying existing

traffic parameters.

Network Topology InformationWe will now give an overview of network topol-

ogy information by studying the IP user network. In

order to perform the network engineering function,

the user network needs to have information on the

following topologies:

• IP routes—These are conventional routes to IP

peers. They are IP routes to various destinations

learned through conventional IS-IS, OSPF, or BGP

protocols.

• Forwarding adjacencies—These represent IP neighbors

that are connected by an underlying network con-

nection through a provider network. Forwarding

adjacencies are dynamic in nature and can be set up

and torn down as needed [10]. Both IP routes and

forwarding adjacencies are used for packet forward-

ing. A provider network treats traffic carrying this

kind of information as user traffic.

• Potential forwarding adjacencies—These involve

network elements that are potentially connectable

via reconfiguration of the underlying circuit-

switched network.

• Accessible IP routers via potential forwarding

adjacency—This provides information about all the

IP routers reachable through a potential forward-

ing adjacency endpoint. Such information can be

used for the network engineering function to

determine the source and destination of the

network connections.

Note that there are various options that can be used to

disseminate the above information. For example, in-

formation dissemination can be via established user

network connections or via a dedicated user network

control plane.

To the provider network, network engineering is

a new functional module that complements conven-

tional traffic engineering [5]. For the purpose of IP

network engineering, it understands the IP network

topology and does not require circuit network topol-

ogy information to compute the output. Since it does

not require server network topology information, net-

work engineering is independent of any network

models (e.g., “peer,” “overlay”). Existing network

models are all able to support a network engineering

function. Note that network engineering considers

user link configuration within a provider domain.

Hence, the choice of the signaling protocol, which will

carry parameters needed for a network engineering

function, is not relevant for the time being.

Triggering MechanismsThe triggering mechanisms stimulate actions to

add/delete/modify circuit-switched links. Some trig-

gering examples are as follows:

• Direct user request—The user should be able to

request a network engineering function. For

example, an operator may need to do online or

offline capacity planning or business modeling for

his/her network. Network engineering provides

Page 8: Network engineering—Control of dynamic link topology in user networks

214 Bell Labs Technical Journal

a most efficient network topology as a basic start-

ing point for traffic engineering to consider.

• Request to add a link—Network engineering has the

capability to add a link to the network. The re-

quest may be a result of future planning for the

network or an unexpected traffic load. Such a re-

quest can also come from traffic engineering.

• Request to delete a link—Network engineering has

the capability to delete a link from the network.

The request may be a result of future planning

for the network or an expected reduced traffic

load. Such a request can also come from traffic

engineering.

• Request to modify a link—Network engineering has

the capability to modify a link to the network. If

the request is to increase the capacity of a link, it

may be a result of future planning for the net-

work or an unexpected traffic load. If the request

is to decrease the capacity of a link, it may be a re-

sult of future planning or reduced traffic loads.

• Congestion condition—The congestion problem is

probably one of the most studied problems in

contemporary Internet networks. When a con-

gestion condition is not properly handled, it will

significantly degrade network performance.

Traffic engineering should be able to detect a con-

gestion condition, reengineer the network, up to

its maximal achievable limit, and then activate

the network engineering function.

• Router overload—When a router overload condi-

tion occurs, it is desirable to find an alternative

route for traffic to bypass the overloaded router.

• Port overload—When a port overload condition oc-

curs, it is desirable to add more capacity for traffic

to bypass the port. If this is not feasible with the

same router, it is desirable to increase capacity over

a different route and redirect some of the existing

traffic to the newly added link to relieve the over-

loaded condition. However, this has to be done

carefully, as all existing traffic should not get redi-

rected on the newly added link, leading to a mere

shift in the location of the overload condition.

• Maintenance—In the maintenance phase, it may

be desirable to run a network engineering

function.

• Failure—When a failure situation occurs (e.g., link

failure, network failure), it may be desirable to

run the network engineering function.

• Routine—If none of the above conditions occur, it

may be desirable to run the network engineering

function on a regular basis. Routinely running

the network engineering function can ensure that

the network is making the most efficient use of its

network resources. The interval of the routine

depends on network size and traffic load.

Designing and Choosing the Optimal LinkThe optimization aspect of the network engi-

neering function decides where to add/delete/mod-

ify circuit-switched links. It involves path computation

depending upon the knowledge of the layered net-

work. The problem of selecting a new link that best

satisfies demands for bandwidth is a multidimensional

optimization problem, i.e., one that is mathematically

difficult and probably not possible to compute per-

fectly, even in principle. The fundamental issue is de-

sign, ranking, and choice of an IP link, which is to be

established by the circuit-switched layer [6]. This

problem can be decomposed into two parts—link

design and link choice. The design phase identifies a

set of links that satisfies some needs, whereas the

choice phase chooses a final set of links based on max-

imizing the total value.

First of all, link design provides a set of potential

links that meet the following criteria:

• The links must be configurable in the circuit-

switched layer (e.g., with free ports, and free

capacities), and

• The links must have enough capacity to meet traf-

fic demands.

During the design phase, we distinguish different

types of links based on the number of hops the link

bridges. The output of link design can be zero or more

candidate links to be configured (e.g., added, deleted,

modified) in the circuit-switched layer. If no link is

selected, it means blocking in the circuit layer. This

can be the result of unavailable ports or insufficient

capacity.

Second, in the choice phase, a set of criteria for

choosing a final link among a set of candidate links is

needed. Within the circuit layer, a choice between

Page 9: Network engineering—Control of dynamic link topology in user networks

Bell Labs Technical Journal 215

candidate switched circuits can be made for the IP

network in order to have the maximum advantages

of a dynamic configuration. The criteria are based

on the consideration of bandwidth, distance, and

duration.

Link choice ranks the set of potential links, se-

lected from the above link design, according to the

following example set of parameters:

• Number of hops bridged,

• Bandwidth utilization of the new link,

• Value (or more correctly, rate of return on

investment), and

• Number of switched ports spared.

It then requests the maximum value from the po-

tential links. One implication of the link choice is that

a link design may get refused for reasons of insuffi-

cient value. The value of any link requested can be

expressed by multiple attributes (as listed for link

choice) or a single metric function of multiple

parameters—that is, a single valued function of the

parameters (e.g., bandwidth, time, distance).

To avoid the problems of multidimensional opti-

mization as described above, a single metric can be

chosen as precedent, based on which a choice is made

if all other requirements are satisfied. Administrative

policies, constraints, directives, and guidance can also

be weighted and contributed to the single metric.

Some will be simple Boolean (go/nogo) parameters,

so if we are converting to cost, nogo parameters might

have to be represented as an infinite cost value to

force rejection of the option.

Other parameters can be given weights and

factored in to the single metric for the multidimen-

sional tradeoffs. So far we have been focusing on

shortest-path costing only, but we realize that net-

work engineering must support additional adminis-

trative inputs.

All circuits that may be configured are expected to

be within the bounds requested by the IP layer; oth-

erwise, the request is rejected. Note that the estab-

lishment of an IP link usually incurs a certain cost to

the user IP network. It is important that the cost be

within a planned budget. This may involve reopti-

mization and tearing down some IP links. The

provider network has limited resources, so it is im-

portant that the total value of the links established

from the provider network be maximized.

Link Type and Link OperationThis section describes various link types and link

operations.

Link TypeTo ease the choice of candidate links, we study the

characteristics of links. Here, we will distinguish three

fundamentally different types of links, which can be

constructed in response to user network demands. As

illustrated in Figure 4, there are Type 1, Type 2, and

Type 3 links. These three types of new links differ not

only in the number of hops they bridge in the user

network, but also in the user network topology

changes that occur by adding the new link.

2

3

1

A

a

Provider Network

d

c

b

gf

h

Z

e

User network

Figure 4.Three types of new links; Type 1, Type 2, and Type 3.

Page 10: Network engineering—Control of dynamic link topology in user networks

216 Bell Labs Technical Journal

As illustrated in Figure 4, an example of user and

provider networks, the traffic flows from the source

node A to the destination node Z. Three fundamen-

tally different types of links can be configured by the

network engineering function:

• Type 1 (Capacity)—This link parallels an existing

link, thereby increasing the capacity of the route

but not otherwise changing the topology of the

user network.

• Type 2 (Local)—This link is the shortest new link

that can be added in some network path around

one node. The topology of the user network

is modified and routing tables will need to be

modified.

• Type 3 (Regional)—This link is the longest new link

that can be added in the provider network

between two nodes. The topology of the user

network is modified and routing tables will need

to be modified.

In the above definition, by “shortest” we mean

the minimum reduction in hop count for a given

path, and by “longest” we mean the maximum re-

duction in hop count for a given path. The distinc-

tion between Type 2 and Type 3 is subtle. The

distinction is in the reason for adding the new link.

The Type 2 link is the one that best satisfies the reac-

tive needs to overloads at a single node based on local

knowledge of the traffic between two nodes within a

short distance (e.g., small number of hops) flows. The

Type 3 link is the one that best satisfies the predictive

needs identified by some external contract or SLA for

future traffic.

Link OperationAs we have mentioned earlier, in order to change

the network topology, network engineering has to be

able to add, delete, and modify circuit-switched links.

We distinguish the following four types of link oper-

ations:

• Add a link—To add a link is to add a potential

route with zero capacity. This type of link bridges

two or more hops of existing IP links. Note that it

changes the IP network topology without adding

new capacity. Routing tables will need to be up-

dated. In Figure 4, “add a link” would mean to

add a link of Type 2 or Type 3.

• Add a link-connection—To add a link-connection is

to increase the capacity of an existing link. This

new link (connection) bridges a single hop of IP

link. It parallels an existing link and increases ca-

pacity of this link without changing the network

topology. Figure 4 shows adding a new link con-

nection b-f to an existing circuit-switched link b-f.

Referring to this figure, “add a link-connection”

would mean to add a Type 1 link.

• Delete a link-connection—This operation is only

allowed when the traffic of that link-connection

is zero. To delete a link-connection is to decrease

the capacity of a link. It may reduce the capacity

of a link to zero. Note that a combination of link-

connection addition and deletion is effectively a

link-connection rearrangement across links.

• Delete a link—This operation is only allowed when

the number of link-connections in the link is

zero—that is, when the capacity of the link is

zero. To delete a link is to remove a potential

route.

For the purpose of network engineering, if we need a

new link with some capacity, we need to add a link

(with zero capacity), and then add a new link-

connection to increase the capacity of this new link. If

we need to delete a link, we will need to delete all

the link-connections in that link before deleting the

link.

SummaryIn this paper, we have defined a new process to

complement network planning and traffic engineer-

ing, a process that we call network engineering.

Network engineering refers to optimizing a user net-

work link topology in support of traffic engineering

processes. It is the automated, mid-term optimization

process of increasing or decreasing the capacity of a

network by changing the capacities of existing links,

by adding new links, or by dropping the existing ones.

Traffic problems, which may occur due to insuf-

ficient network capacity or low resource utilization,

can trigger network engineering processes to better

optimize overall network performance. The network

engineering process can also be triggered by traffic

measurements that indicate insufficient or excess

Page 11: Network engineering—Control of dynamic link topology in user networks

Bell Labs Technical Journal 217

network capacity. The traffic engineering process,

which operates on a shorter/faster timescale, can

then utilize the flexible topology established by net-

work engineering to optimally route traffic flows.

Thus, with the network engineering process, the de-

cisions to connect and disconnect a link can now be

efficiently based on aggregated traffic flows (not in-

dividual packets) at timescales larger than individual

traffic units.

The principles of network engineering are gen-

eral. They are applicable to any user–provider rela-

tionship network. While we have illustrated the

concept of network engineering with an example of

an IP network using an automatically switched opti-

cal network, the specific choice of provider network

implementation is not relevant. We conclude that the

network engineering process is applicable to any

IP/SDH/SONET/DWDM network.

AcknowledgmentsThe authors wish to acknowledge the contribu-

tions of Maarten Vissers and Yangguang Xu to this

work.

References[1] G. Almes, S. Kalidindi, and M. Zekauskas,

“A One-Way Packet Loss Metric for IPPM,”RFC 2680, Sept. 1999, <ftp://ftp.rfc-editor.org/in-notes/rfc2680.txt>.

[2] J. Ash, “Traffic Engineering & QoS Methods forIP- , ATM- , & TDM-based MultiserviceNetworks,” IETF draft, Oct. 2001,<http://www.ietf.org/internet-drafts/draft-ietf-tewg-qos-routing-04.txt>.

[3] D. Awduche, J. Malcolm, J. Agogbua, M.O’Dell, and J. McManus, “Requirements forTraffic Engineering over MPLS,” RFC 2702,Sept. 1999, <ftp://ftp.rfc-editor.org/in-notes/rfc2702.txt>.

[4] D. Awduche, A. Chiu, A. Elwalid, I. Widjaja,and X. Xiao, “Overview and Principles ofInternet Traffic Engineering,” May 2002,<ftp://ftp.rfc-editor.org/in-notes/rfc3272.txt>.

[5] L. Cheng , Y. Cao, J. Ellson, A. Elwalid,T. Hashimoto, H. Ishimatsu, A. Jukan, Li Mo,A. Nagarajan, Y. Oyama, L. Qian, I. Saniee,E. Snyder, S. Tanaka, M. Vissers, I. Widjaja, Y.Xu, and S. Yoneda, “A Framework for InternetNetwork Engineering,” IETF draft, July, 2001,<http://www.ietf.org/proceedings/01dec/

I-D/draft-cheng-network-engineering-framework-01.txt>.

[6] J. Ellson, L. Cheng, and A. Jukan, “LinkProvisioning,” IETF draft, Mar. 2001,<http://www1.ietf.org/mail-archive/ietf-announce/Current/msg11757.html>.

[7] F. Le Faucheur and W. Lai, “Requirements forSupport of Diff-Serv-aware MPLS TrafficEngineering,” IETF draft, Feb. 2003,<http://www.ietf.org/internet-drafts/draft-ietf-tewg-diff-te-reqts-07.txt>.

[8] J. A .S. Monteiro and M. Gerla, “TopologicalReconfiguration of ATM Networks,” INFOCOM‘90, pp. 207–214.

[9] G. Newsome, “IP Traffic Engineering Resultingin Optical Layer Connections,” IETF draft, Nov.2000, <http://www1.ietf.org/mail-archive/ietf-announce/Current/msg10176.html>.

[10] Y. Xu, A. Basu, and Y. Xue, A BGP/GMPLSSolution for Inter-Domain Optical Networking,IETF draft, June 2002, <ftp://ftp.rfc-editor.org/in-notes/internet-drafts/draft-xu-bgp-gmpls-02.txt>.

(Manuscript approved March 2003)

LILY CHENG was formerly a member of technical staffin the Advanced Networking Technologies andStandards Group at Lucent Technologies in Holmdel,New Jersey. She holds a bachelor’s degree in chemistryfrom National Taiwan Normal University and master’sand doctoral degrees in computer science from theUniversity of Missouri-Columbia and Michigan StateUniversity in Lansing, respectively. At Lucent, Dr. Chengwas involved in optical data networking architectureand standards-related activities. Her prior experiencealso includes work focused on system design, trafficmanagement, and traffic engineering, as well as workon several projects including the Video ServicesPlatform access subsystem specification and ATM trafficengineering.

JOHN ELLSON was formerly a member of technical staffin the Advanced Networking Technologiesand Standards Group at LucentTechnologies in Holmdel, New Jersey. Hehas many years of experience intelecommunications (United States and

Canada) and aerospace (United Kingdom). Hecontributed significantly to T1X1 and ITU standards forSONET/SDH; his contributions include the invention ofthe pointer mechanism. Mr. Ellson holds a B.Sc. degree

Page 12: Network engineering—Control of dynamic link topology in user networks

218 Bell Labs Technical Journal

from the Faculty of Electrical Engineering at theUniversity of Surrey, Guildford, United Kingdom.

ADMELA JUKAN received the M.S and Ph.D. degreesfrom Polytechnic of Milan (CEFRIEL), Italy, and ViennaUniversity of Technology in Austria, respectively.Since 1996, she has been with the Institute ofCommunication Networks at Vienna University ofTechnology in Austria. Currently, she is serving as theProgram Director in Networking Research at NationalScience Foundation in Arlington, Virginia, and she is avisiting faculty member at the School of Electricaland Computer Engineering at Georgia Institute ofTechnology in Atlanta. Her research interests includeprotocols and architecture for wavelength-routednetworks, as well as QoS-routing, partitioning, and self-organizing for broadband networks. When thework that is the subject of this paper was performed,Dr. Jukan was a visiting member of technical staff inthe Advanced Networking Technologies and StandardsGroup at Lucent Technologies in Holmdel, New Jersey.

PATRICE LAMY is a technical manager in the OpticalNetworking Group CTO organization atLucent Technologies in North Andover,Massachusetts. He has worked on SONETand SDH, ATM, and optical networking.His work extended from network element

and network architectures to network and servicemanagement. He was formerly Rapporteur in the ITUStudy Group 15 for transport management questionsand the vice chair of the ATM-Forum NetworkManagement Working Group. His current interests arein optical network management, the integration ofdata and optical management, and the extensions ofMPLS for the optical networks. Mr. Lamy graduatedfrom Ecole Centrale de Paris and also from theUniversity of Wisconsin-Madison, where he receivedan M.S. degree in chemical engineering.

EVE VARMA is a technical manager in the OpticalNetworking Group at Lucent Technologiesin Holmdel, New Jersey. She has manyyears of experience within thetelecommunications industry. She formerlyled the Advanced Networking Technologies

and Standards Group. Her more recent work hasbeen focused on the automatically switchedtransport network (ASTN/GMPLS) and intelligenttransport networking. In prior years, she led the teamresponsible for designing and prototyping a distributed

optical network management system as part of theMultiwavelength Optical NETworking (MONET)Consortium (partially supported by DARPA). Previouswork efforts include characterization, analysis, anddevelopment of transmission jitter requirements andSDH/SONET systems engineering and standards.Ms. Varma has been an active contributor to globalstandards and industry forums for many years and hasco-authored two books, Achieving Global InformationNetworking, Artech House (1999) and Jitter in DigitalTransmission Systems, Artech House (1989). She holdsan M.A. degree in physics from the City University ofNew York. �