communication network architecture and design...
Post on 21-Apr-2018
220 Views
Preview:
TRANSCRIPT
◆ Communication Network Architecture and Design Principles for Smart GridsKenneth C. Budka, Jayant G. Deshpande, Tewfik L. Doumi, Mark Madden, and Tim Mew
An integrated high performance, highly reliable, scalable, and securecommunications network is critical for the successful deployment andoperation of next-generation electricity generation, transmission, anddistribution systems—known as “smart grids.” Much of the work done todate to define a smart grid communications architecture has focused onhigh-level service requirements with little attention to implementationchallenges. This paper investigates in detail a smart grid communicationnetwork architecture that supports today’s grid applications (such assupervisory control and data acquisition [SCADA], mobile workforcecommunication, and other voice and data communication) and newapplications necessitated by the introduction of smart metering and home area networking, support of demand response applications, andincorporation of renewable energy sources in the grid. We present designprinciples for satisfying the diverse quality of service (QoS) and reliabilityrequirements of smart grids. © 2010 Alcatel-Lucent.
cells is being deployed in homes and enterprises.
Introduction of alternate and renewable sources of
energy and new storage technologies is fundamen-
tally altering the centralized power generation and
distribution paradigm that predominates today.
Furthermore, variations in the output power of renewa-
ble sources caused by changes in weather and time
of day are driving the control of distribution networks
to finer and finer time scales.
“Smart grid is a concept for transforming . . . [the]
electric power grid by using advanced communica-
tions, automated controls, and other forms of infor-
mation technology. This concept, or vision, integrates
energy infrastructure, processes, devices, information,
and markets into a coordinated and collaborative
IntroductionThe global electric power industry is entering a
period of significant transformation. Generation,
transmission, distribution, and control infrastructure
are aging while energy consumption is increasing.
Figure 1, which was developed using data from the
U.S. Department of Energy [18], illustrates the trend
in worldwide electricity consumption between 1980
and 2006.
Smart metering and other demand-side tech-
niques have become increasingly necessary to control
demand during peak and off-peak hours. Industrial-
scale wind and solar power plants are being connected
to the grid as part of worldwide efforts to reduce car-
bon emissions. Smaller-scale micro-generation in the
form of small wind turbines and photovoltaic (PV)
Bell Labs Technical Journal 15(2), 205–228 (2010) © 2010 Alcatel-Lucent. Published by Wiley Periodicals, Inc. Published online in Wiley Online Library (wileyonlinelibrary.com) • DOI: 10.1002/bltj.20450
206 Bell Labs Technical Journal DOI: 10.1002/bltj
process which allows electricity to be generated, dis-
tributed, and consumed more effectively and effi-
ciently” [13]. A high performance, reliable, and secure
communication network is one of the fundamental
building blocks to the introduction of smart grid appli-
cations.
This paper addresses network architecture and
design principles for an integrated smart grid commu-
nication network. We examine some of the challenges
faced in supporting a diverse set of applications each
with varying network performance requirements, relia-
bility requirements, and traffic characteristics, as well
Panel 1. Abbreviations, Acronyms, and Terms
AC—Alternating currentADR—Automated demand responseAMI—Advanced metering infrastructureBPL—Broadband over power lineCCTV—Closed circuit televisionCDMA—Code division multiple accessCPP—Critical peak pricingCS—Class selectorDER—Distributed energy resourceDiffServ—Differential servicesDSCP—Differential services code pointDSL—Digital subscriber lineEDGE—Enhanced data rates for GSM EvolutionEF—Expedited forwardingEMS—Energy management systemEPRI—Electric Power Research InstituteGPON—Gigabit passive optical networkGPS—Global positioning systemGSM—Global System for Mobile
CommunicationsGtCO2e—Giga (metric) tonne carbon dioxide
equivalentHAN—Home area networkHSPA—High-speed packet accessIEC—International Electrotechnical CommissionIED—Intelligent electronic deviceIEEE—Institute of Electrical and Electronics
EngineersIETF—Internet Engineering Task ForceIntServ—Integrated servicesIP—Internet ProtocolISM—Industrial, scientific, and medicineISO—Independent system operatorL1, L2, L3—Layer 1, 2, 3 (of OSI model)LAN—Local area networkLMR—Land mobile radioLTE—Long Term EvolutionMDMS—Meter data management systemMPLS—Multiprotocol Label SwitchingNAN—Neighborhood area networkNASPI—North American SyncroPhasor Initiative
NASPInet—NASPI networkNERC—North America Electric Reliability
CorporationNIST—National Institute of Standards and
TechnologyOFDM—Orthogonal frequency division
multiplexingOSI—Open System InterconnectionP2P—Peer-to-peerPEV—Plug-in electric vehiclePHEV—Plug-in hybrid electric vehiclePLC—Power line carrierPMU—Phasor measurement unitPRIME—PoweRline Intelligent Metering
EvolutionPTT—Push-to-talkPV—Photovoltaic (cells)QoS—Quality of serviceRF—Radio frequencyRFC—Request for commentsRTO—Regional transmission organizationRTP—Real time pricingRTU—Remote terminal unitSCADA—Supervisory control and data
acquisitionSDH—Synchronous digital hierarchySONET—Synchronous optical networkTDM—Time division multiplexingTOU—Time of use (pricing)UMTS—Universal Mobile Telecommunications
SystemUPS—Uninterruptible power supplyVAR—Volt-ampere reactiveVoIP—Voice over IPVPN—Virtual private networkVVWC—Volt, VAR, Watt controlWAMS—Wide area measurement systemWAN—Wide area networkWiMAX—Worldwide Interoperability for
Microwave Access
DOI: 10.1002/bltj Bell Labs Technical Journal 207
as the challenges faced with supporting legacy appli-
cations and networks. While there are many legacy,
new, and evolving applications, the following five
classes of applications (not necessarily mutually exclu-
sive) will be used as examples in presentation of
communication network architecture and design prin-
ciples in this paper:
• Smart metering, also known as advanced meter-
ing infrastructure (AMI),
• Automated demand response (ADR),
• Teleprotection,
• Distribution automation, and
• Micro grid management.
We begin with an overview of the evolution of a
traditional power grid to the smart grid. Next, we
present a high-level characterization of smart grid
applications including brief descriptions of the appli-
cation examples listed above. A smart grid communi-
cation network architecture is presented including the
physical connectivity architecture, examples of logical
connections, access network options, and the archi-
tectural implications of shared ownership of networks.
We then address specific quality of service (QoS) and
reliability design considerations for integrated smart
grid communication networks. We illustrate the
“green benefits” of a smart grid—and by implication
those of the integrated communication network—and
offer our conclusions and recommendations on areas
for future work.
Complete treatment of smart grids requires dis-
cussion of a wide variety of technologies and topics.
Due to space restrictions, we have had to limit scope.
0
2,000
4,000
6,000
8,000
10,000
12,000
14,000
16,000
18,000
Year1980
Electricity demand is increasing in Asian countries, and in China in particular, which saw demand for energy grow nearly tenfoldover a 25 year span. The North American market experienced a twofold increase over the same period despite drastic reductionin the manufacturing industry and slow population growth. Along with the pent up demand, energy sources are becoming scarceand the cost of generating electricity is becoming prohibitive. Therefore, making efficient use of electric energy should, in theory,help reduce dependence on fossil fuel and combat carbon emissions.
1982 1984 1986 1988 1990 1992 1994 1996 1998 2000 2002 2004 P2006
Ener
gy
con
sum
pti
on
in b
illio
n K
WH
Asia & OceaniaAfricaMiddle EastEurasiaEuropeCentral & South AmericaNorth America
Figure 1.Worldwide electricity consumption.
208 Bell Labs Technical Journal DOI: 10.1002/bltj
An essential topic not addressed in this paper is net-
work security—a topic worthy of several papers on
its own. Furthermore, details of local area networks
(LAN) or home area networks (HAN) are outside of
the scope of this paper.
An Overview of Smart GridThere is a wealth of information on the smart grid
concept and its evolution in the public domain. The
most comprehensive smart grid overviews are found
in the 2009 reports to U.S. National Institute of
Standards and Technology (NIST) prepared by the
Electric Power Research Institute (EPRI) [5] as well as
the final NIST Framework and Roadmap document
[17]. These extensive reports draw on contributions
from consensus-gathering workshops attended by
representatives from government agencies, regula-
tors, vendors, (communication) service providers,
academia, and standards organizations. Some of the
earlier EPRI work on IntelliGrid* can be found in [6].
The following brief smart grid presentation will
be used to set context for network architecture and
design. In a traditional power grid of an electric power
system (or utility), electricity flows from bulk power
generators to consumers over a grid of transmission
lines and distribution feeders, as shown in Figure 2.A hierarchy of transmission lines is connected
through a series of transmission substations leading
to distribution substations at the edge of the grid.
(See [1] for this classification of substations.) Step-up
Transmissionsubstation
Transmissionsubstation
Transmissionsubstation
Transmissionsubstation
Transmissionsubstation
Distributionsubstation
Thermal (coal,gas), hydro-electric,nuclear
CHP—Combined heat and powerDER—Distributed energy resourcePHEV—Plug-in (hybrid) electric vehiclePV—Photovoltaic (cell)UPS—Uninterruptible power supply
Business
Residence
Residence
Largebusiness,industrialcomplex
Storage
DER
DER
PHEV
Micro-generation(PV,…)
Large scale(PV, wind,diesel, UPS,CHP, …)
Alternate,renewable
energysource
Wind,PV,
bio mass,hydro,
tidal,fuel cell,
…
(Hierarchy of)micro grids
Storage
Bulk powergeneration
Distributionsubstation
To regionalor national grid
PV
Extr
a h
igh
an
d h
igh
vo
ltag
etr
ansm
issi
on
lin
es
Med
ium
vo
ltag
e an
d(s
ub
)-tr
ansm
issi
on
lin
es
Feed
er
Bulk powergeneration
Transformer(s)
Generator
Feed
er
Alternate,renewable
energysource
Figure 2.Generation, transmission, and distribution in smart grid.
DOI: 10.1002/bltj Bell Labs Technical Journal 209
or step-down transformers at the substations are used
to change voltages to levels appropriate for the corre-
sponding transmission lines and feeders. Finally, a dis-
tribution transformer on a feeder (such as those
mounted on utility poles or underground) steps down
the voltage to the standardized level required at the
consumer locations. Some industrial and large busi-
ness customers may connect to the grid at the feeder
voltages or even at the sub-transmission voltages.
An example of voltages used in a typical power
system [7] uses generation at 15–25 kV; hierarchy of
transmission lines at 500 kV, 230 kV, and 69 kV (sub-
transmission); distribution feeders at 12kV; residential
customers at 115/230 V. Over time, the power grids of
many utilities have been interconnected to form
regional, national, and international grids improving
energy management and transmission reliability.
With the advent of cost-effective generation of
renewable and/or alternate sources of energy it is now
possible to connect these energy sources of various
capacities throughout the grid (see Figure 2). As a
result, the direction of electricity transfer will fluctu-
ate based on local weather conditions, the position of
the sun, and other environmental effects. To com-
pensate for the variable nature of photovoltaic and
wind generation sources, for example, storage ele-
ments are deployed. Some of the new storage systems
(whether associated with power generation or stand-
alone) include batteries, high-energy flywheels,
(ultra) capacitors, pumped hydro, and compressed air
energy systems. One important class of storage devices
that are expected to be prevalent in the future are
plugged-in (hybrid) electric vehicles (PEV, PHEV). In
addition to supporting transportation, when parked,
the vehicles with charged batteries can potentially be
used to supply electricity to the grid.
Smart Grid ApplicationsThe requirements of smart grid applications drive
the design and architecture of an integrated smart grid
communication network.
Traditional ApplicationsTeleprotection and supervisory control and data
acquisition (SCADA) applications have been employed
for power grid management. Teleprotection refers to
the use of signal-aided relay-to-relay communication
between adjoining substations (i.e., substations con-
nected by a transmission line). If protection equip-
ment at either end detects a fault, the other end is
notified, and protective actions such as tripping
(power circuit disconnect) are initiated in order to iso-
late the fault. SCADA systems consist of remote ter-
minal units (RTUs), programmable logic connectors,
and other intelligent electronic devices (IEDs) con-
nected over communication networks. These sensors
and actuators are located at power stations, substations,
distribution transformers, and other grid locations.
They communicate with their respective manage-
ment systems at the utility data center (centralized)
or substations (distributed). In addition to grid opera-
tions, utility communication needs may also include
support for enterprise voice and data applications.
Many utilities have deployed private land mobile
radio (LMR) networks for their mobile workforce for
group voice communications (push-to-talk) as
well as some peer-to-peer voice communication
needs.
Examples of Smart Grid ApplicationsThe NIST/EPRI roadmap documents, [5] and [17],
divide the smart grid conceptual model into seven
domains that together represent the smart grid
community of interest. These domains are bulk gen-
eration, transmission, distribution, customers (con-
sumers), grid operations, service providers (for
services such as billing and third party providers), and
markets (wholesale, retail, and trading). While some
of these domains are connected by the electric grid
(generation transmission, distribution, and cus-
tomers), all of them must communicate with each
other. The five classes of applications listed earlier are
briefly described in this section. These applications
have been chosen for their diverse network require-
ments and together they incorporate many of the net-
work architecture elements covered in this paper.
Smart metering. Smart metering is one of the first
new smart grid applications deployed by most utilities.
Smart metering encompasses much more than peri-
odic energy measurement. Many new smart grid
applications require frequent power (both active and
reactive) and power quality (e.g., voltage, frequency)
210 Bell Labs Technical Journal DOI: 10.1002/bltj
measurements. Such measurements (provided by
smart meters) may be required as often as once every
15 minutes to support energy management applica-
tions. Measurements provided by smart meters are
also used to support real time pricing (RTP), time of
use (TOU) pricing, and critical peak pricing (CPP) fea-
tures for billing and demand response applications.
1. Depending on the size of a utility, the number of
smart meters in the network can vary from a few
thousand to several million.
2. Regulatory requirements, lack of timely smart
metering standards, and cost considerations have
led to deployment of vendor-proprietary smart
metering solutions based on neighborhood area
networks (NAN). These solutions can be readily
deployed using wireless technologies deployed in
unlicensed spectrum or using power line carrier
(PLC) technologies. A meter concentrator con-
nects to the meters over the NAN and is respon-
sible for aggregating data collected from the
meters it serves. The number of meters served by
a concentrator can vary from a handful for a PLC-
based NAN to several hundred or even several
thousand for a radio frequency (RF) mesh-based
solution. The meter concentrator connects to the
meter data management system (MDMS) over an
Internet Protocol (IP) network.
3. There are smart meter products with direct inter-
faces to a wireless service or Ethernet interfaces to
connect to wireline services. For such deployment
there is no meter concentrator and the meter
communicates directly with the MDMS over an IP
network.
4. Smart meter connections to home area networks
are fundamental to residential or building energy
management. Such connections, for example,
allow appliances to respond to pricing signals or
other triggers carried over the smart grid.
5. Under normal operating conditions, the accepta-
ble response times for completing meter transac-
tions can be high. Thus, one-way packet latency
allowances can also be high—on the order of sev-
eral seconds. The availability of an individual
meter may not be considered too critical to net-
work operations; hence, an availability objective
of 99.95 percent should be reasonable (corre-
sponding to an average downtime of 263 min-
utes/year).
6. However, some smart grid applications may
require data transfer from all linked meters over
a relatively short timeframe (several minutes),
which requires low latency for each of the indi-
vidual meters, even if higher latency may be
acceptable for billing purposes.
7. While security issues are out of the scope of this
paper, it is important to note that smart meters
are, perhaps, the weakest link in smart grid secu-
rity. In addition to the security threat to electric-
ity usage data and unauthorized physical access to
the meter itself, threats attributable to wireless
connectivity (for meters thus connected) must be
considered in the architecture and design of the
network.
Automated demand response. Demand response
activity is an action taken to reduce electricity demand
in response to price, monetary incentives, or utility
directives so as to maintain reliable electric service or
avoid high electricity prices [20]. Demand response
is a temporary change in electricity consumption in
response to supply conditions or other events in the
grid [5]. The inclusion of new energy sources and
storage elements combined with the need to reduce
peak loads and conserve energy has driven the intro-
duction of distributed automated demand response
applications. ADR applications, for example, can be
used to reduce the amount of energy consumed by
appliances during peak power periods. While demand
response has been used by utilities over the years
through scheduled load shedding and manually man-
aged consumption reduction with a few large con-
sumers, ADR is much wider in scope—bringing
dynamic load management directly to residential con-
sumers. ADR often works in concert with distributed
energy resources (DER) closer to the point of con-
sumption or other energy sources connected into the
grid. Thus, in some cases, ADR may not necessarily
reduce overall energy consumption, but only transfer
the source of some of the consumed energy to DER.
Such load shifting will result in reduced carbon emis-
sions if the DER is a renewable energy resource.
DOI: 10.1002/bltj Bell Labs Technical Journal 211
Dynamic pricing mechanisms such as CPP and TOU
pricing through smart meters contribute to efficient
ADR implementation and possible cost reduction.
1. ADR is still evolving. The Demand Response
Research Center OpenADR communication speci-
fication is a data model for information exchange
between the utility and consumer facility and is
designed to automate demand response actions
at the customer location [12].
2. The latency allowance between a utility (or an
independent service operator) and a single con-
sumer’s premise should be less than one minute.
As discussed earlier, however, to span large
numbers of locations (through their respective
meters), much smaller latencies will be required
to support “sweeping” through the meters in a
“neighborhood.”
3. Electric vehicles offer another form of storage for
the smart grid, one with unique communication
networking challenges. Even with the most con-
servative estimates of PEVs and PHEVs, utilities
are ill equipped to support the increased demand
with currently deployed bulk power generation.
P(H)EVs will add tens of thousands of mobile and
roaming endpoints not only to the utility grid, but
also to communication networks. A P(H)EV can
be charged or discharged at an owner’s home,
special charging stations (e.g., at parking lots), or
other locations. Thus vehicle mobility and energy
transfer must be accurately captured through the
relationship between the vehicle communication
port and the electric port connected to the grid.
Teleprotection. The IEEE 1646 standard [8] lists
latency requirements for some of the substation opera-
tions at as little as 1/4 of a cycle (which translates to
about 4 ms and 5 ms, for 60 Hz and 50 Hz AC fre-
quencies, respectively). For applications requiring
communication between substations, the latency
requirements are relaxed to 1/2 cycle. (See Inter-
national Electrotechnical Commission [IEC] specifi-
cation 61850-5 [9] as well as [8].) Thus remote
activation of a protection scheme at a substation is
needed within 8 ms to 10 ms after a fault at that sub-
station has been remotely detected at an adjoining
substation.
Teleprotection applications require extremely
high network availability: failure of such applications
may result in destruction of grid infrastructure and,
potentially, loss of life. For this reason, utilities have
deployed redundant communication links between
substations using a variety of options including pilot
wires, leased voice grade lines, leased data lines, PLC,
and fiber including Ethernet, synchronous optical net-
work/synchronous digital hierarchy (SONET/SDH),
and microwave. To support the low latency require-
ments, connections are typically point-to-point along
the transmission line between the substations, which
is seldom longer than 300 km.
Distribution automation. Distribution automation
extends monitoring and control much deeper into the
distribution network to encompass line reclosers, volt-
age regulators, capacitor banks, sectionalizers, line
switches, fault indicators, circuit breakers, load tap
changers, and transformers. In addition to these ele-
ments, new IEDs will also need to be supported.
To date, power utilities have been accustomed to
managing a limited number of monitoring and control
points, e.g., at hundreds of substations. New commu-
nications technologies need to be introduced into their
grid operations in order to connect tens of thousands of
endpoints encountered in distribution automation in
substations and feeders. Communications with these
widely deployed endpoints can be challenging depend-
ing on the available network access mechanisms.
Micro grid management. The energy management
system (EMS) of a typical utility consists of multiple
centralized or distributed systems. The smart grid may
include and/or connect to “micro grids” possibly man-
aged by individuals or organizations. Taking the sys-
tem approach presented in [11], a micro grid with its
generation, storage, power lines, and loads becomes a
subsystem of the larger utility grid. Micro grids can
be small and simple such as within a home or may
span an interconnection of grid elements in a neigh-
borhood, or over a single feeder-based system, or over
a collection of systems connected to a distribution sub-
station (see Figure 2). In most cases, the micro grid is
an autonomous system since it may be disconnected
(involuntarily or voluntarily) from the larger grid, and
still support its consumers adequately. We assume the
212 Bell Labs Technical Journal DOI: 10.1002/bltj
existence of an EMS for the micro grid that can com-
municate with the EMS of the utility grid or other
micro grids for maintaining grid stability and to sup-
port ADR and other applications.
1. The communication network architecture should
be divided in a hierarchy consistent with micro
grids supporting local reliability if disconnected.
2. It is possible that the owner of the micro grid
communication network is distinct from the
owner of the utility network.
Overview of Smart Grid Application RequirementsTable I lists a few of the applications (or classes of
applications) and their qualitative requirements. Note
that:
1. Some of the table entries are based on [8] and [9].
2. The quantified values of the requirements depend
on the specific nature of applications and associ-
ated utility requirements.
3. “Micro grid management” refers to communica-
tion between a micro grid EMS and the EMSs in
the micro grid hierarchy.
4. Requirements for some of the applications may be
different from Table I under certain circumstances.
For example, lower latency and higher reliability
may be needed for smart metering during ADR
and emergency load management activities.
Network ArchitectureThe new grid in Figure 2 is much more than an
interconnection of transmission lines and distribution
feeders for delivering electricity to homes and busi-
nesses. Our working and descriptive definition of the
smart grid is a power grid, where its applications are man-
aged by state-of-the-art information technologies over an
integrated high-performance, reliable, and secure commu-
nication network.
Data Rate/ (One Way)Application
Scope HSData Volume Latency Reliability Securityor P2P(at Endpoint) Allowance
Smart metering HS Low/v. low High Medium High
Inter-site rapid response (e.g., P2P High/low Very low Very high Very highteleprotection)
SCADA P2P, HS Medium/low Low High High
Operations data HS Medium/low Low High High
Distribution automation HS, P2P Low/low Low High High
Distributed energy managementand control (including ADR, HS, P2P Medium/low Low High Highstorage, PEV, PHEV)
Video surveillance HS High/medium Medium High High
Mobile workforce (push-to-X) HS Low/low Low High High
Enterprise (corporate) data HS Medium/low Medium Medium Medium
Enterprise (corporate) voice P2P Low/v. low Low High Medium
Micro grid management HS, P2P High/low Low High High(between EMSs)
Table I. Qualitative comparison of application requirements.
ADR—Automated demand responseEMS—Energy management systemHS—Hub-spokeP2P—Peer-to-peerP(H)EV—Plug-in (hybrid) electric vehicleSCADA—Supervisory control and data acquisition
DOI: 10.1002/bltj Bell Labs Technical Journal 213
Figure 3 shows the composition of the integrated
communication network for a utility.
The communication network supports communi-
cation between the sensors and actuators attached to
the grid elements, and the smart grid applications
enforcing grid policies through the actuators based on
these measurements. In addition to the smart grid con-
trol applications, the integrated network supports other
utility needs such as multimedia data transfer (e.g.,
closed circuit television [CCTV]) from substations
and voice and data applications for its “enterprise” and
mobile work force. Furthermore, the network must
connect to other utilities’ smart grid networks as well as
to other entities such as the independent system opera-
tors (ISOs) and regional transmission organizations
(RTOs). Finally, for operational simplicity, it is prudent
to implement a network hierarchy consistent with the
micro grid hierarchy as shown in Figure 3.
A smart grid communication network architec-
ture is presented in this section including physical
connectivity architecture, examples of logical con-
nections, access network options, and architectural
implications of shared ownership of networks.
Physical ConnectionsFigure 4 represents the essence of networking
architecture for connecting most of the smart grid and
other endpoints, their locations, and implied applica-
tions. (Also see [17].)
As illustrated, where a network endpoint is
shown to connect to more than one network, not all
connections may be applicable in an implementation.
For example, a building may connect only to one of
four possible network connection options shown: a
renewable energy source may connect to the wire-
less or wireline access network, while a substation
may require connections to more than one network.
It is not possible to show all smart grid elements and
there will be differences from Figure 4 in connection
arrangements in an actual deployed network. For
example, a large traditional generating station in an
outlying area may connect through an access net-
work. Many control and management systems are
named generically: their implementation will depend
on the utilities and vendor products.
Despite the broad consensus that IP is a reasona-
ble choice for a smart grid communication protocol,
Electrical power network
(Hierarchy of)micro grids
Communication network
Enterprisevoice, data
Mobileworkforce
Extranet
Videosurveillance
Sensors and actuators
Figure 3.Communication network beyond smart grid control network.
214 Bell Labs Technical Journal DOI: 10.1002/bltj
NIST has stopped short of mandating IP [17]:
“Among smart grid stakeholders, there is a wide
expectation that Internet Protocol (IP)-based net-
works will serve as a key element for the smart grid
information networks. . . . An analysis needs to be
performed for each set of smart grid requirements to
determine whether IP is appropriate.” We do assume
here that IP is the networking protocol of choice for
the integrated network. Additionally, for many utili-
ties it may be prudent to implement Multiprotocol
Label Switching (MPLS) virtual private networks
(VPNs), with each VPN supporting a set of applications or
user communities. In Figure 4, direct connections
to the core network are assumed to be over point-
to-point (layer 1 or layer 2) connections. There may
be more than one data and control center for relia-
bility and load sharing of the systems. It is expected
that many smart grid applications will require dis-
tributed control and management systems, which
could be located at the substations and in the corre-
sponding micro grids. The network must also con-
nect to the networks of other utilities in the
regional/national smart grid. Also (extranet) con-
nectivity is required to the utility’s partner ISO/RTOs
and corporate service providers for billing, installa-
tion, and other services.
CCTV—Closed circuit TVDER—Distributed energy resourceEMS—Energy management systemHAN—Home area networkIP—Internet Protocol
Power station(large, traditional)
Distributed alternate renewable power generation(e.g., PV, wind, bio mass, tidal, microturbines)
Meter dataman. sys.
Voice/data/push-to-xsystems
SCADAman. sys. …
Utility data and control center
ExtranetconnectivityEMS
RTO/ISO
Wir
eles
s ac
cess
net
wo
rk
Wir
elin
e ac
cess
net
wo
rk
Neighborhood area network
Power line communication network
HAN/(enterprise) LAN
Meter
Building (residential, business, industrial, other)
Distributed energyresources
Utility “pole”
Vehicle(PEV, PHEV)
SCADAMeter
concentrator
Essential
Storage
Vehicle chargingstation
Mobileworkforce
Distributionman. sys.
Videosurveillance
if present if present if present
Billingsystem
Voice/data
Utility office
Micro gridEMS
Storage
Substation
SCADAMeter
concentrator…Protection CCTV
Voicedata
SCADAman. sys.
EMS
if present
EMS
(IP/MPLS) core network
PV
PMU
WAMSman. sys.
PMU—Phasor measurement unitPV—PhotovoltaicRTO—Regional transmission organizationSCADA—Supervisory control and data acquisitionWAMS—Wide area monitoring system
ISO—Independent system operatorLAN—Local area networkMan. sys.—Management systemMPLS—Multiprotocol Label SwitchingPE(H)V—Plug-in (hybrid) electric vehicle
Figure 4.Physical connectivity architecture.
DOI: 10.1002/bltj Bell Labs Technical Journal 215
Every consumer location is expected to have a
smart meter that is connected to the network. A con-
sumer building may have DERs, energy storage ele-
ments, and/or electric vehicles. DERs may be classified
as microgeneration (at residences and small busi-
nesses) or large-scale (at large business or industrial com-
plexes). Depending on their engineered capacity, DERs
provide between 3kW and 10,000kW power. DERs are
typically located in or close to a consumer building
[4]. When present in consumers’ premises, DER, stor-
age, vehicles, and meters may connect over a HAN
or a LAN, which is in turn linked to the communica-
tion network—often through the meter for residential
consumers.
The grid equipment at distribution points
(e.g., utility poles) connects to the network over one
or more of the connections shown in Figure 4.
Standalone (and large scale) alternate/non-traditional/
renewable energy sources, storage elements, and
vehicle charging stations connect to the wireless
and/or wireline access networks. Each micro grid has
its own communication network similar (but smaller
in scale) to that shown in Figure 4. The micro grid
EMS connects to the utility or other micro grid net-
work.
It is expected that Voice over Internet Protocol
(VoIP) will be used for peer-to-peer (P2P) voice and
push-to-talk (PTT) communication of the mobile
work force. It is also expected that mobile wireless
data applications will migrate to broadband. Thus, all
voice, push-to-x, data, and video needs can be satis-
fied by the mobile access terminal connecting to the
wireless broadband access network. Until VoIP com-
munication is available, gateways will be needed to
connect legacy voice systems to the smart grid com-
munication network.
Depending on its location and size, a substation
may house many smart grid and other systems that
require communication with other endpoints. Only
a few of these systems are shown in Figure 4. Figure 5is a more detailed schematic of a substation’s com-
munications infrastructure and its connectivity to vari-
ous possible wide area networks (WANs). This
illustration depicts only the generic systems requir-
ing communication with the outside world.
The IEC 61850 standard provides comprehensive
specifications for substation automation for connect-
ing substation systems that support grid operations.
These systems include legacy analog and digital sys-
tems as well as new IEDs (e.g., for SCADA and pro-
tection). The resulting substation automation LAN
may be implemented as a hierarchy of Ethernet LANs
where a station bus connects many process busses
with each process bus connected to multiple modern
and legacy substation systems. For a typical substa-
tion LAN see [2].
The substation may have a separate LAN for
applications that do not directly contribute to automa-
tion. The substation router may need to connect to
more than one network from the set of networks
shown in Figure 5. Depending on the network tech-
nology, network access adapters or gateways will be
needed. An application such as teleprotection may
need network connections to an adjoining substation
in addition to or instead of the connections through
the substation router. Over time, different networking
technologies have been used with network-specific
gateways facilitating such connections. IEC is working
on extending the 61850 standard to support commu-
nication between substations using multicast over
Ethernet, which would allow substation automation
LANs to connect over an Ethernet network. Further,
tunneling Ethernet over an IP connection through
the router is a possibility if latency objectives can be
met. (See [21] for an overview of teleprotection con-
nectivity options, including Ethernet.)
Logical Connection ModelsIrrespective of their physical network connec-
tions, it is important to determine endpoints of an
application carried over the network. As an illustra-
tion, logical connection models for a few of the appli-
cations are shown in Figure 6, highlighting the
interdependence of the applications—particularly
among ADR, SCADA management, and distribution
management.
Depending on the grid elements under its con-
trol, an EMS communicates only with a subset of the
generation and storage elements shown in Figure 6.
The smart meter that plays a central role in many
applications has been replicated (as entity M) for ease
216 Bell Labs Technical Journal DOI: 10.1002/bltj
of presentation. Newer distribution automation appli-
cations such as the Volt, VAR, Watt control (VVWC)
application will increasingly use periodic as well as
on-demand smart meter measurements. Additionally,
the smart meter is a centerpiece of ADR applications
and home energy management. (Also see [5].)
Networking TechnologiesThis section presents networking technologies for
the network segments shown in Figure 4.
PLC networks. Communication over power line
carriers in various bands between 10 Hz and 500 KHz
have been used by utilities for some of their commu-
nication needs. The main advantage of PLC is the
availability of power lines connecting every endpoint
that needs to be connected to the communication net-
work. However, PLC shortcomings include low data
rates, interference over long distance high voltage
lines, and the need to connect PLCs across trans-
formers (except when the PLC carrier frequency is
identical to the 50 Hz or 60 Hz line frequency.) While
traditionally data rates have varied from about 15 bits
per second for line frequency carriers to about 3 to 4
kilobits per second (kb/s), new orthogonal frequency
division multiplexing (OFDM)-based techniques will
allow for rates as high as 130 kb/s [15]. The current
draft of the emerging IEEE P1901 Broadband Over
Power Line (BPL) standard indicates future support
for much higher rates, i.e., in the tens of megabits per
second range.
Neighborhood area networks. PLC is one example
of NAN connecting buildings in a neighborhood.
Another example is RF mesh over unlicensed spec-
trum, such as over the 900 MHz industrial, scientific,
and medicine (ISM) band or the 2.4 GHz band. While
NANs are predominantly used for smart metering,
One or more network-specific adapters, or gateways
Transmissionline PLC
Feeder PLC NANWireless
access networkWireline
access network
Corenetwork
Substation
Router
Ad
dit
ion
al n
etw
ork
(s)
for
tele
pro
tect
ion
Meterconcentrator CCTV
ADRmanagement
system
Data
Voice…
ADR—Automated demand responseCCTV—Closed circuit TVEMS—Energy management systemLAN—Local area network
EMSPMUProtectionSCADA
managementsystem
SCADA
Ethernet,optical,PDH, SDH/SONETPLC, other
Substation automation LAN(of station bus and process bus)
…
“Gat
eway
”
SCADA—Supervisory control and data acquisitionSDH—Synchronous digital hierarchySONET—Synchronous optical network
NAN—Neighborhood area networkPDH—Plesiochronous digital hierarchyPLC—Power line carrierPMU—Phasor measurement unit
Figure 5.Substation network.
DOI: 10.1002/bltj Bell Labs Technical Journal 217
other applications may also be carried such as SCADA
RTUs and IEDs located at the neighborhood trans-
formers. Lately Wi-Fi has also become a viable NAN
technology for these applications.
Wireless access. Wireless broadband access is one
of the catalysts for successful deployment of the smart
grid. In many countries, with the exception of unli-
censed spectrum, most of the spectrum is owned by
service providers and available for broadband services
over technologies such as Code Division Multiple
Access 2000 (CDMA2000), Enhanced Data Rates for
GSM Evolution (EDGE), Universal Mobile Telecom-
munications System (UMTS), High Speed Packet
Access (HSPA), Worldwide Interoperability for Micro-
wave Access (WiMAX), and Long Term Evolution
(LTE). Depending on the technology and configura-
tion, data rates between 500 Kb/s and 3 Mb/s or more
are possible (1 Mb/s to 6 Mb/s or more on downlinks).
With little spectrum available for their exclusive use,
utilities may subscribe to wireless broadband services
Meter
Meter
Storage Meter
Buildingor other energy
consumption entity
Storage
Power station(large,
traditional)
Distributed alternate renewablepower generation
(e.g., solar, wind, bio mass,tidal, microturbines)
Meter dataman. system
SCADAman. sys.
(Utility) EMS
RTO/ISO
SCADA(sensor)
Micro gridEMS
Electric vehiclechargingstation
Distributionman. sys.
Billingsystem
Measurements(periodic,
on-demand)
Measurements(periodic)
Power qualitymeasurements
(periodic,on-demand)
Measurements(periodic,
on-demand)
On-demand
Regulate
SCADA systems
RegulateStatus,
measurements,incidents
Releaseenergy
Releaseenergy
Increase,decrease
shut-down
Increase,decrease,
shut-down
Increase,decreaseenergy
Demand
Storeenergy
Storeenergy
Demand
Demand
DERElectricvehicle
Building
Micro grid
ADRman. sys.
M
M
M
Homeappliance
Schedule,stop
ADR—Automated demand responseCCTV—Closed circuit televisionDER—Distributed energy resourceEMS—Energy management systemISO—Independent system operatorMan. sys.—Management system
SCADA(actuator)
Regulate
PMU
WAMSman. sys.
(Processed)measurements
Measurements
To otherutilities,
control centers,etc.
Demand
Increase,decrease
Pricing(policies)
Pricingsignals
Pricing signals,periodic poll,on-demand
P(H)EV—Plug-in (hybrid) electric vehiclePMU—Phasor measurement unitRTO—Regional transmission organizationSCADA—Supervisory control and data acquisitionWAMS—Wide area measurement system
On-demand
Measurements
Status
Figure 6.Logical connection models for a few smart grid applications.
218 Bell Labs Technical Journal DOI: 10.1002/bltj
if service providers offer the required coverage, satisfy
the security and reliability requirements, and support
preferential treatment for critical utility applications
when needed. As an example, utilities in the United
States have dedicated narrowband spectrum, essen-
tially to carry voice services over their private mobile
radio network, with little to no prospect for future
broadband allocations. In some cases, utilities may
be able to acquire spectrum for their smart grid net-
work if such spectrum is available with the corre-
sponding product support. The recent allocation of 30
MHz of spectrum in the 1.8 GHz band to electric utili-
ties in Canada is a prime example where spectrum was
indeed assigned but in a band not considered by wire-
less standardization bodies at this time, thus requiring
product customization.
While spectrum can be made available in a few
countries, one possibility is for utilities to enter into a
partnership with wireless service providers or a spec-
trum licensee, for sharing resources such as spectrum,
towers, equipment cabinets, and/or even network
equipment with acceptable logical partitioning. The
relationship between a utility and a wireless service
provider, whether a partnership for resource sharing
or direct customer-provider access agreements, will
have the corresponding impact on network architec-
ture and design.
Wireline access. Broadband wireline services like
digital subscriber line (DSL), cable, Gigabit passive opti-
cal network (GPON), and BPL can provide the band-
width needed for smart meters, SCADA equipment,
and meter concentrators located at the distribution
transformers, or aggregation of substation-based appli-
cations traffic through substation routers. Some utili-
ties may be averse to allow residential broadband
connections to be used for smart meter traffic since
the connection is shared among other applications
of the homeowner. On the other hand, if the broad-
band service and the utility are owned by the local
government of a (small or medium sized) commu-
nity, the residential broadband connection may in fact
be preferred by the utility. (Note that, in spite of its
name—broadband over power line—BPL uses the
power line only between the secondary of the distri-
bution transformer and home. The BPL connection
between the utility pole and the central office is often
a fiber connection).
As noted before, direct SDH/SONET and direct
Ethernet networks are also used for connecting adjoin-
ing substations for applications like teleprotection.
Core network. Depending on the number of com-
munication endpoints of a utility, their locations, their
communication requirements, and other require-
ments, a core network may consist of a router at the
utility data center, or routers connected over an opti-
cal ring or metro-Ethernet in a metropolitan area, or
a mesh of routers connected over point-to-point links.
For the general mesh, the point-to-point links are
either utility-owned or leased from layer 1 (L1) (pri-
vate lines) and L2 (e.g., Ethernet, frame relay) ser-
vice providers. If the performance, reliability, and
security requirements are acceptable, the core net-
work can also be an MPLS VPN from an L3 service
provider with the utility routers connecting to the
provider edge routers of the MPLS VPN service [16].
The utility may itself provide MPLS VPN service so
that groups of users, applications, and/or locations can
manage their networking needs in an autonomous
fashion. In a case where the core network itself is an
MPLS VPN over a service provider network, the VPNs
within the utility can still be created independent of
and within the service provider VPN.
Network Ownership: Utility-Owned Versus Public Carriers
Most utilities prefer exclusive end-to-end owner-
ship of the network though this may not always be
possible because of cost considerations, spectrum availa-
bility, the need for deploying applications in an expe-
dient manner, and other considerations. There are
many advantages and a few serious drawbacks
(e.g., costs) for utility-owned networks over shared own-
ership of network segments with service providers. The
fact that multiple parties share ownership of the net-
work segments within an integrated network does
affect the network architecture’s physical and logical
connectivity, routing, reliability, security, and other
factors. In addition to commercial and business con-
siderations, interoperability agreements between the
utility and network service providers can influence
DOI: 10.1002/bltj Bell Labs Technical Journal 219
the network architecture in its end-to-end network
operations, monitoring, and management challenges.
Even if the utility owns the end-to-end network,
the point-to-point L1 and L2 connections may still be
leased from service provides with acceptable capacity,
reliability, security, and preferably with exclusivity.
However, implementing exclusivity on L1 and L2
links over a wireless service provider network sup-
porting utility requirements for their mission critical
applications can be challenging.
Interconnection With Network of SynchrophasorsWidespread blackouts at the beginning of this cen-
tury have underscored the necessity of wide area mea-
surement systems (WAMS) for regional or national
grids across all the connected utilities’ power systems.
WAMS employ phasor measurement units (PMUs) to
measure voltage and current phasors (phase vectors)
of the corresponding alternating current waveform
and its harmonics. PMUs are considered to be the state
of the art SCADA RTU and are expected to be
deployed at a large number of locations in participat-
ing utility grids. PMUs (often called synchrophasors)
are synchronized to a common clock, usually derived
from the Global Positioning System (GPS). This allows
for time-stamped measurements shared among
utilities, regulatory bodies, and other organizations
through PMU gateways connected to a regional or
national network such as the one being developed and
deployed by the North America SynchroPhasor
Initiative (NASPI) [14]. The NASPI network (NASPInet)
will be a high performance, reliable, and secure com-
munication network that connects PMU gateways
among utilities in a region to a distributed data bus,
allowing for PMU data sharing (almost instantaneously
for some applications) between utilities as well as orga-
nizations such as the North America Electric Reliability
Corporation (NERC).
Therefore, a utility smart grid network will need
to connect to (and be a part of) a network like
NASPInet.
Network Design Principles to Facilitate Smart Grid Applications
This section deals with topology, QoS, and relia-
bility considerations that may differ from the time-
honored design methodologies for service provider
networks and even many large enterprise networks. A
few implementation and product development chal-
lenges also will be identified, with suggestions for
workarounds where possible.
Network TopologyFrom the earlier discussions on smart grid, utility
applications, and the placement of application end-
points, the obvious choice for the network topology is
predominantly a tree structure. The initial topology
of an iterative design will be similar to the one shown
in Figure 7 before QoS, reliability, and other require-
ments are applied to determine the final topology
design.
In addition to the centralized destination of the
utility data and control center for a significant amount
of applications traffic, a substation router is perhaps
the other most identifiable location of traffic aggre-
gation as shown in Figure 5. Traffic from multiple
(smaller) substations may be aggregated at another
(large) substation. Depending on the meter technolo-
gies deployed, the metering traffic may be aggregated
at the meter concentrators at substations, or the
meters or concentrators may connect directly to
the core network. Finally, other smart grid endpoints
such as energy sources and storage units connect to
the substations or directly to the core network routers,
depending on their locations and/or the location of
the corresponding EMSs.
In principle, peer-to-peer applications can be sup-
ported over the tree topology, since connectivity is
always possible through the core routers. However,
the latency requirements of some applications may
not be satisfied by routing that traffic through the core
network. Many of these low-latency applications have
endpoints in adjoining substations. Thus, it is prudent
to maintain direct communication link(s) between
substations as shown in Figure 6 and described fol-
lowing Figure 5. These direct inter-substation links
may additionally provide the possibility of a preferred
path for other P2P traffic such as VoIP bearer.
The topology design is also affected by the fact
that the choice of access network is driven more
by the coverage of a large number of endpoints than
220 Bell Labs Technical Journal DOI: 10.1002/bltj
by the traffic volume. Further, for most centralized
applications, the upstream traffic volume is greater
than the downstream traffic, requiring special design
considerations when carried over service provider
networks that are generally optimized for higher
downstream traffic than upstream traffic.
Integrating Legacy Applications and NetworksWhile new applications and new smart grid sys-
tems are expected to support IP connectivity, the inte-
grated smart grid communication network will also
have to connect to applications at legacy systems for
a period of time. In most cases, gateways to the legacy
systems will be required. Depending on the evolution
of end systems, these gateways can be as simple as
those providing serial-to-Ethernet conversion to those
supporting full application layer protocol conversion.
For a few analog applications including push-to-talk
voice, circuit emulation (time division multiplexing
[TDM] over Ethernet or IP) will have to be provided
until these applications migrate to IP (e.g., VoIP).
Quality of ServiceThe two important QoS factors considered here are
a wide range of latency requirements and (dynamic)
association of flow priority to applications consistent
with smart grid operations. With smaller data vol-
umes, efficiency in bandwidth allocation to applica-
tions may not be the most important QoS objective in
smart grid network design.
Managing latencies. Throughout this paper, the
need for supporting applications with diverse latencies
(from about 8 ms to 1 second or more) was empha-
sized. But lower latency does not always imply higher
(IP/MPLS) core network
Utility data and control center
Substation
SubstationSubstationSubstation
Building BuildingBuilding
SCADA
Meterconcentrator
Mobileworkforce
Micro gridEMS
Micro gridEMS
RTO/ISOPower station
(large,traditional)
Distributed alternate renewablepower generation (including DER)
Vehicle chargingstation
Storage
Mobileworkforce
Distributed alternate renewablepower generation (including DER)
Utility office
Vehicle chargingstation
Substation
DER—Distributed energy resourceEMS—Energy management systemIP—Internet ProtocolISO—Independent system operator
Storage
MPLS—Multiprotocol Label SwitchingRTO—Regional transmission organizationRTU—Remote terminal unitSCADA—Supervisory control and data acquisition
Meter Meter Meter
Figure 7.Network topology.
DOI: 10.1002/bltj Bell Labs Technical Journal 221
priority. For example, according to Internet Engineering
Task Force (IETF) Request for Comments (RFC) 4594
guidelines [3], network control traffic with low
latency (delay tolerance) characterization is given
higher priority with a differential services (DiffServ)
class selector 6 (CS6) while the VoIP bearer traffic
with very low latency is allocated to a lower priority
expedited forwarding (EF) class. In most data net-
working implementations, as in RFC 4594, the VoIP
bearer traffic is given a higher priority than any class
of applications (other than network control), often
assigning it to the strict priority egress queue at each
router. The primary reason for such a high priority is
to manage the jitter and packet loss of the bearer traf-
fic even if there may be real-time or business appli-
cation traffic that requires higher priority. Managing a
diverse set of application priority and latency require-
ments for a smart grid network will require a different
QoS design approach. The QoS design should begin
with listing all utility applications and their priority
and latency requirements. Table II is a sample of
utility applications with their priority and latency
requirements.
Using VoIP bearer traffic as a reference, applica-
tions such as teleprotection and PMU data transfer to
NASPInet have much lower latency requirements
than the latency allowance of up to 175 ms to 200 ms
necessary for good voice quality under most condi-
tions, yet voice traffic latency is considered very low
in RFC 4594. As can be seen from Table II, there are
smart grid applications with much lower latency
requirements and very high priority. The only plausi-
ble design choice for satisfying 8 ms to 16 ms latency
requirements may be in directly connecting the appli-
cation endpoints such as two substations, thus elimi-
nating intermediate hops and reducing propagation
LatencyApplication (only a few allowanceexample applications (assumed,
considered) Application setting unverified) Comments
Teleprotection All 8 ms, 10 ms For 60 Hz and 50 Hz, respectively
Phase measurement unit Class A data service 16 ms60 messages per second stipulated for Class A data service in [14]
Push-to-talk signaling Incident-related 100 ms
Connect to many Example: ADR within 1 minute for up Smart meter meters in a short 200 ms to 300 meters connected over a shared
time medium
SCADA data: poll response 200 ms See [8].
VoIP bearer 175–200 ms Includes P2P and all PTT
VoIP signaling 200 ms Includes non-incident-related PTT
Phase measurement unit Class C data service 500 msPost event (latency value assumed).See [14].
On demand SCADA 1 second See [8].
Smart meterPeriodic meter
� 1 secondSay, once an hour or lower frequency
reading of reading
Table II. (Representative) latency requirements of smart grid applications.
ADR—Automated demand responseP2P—Peer-to-peerPTT—Push-to-talkSCADA—Supervisory control and data acquisitionVoIP—Voice over IP
In t
he
ord
er o
f d
ecre
asin
g p
rio
rity
c
222 Bell Labs Technical Journal DOI: 10.1002/bltj
delay (also see Figure 7). Packet loss and jitter con-
siderations for some other applications (e.g., VoIP)
carried over this inter-substation connection may
have to take a back seat, with lower priority. (But
note that many new codecs do correct for packet loss
and jitter.) Even though RFC 4594 lists many appli-
cation classes that may be implemented with judi-
cious choices of DiffServ values, for many practical
QoS implementations in service providers or enter-
prise data networks supporting typical multimedia
applications, four or fewer classes of service are usu-
ally provided. It is up to the network customers to
map their applications to the pre-defined QoS classes.
One such typical classification is shown in Figure 8,
alongside a more granular smart grid application pri-
ority hierarchy that is similar to Table II.
Thus with DiffServ QoS, differential services code
point (DSCP) allocation to smart grid applications will
have to be different from RFC 4594 guidelines. In
addition, while preemption of a packet under trans-
mission (either one partially transmitted or transferred
to a very small line buffer just before transmission) is
not allowed in most products or network implemen-
tations, critical high priority smart grid applications
may require the preemption feature. The use of inte-
grated services (IntServ) QoS will have to be consid-
ered for some of the applications. Since existing
networking products may not support some of these
Dec
reas
ing
pri
ori
ty
Network controlNetwork control
Teleprotection
PMU (class A data service)
PTT signaling (incident-related)
Smart metering(access many meters in a short time)
SCADA (poll response)
VoIP bearer (including PTT)
VoIP signaling (including some PTT)
PMU (class C data service)
On demand SCADA
Smart metering(periodic meter reading)
Critical enterprise/operation data
Non-criticalenterprise/operations data
Active ADR
Best effort data
VoIP bearer
Critical data
VoIP signaling
Video Video
Non-critical data
Best effort data
Smart grid application priorities Typical multimedia networkapplication priorities and QoS classes
Class 1
Class 4
Class 3
Class 2
ADR—Automated demand responsePMU—Phasor measurement unitPTT—Push-to-talk
QoS—Quality of serviceSCADA—Supervisory control and data acquisitionVoIP—Voice over IP
Figure 8.Smart grid application priorities.
DOI: 10.1002/bltj Bell Labs Technical Journal 223
required features, clever workarounds will be required;
using per flow QoS is one such possibility. Future net-
working product architects must seriously consider
developing features that support the smart grid com-
munication networks’ QoS requirements.
Finally, the importance of respecting latency
requirements of some of the ADR applications per-
taining to renewable variable energy sources such as
wind and PV cannot be overstated. Longer transients
due to a lack of efficient QoS design can lead to per-
turbation in the power grid.
Managing dynamic flow priority. Latency tolerance
and/or priority for smart grid applications may depend
on the context or setting of the corresponding grid
operation or environment. For example (see Table II
and Figure 8), periodic meter reading traffic may be
given lower priority and liberal delay allowance,
whereas traffic from meters located in an area with
active demand response processes or outage manage-
ment must be given higher priority and a lower delay
allowance. Setting a higher priority than normal to
PTT signaling and bearer traffic during an emergency
and widespread blackout or to video surveillance traf-
fic after detection of an incident are other examples.
One possible workaround is to treat an application
with varying latency and priority requirements as
multiple distinct applications. If DSCP is used, depend-
ing on the application setting, different DSCP value
should be assigned to the same application. Since
workarounds may not always be possible, standards
and product capabilities are needed for a generalized
mapping of the tuple �application, priority� to a QoS
class.
ReliabilityIt is extremely important that the (smart) grid
reliability requirements are translated into consistent
communication reliability requirements.
As noted earlier, communication network relia-
bility objectives for different applications can be very
different: 99.95 percent availability (with an average
downtime of 263 minutes/year) may be sufficient for
periodic meter reading, but 99.999 percent availabil-
ity (with an average downtime of 5.3 minutes/year)
may be low for teleprotection. To support the latter
requirement, multiple point-to-point connections
between a pair of adjoining substations are essential.
Depending on the connection option, availability of
the corresponding IP network elements, regulatory
requirements and standards, and/or a utility’s prefer-
ence, one or more of these inter-substation connec-
tions may not be included in the integrated IP
network design. However, every effort must be made
to include at least one of these links in the IP network
(e.g., Ethernet connection [21]). In addition to incor-
porating the teleprotection application in the
integrated network, such interconnections between
substations provide increased reliability for all appli-
cations carried over that connection. If needed, a judi-
cious design of the routing protocol and/or MPLS VPN
implementation will help limit the use of these mul-
tiple links to applications with high reliability require-
ments and for peer-to-peer applications.
Conventional but critical reliability design ele-
ments including substation communication link diver-
sity, redundant meter concentrators, and disaster
recovery plans for data centers and other establish-
ments must be considered to achieve the required net-
work availability goals. Since the communication
network is used for managing the power grid, it
is imperative that the network elements are not
impaired by power outages. At a minimum, battery
backups or an uninterruptible power supply (UPS) is
necessary for communication systems as well as for
some end systems. Further, PLC cannot be the sole
means of connectivity for many applications.
Green BenefitsThe introduction of smart grid applications does
contribute to green benefits as highlighted in the
Smart2020 report [10], which predicts a 14 percent
reduction by year 2020 for global carbon emission
attributable to smart grid evolution, corresponding to
a reduction of 2.03 GtCO2e, from the current emis-
sion of 14.26 GtCO2e. It is believed that 24 percent
of the total carbon emission today is attributed to the
power sector. Accordingly, the expected reduction in
overall carbon emission due to the electric power sec-
tor should be about 3 percent. Of course, this is pos-
sible partly due to the use of renewable and alternate
energy sources, peak power reduction, and energy
224 Bell Labs Technical Journal DOI: 10.1002/bltj
sources closer to load. The last item mostly refers to
reduction in transmission (I2R) losses, since a signifi-
cant amount of the electricity carried over transmis-
sion lines is generated by fossil fuels today.
An integrated communication network helps
make a smart grid more efficient, thus indirectly con-
tributing to green benefits.
There are also some green benefits, however
small, that are directly related to the time saved in
automated policy execution through communication
networks. For example, automated demand response
with effective communication can reduce the amount
of time needed for manually shifting from bulk elec-
tric sources to DER. For illustration, if it takes 20 min-
utes for manual shifting of an energy source, for every
kW of power shifted using ADR, 0.333 kWh less
energy will be drawn from the bulk electricity source
than with manual operation. Under the assumption
that the bulk electricity source is coal, a 40 percent
thermally efficient power plant could produce an
average of about 0.83 kg of CO2 emissions per kWh of
generated electricity [19]. Further, if the resulting
DER used with ADR is a renewable source of energy,
our assumption of a 20 minute savings in ADR opera-
tion would yield a reduction of 0.277 kg in CO2 emis-
sion for every kW of power shifted. Assuming that
such a shift of energy sources occurs once every day,
the average annual carbon savings achieved by the
use of ADR over a manual demand response opera-
tion is about 100 kg of CO2 for every kW of power
shifted.
ConclusionsThis paper presents a network architecture for an
integrated high performance and highly reliable com-
munications network for the successful deployment
and operation of a smart grid. The architecture frame-
work was driven by the smart grid applications—
mission-critical and otherwise—as well as other utility
applications that must be carried over the integrated
network that meet or exceed their individual require-
ments. Throughout the paper, a few representative
smart grid applications were used to illustrate the net-
work architecture. These applications ranged from
smart metering with a very large number of endpoints,
to teleprotection with extremely low latency require-
ments, to communication between autonomous
micro grid networks. Network security (including
cyber security), an extremely important aspect of
network architecture, was not considered in this
paper.
It is clear that communication network design for
the smart grid requires network topology, QoS, and
reliability considerations that may not be common-
place in designing service provider or enterprise data
networks. While it may not be possible to implement
the optimal design with available product and net-
work technologies, workarounds may be used.
Finally, the “green benefits” of the smart grid—
and by implication, that of the integrated communi-
cation network—were presented in terms of carbon
reduction.
Recommendation for Future WorkWe believe that the correlation between the smart
grid architecture and the corresponding physical and
logical connectivity of the network architecture must
be exploited in developing the smart grid architec-
ture. This holistic view of the smart grid and its com-
munication network will facilitate an easier extraction
of the network architecture from the smart grid archi-
tecture, including direct connection between their
respective performance, reliability, and security re-
quirements. Even if such “greenfield” smart grid
implementations may not be practical in most
instances, ongoing development of smart grid archi-
tecture and design, as well as new grid applications,
can facilitate the corresponding development in net-
work architecture and design. For example,
1. Tools that help determine network configurations
as an integral part of new application develop-
ment and deployment.
2. Network protocols that help reduce power tran-
sients, particularly those attributable to variable
energy resources connected into the grid.
3. Automatic setting of QoS configurations when
application requirements change based on grid
events.
4. Translating the self-healing grid to the self-healing
communication network.
DOI: 10.1002/bltj Bell Labs Technical Journal 225
Additional recommendations for future work:
1. QoS management of applications traffic with a
large variety of performance requirements includ-
ing latency and priority.
2. Extending the smart grid architecture to specific
micro grids such as a micro grid spanning a build-
ing, a feeder, and an electric vehicle charging
station.
3. Extending the architecture and design principles
introduced in this paper to include network secu-
rity. It is important to note that security consid-
erations must be incorporated at the beginning of
network architecture and design process.
AcknowledgementsWe want to thank Marc Benowitz and Sam
Samuel, co-editors of this special issue, and an anony-
mous reviewer for their review and valuable sugges-
tions. We also thank Joe Morabito for his comments
on an early draft of the paper.
*TrademarkIntelligrid is a registered trademark of Electric Power
Research Institute, Inc.
References[1] R. Albert, I. Albert, and G. L. Nakarado,
“Structural Vulnerability of the North AmericanPower Grid,” arXiv:cond-mat/ 0401084v, Jan.2004, �http://arxiv.org/ PS_cache/cond-mat/pdf/0401/0401084.pdf�.
[2] L. Andersson, C. Brunner, and F. Engler,“Substation Automation Based on IEC 61850with New Process-Close Technologies,” Proc.IEEE Bologna PowerTech Conf. (PowerTech‘03) (Bologna, It., 2003), vol. 2.
[3] J. Babiarz, K. Chan, and F. Baker,“Configuration Guidelines for DiffServ ServiceClasses,” IETF RFC 4594, Aug. 2006,�http://www.ietf.org/rfc/rfc4594.txt�.
[4] California, The California Energy Commission,“California Distributed Energy ResourcesGuide,” 2009, �http://www.energy.ca.gov/distgen/index.html�.
[5] Electric Power Research Institute, Report toNIST on the Smart Grid InteroperabilityStandards Roadmap, Contract No. SB1341-09-CN-0031—Deliverable 7, June 17, 2009,�http://nist.gov/smartgrid/InterimSmartGridRoadmapNISTRestructure.pdf�.
[6] Electric Power Research Institute andElectricity Innovation Institute, IntelliGridArchitecture Report: Volume 1, IntelliGridUser Guidelines and Recommendations,1012160, Final Report, 2002.
[7] L. L. Grigsby (ed.), Electric Power Generation,Transmission, and Distribution, 2nd ed., CRCPress, Boca Raton, FL, 2007.
[8] Institute of Electrical and ElectronicsEngineers, “IEEE Standard CommunicationDelivery Time Performance Requirements forElectric Power Substation Automation,” IEEE1646–2004, Feb. 25, 2005.
[9] International Electrotechnical Commission,“Communication Networks and Systems inSubstations,” IEC 61850-1–61850-10, 2003.
[10] C. Kruse and B. Singanayagam, “SMART2020:ICT and a Low-Carbon Economy,” J. P.Morgan, Europe Equity Research, July 9,2008.
[11] R. H. Lasseter and P. Paigi, “Microgrid: AConceptual Solution,” Proc. 35th Annual IEEEPower Electronics Specialists Conf. (PESC ‘04)(Aachen, Ger., 2004), vol. 6, pp. 4285–4290.
[12] Lawrence Berkeley National LaboratoryDemand Response Research Center, OpenAutomated Demand ResponseCommunications Specification (Version 1.0),CEC-500-2009-063, California EnergyCommission, Public Interest Energy ResearchProgram, Apr. 2009, �http://drrc.lbl.gov/openadr/pdf/cec-500-2009-063.pdf�.
[13] Michigan, Department of Labor and EconomicGrowth, “21st Century Energy Plan,”Alternative Technologies Workgroup, MeetingHandout 1, July 27, 2006, �www.dleg.state.mi.us/mpsc/electric/capacity/energyplan/alttech/smartgrid_draftreportoutlinejul19_2006.pdf�.
[14] National Energy Technology Laboratory,“Specification for North AmericanSynchroPhasor Initiative (NASPI),”Attachment A, Statement of Work, May 2008,�http://www.naspi.org/ resources/dnmtt/naspinet/quanta_sow.pdf�.
[15] PRIME Project, “PHY, MAC and ConvergenceLayers,” White Paper, v1.0, 2008, �http://www.iberdrola.es/webibd/gc/prod/en/doc/MAC_Spec_white_paper_1_0_080721.pdf�.
[16] E. Rosen and Y. Rekhter, “BGP/MPLS IPVirtual Private Networks (VPNs),” IETF RFC4364, Feb. 2006, �http://www.ietf.org/rfc/rfc4364.txt�.
226 Bell Labs Technical Journal DOI: 10.1002/bltj
[17] United States, Department of Commerce,National Institute of Standards andTechnology, Office of the National Coordinatorfor Smart Grid Interoperability, NIST SpecialPublication 1108, NIST Framework andRoadmap for Smart Grid InteroperabilityStandards, Release 1.0, Jan. 2010,�http://www.nist.gov/public_affairs/releases/smartgrid_interoperability_final.pdf�.
[18] United States, Department of Energy, EnergyInformation Administration, “InternationalEnergy Statistics, Total Electricity NetConsumption (Billion Kilowatthours),”�http://tonto.eia.doe.gov/cfapps/ipdbproject/iedindex3.cfm?tid�2&pid�2&aid�2&cid�&syid�1980&eyid�2007&unit�BKWH�.
[19] United States, Department of Energy, EnergyInformation Administration, “VoluntaryReporting of Greenhouse Gases Program—Fuel and Energy Source Codes and EmissionCoefficients,” �http://www.eia.doe.gov/oiaf/1605/coefficients.html�.
[20] United States, Department of Energy, FederalEnergy Regulatory Commission, 2007Assessment of Demand Response andAdvanced Metering, Staff Report, Sept. 2007,�http://www.ferc.gov/legal/staff-reports/09-07-demand-response.pdf�.
[21] S. Ward, W. Higinbotham, E. Duvelson, and A. Saciragic, “Inside the Cloud—NetworkCommunications Basics for the RelayEngineer,” Proc. 61st Annual Conf. forProtective Relay Engineers (CPRE ‘08)(College Station, TX, 2008), pp. 273–303.
(Manuscript approved March 2010)
KENNETH C. BUDKA is a senior director of the Network and Performance Reliability Department atAlcatel-Lucent Bell Labs in Murray Hill, New Jersey. He received a B.S. degree(summa cum laude) in electrical engineeringfrom Union College in Schenectady, New
York, and M.S. and Ph.D. degrees in engineering sciencefrom Harvard University in Cambridge, Massachusetts.Dr. Budka’s professional interests are in the devel-opment of next-generation wireless and wirelinecommunication technologies and their application tomission-critical communications systems for publicsafety agencies and utilities. He is a senior member ofthe IEEE and a former distinguished member oftechnical staff at Bell Labs. He holds 18 U.S. patents
JAYANT G. DESHPANDE is a member of technical staff in the Network Performance and ReliabilityDepartment at Alcatel-Lucent Bell Labs inMurray Hill, New Jersey. He holds a B.E.degree from Nagpur University, India;master’s degrees from the Indian Institute
of Technology, Kanpur, India, and Princeton University,Princeton, New Jersey; and a Ph.D. from the Universityof Texas at Austin. His professional interests are insmart grid architecture, design, and performance. Hehas spent the last 26 years at Alcatel-Lucent Bell Labsand AT&T Labs with a brief tenure at Cisco Systems. He has worked on data and voice networking servicesdevelopment, network architecture, design, and QoS.Dr. Deshpande was a faculty member of computerscience and electrical engineering at the IndianInstitute of Technology, New Delhi, India, from 1973 to 1982. After spending one year as a visiting facultymember at Pennsylvania State University, he joined BellLabs in 1983.
TEWFIK L. DOUMI is a principal in the Network and Performance Reliability Department atAlcatel-Lucent Bell Labs in Murray Hill,New Jersey. He holds a license in physicsfrom the University of Algiers, Algeria; anM.S. degree in electrical engineering from
Stevens Institute of Technology, Hoboken, NewJersey; and a Ph.D. degree in electrical engineeringfrom the University of Bradford in England. Dr. Doumi’s professional interests are in spectrummanagement and radio engineering techniques fornext-generation wireless systems. He is a member ofthe Alcatel-Lucent Technical Academy and a seniormember of the IEEE.
MARK MADDEN is the regional vice president for Energy Markets in Alcatel-Lucent’sAmericas Region. He is responsible forAlcatel-Lucent’s North American marketstrategy, strategic partnerships, andbusiness development in the utility, oil,
and gas markets. Mr. Madden joined Alcatel-Lucent in1996. He has over 25 years experience with leadingcompanies in the information and communicationstechnologies industry and has been actively engagedproviding consulting to various customers within the electric utility sector on mission-criticaltelecommunications technologies for the last five years.
DOI: 10.1002/bltj Bell Labs Technical Journal 227
TIM MEW is a member of Alcatel-Lucent’s Global Services team, responsible for defining,developing, and managing complexsystems integration services solutions forthe energy and utilities sector. Prior to this,he was the head of the Solution Design and
Innovation team in Australasia, focusing on railways,highways, oil and gas, and security services solutions.Mr. Mew’s generalist background has includedarchitecture, technology planning, and servicedevelopment disciplines in the technology areas ofnext-generation networks, VoIP, intelligent networks,PSTN, CTI, Internet and IP, CCTV, e-commerce, andwireless communications. Before joining Alcatel-Lucent, he held diverse roles ranging from architecturemanager, senior engineer, to brand manager and CTOroles in a number of industries including carriers, ISPs,and e-commerce. ◆
top related