High Speed Optical Networks:
An Evolution of Dependency
November 2, 2001
Todd Sands, Ph.D
WEDnet Project
www.wednet.on.ca
University of Windsor
Latency
• The result of an event in time that slows the transport or processing of information
• E.g. Machine (processing) latency in microsecs (n =1.2)
• E.g. Network latency in millisecs (x < 130 ms)• Optical transport max. = 300,000 km/sec• Physical parameters of the transport media • Convergence of voice, image and data in the path• Switched cells and packet network behaviours• Potential of WDM optically switched and SONET
architectures
OSI Reference Model – Networking 101
• Application• Presentation• Session• Transport• Network• Data-link• Physical
When two computers communicate on a network, the software at each layer on one computer assumes it is communicating with the same layer on the other computer.
e.g. For communication at the transport layers, that layer on the first computer has no regard for how the communication actually passes through the lower layers of the first computer, across the physical media, and then up through the lower layers of the second computer.
Do we know the effects of latency!
Suspect that the answer is yes! We see it every day! No. of processors, power requirements, processing capability, storage
capacity, and the needs of research that use most facilities can be intensive. HPCS resources supplied and funded through a needs-based process, but
this can also be because of research What about a GRID? Is it on the same path? Are we mindful of details, such as latency…with respect to one of the most
fundamental parts of the GRID… THE NETWORK Do we know how computing resources connect to the outside world?…
Maybe… Do we have any control over the “extranet”?
Primary Network InterfaceTo Machine Resources
These switches provideEthernet to ATM SONET WAN interfacesfor TCP/IP traffic
PACKETS VS. CELLS VS. FRAMES
• Frames – used for larger data amounts over high-speed, low error rate links– 2,000 – 10,000 characters in size– Data corrections not link by link– Therefore link by link error checking impacts network latency greatly
• Packets – used for smaller data amounts across lower speed, high error rate links– 128 – 256 (bytes) characters in size– Lower chances of error in each packet, small amounts re-transmitted– Prioritization through tagging of packets leads to QoS
• Cells – very small amounts of data with sometimes no error checking– Highly reliable optical networks sometimes with no error checking– Up to 48 - 53 (bytes) characters in size– Small size allows for load balancing of traffic on network– No payload in cells, no transmission - full payload, then transmission– Uses ATM Adaptation Layers – AAL’s 1-5 for shaping the network
Optical Carrier Designations
• OC-1/STS-1 51.84 Mbps
• OC-3 155.52 Mbps
• OC-12 622.08 Mbps
• OC-48 2,488.32 Mbps
• OC-192 9,953.28 Mbps
• OC-768 39,813.12 Mbps
SONET• digital hierarchy based on Optical Carriers
(OC’s)
• maximum t-speed of 39.81312 Gbps
• defines a base rate of 51.84 Mbps = STS-1s
• OC’s are multiples of the t-speed
• defines Synchronous Transport Signals – STS’s and STS-3c = OC 3 = 155 Mbps
Overheads
• SONET carries 8,000 frames per second, 810 characters in size (36 characters of overhead and 774 characters of payload
• Section Overhead includes:– STS channel performance monitoring
– Data channels for management such as channel monitoring, channel administration, maintenance functions and channel provisioning
– Performs functions necessary for repeaters, add drop multiplexers (ADMs), termination gear, and digital access and cross connect systems (DACS)
• Line Overhead includes:– STS-1c performance monitoring
– Data channel management, payload pointers, protection switching information, line alarm signals, and far-end failure to receive indicators
• In addition to these overheads there are also Path overheads
Optical Wave Division• WDM multiplies (up to 32 more times) the capacity of
existing fibre spans – cross (wide)-band, narrow band or dense band transmission options
• DWDM Red waves 1550, 1552, 1555 & 1557 nm
• DWDM Blue waves 1529, 1530, 1532& 1533 nm
• Now can support 100 wavelengths with each wavelength supporting a channel rate of up to 10 Gbps
Local Area Access Architectures
ATM Network – OC12-OC48
GbE
Access Routers
1MM
System Processors and Interfaces 100 Mb- 1Gb
Router OC-12ATM
GbE
1MM
1-Meg or xDSL Modem Services in Communities
PVCs – on carriers network
Alternate CarrierMANs also Interface
1000 Mb GbE
Grid Access Node – GigaPoP?
Off Ramps -WDM
Central CO for Access Nodes
All PVCs (SVCs or PVPs) usually terminate on 1 or more Centralized Access Routers
Most carrier PVCs are UBR with access at minimum OC48 speeds 2.4 Gb/sec
Backbone may be optically switched with P.O.S on wavelengths using TCP/IP as the
main transport protocol but getting direct access to it is the key!
Direct access will also minimize latency and the synergistic effects of latency
What does a 5 minute average measurementshow us with MRTG?
PC 1MM
EthernetSwitch
(Catalyst)Network
(ATM)LAC
(SMS-1000)1MMDBIC
Network Protocol Stack Models (WAN with IP)
SONET/SDH1MMQAM
ATMATM
SARSAR
AAL5AAL5
LLC/SNAP(1483)
Ethernet
1MMQAM
SONET/SDH
ATMATM
SONET/SDH
ATMATM
SARSAR
AAL5AAL5
LLC/SNAP(1483)
Ethernet
PPPOEPPPOE
SONET/SDH
ATMATM
SARSAR
AAL5AAL5
LLC/SNAP(1483)
IP
UDP
L2TP
Ethernet Ethernet
10BaseT
Ethernet
PPPOEPPPOE
PPPPPP
IP
10BaseT
Ethernet
SONET/SDH
ATMATM
SARSAR
AAL5AAL5
LLC/SNAP(1483)
IP
UDP
L2TP
PPPPPP
IP
LNS
Making a CallTelevision
Video
V-Room
LE25
WEDnet uses WUC asa carrier such as Bellor METROnet with core gearLS1010 and 7200 series forATM and IP routing
TelevisionVideo
V-Room
LE25
IBM 8274 9 slot
LE 25 SMF
P-Tel Video
FVC VGATE
Universityof
Windsor
SharedH.261 ISDN
25 Mb ATM
OC3
Dial - up
FVC V-room
WRH WesternCampus
HDGH
LE25
Making a CallTelevision
Video
V-Room
LE25
WEDnet uses WUCas a carrier suchas a Bell or METROnetwith core gear LS1010 and 7200 series for ATM andIP routing
TelevisionVideo
V-Room
LE25
IBM 8274 9 slot
LE 25 SMF
P-Tel Video
FVC VGATE
Universityof
Windsor
SharedH.261 ISDN
25 Mb ATM
OC3
Dial - up
FVC V-room
WRH Western
Campus
HDGH
LE25
Making a CallTelevision
Video
V-Room
LE25
WEDnet uses WUC asa carrier such as Bellor METROnet with core gear LS1010 and 7200 series for ATM andIP routing
TelevisionVideo
V-Room
LE25
IBM 8274 9 slot
LE 25 SMF
P-Tel Video
FVC VGATE
Universityof
Windsor
SharedH.261 ISDN
25 Mb ATM
OC3
Dial - up
FVC V-room
WRHWestern
Campus
HDGH
LE25
Codec NegotiationTelevision
Video
V-Room
LE25
WEDnet uses WUC asa carrier such as a Bellor METROnet with core gearLS1010 and 7200 series for ATM and IP routing
TelevisionVideo
V-Room
LE25
IBM 8274 9 slot
LE 25 SMF
P-Tel Video
FVC VGATE
Universityof
Windsor
SharedH.261 ISDN
25 Mb ATM
OC3
Dial - up
FVC V-room
WRHWestern
Campus
HDGH
LE25
Successful CallTelevision
Video
V-Room
LE25
WEDnet uses WUC asa carrier such as a Bell or METROnet with core gear LS1010 and 7200 series forATM and IP routing
TelevisionVideo
V-Room
LE25
IBM 8274 9 slot
LE 25 SMF
P-Tel Video
FVC VGATE
Universityof
Windsor
SharedH.261 ISDN
25 Mb ATM
OC3
Dial - up
FVC V-room
WRHWestern
Campus
HDGH
LE25
Making an ISDN Call
TelevisionVideo
V-Room
LE25
WEDnet uses WUCas a carrier such as a Bellor METROnet with core gearLS1010 and 7200 series forATM and IP routing
TelevisionVideo
V-Room
LE25
IBM 8274 9 slot
LE 25 SMF
P-Tel Video
FVC VGATE
Universityof
Windsor
SharedH.261 ISDN
25 Mb ATM
OC3
Dial - up
FVC V-room
WRHWestern
Campus
HDGH
LE25
Making an ISDN Call
TelevisionVideo
V-Room
LE25
WEDnet uses WUC asa carrier such as a Bellor METROnet with coregear LS1010 and 7200series for ATM and IProuting
TelevisionVideo
V-Room
LE25
IBM 8274 9 slot
LE 25 SMF
P-Tel Video
FVC VGATE
Universityof
Windsor
SharedH.261 ISDN
25 Mb ATM
OC3
Dial - up
FVC V-room
WRHWestern
Campus
HDGH
LE25
Making an ISDN Call
TelevisionVideo
V-Room
LE25
WEDnet uses WUC asa carrier such as a Bellor METROnet with coregear LS1010 and 7200series for ATM and IProuting
TelevisionVideo
V-Room
LE25
IBM 8274 9 slot
LE 25 SMF
P-Tel Video
FVC VGATE
Universityof
Windsor
SharedH.261 ISDN
25 Mb ATM
OC3
Dial - up
FVC V-room
WRHWestern
Campus
HDGH
LE25
TelevisionVideo
V-Room
LE25
WEDnet uses WUCas a carrier such as a Bellor METROnet with core gearLS1010 and 7200 series for ATM and IP routing
TelevisionVideo
V-Room
LE25
IBM 8274 9 slot
LE 25 SMF
P-Tel Video
FVC VGATE
Universityof
Windsor
SharedH.261 ISDN
25 Mb ATM
OC3
Dial - up
FVC V-room
WRHWestern
Campus
HDGH
LE25
Making an ISDN Call
TelevisionVideo
V-Room
LE25
WEDnet uses WUCas a carrier such as a Bell or METROnet with coregear LS1010 and 7200 seriesfor ATM and IP routing
TelevisionVideo
V-Room
LE25
IBM 8274 9 slot
DEC Gigaswitch18 gbps
LE 25 SMF
P-Tel Video
FVC VGATE
Universityof
Windsor
SharedH.261 ISDN
25 Mb ATM
OC3
OC3
Dial - up
FVC V-room
WRHWestern
Campus
HDGH
LE25
TelevisionVideo
Centrex module
Leamington District Memorial Hospital
Codec Negotiation
TelevisionVideo
V-Room
LE25
WEDnet uses WUC asa carrier such as a Bellor METROnet with core gear LS1010 and 7200 series for ATM and IP routing
TelevisionVideo
V-Room
LE25
IBM 8274 9 slot
LE 25 SMF
P-Tel Video
FVC VGATE
Universityof
Windsor
SharedH.261 ISDN
25 Mb ATM
OC3
Dial - up
FVC V-room
WRHWestern
Campus
HDGH
LE25
TelevisionVideo
Centrex module
LeamingtonDistrict MemorialHospital
Successful Call
April 20, 2023 Bhavani Krishnan
AT&T and Regional Gigapop
IP Architecture
ATM interconnectivity
Router / RFC1577 Client
LAN interconnect
WEDNetSureNet
ATM/w SVC
AT&T RouteServer
AT&T Gigapop
AT&T Network
CA*net3
OCRINet /wOHIiB
GP
iBGP
iBG
P
AT&T AS iBGP
CA*Net AS iBGP
Regional IGP
Reg
iona
l IG
P
Reg
iona
l IG
P
BGPiBGP
From LAN to WAN
This server and control facility houses multiple Digital Alpha, DEL PowerEdge, IBM Netfinity and RS/6000 servers. Located at a single campus the facility supports 400 nodes locally and 800 nodes 7.5 km away. SVCs are provisioned on separate PVPs for security and LANE services provide VLANs for ADT systems, pharmacy, and document imaging. The systems use GUI interfaces to assist visual references for end-users
In the Ideal World!
Dark fibre between nodes Homogenous switched architecture with minimal breakouts Low latency at all layers
We will likely be dealing with something much different, unless there is about
$500 M available to support and sustain the network side of grids to help
minimize the synergistic effects of latency on applications Latency studies are important and the synergy of latency effects are
important from the processor to the I/O architectures, to the network layers If commercial carriers are to be used anywhere in the path, latency should
become a factor for selecting them as providers Effective monitoring and support of the extranet is important to the success
of a GRID unless the GRID middleware can accommodate different types of
latency and the variation that exists Internet routing is “best effort” with variable paths every time – not likely the
best GRID platform Research networks like CA*net 3, Internet 2, ORION, etc. are the next best
bet! However, the last mile issue still has to be addressed.
The Future• “It is conceivable that future Internet networks may be a seamless composite of a variety of transport
protocols. An Optical Internet might be used for high volume, best efforts computer to computer traffic, while IP over ATM might be used to support VPNs and mission critical IP networks, while IP over SONET would be used to aggregate and deliver traditional IP network services that are delivered via T1s, DS3s, and Gigabit uplinks”
• From, Dr. Bill St. Arnaud, CANARIE