optiputer infostructure: east of i-5*
Post on 12-Feb-2016
48 Views
Preview:
DESCRIPTION
TRANSCRIPT
OptIPuter Infostructure: East of I-5*
Tom DeFanti, Maxine Brown, Jason Leigh, Oliver Yu, Tom Moher, Bob Grossman, Joe Mambretti, Valerie Taylor, Cees de Laat
“East of I-5” OptIPuter
• “East of I-5” OptIPuter is focused on large-data applications using experimental technology
• Experimental means that – It is obtainable and affordable– It works 99% of the time– It is programmable– Brute force is acceptable
• Our goal: to make photonics controllable by Grid middleware as soon as possible
What is a Lambda?
• A lambda, in networking, is a fully dedicated wavelength of light in an optical network, typically used today for 1-10Gbps.
• We are now working with 1Gb dedicated layer2 circuits that act like lambdas
• We need enough to schedule and manipulate: up to 40 1Gb “sub” lambdas will be available to the OptIPuter locally, regionally, internationally
• We expect 10Gb lambdas to be available to the OptIPuter in a few years; first locally, then regionally, then (inter)nationally
Gross Optical Burst Switching (GOBS)
• Move a terabyte or petabyte on a schedule• Roam a 375600 x 375600 pixel remote database• Applications will be able to request dedicated
Lambdas using the routed infrastructure• Bypass the routers in between
PC
PC
PC
PC
PC
PC
Photonic Switch(Glimmerglass, Calient)
Cluster Visualization
5x3 Grid of 1280x1024 Pixel LCD Panels Driven by 16-PC Cluster Resolution=6400x3072 Pixels, or ~3000x1500 pixels in Autostereo
NTT 4Kx2K Compressed Video from Chicago to Los Angeles
• Pre-compressed to 300 Mbps in Chicago using an experimental JPEG 2000 SHD codec
• Received in LA at USC Zemeckis Center by an NTT real-time decoder, and fed to NTT's prototype SHD frame-buffer and 8-megapixel full-color D-ILA projector for display on a large screen.
• SHD= 4xHDTV or 16xDVD
http://www.ntt.co.jp/news/news02e/0211/021113.html
• Lambdas for high bandwidth applications:– Bypass production network– Middleware request for
optical pipes• Rationale:
– Lower the cost of transport per packet
Application
Middleware
Transport
Application
Middleware
Transport
Router
Router
UvA
Router
Router
CA*net4
Router
ams
chi
SURFnet5
UBC Vancouver
Switch
GbE
GbE
GbE
Lambda
Switch
Lambda
Switch
Switch
Router
High bandwidth application
Scale 2-20-200
EVLEVL
OMNInet Chicago 2x10GE LambdaGrid Scale: 2ms
• A four-site network in Chicago -- a 10GE service trial• A test bed for all-optical switching and advanced middleware• Partners: SBC, Nortel, iCAIR at Northwestern, EVL, CANARIE, ANL
2x10GE
Northwestern U
OpticalSwitchingPlatform
Passport8600
ApplicationCluster
ApplicationCluster
OpticalSwitchingPlatform
Passport8600
2x10GE
StarLight
OPTera Metro5200
ApplicationCluster
OpticalSwitchingPlatform
Passport8600
2x10GE8x1GE
UIC
Carrier Hotel
OpticalSwitchingPlatform
Passport8600
2x10GE8x1GE
8x1GE
8x1GELoop back
UIUC/NCSA
Starlight(NU-Chicago)Argonne
UChicagoIIT
UIC
Illinois Century NetworkJames R. Thompson CtrCity HallState of IL Bldg
4 pair
12 pair
4 pair
2 pair 2 pair
4 pair
18 pair
4 10 pair
12 pair
2 pair
Level(3)111 N. Canal
McLeodUSA151/155 N. MichiganDoral Plaza
Qwest455 N. Cityfront
UC Gleacher450 N. Cityfront
Illinois’ I-WIRE
TransPAC/APAN
Euro-LinkSurfNET
Euro-LinkCERN
GÉANT
CA*net4
Abilene
Euro-Link Funded 10Gb LinkEuro-Link Co-Funded 10Gb LinkTransPAC Co-Funded 10Gb LinkPartner Matching 10Gb LinkGÉANT Matching 2.5Gb Links
Euro-Link/Trans-Light andTransPAC/Trans-Light SupplementFunded, Co-Funded and Matching Links
I-5/PLR
• A Production Network 1GigE and 10GigE exchange• An Experimental Network lambda exchange• A Research Network 1GigE and 10GigE MEMS-switched
exchange• Host to DTFnet, the TeraGrid’s 4x10Gb T640-based
Experimental Network, perhaps for future collaborations• A co-location space with 66 racks for networking and
computing and data-management equipment • An OIX with fiber and/or circuits from SBC/Ameritech,
Qwest, AT&T, Global Crossing, Looking Glass Networks, Level 3, RCN, Deutsche Telekom/T-Systems, I-WIRE
• A facility for links coming from NetherLight, CERN/DataTAG, CA*net4, and proposed from UK-Light and APAN forming Trans-Light
StarLight supports the OptIPuter as
Clusters East of I-5
• Each site (StarLight, UIC, NetherLight) has – Several clusters with dual processors and dual GigE
cards– Electronic switching and routing– Optical switching– Specialized Clusters
• Computing • Data Mining and Serving• Visualization
• Upgrade to Itanium Clusters in progress• Upgrade to 10GigE NICs next year
Lambdas East of I-5, 2003
• Illinois– 16 GigEs to UIC– 8 GigEs to NorthWestern/Evanston– GigEs to ANL, NCSA
• Canada– 8 GigEs Chicago to NYC– 8 GigEs Chicago to Seattle
• Europe– 16 GigEs Chicago to Amsterdam– 4 GigEs Chicago to CERN– 2 GigEs CERN to Amsterdam
Optical MEMS Switching
Why Optical Switching?• No need to look at every packet when transferring
a terabyte of information– 1% the cost of routing– 10% the cost of switching– 64x64 10Gb:
• $100,000 O-O-O switched• $1,000,000 O-E-O switched• $10,000,000 O-E-O Routed
• Spend the savings on links, computing and collaboration systems instead!
• Replaces patch panels; allows rapid reconfiguration of 1 and 10Gb experiments
UIC Procurement of 3D MEMS Switches
• Sent out bid request for (2) 64x64 3D MEMS Switches
• GlimmerGlass Networks and Calient switches were tested with a small cluster– Using both MMF (with adapters) and SMF NICs– Computer controlled– No problems encountered
• Calient won the bid for 2 switches; UIC is upgrading one switch to 128x128
• GlimmerGlass Networks will provide EVL a loaner 64x64 switch
Optical Switches at StarLight and NetherLight
N E T H E R L I G H T
A “groomer” is a box that accepts multiple circuits of varying types (e.g., 1GigE, 10GigE) and aggregates and/or disseminates over the 10Gbps transoceanic link. As the amount of transoceanic connectivity increases, we aim to “bandwidth match” the amount of data being sent and/or received by clusters across continents.
GigE = Gigabit Ethernet (Gbps connection type)
8-processor cluster
16-processor cluster
Switch/Router
8 GigE16 GigE
8 GigE16 GigE
Control plane
Data plane
“Groomer” at StarLight
8 GigE
2 GigE
128x128MEMS
Optical Switch
N-processor cluster
8 GigE
N GigE
“Groomer” at NetherLight
Control plane
Data plane
2 GigE
OC-192
(10Gbps)
N GigE
64x64MEMS
Optical Switch
Switch/Router
SL 650916-processor cluster
8-processor cluster
ONI DWDM
SL OPSW
NL 15454
16
8
2
6
6
2
8
OC-192
128x128 MEMSOptical Switch
OMNInet8
4
10Gb
10Gb
Layer2/3 Switch/Router
EVL 6509
16-processor cluster
16
16
EVL OPSW
64x64 MEMS
16
ONI DWDM
OMNInet
6
8
2
Layer2/3 Switch/Router 16
UIC/EVL LAN
StarLightStarLight LAN
NetherLight/UvA
NL 15454
NL OPSWNL 6509
Layer2/3 Switch/Router
6
2
8-processor cluster
8
8
Fiber Use in Pairs
GE ElectronicallySwitched
GE OpticallySwitched
64x64 MEMSOptical Switch
OptIPuterEVL/StarLight/NetherLight/UvA
Summer 2003 Plan
Hard Infostructure Problems
• Internet is not designed for single large-scale users—TCP is not usable for long fat applications
• Circuits are not scalable, but neither are router$• All intelligence has to be on the edge• Tuning compute, data, visualization, networking
using clusters to get order of magnitude improvement
• Security at 10Gb line speed
Thanks to…• StarLight planning, research, collaborations, and outreach
efforts are made possible, in major part, by funding from: – National Science Foundation (NSF) awards ANI-9980480, ANI-9730202,
EIA-9802090, EIA-9871058, ANI-0225642, and EIA-0115809– NSF Partnerships for Advanced Computational Infrastructure (PACI)
cooperative agreement ACI-9619019 to NCSA– State of Illinois I-WIRE Program, and major UIC cost sharing– Northwestern University for providing space, engineering and
management• NSF/CISE/ANIR and DoE/Argonne National Laboratory for StarLight and
I-WIRE network engineering and planning leadership• NSF/CISE/ACIR and NCSA/SDSC for DTF/TeraGrid/ETF opportunities• The OMNInet Initiative• UCAID/Abilene for Internet2 and ITN transit; IU for the GlobalNOC • Bill St. Arnaud of CANARIE, Kees Neggers of SURFnet, Olivier Martin of
CERN, Michael McRobbie of IU, and Harvey Newman of CalTech
OptIPuter: East of I-5
top related