icfa standing committee on interregional connectivity (scic) icfa standing committee on...
TRANSCRIPT
ICFA Standing Committee onICFA Standing Committee onInterregional Connectivity (SCIC)Interregional Connectivity (SCIC)
Harvey B. Newman Harvey B. Newman California Institute of TechnologyCalifornia Institute of Technology
CHEP2009, Prague, March 23 2009CHEP2009, Prague, March 23 2009
Global Networks for HEP in 2009
World Internet Connection Density 2008http://www.chrisharrison.net/projects/InternetMap/
The World At Nighthttp://www.bertc.com/subfour/truth/nightworld.htm
http://monalisa.caltech.edu:8080/Slides/Public/SCICReports2009Finalhttp://monalisa.caltech.edu:8080/Slides/Public/SCICReports2009Final
SCIC in 2009 http://cern.ch/icfa-scic
SCIC in 2009 http://cern.ch/icfa-scic
Three 2009 Reports: An Eventful YearThree 2009 Reports: An Eventful YearRapid Progress, Dawn of an Era; Deepening Digital DivideRapid Progress, Dawn of an Era; Deepening Digital Divide
Main ReportMain Report: “Networking for HEP”: “Networking for HEP” [HN, A. Mughal et al.][HN, A. Mughal et al.] Includes Updates on the Digital Divide, World Network Includes Updates on the Digital Divide, World Network
Status; Brief Updates on Monitoring, Advanced Status; Brief Updates on Monitoring, Advanced Technologies; Technologies; Focus on Digital Divide IssuesFocus on Digital Divide Issues
3939 Annexes: A World Network Overview Annexes: A World Network Overview Status and Plans of Nat’l & Regional Networks, HEP Labs, Status and Plans of Nat’l & Regional Networks, HEP Labs, & Optical Net Initiatives & Optical Net Initiatives [[35 Updated in 200935 Updated in 2009]]
Monitoring Working Group ReportMonitoring Working Group Report [R. Cottrell, U. Kalim] [R. Cottrell, U. Kalim] Also See: Also See: TERENA TERENA 2008 Compendium2008 Compendium ((www.terena.nlwww.terena.nl)): : R&E Networks in EuropeR&E Networks in Europe http://internetworldstats.comhttp://internetworldstats.com: Worldwide Internet Use: Worldwide Internet Use OECD Broadband PortalOECD Broadband Portal http://www.oecd.org/sti/ict/broadbandhttp://www.oecd.org/sti/ict/broadband
Broadband: the new Digital Divide ?Broadband: the new Digital Divide ? SCIC 2003 Digital Divide ReportSCIC 2003 Digital Divide Report [A. Santoro et al.][A. Santoro et al.]
North Am.
Revolutions in Networking Revolutions in Networking
Explosion of BW Use: Explosion of BW Use: Now ~4,000 PB/moNow ~4,000 PB/mo
Raw capacity still Raw capacity still mostly unusedmostly unused
Rise of broadbandRise of broadband Rise of Video + Mobile Rise of Video + Mobile
Traffic: ~20 ExabytesTraffic: ~20 ExabytesPer mo. (64%) by 2013Per mo. (64%) by 2013
Web 2.0: Billions Web 2.0: Billions of Web Pages, of Web Pages, embedded apps.embedded apps. Facebook, Twitter, Facebook, Twitter,
iPhone/Gphone, iPhone/Gphone, GEarth; SkypeGEarth; Skype
Beginnings of Web 3.0: Beginnings of Web 3.0: streaming content; streaming content; ubiquitous informationubiquitous information
11stst Revolution: “Long Dawn” of the Information Age Revolution: “Long Dawn” of the Information Age
1.6B1.6B Internet Users; 400+M with Broadband (12/31/08) Internet Users; 400+M with Broadband (12/31/08)http://internetworldstats.com
%0 10 20 30 40 50 60 70 80
73%
49%24%
17%22%
5.4%
60%
Asia
Latin Am.
Africa
Mid. East
Australasia/Oceana
World Av. 23%
Broadband: 80M Each in the US and ChinaBroadband: 80M Each in the US and China
World Penetration Rates (12/31/08)
Europe
SCIC Report 2009 - Main Trends Accelerate:SCIC Report 2009 - Main Trends Accelerate: Dark Fiber Nets, Dynamic Circuits, 40-100GDark Fiber Nets, Dynamic Circuits, 40-100G
SCIC Report 2009 - Main Trends Accelerate:SCIC Report 2009 - Main Trends Accelerate: Dark Fiber Nets, Dynamic Circuits, 40-100GDark Fiber Nets, Dynamic Circuits, 40-100G
Current generation of 10 Gbps network backbones and major Int’l Current generation of 10 Gbps network backbones and major Int’l links arrived in 2002-8 in US, Europe, Japan, Korea; Now links arrived in 2002-8 in US, Europe, Japan, Korea; Now China China
Bandwidth Growth: from 16X to Bandwidth Growth: from 16X to 10,000X in 7 Yrs. >> Moore’s Law10,000X in 7 Yrs. >> Moore’s Law Proliferation of 10G links across the Atlantic & Pacific since 2005Proliferation of 10G links across the Atlantic & Pacific since 2005
Bandwidth for LHC well above 100 Gbps, in aggregateBandwidth for LHC well above 100 Gbps, in aggregate Rapid Spread of “Dark Fiber” and DWDM: Emergence of Rapid Spread of “Dark Fiber” and DWDM: Emergence of Continental, Continental,
Nat’l, State & MetroNat’l, State & Metro N X 10G “Hybrid” Networks in Many NationsN X 10G “Hybrid” Networks in Many Nations Demand may still exceed capacity by ~2010, in some regionsDemand may still exceed capacity by ~2010, in some regions
Point-to-point “Light-paths” for HEP and “Data Intensive Science”Point-to-point “Light-paths” for HEP and “Data Intensive Science” Now Dynamic Circuits, and Managed Bandwidth ChannelsNow Dynamic Circuits, and Managed Bandwidth Channels
Technology continues to drive Performance Higher, Costs Lower Technology continues to drive Performance Higher, Costs Lower Commoditization of GE now 10 GE ports on serversCommoditization of GE now 10 GE ports on servers Multicore processors with Multi-Gbyte/sec interconnectsMulticore processors with Multi-Gbyte/sec interconnects Cheaper and faster storage ($ < 100/Tbyte); 100+ Mbyte/sec disksCheaper and faster storage ($ < 100/Tbyte); 100+ Mbyte/sec disks
TransitionTransition to 40G, 100G links to 40G, 100G links from 2011 (on land), ~2012 (sub-sea) from 2011 (on land), ~2012 (sub-sea) OutlookOutlook:: Continued growth in bandwidth deployment & useContinued growth in bandwidth deployment & use
Core NREN Capacity of EU/EFTA
Countries 2004-2008 TERENA Compendium 2008:
www.terena.org/activities/compendium/100G
10G
1G
0.1G
20G
atbe
bgcy
czdk
ee
frde
grhu
isie
it ltlu
mtnl
noplfi lv
sksi
esse
chuk
ptro
LHC Optical Private Network (The LHCOPN)
LHC Optical Private Network (The LHCOPN)
Edoardo Martelli (CERN)
Close collaboration with CERN & GEANT2 (11 10G Links)
Working with GEANT2 to achieve fiber diversity for European circuits to/from CERN
Inter-Tier1 links for resilience, redundancy:
BNL, FNAL: US LHCNetCNAF, SARA, GridKa, IN2P3:
Cross-border dark fiberRAL (UK): 2nd 10G in ProcessNDGF, ASGC, Taiwan, TRIUMF:
Reduced BW backupPIC (Spain): Still no backup
Evolution to N X 10 Gbps: Fermilab, BNL, US LHCNet; SURFnet, GARR-X
GÉANT2 Pan-European BackboneDark Fiber Core
Among 19 Countries: Austria Belgium Croatia Czech Republic Denmark Finland France Germany Hungary Ireland Italy Netherlands Norway Slovakia Slovenia Spain Sweden Switzerland United KingdomC. Stover (DANTE)
34 NRENs, ~30M Users; 50k km Leased Lines 12k km Dark Fiber; Point to Point ServicesGN3 Next Gen. Network Proposed 9/2008
Projected Start Q2 2009
SNLL
PNNL
LIGO
INEEL
LANL
MIT
ANLAMES
LLNL
GA
DOE-ALB
JGI
LBNL
SLACCHI
IARC
DenverLNOC
NERSC
BNL
FNAL
CH
I-SL
NYC
NYC
CERN
Amsterdam
By 2006
US LHCNet 10Gb/sUS LHCNet 20 Gb/sInternational (high speed)10 Gb/s SDN core10G/s IP coreMAN rings (≥ 10 G/s)Lab supplied linksOC12 / GigEthernetOC3 (155 Mb/s)45 Mb/s and less
ESnet4 IP Core (10 Gbps)
ESNet4 Science Data Net Core (n*10 Gbps)
Redundant “light-paths” to BNL and FNAL; dark fiber to FNAL
Connections to ESnet MANs in NYC & Chicago 10 Gbps peerings with Abilene (2) and GEANT2
US LHCNet (4x10Gbps)
US LHCNet + ESnet4 Today
SURFNet
In 2007
US-LHCNet Plan 2008-10: 40, 60, 80
GbpsNY-CHI-GVA-AMS
In 2008
A Global Partnership of R&E Networks and Advanced R&D Projects Supporting the LHC Program
14 to 16 10G Trans- Atlantic Links in 2009
Partnership with Internet2, NLR, ESnet in the US
Computing Model Progress CMS Internal Review of Software and Computing
CMS Data Transfer Volume (May – Aug. 2007)
10 PetaBytes Transferred Over 4 Mos. = 8.0 Gbps Avg. (15 Gbps Peak)
2008: 9-12 Gbps T1-T2 Flows
To UCSD, Nebraska, Caltech
15Log Plot of ESnet Monthly Accepted Traffic, January 1990 – December 2008
Remarkable Historical ESnet Traffic Trend Cont’d in 2008T
erab
ytes
/ m
on
th Oct 19931 TBy/mo.
Aug 1990100 MBy/mo.
Jul 199810 TBy/mo.
38 months
57 months
40 months
Nov 2001100 TBy/mo.
Apr 20061 PBy/mo.
53 months
July 201010 PBy/mo.
J. Metzger
ESnet Traffic Increases by10X Every 47 Months on Avg.
10 PBytes/mo. By ~July 2010Equal to 30 Gbps Continuous
Bandwidth Roadmap for Major Links (in Gbps): US LHCNet Example
Bandwidth Roadmap for Major Links (in Gbps): US LHCNet Example
Year Production Experimental Remarks
2001 0.155 0.622-2.5 SONET/SDH
2002 0.622 2.5 SONET/SDH DWDM; GigE
Integ.
2003 2.5 10-20 DWDM; 1 + 10 GigE
Integration
2005-6 10-20 2-10 X 10 Switch; Provisioning
2007-8 3-4 X 10 ~10 X 10; 100 Gbps
1st Gen. Grids
2009-10 6-8 X 10
~20X10, 5X40 or ~2 X 100
100 Gbps Switching
2011-13 ~12 X 10 to 0.25 Tbps
~10 X 100 2nd Gen Grids Terabit Networks
2014-6 ~Terabit ~MultiTbps ~Fill One Fiber
Paralleled by ESnet Roadmap for Data Intensive Sciences
We are progressing along this roadmap
Science Network Requirements Aggregation SummaryScience Drivers
Science Areas / Facilities
End2End Reliability
Near Term End2End
Band width
5 years End2End
Band width
Traffic Characteristics
Network Services
HEP:
LHC (CMS and Atlas)
99.95+%
(Less than 4 hours
per year)
73Gbps 225-265 Gbps
•Bulk data•Coupled analysis workflows
• Collaboration services
• Grid / PKI• Guaranteed
bandwidth• Monitoring / test
tools
NP:
CMS Heavy Ion
- 10Gbps (2009)
20 Gbps •Bulk data • Collaboration services
• Deadline scheduling• Grid / PKI
NP:
CEBF (JLAB)
- 10Gbps 10 Gbps •Bulk data • Collaboration services
• Grid / PKI
NP:
RHIC
Limited outage
duration to avoid analysis pipeline
stalls
6Gbps 20 Gbps •Bulk data • Collaboration services
• Grid / PKI• Guaranteed
bandwidth• Monitoring / test
tools
Immediate Requirements and Drivers for ESnet4
HENP: ~300 Gbps by 2013
ESnet Science Network Requirements Aggregation SummaryScience DriversScience Areas / Facilities
End2End Reliability
Near Term End2End
Band width
5 years End2End
Band width
Traffic Characteristics Network Services
BES:Chemistry and Combustion
- 5-10Gbps 30 Gbps • Bulk data• Real time data
streaming
• Data movement middleware
BES:Light Sources
- 15Gbps 40-60 Gbps
• Bulk data• Coupled simulation
and experiment
• Collaboration services• Data transfer facilities• Grid / PKI• Guaranteed BW
BES:Nanoscience Centers
- 3-5Gbps 30 Gbps •Bulk data•Real time data streaming•Remote control
• Collaboration services• Grid / PKI
Fusion ES:Int’l Collaboration
- 100Mbps 1 Gbps • Bulk data • Enhanced collaboration services
• Grid / PKI• Monitoring / test tools
Fusion ES:Instruments and Facilities
- 3Gbps 20 Gbps • Bulk data• Coupled simulation
and experiment• Remote control
• Enhanced collaboration service
• Grid / PKI
Fusion ES:Simulation
- 10Gbps 88 Gbps • Bulk data• Coupled simulation
and experiment• Remote control
• Easy movement of large checkpoint files
• Guaranteed bandwidth• Reliable data transfer
Fusion + BES + Bioinformations ~Equal to HEP
Cle
vela
nd
Primary DOE Labs
IP core hubsSDN hubs
Europe(GEANT)
Asia-Pacific
New YorkChicago
Washington DC
Atl
anta
Seattle
Albuquerque
Au
stra
lia
San Diego
LA
Production IP core (10Gbps)
SDN core (20-30-40Gbps)
MANs (20-60 Gbps) or backbone loops for site access
International connections
SunnyvaleDenver
South America(AMPATH)
South America(AMPATH)
Canada(CANARIE)
US LHCNet to CERN (60-80; 120-180 Gbps)
Canada(CANARIE)
Europe(GEANT)
Asi
a-Pac
ific
Asia Pacific
High Speed Cross connects with Ineternet2/Abilene
GLORIAD (Russia and
China)
Boise
Houston
Jacksonville
Possible hubs
Tulsa
Boston
Science Data Network Core
IP Core
Kansa
s City
Au
stra
lia
Fiber path is ~ 14,000 miles / 24,000 km
www.es.net/ESNET4ESnet4 50-60 Gbps by 2009-10; 500-600 Gbps 2011-12
100 Gbps waves “production-ready” within ~1.5-2 years (Infinera; Also CIENA)
Computing Model Progress CMS Internal Review of Software and Computing
Implementation: US LHCNet Plan Phase 8 Implementation: US LHCNet Plan Phase 8 (2014): Transition to Full Use of 100G(2014): Transition to Full Use of 100G
Using OTU-4 Using OTU-4 (100G) Links(100G) Links+ Next-Gen. + Next-Gen.
Optical MuxesOptical Muxes
4 X 100G4 X 100GTrans-Trans-
AtlanticAtlantic
FollowingFollowingan 8 Phasean 8 Phase
Plan Plan 2007-20142007-2014
US LHCNet Bandwidth Roadmap Matches LHC Storage Roadmap
US LHCNet Bandwidth Roadmap Matches LHC Storage Roadmap
20082009
20102011
20122013
Atlas Disk Ratio wrt 2008
AVG Disk Ratio wrt 2008
USLHCNet Bandwidth
CMS Disk Ratio wrt 2008
0
1
2
3
4
5
6
7
8
9
10Atlas Disk Ratio wrt 2008
AVG Disk Ratio wrt 2008
USLHCNet Bandwidth
CMS Disk Ratio wrt 2008
2008 2009 2010 2011 2012 2013
Atlas Disk Ratio wrt 2008 1 1.85 3.39 4.77 6.16 9.51
AVG Disk Ratio wrt 2008 1 1.55 2.63 3.62 4.62 6.60
USLHCNet BW Ratio [40G] 1 [60] 1.50 [80] 2.0 [120] 3.0 [180] 4.5 [260] 6.5
CMS Disk Ratio wrt 2008 1 1.25 1.87 2.47 3.08 3.69
40G in 200840G in 2008to to
260G in 2013:260G in 2013:
6.5X in 5 Yrs.6.5X in 5 Yrs.
SlowerSlower than than historical historical
trends:trends:[20-100X][20-100X]
Computing Model Progress CMS Internal Review of Software and Computing
Downward Price Evolution
on TA 10G Links Continues
(-31%/Yr 2005-8)
Important for the LHC
There Are Price “Bumps” Along
the Way
Prices in Other Regions are
Higher
Question
How will 40 + 100G Link Prices
Evolve in
2010-14 ?
24
Use of Dark Fiber in NREN Backbones 2005 – 2008Greater or Complete Reliance on Dark Fiber
TERENA Compendium 2008: www.terena.org/activities/compendium/
2005 2008
SURFNet and NetherLight: 8000 Km Dark FiberFlexible Photonic Infrastructure
SURFNet and NetherLight: 8000 Km Dark FiberFlexible Photonic Infrastructure
Cross Border Fibers to Germany: X-Win; On to NORDUnetCross Border Fibers to Germany: X-Win; On to NORDUnet
5 Photonic Subnets
λ Switching Services
to 10G
Fixed or Dynamic
Lightpaths: LCG, GN2, EXPRES DEISA
CineGrid
5 Photonic Subnets
λ Switching Services
to 10G
Fixed or Dynamic
Lightpaths: LCG, GN2, EXPRES DEISA
CineGrid
Erik-Jan Bos
LCG/EGEEPOLTIER2
Distributed Tier2 (Poznan, Warsaw, Cracow) Connects to Karlsruhe Tier1
Cross Border Dark Fiber Links
to Russia, Lithuania,
Belarus, Czech Republic,
and Slovakia
R. Lichwala2 X 10G Among 20 Major University Centers
Existing 1Q 20094Q 2009
POLAND: PIONIER 6000 km Dark Fiber Network in 2009
POLAND: PIONIER 6000 km Dark Fiber Network in 2009
Czech Republic: CESNET2 Reconfigurable Optical Backbone in 2009
Czech Republic: CESNET2 Reconfigurable Optical Backbone in 2009
2500+ km 2500+ km Dark FibersDark Fibers(since 1999)(since 1999)
N X 10 GbE N X 10 GbE Light-PathsLight-Paths
10 GbE CBDF10 GbE CBDFSlovakiaSlovakiaPolandPolandAustriaAustria
NetherlightNetherlightGEANT2GEANT2
Czech Tier2:Czech Tier2: 1 Gigabit Lightpaths to the Tier1s 1 Gigabit Lightpaths to the Tier1s at Fermilab, BNL, Karlsruhe and Taiwan;at Fermilab, BNL, Karlsruhe and Taiwan;
Similar scheme: Similar scheme: it, br, nl, de, ….it, br, nl, de, ….
H. Sverenyak
2002 - 2004: Dark Fiber Links to Austria, Czech Republic, Poland
2005-6: Complete 1 GbE links to all main sites 2006: 10 GbE Cross-Border Dark Fiber to Austria & Czech
Republic; 8 X 10G over 224 km with Nothing In-Line Deonstrated
2007-8: Transition Backbone to 10G Done; All CB Dark Fibers to 10G
~10,000x Increase Since
2002
WeisHorvath http://www.sanet.sk/en/index.shtm
SLOVAK Academic Network January 2009:
Now ~All 10 GbE Switched Ethernet
SANET to Schools 1GE to
500 Schools
In 54 Cities
By 2012
The Emergence of “Hybrid” Networks The Emergence of “Hybrid” Networks With Dynamic Circuits with BW GuaranteesWith Dynamic Circuits with BW Guarantees
W. Johnston, ESnet W. Johnston, ESnet On Circuit-Oriented Network ServicesOn Circuit-Oriented Network Services
Traffic Isolation; Security; Deadline Scheduling; Fairness; High UltilizationTraffic Isolation; Security; Deadline Scheduling; Fairness; High Ultilization
US LHCNet Configuration (2H 2009)Non-stop Operation; Circuit-oriented Services US LHCNet Configuration (2H 2009)
Non-stop Operation; Circuit-oriented Services
Robust fallback at layer 1 + next-generation hybrid optical network: Robust fallback at layer 1 + next-generation hybrid optical network: Dynamic Dynamic circuit-oriented network services with BW guarantees circuit-oriented network services with BW guarantees
Emerging Emerging StandardsStandards
VCAT, LCASVCAT, LCAS
CIENA Core CIENA Core DirectorsDirectors
Equipment Equipment and link and link
RedundancyRedundancy
Also Internet2 and SINET3
(Japan)
2007 Outbound Traffic13.3 PBytes
77% on Circuits: Peak 18 Gbps
6.7 Gbps Average
2007 Outbound Traffic13.3 PBytes
77% on Circuits: Peak 18 Gbps
6.7 Gbps Average
Large Scale Flows Mostly Handled by Dynamic
CircuitsUsing Software by
Fermilab and Caltech
Large Scale Flows Mostly Handled by Dynamic
CircuitsUsing Software by
Fermilab and Caltech
Traffic on CircuitsTraffic on Circuits
CMS data transfer between FNAL and UNL using Internet2's
DCN and LambdaStation Software (FNAL + Caltech) Cumulative transfer volume (top) and data rates (bottom)
CMS data transfer between FNAL and UNL using Internet2's DCN and LambdaStation Software (FNAL + Caltech)
Cumulative transfer volume (top) and data rates (bottom)
Entire 50 TByte Tier2 Buffer Filled
in ~1 Day
Entire 50 TByte Tier2 Buffer Filled
in ~1 Day
9 Gbps PeaksUsing Circuits
on ESNet (Green)I2 DCN (Red)
9 Gbps PeaksUsing Circuits
on ESNet (Green)I2 DCN (Red)
NetherLight 2009: 37 Lambdas, 201 GbpsNetherLight 2009: 37 Lambdas, 201 Gbps
Convergence of Many Partners on Common Circuit Concepts
Internet2, ESnet, GEANT2, US LHCNet; cz,nl, ru, es, tw, kr, hk, in, nordic; jp
Convergence of Many Partners on Common Circuit Concepts
Internet2, ESnet, GEANT2, US LHCNet; cz,nl, ru, es, tw, kr, hk, in, nordic; jp
~512 CPU Cores and100 10GE NICs
in 1 Rack of Servers64 10GE Switch Ports;
~100 TB Disk
Research Partners: FNAL, BNL, Florida, Michigan, Brazil, Korea;
ESnet, NLR, FLR, Internet2, ESNet, CWave, AWave, IRNC,
CANARIE, SURFNET
40
6050
30
1020
70
0
2010
30
40
In (
Gb
ps)
Tra
ffic
: O
ut
SC08
Using FDT and FDT/dCache Storage to Storage
Max. 114 Gbps; 110 Gbps Sustained; 71 Gbps Outbound
Caltech and CIENA: 191 Gbps Avg., 199.90 Gbps Max on An OTU4
(Standard 100G) Wave at SC2008
10 X 10G Waves at the Caltech HEP Booth
Used Fully, in Both Directions with Caltech’s
FDT (TCP-Based Java Application)
Previewing the US LHCNet Transition to
4 X 100G by ~20141.02 Petabytes Overnight
SC08
Plots Courtesy the PingER Project
Data from http://internetworldstats.com
Digital Divide: North Vs. South East Vs. West
Work on the Digital Dividefrom Several PerspectivesWork on the Digital Dividefrom Several Perspectives
Share Information: Share Information: Monitoring,Monitoring, Tracking Tracking BW ProgressBW Progress;; Dark Fiber Projects & Pricing Dark Fiber Projects & Pricing Track Planning (focus on LHC) and Leading Edge ProgressTrack Planning (focus on LHC) and Leading Edge Progress Model Cases: Poland, Slovakia, Czech Rep., Brazil, China …Model Cases: Poland, Slovakia, Czech Rep., Brazil, China … Encourage Access to Dark Fiber; Modern technology choicesEncourage Access to Dark Fiber; Modern technology choices
Raise Awareness: Locally, Regionally & GloballyRaise Awareness: Locally, Regionally & Globally Digital Divide Workshops Digital Divide Workshops [Rio, Gaegu, Cracow, Sinaia, Mexico City][Rio, Gaegu, Cracow, Sinaia, Mexico City] Diplomatic Events: WSIS, RSIS, Bilateral: US-South Asia, EU-Africa,…Diplomatic Events: WSIS, RSIS, Bilateral: US-South Asia, EU-Africa,…
Technical Help with Modernizing the Infrastructure: Technical Help with Modernizing the Infrastructure: Provide Tools for Effective Use: Provide Tools for Effective Use: Data Transport, Monitoring, Data Transport, Monitoring,
Remote Collaboration and e-LearningRemote Collaboration and e-Learning Design, Commissioning, Development Design, Commissioning, Development India “Knowledge Network”, Brazil’s Nat’l and Int’l NetworksIndia “Knowledge Network”, Brazil’s Nat’l and Int’l Networks
Encourage, and Work on Inter-Regional Projects Encourage, and Work on Inter-Regional Projects GLORIAD,GLORIAD, Russia-China-Korea-US-Europe Optical Ring Russia-China-Korea-US-Europe Optical Ring Latin America: Latin America: CHEPREO/WHRENCHEPREO/WHREN (US-Brazil); (US-Brazil); RedCLARARedCLARA Mediterranean: Mediterranean: EUMEDConnect; EUMEDConnect; Asia-Pacific: Asia-Pacific: TEIN3; TEIN3; Asia:Asia: ORIENT ORIENT
SCIC Monitoring WG PingER (Also IEPM-BW) SCIC Monitoring WG PingER (Also IEPM-BW)
Measurements from 1995 OnMeasurements from 1995 OnReports link reliability & qualityReports link reliability & quality
Countries monitoredCountries monitored Contain Contain 95.5%95.5% of world pop. of world pop. 99% of World’s Internet Users99% of World’s Internet Users
890890 remote nodes at remote nodes at 650650 sites in sites in 165 nations165 nations; ; 45 monitoring nodes;45 monitoring nodes; 95 Sites in 50 African countries95 Sites in 50 African countries
Strong Collaboration with ICTP Strong Collaboration with ICTP Trieste and NUST/SEECS (Pakistan)Trieste and NUST/SEECS (Pakistan)
Excellent, Vital WorkExcellent, Vital Work
Monitoring & Remote Nodes (1/09)
Countries: N. America (3), Latin America (21), Europe (33), Balkans (10), Africa (50), Middle East (13), Central Asia (9), South Asia (8), East Asia
(4), SE Asia (10), Russia (1), China (1) and Oceania (4)
R. Cottrell
Number of Hosts MonitoredBy Region: 1998 – End 2008 Number of Hosts MonitoredBy Region: 1998 – End 2008
R. Cottrell
AfricaAfrica
End
SCIC Monitoring WG - Throughput Improvements 1998-2008
Bandwidth of TCP < MSS/(RTT*Sqrt(Loss)) Matthis et al., Computer Communication Review 27(3), July 1997Matthis et al., Computer Communication Review 27(3), July 1997
30% annual improvement
~14X Per 10 yrs
Progress: but Digital Divide is Mostly Maintained
Behind Europe3-6 Yrs: Russia, Latin America, SE Asia 9-12 Yrs: South Asia, Central Asia16 Years: Africa
South & Central Asia, EspeciallyAfrica are Falling Farther Behind:
100X-1000X Worse by 2019 ?
R. Cottrell
http://www-iepm.slac.stanford.edu/
98
99
00
01
02
03
04
05
06
07
08
09
10k
100k
1M
10M
Th
rou
gh
pu
t (b
ps)
Three undersea cables cut: traffic greatly disturbed between Europe and Asia/Near
East zone France Telecom observed Dec. 19 that 3 major underwater cables were cut:
“Sea Me We 4” at 7:28am, “Sea Me We3” at 7:33am and FLAG at 8:06am. Causes of the cut, located in the Mediterranean between Sicily and Tunisia,
on sections linking Sicily to Egypt, remain unclear. Most traffic between Europe and Asia rerouted through the USA. Traffic from Europe to Near East & Asia interrupted to greater or lesser extent.First estimate of percentage of out-of-service capacity): Saudi Arabia: 55% out of service Djibouti: 71% out of service Egypt: 52% out of service UAE: 68% out of service India: 82% out of service Lebanon: 16% out of service Malaysia: 42% out of service Maldives: 100% out of service Pakistan: 51% out of service Qatar: 73% out of service Syria: 36% out of service Taiwan: 39% out of service Yemen: 38% out of service Zambia: 62% out of service
www.orange.com/en_EN/press/press_releases/att00006557/CP_cables_19dec08_EN.pdf
Computing Model Progress CMS Internal Review of Software and Computing
Cable Cuts on Dec. 19.
Partial Recovery by Dec. 23
PingER Measurementsof the Impact of the Cuts on Throughput
Vs. Time
1 3 5 7 9 11 1315 17 19
21 23 9 2727
Brazil: RNP2 Next-Generation “Ipê” Backbone
Brazil: RNP2 Next-Generation “Ipê” Backbone
M. Stanton
New vs. OldBandwidth 70 to 300X
2007-9 Buildout of dark fiber nets in 27 cities: Connecting 200 Univ. & Institutes at 1 Gbps
2009: Upgrade 2.5G Sao Paulo –Miami Link to 2 X 10G (RNP+ANSP+NSF)10G to Grid-UNESP
Tier1, Tier2s in Rio and Sao Paulo
Dark Fiber (622 Mbps) across the Amazon Porto Velho to Manaus
Extending the Ipê Fiber footprint to the East and South
SCIC: Networks for HEPMain Conclusions for 2009
SCIC: Networks for HEPMain Conclusions for 2009
The major R&E networks serving HEP have progressed rapidly over The major R&E networks serving HEP have progressed rapidly over the last few years; to N X10G in many casesthe last few years; to N X10G in many cases Worldwide transition to 40G & 100G links will occur in 2010 – 14Worldwide transition to 40G & 100G links will occur in 2010 – 14
Our BW usage has kept pace; may soon outstrip the planned capacityOur BW usage has kept pace; may soon outstrip the planned capacity Groups in HEP have developed state of the art methods to use these Groups in HEP have developed state of the art methods to use these
networks most effectivelynetworks most effectively Widespread deployment, end-to-end monitoring, training requiredWidespread deployment, end-to-end monitoring, training required
Adapting the LHC Computing Models to fully exploit networks Adapting the LHC Computing Models to fully exploit networks would have a profound positive impact on the LHC programwould have a profound positive impact on the LHC program
It is more urgent than ever that we act It is more urgent than ever that we act to Close the Digital Divideto Close the Digital Divide To make physicists in all regions full partners in the discoveries To make physicists in all regions full partners in the discoveries
We are learning to do this effectively, in some cases, in partnership We are learning to do this effectively, in some cases, in partnership with many agencies & HEP groups:with many agencies & HEP groups: Central Europe Central Europe andand Brazil; India Brazil; India and and PakistanPakistan
But we are leaving other countries and regions behind, for example:But we are leaving other countries and regions behind, for example: the Middle East, the Rest of Latin America; Africa the Middle East, the Rest of Latin America; Africa
A great deal of work remainsA great deal of work remains