sc16 architecture - supercomputing · pdf filemajor changes in 2016 • otn used to...
TRANSCRIPT
SC16Architecture
NDISworkshopNov13th,2016
SC16NetworkDiagram
2 - 11/12/16
SLC POPCenturyLink
POP
SLC Level3 POP
CenturyLink
XT500
CenturyLink
XT500
CenturyLink
XT500
CenturyLink
XT500
DDC
SLC POP
SC01
DDC
FSP3000C
FSP3000C
SC24
-SLC
SC25
-SLC
SC26
-SLC
SC27
-SLC
SC23
-SLC
SC32
-SLC
SC33
-SLC
p1 p2
Line 1
Line 2
Line 1
Line 2
2/11/1
SC29
-Chi
cago
Commodity Internet Service
GROUP
SC38
-Sun
nyva
leSC
39-S
unny
vale
SC40
-Sun
nyva
le
SC31
-SLC
SC01
-NYC
SC02
-NYC
SC03
-NYC
Commodity Internet Service
CenturyLink
XT500
9/3 9/4 9/5 7/2 7/37/1 7/4 5/1 5/27/5 5/3 5/5 8/15/4 8/2 8/3 8/5 6/18/4
6/59/2
9/1 5p1
3p5 4p4 5p2 1p3 4p15p4 4p5 2p35p3 1p2 3p2 1p43p4 4p2 4p3 1p5 2p15p1
1p1
3p3
2p5
2p5
5p5 2p2
6/2 6/3 6/4
SC37
-Los
Ang
eles
SC36
-Los
Ang
eles
SC35
-Sun
nyva
leSC
34-S
unny
vale
SC22
-Sun
nyva
leSC
21-L
os A
ngel
esSC
20-P
ased
ena
SC19
-Los
Ang
eles
SC16
-Sea
ttle
SC14
-Chi
cago
SC13
-Chi
cago
SC12
-Chi
cago
SC09
-Chi
cago
SC08
-Chi
cago
SC07
-Was
hing
ton,
DC
SC06
-Was
hing
ton,
DC
SC05
-Was
hing
ton,
DC
SC04
-NYC
SC10
-Chi
cago
SC11
-Chi
cago
SC15
-Sea
ttle
SC28
-Mia
mi
xp1
Waveserver
1
SC41
-SLC
Waveserver1 6500-32
1/33/1
1/31/1 1/34/1
6500-141/1/1
1/2/1
2150
2p4
1/3/1
SC30
-St.
Geo
rge,
UT
1000-TX
10 Gigabit Ethernet
40 Gigabit Ethernet
100 Gigabit Ethernet
OTN / DWDM
1 Gigabit Ethernet
Wi-Fi Access Point
Network Architecture V1.19
Matt SmithMarc LyonnaisKathryn Mace
November 9, 2016
bwctl10g-demo
ferst
bwctl10g-external
techwood
bwctl3
140.221.249.133
owamp
140.221.249.141
2x10
0GE
Exhi
bito
rC
onne
ctio
ns
10G
E Ex
hibi
tor
Con
nect
ions
1GE
Exhi
bito
rC
onne
ctio
ns
2x10
0GE
Exhi
bito
rC
onne
ctio
ns
100G
E Ex
hibi
tor
Con
nect
ions
100G
E Ex
hibi
tor
Con
nect
ions
10G
E Ex
hibi
tor
Con
nect
ions
1GE
Exhi
bito
rC
onne
ctio
ns
10G
E Ex
hibi
tor
Con
nect
ions
1GE
Exhi
bito
rC
onne
ctio
ns
10G
E Ex
hibi
tor
Con
nect
ions
1GE
Exhi
bito
rC
onne
ctio
ns
10G
E Ex
hibi
tor
Con
nect
ions
1GE
Exhi
bito
rC
onne
ctio
ns
10G
E Ex
hibi
tor
Con
nect
ions
1GE
Exhi
bito
rC
onne
ctio
ns
10G
E Ex
hibi
tor
Con
nect
ions
1GE
Exhi
bito
rC
onne
ctio
ns10
GE
Exhi
bito
r C
onne
ctio
ns1G
E Ex
hibi
tor
Con
nect
ions
10G
E Ex
hibi
tor
Con
nect
ions
1GE
Exhi
bito
rC
onne
ctio
ns
10G
E Ex
hibi
tor
Con
nect
ions
1GE
Exhi
bito
rC
onne
ctio
ns
10G
E Ex
hibi
tor
Con
nect
ions
1GE
Exhi
bito
rC
onne
ctio
ns
10G
E Ex
hibi
tor
Con
nect
ions
1GE
Exhi
bito
rC
onne
ctio
ns
2x10
0GE
Exhi
bito
rC
onne
ctio
ns
100G
E Ex
hibi
tor
Con
nect
ions
noc3001-test1140.221.241.183
core-rtr1
QFX10K81/0/13
agg-rtr1
MLXe-8L0:140.221.251.240/32
2001:0468:1f07:ffff:0240/128
3/0/13
L0:140.221.251.243/322001:0468:1f07:ffff:0243/128
L0:140.221.251.243/322001:0468:1f07:ffff:0243/128
L0:140.221.251.242/322001:0468:1f07:ffff:0242/128
Corsa
Corsa Scheduler
DP2400Corsa-3961-1
XGS2
7100 Nano coriant-7100-1-noc
mgmt-dist-sw
140.221.241.144
agg-rtr2
ASR9010L0:140.221.251.239/32
2001:0468:1f07:ffff:0239/128
1/0/3 1/0/5
core-rtr2
7504RL0:140.221.251.241/32
2001:0468:1f07:ffff:0241/128
noc-sw27050SX-64
OpenFlow controlled
noc-sw1ICX7750 OpenFlow controlled
1830-PSS-16 1830-1-noc
corsa-noc-1DP2400
3/0/1 1/0/1 1/0/2 1/0/3 1/0/4 1/0/6 1/0/8 3/0/0 3/0/3 3/0/4 3/0/61/0/0 3/0/8 3/7/1 3/10/1 5/8/1 5/9/1 5/1/15/7/1 3/1/1
3/0/5
1/0/7
3/0/7
3/2/1
5/2/1
3/3/1
5/3/1
1/0/17 3/0/13 1/0/19 1/0/20 3/0/19 1/0/15
1/0/16
1/0/14 3/0/121/0/183/0/18 3/0/16 5/15/1 3/13/1 3/14/1 3/15/1 5/5/1 3/6/15/13/1 5/6/13/5/1
4/1/16/1/1
3/0/0 5/0/0 4/0/0 3/1/0 5/1/0 3/1/1 5/1/1 3/0/1 5/0/1 4/0/1 6/0/16/0/05/1 7/1 6/1 5/3 7/3 5/4 7/4 5/2 7/2 6/2 8/28/1
1/1 3/1
6/48/4
12
1-3-c11-3-c2
1-33
SPT-N4U
OTDR
2-332-37
1/0/11
3/0/11
3/4/1
5/4/1
1/0/5
5/14/1
1/2 3/2 1/0/2 1/0/4
CorsaDP2400
Corsa-2520-1
1
DCI
Cloud Xpressinfn-cx100-b2537
DCI
Cloud Xpressinfn-cx100-b2611 NCS2006
cisco-otn-2-d2520
3/36/1
4/36/1
5/0/05/0/1
500G
1830-PSS-16
nokia-1830-2-dn3961
dwdm
dwdm
400G
8700-4
ciena-8700-noc
NCS2006cisco-otn-3-d3931
400G
NCS2006 cisco-otn-1-noc
dwdm
400G
coriant-7100-3-dn3976
7100 Nano
400G
coriant-7100-2-dn1423
7100 Nano
500G
1830-PSS-8
nokia-1830-3-dn533
dwdm
ciena-8700-dn25208700-4
400G
6500-2
ciena-2150-dn2520
ciena-8700-dn3931 8700-4
2150
2150
400G
2/0/12/0/0
SC01
-NYC
SC24
-SLC
SC13
-Chi
cago
SC15
-Sea
ttle
SC21
-Los
Ang
eles
SC34
-Sun
nyva
leSC
36-L
os A
ngel
esSC
03-N
YC
SC26
-SLC
SC16
-Sea
ttle
SC22
-Sun
nyva
leSC
35-S
unny
vale
SC37
-Los
Ang
eles
SC25
-SLC
SC02
-NYC
SC14
-Chi
cago
3/8/1 3/9/1
SC27
-SLC
SC12
-Chi
cago
SC04
-NYC
SC19
-Los
Ang
eles
Apollo
eci-dci-2-d2520
400G
SC08
-Chi
cago
DCI
g30-1-noc
3/0/14
dwdm
1/0/13dwdmdwdm
dwdmdwdm
dwdm
dwdm
dwdm
dwdm
dwdm
ciena-2150-dn3931
dwdm
dwdmdwdmdwdm
dwdm
dwdm
dwdm
dwdm
dwdm
dwdm
1-1 1-22-1
dwdm
34
5
6 7 85
12
12
M-4 1-B 2-B 2-T 3-T 4-T 1-T 1-M 2-M
DCI
Waveserver dwdm
1 2 3 4 5
3/0/15
222-15 2-25
p1 p2
2x10
0GE
Exhi
bito
rC
onne
ctio
ns
1-3-c1 1-3-c2 1-3-c{1-2} 2-1
1-33
p1
1 2 3 4
4x10
0GE
Exhi
bito
rC
onne
ctio
ns
12
1-5-c21-5-c1dwdm
1/0/29
3/0/2
SC29
-Chi
cago
1/0/211/0/15 1/0/12
1-37
dwdm
1-37 1-33
Apolloeci-dci-1-noc
4
2
dwdm
3
dwdm
3/17/12/12/23/0/29
1/1/1 1/1/2
3/0/10
400G
bwctl100g
5/22/1
mgmt2611-sw
140.221.241.137
mgmt2537-sw
140.221.241.136
800G
Caltech Booth 2537
Starlight Booth 2611
12x1
00G
E A
pplia
nces
DCI
ciena-wserver-b2611
10x1
00G
E A
pplia
nces
DCI
coriant-g30-2-b2437
400G
1.2T
b
dwdmdwdm
SCinet DNOC Booth 1423
SCinet DNOC Booth 533
SCinet DNOC Booth 2520
SCinet DNOC Booth 3931
SCinet DNOC Booth 3976
SCinet DNOC Booth 3961
dnoc1423-sw27050SX-64
OpenFlow controlled
dnoc1423-sw1ICX7750
OpenFlow controlled
dnoc533-sw27050SX-64
OpenFlow controlled
dnoc533-sw1ICX7750
OpenFlow controlled
dnoc2520-sw27050SX-64
OpenFlow controlled
dnoc2520-sw1ICX7750
OpenFlow controlled
dnoc3931-sw27050SX-64
OpenFlow controlled
dnoc3931-sw1ICX7750
OpenFlow controlled
dnoc3961-sw2
7050SX-64 OpenFlow controlled
dnoc3961-sw1ICX7750
OpenFlow controlled
dnoc3976-sw27050SX-64
OpenFlow controlled
dnoc3976-sw1ICX7750
OpenFlow controlled
mgmt533-sw
140.221.241.147
mgmt3961-sw
140.221.241.140
mgmt1423-sw
140.221.241.143
mgmt2520-sw
140.221.241.142
mgmt3931-sw
140.221.241.141
mgmt3976-sw
140.221.241.138
rack-sw 0/1/02/1/0
3/1/0 4/1/0
wlc2
wlc1
140.221.174.3
140.221.174.2
140.221.174.4
prime
corsa schedulerodl scheduler
kioskmeas-displays
ndtscdl
rancidrepo
scinetsyslog
its2140.221.249.180
its-nas
140.221.249.181
its0140.221.249.178ns2
ns0dhcp
140.221.251.250
ns1
its1140.221.249.179
conf-rtr
MX-960
General Conference Support
4/1/0
4/1/99/0/7
2/1/3
9/1/3
9/1/0
11/1/7
9/0/1
9/0/0
9/1/1
2/1/1
0/0/7
2/0/2
11/0/6
0/0/6
2/0/1
0/1/6
11/1/6
9/2/1
9/2/2
2/1/0
L0:140.221.251.243/322001:0468:1f07:ffff:0243/128
swidf-b55
140.221.241.22
2/0/3
9/0/3
2/1/0
swidf-mdf2
140.221.241.13
0/1/7
swidf-a48
140.221.241.14
9/1/2
swidf-c315
140.221.241.19
0/3/1
9/0/2
4/2/0
7/2/0
2/1/2
swidf-a51
140.221.241.4
swidf-a53
140.221.241.11
swidf-a107
140.221.241.16
swidf-a115
140.221.241.18
swidf-a124-f
140.221.241.40
swidf-b215
140.221.241.21swidf-b51
140.221.241.20
swidf-a124
140.221.241.15
swidf-mdf1
140.221.241.10
swidf-b54
140.221.2418
swidf-b53
140.221.241.9
swidf-a133
140.221.241.7
swidf-a56
140.221.241.3
swidf-a49
140.221.241.5
swidf-a52
140.221.241.2
7/1/0
2/3/3
9/3/3
swnoc-151g-1
140.221.241.12
swidf-a107-f
140.221.241.41
0/2/0
140.221.251.254
140.221.251.130
11/2/0
swidf-a46
140.221.241.6
swnoc-151g-6
140.221.241.27
swnoc-151g-5
140.221.241.26
swnoc-151g-4
140.221.241.25
swnoc-151g-3
140.221.241.24
swnoc-151g-2
140.221.241.23
11/0/7
2/2/0swidf-mdf-f
140.221.241.36
sw255f
140.221.241.33
sw251a
140.221.241.35
sw251d
140.221.241.34
7/1/104/1/10
9/2/0
swidf-a56-f
140.221.241.39
2/2/2swidf-a52-f
140.221.241.38
2/2/1
swidf-a48-f
140.221.241.37
swscc-mgmt
140.221.241.42
swscc-teams
140.221.241.43
7/2/1
4/2/1
swnoc-a-2
140.221.241.28
swnoc-a-3
140.221.241.29
swnoc-a-12
140.221.241.59
swnoc-a-13
140.221.241.60
swnoc-a-1
140.221.241.17
7/1/9 7/1/94/1/9em0 em0 em0
11/3/1
swnoc-a-11
140.221.241.58
swnoc-a-10
140.221.241.57
swnoc-a-7
140.221.241.54
swnoc-a-5
140.221.241.31
swnoc-a-4
140.221.241.30
swnoc-a-6
140.221.241.32
swnoc-a-8
140.221.241.55swnoc-a-9
140.221.241.56
2/2/39/2/3
5/0/6:0
2/0/0
hc2
48-P
assiv
e Fi
ber N
etwo
rk T
aps
140.221.240.17
conf-fw
SRX1500
0/0/18
0/0/19
hc2
48-P
assiv
e Fi
ber N
etwo
rk T
aps
Tap aggregation fabric
sec-tap-2
10 ta
pped
1G
inte
rface
s13
tapp
ed 1
0G in
terfa
ces
2x10
0GE
to T
A100
TA100-taps
4*10
0G in
terfa
ces
140.221.240.10
Tap
aggr
egat
ion
fabr
ic
140.221.240.10
L7-analytics
PA-7080140.221.242.10
scc-fw
SRX1500
sensor-bro {1-6}
R-Scope140.221.242.{12-17}
sec-tap-3
sec-tap-4
2x100GE
to TA100
TA100-taps
sec-metaflows-ctl
140.221.242.4
0/0/180/0/19
1Tb tap-agg-100
GigaVUE-TA100140.221.242.20
tap-agg-mdf
GigaVUE-HC2140.221.242.8
sec-metaflows-sandbox
140.221.242.3
sec-metaflows-sensor
140.221.242.2
info-agg1info-agg2
140.221.242.{6,9}
sensor-splunk {1-4}
140.221.242.{25-28}
honeynet
140.221.242.{21-24}
analytics
Ensign140.221.242.5
tap-agg-noc
GigaVUE-HD8140.221.242.18
alpha-flow
R-Scope140.221.242.11
sec-tap-1
hc2
48-P
assiv
e Fi
ber N
etwo
rk T
aps
noc3001-test2140.221.241.184
140.221.241.131
140.221.241.132
5/17/1
100G
100G
200G
200G
200G
500G
400G
400G
p2
1-37
Majorchangesin2016• OTNusedtotransport100GE
– Fiberinfrastructure• FECwillincreasethereliabilityofthefiberinterconnectionfromtheNOCtothebooth
– MetroRing• 400Grings
– DataCenterInterconnect(DCI)• Terabituplinks
• SecurityCoreFlow– EnrichingofSecurityeventswithotherinformationsources(Ralph)
• SDNtothebooth– Evolvingfromlastyear’sOpenFlow experimentationtoproduction
• EvolutionwithCorsa (seenextslide)
3 - 11/12/16
TheSCinet Corsa Experiment
• Thegoalistoautomateschedulingofprioritynetworkaccess
• Provideadatabasedrivenautomationplatformtoprovidetheschedulingdetails
• Executeschedulingallowingallexperimentsaccesstothenetworkatalltimes,withprioritytothescheduledexperiment.
4 - 11/12/16