nl-t1 expectations , findings , and innovation
DESCRIPTION
NL-T1 Expectations , findings , and innovation. Bas Kreukniet, Sr Network Specialist at SURFsara. Geneva Workshop 10 Februari 2014. Outline. Expectations from NL-T1 grid administrators Findings while connecting to the LHCONE Innovation : Ethernet OAM and NSI. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: NL-T1 Expectations , findings , and innovation](https://reader036.vdocument.in/reader036/viewer/2022062410/568165b0550346895dd8a258/html5/thumbnails/1.jpg)
Bas Kreukniet, Sr Network Specialist at SURFSARA
NL-T1 Expectations, findings, and innovation
Geneva Workshop 10 Februari 2014
![Page 2: NL-T1 Expectations , findings , and innovation](https://reader036.vdocument.in/reader036/viewer/2022062410/568165b0550346895dd8a258/html5/thumbnails/2.jpg)
Outline
1. Expectations from NL-T1 grid administrators2. Findings while connecting to the LHCONE3. Innovation: Ethernet OAM and NSI
![Page 3: NL-T1 Expectations , findings , and innovation](https://reader036.vdocument.in/reader036/viewer/2022062410/568165b0550346895dd8a258/html5/thumbnails/3.jpg)
Expectations from NL-T1 administrators
Advice from grid administrators NL-T1:
• Bulk data with simple applications (grid-FTP). “Keep it simple” • Network provisioning from application is
considered “complex”• Don’t rush to merge LHCOPN with LHCONE
![Page 4: NL-T1 Expectations , findings , and innovation](https://reader036.vdocument.in/reader036/viewer/2022062410/568165b0550346895dd8a258/html5/thumbnails/4.jpg)
NL-T1 connected to LHCONE since 20 Jan 2014
![Page 5: NL-T1 Expectations , findings , and innovation](https://reader036.vdocument.in/reader036/viewer/2022062410/568165b0550346895dd8a258/html5/thumbnails/5.jpg)
BGP routing starts at the connected site.
BGP routing for T1’s:
Not only a “NREN thing” – it already starts at your organisation!
Focus on some BGP topics for connecting sites to LHC networks
![Page 6: NL-T1 Expectations , findings , and innovation](https://reader036.vdocument.in/reader036/viewer/2022062410/568165b0550346895dd8a258/html5/thumbnails/6.jpg)
LHCOPN connectivity
![Page 7: NL-T1 Expectations , findings , and innovation](https://reader036.vdocument.in/reader036/viewer/2022062410/568165b0550346895dd8a258/html5/thumbnails/7.jpg)
LHCONE connectivity
128.142.0.0/16 *[BGP/170] 1w2d 04:07:47, MED 120, localpref 100 AS path: 20965 20641 513 I > to 62.40.126.161 via xe-4/0/1.2012
NL-T1AS1162
GÉANTAS20965
LHCONE-RS CERN
AS20641
CERNAS513
LHCONE.inet.0: 133 destinations, 133 routes (133 active, 0 holddown, 0 hidden)+ = Active Route, - = Last Active, * = Both
![Page 8: NL-T1 Expectations , findings , and innovation](https://reader036.vdocument.in/reader036/viewer/2022062410/568165b0550346895dd8a258/html5/thumbnails/8.jpg)
Preferred route: LHCOPN, LHCONE or Internet
• primary: direct T1–T1 over LHCOPN• secondary: T1–T1 over LHCOPN via another T1 • tertiary: LHCONE• quaternary: Internet
![Page 9: NL-T1 Expectations , findings , and innovation](https://reader036.vdocument.in/reader036/viewer/2022062410/568165b0550346895dd8a258/html5/thumbnails/9.jpg)
Preferred route: route (a)symmetry
If everyone makes this choice, all connections are symmetric.But: sites may have different preferences:
10 Gb/s 10 Gb/s
1 Gb/s 100 Gb/ssite 1 site 2
ISP-B
ISP-A
![Page 10: NL-T1 Expectations , findings , and innovation](https://reader036.vdocument.in/reader036/viewer/2022062410/568165b0550346895dd8a258/html5/thumbnails/10.jpg)
Tie-breaker between LHCOPN and LHCONE
CNAF routers at NL-T1
LHCOPN131.154.128.0/17 *[BGP/170] 14:58:18, localpref 100 AS path: 34878 137 I
LHCONE131.154.128.0/17 *[BGP/170] 1w2d 04:23:40, MED 120, localpref 100 AS path: 20965 137 I
![Page 11: NL-T1 Expectations , findings , and innovation](https://reader036.vdocument.in/reader036/viewer/2022062410/568165b0550346895dd8a258/html5/thumbnails/11.jpg)
LHCOPN / LHCONE route preference
BNL routes at NL-T1
LHCOPN130.199.185.0/24 *[BGP/170] 2w5d 09:35:43, MED 10, localpref 100 AS path: 513 43 I > to 192.16.166.73 via xe-1/1/0.0 [BGP/170] 3d 01:15:22, MED 51, localpref 100 AS path: 39590 513 43 I > to 109.105.124.17 via xe-2/1/0.0
LHCONE130.199.185.0/24 *[BGP/170] 1w2d 04:27:43, MED 120, localpref 100 AS path: 20965 293 43 I > to 62.40.126.161 via xe-4/0/1.2012
![Page 12: NL-T1 Expectations , findings , and innovation](https://reader036.vdocument.in/reader036/viewer/2022062410/568165b0550346895dd8a258/html5/thumbnails/12.jpg)
Problems and concerns regarding BGP
• A site advertising his routes has no control who to send a route to. At best they can give hints with BGP communities.
• The site receiving a route decides which route to accept and how to accept.
![Page 13: NL-T1 Expectations , findings , and innovation](https://reader036.vdocument.in/reader036/viewer/2022062410/568165b0550346895dd8a258/html5/thumbnails/13.jpg)
BGP in LHC networks
Only As is not enough. More info needed. community for origin of a route T1/T2?Or even better: -site connected to LHCOPN-site connected to LHCONE- Site connected to both:Type A “prefer LHCONE for this route”Type B “prefer LHCOPN for this route”- Specials: dedicated link between two (T1) sites. (“VPN”or “private link”)
![Page 14: NL-T1 Expectations , findings , and innovation](https://reader036.vdocument.in/reader036/viewer/2022062410/568165b0550346895dd8a258/html5/thumbnails/14.jpg)
Route Preference Solutions
• Idea: Tag routes with two types of communities: • One for origin or source• One for destinations• See also: BGP hinting by Martin Sweeny (Indiana U)
• BGP Always-compare-MED always on. We sometimes add metrics on incoming routes.
![Page 15: NL-T1 Expectations , findings , and innovation](https://reader036.vdocument.in/reader036/viewer/2022062410/568165b0550346895dd8a258/html5/thumbnails/15.jpg)
Operational Issues
• We received routes over LHC from a site,but traffic we send is blackholed by that site
• The site was still reachable over the Internet.• This happened to us twice recently: on LHCOPN (accidental route
redistribution) and LHCONE (incoming IP filter).• Configuration errors will be made (we’re also just human)
• LHCOPN Link NL-T1 – TRIUMF link is still unstable• 31 outages last 4 months
Monitoring remains important!
![Page 16: NL-T1 Expectations , findings , and innovation](https://reader036.vdocument.in/reader036/viewer/2022062410/568165b0550346895dd8a258/html5/thumbnails/16.jpg)
LHCONE and LHCOPN layer 2 monitoring
Ethernet OAM monitoring or Layer 2 monitoring
NIKHEF, TRIUMF to participate as Measurement PointsLooking for T1’s and T2 to participate
Advantages:• Layer2 keep-alive, ping and traceroute• Interdomain, intervendor solution• L2 devices can be made visible• unidirectional fibercuts can be signalled
![Page 17: NL-T1 Expectations , findings , and innovation](https://reader036.vdocument.in/reader036/viewer/2022062410/568165b0550346895dd8a258/html5/thumbnails/17.jpg)
LHCONE Innovation
NL-T1 likes to partcipate in NSI for LHCONE
NSI experiences so far:• Use-case: Life Science Grid (LSG) in Holland
makes use of NSI since autumn 2013. • Freek contributed to standard• Sander wrote NSI client and implemented it for
“Cloud Bypassing” in the Life Science Grid.
![Page 18: NL-T1 Expectations , findings , and innovation](https://reader036.vdocument.in/reader036/viewer/2022062410/568165b0550346895dd8a258/html5/thumbnails/18.jpg)
Cloud Bypassing on Life Science Grid (LSG)
Compute clusters at ±10 locations in the Netherlands
![Page 19: NL-T1 Expectations , findings , and innovation](https://reader036.vdocument.in/reader036/viewer/2022062410/568165b0550346895dd8a258/html5/thumbnails/19.jpg)
Cloud Bypassing on Life Science Grid (LSG)
Offloading campus networks (some only have 1 Gb/s Internet)
![Page 20: NL-T1 Expectations , findings , and innovation](https://reader036.vdocument.in/reader036/viewer/2022062410/568165b0550346895dd8a258/html5/thumbnails/20.jpg)
LHCONE Innovation: Offloading is cheaper
Internet full routing (incl backup): € 8k – €10k per month for 10 Gb/sLHCONE or dynamic lightpath: € 2k – 3k per month for 10 Gb/s
![Page 21: NL-T1 Expectations , findings , and innovation](https://reader036.vdocument.in/reader036/viewer/2022062410/568165b0550346895dd8a258/html5/thumbnails/21.jpg)
Questions?
Erik Ruiter
Bas Kreukniet
Diederik Vandevenne
Sander Boele
Farhad Davani
Freek Dijkstra