the blu project - itnog · quagga (bgp) the blu project bluos eolo customizations •...
TRANSCRIPT
4
About EOLO
• Atthebeginning(2006):• Broadbandforruralareas(innorthernItaly)
• Today:• Coverage:>90%northernItaly,>50%centralItaly• 230Kcustomers(4%marketshare)• From30Mbpsto1Gbpsandwhite-labelwholesale
• Eachmonth:• 13Knewcustomers
5
FWA– AnatomyofaBaseTransceiverStation
Tothetop• Backhaul(PTPlinks)• Clientaccess(PMPlinks)
Totheground• Routers,switches• Energystations
6
About EOLO
• FixedWirelessAccess(FWA)ISP
• CoreinMilan
• Backhaulconnectionsbymeansofwireless linksestablishedbetween BaseTransceiverStation
7
(Avery)Distinguishing feature
Opportunisticapproach• NewlinkestablishedwhenevertwoBTSs“seeeachothers”
11
TheBLUprojectRouters
• TileGX architecture• 72coreCPU• 16GBDDR3RAM• 128GBSSD• Lowpower:90Wavg,150Wmax• 24x1Gports,2x10Gports
• IEEE1613compliant• Electromagneticcompatibility• Error-freeoperationin-15C/+75Cenvironment• EMIfieldsofupto180V/m
12
Basedon6windgateby6WINDS.A.• Linux3.10.61
• OpenVSwitch
• 6WINDuserspace networkstack• BasedonDPDK• Associatecorestoports(differentconfigurationbasedontherouterstasks/position)
• Userspace dataplane – HIGHLYEFFICIENT
• PPPdaemon
• DHCPrelay
• Quagga(BGP)
TheBLUprojectBLUos
EOLOcustomizations• OpenVSwitch mac-learningimproved
• RFC3107insideQuaggaBGP
• Automaticset-up/rescueprocedure• Self-managementstrategies• Corruptedconfigurations• Hosttemporarilydisconnected
• Monitoringdaemons• LLDPneighboringdetails• Bandwidth(radiolinkssufferbadweather,interferences,etc.)
• Tx/Rxtrafficoneachinterface(bothBHandAccess)
13
• Avoid(or,atleast,limit)humanerror
• FullmanagementofourBLUrouters• Parallelexecutionofcommands• Schedulerfornightupdater• Success/failure/rollback
• CollectionandanalysisofdatafromtheBLUrouters• Topologyupdates• Statistics,logsanderrormessages
• Statisticsvisualization
TheBLUprojectBLU-GW
20
TheBLUprojectPhase1- Challenges
• Backhaul:• Wirelesslinks• PVST+L2
• Veryhighnumberofmac-address!
• PPP?DHCP?
• Trafficbalancing?Overloadedlinks?
21
TheBLUprojectPhase1- Solution
• Backhaul:• Wirelesslinks• PVST+L2
• Veryhighnumberofmac-address!• PPPserver/DHCPrelayontheBLU• EachBLUadvertisestothecoreitsPPP/DHCPIPpools
• Trafficbalancing?Overloadedlinks?• MultipleBHvlans (costchangesdeterminedifferentpaths)
• Costchangesperformedmanually!
• OVSstillsuffersthehighnumber(>1KperBLU)ofmac-addresses!• Flows-handlingperformancedecreases– trafficgetslost
22
TheBLUprojectBLUos – HackingOVS
in_port(2),eth(src=00:00:00:00:00:01,dst=00:00:00:00:05:03),…,actions:9in_port(9),eth(src=00:00:00:00:05:03,dst=00:00:00:00:00:01),…,actions:2
00:00:00:00:00:01
00:00:00:00:00:02
00:00:00:00:05:03
eth2 eth9
23
TheBLUprojectBLUos – HackingOVS
in_port(2),eth(src=00:00:00:00:00:01,dst=00:00:00:00:05:03),…,actions:9in_port(2),eth(src=00:00:00:00:01:01,dst=00:00:00:00:05:03),…,actions:9in_port(2),eth(src=00:00:00:00:02:01,dst=00:00:00:00:05:03),…,actions:9in_port(2),eth(src=00:00:00:00:03:01,dst=00:00:00:00:05:03),…,actions:9
in_port(9),eth(src=00:00:00:00:05:03,dst=00:00:00:00:00:01),…,actions:2in_port(9),eth(src=00:00:00:00:05:03,dst=00:00:00:00:01:01),…,actions:2in_port(9),eth(src=00:00:00:00:05:03,dst=00:00:00:00:02:01),…,actions:2
in_port(9),eth(src=00:00:00:00:05:03,dst=ff:ff:ff:ff:ff:ff),…,actions:2,3,4,5in_port(9),eth(src=00:00:00:00:01:03,dst=ff:ff:ff:ff:ff:ff),…,actions:2,3,4,5
00:00:00:00:00:01
00:00:00:00:00:02
00:00:00:00:05:03
00:00:00:00:01:01
00:00:00:00:03:01
00:00:00:00:02:01
00:00:00:00:04:03
00:00:00:00:06:03
eth2 eth9
24
TheBLUprojectBLUos – HackingOVS
in_port(2),eth(src=*,dst=00:00:00:00:05:03),…,actions:9in_port(2),eth(src=00:00:00:00:01:01,dst=00:00:00:00:05:03),…,actions:9in_port(2),eth(src=00:00:00:00:02:01,dst=00:00:00:00:05:03),…,actions:9in_port(2),eth(src=00:00:00:00:03:01,dst=00:00:00:00:05:03),…,actions:9
in_port(9),eth(src=*,dst=00:00:00:00:00:01),…,actions:2in_port(9),eth(src=00:00:00:00:05:03,dst=00:00:00:00:01:01),…,actions:2in_port(9),eth(src=00:00:00:00:05:03,dst=00:00:00:00:02:01),…,actions:2
in_port(9),eth(src=*,dst=ff:ff:ff:ff:ff:ff),…,actions:2,3,4,5in_port(9),eth(src=00:00:00:00:01:03,dst=ff:ff:ff:ff:ff:ff),…,actions:2,3,4,5
00:00:00:00:00:01
00:00:00:00:00:02
00:00:00:00:05:03
00:00:00:00:01:01
00:00:00:00:03:01
00:00:00:00:02:01
00:00:00:00:04:03
00:00:00:00:06:03
eth2 eth9
26
TheBLUprojectPhase2
• BLUGWknows• Linkcapacity• ResidualBHlinkcapacity• BLUs’“desiredbandwidth”(accessifaces bandwidth)
• AcentralizedSDNcontrollerdecidestherouting• Residualcapacitygetsoptimized• InformstheBLUonhowtohandletraffic
• Openflow rulesforMPLSlabelswitching
• RFC3107for(downlink)trafficlabeling
27
TheBLUprojectPhase2
R7
R6
R5R3
R4
R2
Core
0.4Gbps
10
1
0.5
0.5
0.75
0.35 0.45
0.5
6
0.15
Internet
28
TheBLUprojectPhase2
R7
R6
R5R3
R4
R2
Core
0.4Gbps
10
1
0.5
0.5
0.15
0.75
0.35 0.45
0.5
6
Internet
BLUGW
29
TheBLUprojectPhase2
R7
R6
R5R3
R4
R2
Core
0.4Gbps
10
1
0.5
0.5
0.15
0.75
0.35 0.45
0.5
6
in_port(2),mpls=L7,out_port(3) in_port(5),mpls=L7,out_port(9)
Internet
Dst=R7,Label=L7
BLUGW
30
TheBLUprojectPhase2
R7
R6
R5R3
R4
R2
Core
0.4Gbps
10
1
0.5
0.5
0.15
0.15
0.35 0.45
0.5
6
Internet
Dst=R7,Label=L7
BLUGW
31
TheBLUprojectPhase2
R7
R6
R5R3
R4
R2
Core
0.4Gbps
10
1
0.5
0.5
0.15
0.15
0.35 0.45
0.5
6
in_port(2),mpls=L7,out_port(3)
in_port(1),mpls=L7,out_port(4)
in_port(8),mpls=L7,out_port(9)
Internet
in_port(7),mpls=L7,out_port(2)
Dst=R7,Label=L7
BLUGW
in_port(2),mpls=L7,out_port(3) in_port(5),mpls=L7,out_port(9)
32
TheBLUprojectPhase2- Issues
• OpenFlow granularityisrathersmall• AsingleflowthroughthenetworkrequiresalltheBLUsalongthepathtobe(re)programmed
• Everytimethenetworkchanges(!!!)thecontrollerneedstore-computetheroutesandupgradetheBLU
• Upgradesneedtobeatomic,highriskofinconsistentstateofthenetwork
• Thereisnoa“BIGREDBUTTON”tobepressedincaseofcriticalissues!• Howtogobacktoasafeconfigurationincaseoftroubles?
33
TheBLUprojectPhase3
• SegmentRoutingphilosophy
• IGPrunningontheBLUrouters• eBGP,RFC3107
• RFC3107+SRpolicyfordownlinktrafficlabeling
• Labelswitching(OVS)• eBGP governsOpenvSwitch:activeroutesdeterminelabelswitchingflows
34
TheBLUprojectPhase3
R7
R6
R5R3
R4
R2
Core
0.4Gbps
10
1
0.5
0.5
0.75
0.35 0.45
0.5
6
0.15
Internet
35
TheBLUprojectPhase3
R7
R6
R5R3
R4
R2
Core
0.4Gbps
10
1
0.5
0.5
0.75
0.35 0.45
0.5
6
0.15
Internet
mpls=L7,out_port(3)
mpls=L7,out_port(4)
mpls=L7,out_port(9)
mpls=L7,out_port(2)
mpls=L7,out_port(5)
mpls=L7,out_port(3)
Dst=R7,Label=L7
36
TheBLUprojectPhase3
R7
R6
R5R3
R4
R2
Core
0.4Gbps
10
1
0.5
0.5
0.75
0.35 0.45
0.5
6
0.15
Internet
mpls=L7,out_port(3)
mpls=L7,out_port(4)
mpls=L7,out_port(9)
mpls=L7,out_port(2)
mpls=L7,out_port(5)
mpls=L7,out_port(3)
Dst=R7,Label=L7
37
TheBLUprojectPhase3– TrafficEngineering
• Howtooptimizethetrafficflows?
• BLUGWstillknows• (quasi-)Realtimetopology• Residualbandwidth• Desiredaccessifaces bandwidth• NEW:BLUs’IGProutingtables
• Optimizationgoesthroughthesepoints:• Findawaypointwheretoroutethetraffictobeoptimized• InstructtheCOREontherequiredlabelstack
38
TheBLUprojectPhase3
R7
R6
R5R3
R4
R2
Core
0.4Gbps
10
1
0.5
0.5
0.75
0.35 0.45
0.5
6
0.15
Internet
mpls=L7,out_port(3)
mpls=L7,out_port(4)
mpls=L7,out_port(9)
mpls=L7,out_port(2)
mpls=L7,out_port(5)
mpls=L7,out_port(3)
Dst=R7,Label=L7
mpls=L6,out_port(2)
mpls=L6,out_port(4)mpls=L6,out_port(5)
mpls=L6,out_port(1)mpls=L6,out_port(7)
mpls=L6,out_port(7)
39
TheBLUprojectPhase3
R7
R6
R5R3
R4
R2
Core
0.4Gbps
10
1
0.5
0.25
0.75
0.35 0.45
0.5
6
0.15
Internet
mpls=L7,out_port(3)
mpls=L7,out_port(4)
mpls=L7,out_port(9)
mpls=L7,out_port(2)
mpls=L7,out_port(5)
mpls=L7,out_port(3)
Dst=R7,Label=L7
mpls=L6,out_port(2)
mpls=L6,out_port(4)mpls=L6,out_port(5)
mpls=L6,out_port(1)mpls=L6,out_port(7)
mpls=L6,out_port(7)
40
TheBLUprojectPhase3
R7
R6
R5R3
R4
R2
Core
0.4Gbps
10
1
0.5
0.25
0.75
0.35 0.45
0.5
6
0.15
Internet
mpls=L7,out_port(3)
mpls=L7,out_port(4)
mpls=L7,out_port(9)
mpls=L7,out_port(2)
mpls=L7,out_port(5)
mpls=L7,out_port(3)
Dst=R7,Label=[L6,L7]
mpls=L6,out_port(2)
mpls=L6,out_port(4)mpls=L6,out_port(5)
mpls=L6,out_port(1)mpls=L6,out_port(7)
mpls=L6,out_port(7)
41
TheBLUprojectPhase3
R7
R6
R5R3
R4
R2
Core
0.4Gbps
10
1
0.5
0.25
0.75
0.35 0.45
0.5
6
0.15
Internet
mpls=L7,out_port(3)
mpls=L7,out_port(4)
mpls=L7,out_port(9)
mpls=L7,out_port(2)
mpls=L7,out_port(5)
mpls=L7,out_port(3)
Dst=R7,Label=[L6,L7]
mpls=L6,out_port(2)
mpls=L6,out_port(4)mpls=L6,out_port(5)
mpls=L6,out_port(1)mpls=L6,out_port(7)
mpls=L6,out_port(7)
42
TheBLUprojectPhase3- Pros
• Noneedtore-programtheBLUsfromBLUGW• TheyautomaticallylearnhowtoforwardMPLSlabelsbasedonIGProutes
• Optimizer:• Comesintoplayonlyincaseofcongestion• Manytoolstotackletheoptimizationproblem
• Operationsresearch,LP,SMT,etc.+alotofheuristics
• Wedohavea“BigRedButton” now!• Incaseoftroubles,turningofftheoptimizerrevertstoshortestpathforwarding
43
TheBLUprojectTheFuture
• Newhardware()• Motivation:needofmorethan2x10Ginterfacesonimportantsites• SingleinsteadofmanyBLUs(powerefficiency)• Intelbased• BLUos compliant
• Latency-basedoptimizationsfor“delay-aware”traffic(e.g,voice,videostreaming,etc.)• BLUscanmeasurelatencyfortheirBHlinks• Optimizercantakethesevaluesintoconsideration
• BLU2BLU(almost)forfree!• src BLUtagstrafficwiththedst BLU’slabel• …orstackoflabels(dependingonwhattheoptimizersays)
44
TheBLUproject
Thankyou!
IfyouareinterestedintheBLUproject,donothesitatetocontactus!