when devops and networking intersect by brent salisbury of socketplane.io
DESCRIPTION
When DevOps and Networking Intersect by Brent Salisbury of socketplane.ioTRANSCRIPT
![Page 1: When DevOps and Networking Intersect by Brent Salisbury of socketplane.io](https://reader034.vdocument.in/reader034/viewer/2022052601/558e684a1a28ab81218b47c2/html5/thumbnails/1.jpg)
when network and devops intersect
Brent Salisbury socketplane.io
![Page 2: When DevOps and Networking Intersect by Brent Salisbury of socketplane.io](https://reader034.vdocument.in/reader034/viewer/2022052601/558e684a1a28ab81218b47c2/html5/thumbnails/2.jpg)
socketplane.io - docker networking
John Willis Co-Founder & VP Business Development Formerly: Formerly CTO Stateless Networks
Madhu Venugopal Co-Founder & President Formerly: Principal Engineer Office of the CTO, Red Hat
Brent Salisbury Co-Founder & VP Engineering Formerly: Senior Engineer Office of the CTO, Red Hat
Dave Tucker Co-Founder, VP Product Formerly: Senior Engineer Office of the CTO, Red Hat
![Page 3: When DevOps and Networking Intersect by Brent Salisbury of socketplane.io](https://reader034.vdocument.in/reader034/viewer/2022052601/558e684a1a28ab81218b47c2/html5/thumbnails/3.jpg)
lessons_learned struct
1. the evolving network!2. lessons learned from controller development!3. netops from an operational+dev view!4. looking ahead
![Page 4: When DevOps and Networking Intersect by Brent Salisbury of socketplane.io](https://reader034.vdocument.in/reader034/viewer/2022052601/558e684a1a28ab81218b47c2/html5/thumbnails/4.jpg)
the problem
![Page 5: When DevOps and Networking Intersect by Brent Salisbury of socketplane.io](https://reader034.vdocument.in/reader034/viewer/2022052601/558e684a1a28ab81218b47c2/html5/thumbnails/5.jpg)
Cos
tNetwork
Compute - Storage
VerticalIntegration
Horizontal Scale
Number Widgets - Economies of Scale
![Page 6: When DevOps and Networking Intersect by Brent Salisbury of socketplane.io](https://reader034.vdocument.in/reader034/viewer/2022052601/558e684a1a28ab81218b47c2/html5/thumbnails/6.jpg)
Network Capacity
Needs
Over Provisioned
Netw
ork
Usag
e G
rowt
h
Time
Under Provisioned
![Page 7: When DevOps and Networking Intersect by Brent Salisbury of socketplane.io](https://reader034.vdocument.in/reader034/viewer/2022052601/558e684a1a28ab81218b47c2/html5/thumbnails/7.jpg)
Network Capacity
Needs
Net
wor
k U
sage
Gro
wth
Time
Efficient Provisioning
![Page 8: When DevOps and Networking Intersect by Brent Salisbury of socketplane.io](https://reader034.vdocument.in/reader034/viewer/2022052601/558e684a1a28ab81218b47c2/html5/thumbnails/8.jpg)
Where we were
• CLI for everything • vendor management tools did everything and nothing. • used to be Perl, TCL and later Python • zero ip management !• turned into a who can make the best obscure magic !
![Page 9: When DevOps and Networking Intersect by Brent Salisbury of socketplane.io](https://reader034.vdocument.in/reader034/viewer/2022052601/558e684a1a28ab81218b47c2/html5/thumbnails/9.jpg)
Where we are
• CLI for everything • vendor management tools did everything and nothing. • used to be Perl, TCL and later Python • zero ip management !
• turned into a who can make the best obscure magic !
![Page 10: When DevOps and Networking Intersect by Brent Salisbury of socketplane.io](https://reader034.vdocument.in/reader034/viewer/2022052601/558e684a1a28ab81218b47c2/html5/thumbnails/10.jpg)
where we are(ish)
• exponential growth with flat operating budgets!• incessant pressure for uptime + capex/opex cost
reduction!• the majority of networks still maintain proprietary hw,
sw and api!• datapaths are still barely programmable !• netops manages very little beyond the ToR.
![Page 11: When DevOps and Networking Intersect by Brent Salisbury of socketplane.io](https://reader034.vdocument.in/reader034/viewer/2022052601/558e684a1a28ab81218b47c2/html5/thumbnails/11.jpg)
quick review of node distribution
• distributed!• centralized!• de-centralized
![Page 12: When DevOps and Networking Intersect by Brent Salisbury of socketplane.io](https://reader034.vdocument.in/reader034/viewer/2022052601/558e684a1a28ab81218b47c2/html5/thumbnails/12.jpg)
Centralized
![Page 13: When DevOps and Networking Intersect by Brent Salisbury of socketplane.io](https://reader034.vdocument.in/reader034/viewer/2022052601/558e684a1a28ab81218b47c2/html5/thumbnails/13.jpg)
Centralized
Forwarding Population
Controller
Match + Action
the sdn approach
![Page 14: When DevOps and Networking Intersect by Brent Salisbury of socketplane.io](https://reader034.vdocument.in/reader034/viewer/2022052601/558e684a1a28ab81218b47c2/html5/thumbnails/14.jpg)
Decentralized
![Page 15: When DevOps and Networking Intersect by Brent Salisbury of socketplane.io](https://reader034.vdocument.in/reader034/viewer/2022052601/558e684a1a28ab81218b47c2/html5/thumbnails/15.jpg)
Decentralized
Topology
Forwarding Population + Clustered Controller
Orchestration
Match + Action
the sdn approach
![Page 16: When DevOps and Networking Intersect by Brent Salisbury of socketplane.io](https://reader034.vdocument.in/reader034/viewer/2022052601/558e684a1a28ab81218b47c2/html5/thumbnails/16.jpg)
similarly both hard problemsRouting Engine
Line Card 1
P...P1 P2
MAC Source Addres
s
MAC Destinati
on
IP Source Address
IP Destinati
on
Source
Port
Destination Port Instructions
Ingress
Port
Priority
* * * * * *
GOTO/Drop/
Controller/Normal
*.0
Protocol
*
Data Plane
Line Card 2
P...P1 P2
MAC Source Addres
s
MAC Destinati
on
IP Source Address
IP Destinati
on
Source
Port
Destination Port Instructions
Ingress
Port
Priority
* * * * * *
GOTO/Drop/
Controller/Normal
*.0
Protocol
*
Data Plane
Line Card ...
P...P1 P2
MAC Source Addres
s
MAC Destinati
on
IP Source Address
IP Destinati
on
Source
Port
Destination Port Instructions
Ingress
Port
Priority
* * * * * *
GOTO/Drop/
Controller/Normal
*.0
Protocol
*
Data Plane
Controller
OVS
P...P1 P2
MAC Source Addres
s
MAC Destinati
on
IP Source Address
IP Destinati
on
Source
Port
Destination Port Instructions
Ingress
Port
Priority
* * * * * *
GOTO/Drop/
Controller/Normal
*.0
Protocol
*
Data Plane
OF Switch
P...P1 P2
MAC Source Addres
s
MAC Destinati
on
IP Source Address
IP Destinati
on
Source
Port
Destination Port Instructions
Ingress
Port
Priority
* * * * * *
GOTO/Drop/
Controller/Normal
*.0
Protocol
*
Data Plane
Random Agent
P...P1 P2
MAC Source Addres
s
MAC Destinati
on
IP Source Address
IP Destinati
on
Source
Port
Destination Port Instructions
Ingress
Port
Priority
* * * * * *
GOTO/Drop/
Controller/Normal
*.0
Protocol
*
Data Plane
Bus EthernetFabric
![Page 17: When DevOps and Networking Intersect by Brent Salisbury of socketplane.io](https://reader034.vdocument.in/reader034/viewer/2022052601/558e684a1a28ab81218b47c2/html5/thumbnails/17.jpg)
Distributed
![Page 18: When DevOps and Networking Intersect by Brent Salisbury of socketplane.io](https://reader034.vdocument.in/reader034/viewer/2022052601/558e684a1a28ab81218b47c2/html5/thumbnails/18.jpg)
Distributedthe internets scales
![Page 19: When DevOps and Networking Intersect by Brent Salisbury of socketplane.io](https://reader034.vdocument.in/reader034/viewer/2022052601/558e684a1a28ab81218b47c2/html5/thumbnails/19.jpg)
Host 1
L2 Flooding and Learning
Host 2
Data PlaneData PlaneFlooding Flooding
VLAN xVLAN x
!• Live workload migration cripples network ops!• subnets for policy groupings are the only reason to think
in those terms anymore
the barrier to scale
![Page 20: When DevOps and Networking Intersect by Brent Salisbury of socketplane.io](https://reader034.vdocument.in/reader034/viewer/2022052601/558e684a1a28ab81218b47c2/html5/thumbnails/20.jpg)
shit that doesn't scale• the next few slides are
things i thought were possible at some point around the problem of L2!!
• lesson learned prototype and fail faster!!
• ask your team why they really need L2
![Page 21: When DevOps and Networking Intersect by Brent Salisbury of socketplane.io](https://reader034.vdocument.in/reader034/viewer/2022052601/558e684a1a28ab81218b47c2/html5/thumbnails/21.jpg)
Host 1
OpenFlow Controller
Proactive L2 Flooding and Learning with Legacy VLANs
Host 2
Data PlaneData Plane
Maintaining Legacy Broadcast Domains Controller Never Punts ARP
Flooding FloodingVLAN x
VLAN x
Proactive Rule - Match: ARP Action: Normal
Can Also Serve as a Fallback Failure Mode or Hybrid Mirgration Strategy
![Page 22: When DevOps and Networking Intersect by Brent Salisbury of socketplane.io](https://reader034.vdocument.in/reader034/viewer/2022052601/558e684a1a28ab81218b47c2/html5/thumbnails/22.jpg)
OpenFlow Switch
Data PlaneP3P1 P2
Svr 2 Svr 3Svr 1
OpenFlow Controller
MAC Source Addres
s
MAC Destinati
on
IP Source Address
IP Destinati
on
Source
Port
Destination Port Instructions
Ingress
Port
Priority
* * * * * *
GOTO/Drop/
Controller/Normal
*.0
Protocol
*
Packet-In A Flowmod Installs a Flow Rule for Subsequent Matching Packets
Reactive OpenFlow Flow Policy1st Packet in Flow
![Page 23: When DevOps and Networking Intersect by Brent Salisbury of socketplane.io](https://reader034.vdocument.in/reader034/viewer/2022052601/558e684a1a28ab81218b47c2/html5/thumbnails/23.jpg)
Controller Intercepting ARP and Proxy the Reply
Host 2
Data PlaneData Plane
VLAN ID Constraints Becomes IrrelevantTenancy Maintained in the Controller
Switch 1 Switch 2
Host 1
Match: ARP Action: Controller Match: ARP Action: Controller
ARP Requestand Reply
Controllers can Answers and/or Sends ARP (proxy)
OpenFlow Controller
Host 2 IP, MAC,Tenant ==> Tunnel 200 Tep IP Host (Key) Location (Value)
![Page 24: When DevOps and Networking Intersect by Brent Salisbury of socketplane.io](https://reader034.vdocument.in/reader034/viewer/2022052601/558e684a1a28ab81218b47c2/html5/thumbnails/24.jpg)
Controller Connect Source and Destination Hosts via Packet-In and Flowmods
Host 2
Data PlaneData PlaneSwitch 1 Switch 2
Host 1
Match: ARP Action: Controller Match: ARP Action: Controller
ARP Request
Data Path (Tunnel, or Flow Path
FlowmodBuilding Data Path
OpenFlow Controller
Host 2 IP, MAC,Tenant ==> Tunnel 200 Tep IP Host (Key) Location (Value)
FlowmodBuilding Data Path
VLAN ID Constraints Becomes IrrelevantTenancy Maintained in the Controller
![Page 25: When DevOps and Networking Intersect by Brent Salisbury of socketplane.io](https://reader034.vdocument.in/reader034/viewer/2022052601/558e684a1a28ab81218b47c2/html5/thumbnails/25.jpg)
not if but when
!• build infrastructure for the worst
case scenario, because it will be worse.!
• cascading failure suck!• focus on solving the problem
not the implementation!• intelligence in the datapath HW
is a good thing as long ideally if coupled with open and programmatically manageable
P3P1 P2
DPID DPIDDPID
Control Plane
Control and Data Plane Split Brain
?
?
???
Data Plane - DPID ::00:01
X
![Page 26: When DevOps and Networking Intersect by Brent Salisbury of socketplane.io](https://reader034.vdocument.in/reader034/viewer/2022052601/558e684a1a28ab81218b47c2/html5/thumbnails/26.jpg)
Linux Bridging
BridgeFrame In
IPTables
Frame Egress
HAProxy Functions X,Y, Z
this movie has a shitty ending
![Page 27: When DevOps and Networking Intersect by Brent Salisbury of socketplane.io](https://reader034.vdocument.in/reader034/viewer/2022052601/558e684a1a28ab81218b47c2/html5/thumbnails/27.jpg)
What Works: Performance and Reliability First
OVS/DPDK Packet Forwarding Pipeline
Classifier
Table 0Frame In
FunctionFoo
Table 2
FunctionBar
Frame Out…….. Table n
Stages
![Page 28: When DevOps and Networking Intersect by Brent Salisbury of socketplane.io](https://reader034.vdocument.in/reader034/viewer/2022052601/558e684a1a28ab81218b47c2/html5/thumbnails/28.jpg)
Data CenterL3 Core
Data CenterL3 Core
PhysicalSwitch vSwitch Physical
Switch
vSwitch PhysicalSwitch vSwitch
Firewall
North/South Security Policy
Data Center Today
traffic alignment from the 90’s
![Page 29: When DevOps and Networking Intersect by Brent Salisbury of socketplane.io](https://reader034.vdocument.in/reader034/viewer/2022052601/558e684a1a28ab81218b47c2/html5/thumbnails/29.jpg)
Data CenterL3 Core
Data CenterL3 Core
PhysicalSwitch vSwitch Physical
Switch
vSwitch PhysicalSwitch vSwitch
East West Security Policy
Distributed Policy Application For Data Center
new architectures for new workloads
![Page 30: When DevOps and Networking Intersect by Brent Salisbury of socketplane.io](https://reader034.vdocument.in/reader034/viewer/2022052601/558e684a1a28ab81218b47c2/html5/thumbnails/30.jpg)
trust what you know• rely your own operational experiences, if you don't have any go
get some even if its stalking customers!• don't fall in love with implementations, they are probably wrong!• ask questions but be open minded!• avoid slide jockeys!• avoid the vendor wars!• avoid cults!• complexity w/o abstraction fails!• almost all abstractions fail
![Page 31: When DevOps and Networking Intersect by Brent Salisbury of socketplane.io](https://reader034.vdocument.in/reader034/viewer/2022052601/558e684a1a28ab81218b47c2/html5/thumbnails/31.jpg)
serenity now, insanity later
• make time for research and planning!!• wether it is a big infra project or an dev sprint, don't
let the oppressive demand of execution compromise a practical design!!
• that said, if the plan sucks, change it.
![Page 32: When DevOps and Networking Intersect by Brent Salisbury of socketplane.io](https://reader034.vdocument.in/reader034/viewer/2022052601/558e684a1a28ab81218b47c2/html5/thumbnails/32.jpg)
nothing is easy, don't make it harder
• prototyping and early feedback should be your compass
• when users says, this seems a little too complex, LISTEN!
• odds are you aren't going to be able to get the right abstraction to hide your over-engineering
![Page 33: When DevOps and Networking Intersect by Brent Salisbury of socketplane.io](https://reader034.vdocument.in/reader034/viewer/2022052601/558e684a1a28ab81218b47c2/html5/thumbnails/33.jpg)
performance and reliability first
• network operators are measured in uptime first • don't compromise reliability for cost savings without
making it very clear to all leadership, not just the IT manager heroes.
• perform consistency checking
![Page 34: When DevOps and Networking Intersect by Brent Salisbury of socketplane.io](https://reader034.vdocument.in/reader034/viewer/2022052601/558e684a1a28ab81218b47c2/html5/thumbnails/34.jpg)
/dev• understand the problem first!!
• if you don't understand the problem stalk someone who does!!
• make readable code!!
• code for the worst case scenario
![Page 35: When DevOps and Networking Intersect by Brent Salisbury of socketplane.io](https://reader034.vdocument.in/reader034/viewer/2022052601/558e684a1a28ab81218b47c2/html5/thumbnails/35.jpg)
architecture• if it isn't broke, don't break it • architects need understandable components • architects need predictable components • predictive analysis is a big data problem • predict problems with operational tools and data • don't build a nuclear submarine when a bicycle will do
![Page 36: When DevOps and Networking Intersect by Brent Salisbury of socketplane.io](https://reader034.vdocument.in/reader034/viewer/2022052601/558e684a1a28ab81218b47c2/html5/thumbnails/36.jpg)
test and prototype!
• verify before you hit enter!• automate all production changes!• setup rollback processes!!
• the result:!• should be shorter change windows!• faster rollbacks!• better trained operators
![Page 37: When DevOps and Networking Intersect by Brent Salisbury of socketplane.io](https://reader034.vdocument.in/reader034/viewer/2022052601/558e684a1a28ab81218b47c2/html5/thumbnails/37.jpg)
everybody is smart
• "A great team doesn’t mean that they had the smartest people. What made those teams great is that everyone trusted one another. It can be a powerful thing when that magic dynamic exists." -Gene Kim
![Page 38: When DevOps and Networking Intersect by Brent Salisbury of socketplane.io](https://reader034.vdocument.in/reader034/viewer/2022052601/558e684a1a28ab81218b47c2/html5/thumbnails/38.jpg)
team culture
• not proving how much smarter you are then your co-workers.
• give credit to the team first, its just weird otherwise
• don't hoard contacts • find peoples passion and
maximize it • protect your cultures morale like it
is your bank account
![Page 39: When DevOps and Networking Intersect by Brent Salisbury of socketplane.io](https://reader034.vdocument.in/reader034/viewer/2022052601/558e684a1a28ab81218b47c2/html5/thumbnails/39.jpg)
• starting out!• no one can learn for you, find your passion!• learn linux!• explore vswitches, I recommend http://openvswitch.org!• connect with peers in the community and share experiences
where to start?
• explore compute (containers, hypervisors and everything else beyond the top of rack!!
• further along!• code, i recommend Golang atm fwiw!• learn CI tools and sw dev processes!• contributes to upstream open source!• build something that solves others
problems and open source it