modular overlay networking solutions with the container...
TRANSCRIPT
Modular Overlay Networking Solutions with the Container Network Interface
Murali PaluruPrincipal Software Engineer,
Rancher Labs, Inc.
Open Source Summit North America,13th September 2017
Slides (without animation) are posted in PDF format here:https://ossna2017.sched.com/event/BDpo/modular-overlay-networking-solutions-with-the-container-network-interface-murali-paluru-rancher-labs-inc
Original Slides (with animation) can be found here:https://goo.gl/L2bdZt
Slides availability
Agenda● Introduction● Evolution of Container Networking● History of Container Network Interface (CNI)● CNI Overview● Walkthrough of CNI Specification● SDK + Reference CNI plugins● Build a simple CNI runtime + CNI plugin● Questions (You don’t have to wait till the end, interrupt!)
Agenda● Introduction● Evolution of Container Networking● History of Container Network Interface (CNI)● CNI Overview● Walkthrough of CNI Specification● SDK + Reference CNI plugins● Build a simple CNI runtime + CNI plugin● Questions (You don’t have to wait till the end, interrupt!)
Who am I?● Plumber (Principal Software Engineer) at Rancher Labs.
Build the pipes between containers (Take care of networking)
What have I done?https://github.com/leodotcloud
- Build, maintain networking microservices for Rancher product (complete container management platform)- CNI plugins: VXLAN, IPSec- Network Policy- Solve networking issues in customer setups
Past: Startup, Gigamon, Cisco
How to reach out to us/me?Twitter: @leodotcloud
Email: [email protected] or [email protected]
Rancher Product: https://github.com/rancher/rancher
Community Users: https://forums.rancher.com
Slack: https://slack.rancher.io
If you want to play with containers/microservices: https://try.rancher.com
Agenda● Introduction● Evolution of Container Networking● History of Container Network Interface (CNI)● CNI Overview● Walkthrough of CNI Specification● SDK + Reference CNI plugins● Build a simple CNI runtime + CNI plugin● Questions (You don’t have to wait till the end, interrupt!)
Docker Networking made easy
Subnet 172.17.0.0/16: {172.17.0.1, 172.17.0.2, … } { Amy, Bob, Deb, Eli, Kim …. }
HOST 1
One of the solutions: Published ports
10.250.0.100:300010.250.0.100:32769...
10.250.0.200:808010.250.0.200:32769...
Published ports (made easy)
10.250.0.100:300010.250.0.100:32769...
10.250.0.200:808010.250.0.200:32769...
Bob
Deb
Amy
Bob
Deb
Amy
Floor 1
Floor 2
Floor 1
Floor 2
Building Alpha Building Bravo
Published ports: Containers on different servers can talk!
10.250.0.100:300010.250.0.100:32769...
10.250.0.200:808010.250.0.200:32769...
Bob
Deb
Amy
Bob
Deb
Amy
Floor 1
Floor 2
Floor 1
Floor 2
Building Alpha Building Bravo
Hi, this is the person calling from second floor of Building Bravo, can I talk to to the person living on first floor of Building Alpha?
Rancher’s solution: Unique subnet for containers on all hosts
Subnet 172.17.0.0/16: {172.17.0.1, 172.17.0.2, … } { Amy, Bob, Deb, Eli, Kim …. }Subnet 10.42.0.0/16: {10.42.0.1, 10.42.0.2, …} { Gandhi, Ram, Budh, Vyas, Meera …. }
Rancher’s solution: Unique subnet for containers on all hosts
Bob (Ram)
Deb (Budh)
Amy (Gandhi)
Bob (Vyas)
Deb (Meera)
Amy (Gandhi)
Kubernetes Solution (made easy)
Homework for your imagination
Subnet 10.10.1.0/24: {10.10.1.1, 10.10.1.2, … } { ?, ?, ? …. }Subnet 10.10.2.0/24: {10.10.2.1, 10.10.2.2, … } { ?, ?, ? …. }
Agenda● Introduction● Evolution of Container Networking● History of Container Network Interface (CNI)● CNI Overview● Walkthrough of CNI Specification● SDK + Reference CNI plugins● Build a simple CNI runtime + CNI plugin● Questions (You don’t have to wait till the end, interrupt!)
Industry wants a standardA standard/interface/API is required so that 3rd party vendors can plug in their custom network solution into a given container runtime.
Docker launches CNM (libnetwork)https://www.sdxcentral.com/articles/news/docker-acquires-networking-startup-socketplane/2015/03/
CNM: Container Network Model
Source: https://github.com/docker/libnetwork/blob/master/docs/design.md
CoreOS(rkt) releases CNI as part of appcCNI: Container Network Interface
Organisation for the App Container specification, including the schema and associated tooling: https://github.com/appc
Part of effort to define open standards/specifications to support other runtimes.
CNM vs CNIWorks with Docker runtime.
Daemon
Project Calico, Weave, Cisco Contiv, Kuryr, Open Virtual Networking (OVN), VMware, …
Needs a KV store for global plugins.
Built in: none, bridge, macvlan, ipvlan, etc.
Support for multiple networks.
Works with rkt, kurma, …
Binary
Kubernetes, Rancher, Project Calico, Weave, Cisco Contiv, Apache Mesos, Cloud Foundry, Kurma, Cilium, Infoblox, …
It’s up to the plugin to decide.
Built in: loopback, bridge, macvlan, ipvlan, etc.
Multiple networks: Work in progress
Disadvantages:Some problems with docker ps and docker inspect
CNI adoptionKubernetes(k8s) adopted CNI and blogged about the reasons behind their decision: http://blog.kubernetes.io/2016/01/why-Kubernetes-doesnt-use-libnetwork.html
Project Calico.
Weave.
Mesos embraces CNI.
Rancher embraces CNI.
CNCF accepted CNI. https://www.cncf.io/blog/2017/05/23/cncf-hosts-container-networking-interface-cni/
Agenda● Introduction● Evolution of Container Networking● History of Container Network Interface (CNI)● CNI Overview● Walkthrough of CNI Specification● SDK + Reference CNI plugins● Build a simple CNI runtime + CNI plugin● Questions (You don’t have to wait till the end, interrupt!)
What are the responsibilities of “network plugin”?
1. Take care of plumbing, plugging a container in a network, provide connectivity, etc.
2. Also need to give an identity (IP, etc, etc) to the container.○ Sometimes, the plugin can do this myself○ In general the plugin needs help and ask another plugin (IPAM plugin) for this information.
Step 1: Runtime adds/deletes a container
runtime container1
NOTE: In case of using docker, the container is started with “--net=none”
Step 3: CNI network plugin invokes IPAM plugin (optional)
runtime container
CNI network plugin
1
2
CNI IPAM plugin3
Step 4: IPAM plugin allocates/deallocates, returns IPAM info (optional)
runtime container
CNI network plugin
1
2
CNI IPAM plugin3
4
Step 5: CNI network plugin setup/cleanup networking for container
runtime container
CNI network plugin
1
2
CNI IPAM plugin3
4
5
Step 6: CNI plugin informs runtime about success/failure
runtime container
CNI network plugin
1
2
CNI IPAM plugin3
4
56
Agenda● Introduction● Evolution of Container Networking● History of Container Network Interface (CNI)● CNI Overview● Walkthrough of CNI Specification● SDK + Reference CNI plugins● Build a simple CNI runtime + CNI plugin● Questions (You don’t have to wait till the end, interrupt!)
JSON => Step 2: Run time invokes CNI network plugin
runtime container
CNI network plugin
1
2
●●
CNI_VERSIONCNI_CONTAINERIDCNI_NETNSCNI_IFNAMECNI_COMMANDCNI_PATHCNI_ARGS
JSON => Step 3: CNI network plugin invokes IPAM plugin (optional)
runtime container
CNI network plugin
1
2
CNI IPAM plugin3
●● CNI_VERSION
CNI_CONTAINERIDCNI_NETNSCNI_IFNAMECNI_COMMANDCNI_PATHCNI_ARGS
JSON => Step 4: IPAM plugin allocates/deallocates, returns IPAM info (optional)
runtime container
CNI network plugin
1
2
CNI IPAM plugin3
4
●●
JSON => Step 4: IPAM plugin allocates/deallocates, returns IPAM info (optional)
runtime container
CNI network plugin
1
2
CNI IPAM plugin3
4
●●
JSON => Step 6: CNI plugin informs runtime about success/failure
runtime container
CNI network plugin
1
2
CNI IPAM plugin3
4
56
●●
JSON => Step 6: CNI plugin informs runtime about success/failure
runtime container
CNI network plugin
1
2
CNI IPAM plugin3
4
56
●●
Agenda● Introduction● Evolution of Container Networking● History of Container Network Interface (CNI)● CNI Overview● Walkthrough of CNI Specification● SDK + Reference CNI plugins● Build a simple CNI runtime + CNI plugin● Questions (You don’t have to wait till the end, interrupt!)
CNI Github Repositories● https://github.com/containernetworking/cni
○ contains the Go source code of a library for integrating CNI into applications (libcni)○ example command-line tool (cnitool)
● https://github.com/containernetworking/plugins○ Main: bridge, loopback, ipvlan, macvlan, ptp, vlan○ IPAM: host-local, dhcp○ Meta: tuning, portmap, flannel
The above plugins can be forked and customized to your needs.Example: rancher-cni-bridge, rancher-cni-ipam
Agenda● Introduction● Evolution of Container Networking● History of Container Network Interface (CNI)● CNI Overview● Walkthrough of CNI Specification● SDK + Reference CNI plugins● Build a simple CNI runtime + CNI plugin● Questions (You don’t have to wait till the end, interrupt!)
Files for demo
https://github.com/leodotcloud/ossna2017-cni-presentation
Agenda● Introduction● Evolution of Container Networking● History of Container Network Interface (CNI)● CNI Overview● Walkthrough of CNI Specification● SDK + Reference CNI plugins● Build a simple CNI runtime + CNI plugin● Questions (You don’t have to wait till the end, interrupt!)