openstack summit pluggable ipam
TRANSCRIPT
OPENSTACK PLUGGABLE IPAM
Development and deployment experience with Romana Cloud Native networks
Chris MarinoRobert Starmer
OpenStack Summit Austinromana.io
Before Pluggable IPAM• Workflow
• Each Tenant creates segment• Manually Assigns CIDR• DHCP server issues IPs as they get launched
• Problems and issues• Many manual and error prone steps• Coordination with DC VLANs• Gateway misconfiguration• Duplicate IPs• Integration with enterprise IPAM
April 2016 Slide 2
OpenStack Summit Austinromana.io
Had to change….• Old approach
• Monolithic with Neutron plugins and needed to be pulled out separately
• New requirements• Separated IPAM driver with pluggable back end
• Support vendor specific back end implementation
• Large development effort to refactor code• Congrats to John Belamaric and rest of team
April 2016 Slide 3
OpenStack Summit Austinromana.io
Old/New IPAM sequencing
April 2016 Slide 4April 2016romana.io
NeutronPlugin
NeutronPlugin
NeutronDB Plugin
NeutronDB Plugin v2
IPAMDriver
PluggableIPAM
NeutronDB
IPAMSubnet
create_portcreate_port
get_subnet
Allocate_IP
Allocate_IP
IP
IPAMSubnet
port, IP data port, IP data
port, IP data
OpenStack Summit Austinromana.io
Neutron
Node nNode nNode nNode nNode n
VM VM
External IPAM
Typical Deployment
April 2016 Slide 5
vSwitch
ML2IPAM
iptablesL2
OpenStack Summit Austinromana.io
Pluggable IPAM advantages• IPAM necessary for many enterprise deployments• Enables innovative deployment alternatives• Intelligent IP address assignment
• Simplify OpenStack operations • Increase performance• Enable nested endpoints for container networking
April 2016 Slide 6
OpenStack Summit Austinromana.io
Romana Project• Network and Security Automation• Layer 3 based isolation and tenancy model
• Assign tenants and segments physical IP ranges• Hierarchical addressing enables route aggregation• Apply security directly to physical network
• Requires nothing more than standard L3 routing• No virtual network required
• Native performance and visibility• Eliminates overlays
• Works for nested container endpoints too! • Intelligent IPAM combined with route control
April 2016 Slide 7
OpenStack Summit Austinromana.io
Romana Project
April 2016 Slide 8
REST Call Returns IP
IPAM Driver
Romana
IPAM
Routes
NeutronPlugin
NeutronDB Plugin v2
IPAMDriver
PluggableIPAM
NeutronDB
IPAMSubnet
OpenStack Summit Austinromana.io
Neutron
Node nNode nNode nNode nNode n
Agent
VM VM
Romana Deployment
April 2016 Slide 9
Romana
IPAM
Routes
ML2IPAM
iptables
OpenStack Summit Austinromana.io
Romana REST API
April 2016 Slide 10
{ # In case of OpenStack, this is the project's UUID "tenant_id" : "Tenant ID",
# Segment ID. In case of OpenStack, this is # the value of the metadata tag whose name is 'romanaSegment' "segment_id" : "Segment ID",
# Host ID. In case of OpenStack, this is the value of # 'binding:host_id' field of port object. "host_id" : "Host ID"
# Optional "name" : "Endpoint name",}
{ "ip" : "10.0.0.3", "id" : 37,
# In case of OpenStack, this is the project's UUID "tenant_id" : "Tenant ID",
# Segment ID. This is the OpenStack equivalent of L3 network' "segment_id" : "Segment ID",
# Host ID. "host_id" : "Host ID"
# Optional "name" : "Endpoint name",}
Example: Get new IP AddressPOST
Response
• Available Resources• Tenants, Segments, Endpoints, Hosts, Policies
OpenStack Summit Austinromana.io
Example
April 2016 Slide 11
Bit location 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32FieldCapacity 0 0 0 0 1 0 1 0
Example: Bits Length Purpose10.0 Network 8 Full Network (10/8)Hosts 8 Up to 255 HostsTenants 4 Up to 16 TenantsSegments 4 Up to 16 Segments per TenantEndpoints 8 Up to 255 Endpoints per Segment
Bit location 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32Field Host ID Bits (4)Capacity 1 0 1 0 1 1 0 0 0 0 0 1 Up to 16 Hosts
Example: Bits Length Purpose172.16 Network 12 Full Network (172.16/12)Hosts 4 Up to 16 Hosts
4 Up to 16 TenantsSegments 4 Up to 16 Segments per TenantEndpoints 8 Up to 255 Endpoints per Segment
Endpoint IDUp to 255 Hosts Up to 255 Tenant/Segments 255 Endpoints
20 17-20
10/8 Net Mask Host ID Bits (8) Tenant and Segment ID Bits (8)
Location8 1-816 9-16
24 21-2432 25-32
OpenStack Summit Austinromana.io
Physical Deployment
April 2016 Slide 12
192.168.0.10 192.168.0.11 192.168.0.12
Host 1
VM 1: 10.1.1.22
G/W: 10.1.0.1/16
VM 1: 10.1.1.33
VM 1: 10.1.2.44
VM 1: 10.1.2.55
10.2/16 -> 192.168.0.1110.3/16 -> 192.168.0.12
Host 2
VM 1: 10.2.1.22
G/W: 10.2.0.1/16
VM 1: 10.2.1.33
VM 1: 10.2.2.44
VM 1: 10.2.2.55
10.1/16 -> 192.168.0.1010.3/16 -> 192.168.0.12
Host 3
VM 1: 10.3.1.22
G/W: 10.3.0.1/16
VM 1: 10.3.1.33
VM 1: 10.3.2.44
VM 1: 10.3.2.55
10.1/16 -> 192.168.0.1010.2/16 -> 192.168.0.11
OpenStack Summit Austinromana.io
North/South Traffic
April 2016 Slide 13
• Neutron Network node routes traffic between segments
• Network node performs all L3 functions
• East/West traffic encapsulated, but is direct to destination host
VXLAN Decap
VXLAN Decap
VXLAN Encap
VXLAN Encap
2 Top of Rack Round Trips
East/West Traffic
Per Instance Security
OpenStack Summit Austinromana.io
Direct routed paths• Latency dramatically
reduced• No Network node• No encap
• Identical path for East/West traffic
April 2016 Slide 14
Eliminated
Bypassed
Bypassed
Romana
Romana
1 Top of Rack Round Trip
Per Network Security
OpenStack Summit Austinromana.io
Direct Routing Latency
• North/South Latency reduced 50%-85%• 10% improvement for East/West traffic between hosts (no encap)• No performance penalty for local on-host East/West traffic
April 2016 Slide 15
North/South
(Routed)East/West (Switched)
Time (ms) Local Remote Local Remote
Native OpenStack 1.51* 1.51 0.24 0.85
Pani Networks 0.24 0.77 0.24** 0.77**
Relative Performance Local Remote Local Remote
Native OpenStack 100% 100% 100% 100%Pani Networks 16% 51% 100% 90%
* All N/S OpenStack traffic goes off host** All Pani traffic is routed
OpenStack Summit Austinromana.io
Nested Container Networking
April 2016 Slide 16
Bit location 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32FieldCapacity 0 0 0 0 1 0 1 0
Example: Bits Length Purpose10.0 Network 8 Full Network (10/8)Hosts 8 Up to 255 HostsTenants 4 Up to 16 TenantsSegments 4 Up to 16 Segments per TenantEndpoints 8 Up to 255 Endpoints per Segment
Bit location 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32Field Host ID Bits (4)Capacity 1 0 1 0 1 1 0 0 0 0 0 1 Up to 16 Hosts
Example: Bits Length Purpose172.16 Network 12 Full Network (172.16/12)Hosts 4 Up to 16 HostsTenants 4 Up to 16 TenantsSegments 4 Up to 16 Segments per TenantEndpoints 8 Up to 255 Endpoints per Segment
Endpoint IDUp to 255 Hosts Up to 255 Tenant/Segments 255 Endpoints
Tenant and Segment ID Bits (8) Endpoint IDUp to 255 Tenant/Segments 255 Endpoints
Location12 1-1216
20 17-20
10/8 Net Mask Host ID Bits (8) Tenant and Segment ID Bits (8)
Location8 1-816 9-16
24 21-2432 25-32
13-1620 17-2024 21-2432 25-32
172.16/12 Net Mask
OpenStack Summit Austin
Nested Containers
April 2016romana.io
192.168.0.10 192.168.0.11 192.168.0.12
Slide 17
Host 1
VM 1: 10.1.1.22
G/W: 10.1.0.1/16
10.2/16 -> 192.168.0.1110.3/16 -> 192.168.0.12172.17/16-> 192.168.0.11172.18/16 -> 192.168.0.12
Pod 172.16.1.8
Pod 172.16.2.9
GW 172.16.0.1/16
172.17/16 -> 10.2.0.1172.18/16 -> 10.3.0.1
Host 2
VM 1: 10.2.1.22
G/W: 10.2.0.1/16
Pod 172.17.6.8
Pod 172.17.2.11
GW 172.17.0.1/16
172.18/16 -> 10.3.0.1172.16.16 -> 10.1.0.1
Host 3
VM 1: 10.3.1.22
G/W: 10.3.0.1/16
Pod 172.18.3.8
Pod 172.18.4.9
GW 172.18.0.1/16
172.16/16 -> 10.1.0.1172.17/16 -> 10.2.0.1
10.1/16 -> 192.168.0.1010.3/16 -> 192.168.0.12172.16/16 -> 192.168.0.10172.18/16 -> 192.168.0.12
10.1/16 -> 192.168.0.1010.2/16 -> 192.168.0.11172.16/16 -> 192.168.0.10172.17/16-> 192.168.0.11
OpenStack Summit Austin
Ubernetes
April 2016romana.io
192.168.0.10 192.168.0.11 192.168.0.12
Slide 18
Host 1
VM 1: 10.1.1.22
G/W: 10.1.0.1/16
10.2/16 -> 192.168.0.1110.3/16 -> 192.168.0.12172.17/16-> 192.168.0.11172.18/16 -> 192.168.0.12
Pod 172.16.1.8
Pod 172.16.2.9
GW 172.16.0.1/16
172.17/16 -> 10.2.0.1172.18/16 -> 10.3.0.1
Host 2
VM 1: 10.2.1.22
G/W: 10.2.0.1/16
Pod 172.17.6.8
Pod 172.17.2.11
GW 172.17.0.1/16
172.18/16 -> 10.3.0.1172.16.16 -> 10.1.0.1
Host 3
VM 1: 10.3.1.22
G/W: 10.3.0.1/16
Pod 172.18.3.8
Pod 172.18.4.9
GW 172.18.0.1/16
172.16/16 -> 10.1.0.1172.17/16 -> 10.2.0.1
10.1/16 -> 192.168.0.1010.3/16 -> 192.168.0.12172.16/16 -> 192.168.0.10172.18/16 -> 192.168.0.12
10.1/16 -> 192.168.0.1010.2/16 -> 192.168.0.11172.16/16 -> 192.168.0.10172.17/16-> 192.168.0.11
WAN
OpenStack Summit Austinromana.io
Demo• OpenStack on four physical machines• Launch VMs on private 10/8 network• Kubernetes running on VMs
• Kubernetes Network 172.16/12• Container Network Interface (CNI) configuration of pods
• Romana IPAM allocates IPs for VMs and pods• Chosen specially to maintain static routes and CIDRs to each host
and VM• All IPs reachable by construction
April 2016 Slide 19
OpenStack Summit Austinromana.io
Thank You…• Network and Security Automation
• All details available at romana.io• Open source
• Apache 2.0• github.com/romana
• Release v0.8 available now• Integration with OpenStack and Kubernetes
April 2016 Slide 20