opennebulaconf 2016 - networking, nfvs and sdns hands-on workshop by rubén s. montero

Networking, NVFs & SDN Workshop

OpenNebulaConf 20164th edition

Agenda for the Workshop

● OpenNebula Network Stack

● Hacking the Network Drivers

● Virtual Routers

● IPAM Drivers

● Use Cases

○ OnLife: OpeNebula & ONOS for CORD

○ BEACON: SDN for Networks across Clouds

OpenNebula Network Stack


OpenNebula Network Model: View

Hyper. Hyper. Hyper. Hyper.

Virtual Network

VM VM VM VM

Datacenter Network

● Leaf-Spine switched or routed backbone

● Under-control of SDN

● Most deployments just few TOR switches

Virtual switches

Virtual Network

OpenNebula

SDN

OpenNebula Network Model: “built-in SDN”

Provide a central point to define, consume and set up Virtual Networks for Virtual Machines

XML-RPC API

OpenNebula daemon

Infrastructure Drivers

Northbound Interface● Virtual Network abstract management● Lease addresses to VMs● Reserve addresses for users or groups

Network

Southbound Interface● Uniform interface to interact with network

elements● Specialized for VM Networking

Northbound Interface

Logical AttributesAddresses Space (multiple, disjoint ranges)● IPv4● IPv6 (Global & ULA)● Ethernet - MAC addresses

Security Groups● Managed as a separate entity● Inbound & Outbound, TCP/UDP/ICMP

Custom Tags

Implementation & Physical Attributes● VNET driver* (e.g. bridged,vxlan)● Network specific (e.g. BRIDGE, VLAN_ID*)● VM Context (e.g. DNS, GATEWAY)

Virt

ual N

etw

ork

Def

initi

on

Virtual Network Tips

● Can be used as a logical SuperNet

○ Each AR can be a different entity (drivers, IPs, context…)

○ Example: Public Network as a set of AR’s

● Non VM leases can be put on hold or left out of the range

● Use reservations:

○ special purposes leases (e.g. routers, well-known services)

○ VDC or user assignment

○ Labels come in handy to organize VNETs

Southbound Interface: The Network Drivers

Three-phase setup● pre, post and clean right before, after VM boot and shutdown

● It considers live-migrations

● Includes update_sg to push Security Group changes

OpenvSwtich IPAM

Linux Bridge

802.1Q Bridged

Security Groupiptables rules

VXLAN ebtables Bridged VLAN dummy

NFV Virtual Router

Setup networking for VMs interfacing with network & hypervisors

Stock Network drivers

VLAN Driver Structure

def activate

lock

process do |nic|

get_vlan_dev_name #eth0.200

create_bridge #brctl addbr onebr.200

create_vlan_dev #ip link add ...

exec_and_log("#{command(:brctl)} addif"\

" #{nic[:bridge]} #{nic[:vlan_dev]}")

end

unlock

#!/usr/bin/env ruby template64 = ARGV[0] deploy_id = ARGV[1] hm = VXLANDriver.from_base64(...) exit hm.activate

The Drivers in Action: Example VXLAN

ens3 IP:10.4.4.11 (this IP is used to encapsulate VM traffic in a UDP datagram, can be in a VLAN)

ens3.100 vxlan encap. 239.0.0.100 mc group

ens3.200 vxlan encap. 239.0.0.200 mc group

onbr.100 (bridge)

one-<vm_id>-<nic_id>

Infrastructure set in pre operation, removed in clean

Predictable interface namesVM

switching/routing fabric:● multicast● MTU

Virtual Routers


Virtual Routers (new in 5.0)

Virtual Router

Virtual Router

Virtual Router

VR cluster

Virtual Network Virtual Network

● Alpine Linux

● Link any VLAN network

● HA configuration

● Support for reconfiguration

● First class management entity● Sunstone specialized views● CLI for vrouters management● Can be configured per VDC

The Virtual Routers in Action

● Approach: Virtualize some network functions by packaging into an appliance

● Performance tuning

○ Hugepages*,

○ NUMA scheduling*,

○ PCI PF/SR-IOV co-allocation (fully supported in 5.0.2)

* Supported only to some extent in 4.14

● Other common pitfalls

○ Network MTU

○ virtio and segmentation offloading

NFV Implementation Tips

Security Groups


Security Groups● Defines a set of rules to accept specific traffic to/from a VM

● Rule

● SG’s can be applied to Virtual Networks or Address Ranges● DEFAULT SG allows all traffic!● A SG update will be propagated to running VMs (new 5.0.2)● New SG cannot be added to VM NICs (detach/attach)

Traffic Type

● Protocol (TCP, UDP, IPSEC, ICMP)● Port or ICMP type

Traffic Origin/Destination

● INBOUND/OUTBOUND● IP (or IP set)● OpenNebula Virtual Netwok

Security Groups● Implemented using iptables kernel facilities & ipset

● Only for Linux bridge based virtual networks

Match outbound traffic for VM (at bridge port)

Match inbound traffic for VM (at bridge port)

opennebula chain

inbound chainMatch protocol, port, src or dst (RETURN)

ACCEPT (by default)

DROP (by default)

Match bridged packets

Security Groups in Action

IP Address Management


IP Address Management

● Awesome contribution by

Christophe Palanché & Guillaume Oberlé

● Delegate IP management to external component per AR

● IPAM options in 5.2:

○ Built-in module for address management

○ Develop custom one based on provided template

○ Addons are welcome!

IP Address Management: Network

allocate_address get_address free_addressIPAM driver

AR definition with as many details as neededAR=[

IPAM_MAD="dummy", TYPE="IP4", SIZE="20",

IP="10.0.0.2", GATEWAY="10.0.0.1",

DNS="10.0.0.1", ANY_IPAM_ATTR="doamin.com"

]

register_address_range

AR = [ IPAM_MAD="dummy", TYPE="IP4", SIZE="20" ]

IP Address Management: Addresses

register_address_range allocate_address get_address free_addressIPAM driver

Return ADDRESS requested by VM or ReservationADDRESS = [ IP = "10.0.0.10", SIZE = "5" ]

ADDRESS = [ SIZE = "5" ] (IP optional for allocate)

IPAM Drivers: Integration Sample

Guillaume Oberlé

OnLife: OpenNebula & ONOS for COORD


OnLife: OpenNebula & ONOS for CORD

Central Office Re-architected as Datacenter

● NFVs● SDN● Cloud● Real-time configuration of new customer

services


VPdC


VPdC VPdCVPdC

VPdC VPdC VPdC

VPdC

VPdC

VPdC

VPdC


one-onos-driver

├── clean├── clean.d│ └── onos.sh├── onos.conf├── post├── post.d├── pre└── pre.d └── onos.sh


onos.conf

ONOS_URL=http://<ip>:8181/onos

ONOS_USER=<user>ONOS_PASSWORD=<password>

OLTAPP_URL=$ONOS_URL/ctpd-olt-app/oltappCLOSFWD_URL=$ONOS_URL/closfwd-app/closfwdapp


pre.d/onos.sh (194 lines)

1. OLTAPP (olt_dev, olt_port, bandwidth) ⇒ VLAN2. Update VM template with VLAN through onegate3. Create vPDC bridge4. Tag veth (veth_clos ↔ veth_vpdc)5. Add tagged veth to bridge6. CLOSFWD. args: vlan_id7. CLOSFWD_MULTI Internet. args: vlan_id, mac8. CLOSFWD_MULTI Other services. args: vlan_id, mac


# Get VLAN ID

URL="$OLTAPP_URL/$OLN_OLT_DEV/$OLN_OLT_PORT/$NO_VLAN_TAG/$OLN_B

ANDWIDTH"

RESPONSE=$(curl -s --user $ONOS_USER:$ONOS_PASSWORD -X POST -H

"Content-Type: application/json" "$URL")

VLAN_ID=$(echo "$RESPONSE" | tr , '\n' | grep vlanOut | tr -dc

0-9)


# create bridge

sudo brctl addbr $BR_VPDC

sudo ip link set dev $BR_VPDC up

# tag vpdc side of veth

sudo ip link add link veth_vpdc veth_vpdc.$VLAN_ID type vlan id

$VLAN_ID

sudo ip link set dev veth_vpdc.$VLAN_ID up

# add ifaces to bridges

sudo brctl addif $BR_VPDC veth_vpdc.$VLAN_ID


closfwd() {

SRC_DEV="$1"; DST_DEV="$2"; SRC_PORT="$3"; DST_PORT="$4";

VLAN="$5"; MAC="$6"

URL="$CLOSFWD_URL/$SRC_DEV/$DST_DEV/$SRC_PORT/$DST_PORT/$VLAN/$

MAC"

curl -s --user $ONOS_USER:$ONOS_PASSWORD -X POST -H

"Content-Type: application/json" "$URL"

}

# OLT - L2

closfwd "$OLN_CLOS_IN_DEV" \

"$OLN_VPDC_IN_DEV" \

"$OLN_CLOS_IN_PORT" \

"$OLN_VPDC_IN_PORT" \

"$VLAN_ID" \

"$ANY_MAC"


closfwd_multi() {

# regular fwd

closfwd "$SRC_DEV" "$DST_DEV"...

# multicast fwd

MAC=$(echo "$MAC" | awk -F ":" '{print

"33:33:ff:"$4":"$5":"$6}')

closfwd "$SRC_DEV" ...

}

# L3 - VR - INTERNET

closfwd_multi "$OLN_VPDC_OUT_DEV" \

"$OLN_CLOS_OUT_DEV" \

"$OLN_VPDC_OUT_PORT" \

"$OLN_CLOS_OUT_PORT" \

"$OLN_CLOS_OUT_VLAN_INTERNET" \

"$OLN_VPDC_OUT_MAC_INTERNET"


pre.d/clean.sh (131 lines)

1. Delete veth2. OLTAPP: DELETE. args: port3. CLOSFWD_DEL. args: vlan_id4. CLOSFWD_MULTI_DEL Internet. args: vlan_id, mac5. CLOSFWD_MULTI_DEL Other services. args:

vlan_id, mac

BEACON: SDN for Networks across Clouds


BEACON: Project Architecture

BEACON: Federated SDN

Virtual RouterVirtual Router

VM

Virtual Network (VLAN)Virtual Network (VLAN)

VM VM

Cross-siteNetwork

VM

Ethernet over IP (GRETAP) + IPSec

● Link multiple virtual networks (L3/L2)● Support for hybrid configurations● Support for L3 inter-DC

BEACON: Federated Cloud Networks

OpenNebula

KVM HostEU

EC2US EC2 AWS

us_east

Demo: federated network overlay between local OpenNebula and EC2 VPC (Virtual Private Cloud)


KVM HostEU

EC2US

InternetPrivatebridge

Publicbridge

VM VM FA

192.168.0.x

VM VMFA

AWS VPC

10.0.1.x147.96.81.134 52.71.148.30


KVM HostEU

EC2US

InternetPrivatebridge

Publicbridge

VM VM

192.168.0.x

VM VM

AWS VPC

10.0.1.x147.96.81.134 52.71.148.30

GRE tunnels

192.168.0.xNetwork overlay

FAFA


Platinum

Gold

Silver

Community

THANKS!

opennebulaconf 2016 - networking, nfvs and sdns hands-on workshop by rubén s. montero

Technology