CGNAT on VSM in 5.1.1

What is VSM?

Virtualized Services Module(VSM) is virtualized platform in ASR9K to host multiple Service applications.

This document will be focusing on CGN/CGNv6(NAT44) as an example .

VSM Architecture

FabricASIC 0

IvyBridge

BACKPLANE

32GBDDR3

48ports10GE

Application Processor Module (APM) Service Infra Module (SIM)

TyphoonNPU

FabricASIC 1

TyphoonNPU

XAUI

Niantic

Niantic

Niantic

Niantic

Niantic

Niantic

Niantic

PCIe

IvyBridge

IvyBridge

IvyBridge

32GBDDR3

32GBDDR3

32GBDDR3

QuadPHY

SFP+SFP+SFP+SFP+

Crypto/DPIAssist

Crypto/DPIAssist

Crypto/DPIAssist

Crypto/DPIAssistCan be

used for FCOE; not

enabled for 5.1.1 FCS

Intel Cavecreek

chipset48-Port Niantic switch

Intel x86 Ivy Bridge CPU1 Intel CPU with 10 coresTotal of 4 CPU with 40 Cores.With Intel Hyper-threading technology total of 80 cores for 4CPU; 20 cores per CPU can be achieved.

VSM Hardware

• Intel x86 Ivy Bridge CPU• 1 Intel CPU with 10 cores• Total of 4 CPU with 40 Cores.• With Intel Hyper-threading technology total of 80 cores for

4CPU; 20 cores per CPU can be achieved.• Intel Cavecreek Chipset provides Crypto/DPI assist

functionality.

Virtualized Software Infrastructure

KVM hypervisor runs on Linux.Multiple Service Applications can be hosted.Service chaining of applications can be achieved in two ways:1) Via static route 2) Via OnePK

Interface Terminologies

a) SVI Infra (identified by ‘interface ServiceInfra’) –used to send SVI and CGv6 related control/mgmt traffic between XR and Linux side b) SVI App (identified by ‘interface ServiceApp’) –used to send CGv6 data traffic to/from CGv6 applications.

Service Instantiation and Configuration

Installing the CGv6 ova package

Step1 : install 5.1.1 IOS-XR image along with services.pie and services-infra.pie.Step 2: copy the cgn.ova file to RSP (eg: disk0:)

Step 3 : Enable virtual-serviceRP/0/RP0/CPU0:Starscream-UI-va(config)#virtual-service enable RP/0/RP0/CPU0:Starscream-UI-va(config)#

Step 4: Install CGN VM , 0/3/CPU0 is location of VSM card.RP/0/RP0/CPU0:Starscream-UI-va#virtual-service install name cgn123 package disk0:vsmcgv6_ivybridge.ova node 0/3/CPU0

CGv6 Installation statusStep 5: Status of Installation

RP/0/RP0/CPU0:Starscream-UI-va#sh virtual-service list Virtual Service List:

Name Status Package Name Node Name ______________________________________________________________________________cgn123 Installing vsmcgv6_ivybridge.ova 0/3/CPU0

RP/0/RP0/CPU0:Starscream-UI-va#sh virtual-service list

Virtual Service List:

Name Status Package Name Node Name ______________________________________________________________________________cgn123 Installed vsmcgv6_ivybridge.ova 0/3/CPU0 RP/0/RP0/CPU0:Starscream-UI-va#

CGv6 VM activateStep 6: Configure CGv6 VM

RP/0/RP0/CPU0:Starscream-UI-va(config)#virtual-service cgn123RP/0/RP0/CPU0:Starscream-UI-va(config-virt-service)# vnic interface TenGigE0/3$RP/0/RP0/CPU0:Starscream-UI-va(config-virt-service)# vnic interface TenGigE0/3$RP/0/RP0/CPU0:Starscream-UI-va(config-virt-service)# vnic interface TenGigE0/3$RP/0/RP0/CPU0:Starscream-UI-va(config-virt-service)# vnic interface TenGigE0/3$RP/0/RP0/CPU0:Starscream-UI-va(config-virt-service)# vnic interface TenGigE0/3$RP/0/RP0/CPU0:Starscream-UI-va(config-virt-service)# vnic interface TenGigE0/3$RP/0/RP0/CPU0:Starscream-UI-va(config-virt-service)# vnic interface TenGigE0/3$RP/0/RP0/CPU0:Starscream-UI-va(config-virt-service)# vnic interface TenGigE0/3$RP/0/RP0/CPU0:Starscream-UI-va(config-virt-service)# vnic interface TenGigE0/3$RP/0/RP0/CPU0:Starscream-UI-va(config-virt-service)# vnic interface TenGigE0/3$RP/0/RP0/CPU0:Starscream-UI-va(config-virt-service)# vnic interface TenGigE0/3$RP/0/RP0/CPU0:Starscream-UI-va(config-virt-service)# vnic interface TenGigE0/3$RP/0/RP0/CPU0:Starscream-UI-va(config-virt-service)#commitRP/0/RP0/CPU0:Starscream-UI-va(config-virt-service)#activate RP/0/RP0/CPU0:Starscream-UI-va(config-virt-service)#commit

Step 7: Check the status of the CGv6 VMRP/0/RP0/CPU0:Starscream-UI-va#sh virtual-service list Virtual Service List:

Name Status Package Name Node Name ______________________________________________________________________________cgn123 Activated vsmcgv6_ivybridge.ova 0/3/CPU0 RP/0/RP0/CPU0:Starscream-UI-va#

VSM-NAT44Basic Configuration Steps

VSMIngress LC Egress LC

Int Gige 0/6/1/13VRF: Nat-insideIPv4:31.1.1.1/24

int ServiceApp 1VRF: Nat-insideIPv4: 14.1.1.1/24Service-Type: cgn123/nat44

int ServiceApp 2[VRF: Nat-outside]IPv4: 15.1.1.1/24Service-Type: cgn123/nat44

Int Gige 0/6/1/14[VRF: Nat-outside]IPv4:41.1.1.1/24

VRF “Nat-inside” VRF “Nat-outside”CGN “cgn123/nat44”Public IPv4 Pool (Nat-inside to Nat-outside): 100.2.0.0/24

router static vrf Nat-inside address-family ipv4 unicast 0.0.0.0/0 ServiceApp1

router static [vrf Nat-outside]address-family ipv4 unicast 100.2.0.0/24 ServiceApp2

Install asr9k-services-p.pieInstall asr9k-services-infra.pie Int ServiceInfra1

IPv4:200.1.1.1/24

Getting started for CGv6/CGNAT

• Sample Ingress/Egress LC configuration:

vrf Nat-inside address-family ipv4 unicast

interface GigabitEthernet0/6/1/13.100 vrf Nat-inside ipv4 address 31.1.1.1 255.255.255.0 load-interval 30 encapsulation dot1q 100

vrf Nat-outside address-family ipv4 unicast

interface GigabitEthernet0/6/1/14.100 vrf Nat-outside ipv4 address 41.1.1.1 255.255.255.0 load-interval 30 encapsulation dot1q 100

Service CGN and service-type******** CGN instance *******service cgn cgn123 service-location preferred-active 0/3/CPU0

*****CGNAT service-type ****** service-type nat44 nat123 portlimit 65535

inside-vrf Nat-inside map outside-vrf Nat-outside address-pool 100.2.0.0/24 !protocol udp session initial timeout 65535 session active timeout 65535 ! protocol tcp session initial timeout 65535 session active timeout 65535 !

Service interfaces

interface ServiceInfra1 ipv4 address 75.1.1.1 255.255.255.0 service-location 0/3/CPU0

ServiceApp interfaces per vrf along with service cgn and service-type.

interface ServiceApp1 vrf Nat-inside ipv4 address 14.1.1.1 255.255.255.0 service cgn cgn123 service-type nat44

interface ServiceApp2 vrf Nat-outside ipv4 address 15.1.1.1 255.255.255.0 service cgn cgn123 service-type nat44

Static routes

Static route for Inside-to-outside; Redirect all traffic to Inside ServiceApp interface:

vrf Nat-insideaddress-family ipv4 unicast 0.0.0.0/0 ServiceApp1

Static route for Outside-to-inside traffic; IP address should match Public pool configured under service cgn:

vrf Nat-outside address-family ipv4 unicast 100.2.0.0/24 ServiceApp2

VSM-NAT44

VSMIngress LC Egress LC

Src:31.1.1.2:1000Dest: 41.1.1.2:1000

int ServiceApp 1VRF: Nat-insideIPv4: 14.1.1.1/24Service-Type: cgn123/nat44

Int Gige 0/6/1/14[VRF: Nat-outside]IPv4:41.1.1.1/24

VRF “Nat-inside” VRF “Nat-outside”

CGN “cgn123/nat44”Public IPv4 Pool (Nat-inside to Nat-outside): 100.2.0.0/24Nat entry created:31.1.1.2:1000 | 100.2.0.52:1000

Int ServiceInfra1IPv4:200.1.1.1/24

Inside to outside translationsh cgn nat44 nat123 inside-translation protocol udp inside-vrf Nat-inside inside-address 31.1.1.2

port start 1 end 65535

Fib lookup happens and traffic passes to the outside-vrf on

the Egress LC

Src:100.2.0.52:1000Dest:41.1.1.2:1000

Inside-to-Outside Packet flow

1) Inside vrf is connected to a traffic Generator2) Packet enters from private Inside VRF to the ingress Linecard. 3) Static route from inside vrf redirects all traffic to ServiceApp1 on VSM.4) CGNAT application does the NAT processing for the packet and assigns a public IP address

from the public pool creating a NAT entry.5) After the Nat translation forwarding lookup will be done for destination address in the

outside vrf and packet is sent to the Egress LC interface.6) Egress line card send the packet to the Public side connected to another traffic generator.

VSM-NAT44Outside to Inside translation

VSMIngress LC Egress LC

Int Gige 0/6/1/13VRF: Nat-insideIPv4:31.1.1.1/24

int ServiceApp 2[VRF: Nat-outside]IPv4: 15.1.1.1/24Service-Type: cgn123/nat44

Int Gige 0/6/1/14[VRF: Nat-outside]IPv4:41.1.1.1/24

VRF “Nat-inside” VRF “Nat-outside”CGN “cgn123/nat44”Public IPv4 Pool (Nat-inside to Nat-outside): 100.2.0.0/24

Src: 41.1.1.2:1000Dest:100.2.0.52:1000

sh cgn nat44 nat123 outside-translation protocol udp outside-vrf Nat-outside outside-address 100.2.0.52

port start 1 end 65535

Fib lookup happens and traffic passes to the inside-vrf on the

Egress LC

Outside to Inside Packet flow (reverse-nat)

1) Packet enters from Outside vrf - Public side 2) Based on Static route defined packet should be forwarded to the VSM card via the

ServiceApp2 in the outside-vrf.3) CGNAT application does the Nat processing and looks for corresponding NAT entry if present.

If not it drops the packet. If the entry is present then it replaces destination ip and port with the corresponding Private IP address.

4) After the Reverse Nat translation forwarding lookup will be done for the destination IP address in the inside vrf and packet is sent to the Egress LC interface

5) Egress line card send the packet out to the Private side/ inside vrf.

Caveats in 5.1.1

VSM on Cluster is not supported

Commit replace and rollback: i) Commit replace does not have this restriction but its safer to deactivate Virtual- services in all cases. ii) Rollback:Virtual-services need to be deactivated before doing config rollback.

IP address configuration is not supported on the Tengig interfaces of the VSM LC.

4 Front Panel SFP+ ports are not enabled and cannot be used.

CGNAT Show commandsInside-to-outside translation:

sh cgn nat44 nat123 inside-translation protocol udp inside-vrf Nat-inside-101 inside-address 32.1.1.2 port start 1 end 65535

RP/0/RP1/CPU0:Starscream-UI-va#sh cgn nat44 nat123 inside-translation protocol$Inside-translation details---------------------------NAT44 instance : nat123Inside-VRF : Nat-inside-101-------------------------------------------------------------------------------------------- Outside Protocol Inside Outside Translation Inside Outside Address Source Source Type to to Port Port Outside Inside Packets Packets-------------------------------------------------------------------------------------------- 101.2.0.58 udp 1000 34656 dynamic 1805831 1294025 RP/0/RP1/CPU0:Starscream-UI-va#

Outside-to-Inside Translation:

RP/0/RP0/CPU0:va#SH cgn nat44 nat123 outside-translation protocol udp outside-address 101.2.0.58 port start 1 end 65535Outside-translation details---------------------------NAT44 instance : nat123Outside-VRF : default-------------------------------------------------------------------------------------------- Inside Protocol Outside Inside Translation Inside Outside Address Destination Destination Type to to Port Port Outside Inside Packets Packets-------------------------------------------------------------------------------------------- 32.1.1.2 udp 34656 1000 dynamic 107491158 101560603 RP/0/RP0/CPU0:va#

Cef commandsRP/0/RP0/CPU0:va#sh cef vrf Nat-inside 31.1.1.2 location 0/3/CPU0 31.1.1.0/24, version 19, attached, connected, internal 0xc0000c1 0x0 (ptr 0x7c12a064) [1], 0x0 (0x7c071008), 0x0 (0x0) Updated Jan 22 15:17:43.521 remote adjacency to GigabitEthernet0/6/1/13.100 Prefix Len 24, traffic index 0, precedence n/a, priority 0 via GigabitEthernet0/6/1/13.100, 2 dependencies, weight 0, class 0 [flags 0x8] path-idx 0 NHID 0x0 [0x7e1624d8 0x0] remote adjacencyRP/0/RP0/CPU0:va#

RP/0/RP0/CPU0:va#sh cef vrf Nat-outside 101.2.0.58 location 0/3/CPU0 0.0.0.0/0, version 0, proxy default, default route handler, drop adjacency, internal 0x4002021 0x0 (ptr 0x7c1241e4) [1], 0x0 (0x7c066290), 0x0 (0x0) Updated Jan 22 15:17:24.341 Prefix Len 0, traffic index 0, precedence n/a, priority 0 via point2point, 144 dependencies, weight 0, class 0 [flags 0x0] path-idx 0 NHID 0x0 [0x7bacf23c 0x0] next hop point2point drop adjacencyRP/0/RP0/CPU0:va#

CGNAT Statistics summaryRP/0/RP0/CPU0:va#sh cgn nat44 nat123 statistics

Statistics summary of NAT44 instance: 'nat123'Number of active translations: 14Number of sessions: 100Translations create rate: 0Translations delete rate: 0Inside to outside forward rate: 67875Outside to inside forward rate: 8539Inside to outside drops port limit exceeded: 0Inside to outside drops system limit reached: 0Inside to outside drops resource depletion: 0No translation entry drops: 13PPTP active tunnels: 0PPTP active channels: 0PPTP ctrl message drops: 0Number of subscribers: 14Drops due to session db limit exceeded: 0Drops due to source ip not configured: 0

Pool address totally free: 498Pool address used: 14Pool address usage:------------------------------------------------- External Address Ports Used ------------------------------------------------- 200.2.0.48 1 200.2.0.49 1 200.2.0.50 1 200.2.0.51 1 200.2.0.53 1 200.2.0.56 1