aci fundamentals lab guide 2 -...

1

ACI Fundamentals Lab Guide 2.2 March 10

th 2016

Ivan Andjelkovic

Systems Engineer

[email protected]

2

Foreword

Welcome to the Application Centric Infrastructure (ACI) Fundamentals Lab.

This lab is created for Field Engineers, System Engineers, Architects and others who would like

to get familiar with fundamental ACI concepts and techniques using a hands-on approach.

For scaling purposes, some of the ACI setup steps are done ahead of time. All of those steps are

documented in the additional ACI Lab Setup and Connectivity document.

Combined with this ACI Fundamentals Lab Guide, these two documents contain information

needed to set up a new ACI system.

For students that would like to go through the ACI setup steps, they can do so using the

simulator and following the document Optional ACI Simulator Lab.

Special thanks to the Business Technology Architects (BTA) whose original lab was the

foundation for this lab.

3

Foreword .......................................................................................................................................... 2 Revision Notes ................................................................................................................................. 4 Introduction ...................................................................................................................................... 5 Lab Format and Objectives .............................................................................................................. 7 Lab Topology and Lab Preparation .................................................................................................. 8 Lab Access ....................................................................................................................................... 9

Module 1: Student Pre-Lab setup verification ..................................................... 13

Task 1: APIC GUI Overview .......................................................................................................... 13 Task 2: Verify APICs, ACI Fabric and OOB Management ............................................................. 17 Task 3: Verify vPC connectivity to the UCS system ...................................................................... 19

Module 2: Using API Inspector and Postman ...................................................... 23

Task 1: Explore API Inspector and Postman ................................................................................. 24 Task 2: Create and Delete ACI User using GUI ............................................................................ 26 Task 3: Capture JSON script using the API Inspector ................................................................... 27 Task 4: Create and Delete ACI User using Postman .................................................................... 29

Module 3: Building ACI Forwarding Constructs ................................................. 32

Task 1: Create a Tenant ................................................................................................................ 33 Task 2: Create a VRF and a Bridge Domain (BD) for Web Servers .............................................. 35 Task 3: Create a Bridge Domain (BD) for App Servers ................................................................. 37

Module 4: Configuring Application Profile .......................................................... 41

Task 1: Create Application Network Profile (ANP) ........................................................................ 43

Module 5: Configuring VMM Integration .............................................................. 48

Task 1: Configure VMM Integration ............................................................................................... 49 Task 2: Add VMs to the port group from the VMware vCenter ...................................................... 58 Task 3: Verify and Explore ACI Contracts ..................................................................................... 66

Module 6: Configuring Ext. Layer 2 Connectivity ............................................... 71

Task 1: Create Access Policy to Nexus Switch ............................................................................. 73 Task 2: Enable and Verify App Traffic with the External L2 ........................................................... 79

Module 7: Connecting Ext. Layer 3 Connectivity ................................................ 84

Task 1: Create BGP Route Reflector ............................................................................................. 84 Task 2: Create Layer 3 External Connectivity................................................................................ 85 Task 3: Enable App Traffic with the External L3 ............................................................................ 90 Task 4: Verify External L3 Connectivity ......................................................................................... 95

Appendix A: Troubleshooting ............................................................................... 98

T1. Host and vCenter credentials issue ......................................................................................... 98

4

Revision Notes

The initial version 1.2 of the lab guide was done based on the ACI version 1.0.

The version 2.0 of the lab guide is based on the next major ACI version 1.1.

This version 2.2 of the lab guide is based on the latest major ACI release 1.2.

The screenshots are updated to reflect changes in the new ACI version.

The new comments in this revision are mostly related with the differences between the ACI

versions. All new comments are done using the blue font throughout the document.

The main content of the lab is unchanged.

The details around the new software and hardware features with this major 1.2 release can be

found under the release notes.

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/1-x/release/notes/apic_rn_121i.html

5

Introduction

“Cisco Application Centric Infrastructure (ACI) is a data center fabric that enables you to

integrate virtual and physical workloads in a highly programmable multi-hypervisor

environment that is designed for any multi-service or cloud data center.

The ACI fabric consists of discrete components that operate as routers and switches (Leaves and

Spines) but are provisioned and monitored as a single entity. The operation is like a distributed

switch and router configuration that provides advanced traffic optimization, security, and

telemetry functions, stitching together virtual and physical workloads.

The controller, called the Application Policy Infrastructure Controller (APIC), is the central

point of management of the fabric. That is the device that distributes Application Network Profile

(ANP) and other policies to the devices that are part of the fabric.”

Quote from “The Policy Driven Data Center with ACI” by Lucien Avramov, et al.

At the top level, the ACI object model is built on a group of one or more tenants, allowing the

network infrastructure administration and data flows to be segregated. Tenants can be used for

customers, business units, or groups, depending on organizational needs. For instance, an

enterprise may use one tenant for the entire organization, and a cloud provider may have

customers that use one or more tenants to represent their organizations.

Tenants can be further divided into contexts, which directly relate to Virtual Routing and

Forwarding (VRF) instances, or separate IP spaces. Each tenant can have one or more contexts,

depending on the business needs of that tenant. Contexts provide a way to further separate the

organizational and forwarding requirements for a given tenant. Because contexts use separate

forwarding instances, IP addressing can be duplicated in separate contexts for multitenancy.

Within the context, the model provides a series of objects that define the application. These

objects are endpoints (EP) and endpoint groups (EPGs) and the policies that define their

relationship (Figure 2). Note that policies in this case are more than just a set of access control

lists (ACLs) and include a collection of inbound and outbound filters, traffic quality settings,

marking rules, and redirection rules.

The figure above shows a tenant with two contexts and the applications that make up those

contexts. The EPGs shown are groups of endpoints that make up an application tier or other

logical application grouping. For example, Application B, shown expanded on the right side of

6

the figure, may consist of a web tier (blue), application tier (red), and database tier (orange). The

combination of EPGs and the policies that define their interaction is an Application Network

Profile in the ACI model. More details can be found here.

Industry shifts are redefining IT at all levels. On-premise IT consumption models are shifting to

cloud-based services. Infrastructure as a service (IaaS) is being supplanted by applications as a

service. Separate development and operations teams are moving toward integrated development

and operations (DevOps) groups. Box-centric management models are migrating to application-

centric management models.

Business agility requires application agility, so IT teams need to provision applications in hours

instead of months. Resources need to scale up (or down) in minutes, not hours.

ACI enables IT departments to meet those needs through:

Simplified automation by an application-driven policy model

Centralized visibility with real-time, application health monitoring

Open software flexibility for DevOps teams and ecosystem partner integration

Scalable performance and multi-tenancy in hardware

The basic components of ACI are:

The Nexus 9000 series of switches forming Leaf-Spine (Closs) Fabric

A centralized policy management using the Application Policy Infrastructure Controller

(APIC)

An Application Virtual Switch (AVS) for the virtual network edge

Software and hardware innovations

Integrated physical and virtual infrastructure components

An open ecosystem of network, storage, management, and orchestration vendors

The future of networking with ACI is about providing a network that is deployed, monitored, and

managed in a fashion that supports DevOps and a consistently changing application

environment. ACI does so through the reduction of complexity and a common policy framework

that can automate provisioning and managing of resources.

http://www.cisco.com/en/US/prod/collateral/netmgtsw/ps13004/ps13460/white-paper-c11-729906_ns1261_Networking_Solutions_White_Paper.html

7

Lab Format and Objectives

This is a SELF-PACED lab. The following table describes the objectives of the different

sections found in this document. Students are absolutely welcome to experiment and explore the

various product menus and options beyond what is covered in this guide with the understanding

that the ACI fabric is a shared resource among the students. Given the use of shared

infrastructure, it is important to further note that your work can impact others.

All steps in the Modules 3-7 (except Task 2 in Module 5) have a json script prepared. If in the interest of time you would like to avoid configuring certain steps, you can execute a json script using Postman. Details about Postman can be found in Module 2.

Each Module starts with a description of the objectives, what will be covered, and additional

relevant information.

For students that would like to go through steps that could not be part of the main lab due to

scaling issues (APIC Setup, ACI fabric discovery and vPC to UCS connectivity), there is an

optional lab that can be done on the simulator by following the Optional ACI Simulator Lab

document.

Module Objectives Duration

Module 1 Familiarize yourself with the APIC GUI options and verify that the Pre-Lab

configuration has been completed successfully. 20 minutes

Module 2 Learn about API Inspector and Postman. 30 minutes

Module 3 Build ACI Forwarding constructs such as Tenants, VRFs (Private Networks)

and Bridge Domains 25 minutes

Module 4 Configure a 2-Tier Application Network Profile (ANP) 25 minutes

Module 5 Integrate APIC with VMware vCenter and attach Virtual Machines to the

designated Endpoint Groups. 50 minutes

Module 6 Connect the ACI Fabric with an existing External Layer 2 network 40 minutes

Module 7 Connect the ACI Fabric with an existing external Layer 3 Network 35 minutes

8

Lab Topology and Lab Preparation

The overall lab topology is represented in the figure below:

Pay close attention to the simplified logical cabling setup between devices. The UCS system and

the Nexus switches are preconfigured following FlexPod with vSphere 5.5 and ACI Design

Guide and are not the focus of today’s lab.

Additional documents

- For Hardware installation:

“Application Centric Infrastructure Fabric Hardware Installation Guide”

- For first-time access, APIC setup, Fabric Initialization, Out-Of-Band (OOB)

Management access and more: “Cisco APIC Getting Started Guide”:

- For the details regarding the configuration done on the APIC before this lab, consult the

ACI Lab Setup and Connectivity document associated with this lab.

http://www.cisco.com/c/dam/en/us/td/docs/unified_computing/ucs/UCS_CVDs/flexpod_esxi55u1_n9k_aci.pdf

http://www.cisco.com/c/dam/en/us/td/docs/unified_computing/ucs/UCS_CVDs/flexpod_esxi55u1_n9k_aci.pdf

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus9000/hw/aci_hig/b_aci_hardware_install_guide.html

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/1-x/getting-started/b_APIC_Getting_Started_Guide/b_APIC_Getting_Started_Guide_chapter_01.html

9

Lab Access

This section describes access to the lab environment.

There are 2 ways to access your jumpstation:

- Using VPN AnyConnect and RDP client (recommended for the best performance)

- Using Clientless access via browser (if you do not have VPN AnyConnect or RDP client)

Note: If you have any problems connecting to the lab or experience performance issues, please contact your Lab proctor for assistance.

Access using VPN AnyConnect:

Open your VPN AnyConnect client, put dctraining.cisco.com/aci-student as a 1.

destination and click Connect.

On the security warning page click Connect Anyway. 2.

Enter ACIstudentX for the username and use the password supplied by your instructor at 3.

the beginning of the lab (Note each pod has a unique username). Click OK.

Note: X is your double digit pod number, for example pod 8 will use ACIstudent08

Click Accept at the welcoming screen. 4.

Note: Now you have access to your jumpstation using Remote Desktop Connection client.

Open your RDP client. For Windows machines type mstsc in the windows search field. 5.

10

Type 192.168.199.(100 + X) where X is your pod number and click Connect. 6.

Note: For example, pod 7 will have ip address 192.168.199.107

Click Connect when asked “So you trust this remote connection”. 7.

Use username acistudent and password 1234Qwer to access your Windows jumpstation. 8.

Click OK.

Note: IP address below belongs to pod 12.

Click Yes if asked regarding the identity of the remote computer. 9.

You are now connected to your jump station. Skip the rest of this section that describes how to

access that jump station using clientless browser based access.

11

Clientless Access using your any browser:

Open your web browser and access https://dctraining.cisco.com/aci-student 10.

If the warning below (or similar, it is browser dependent) is displayed, click I 11.

Understand the Risks to proceed and accept any windows that follow.

Enter ACIstudentX for the username and use the password supplied by your instructor 12.

at the beginning of the lab (Note each pod has a unique username).

Note: X is your double digit pod number, for example pod 8 will use ACIstudent08

Click Continue. 13.

Click on the Jumpbox VNC bookmark to access your Windows Workstation via VNC. 14.

Note: VNC has the best performance, and the screen resolution can be adjusted. It is important to point out that VNC does not support Copy&Paste functionality from within VNC to your personal workspace.

If you want to Copy&Paste details from the remote session to your computer, you can click on the RDP bookmarks. Unfortunately, the resolution for RDP sessions is not modifiable.

https://dctraining.cisco.com/aci-student

12

Click Continue at Security Warning pop-up. 15.

Check I accept the risk and click Run. 16.

Note: If this window does not pop up, it might be due to Java security settings. Read the next step.

If you are asked for various Java approvals, please approve them all. 17.

Note: You might have to go to Configure Java and Security Tab and add https://dctraining.cisco.com to the list of the approved sites.

Afterward, you will have to close all your browser windows for this change to take effect, and start the access process again.

Contact instructor for details.

Select user ACIStudent and password 1234Qwer for your Windows login credentials. 18.

You are connected to your jump station and ready to proceed with the lab.

13

Module 1: Student Pre-Lab setup verification

Note: This Module can be skipped and done later without affecting the rest of the lab.

To be able to scale beyond one student per lab, the following tasks were performed ahead of

time:

1) Initial Setup of the APICs

2) ACI Fabric Initialization and Discovery

3) Configuration of Out of Band (OOB) Management of ACI Nodes

4) vPC connectivity between ACI Leaves and the UCS System

Detailed steps can be found in the ACI Lab Setup and Connectivity document.

In this Module, we will get familiar with ACI GUI options while verifying that the pre-work has

been done properly.

Task 1: APIC GUI Overview

In this task, you will review top level options in the ACI GUI. This will be the start of a high

level overview. During the lab we will go into most options in much greater detail.

Note: Most of the GUI options either contain or start with a Quick Start screen by default. You can watch how-to videos, read configuration tutorials or start configuration wizards from the Quick Start. Wizards are intended to make configurations easier. We will avoid using them as a part of this lab to enable a deeper understanding of what we are configuring.

From the Windows jumpstation, open the Chrome Browser 1.

Go to APIC1 address https://192.168.199.1 2.

If you encounter a Certificate warning, click on Advanced and then click on Proceed to 3.

192.168.199.1 (unsafe).

https://192.168.199.1/

14

Login with the User ID of admin and password of 1234Qwer. Chose Advanced Mode. 4.

Note: As of version 1.2, there is an option to choose the Basic Mode. The Basic Mode allows us to do the most common tasks in the simpler way but limits us in what we can do in that mode. Due to the limitations, we will be using the Advanced Mode in this lab.

Click on the welcome, admin section of the GUI on the far right of the screen. 5.

Familiarize yourself with the options that can be done with this pull down menu. This 6.

includes all of the AAA options, as well as API Inspector and API Documentation.

Note: We have an entire detailed module dedicated to API Inspector.

Next click on the search Icon. 7.

15

This is a tool that can be used to search for items in the GUI based on Category as shown 8.

below.

Next click on the L4-L7 Services. 9.

Notice Inventory and Packages sub items. We first import the L4-L7 device package 10.

(from Cisco, F5, Citrix or other 3rd

party vendors) using the Packages submenu, and then

we create that device in the Inventory menu.

Next click on the VM Networking menu item. 11.

Note: In ACI versions prior to 1.2 there were submenus Inventory and Policies. As of version 1.2, all options are under a single submenu Inventory.

The Inventory menu displays the VMs, hypervisors, and virtual switches belonging to the fabric.

This menu also provides VM statistics including packet counters, byte counters, CPU usage, and

memory usage.

As of ACI version 1.2, we will use Inventory menu to do what was in the prior versions done

under the Policies menu. We will configure connectivity policies for virtual machines managed

by Virtual Machine Manager (VMM) tools such as VMware (vCenter, vShield), Microsoft

(SCVMM) or OpenStack.

Next click on the Fabric menu item. 12.

16

Familiarize yourself with the options. 13.

Note: Later in this Module we will use Inventory options extensively to verify configurations already existing on the system.

The Fabric Policies section allows policy creation and modification for interfaces that connect

spine and leaf switches. Fabric policies can enable features such as monitoring (statistics

collection and statistics export), troubleshooting (on-demand diagnostics and SPAN), or NTP. It

also includes internal connectivity policies.

The Access Policies section allows policy creation and modification for access-edge (external-

facing) interfaces that do not connect to a spine switch. External-facing interfaces connect to

external devices such as virtual machine controllers, hypervisors, hosts, routers, or fabric

extenders (FEX). Access policies provide configuration policy for individual ports, port channels

and virtual port channels, as well as protocols such as LLDP, CDP or LACP, and features like

monitoring or diagnostics.

Next click on the Tenants menu item. 14.

A Tenant is a logical container or a folder for application policies. This container can represent

an actual tenant, an organization, security zone, application or a domain. A Tenant can also just

be used for organizing information in a way that is convenient. Overall, a tenant represents a

unit of isolation from a policy perspective.

Notice three Tenants are preconfigured out of the box common, infra and mgmt. 15.

Note1: The common tenant is preconfigured for defining policies that provide common behavior for all the tenants in the fabric. A policy defined in the common tenant is usable by any tenant.

Note2: In the lab, the screen might not appear as below since other tenants will be created. To see all tenants including preconfigured ones, use the option All Tenants.

Click on the System menu item at the top of the APIC GUI and observe the sub-menu 16.

options.

17

The Quick Start section on this page assists you in performing common and basic procedures.

The Concepts menu displays APIC online help that covers the building blocks of ACI.

Researching through documentation in Quick Start and Concepts is a great way to start learning

about ACI or refreshing your knowledge during implementation.

Dashboard provides an overview of the system health.

The Controllers menu displays property and status information about the APIC instances and

clusters.

Click on the Operations Menu item. Observe sub-menu options. 17.

Visibility & Troubleshooting option provides way of troubleshooting traffic between

the two specific end points as well as their location.

Capacity Dashboard provides visibility into the ACI fabric resources utilization.

ACI Optimizer – Details about this tool can be found at this KB article.

EP Tracker – Locate EndPoints based on IP or MAC address. More details can be found

at this ACI Toolkit document.

Visualization – Shows nodes utilization. More info can be found at this github link.

Note: Tested, community written python based tools get implemented in the ACI releases. In the release 1.2, ACI Optimizer, EP Tracker and Visualization components of the ACI Toolkit have been added as the integrated parts of the ACI release.

At this point you should be comfortable navigating the top level options of the APIC GUI.

Again, we will explore most of these menu options in much greater detail throughout the

lab.

Task 2: Verify APICs, ACI Fabric and OOB Management

To verify the status of the controllers, click on System->Controllers. Expand 18.

Controllers and verify that all three APICs are present. Click on the Controllers folder

to verify the APIC connectivity topology.

18

Click on apic1 in the left navigation pane or double-click the apic1 object in the right 19.

content pane, then select the General tab to see the information about this APIC.

Note: Explore the rest of the information about the APIC by expanding apic1 and selecting different tabs as time permits.

Click on Fabric at the top of the GUI. Select the Inventory option and click on 20.

Topology option from the menu on the left to see the whole ACI fabric topology.

Note: We can see the same information from the multiple places in the GUI.

This is an internal

VXLAN tunnel Endpoint

(VTEP) address of this

APIC automatically

assigned based on the

address pool we chose

during the initial APIC’s

setup.

19

Expand Pod 1, click on Leaf101 and select the General tab to see information about this 21.

leaf including the OOB IP address. Explore the rest of the information about the leaves

and spines as time permits.

Note: OOB information about ACI nodes is also available where the policy configuration tasks are performed, at Tenants->mgmt->Node Management Addresses->Node.

Task 3: Verify vPC connectivity to the UCS system

Open Putty and SSH to the Leaf101’s IP address 192.168.199.6. 22.

Note: The address is visible in the previous step.

Click Yes to Putty Security Alert. 23.

20

Login with the User ID of admin and password of 1234Qwer. 24.

Type the where command. Notice that you are in bash shell. 25.

Note1: In ACI version 1.0, to issue NX-OS style commands we would have to use vsh command first to get to vshell. As of version 1.1 that is not needed any more. All NX-OS commands can be issued straight from the iBash shell.

Note2: iBash shell is just a bash shall plus a set of custom commands.

Type show with space after it, and press Tab key twice to see all available options. 26.

Note1: In iBash shell, pressing Esc key twice replaces NX-OS/IOS ‘?’ option.

Press Esc key twice to see both, options and their descriptions.

Note2: Pressing Tab key once completes the command like in NX-OS and IOS.

Issue the show vpc command. 27.

Note: vPC numbers and Port channel numbers could vary as the system assigns these dynamically at time of the creation.

21

Use the common NX-OS Show commands to confirm that the above port channels are indeed

ones leading to UCS. Explore Leaf 101 and other nodes as time permits.

Additional suggested commands include:

conf t (this will not work since all configuration of the nodes is done using APIC)

show port-channel summary

show cdp neighbors

show lldp neighbors

show version (Processor Board ID is the same as Serial Number in GUI)

show vpc peer-keepalive (there is no peer-link or specific peer-keepalive for vPC in ACI. The

status and information about the peer is done automatically via APIC and fabric).

show ip interface vrf management

We can obtain the same information about vPC from the GUI. Go to Fabric->Inventory 28.

and expand Pod1->Leaf101->Interfaces->VPC Interfaces within the left menu and

then click on VPC Interfaces and explore options as shown below.

Within the same menu on the left, expand Protocols->CDP and click on Neighbors to 29.

see the equivalent of the show cdp neighbors command in GUI.

22

To verify who the vPC peers are and what the vPC Domain ID is, go to Fabric->Access 30.

Policies, and within the navigation pane on the left expand Switch Policies->Policies and

click on Virtual Port Channel Default.

In this module we performed a high level overview of the APIC GUI and verified configuration

work done as a part of lab preparation.

Congratulations! You are done with this module.

23

Module 2: Using API Inspector and Postman

Note: This Module can be skipped and done later without affecting the rest of the lab.

ACI can be configured using the GUI, CLI and REST API.

Throughout the lab we are using the GUI as it is the most efficient way to learn about ACI.

It is important to note however that the REST API, taking advantage of xml or json based scripts,

provides a way to automate and optimize configuration deployments making roll out

significantly faster.

When you perform a task in the APIC GUI, the GUI creates and sends internal API messages to

the operating system to execute the task. By using the API Inspector, which is a built-in tool of

the APIC, you can view and copy these API messages.

Note: As a part of this lab experience, you have the flexibility to invest your time in topics that are of more interest to you. Should you decide to do so, you can skip the entire modules by using pre-created JSON scripts.

For this lab, we created scripts by collecting and modifying data output from the API Inspector¸

effectively using it as a “GUI to JSON” translator.

To reduce the time needed for the lab steps configuration, you can optionally execute those

scripts by using the Chrome application Postman.

This module will provide an overview of these two tools by using an example with the following

steps:

1) Access API Inspector and Postman

2) Create an ACI User and then delete it using GUI

3) Capture and create a JSON script using the API Inspector output corresponding to step 2

4) Create the same ACI User and then delete it using the JSON script and Postman

More information about REST API can be found in Cisco APIC REST API User Guide.

If you are skillful with JSON or XML, you could have written the above script yourself by using

examples from the Cisco ACI configuration guides.

For advanced automation using Python, you can download sample codes from repositories such

as GitHub, but that is beyond the scope of this lab.

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/1-x/api/rest/b_APIC_RESTful_API_User_Guide.html

https://github.com/datacenter

24

Task 1: Explore API Inspector and Postman

To open API Inspector, click on the welcome, admin option in the upper right hand 1.

corner to view the drop-down list and select Show API Inspector.

Note: The new browser window will open up displaying API Inspector.

We can monitor all APIC activities here.

Overview of API Inspector options 2.

The API Inspector screen will fill up quite quickly since API Inspector captures all levels of

APICs communications by default, including all API commands (POST, GET and DELETE).

Note: POST, GET and DELETE commands are at the debug level.

Filter only affects what is displayed; API Inspector will continue collecting all information

regardless of the filter setting, but simply will not display it.

Search helps us to additionally narrow down what commands will be displayed.

Leave the API Inspector browser window open.

Double click the Postman shortcut on the desktop 3.

Note: Postman is a free Google APP. Check here for more information.

Expand Lab Collections in the menu on the left and click on POST Login. 4.

http://www.getpostman.com/

25

Overview of the Postman options 1-5. 5.

1) Collection of previously created scripts organized in folders. In this lab, you will use

those scripts if you want to save time by using REST API instead of the GUI.

2) On the right hand side is the content of the Login script we selected. This script will

be used for authentication with the APIC.

3) Shows url where in the Management Information Tree (MIT) we want to impact the

object with the Login Script.

4) Shows Method of what we want to do with the object (POST, GET and DELETE are

supported).

5) The content of the message - Payload, what and how we want to create/modify the

object.

Note: We can obtain url, Method and Payload information from the API Inspector.

Leave Postman open and return to the ACI browser window.

We are now going to create and delete a single user using GUI. The equivalent JSON scripts will

be captured by the API Inspector.

Make sure you kept API inspector window open!

1

5

4 3

2

26

Task 2: Create and Delete ACI User using GUI

Click on ADMIN and select AAA option. 6.

Expand Security Management, right click on Local User and select Create Local User. 7.

Leave all options unchecked and click Next. 8.

Type Student<Pod Number>Local for Login ID, and 1234Qwer for a Password and 9.

Confirm Password and click Finish.

Note: Replace <Pod Number> with your pod number.

27

Expand Local Users, right click on the newly created user Student<Pod Number>Local 10.

and select Delete.

Click Yes to confirm deletion. 11.

All configurations we did in this task using GUI should have been captured by the API Inspector.

In the following task we will use those captured JSON scripts.

Task 3: Capture JSON script using the API Inspector

Go back to the API Inspector browser window. By now, there are a lot of messages there from

yours and from other student’s activities. We want to find the POST actions that created and

deleted the user.

Type Student<Pod Number>Local in the search window, check Filter option and 12.

uncheck Wrap and Scroll to latest options.

Note: Replace <Pod Number> with your pod number

The first output will be POST command to create the user. Notice two of the components we

need:

1) url which describes where to place the object (new user and its properties)

2) payload which tells APIC what the characteristics of the object are

Note: This is plain text, and can be copied, pasted and modified in any text editor.

28

Read through the captures and find the second POST command that contains information 13.

on how to delete the user. Copy POST output for both creating and deleting the user and

paste into notepad to prepare the JSON script to be sent by Postman.

Note: You should have your pod number below instead of “PodNumber” in the name.

The second POST command should be toward the bottom of the list.

Validate your script to create the user against this one:

method: POST

url: https://192.168.199.1/api/node/mo/uni/userext/user-StudentPodNumberLocal.json

payload

{"aaaUser":{"attributes":{"dn":"uni/userext/user-

StudentPodNumberLocal","name":"StudentPodNumberLocal","pwd":"1234Qwer","rn"

:"user-StudentPodNumberLocal","status":"created"},"children":[]}}

Validate your script to delete the user against this one:

method: POST

url: https://192.168.199.1/api/node/mo/uni/userext.json

payload

{"aaaUserEp":{"attributes":{"dn":"uni/userext","status":"modified"},"children":[{"aaa

User":{"attributes":{"dn":"uni/userext/user-

StudentPodNumberLocal","status":"deleted"},"children":[]}}]}}

The format of the above commands might be hard to read. When using Postman in your test and

production environment, you can make it more readable by using online formatting programs

such as codebeautify. You should run only the payload portion through that program. Below is

the example of how the new outputs would look like for our code.

Note: Since in this lab you do not have access to the internet, you cannot access codebeautify site to make JSON more readable. Below is the “beautified” JSON code just for your reference.

‘Beautified’ payload to create the user

{

"aaaUser": {

"attributes": {

"dn": "uni/userext/user-StudentPodNumberLocal",

"name": "StudentPodNumberLocal",

"pwd": "1234Qwer",

"rn": "user-StudentPodNumberLocal",

"status": "created"

},

"children": []

}

}

‘Beautified’ payload to delete the user

{

http://codebeautify.org/jsonviewer

29

"aaaUserEp": {

"attributes": {

"dn": "uni/userext",

"status": "modified"

},

"children": [

{

"aaaUser": {

"attributes": {

"dn": "uni/userext/user-StudentPodNumberLocal",

"status": "deleted"

},

"children": []

}

}

]

}

}

Next we will use JSON text scripts from this task together with Postman to create and delete the

user.

Task 4: Create and Delete ACI User using Postman

Go to APIC GUI and verify that there is no Student<PodNumber>Local user at 14.

ADMIN->AAA->Security Management->Local Users.

Note: Replace <Pod Number> with your pod number

Go to Postman, click on POST Login on the left and click Send on the right to 15.

authenticate with APIC.

30

Verify that POST was successful by observing the Status “200 OK” message in the 16.

output below the Send button.

Note: For all the scripts that you will be running in the lab this message confirms success.

To create a new user, enter the corresponding url and payload from the previous task, 17.

select POST and click Send. Make sure that JSON is selected as a format.

Observe that the new student is created in the APIC GUI. 18.

To delete the user, enter the corresponding url and payload from the previous task within 19.

Postman and click Send.

31

Verify that the user is now removed in the APIC GUI. 20.

The main purpose of this module was to provide you with the very basics of REST API

programming with JSON using API Inspector and Postman, so that you can use existing scripts

to skip specific tasks. That way you can have more time to focus on the parts of the lab of most

interest to you.

In today’s world of automation, API and programmability from the various orchestrator tools

(UCS Director, Puppet …) are present and gaining more traction in the Datacenter. While you

will not have to become a JSON script expert, some basic understanding about this technology is

very important. The second goal of this module is to provide that understanding.

32

Module 3: Building ACI Forwarding Constructs In the previous modules we verified the setup of the fabric and got familiar with the GUI and

basic JSON programmability. From this module on we will focus on the management of the ACI

using the ACI policy model.

The ACI policy model enables the specification of application requirements policies.

This policy model manages the entire fabric, including infrastructure, authentication, security,

services, applications, and diagnostics.

In simplified terms, ACI brings to the data center networking environment what UCS brought to

the computing environment with service profiles and stateless hardware.

A Tenant is a logical container for application policies that enables domain-based access

control. It represents a unit of isolation from a policy perspective. These logical containers can

be used whether we want to separate customers, organizations, domains or just to group policies.

Note: In this lab we use the concept of Tenant to separate policies from multiple students that are using a single ACI fabric.

The following figure depicts elements of the Tenant. In this Module we are focusing on Context

(VRF), Bridge Domain and Subnet. The other elements will be covered later in the lab.

A Context (Private Network) defines a Layer 3 (L3) address domain. It is equivalent to a

virtual routing and forwarding (VRF). We can have one or more Contexts per tenant.

As of the ACI version 1.2, term VRF is used in the GUI instead of the terms Private Network or

Context. Those three terms are being used interchangeably.

33

A Bridge Domain (BD) represents a Layer 2 (L2) forwarding construct. It is a container of

subnets and may be configured as a L2 flood domain/boundary, but it does not have to be. We

can have one or more Bridge Domains associated with the same Context (VRF). A Bridge

Domain can span multiple switches and can contain one or more Subnets.

More details about ACI Policy Model can be found in Cisco Application Centric Infrastructure

Fundamentals document.

At the end of the Module, you will create your Tenant (TenantX, where X is your pod number).

This Tenant will contain one VRF (TX_Production) and two Bridge Domains (VMData-Web

and VMData-App).

Task Summary: Task 1: Create a Tenant

Task 2: Create a VRF for the new tenant

Task 3: Create Bridge Domains for VMData-Web and VMData-App

These constructs will be used in later lab exercises.

Note: If you want to save lab time, you can configure this module task by task using Postman. Please check the Module 2 for details on how to use this tool.

Task 1: Create a Tenant

In this task, you will create TenantX, where X is your pod number. Before creating the tenant,

we will create Security Domain for this tenant first.

The Security Domain is a tag that will be associated with this tenant in the APIC Management

Information Model. APIC will use the Security Domain tag to restrict Tenant Admin access to

this tenant only using RBAC. The Security Domain tag is necessary to assign a Tenant Admin at

a later date.

Select Admin->AAA. 1.

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/1-x/aci-fundamentals/b_ACI-Fundamentals/b_ACI_Fundamentals_BigBook_chapter_010.html

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/1-x/aci-fundamentals/b_ACI-Fundamentals/b_ACI_Fundamentals_BigBook_chapter_010.html

34

Expand Security Management, right click on Security Domains and select Create 2.

Security Domain.

Type TenantX for a name (Replace X with your pod number) and click Submit. 3.

Note: Screenshot below is taken for pod 5.

Select the Tenants -> Add Tenant. 4.

Enter TenantX (Replace X with your pod number) as your tenants Name, check 5.

TenantX Security Domain and click Submit.

Note: As soon as you click Submit, you will be taken to the Networking option of the Tenant you just created.

35

Your domain TenantX is created. 6.

Note1: The URL is updated to reflect that you are in your tenant object.

Note2: A quick way to get to your tenant is by clicking on its name in the tenant bar. A tenant bar shows only the most recent tenants. Since there are many students using the same ACI fabric, your tenants name might not be there at all times.

To view all tenants, select the ALL TENANTS tab. To get back to your tenant, locate 7.

your tenant in the table and double click on it.

Note1: This is another way to get to the desired tenant if it is not on the tenant bar. Your tenant might be on the second page of the list.

Note2: You could also perform a search by name. The search is case sensitive.

Your Tenant is successfully created and you are done with this task.

Task 2: Create a VRF and a Bridge Domain (BD) for Web Servers

In this task, you will create a VRF (Virtual Routing Forwarding) instance for your tenant.

In addition, you will create a Bridge Domain (BD) for the Web Servers tier.

Note: Prior to the ACI version 1.2, term Private Network was used in the GUI. Other terms used interchangeably in documentations are Context and Private Network Context.

VRF is an L3 object in the ACI network and allows for the separate routing instances. It can also be used for the administrative L3 separation. All subnets for a particular tenant must be associated with a VRF for that tenant (via Bridge Domain), but a tenant can have more than one VRF.

Subnets cannot overlap within the same VRF.

36

Expand Tenant TenantX (where X is your pod number) created in the previous task and 8.

click on the Networking option. Observe available Drag and drop options.

Note: As of the ACI version 1.2, we can use drag and drop method to configure L3 Network (VRF), L2 Bridge Domain (BD), External Bridged (L2) and External Routed (L3) networks. You can use this method in production.

In this lab we will not use this method to be consistent with the previous ACI versions and to show in more details where each object is created.

Note: Make sure you are in your Tenant!

Expand Networking, right click on VRFs and choose Create VRF. 9.

Note: In the ACI GUI, “right clicking” and the Actions menu (to the far right) usually produce the same options. Some laptops are not set up to right click, so instead look to the far right and select ACTIONS->Create VRF.

Enter TX_Production (Replace X with your pod number) and click Next. 10.

Note: We could have unchecked Create A Bridge Domain, and create BD separately.

37

Enter the Bridge Domain name VMData-Web for the name and click Finish. 11.

Click on Networking to see VRF and BD you just created. 12.

Note: To see the name of the network object, hover with the mouse over the circle.

Task 3: Create a Bridge Domain (BD) for App Servers

In this task, you will add a subnet to the previously created bridge domain (BD) VMData-Web

for web servers, and create a new bridge domain VMData-App for app servers.

BD gives network engineers layer 2 design capabilities that go way beyond what they could do

with the vlan alone.

While we can design BD to behave like the traditional vlan, we can also provide optimized

design that will use the ACI fabric intelligence to avoid unnecessary flooding and broadcasts.

A Bridge Domain (BD) also contains subnets. If we chose so, BD will provide the distributed

default gateway for those subnets.

A Bridge Domain must be associated with a VRF.

38

Expand Networking->Bridge Domain->VMData-Web, right click on Subnets and 13.

select Create Subnet.

Note: Throughout the screenshots you will notice ND abbreviation like in ‘ND Proxy Subnets’ below. ND stands for IPv6 Neighbor Discovery.

Support for IPv6 started in ACI version 1.1. IPv6 is not part of this lab.

Enter 192.168.10.1/24 as the Web Servers Network Gateway IP and click Submit. 14.

Note: In ACI version 1.0, there was an additional mask field that would auto-populate based on / notation. Mask field would be 255.255.255.0 in the example below.

Note: After you click submit, subnet will be displayed in the Work Pane.

We will now create a second Bridge Domain for the App Server called VMData-App.

Unlike with VMData-Web, we will define subnets while creating VMData-App bridge domain.

There is typically more than one way to create the same object in ACI using GUI.

39

Right-click on Bridge Domains and select Create Bridge Domain. 15.

Enter VMData-App for the name, for VRF select TenantX/TX_Production from the 16.

drop down menu and click Next.

Note: By the default, L2 design is optimized. If you would like to design L2 differently (for example to behave like a vlan), select option customize and modify L2 behavior.

Click + to add subnets that belong to this BD. 17.

Enter 192.168.11.1/24 as the App Servers Network Gateway IP and click OK. 18.

Click Next. 19.

We will not define the Monitoring Policy in this lab. Click Finish. 20.

40

Click on Networking to verify that we have created and configured two Bridge Domains 21.

and associated them with the VRF TX_Production.

ARP Flooding is off by default because the assumption is that hosts are not silent and will source

frames and the fabric (leaf) will learn about the endpoint.

Later in the lab we will use a Switch Virtual Interface (SVI) on a Cisco Nexus 5548 Switch to

act as an endpoint. The SVI will not source traffic and therefore the fabric will not learn the

SVI’s IP Address. When ARP flooding is disabled (which is the default), unicast routing is

performed on the target address. In our scenario, the SVI is not sending traffic so the host route

is not in the routing table and the ARP is dropped. By turning on ARP flooding the “ARP

Request” is sent out of the interface toward the N5K and when the “ARP Response” returns (i.e.

arrives at the leaf) the fabric dynamically learns about the SVI (IP address and MAC).

Click on VMData-App, select ARP Flooding checkbox and click Submit. 22.

Repeat the same step for the other Bridge Domain. Click on VMData-Web, select ARP 23.

Flooding checkbox and click Submit.

If Policy Usage Warning window pops up, select Submit Changes to confirm. 24.

You created your Tenant and the basic L2 and L3 constructs for it. You are done with Module 3.

41

Module 4: Configuring Application Profile In the previous Module, we were introduced to the ACI networking policy model, the concept of

a Tenant and configured some of the Tenants networking elements, namely Context (also known

as Private Network or VRF) and Bridge Domain.

In this module we will focus on the Application Profile and Security Policies.

The ACI policy model is designed to provision a network based on application requirements.

An Application Profile models those application requirements. It consists of Endpoint Groups

(EPGs) and the policies that define the communication between them - Contracts.

Endpoint Group (EPG) is a grouping of application or application components independent of

other network constructs. Examples include groupings based on common security, Virtual

Machine Mobility (VMM), QoS or L4-L7 services. Endpoints can be physical (for example bare

metal servers, NAS) or virtual (for example Virtual Machines). Policies are never applied

directly to the endpoints but to the EPG instead.

Endpoints within the same EPG can communicate freely. For communication between endpoints

belonging to the different Endpoint Groups there has to be mandatory Contract between the

two EPGs.

Contracts are groups of subjects which define communication between EPGs. One EPG

provides the contract (sets the rules for communicating with it), and the other EPG consumes

the contract (gets information about the rules to access the provider EPG). An EPG can be both a

provider and consumer of multiple contracts.

Subjects build definitions of communications between EPGs. They contain filters that classify

traffic of interest and an Action that defines what to do with that traffic (similar to Access

42

Control Lists). There is an optional Label identifier for more complex relationship mappings.

Label Identifiers go beyond the scope of this lab.

In this module we will create a 2-Tier Application Profile. We will create two Endpoint Groups:

1) App_Servers EPG which will contain our App Server Virtual Machines introduced later

in the lab.

2) Web_Servers EPG which will contain our Web Server Virtual Machines also introduced

later in the lab.

We will define traffic we want to allow between these two EPGs via a contract App_Contract.

The final configuration for this module is depicted below where the App_Servers EPG provides

App_Contract, while the Web_Servers EPG consumes that contract as indicated by the arrows.

Note: If you want to save lab time, you can configure this module’s tasks using Postman. Please check the Module 2 for details.

43

Task 1: Create Application Network Profile (ANP)

If you are not already in your tenant TenantX (where X is your pod number), go there by 1.

selecting ALL TENANTS tab and double clicking on the name of your TenantX.

Note: Your tenant might be on the second page of the list.

Right click on Application Profiles and select Create Application Profile. 2.

Note: Make sure you are in your Tenants environment.

Enter Name TX_AppProfile (where X is your pod number) and click + under EPGs to 3.

create an Endpoint Group.

44

Enter name Web_Servers, for the Bridge Domain select VMData-Web. Expand the 4.

Consumed Contract and select Create Contract.

Note: You will associate this EPG to a Virtual Machine Manager (VMM) Domain Profile later in the lab.

Enter Name App_Contract and click + to create a Subject. 5.

Note: In older versions of the APIC, for scope option, the words Context or Private Network were used instead of VRF.

Enter Name App_Services, click + next to Filters. Click anywhere on the Name box 6.

where it reads “select an option” and then click on the new + sign to create a new filter.

45

Name the Filter App_Service_Ports and click + to add Entries. Enter data from the table 7.

below and click Update. Name ICMP

EtherType IP

IP Protocol icmp

Click + next to Entries to add another Filter Entry. Enter data from the table below and 8.

click Update. Name TCP5000

EtherType IP

IP Protocol tcp

Destination port range From/To 5000/5000

Click Submit in the Create Filter dialog. 9.

Click Update on the Filters name. 10.

Click OK to Create Contract Subject. 11.

46

Click Submit to Create Contract. 12.

Note: In this part of the lab we used wizard to create Contract, Subject and Filters. In module 7 (L3 external connectivity), we will show how to create these objects without the wizard.

Expand the Consumed Contract, select the newly created App_Contract and click 13.

Update.

Click + under EPGs to add the second Endpoint Group of the 2-tier application. Name 14.

EPG App_Servers, for Bridge Domain select VMData-App, select App_Contract as

the provided contract and click Update.

Click Submit. 15.

Expand Application Profiles and click on TX_AppProfile. Click on any object in the 16.

Application graph to see the directions of the contract.

Note: Direction of the contract is from the provider to the consumer, while the first packet goes the opposite way.

47

Expand the App_servers and Web_Servers EPGs and select Contracts to verify that the 17.

Contractual relationship was created. The state will show as Formed.

Note: We could already see that relationship by reading the graph arrows from before.

Contract and Filters are under Tenant’s Security Policies. To see Contract and Filter we 18.

created expand the Tenant Tree as shown below.

Note: We created Contract and Filters as a part of Application Profile wizard. We could have created them separately ahead of time. We can reuse these Security Policies in different ANPs (Application Network Profile) later.

In this Module we created an Application Network Profile for a simple 2-tiered application and

defined a Provider/Consumer relationship between two Endpoint Groups.

So far in the lab we have created the necessary elements of the Logical Policy Model.

In the following Module we will assign Endpoints to the Endpoint Group.

48

Module 5: Configuring VMM Integration

In the previous Modules, we focused on Logical Policy Model elements. We created a Tenant, an

Application Network Profile, and Network elements such as a VRF and Bridge Domains.

We defined two Endpoint Groups as a part of Application Network Profile (ANP) representing

an example of 2-tier application.

In this lab, we will define association of Endpoints with Endpoint Groups.

Again, Endpoints can be physical (for example Bare metal servers, NAS) or virtual (for example

Virtual Machines).

Endpoints within the same EPG can communicate freely. For communication between endpoints

belonging to different Endpoint Groups, there has to be mandatory Contract between these two

EPGs.

In this lab, we will integrate VMware vCenter with APIC and enable connectivity between two

preconfigured Virtual Servers using the Logical Policy Model elements we configured in the

previous two modules.

We will explore the impact of the Contract we created earlier to connectivity between these two

servers, given that each belongs to a different Endpoint Group.

Note: All tasks in this module have to be performed to be able to continue with the lab. If you want to save lab time, you can configure all tasks except Task 2 using Postman. Task 2 in this module is done in the vCenter; therefore there is no Postman shortcut.

49

Task 1: Configure VMM Integration

In this task, you will integrate a Virtual Machine Manager (VMM) with ACI.

The table below shows the naming convention for the different objects per pod.

Note: Attachable Access Entity Profile (AEP) is created ahead of time as a part of ACI-UCS connectivity. Details are in the ACI Lab Setup and Connectivity document.

Student Tenant VMM Name Associated AEP VLAN Pool Name

Pod1 Tenant1 T1-vCenter

UCS_Domain_01

T1-VLAN-Pool

Pod2 Tenant2 T2-vCenter T2-VLAN-Pool















Select VM Networking. Right click at VMware and select Create vCenter Domain. 1.

Note: vCenter domain references a particular vCenter manager and a particular pool of VLANs or VxLANs that will be used.

In the older ACI versions there was the additional Policies tab which has been removed as of 1.2 version.

50

Enter TX-vCenter as Name (where X is your pod number), select UCS_Domain_1 for 2.

Associated Attachable Access Entity Profile, click on VLAN Pool and select Create

VLAN Pool.

Note: Attachable Access Entity Profile (AEP) is created ahead of time. Details are in the ACI Lab Setup and Connectivity document. The AEP specifies which leaf ports will be used to connect to the hosts and which VLANS are valid on those connections.

Enter TX-VM-VLAN-Pool as a Name (where X is your pod number) and click on + next 3.

to Encap Blocks.

Note: Make sure that Allocation Mode is Dynamic Allocation. For VLAN Pools associated with Virtual Domains, we want Dynamic Allocation.

Add the VLAN Range as per the table below and click OK 4.

Note: VLAN range for pod5 is in the screenshot.

Pod VLAN Range Pod VLAN Range Pod VLAN Range Pod VLAN Range

Pod1 2110–2119 Pod5 2150–2159 Pod9 2190–2199 Pod13 2230–2239

Pod2 2120–2129 Pod6 2160–2169 Pod10 2200–2209 Pod14 2240–2249

Pod3 2130–2139 Pod7 2170–2179 Pod11 2210–2219 Pod15 2250–2259

Pod4 2140–2149 Pod8 2180–2189 Pod12 2220–2229 Pod16 2260–2269

51

Click Submit in the CREATE VLAN POOL dialog. 5.

Note: The VLAN pool we just created is at Fabric->Access Policies->Pools. We could have created it there ahead of time and have it available in the pull down menu.

To select a Security Domain, click + select your Tenant’s Security domain TenantX 6.

(where X is your pod number) and click Update.

Note: You created Security Domain at the beginning of the Module 3.

Click + next to vCenter Credentials. 7.

Enter AdministratorX as a Name, for Username enter root with password 1234QwerX 8.

(where X is your pod number) and click OK.

Note: For example, pod 7 will have password 1234Qwer7

Click + next to vCenter/vShield. 9.

52

Refer to the following table to assign VCENTER/VSHIELD CONTROLLER options. 10.

Choose DVS Version 5.5 for DVS version and click OK.

Note: Enter the Datacenter name exactly as the table below (case-sensitive – must match exactly). You can also Cut-and-Paste the Datacenter name from the vCenter.

Name Datacenter vCenter IP Associated

Credential

T1_VC_Controller Tenant1 192.168.199.201 Administrator1
















Click Submit to CREATE VCENTER DOMAIN. 11.

53

Your VM Provider Profile will appear in the vCenter Domains. 12.

To verify connectivity with vCenter, expand VMware and click on Tx-vCenter. Select 13.

Operational tab and verify that the State is Online.

Note: If status is not online, check troubleshooting section at the end of this document. If that does not help, please contact your instructor.

Expand the whole tree under the Tx-vCenter to verify that the vCenter inventory is 14.

discovered. Verify the APIC created a DVS in this vCenter and notice that there is no

VM facing DVS Portgroup.

Note: You can also verify the creation of the DVS in vCenter, which you will do in the next task.

In this Task so far, we connected APIC with vCenter. APIC instructed vCenter to create a

Distributed Virtual Switch (DVS).

APIC will NOT instruct vCenter as to which servers it should connect to the DVS or VMware

port group. This task has to be done by the Server Administrator within vCenter, and that is what

we will do in the next task.

54

Next we will define what will qualify an Endpoint to belong to an Endpoint Group (EPG).

In the vCenter based server environment, this is done using VMware Port Groups. VMs

belonging to the same Port Group will belong to the same EPG.

In the previous module we created Application profile TX_AppProfile (where X is your pod

number) that contains two EPGs. By associating each of those two EPGs with VMM domain

TX_vCenter we just created, APIC will instruct vCenter to create two corresponding port groups.

Any VM the Server Administrator connects to those port groups will correspond to an EPG

member from the APICs perspective.

Select Tenants -> TenantX (where X is your pod number). 15.

Note: You can also get to your Tenant using the All Tenants option as before if your tenant is not visible in the most recent tenants bar.

Verify that your Tenant has TX_AppProfile created with two EPGs by expanding 16.

Application Profiles and clicking on TX_AppProfile. Click on any object to verify

contracts provider-consumer relationship of the EPGs.

Expand TX_AppProfile->Application EPGs and select EPG App_Servers, choose the 17.

Operational Tab. Notice that there are no client end-points in this EPG.

55

Expand EPG App_Servers, right click Domains (VMs and Bare-Metals) and select Add 18.

VMM Domain Association.

For the VMM Domain Profile, choose your VMM Domain instance VMware/TX-19.

vCenter (where X is your pod number) set Deploy and Resolution Immediacies to

Immediate and click Submit.

Resolution Immediacy defines when APIC downloads Policies to the Leaf:

Immediate: All policies (VLAN / NVGRE / VXLAN bindings, Contracts, Filters) are pushed to the Leaf node upon Hypervisor pNIC attachment. LLDP or OpFlex are used to detect Hypervisor to Leaf node attachment.

On Demand: Policies only pushed to the Leaf node upon pNIC attachment AND vNIC association with port-group (EPG). Purpose is to save Leaf resources if needed.

Pre-Provision: The same as Immediate, except that policies are pushed to the Leaf node even if Hypervisor is not attached yet. This is a new option as of ACI version 1.1.

Deploy Immediacy defines when we are programming Policy in Leaf node hardware.

Immediate: Policies programmed in Policy CAM once received by APIC as defined by Resolution Immediacy Policy.

On Demand: Polices programmed in Policy CAM only when reachability is learned through data path. Purpose is to save ASIC resources if needed.

56

Verify that the STATE is formed. 20.

Note: As soon as you click Submit, the new port group will be created in the vCenter.

Click at EPG App_Servers and select the Faults Tab to verify there were no faults. 21.

Note: Faults tab exists for the most of the objects in ACI and is an excellent starting point for troubleshooting.

It is important not to overreact on the fault. Sometimes faults are reported to the APIC during the transition phase and get resolved by themselves. If the fault is resolved after the transition, their Lifecycle status will eventually become Clearing. This indicates that fault is no more a fault. You might encounter that case in this step.

Select the OPERATIONAL tab and note that there are still no Endpoints associated. 22.

Expand EPG Web_Servers, right click Domains (VMs and Bare-Metals) and select 23.

Add VMM Domain Association.

Note: You are repeating the same last few steps just for Web EPG.

57

For VMM Domain Profile choose your VMM Domain instance VMware/TX-vCenter, 24.

set Deploy and Resolution Immediacies to Immediate and click Submit.

Verify that the STATE is formed. 25.

Click EPG Web_Servers, select the OPERATIONAL tab and note that there are still no 26.

Endpoints associated.

APIC is now ready for virtual App and Web endpoints (virtual servers) to get associated with

their corresponding EPGs.

In the next task, as a Server Administrator, you will attach virtual servers to the port groups

created by APIC. That way those servers will be associated with the proper EPG.

58

Task 2: Add VMs to the port group from the VMware vCenter

In this task we will associate an existing host with the DVS and then attach each of the two

servers to the proper port group on that DVS.

From the student desktop, double click on the vSphere Client icon, use address and 27.

credentials from the table below, and click Login. Click Ignore if the warning pops up.

Pod vCenter IP User Password Pod vCenter IP User Password

1 192.168.199.201 root 1234Qwer1 9 192.168.199.209 root 1234Qwer9








59

Navigate to Hosts and Clusters, select TenantX Datacenter (where X is your pod 28.

number) and select the Tasks & Events tab to verify DVS and port group creation.

Note: You can also navigate to the Networking view to see if the APIC DVS and port groups were successfully created in the previous task.

ESXi hosts addressing is in the following table:

Datacenter Host Datacenter Host

Tenant1 192.168.191.51 Tenant9 192.168.191.59

Tenant2 192.168.191.52 Tenant10 192.168.191.60

Tenant3 192.168.191.53 Tenant11 192.168.191.61

Tenant4 192.168.191.54 Tenant12 192.168.191.62

Tenant5 192.168.191.55 Tenant13 192.168.191.63

Tenant6 192.168.191.56 Tenant14 192.168.191.64

Tenant7 192.168.191.57 Tenant15 192.168.191.65

Tenant8 192.168.191.58 Tenant16 192.168.191.66

Verify that your host and two VMs appear under your TenantX Datacenter. 29.

Note: There will be other VMs under your host. Ignore them.

Click on Hosts and Clusters and navigate to Networking to add the ESXi server to the 30.

DVS created by APIC.

60

Expand ACI-vCenterX -> TenantX -> TX-vCenter folder (where X is your pod 31.

number).

Note: Notice that the folder name is the same as the VMM domain that you created in the APIC in the previous task. Also notice the two port groups created by APIC named after the Tenant, Application Profile and EPG names.

Right click at DVS TX-vCenter switch and select Add Host. 32.

The Add Host to vSphere Distributed Switch wizard will guide you through the process of

adding your server into the APIC DVS switch.

Select your Host and two unassigned physical adapters (vmnics 0 & 1) and click Next. 33.

Note: Notice that two physical adapters (vmnic2 and vmnic3 in the example below) are already assigned to vswitch0. These provide management connectivity between the ESXi host and vCenter. Do not change these adapter mappings.

61

Click Next -> Next-> Finish. 34.

Note: Notice Port Groups with dynamically created VLANs form the APIC VLAN pool.

VLANs in the screen capture belong to pod2. Specific VLAN numbers might vary.

Verify that the host is Connected with a VDS status of Up by selecting TX-vCenter and 35.

the Tab Hosts.

Go back to Hosts and Clusters view. 36.

Note: We will now assign Virtual Servers to their port groups.

62

Right click on Web_Server VM and select Edit Settings 37.

Click on the Network adapter, set the Network Connection to the 38.

TenantX|TX_AppProfile|Web_Servers Port Group and click OK.

Note: Notice the name of the port group includes the names of the Tenant, Application Profile and End Point group.

As soon as you are done with this step, vCenter will communicate with the APIC and the Endpoint will be registered with EPG even if the VM is powered off.

Right click on App_Server VM and select Edit Settings 39.

63

Click on the Network adapter, set the Network Connection to the 40.

TenantX|TX_AppProfile|App_Servers Port Group and click OK.

Go to the Networking menu by selecting Hosts and Clusters, click on the port groups 41.

and the Virtual Machine tab for each to verify the Network settings for App-Servers

and Web-Servers is applied.

64

Go back to your Tenant in the APIC to observe registered Endpoints, Tenants-42.

>TenantX (where X is your pod number).

Go to Tenant TenantX->Application Profiles->TX_AppProfile->Application EPGs 43.

and click on EPG Web_Servers. Click on the Operational tab and observe information

about the endpoint.

Note: End point is there, but you might see IP address 0.0.0.0 or ---. Why?

Since the Web Server in the lab environment is a silent host, APIC knows about its existence from the vCenter but did not receive any packets from the server and hence does not know its IP address.

The IP addresses under the Interface column belongs to the UCS Fabric Interconnects.

Encap VLAN is from the VLAN pool we created earlier in the lab.

Vmm indicates that the endpoint is a virtual machine from the vmm (vCenter).

Go back to the vCenter and revert to Hosts and Clusters menu, right click on Web-44.

ServerX and Open Console.

Login using login student and password 1234Qwer. 45.

65

Ping the Web Servers Defaut Gateway ping 192.168.10.1 46.

Note: We defined Default Gateway when we created the Bridge Domain for 192.168.10.0/24 subnet. The Leaf is the default gateway for this subnet.

To stop ping press Ctrl-C.

Note: This ping should be successful. If ping is unsuccessful, Power off (Guest Shutdown) the Linux VM (from vCenter) and then after shutdown is competed, Power On; retry Ping.

Go back to the APIC GUI and observe information about the endpoint. 47.

Note: Since Web_Server generated traffic, APIC discovered the Web_Servers’s IP address.

To view endpoint’s details, double Click on Web-ServerX output depicted above. Click 48.

Close when done.

Note: Under Interfaces (need to scroll down) you can see vPC Policy Group related with APIC and UCS connectivity.

66

If time permits, verify that App_Server endpoint is registered to the EPG App_Servers.

In this Task, playing a role of a Server Administrator we connected the App_Server and

Web_Server from the vCenter to the virtual networking environment created by APIC.

We also verified that endpoints were properly associated with their corresponding EPGs.

Task 3: Verify and Explore ACI Contracts

In Task 1 of this module, we defined the contract between two EPGs, with the App EPG

providing the contract, and the Web EPG consuming it. In that contract, we defined ping and

TCP port5000 as the allowed traffic leveraging a Filter. In this task, we will test that contract

and modify it.

Note: Contracts are based on the Whitelist method. We list the traffic that we want to allow. If traffic is not listed, it is not allowed. Contracts apply for traffic between two different EPGs. All traffic is allowed between two endpoints belonging to the same EPG.

Go back to vCenter and Web-Server’s Console window. Type ifconfig to obtain the 49.

address of your Web-Server. The forth octet value should be 100+X (where X is your pod

number).

Note: Web-Servers are in 192.168.10.0/24 and App-Servers are in 192.168.11.0/24 subnets. The forth octets are the same for both servers within the same pod. Screenshot is taken from pod2, (100+2) = 102.

Ping the App-ServerX ping 192.168.11.(100+X) where X is your pod number. 50.

Note: To stop ping press Ctrl-C.

67

After that was successful, try to SSH from web-server to app-server. Type command ssh 51.

192.168.11.100+X (where X is your pod number). Default port is port 22.

Note: SSH will time out eventually. To reduce the wait time, press Ctrl-C to interrupt.

Now try to SSH from web-server to app-server again but use tcp port 5000 52.

ssh –p 5000 192.168.11.100+X (where X is your pod number)

Note: Immediately the app-server refuses the connections. That means that the packet actually made it to the App_Server.

SSH TCP connection with destination Port 5000 was allowed by the contract to reach App_Server, while the default port 22 was blocked since it was not listed on the contract.

Important for MAC-Users: Control + option/alt key combination might NOT allow you to exit the console session. Hold down Control + Command + Option/Alt until you are released from the console window.

We will now access App_Server and do a ssh test. What do you expect will work?

From the Hosts and Clusters menu, right click on App-Server02 and Open Console. 53.

Login using login student and password 1234Qwer. 54.

SSH will not work, the same as before. Let’s try ssh on port 5000 ssh –p 5000 55.

192.168.10.100+X (where X is your pod number).

Note: This time, ssh on port -5000 did not work. Why? Let’s check contract in the APIC. Note: Pinging Web-ServerX at 192.168.10.(100+X) will work as an exception.

mailto:[email protected]+X

68

Go back to your Tenant TenantX in the APIC. 56.

Expand Tenant TenantX->Security Policies->Contracts->App_Contract and select 57.

App_Services. Note the name of the filter is App_Service_Ports.

Expand Security Policies->Filters and click on App_Service_Ports. 58.

Note: ICMP is allowed and Destination TCP port 5000 is also allowed.

Expand Tenant TenantX->Application Profiles->TX_AppProfile->Application 59.

EPGS->EPG App_Servers and click on Contracts. From this screen you can see why

ssh from the App_Server on port 5000 failed.

Note: App EPG is providing App_Contract (to the Web EPG) with the list of allowed traffic, but there is no contract that App EPG consumes! If we want ssh using port 5000 from the App_Server to work, we need contract where Web EPG will provide such contract to the App EPG and App EPG will consume it.

69

Click on Actions and select Add Consumed Contract 60.

Note: We could right click on Contract and have the same choices.

On the contract drop down list, select the TenantX/App_Contract created earlier and 61.

click Submit.

Note: We could have created a new contract as well.

Verify that App_Servers EPG is both Providing and Consuming the App_Contract. 62.

Expand Application EPGs->EGP Web_Servers, right click on Contracts and select 63.

Add Provided Contract.

Note: Previously, we made App_Servers EPG to consume the contract. We also need Web_Servers EPG to provide that contract.

70

From the Contract drop-down menu, select TenantX/App_Contract and click Submit. 64.

Verify that Web_Servers EPG is both Providing and Consuming the App_Contract. 65.

Expand Security Policies->Contracts and click on App_Contract for a visual 66.

presentation of the EPG - App_Contract relationships.

Return to the App-Server console in vSphere and try again to ssh using port 5000 to the 67.

Web-ServerX by issuing the following command: ssh –p 5000 192.168.11.100+X

Note: Now the behavior changes and the Connection is refused immediatelly. The difference is the login is being refused by the Web-Server,whereas previously it was being denied by the ACI fabric. Imagine that port tcp 5000 is a middleware. Would you want traffic arriving at the web server on this port? Probably not, but if you do make sure that the Web_Server EPG is a provider for that service.

Congratulations, you are done with the VMM Integration Module!

71

Module 6: Configuring Ext. Layer 2 Connectivity

In a typical ACI deployment, there will be an existing Data Center infrastructure that ACI has to

integrate with. This module shows how to integrate with that infrastructure at the Layer 2 (L2)

level. The next module will focus on the existing Layer 3 (L3) connectivity.

Additional details and options can be found in Connecting Application Centric Infrastructure

(ACI) to Outside Layer 2 and 3 Networks white paper.

We are in essence extending the bridge domain out of the ACI fabric in this module.

While for this exercise and in live production use cases it might be easier to use the Configure

an interface, PC and VPC wizard available from the Quick Start section under the Fabric-

>Access Policies menu, in the interest of ensuring that you know exactly what you are

configuring, we will configure all necessary policies and pools individually.

The Overall workflow for creating an access policy for connectivity is depicted below.

Note: The arrow orientation is different from most Cisco Documentations since those documents are focusing on the relationship between objects. Directions below show one possible configuration workflow.

Interface Policies control the configuration of an individual feature, CDP on/off, LLDP on/off,

LACP mode, port-speed etc. Necessary Interface Policies were already created as a part of initial

lab preparation. Details can be found in the ACI Lab setup and connectivity document.

Interface Policy Group is a container consisting of multiple Interface Policies grouped

together.

http://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/application-centric-infrastructure/white-paper-c07-732033.html

http://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/application-centric-infrastructure/white-paper-c07-732033.html

72

Interface Profile consists of a range of interfaces sharing similar configuration so that we can

apply the same Interface Policy Group across multiple interfaces in one place.

Switch Profile defines on which Nodes we will use a specific Interface Policy Group.

Whether you connect physical or virtual servers to the Cisco ACI fabric, you define a physical

or a virtual domain. Virtual domains reference a particular virtual machine manager and a

particular pool of VLANs or VxLANs that will be used.

Attachable Access Entity Profile (AEP) connects the concept of Domains (and corresponding

VLAN/VxLANs) with interfaces that tie to that domain.

Note: For VLAN/VxLAN to function on the specific Leaf, both the AEP and End Point Group (EPG) have to be provisioned.

The person who administers the VLAN or VxLAN space is the Infrastructure Administrator. The

person who consumes the domain is the Tenant Administrator. The Infrastructure Administrator

associates domains with a set of ports that are entitled or expected to be connected to virtualized

servers or physical servers through an Attachable Access Entity Profile (AEP).

The Tenant Administrator is in charge of creating Application Network Profiles (ANP) and

maintaining the Tenant’s resources.

vPC Explicit Protection group defines the vPC domain ID and which switches participate in

the domain. That configuration is not needed for the single interface connectivity represented in

this module.

Note: To connect the ACI fabric to Nexus devices, typically we will need only one object listed above. Due to lab scalability, we will have up to 16 times more objects created.

In this Module, in Task 1 you will configure single interface (not port channel) connectivity with

the external Nexus 5548 by creating an access policy and then, in Task 2, tie the Nexus L2

Outside Bridged Domain with the Application Network Profile you created earlier in the lab to

enable the traffic between VMs and Hosts behind Nexus.

Note: If you want to save lab time, you can configure this module task by using Postman. Please refer back to Module 2 for details on how to use this tool.

There are two types of policies under the Fabric option.

Fabric policies configure interfaces that connect spine and leaf switches. Fabric policies can

enable features such as monitoring (statistics collection and statistics export), troubleshooting

(on-demand diagnostics and SPAN), or NTP.

Access policies configure external-facing interfaces that do not connect to a spine switch.

External-facing interfaces connect to external devices such as virtual machine controllers and

hypervisors, hosts, routers, or fabric extenders (FEX). Access policies enable configuring port

channels and virtual port channels, protocols such as LLDP, CDP or LACP, and features like

monitoring or diagnostics.

73

Task 1: Create Access Policy to Nexus Switch

Click Fabric->Access Policies. 1.

Expand Interface Policies, right click on Policy Groups and select Create Access Port 2.

Policy Group.

Note: To check the policies we just created or to see the default policies, just expand the feature you want to see by clicking on + sign next to it and selecting policy.

Type TX_L2_Out (where X is your pod number) for the Name, CDP_Enable and 3.

LLDP_Disable policies and click Submit.

Note: All other policies will automatically leverage default values.

Note: If Attachable Access Entity Profile (AEP) was already created this would be the place to connect it. We will connect the AEP and Interface Policy Group later when we create the AEP.

74

Right click on Profiles and select Create Interface Profile. 4.

Type ip_AccX_L2_N5K as a name (where X is your pod number) and click + to specify 5.

Interface Selectors.

Enter the Interface Name and Interface ID based on the table below, Select the 6.

TX_L2_Out Access Port Policy Group you just created (where X is your pod number)

and click OK.

Pod Name Int ID Pod Name Int ID

Pod1 Port17 1/17 Pod9 Port25 1/25








Note: The Screenshot below is from pod5.

75

Click SUBMIT to complete Interface Profile. 7.

Note: We just created the Interface Profile for a single interface that will tie to the Nexus 5548 via the port specified in the table. Next we will define from which Switch we will use this port.

Expand Switch Policies->Profiles and click on the Profile Leaf101. 8.

Note: According to best practices, the Leaf101 placeholder for anything connecting just to this Switch/Node/Leaf was created ahead of time. Details can be found in the ACI Lab setup and connectivity document.

Click + next to the Associated Interface Selector Profile to associate the Interface 9.

Profile with the Switch Profile.

Select Interface Profile ip_AccX_L2_N5K created in the previous steps and click 10.

Submit.

76

Up until now we have configured all tasks on the left hand side of the graph that were referenced

at the beginning of this module.

In the final steps we will:

1) Create the VLAN Pool and the L2 Domain for Nexus connectivity.

2) Tie that L2 Domain with the Interface Policy Group association using an Attachable

Access Entity Profile (AEP).

Right click on Pools and select Create VLAN Pool. 11.

Type vp_TX_L2Out (where X is your pod number) as a Name, select Static Allocation 12.

and click + to specify the VLAN range.

77

We are using a single VLAN with the number 2000+X (where X is your pod number) per 13.

pod. Type 2000+X in both Range spots and click OK.

Example: For pod 13 vlans will be 2013-2013.

Note: The following screenshot shows input for pod5. This VLAN will operate between the ACI Border Leaf and the Nexus 5548 Switch.

Click Submit. 14.

Expand Physical and External Domains, right click on External Bridged Domains and 15.

select Create Layer 2 Domain.

Type TX_L2_ExtDom (where X is your pod number) as a name, select vp_TX_L2Out-16.

static for the VLAN pool, select TenantX for a Security Domain and click SUBMIT.

Now we will create the Attachable Access Entity Profile to connect the Bridge Domain and

VLANs with the Interface properties we created earlier in this task.

78

Expand Global Policies, right click on Attachable Access Entity Profiles and select 17.

Create Attachable Access Entity Profile.

Type TX_AEP as AEP name and click + to associate with the domain. 18.

Select TX_L2_ExtDom as the L2 External domain we want to tie to this AEP and click 19.

Update.

Click NEXT. 20.

Select All in the Select Interface choice for TX_L2_Out Interface Policy Group and 21.

click FINISH.

Note: If you select All for the wrong Interface Policy Group you will need to cancel out of the wizard and start over.

This concludes the setup for the access policy to enable the connection between the ACI fabric

and the Nexus Switching infrastructure.

So far we have the EPG that defines the App_Servers and Web_Servers. To enable connectivity

to the outside L2 network for the applications, we need to create an EPG to represent the outside

L2 network and use contracts to allow classified traffic between the EPGs.

Do not forget to select

All

79

Task 2: Enable and Verify App Traffic with the External L2

In the previous Task, we provided policies for overall connectivity between ACI fabric and

Nexus Switch.

In this Task, we will define and enable connectivity from the Tenant and Application Network

Profile (ANP) perspective.

Go to your Tenant TenantX (where X is your pod number). 22.

Expand Tenant TenantX->Application Profiles and click on TX_AppProfile. 23.

Verify that this Application profile consists of the two EPGs you created previously. 24.

Note: As of ACI version 1.2 we can connect EPG’s, external L2 and L3 and various existing VMM domains with contracts using drag and drop method.

Expand Networking, right click on External Bridged Networks and select Create 25.

Bridged Outside.

80

Enter the following information and click ADD (Do not click Next yet!). 26.

Name: TX_L2_Out (Where X is your pod number)

External Bridged Domain: TX_L2_ExtDom

Bridge Domain: TenantX/VMData-App

Encap: See table below

Path Type: Port

Path: See table below

Note: The vlan must be in the vlan pool you created earlier or you will get an invalid-vlan fault. Screenshot is from pod5.

Note: The process is simplified in this ACI version 1.2 compared to the older versions.

Pod Encap Path Pod Encap Path

Pod1 vlan-2001 Node-101/eth1/17 Pod9 vlan-2009 Node-101/eth1/25








Click Next. 27.

81

To create an External EPG, click + next to the External EPG Networks. 28.

For the Name, type TX_L2Out_EPG (where X is your pod number) and click OK. 29.

Click Finish. 30.

We defined the external Bridged Domain, the Bridge Domain that is on the ACI side and EPG

for our external L2 connectivity within the Tenant. To enable traffic we have to associate policy

between external L2 EPG and Server’s EPGs.

Expand External Bridged Networks->TX_L2_Out->Networks and click on 31.

TX_L2Out_EPG.

Note: Next you will add Provided and Consumed contracts to L2Ext EPG. Here we will reuse the App_Contract from the previous Module to allow ICMP. Depending on what services existing servers were providing you could create new contract/filters or even choose the default contract to allow all traffic types.

82

Click + next to Provided Contracts, select the App_Contract and click Update. 32.

Click the + next to Consumed Contracts, select the App_Contract and click Update. 33.

Click Submit. 34.

If Policy Usage Warning window pops up, confirm by selecting Submit Changes. 35.

Expand Tenant TenantX->Security Policies->Contracts and click on App_Contract. 36.

Observe the newly formed relationships with the TX_L2Out_EPG.

Note: EPG TX_L2Out_EPG now has a relationship with App_Servers EPG using the App_Contract. We can test that relationship by reaching outside IP address.

83

Go back to your vCenter. If you are logged out, IP address and credentials are below. 37.

Pod vCenter IP User Password Pod vCenter IP User Password









From the Hosts and Clusters menu, right click on App-ServerX and Open Console. 38.

If you were logged out, log back in using student/1234Qwer credentials and Ping 39.

192.168.11.200

Note: 192.168.11.200 is IP address of the Loopback on Nexus 5548 simulating host beyond it.

Congratulations! You have successfully attached external L2 domain to the ACI fabric.

84

Module 7: Connecting Ext. Layer 3 Connectivity

In the previous Module, we connected ACI fabric to the external L2 network. Whether we want

to connect to the WAN or we just need routing within our Data Center, the external L3

connectivity will be needed.

In this Module, you will create an External Layer 3 connection and import networks from the

external OSPF Domain. Nexus 5548 Layer Switch connected to the Leaf2 will present that

external OSPF source.

To enable application traffic we will create an external L3 EPG and establish its relationship

with the other EPGs using contract.

Note: If you want to save lab time, you can configure this module task by task using Postman. Please check the Module 2 for details how to use this tool.

In ACI version 1.1, there are a few major additional to the external L3 connectivity.

- In addition to iBGP and NSSA OSPF, as of 1.1 we can use EIGRP, OSPF regular area

and eBGP for the external L3 connectivity.

- As of version 1.1, ACI fabric allows for Transit routing.

For more details check release notes.

Task 1: Create BGP Route Reflector

To propagate externally learned L3 addresses within ACI fabric, Multiprotocol BGP (MP-BGP)

is used. To support large number of Leaf Switches, BGP route reflector technology is deployed.

More details can be found in Connecting Application Centric Infrastructure (ACI) to Outside

Layer 2 and 3 Networks.

Note: Since there can be only one Route Reflector per ACI Fabric, this configuration was done ahead of time. Two Spines are Route Reflectors for the ACI fabric. For more details consult ACI Lab Setup and Connectivity document.

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/1-x/release/notes/apic_rn_111.html

http://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/application-centric-infrastructure/white-paper-c07-732033.html#_Toc395143552

http://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/application-centric-infrastructure/white-paper-c07-732033.html#_Toc395143552

85

Task 2: Create Layer 3 External Connectivity

In this task, you will enable External Routing using OSPF within your Tenant. We will connect

ACI Fabric with Nexus 5548 and create needed Networking constructs for that connectivity,

such as interface profile, Node profile, EPG and OSPF parameters.

In the following task we will focus on application connectivity aspect via Application Network

Profile (ANP) to enable communication between external L3 and our Web-Server EPGs.

Navigate to your Tenant. Expand Tenant TenantX->Networking, right click on 1.

External Routed Networks and select Create Routed Outside.

Enter Name N5K-L3-Out, check OSPF checkbox with OSPF Area ID 1. Select 2.

TenantX/TX_Production (where X is your pod number) as a VRF and click + under

Nodes and Interfaces Protocol Profiles.

Note: TX_Production is our Tenant’s private network (VRF) that we created in the Module 3.

86

For the Node Name, enter Leaf102 and click + next to Nodes. 3.

Select Leaf102 (Node-102) as a Node ID, enter 1.0.0.4 for Router ID and click OK. 4.

Note: Defining OSPF properties of the node (Leaf).

Click + to create OSPF Interface Profile. 5.

Enter Name N5K-L3-OSPF-Leaf102 and click + for ROUTED INTERFACES. 6.

Note: Defining OSPF properties of the interface.

87

Refer to the following chart to select the proper interface on the Leaf102 as a Path. 7.

Enter 172.16.2.2/24 as the IP address, MTU as 1500 Bytes and click OK.

Note: Making sure that MTU matches N5K configuration ensures that OSPF neighboring can be established.

Pod Interface Pod Interface Pod Interface Pod Interface

Pod1 1/17 Pod5 1/21 Pod9 1/25 Pod13 1/29

Pod2 1/18 Pod6 1/22 Pod10 1/26 Pod14 1/30

Pod3 1/19 Pod7 1/23 Pod11 1/27 Pod15 1/31

Pod4 1/20 Pod8 1/24 Pod12 1/28 Pod16 1/32

Click OK to finish the creation of the Interface Profile. 8.

Click OK to finish the creation of the Node Profile. 9.

88

Click Next to proceed to Step 2 of the creation of External Routed Networking. 10.

Click + next to External EPG Networks to create EPG for L3 External Network. 11.

Enter Name L3-Out-EPG and click + under Subnet. 12.

Allow all subnets; enter 0.0.0.0/0 and click OK. 13.

Note: In this step we are selecting which outside subnets will belong to this EPG.

We can have a small number of EPGs associated with the same L3 external network and enforce different connectivity rules based on which EPG those external networks belong to.

In this lab, we will have a single EPG and we are associating all external networks to this EPG.

Other options are related with transit routing feature enabled in ACI version 1.1. For more details consult this ACI knowledgebase article or click Info icon.

Note2: For more details about the scope options check this link.

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/kb/b_KB_Transit_Routing.html

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/1-x/aci-fundamentals/b_ACI-Fundamentals/b_ACI-Fundamentals_chapter_010100.html#concept_FF7C6B8316864F03A1CC032B3ADE2F0B

89

Click OK to complete the Create External Network part. 14.

Click Finish to finish configuring External Routed Network. 15.

Click on Tenant TenantX->Networking and notice the new External Routed Network 16.

N5K_L3_Out we just created related with our Tenant’s private network TX_Production.

Expand Networking->External Routed Networks -> N5K-L3-Out as shown on the 17.

picture below and explore all the objects you just created.

The basic L3 connectivity and EPGs are created in this task. We will focus on enabling our

application to connect with the External L3 in the following task.

90

Task 3: Enable App Traffic with the External L3

We established general connectivity with external L3 in the previous task. In this task, the focus

is on enabling communication between Web_Servers EPG and L3 Ext_EPG via Contract.

We will start by creating several Filter entries that will be part of the Contract.

Note: We are doing a very similar configuration as what we did when we created the App_Services contract earlier in the lab, just without using wizard.

Expand Security Policies, right click Filters and select Create Filter. 18.

Enter Name Web_Service_Ports and click + to add the first Entry. 19.

Name Filter Entry ICMP, EtherType is IP and IP Protocol is icmp. Click Update. 20.

Click + to add the second entry. Name it SSH, EtherType IP, IP Protocol is tcp. Set the 21.

Destination Port/Range to 22 by typing values directly in both fields and click Update.

Note: Option Stateful refers to the new distributed stateful firewall feature enforced by Cisco AVS (Application Virtual Switch). For more details check this article.

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/1-x/virtualization/b_ACI_Virtualization_Guide/cisco_application___virtual_switch.html#concept_E89432FC9DDF4F45A3AFB0EA826A7DEA

91

Click + to add the third entry. Name it HTTP, EtherType IP, IP Protocol is tcp. Set the 22.

Destination Port/Range to http from the drop down menu and click Update.

SUBMIT to complete the Filter. 23.

Next we will create a new Contract that uses this Filter. Expand Security Policies, right 24.

click Contracts and select Create Contract.

Enter Name Web_Contract and click + to add a Subject. 25.

Enter Name Web_Subject. Check both Reverse Filter Ports and Apply Both Directions. 26.

Click + and add filter TenantX/Web_Service_Ports you just created. Click Update.

Click OK to Create Contract Subject. 27.

Click Submit to create the Contract. 28.

92

Next we will associate Contract with EPGs. There are several ways we can do that.

So far in the lab, we associated contract with EPG during the creation of the application profile

in module 4. In module 6 we associated contract by designating the specific EPG to be either

provider or consumer of the contract or both.

In this module we will use drag and drop method enabled as of version 1.2.

All of the above methods do the same, and choice is matter of personal preference.

We want Web_Servers EPG to dictate what kind of traffic can get to it from the outside, i.e. to

Provide Contract. L3 External EPG will consume that contract.

Expand Tenant TenantX->Application Profiles and click on TX_AppProfile. Click on 29.

L3 icon and drag it to the area below.

Note: Expand the browser window if you cannot see all of the icons.

Select TenantX/N5K-L3-Out/L3-Out-EPG for the External Network and click OK. 30.

Click on Contract Icon, drag it first to the Web_Servers EPG and then to the L3-Out-31.

EPG.

1

2

93

Verify that L3-Out-EPG is Consumer EPG and epg-Web_Servers is Provider EPG. 32.

Select Choose An Existing Contract radial button, select TenantX/Web_Contract,

uncheck No Filter and select Web_Contract/Web_Subject as a Contract Subject.

Click OK.

Click on Contract Icon to verify provider/consumer relationship and finish configuration 33.

by clicking Submit.

Note: As a challenge, can you find where within your tenant you can see all contracts associated with L3-Out-EPG? Can you find L3-Out-EPG object?

So far we configured ACI OSPF properties to connect with the external L3 and configured

Application Profile to include contract between L3_Out and Web_Servers EPGs.

The last step is to define which subnets will be advertised via OSPF. Since Bridge Domains are

carriers of the subnet information, this is what we will configure next.

94

Expand Networking->Bridge Domains and click on VMData-Web. Select L3 34.

Configuration tab and click + for Associated L3 Outs.

Note: We are binding L3 Outside routed network to VMData-Web bridge domain.

Select TenantX/N5K-L3-Out and click Update. 35.

Expand Networking->Bridge Domain->VMData-Web->Subnets and click on the 36.

subnet 192.168.10.1/24.

Under Scope, check Advertised Externally and click Submit. 37.

95

If Policy Usage Warning pops up, click Submit Changes. 38.

Repeat the same Bridge Domain modification steps for the VMData-App. Associate 39.

TenantX/N5K-L3-Out as L3-Out with VMData-App and advertise externally subnet

192.18.11.1/24.

We are done configuring External L3 Connectivity using OSPF. In the next task we will verify

our configuration.

Task 4: Verify External L3 Connectivity

First we will check OSPF connectivity between the Border Leaf and Nexus 5548.

In the APIC GUI, navigate to Fabric->Inventory. 40.

Expand POD 1->Leaf102->Protocols. Select OSPF and click on the Operational tab. 41.

Verify that OSPF process for your tenant TenantX (where X is your pod number) is UP.

Expand OSPF and click on Ospf for VRF-TenantX: TX_Production (where X is your 42.

pod number). Verify that the state with the external Nexus 5548 is Full.

96

Expand OSPF Databases and select Routes. Verify the presence of the routes 0.0.0.0/0, 43.

172.16.2.0/24, and 10.0.X.1/32 (where X is your pod number).

Note: 10.0.X.1 is N5K loopback interface simulating remote host’s IP address.

Return to vSphere, go to the Web_Server console and ping the IP Prefix 10.0.X.1 (where 44.

X is your pod number). This should be successful.

Note: It might take some time for all OSPF and MP-BGP info to propagate.

From the App_Server console ping the IP Prefix listed in the table above. This ping 45.

should fail though we did make the App_Server subnet public.

Note: Consider why this ping fails but the previous one succeeds. The answer is in a few steps.

We will now test connectivity from the Nexus side.

97

Connect to the N5K using the Putty SSH client on your desktop. Use IP address 46.

192.168.199.11, Username: student and Password:1234Qwer.

Explore routing and set up of Nexus 5548. Perform testing from the Nexus side. 47.

Note: Your access is limited to show, ping and ssh commands.

Suggested commands:

# show run ospf # show ip route vrf tenantX (notice that both subnets 192.168.10.0/24 and

192.168.11.0/24 are in the routing table, so that is not a reason of failed ping.)

# show ip ospf interface brief vrf tenantX

# ping 192.168.10.(100+X) vrf tenantX

(X is your pod number, Example: for pod 2 fourth octet is 102)

# ssh [email protected].(100+X) vrf tenantX (password is 1234Qwer, type exit to

close ssh session)

# ssh [email protected].(100+X) vrf tenantX (this one will fail, why?)

Note: The answer for SSH is the same as it is for failed ping. The ssh to the Web_Server is successful because there is a contract (which includes ssh) between the External L3 EPG and the Web_Servers EPG.

Ping and SSH to the App Server fail since there are no contracts between corresponding EPGs. That is the property of the whitelist models, no contracts, no data traffic allowed.

Telnet to the non-Border Leaf 101 at 192.168.199.6 using credentials admin/1234Qwer 48.

and explore routing from the Leaf 101 perspective.

Suggested Commands:

# show vrf all

# show ip route vrf TenantX:TX_Production (replace 2 X’s with your pod number)

Note: You can use the output of show vrf all command to specify vrf by copying and pasting.

# show ip route vrf overlay-1

CONGRATULATIONS! You completed the ACI Fundamentals lab!

98

Appendix A: Troubleshooting

T1. Host and vCenter credentials issue

Symptom: Simulator VM is not visible or there are issues when associating ACI with vCenter.

When accessing vCenter, there is a yellow triangle next to the host’s name.

Cause: Occasionally during the lab reset process vCenter and Host lose credentials trust.

Solution: Reconnect the host to vCenter.

Right-Click on Host and select Disconnect. 1.

Click Yes to confirm. 2.

Right-Click on Host and select Connect. 3.

Note: Due to disconnect, host and VMs will be greyed out.

Click Yes to confirm. 4.

The error message will pop up. Click Close. 5.

99

New window will pop up. Enter username: root password: 1234Qwer and click Next. 6.

Click Yes when security warning pops up. 7.

Click Next, Next and Finish. 8.

The host is successfully reconnected with the vCenter.

aci fundamentals lab guide 2 -...

Documents