1 introduction - configure out of band management …aci solutions team by tomas de leon...

ACI Solutions Team by Tomas de Leon ([email protected]) 3

1 Introduction - Configure Out Of Band management for the ACI Fabric

The following lab sections will introduce you to some of the common infrastructure components of the fabric. The sections in this lab represent some of the common configuration ACI cases that you may get. The fabric components in this lab are crucial to successful management and fabric operations. Please refer to the ACI Bootcamp reference material provided for each feature section. The material should provide some guidance on configuration and troubleshooting tasks if needed. For this lab, you will setup Node Management Addresses for your designated Fabric. As you may be aware, there are two methods of management access for the ACI Fabric: In-Band Management Access and Out of Band Management Access. This lab will focus on configuring Out of Band Management Access for your designated Fabric. For more details on In-Band Management Access and Out of Band Management Access please refer to the provided materials or online documentation and guides for this topic. In summary, Node Management Addresses are in a managed group which contain a set of nodes that are going to participate in the management network. There are two configuration methods for creating Node Management Addresses for the ACI Fabric: Create Node Management Addresses and Create Static Node Management Addresses. At FCS and subsequent APIC firmware versions in the 1.0(1x) release tree, the only configuration method for creating Node Management Addresses is "Create Node Management Addresses". The "Create Static Node Management Addresses" method was added in APIC firmware version 1.0(2x) release tree. Note: Regardless of which APIC firmware version that is running, Creating a Static Address to a single node is achievable. In section 2 Lab Reference & Topology Information, you have been provided a table of Out of Band (OOB) addresses for your designated Fabric. Please use the enclosed fabric information for the POD1 in your fabric pod assignments. This lab will:

1. Configuring Out-of-Band Management Access Using the APIC Admin GUI 2. Create Node Management Addresses for the Fabric APICs and Switches. 3. The Node Management Addresses assignment tasks will utilize "specific"

and "range" grouping for Nodes. 4. The Node Management Addresses will be created for the Out of Band

Management Network


Note: In a Subsequent Lab, You will have a task to "Statically" assign OOB management addresses for all nodes in your designated Fabric.

LAB - Configuring Out-of-Band Management Access Using the GUI Note: APIC Firmware Version 1.0(1n) is running in Fabric at this point in the lab.

2 Lab Reference & Topology Information For the following sections in this lab, please use the following fabric information for the POD1 in your fabric pod assignments.

Device\Entity NodeID Fabric 1 Fabric 2 APIC 1 (OOB IP Address) 1 10.122.254.211 10.122.254.141 APIC 2 (OOB IP Address) 2 10.122.254.212 10.122.254.142 APIC 3 (OOB IP Address) 3 10.122.254.213 10.122.254.143 Spine 1 (OOB IP Address) 201 10.122.254.244 10.122.254.130 Spine 2 (OOB IP Address) 202 10.122.254.245 10.122.254.131 Leaf 1 (OOB IP Address) 101 10.122.254.241 10.122.254.128 Leaf 2 (OOB IP Address) 102 10.122.254.242 10.122.254.135 Leaf 3 (OOB IP Address) 103 10.122.254.243 10.122.254.136 Leaf 4 (OOB IP Address) 104 10.122.254.154 10.122.254.137 OOB Default Gateway 10.122.254.1 / 24 10.122.254.1 / 24

Device\Entity Fabric 1 Fabric 2 DNS Server 1 (Preferred) 64.102.6.247 64.102.6.247 DNS Server 2 173.37.87.157 173.37.87.157 DNS Server 3 171.70.168.183 171.70.168.183 DNS Server 4 161.44.124.122 161.44.124.122 DNS Domain 1 (Preferred) cisco.com cisco.com DNS Domain 2 insieme.local insieme.local

Device\Entity Fabric 1 Fabric 2 NTP Server 1 (Preferred) 172.18.108.15 172.18.108.15 NTP Server 2 10.81.254.202 10.81.254.202 NTP Server 3 171.68.38.66 171.68.38.66 NTP Server 4 10.81.254.131 10.81.254.131


3 Configure a "Static" Out-of-Band Node Management Address for each APIC Nodes

LAB - Configuring Out-of-Band Management Access Using the GUI Note: APIC Firmware Version 1.0(1n) is running in Fabric at this point in the lab. On the menu bar, choose TENANTS > mgmt. In the Navigation pane, expand Tenant mgmt. Right-click Node Management Addresses, and click Create Node Management Addresses. In the Create Node Management Addresses dialog box, perform the following actions:

3.1. In the Policy Name field, enter a policy name. (Use "apic1-oob" for APIC1 (node 1), "apic2-oob" for APIC2 (node 2), and "apic3-oob" for APIC3 (node 3))

3.2. For the Select Nodes By field, select "Specific" 3.3. In the Nodes field, check the check box next to the appropriate APIC node (1,

2, 3...) 3.4. In the Config field, check the check box for Out of-Band Addresses. Note

The Out-of-Band IP addresses area is displayed. 3.5. In the Out-of-Band Management EPG field, choose the EPG from the drop-

down list (default). 3.6. In the Out-of-Band Gateway field, enter the IP address. (refer to OOB

Address Table for your designated Fabric in Section 2 above) 3.7. In the Mask field, enter the mask if it is not already assigned. 3.8. In the Out-of-Band IP Addresses fields, enter the range of IP addresses that

will be assigned to a SPECIFIC APIC. (note: To statically assign an OOB Node Management Address in this APIC firmware release tree, you create a range of 1 address)

3.9. Click SUBMIT. 3.10. Click YES to CONFIRM.

The node management IP address is configured for the APIC. Repeat Task for Section 3 for the remaining APICs in your designated Fabric.


Section 3 - Screenshot examples


Section 3 - Screenshot examples (cont.)

4 Create a "Pool" of Out-of-Band Node Management Addresses for a "Range" of Node Switches.

This lab section is similar to Section 3, but you will create a "Pool" of Out-of-Band Node Management Addresses for a "Range" of Node Switches. NOTE: Please pay close attention to section 2 Lab Reference & Topology Information, you have been provided a table of Out of Band (OOB) addresses for your designated Fabric. Please use the enclosed fabric information for the POD1 in your fabric pod assignments. I apologize in advance for any confusion but the address space is not contiguous. As a result you will have more then one RANGE for this task. Each FABRIC is different. I provided samples for RANGE segmentation. For example, Fabric1 Pod

Policy Name Node ID(s) IP Address Pools node-range1 101-103, 201-202 10.122.254.241-.245 node-range2 104 10.122.254.154

For example, Fabric2 Pod Policy Name Node ID(s) IP Address Pools node-range1 101 10.122.254.128 node-range2 201-202 10.122.254.130-.131 node-range3 102-104 10.122.254.135-.137


On the menu bar, choose TENANTS > mgmt. In the Navigation pane, expand Tenant mgmt. Right-click Node Management Addresses, and click Create Node Management Addresses. In the Create Node Management Addresses dialog box, perform the following actions:

4.1. In the Policy Name field, enter a policy name. (Use the Range Table listed above for your designated Fabric Pod)

4.2. For the Select Nodes By field, select "Range". 4.3. In the Nodes field, click on the "+" to add the Node "Range". The Node

Range may vary based on your contiguous IP Address Pool. Click "UPDATE" to add the Node Range.

4.4. In the Config field, check the check box for Out of-Band Addresses. Note The Out-of-Band IP addresses area is displayed.

4.5. In the Out-of-Band Management EPG field, choose the EPG from the drop-down list (default).

4.6. In the Out-of-Band Gateway field, enter the IP address. (refer to OOB Address Table for your designated Fabric in Section 2 above)

4.7. In the Mask field, enter the mask if it is not already assigned. 4.8. In the Out-of-Band IP Addresses fields, enter the range of IP addresses that

will be assigned to a SPECIFIC NODE RANGE. (note: Just a reminder that an OOB Node Management address pool can be a range of 1 ip address)

4.9. Click SUBMIT. 4.10. Click YES to CONFIRM. 4.11. In the Navigation pane, expand Node Management Addresses, and click

the policy that you created. In the Work pane, the out-of-band management addresses are displayed against the Node Switches.

The node management IP address is configured for the NODE RANGE. Repeat Task 2 for the remaining NODE RANGES in your designated Fabric.



5 Create an OOB Contract for the Node Management Out-of-Band EPG.

In the Navigation pane, expand Security Policies > Out-of-Band Contracts. Right-click Out-of-Band Contracts, and click Create Out-of-Band Contract. In the Create Out-of-Band Contract dialog box, perform the following tasks:

5.1. In the Name field, enter a name for the contract (oob-contract). 5.2. Expand Subjects. In the Create Contract Subject dialog box, in the Name field,

enter a subject name (oob-subject). 5.3. Expand Filters, and in the Name field, from the drop-down list, choose the

name of the filter (default). Click Update, and click OK. 5.4. In the Create Out-of-Band Contract dialog box, click Submit.

An out-of-band contract that can be applied to the out-of-band EPG is created.




5.5. In the Navigation pane, expand Node Management EPGs > Out-of-Band EPG - default.

5.6. In the Work pane, expand Provided Out-of-Band Contracts. 5.7. In the OOB Contract column, from the drop-down list, choose the out-of-band

contract that you created (oob-contract). Click Update, and click Submit. The contract is associated with the node management Out-of-Band EPG.



6 Create an External Network Instance Profile for the Out-of-Band Management Network.

In the Navigation pane, right-click External Network Instance Profile, and click Create External Management Entity Instance. In the Create External Management Entity Instance dialog box, perform the following actions:

6.1. In the Name field, enter a name (oob-mgmt-ext). 6.2. Expand the Consumed Out-of-Band Contracts field. From the Out-of-Band

Contract drop-down list, choose the contract that you created (oob-contract). Click Update. Choose the same contract that was provided by the out-of-band management.

6.3. In the Subnets field, enter the subnet address. (0.0.0.0/0) Click Update, and click Submit.

Only the subnet addresses you choose here will be used to manage the switches. The subnet addresses that are not included cannot be used to manage the switches.


7 Verify OOB Node Management Addresses configuration and connectivity for all Nodes in the Fabric.

Use the section 2 Lab Reference & Topology Information, you have been provided a table of Out of Band (OOB) addresses for your designated Fabric.

7.1 Using the Admin APIC GUI, verify the IP Address Blocks configured to the APICs and Node Switches. On the menu bar, choose TENANTS > mgmt. In the Navigation pane, expand Tenant mgmt. Under Tenant mgmt, select IP ADDRESS POOLS. In the Work pane, you can verify the GATEWAY ADDRESSES and ADDRESS BLOCKS for the APIC Controllers and NODE Switches.

Figure Task 7.1 - Screenshot example

7.2 Using the Admin APIC GUI, verify the Node Management Out-of-Band EPG default has an OOB Contract configured with the STATE "formed". Also verify that there are NO Configuration Issues and the Configuration State is "applied". On the menu bar, choose TENANTS > mgmt. In the Navigation pane, expand Tenant mgmt. Under Tenant mgmt, expand Node Management EPGs. Select Out-of-Band EPG - default. In the Work pane, you can verify the Configuration Issues, Configuration State, and Provided Out-of-Band Contracts.


Figure Task 7.2 - Screenshot example

7.3 Using the CLI on the APIC Controllers and Node Switches, verify the OOB IP Address configured, the IP Route Table, and test IP Connectivity.

7.3.1 SSH to an APIC and perform the following actions: • ip link | grep oobmgmt • ip route show | grep oobmgmt • ifconfig -a oobmgmt • ping < Default Gateway > • ping < DNS Server 1 (Preferred) 64.102.6.247 > • ping < NTP Server 1 (Preferred) 172.18.108.15 >

Task 7.3.1 – Display Output

TDELEON-M-205R:~ tdeleon$ ssh [email protected] Application Policy Infrastructure Controller admin@apic1:~> ip link | grep oobmgmt 7: bond1: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue

master oobmgmt state UP 29: oobmgmt: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP


admin@apic1:~> ip route show | grep oobmgmt default via 10.122.141.97 dev oobmgmt metric 16 10.122.141.96/27 dev oobmgmt proto kernel scope link src 10.122.141.98 10.122.141.97 dev oobmgmt scope link src 10.122.141.98 admin@apic1:~> ifconfig -a oobmgmt oobmgmt Link encap:Ethernet HWaddr F4:0F:1B:76:C9:DE inet addr:10.122.141.98 Bcast:10.122.141.127 Mask:255.255.255.224 inet6 addr: fe80::f60f:1bff:fe76:c9de/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:9536 errors:0 dropped:0 overruns:0 frame:0 TX packets:4379 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:5340455 (5.0 MiB) TX bytes:5556588 (5.2 MiB)

Use PING to test connectivity to:

• Default Gateway

admin@apic1:~> ping 10.122.141.97 PING 10.122.141.97 (10.122.141.97) 56(84) bytes of data. 64 bytes from 10.122.141.97: icmp_seq=1 ttl=255 time=10.4 ms 64 bytes from 10.122.141.97: icmp_seq=2 ttl=255 time=0.453 ms

• DNS Server 1 (Preferred) 64.102.6.247

admin@apic1:~> ping 64.102.6.247 PING 64.102.6.247 (64.102.6.247) 56(84) bytes of data. 64 bytes from 64.102.6.247: icmp_seq=1 ttl=246 time=0.731 ms 64 bytes from 64.102.6.247: icmp_seq=2 ttl=246 time=0.674 ms

• NTP Server 1 (Preferred) 172.18.108.15

admin@apic1:~> ping 172.18.108.15 PING 172.18.108.15 (172.18.108.15) 56(84) bytes of data. 64 bytes from 172.18.108.15: icmp_seq=1 ttl=59 time=2.73 ms 64 bytes from 172.18.108.15: icmp_seq=2 ttl=59 time=0.656 ms Note: Repeat the above PING tests on each of the APIC Controllers.


7.3.2 SSH to an APIC and attach to each of the switches and perform the following actions:

TDELEON-M-205R:~ tdeleon$ ssh [email protected] Application Policy Infrastructure Controller [email protected]'s password: admin@apic1:~> acidiag fnvread ID Name Serial Number IP Address Role State LastUpdMsgId

---------------------------------------------------------------------------------------

101 fab1-leaf1 SAL1819S0QX 192.168.0.95/32 leaf active 0

102 fab1-leaf2 SAL1817R818 192.168.0.127/32 leaf active 0

201 fab1-spine1 FGE18200AVQ 192.168.0.94/32 spine active 0

202 fab1-spine2 FGE18170ABF 192.168.120.95/32 spine active 0

admin@apic1:~> attach fab1-leaf1 # Executing command: ssh fab1-leaf1

On LEAF Node Switches, execute the following commands: • ip link | grep eth0 • ip route show | grep eth0 • ifconfig -a eth0 • ping < Default Gateway > • ping < DNS Server 1 (Preferred) 64.102.6.247 > • ping < NTP Server 1 (Preferred) 172.18.108.15 >

fab1-leaf1# ip link | grep eth0 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000 fab1-leaf1# ip route show | grep eth0 default via 10.122.141.97 dev eth0 10.122.141.96/27 dev eth0 proto kernel scope link src 10.122.141.103


fab1-leaf1# ifconfig -a eth0 eth0 Link encap:Ethernet HWaddr 50:87:89:a1:e0:d6 inet addr:10.122.141.103 Bcast:10.122.141.127 Mask:255.255.255.224 inet6 addr: fe80::5287:89ff:fea1:e0d6/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:10455 errors:0 dropped:0 overruns:0 frame:0 TX packets:1062 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:2539695 (2.4 MiB) TX bytes:90484 (88.3 KiB) Use PING to test connectivity to:

• Default Gateway fab1-leaf1# ping 10.122.141.97 PING 10.122.141.97 (10.122.141.97): 56 data bytes 64 bytes from 10.122.141.97: icmp_seq=0 ttl=255 time=1.104 ms 64 bytes from 10.122.141.97: icmp_seq=1 ttl=255 time=0.517 ms

• DNS Server 1 (Preferred) 64.102.6.247 fab1-leaf1# ping 64.102.6.247 PING 64.102.6.247 (64.102.6.247): 56 data bytes 64 bytes from 64.102.6.247: icmp_seq=0 ttl=246 time=0.756 ms 64 bytes from 64.102.6.247: icmp_seq=1 ttl=246 time=0.691 ms

• NTP Server 1 (Preferred) 172.18.108.15 fab1-leaf1# ping 172.18.108.15 PING 172.18.108.15 (172.18.108.15): 56 data bytes 64 bytes from 172.18.108.15: icmp_seq=0 ttl=59 time=2.288 ms 64 bytes from 172.18.108.15: icmp_seq=1 ttl=59 time=1.082 ms Note: Repeat the above PING tests on each of the LEAF Node Switches.


admin@apic1:~> attach fab1-spine1 # Executing command: ssh fab1-spine1

On SPINE Node Switches, execute the following commands: • ip link | grep eth6 • ip route show | grep eth6 • ifconfig -a eth6 • ping < Default Gateway > • ping < DNS Server 1 (Preferred) 64.102.6.247 > • ping < NTP Server 1 (Preferred) 172.18.108.15 >

fab1-spine1# ip link | grep eth6 8: eth6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000 fab1-spine1# ip route show | grep eth6 default via 10.122.141.97 dev eth6 10.122.141.96/27 dev eth6 proto kernel scope link src 10.122.141.101 fab1-spine1# ifconfig -a eth6 eth6 Link encap:Ethernet HWaddr e4:c7:22:bd:4e:2c inet addr:10.122.141.101 Bcast:10.122.141.127 Mask:255.255.255.224 inet6 addr: fe80::e6c7:22ff:febd:4e2c/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:10486 errors:0 dropped:0 overruns:0 frame:0 TX packets:1065 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:2547251 (2.4 MiB) TX bytes:90742 (88.6 KiB) Use PING to test connectivity to:

• Default Gateway fab1-spine1# ping 10.122.141.97 PING 10.122.141.97 (10.122.141.97): 56 data bytes 64 bytes from 10.122.141.97: icmp_seq=0 ttl=255 time=1.134 ms 64 bytes from 10.122.141.97: icmp_seq=1 ttl=255 time=0.690 ms

• DNS Server 1 (Preferred) 64.102.6.247 fab1-spine1# ping 64.102.6.247 PING 64.102.6.247 (64.102.6.247): 56 data bytes


64 bytes from 64.102.6.247: icmp_seq=0 ttl=246 time=0.741 ms 64 bytes from 64.102.6.247: icmp_seq=1 ttl=246 time=0.631 ms

• NTP Server 1 (Preferred) 172.18.108.15 fab1-spine1# ping 172.18.108.15 PING 172.18.108.15 (172.18.108.15): 56 data bytes 64 bytes from 172.18.108.15: icmp_seq=0 ttl=59 time=1.356 ms 64 bytes from 172.18.108.15: icmp_seq=1 ttl=59 time=0.637 ms Note: Repeat the above PING tests on each of the SPINE Node Switches.

Reference Material

• Configuring Out-of-Band Management Access Using the GUI http://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/1-x/getting-started/b_APIC_Getting_Started_Guide/b_APIC_Getting_Started_Guide_chapter_01.html#task_B269F0341EA744899DE3CF987DC0A42F

• Cisco Application Policy Infrastructure Controller (APIC) - Configuration

Examples and TechNotes http://www.cisco.com/c/en/us/support/cloud-systems-management/application-policy-infrastructure-controller-apic/products-configuration-examples-list.html

• Video: Cisco APIC - Configuring Out-of-Band Management Access

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/1-x/getting-started/video/cisco_apic_configure_oob_mgmt_access_using_gui.html

End of Document

1 introduction - configure out of band management …aci solutions team by tomas de leon...

Documents