openvswitch _ brezular's technical blog

Brezular's Technical Blog

My worthless personal blog about networking

Part4 – OPENVSWITCH – Playing with Bondingon Openvswitch

DECEMBER 4, 2011 15 COMMENTS (HTTP://BREZULAR.WORDPRESS.COM/2011/12/04/PART4-OPENVSWITCH-PLAYING-WITH-BONDING-ON-OPENVSWITCH/#COMMENTS)

i1 Vote

Bonding (h,p://en.wikipedia.org/wiki/Channel_bonding) is aggregation multiple links to single link inorder to increase throughput and achieve redundancy in case one of links fails. At least, officials say it so itmust be true. Now, let me ask the question. How can we configure bonding?

From what I know about bonding, they are two possibilities to configure it.

The first option (h,p://www.linuxfoundation.org/collaborate/workgroups/networking/bonding) is to loadbonding module to kernel and use ifenslave (h,p://linux.die.net/man/8/ifenslave) utility for itsconfiguration. Configuration steps for Microcore are described here.

h,p://brezular.wordpress.com/2011/01/20/how-to-setup-linux-microcore-3-x-router-qemu-image-in-fedora-linux-part2/ (h,p://brezular.wordpress.com/2011/01/20/how-to-setup-linux-microcore-3-x-router-qemu-image-in-fedora-linux-part2/)

If you want to have fun as I had with bonding configuration, check this lab for reference.

h,p://brezular.wordpress.com/2011/03/10/etherchannelvrrp-dhcp-ospf-configuration-cisco-vya,a-microcore/ (h,p://brezular.wordpress.com/2011/03/10/etherchannelvrrp-dhcp-ospf-configuration-cisco-vya,a-microcore/)

As the tutorial should be dedicated to bonding on Openvswitch (h,p://openvswitch.org/) I should focuson this topic. This is the second option how can we configure bonding without loading bonding module(h,p://www.linuxfoundation.org/collaborate/workgroups/networking/bonding) to Linux kernel.

Let’s say we have two Microcore Linux (h,p://distro.ibiblio.org/tinycorelinux/welcome.html) boxes

Openvswitch | Brezular's Technical Blog http://brezular.wordpress.com/category/openvswitch/

1 de 48 02/10/2013 08:01 a.m.

connected with two links. As usual, our topology is created using GNS3 (h,p://www.gns3.net/) andQemu (h,p://en.wikipedia.org/wiki/QEMU) images of Microcore with installed Openvswitch 1.2.2 andQuagga (h,p://www.quagga.net/) 0.99.20 on the top of Microcore.

(h,p://brezular.files.wordpress.com/2011/12/topology.png)There is nothing in the world what couldprevent you to download my Openvswitch Microcore Qemu image available here.

h,p://sourceforge.net/projects/gns-3/files/Qemu%20Appliances/linux-microcore-4.0-openvswitch-1.2.2-quagga-0.99.20.img/download (h,p://sourceforge.net/projects/gns-3/files/Qemu%20Appliances/linux-microcore-4.0-openvswitch-1.2.2-quagga-0.99.20.img/download)

As for Ethernet card I suggest you to use emulated i82557b Ethernet card. Hint – check GNS3 Qemuse,ing. According to Bill (h,p://en.wikipedia.org/wiki/Bill_Gates), 640 kB is enough for anyone but it isbe,er to assign at least 128 MB RAM for Microcore. Here is a snapshot of Qemu se,ing.


2 de 48 02/10/2013 08:01 a.m.

(h,p://brezular.files.wordpress.com/2011/12/qemu-se,ings.jpeg)

Part1 – Basic Microcore, Openvswitch and QuaggaConfiguration

Before we start to configure Openvswitch we should be aware of initial configuration requirement forOpenvswitch and Quagga. Frankly to say, Openvswitch is not Cisco IOS (h,p://en.wikipedia.org/wiki/Cisco_IOS) which you can configure right after device is booted. First you need(h,p://openvswitch.org/cgi-bin/gitweb.cgi?p=openvswitch;a=blob_plain;f=INSTALL.Linux;hb=HEAD) tocreate database, configuration files, start services etc.

My Qemu Microcore image with Openvswitch and Quaggae is preconfigured for such things so feel freeto use it.

1. Switch 1 configuration – hostname, trunks, vlan interfaces

tc@box:~$ echo "hostname Switch1" >> /opt/bootlocal.sh

tc@box:~$ sudo hostname Switch1

tc@Switch1:~$ sudo su

root@Switch1:~# ovs-vsctl add-br br0

root@Switch1:~# ovs-vsctl add-port br0 vlan10 tag=10 -- set interface vlan10type=internal


Note: I usually specify list of VLANs to be allowed on trunk link

root@Switch1:~# ovs-vsctl add-port br0 eth0 trunks=10,20

root@Switch1:~# ovs-vsctl add-port br0 eth1 trunks=10,20

Unfortunately if we do it now, we cannot create bond0 interface, later in Part2. In this case a warningmessage informs us that eth0 and eth1 already exists on bridge br0. Therefore we skip a trunk definitionhere and we will configure the trunk later together with bonding configuration.

Start Quagga vtysh (h,p://linux.die.net/man/1/vtysh) shell.

root@Switch1:~# vtysh

Hello, this is Quagga (version 0.99.20).

Copyright 1996-2005 Kunihiro Ishiguro, et al.

Switch1# conf t


3 de 48 02/10/2013 08:01 a.m.

Switch1(config)# interface vlan10

Switch1(config-if)# ip address 192.168.10.1/24

Switch1(config-if)# no shutdown

Switch1(config-if)#Switch1(config-if)# interface vlan20



Switch1(config-if)# do wr

Switch1(config-if)# exit

Switch1(config)# exit

Switch1# exit.

Save Microcore configuration.

root@Switch1:~# /usr/bin/filetool.sh -b

Ctrl+S

2. Switch 2 configuration – hostname, trunks, vlan interfaces

tc@box:~$ echo "hostname Switch2" >> /opt/bootlocal.sh

tc@box:~$ sudo hostname Switch2

tc@Switch1:~$ sudo su

root@Switch2:~# ovs-vsctl add-br br0



Start Quagga vtysh shell.

root@Switch2:~# vtysh



Switch2# conf t

Switch2(config)# interface vlan10



Switch2(config-if)#Switch2(config-if)# interface vlan20



Switch2(config-if)# do wr

Switch2(config-if)# exit

Switch2(config)# exit


4 de 48 02/10/2013 08:01 a.m.

Switch2# exit

root@Switch2:~#

Save Microcore configuration.


Ctrl+S

Part2 – Bonding

Now, it is time to start playing game called bonding. They are not any rules in our game but it is not badidea to read documentation first. Page number ten could answer many future question.

h,p://openvswitch.org/ovs-vswitchd.conf.db.5.pdf (h,p://openvswitch.org/ovs-vswitchd.conf.db.5.pdf)

As you can see, they are four bond modes with balance-slb as default mode. Modes are described on page11.

1. Switch1 and Switch2 configuration for static bonding

root@Switch1:~# ovs-vsctl add-bond br0 bond0 eth0 eth1 trunks=10,20

root@Switch2:~# ovs-vsctl add-bond br0 bond0 eth0 eth1 trunks=10,20

2. Static Bonding Testing

Check bond interface bond0 on Switch1.

root@Switch1:~# ovs-appctl bond/show bond0

bond_mode: balance-slbbond-hash-algorithm: balance-slbbond-hash-basis: 0updelay: 0 msdowndelay: 0 msnext rebalance: 9476 mslacp_negotiated: false

slave eth1: enabledmay_enable: true

slave eth0: enabledactive slavemay_enable: true

Now do the same for bond0 on Switch2.



5 de 48 02/10/2013 08:01 a.m.




Now, start pinging IP address 192.168.10.2 from Switch1. Ping is supposed to be working. While pinging,start tcpdump to listen on interface ethernet1 of Switch2

root@Switch2:~# tcpdump -i eth1

tcpdump: WARNING: eth1: no IPv4 address assignedtcpdump: verbose output suppressed, use -v or -vv for full protocol decodelistening on eth1, link-type EN10MB (Ethernet), capture size 68 bytes

11:47:07.214753 IP 192.168.10.2 > 192.168.10.1: ICMP echo reply, id 36360, seq 15, length 6411:47:08.215483 IP 192.168.10.2 > 192.168.10.1: ICMP echo reply, id 36360, seq 16, length 6411:47:09.215823 IP 192.168.10.2 > 192.168.10.1: ICMP echo reply, id 36360, seq 17, length 6411:47:10.216846 IP 192.168.10.2 > 192.168.10.1: ICMP echo reply, id 36360, seq 18, length 6411:47:11.217868 IP 192.168.10.2 > 192.168.10.1: ICMP echo reply, id 36360, seq 19, length 64

<output truncated>

15 packets captured15 packets received by filter0 packets dropped by kernel

Interface Ethernet1 sends icmp echo reply messages back to Switch1. As they are not any ICMP echorequests listed in the output, only the interface Ethernet0 receives these ICMP requests coming fromSwitch1.

My question is, what happens if we shutdown Ethernet0 on Switch2? Will be ICMP requests interruptedand possibly re-forwarded via interface Ethernet0 of Switch1? If yes, how long does it take?

root@Switch2:~# ifconfig eth0 down

Check the bond interface bond0 on Switch2 again. As soon as Switch2 detects failure of its interface,Openvswitch puts slave eth0 do disabled state


bond_mode: balance-slb


6 de 48 02/10/2013 08:01 a.m.

bond-hash-algorithm: balance-slbbond-hash-basis: 0updelay: 0 msdowndelay: 0 msnext rebalance: 7612 mslacp_negotiated: false

slave eth1: enabledactive slavemay_enable: truehash 52: 0 kB load

slave eth0: disabledmay_enable: false

Switch1 cannot detect failure of far end interface Ethernet0. It continues to send ICMP requests toSwitch2 via its Ethernet0 interface. Stop ping command and check bond interface bond0 on Switch1.Obviously, Ethernet0 is still enabled in bundle




slave eth0: enabledactive slavemay_enable: truehash 133: 0 kB load

This behaviour we cannot tolerate. We need such as mechanism which helps to detect far end failure.Therefore LACP must be deployed in your configuration.

According to wiki (h,p://en.wikipedia.org/wiki/Link_Aggregation_Control_Protocol#Link_Aggregation_Control_Protocol), LACP works bysending frames (LACPDUs) down all links that have the protocol enabled. These are two mainadvantages comparing to static bonding.

1. Failover when a link fails when the peer will not see the link down. With static link aggregation the peerwould continue sending traffic down the link causing it to be lost.

2. The device can confirm that the configuration at the other end can handle link aggregation. With Staticlink aggregation a cabling or configuration mistake could go undetected and cause undesirable networkbehaviour.


7 de 48 02/10/2013 08:01 a.m.

Uhm, it looks like exactly what we want. So do not hesitate and let’s go configure bonding with LACP.

3. Switch1 and Switch2 configuration for bonding with LACP

Delete existing bond0 port.

root@Switch1:~# ovs-vsctl del-port br0 bond0


Create a new bond interface.

root@Switch1:~# ovs-vsctl add-bond br0 bond0 eth0 eth1 lacp=active trunks=10,20

root@Switch2:~# ovs-vsctl add-bond br0 bond0 eth0 eth1 lacp=active trunks=10,20

Check bonding on Switch1.


bond_mode: balance-slbbond-hash-algorithm: balance-slbbond-hash-basis: 0updelay: 0 msdowndelay: 0 msnext rebalance: 7763 mslacp_negotiated: true

slave eth0: disabledmay_enable: false


Notice state of Ethernet0 interface. Remember, we shut-downed Ethernet0 on Switch2. Thanks to LACP,Ethernet0 is disabled in bundle on Switch1. It is awesome!

Note Even an interface Ethernet0 of Switch1 is disabled in bundle, it remains in up state

root@Switch1:~# ifconfig eth0

eth0 Link encap:Ethernet HWaddr 00:AA:00:D8:E6:00inet6 addr: fe80::2aa:ff:fed8:e600/64 Scope:LinkUP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1RX packets:112 errors:0 dropped:0 overruns:0 frame:0TX packets:2030 errors:0 dropped:0 overruns:0 carrier:0collisions:0 txqueuelen:1000RX bytes:36156 (35.3 KiB) TX bytes:336741 (328.8 KiB)

4. LACP bonding testing

The last point we want to test is how much time is required for transition for interface in bundle todisabled state, if far end fails. Logically it should depend how often are LACP messages exchanged


8 de 48 02/10/2013 08:01 a.m.

between peers.

Start tcpdump listening on eth0 of Switch1. Then bring interface eth0 up on Switch2. Right after LACPmessages are exchanged via eth0 between Switch1 and Switch2, Ethernet0 interface is brought to enabledstate in bundle. See the output below

root@Switch1:~# tcpdump -i eth0 -v

tcpdump: WARNING: eth0: no IPv4 address assignedtcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 68 bytes

< DHCP requests>

14:00:14.555758 IP (tos 0×0, ,l 64, id 0, offset 0, flags [none], proto UDP (17), length 317) 0.0.0.0.bootpc >255.255.255.255.bootps: BOOTP/DHCP, Request [|bootp]

<IPV6 messages coming from Switch1>

14:00:18.457096 00:aa:00:85:34:00 (oui Unknown) > 33:33:00:00:00:16 (oui Unknown), ethertype IPv6(0x86dd), length 110:0×0000: 6000 0000 0038 0001 0000 0000 0000 0000 `….8……….0×0010: 0000 0000 0000 0000 ff02 0000 0000 0000 …………….0×0020: 0000 0000 0000 0016 3a00 0502 ……..:…14:00:18.957535 00:aa:00:85:34:00 (oui Unknown) > 33:33:ff:85:34:00 (oui Unknown), ethertype IPv6(0x86dd), length 78:0×0000: 6000 0000 0018 3aff 0000 0000 0000 0000 `…..:………0×0010: 0000 0000 0000 0000 ff02 0000 0000 0000 …………….0×0020: 0000 0001 ff85 3400 8700 1173 ……4….s

<LACP message coming from Switch1. Notice System 00:00:00:00:00:00. It means that Switch2 has notknown about partner’s system yet>

14:00:19.217584 LACPv1, length: 110Actor Information TLV (0×01), length: 20System 00:aa:00:85:34:00 (oui Unknown), System Priority 65534, Key 5, Port 6, Port Priority 65535State Flags [Activity, Aggregation, Default]Partner Information TLV (0×02), length: 20System 00:00:00:00:00:00 (oui Ethernet), System Priority 0, Key 0, Port 0, Port Priority 0State Flags [none]Collector Information TLV (0×03), length: 16packet exceeded snapshot

<LACP message sent to Switch2. As Switch1 knows partner ID 00:aa:00:85:34:00 – MAC address ofinterface eth0 of Switch2>

14:00:19.217914 LACPv1, length: 110Actor Information TLV (0×01), length: 20System 00:aa:00:d8:e6:00 (oui Unknown), System Priority 65534, Key 5, Port 6, Port Priority 65535State Flags [Activity, Aggregation, Synchronization, Collecting, Distributing]Partner Information TLV (0×02), length: 20System 00:aa:00:85:34:00 (oui Unknown), System Priority 65534, Key 5, Port 6, Port Priority 65535


9 de 48 02/10/2013 08:01 a.m.

State Flags [Activity, Aggregation, Default]Collector Information TLV (0×03), length: 16packet exceeded snapshot

<LACP message sent to Switch1. Now, Switch1 knows partner’s ID 00:aa:00:d8:e6:00 – MAC address ofinterface eth0 of Switch1>

14:00:19.220189 LACPv1, length: 110Actor Information TLV (0×01), length: 20System 00:aa:00:85:34:00 (oui Unknown), System Priority 65534, Key 5, Port 6, Port Priority 65535State Flags [Activity, Aggregation, Synchronization, Collecting, Distributing]Partner Information TLV (0×02), length: 20System 00:aa:00:d8:e6:00 (oui Unknown), System Priority 65534, Key 5, Port 6, Port Priority 65535State Flags [Activity, Aggregation, Synchronization, Collecting, Distributing]Collector Information TLV (0×03), length: 16packet exceeded snapshot

Explanation

00:aa:00:85:34:00 – MAC address of eth0 interface of Switch200:aa:00:d8:e6:00 – MAC address of eth0 interface of Switch1:

Nice! We can see structure of LACP messages but still do not know anything about time required fortransition. It is time to check it.

Start pinging 192.168.10.2 from Switch1 and with the help of tcpdump find an interface where packetsenter Switch2. In my case it is Ethernet1. Shutdown this interface and cheek the output of ping command.You see, ping is interrupted and it took approximately 50 seconds for ping to be recovered. It means thatfor 50 seconds interface Ethernet1 of Switch1 had been in enabled state. It is too much. It must be a wayto short this time to acceptable level.

Issue the command below on Switch1. LACP statistic can be found there.

root@Switch1:~# ovs-appctl lacp/show bond0

lacp: bond0status: active negotiatedsys_id: 00:aa:00:d8:e6:00sys_priority: 65534aggregation key: 5lacp_time: slow

slave: eth0: current a,achedport_id: 6port_priority: 65535

actor sys_id: 00:aa:00:d8:e6:00actor sys_priority: 65534actor port_id: 6actor port_priority: 65535actor key: 5


10 de 48 02/10/2013 08:01 a.m.

actor state: activity aggregation synchronized collecting distributing

partner sys_id: 00:aa:00:85:34:00partner sys_priority: 65534partner port_id: 6partner port_priority: 65535partner key: 5partner state: activity aggregation synchronized collecting distributing

slave: eth1: current a,achedport_id: 5port_priority: 65535

actor sys_id: 00:aa:00:d8:e6:00actor sys_priority: 65534actor port_id: 5actor port_priority: 65535actor key: 5actor state: activity aggregation synchronized collecting distributing

partner sys_id: 00:aa:00:85:34:00partner sys_priority: 65534partner port_id: 5partner port_priority: 65535partner key: 5partner state: activity aggregation synchronized collecting distributing

Focus on lacp-time parameter which is set to slow. According to documentation (h,p://openvswitch.org/ovs-vswitchd.conf.db.5.pdf) it is:

The LACP timing used on this Port. Possible values are fast, slow and a positive number of milliseconds.By default slow is used. When configured to be fast LACP heartbeats are requested at a rate of once persecond causing connectivity problems to be detected more quickly. In slow mode, heartbeats arerequested at a rate of once every 30 seconds.

That is exactly what we need. Changing lacp_time from slow to fast should help us to detect failure ofpeer’s interface more quickly.

5. Switch1 and Switch2 configuration for bonding with LACP and lacp time set to fast

Delete existing bond interface bond0.



Create a new bond interface.

root@Switch1:~# ovs-vsctl add-bond br0 bond0 eth0 eth1 lacp=active other_config:lacp-time=fast trunks=10,20

root@Switch2:~# ovs-vsctl add-bond br0 bond0 eth0 eth1 lacp=activeother_config:lacp_time=fast trunks=10,20


11 de 48 02/10/2013 08:01 a.m.

Save configuration.



Ctrl+S

If you want to check interval of sending LACP messages, start tcpdump on Switch2 while pinging IPaddress of VLAN interface of Switch2 from Switch1. Now, LACP message are sent every 1 second

root@Switch2:~# tcpdump -i eth0

tcpdump: WARNING: eth0: no IPv4 address assignedtcpdump: verbose output suppressed, use -v or -vv for full protocol decodelistening on eth0, link-type EN10MB (Ethernet), capture size 68 bytes21:42:25.525007 LACPv1, length: 11021:42:26.529846 LACPv1, length: 11021:42:27.532092 LACPv1, length: 11021:42:28.534532 LACPv1, length: 11021:42:29.543080 LACPv1, length: 11021:42:30.545620 LACPv1, length: 11021:42:31.547848 LACPv1, length: 11021:42:32.550717 LACPv1, length: 11021:42:33.551271 LACPv1, length: 11021:42:34.553382 LACPv1, length: 110

10 packets captured10 packets received by filter0 packets dropped by kernel

6. LACP bonding testing with enabled lacp_time=fast option

Start pinging 192.168.10.2 from Switch1 and find interface receiving ICMP echo requests on Switch2. Inmy case it is interface eth0. While pinging, shutdown this interface and check ping statistic

root@Switch1:~# ping 192.168.10.2

PING 192.168.10.2 (192.168.10.2): 56 data bytes64 bytes from 192.168.10.2: seq=0 ,l=64 time=1.810 ms64 bytes from 192.168.10.2: seq=1 ,l=64 time=0.940 ms64 bytes from 192.168.10.2: seq=2 ,l=64 time=1.610 ms64 bytes from 192.168.10.2: seq=3 ,l=64 time=0.755 ms

<output truncated>

— 192.168.10.2 ping statistics —27 packets transmi,ed, 24 packets received, 11% packet lossround-trip min/avg/max = 0.755/3.378/48.484 ms

You see, three packet are lost. That is what we can call link redundancy if one of links fails.


12 de 48 02/10/2013 08:01 a.m.

Following commands helpful during troubleshooting.

Show MAC address table.

root@Switch1:~# ovs-appctl fdb/show br0

List ports.

root@Switch1:~# ovs-vsctl list port

END.

FILED UNDER OPENVSWITCH TAGGED WITH BONDING, CORE, LINUX TEAMING NIC,LLINUX TEAMING NETWORK, MICROCORE, TEAMING

Part3 – OPENVSWICH – Campus Model with Layer2Access, Built with Open-Source Applications

NOVEMBER 23, 2011 1 COMMENT (HTTP://BREZULAR.WORDPRESS.COM/2011/11/23/PART3-OPENVSWICH-CAMPUS-MODEL-WITH-LAYER2-ACCESS-BUILT-WITH-OPEN-SOURCE-APPLICATIONS/#COMMENTS)

i2 Votes

In part one (h,p://brezular.wordpress.com/2011/09/03/part1-openvswich-creating-and-submi,ing-openvswitch-extension-to-microcore-upstream/) we showed how to create Openvswitch(h,p://openvswitch.org/) extension and submit it to Microcore (h,p://distro.ibiblio.org/tinycorelinux/welcome.html) repository (h,p://distro.ibiblio.org/tinycorelinux/4.x/x86/tcz/). There were also presentedafter-install steps for Openvswitch, adapted for specific Microcore needs.

h,p://brezular.wordpress.com/2011/09/03/part1-openvswich-creating-and-submi,ing-openvswitch-extension-to-microcore-upstream/ (h,p://brezular.wordpress.com/2011/09/03/part1-openvswich-creating-and-submi,ing-openvswitch-extension-to-microcore-upstream/)

In part two (h,p://brezular.wordpress.com/2011/06/25/part2-openvswich-vlans-trunks-l3-vlan-interface-intervlan-routing-configuration-and-testing/) we did several tests in order to test feature such as vlans(h,p://en.wikipedia.org/wiki/Virtual_LAN), 8021q trunks (h,p://en.wikipedia.org/wiki/IEEE_802.1Q) andVLAN interfaces (h,p://www.itsyourip.com/cisco/how-to-create-vlan-interfaces-for-intervlan-routing-in-cisco-ios/) widely used in typical multilayer switches (h,p://en.wikipedia.org/wiki/Multilayer_switch).


13 de 48 02/10/2013 08:01 a.m.

h,p://brezular.wordpress.com/2011/06/25/part2-openvswich-vlans-trunks-l3-vlan-interface-intervlan-routing-configuration-and-testing/ (h,p://brezular.wordpress.com/2011/06/25/part2-openvswich-vlans-trunks-l3-vlan-interface-intervlan-routing-configuration-and-testing/)

In part three we are going to create Campus Model (h,p://www.mcmcse.com/cisco/guides/hierarchical_model.shtml) using Linux Microcore with installed Openvswitch extension. As we need toenable routing between Distribution and Core layer (h,p://en.wikipedia.org/wiki/Cisco%27s_3_Layered_Model) to follow reccomendations of design guide, we have to install Quagga(h,p://www.quagga.net/) open-source routing suite.

Keepalived (h,p://www.keepalived.org/) extension helps us to achieve default gateway’s redundancyusing VRRP (h,p://en.wikipedia.org/wiki/Virtual_Router_Redundancy_Protocol) protocol, in case offailure of default gateway. Other extensions such as iproute2 (h,p://en.wikipedia.org/wiki/Iproute2),tcpdump (h,p://en.wikipedia.org/wiki/Tcpdump) are not neccessary but useful and they are installed inour Qemu image (h,p://en.wikibooks.org/wiki/QEMU/Images).

Here is my own linux-microcore-3.0.3 Qemu image with installed Openvswitch 1.2.2, Quagga 0.99.20 andKeepalived 1.2.2 as extensions.

h,p://sourceforge.net/projects/gns-3/files/Qemu%20Appliances/linux-microcore-4.0-openvswitch-1.2.2-quagga-0.99.20.img/download (h,p://sourceforge.net/projects/gns-3/files/Qemu%20Appliances/linux-microcore-4.0-openvswitch-1.2.2-quagga-0.99.20.img/download)

h,p://www.4shared.com/file/CAi_zNuO/linux-microcore-40-openvswitch.html (h,p://www.4shared.com/file/CAi_zNuO/linux-microcore-40-openvswitch.html)

I had to recompile the kernel because a default Microcore kernel is not compiled with Multipath option.Read more here.

h,p://brezular.wordpress.com/2011/11/13/part1-recompiling-linux-microcore-4-0-2-for-equal-cost-routes/(h,p://brezular.wordpress.com/2011/11/13/part1-recompiling-linux-microcore-4-0-2-for-equal-cost-routes/)

The image also contains tcpdump and iproute2 command. I recommend you to use my Qemu imageotherwise additional after install configuration for Quagga and Openvswitch is required. At least youshould put following commands to /opt/bootlocal.sh

# put other system startup commands here

#Load modules to kernel

modprobe openvswitch_mod

modprobe ipv6

#Start ovsdb-server

ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock

--remote=db:Open_vSwitch,manager_options --private-key=db:SSL,private_key

--certificate=db:SSL,certificate --bootstrap-ca-cert=db:SSL,ca_cert --pidfile

--detach

#Initialize databaseovs-vsctl –no-wait init

#Start vswitchd daemon


14 de 48 02/10/2013 08:01 a.m.

ovs-vswitchd –pidfile –detach

#Enable forwarding between interfacessysctl -w net.ipv4.ip_forward=1sysctl -w net.ipv6.conf.all.forwarding=1

#Start Quagga routing daemons/usr/local/sbin/zebra -u root -d -f /usr/local/etc/quagga/zebra.conf/usr/local/sbin/ripd -u root -d -f /usr/local/etc/quagga/ripd.conf/usr/local/sbin/ripngd -u root -d -f /usr/local/etc/quagga/ripngd.conf/usr/local/sbin/ospfd -u root -d -f /usr/local/etc/quagga/ospfd.conf/usr/local/sbin/ospf6d -u root -d -f /usr/local/etc/quagga/ospf6d.conf/usr/local/sbin/bgpd -u root -d -f /usr/local/etc/quagga/bgpd.conf/usr/local/sbin/isisd -u root -d -f /usr/local/etc/quagga/isisd.conf

Note: 8021q module does not have to be loaded to get trunk working with Openvswitch. It has to beloaded when creating sub-interfaces using vconfig (h,p://linux.about.com/library/cmd/blcmdl8_vconfig.htm) command is needed. On the other hand, choosing the right emulated networkcard in GNS3 Qemu se,ings is crucial to get trunk working. You cannot miss with i82557b ethernet card.

Note: Iproute2 extension is needed to show multiple equal-cost routes presented in Linux kernel. Thetopic is described more in detail here.

h,p://brezular.wordpress.com/2011/11/14/part2-testing-equal-cost-routes-in-linux-microcore-4-0-2/(h,p://brezular.wordpress.com/2011/11/14/part2-testing-equal-cost-routes-in-linux-microcore-4-0-2/)


15 de 48 02/10/2013 08:01 a.m.

(h,p://brezular.files.wordpress.com/2011/11/campus_network.jpg)

Part1 – Core Layer

Core layer consists from two Multilayer Core switches – Core1 nad Core2. They are connected with point-to-point layer3 links. The full-mesh topology (h,p://en.wikipedia.org/wiki/Network_topology) betweenCore and Distribution layer provides path even in case of failure of two links. For example, if bothinterfaces eth4 and eth1 fail on switch Distrib1 there is still path to Core Layer via interface eth0.

Quagga routing daemon is running on all Core and Distribution switches. It offers routing capabilitiesusing OSPF routing protoco (h,p://en.wikipedia.org/wiki/Open_Shortest_Path_First)l.

Use vtysh (h,p://linux.die.net/man/1/vtysh) – Quagga shell to configure Core switches as following.

1. Command for accessing Quagga CLI

tc@box:~$ sudo /usr/local/bin/vtysh



2. Configure hostname, interfaces and OSPF routing protocol for Core1

box# configure terminal

box(config)# hostname Core1

Core1(config)# interface eth0

Core1(config-if)# description Link to Core2

Core1(config-if)# ip address 10.10.10.5/30

Core1(config-if)# no shutdown

Core1(config-if)# exit


Core1(config-if)# description Link to Distrib2





16 de 48 02/10/2013 08:01 a.m.


Core1(config-if)# description Link to Distrib1




Core1(config)# router ospf

Core1(config-router)# network 10.10.10.4/30 area 0



Core1(config-router)# do write

Building Configuration…Configuration saved to /usr/local/etc/quagga/zebra.confConfiguration saved to /usr/local/etc/quagga/ripd.confConfiguration saved to /usr/local/etc/quagga/ripngd.confConfiguration saved to /usr/local/etc/quagga/ospfd.confConfiguration saved to /usr/local/etc/quagga/ospf6d.confConfiguration saved to /usr/local/etc/quagga/bgpd.confConfiguration saved to /usr/local/etc/quagga/isisd.conf[OK]

Now, exit from vtysh. As Quagga had not saved hostname “Core1″ to /usr/local/etc/quagga/zebra.conf,configure it from Microcore CLI..

tc@box:~$ echo "hostname Core1" >> /opt/bootlocal.sh

tc@box:~$ sudo hostname Core1

Force Microcore to save Quagga’s configuration files in /usr/locac/etc/quagga/ and other files in/opt/.filetool.lst.

tc@Core1:~$ /usr/bin/filetool.sh -b

Save GNS3. Go to File-> Save or use Ctrl + s.

3. Configure hostname, interfaces and OSPF routing protocol for Core2

Start Core2 and configure router according to topology.

Check if Core switches can see themselves as OSPF neighbours.

Core2# show ip ospf neighbor

Neighbor ID Pri State Dead Time Address Interface RXmtL RqstL DBsmL


17 de 48 02/10/2013 08:01 a.m.

10.10.10.18 1 Full/DR 35.785s 10.10.10.5 eth0:10.10.10.6 0 0 0

Check if routes are properly propagated in the routing table.

Core2# show ip route ospf

Codes: K – kernel route, C – connected, S – static, R – RIP, O – OSPF,I – ISIS, B – BGP, > – selected route, * – FIB route

O 10.10.10.4/30 [110/10] is directly connected, eth0, 00:08:20O>* 10.10.10.8/30 [110/20] via 10.10.10.5, eth0, 00:08:10O 10.10.10.12/30 [110/10] is directly connected, eth1, 00:00:27O>* 10.10.10.16/30 [110/20] via 10.10.10.5, eth0, 00:08:10O 10.10.10.20/30 [110/10] is directly connected, eth2, 00:08:20

Part2 – Distribution Layer

Distribution layer consists from two Multilayer Distribution switches – Distrib1 and Distrib2. The main jobfor Distribution switches is routing between different vlan’s subnet that are terminated here. Any trafficfiltering rules should be configured on Distribution swiches.

Uplink interfaces connecting Distribution switches to Core switches are layer3 interfaces. They participatein OSPF messages forwarding. Downlink interfaces connecting Distribution switches to Access switchesare Layer2 interfaces. They are trunks capable of carrying traffic belongig to multiple VLANs.

The next configuration’s steps are shown for Distrib1 switch only. Continue and make similiarconfiguration for Distrib2 switch.

1. Configure Layer3 interfaces and OSPF routing protocol on Distrib1 switch

tc@box:~$ echo "hostname Distrib1" >> /opt/bootlocal.sh

tc@box:~$ /usr/bin/filetool.sh -b

tc@box:~$ sudo hostname Distrib1

tc@Distrib1:~$ sudo /usr/local/bin/vtysh

Distrib1# conf t

Distrib1(config)# hostname Distrib1

Distrib1(config)# interface eth1

Distrib1(config-if)# description Link to Core1

Distrib1(config-if)# ip address 10.10.10.9/30


18 de 48 02/10/2013 08:01 a.m.

Distrib1(config-if)# no shutdown

Distrib1(config-if)# exit

Distrib1(config)# int eth2

Distrib1(config-if)# description Link to Core2




Distrib1(config)# int eth0

Distrib1(config-if)# description Link to Distrib2




Distrib1(config)# router ospf

Distrib1(config-router)# network 10.10.10.0/30 area 0



Distrib1(config-router)# exit

Distrib1(config)# do write

Exit from vtysh and save content of /usr/local/etc/quagga/ directory.

tc@box:~$ /usr/bin/filetool.sh -b

Ctrl + s

2. Configure Layer3 interfaces and OSPF routing protocol on Distrib2 switch

Configure Distrib1 switch according to the topology. Check if Distrib2 can see all three OSPF neighbours.

Distrib2# show ip ospf neighbor

Neighbor ID Pri State Dead Time Address Interface RXmtL RqstL DBsmL10.10.10.21 1 Full/DR 38.389s 10.10.10.1 eth0:10.10.10.2 0 0 010.10.10.22 1 Full/DR 38.927s 10.10.10.14 eth1:10.10.10.13 0 0 010.10.10.18 1 Full/DR 33.153s 10.10.10.18 eth2:10.10.10.17 0 0 0

Check if routes are properly propagated in the routing table of Quagga.


19 de 48 02/10/2013 08:01 a.m.

Distrib2# show ip route ospf

Codes: K – kernel route, C – connected, S – static, R – RIP, O – OSPF,I – ISIS, B – BGP, > – selected route, * – FIB route

O 10.10.10.0/30 [110/10] is directly connected, eth0, 00:01:47O>* 10.10.10.4/30 [110/20] via 10.10.10.14, eth1, 00:01:17* via 10.10.10.18, eth2, 00:01:17O>* 10.10.10.8/30 [110/20] via 10.10.10.1, eth0, 00:01:17* via 10.10.10.18, eth2, 00:01:17O 10.10.10.12/30 [110/10] is directly connected, eth1, 00:01:37O 10.10.10.16/30 [110/10] is directly connected, eth2, 00:01:24O>* 10.10.10.20/30 [110/20] via 10.10.10.1, eth0, 00:01:31* via 10.10.10.14, eth1, 00:01:31

Exit from Quagga vtysh shell and check routing table of Linux Microcore.

tc@Distrib2:~$ ip route show

10.10.10.0/30 dev eth0 proto kernel scope link src 10.10.10.210.10.10.4/30 proto zebra metric 20nexthop via 10.10.10.14 dev eth1 weight 1nexthop via 10.10.10.18 dev eth2 weight 110.10.10.8/30 proto zebra metric 20nexthop via 10.10.10.1 dev eth0 weight 1nexthop via 10.10.10.18 dev eth2 weight 110.10.10.12/30 dev eth1 proto kernel scope link src 10.10.10.1310.10.10.16/30 dev eth2 proto kernel scope link src 10.10.10.1710.10.10.20/30 proto zebra metric 20nexthop via 10.10.10.1 dev eth0 weight 1nexthop via 10.10.10.14 dev eth1 weight 1127.0.0.1 dev lo scope link

They are two euqal-cost paths for each of network 10.10.10.4/30, 10.10.10.8/30, 10.10.10.20/30 presented inkernel routing table of Distrib2.

3. Openvswitch Configuration – configure Layer2 trunk ports, Layer3 VLAN interfaces on Distrib1switch

Openvswitch does not have separate CLI for its configuration thus all the configuration must be donefrom Microcore CLI.

a) Configure eth3 and to become trunk port to allow carry traffic from VLAN 10 and VLAN 20

tc@Distrib1:~$ sudo ovs-vsctl add-br br0

tc@Distrib1:~$ sudo ovs-vsctl add-port br0 eth3 trunks=10,20

b) Configure eth4 to become trunk port to allow carry traffic from VLAN 30 and 40

tc@Distrib1:~$ sudo ovs-vsctl add-port br0 eth4 trunks=30,40


20 de 48 02/10/2013 08:01 a.m.

c) Create VLAN interfaces

tc@Distrib1:~$ sudo ovs-vsctl add-port br0 vlan10 tag=10 -- set interface vlan10type=internal




d) Check Openvswitch configuration

tc@Distrib1:~$ sudo ovs-vsctl show

a66779ff-0224-40ef-89f1-0deb21b939dBridge “br0″Port “eth3″trunks: [10, 20]Interface “eth3″Port “eth4″trunks: [30, 40]Interface “eth4″Port “vlan20″tag: 20Interface “vlan20″type: internalPort “vlan10″tag: 10Interface “vlan10″type: internalPort “br0″Interface “br0″type: internalPort “vlan30″tag: 30Interface “vlan30″type: internalPort “vlan40″tag: 40Interface “vlan40″type: internal

e) Configure IP addresses of VLAN interfaces and OSPF routing protocol on Distrib1 switch

We have two options here – either to use Linux kernel command – ifconfig (h,p://en.wikipedia.org/wiki/Ifconfig) or Quagga vtysh shell. I chose vtysh – no need to put commands to /opt/bootlocal.sh


21 de 48 02/10/2013 08:01 a.m.

tc@Distrib1:~$ sudo /usr/local/bin/vtysh

Distrib1# conf t

Distrib1(config)# interface vlan10



Distrib1(config-if)# interface vlan20









Distrib1(config-if)# router ospf





Distrib1(config)# do write

Exit from vtysh shell and save configuration.

tc@Distrib1:~$ /usr/bin/filetool.sh -b

Ctrl + s.

Do similar configuration for Distrib2 switch.


22 de 48 02/10/2013 08:01 a.m.

Part3 – Access Layer

Access Layer consists of two Layer2 switches – Access1 and Access2. VLANs are created here andswitchports are assigned to VLANS. As each VLAN is restricted to the local Access switch we call them –local VLANs. Primary task of Access layer is to provide switchports for end users and forward traffic. Since Layer2 Access switches do not route between different VLANs, traffic is sent to Distribution layervia 8021q trunks when routing is required. The design guides recommend a campus model with localVLANs when 20 percent of user’s traffic stays in Campus and 80 percent of traffic is forwarded outsidecampus.

1. Configure hostname, access ports for VLAN 10 and 20 and 8021q trunk ports on Access1 switch

tc@box:~$ echo "hostname Access1" >> /opt/bootlocal.sh

tc@box:~$ sudo hostname Access1

tc@Access1:~$ sudo ovs-vsctl add-br br0

tc@Access1:~$ sudo ovs-vsctl add-port br0 eth0 tag=10

tc@Access1:~$ sudo ovs-vsctl add-port br0 eth1 tag=20

tc@Access1:~$ sudo ovs-vsctl add-port br0 eth3 trunks=10,20

tc@Access1:~$ sudo ovs-vsctl add-port br0 eth4 trunks=10,20

2. Check Openvswitch configuration

tc@Access1:~$ sudo ovs-vsctl show

a66779ff-0224-40ef-89f1-0deb21b939dbBridge “br0″Port “br0″Interface “br0″type: internalPort “eth1″tag: 20Interface “eth1″Port “eth4″trunks: [10, 20]Interface “eth4″Port “eth0″tag: 10Interface “eth0″Port “eth3″trunks: [10, 20]Interface “eth3″


23 de 48 02/10/2013 08:01 a.m.

Save configuration.

tc@Access1:~$ /usr/bin/filetool.sh -b

Ctrl + s.

3. Configure access ports for VLAN 30 and 40 and 8021q trunk ports on Access2 switch

Configure Access2 switch according to topology.

4. Configure hostname and IP seetings for all hosts, PC1 is an example

tc@box:~$ echo "hostname PC1" >> /opt/bootlocal.sh

tc@box:~$ sudo hostname PC1

tc@PC1:~$ echo "ifconfig eth0 192.168.10.10 netmask 255.255.255.0 up" >>

/opt/bootlocal.sh

tc@PC1:~$ sudo ifconfig eth0 192.168.10.10 netmask 255.255.255.0 up

tc@PC1:~$ echo "route add default gw 192.168.10.1" >> /opt/bootlocal.sh

tc@PC1:~$ sudo route add default gw 192.168.10.1

tc@PC1:~$ /usr/bin/filetool.sh -b

5. Test connectivity inside the same VLAN, PC1 is an example

PC1 should ping IP address of vlan10 interface on both Distribution switches.

tc@PC1:~$ ping 192.168.10.2

PING 192.168.10.2 (192.168.10.2): 56 data bytes64 bytes from 192.168.10.2: seq=0 ,l=64 time=1.949 ms64 bytes from 192.168.10.2: seq=1 ,l=64 time=2.342 ms64 bytes from 192.168.10.2: seq=2 ,l=64 time=3.016 ms^C— 192.168.10.2 ping statistics —3 packets transmi,ed, 3 packets received, 0% packet lossround-trip min/avg/max = 1.949/2.435/3.016 ms

tc@PC1:~$ ping 192.168.10.3

PING 192.168.10.3 (192.168.10.3): 56 data bytes64 bytes from 192.168.10.3: seq=0 ,l=64 time=2.297 ms64 bytes from 192.168.10.3: seq=1 ,l=64 time=2.470 ms64 bytes from 192.168.10.3: seq=2 ,l=64 time=2.815 ms^C— 192.168.10.3 ping statistics —3 packets transmi,ed, 3 packets received, 0% packet lossround-trip min/avg/max = 2.297/2.527/2.815 ms2


24 de 48 02/10/2013 08:01 a.m.

Part4 – Virtual Router Redundancy Protocol/VRRP/ Configuration

Things look good now. We can ping a virtual interfaces on Distribution switches from host residing in thesame VLAN. But we still cannot ping any IP address residing in different subnet than host. It is becauseboth Distribution switches have not been configured to act as default gateway yet.

You might have noticed that all the hosts were configured with IP address of default gateway 192.168.x.1.This is a virtual IP address created by Keepalived extension.

Keepalived offers default gateway’s redundancy using VRRP protocol. In case of failure one of theDistribution switches, the other Distrib switch takes responsibility and continues to forward packets.Switch with higher priority is called Master and forwards packet. At least it is always one Master switch inone VRRP group. For example, in VRRP group 10 it is Distrib1 with priority 150. Switch with lowerpriority is called Backup and as it was said, it forwards packets only if Master switch fails. The Backupswitch in VRRP group 10 is Distrib2 with priority 100. The higher is priority, more likely switch becomesMaster switch.

Obviously, it must be communication between switches to let known each other about its existence. Bydefault every one second Advertisiment is sent from Master switch to each memember of VRRP group tomulticast IP address 224.0.0.18. After three missing Advertisiment plus screw time, Backup switch knownsthat Master is down and transition from Backup to Master state occurs. It forwards packets now.

Note:

Virtual IP address is tied with virtual MAC address – 00-00-5E-00-01-XX. The last byte of the address(XX) is the Virtual Router Identifier (VRID), which is different for each VRRP instance in the network.This address is used by only one physical router at a time, and it will reply with this MAC address whenan ARP request is sent for the virtual router’s IP address. If Master fails, the new Master will broadcastGratious ARP containing the virtual router MAC address for the associated IP address. If I understand itcorrectly, nothing has been change in host configuration but Access switches had changed their CAMtable – frames with destination MAC address 00-00-5E-00-01-XX will be send via new path to the newMASTER router.

Keepalived works slightly differently – it uses real MAC address of interface instead of Virtual MACaddress. Check ARP cache of hosts in VRRP testing section – it seems that the current Keepalived/VRRPdoes not support Virtual MAC addresses.

1. VRRP configuration on switch Distrib1

tc@Distrib1:~$ sudo su

root@Distrib1:~# mkdir /usr/local/etc/keepalived/

root@Distrib1:~# echo "/usr/local/etc/keepalived/" >> /opt/.filetool.lst


25 de 48 02/10/2013 08:01 a.m.

root@Distrib1:~# vi /usr/local/etc/keepalived/keepalived.conf

vrrp_instance 10 { # VRRP instance declarationstate MASTER # Start-up default stateinterface vlan10 # Binding interfacevirtual_router_id 10 # VRRP VRIDpriority 150 # VRRP PRI

authentication {auth_type PASS # Simple Passwd or IPSEC AHauth_pass Campus123 # Password string}

virtual_ipaddress { # VRRP IP address block192.168.10.1/24 brd 192.168.10.255 dev vlan10}}

vrrp_instance 20 {state BACKUPinterface vlan20virtual_router_id 20priority 100

authentication {auth_type PASSauth_pass Campus123}

virtual_ipaddress {192.168.20.1/24 brd 192.168.20.255 dev vlan20}}




vrrp_instance 40 {


26 de 48 02/10/2013 08:01 a.m.

state MASTERinterface vlan40virtual_router_id 40priority 150



Start Keepalived daemon. As Distrib2 is not configured for VRRP, all the VRRP instances should transit toMASTER state.

root@Distrib1:~# /usr/local/sbin/keepalived -P -f /usr/local/etc/keepalived/keepalived.conf

Make Keepalived daemon to be started after boot of Microcore.

root@Distrib1:~# echo "/usr/local/sbin/keepalived -P -f /usr/local/etc/keepalived/keepalived.conf" >> /opt/bootlocal.sh

Save Keepalived configuration file.

root@Distrib1:~# /usr/bin/filetool.sh -b

Note: After each change in keepalived.conf file you have to restart keepalived daemon to accept changes.

2. VRRP configuration on switch Distrib2

tc@Distrib2:~$ sudo su

root@Distrib2:~# mkdir /usr/local/etc/keepalived/

root@Distrib2:~# echo "/usr/local/etc/keepalived/" >> /opt/.filetool.lst

root@Distrib2:~# vi /usr/local/etc/keepalived/keepalived.conf

vrrp_instance 10 { # VRRP instance declarationstate BACKUP # Start-up default stateinterface vlan10 # Binding interfacevirtual_router_id 10 # VRRP VRIDpriority 100 # VRRP PRI

authentication {auth_type PASS # Simple Passwd or IPSEC AHauth_pass Campus123 # Password string}


27 de 48 02/10/2013 08:01 a.m.

virtual_ipaddress { # VRRP IP address block192.168.10.1/24 brd 192.168.10.255 dev vlan10}}

vrrp_instance 20 {state MASTERinterface vlan20virtual_router_id 20priority 150



vrrp_instance 30 {state MASTERinterface vlan30virtual_router_id 30priority 150





virtual_ipaddress {192.168.40.1/24 brd 192.168.40.255 dev vlan40


28 de 48 02/10/2013 08:01 a.m.

}}

Start keepalived daeomon.


Distrib1 should become BACKUP router in VRRP group 20 and 30 and stays in MASTER state in VRRP10 and 40. Similarly, DOSTRIB2 is BACKUP router in VRRP group 10 and 40 and MASTER in VRRPgroup 20 and 30.

Make keepalived daeomon to be started after boot of Microcore.

root@Distrib2:~# echo "/usr/local/sbin/keepalived -P -f /usr/local/etc/keepalived/keepalived.conf" >> /opt/bootlocal.sh

Save keepalived configuration file.

root@Distrib2:~# /usr/bin/filetool.sh -b

Part5 – Testing

Connectivity should be established between any two nodes in our Campus network and we are going totest it.

1. Test routing between VLANs

Issue ping from host PC1 in VLAN10 to host P2, PC3, PC4 and check if intervlan routing is working.

tc@PC1:~$ ping 192.168.20.1

PING 192.168.20.10 (192.168.20.10): 56 data bytes64 bytes from 192.168.20.10: seq=0 ,l=63 time=14.443 ms64 bytes from 192.168.20.10: seq=1 ,l=63 time=4.257 ms64 bytes from 192.168.20.10: seq=2 ,l=63 time=4.962 ms^C— 192.168.20.10 ping statistics —3 packets transmi,ed, 3 packets received, 0% packet lossround-trip min/avg/max = 4.257/7.887/14.443 ms

tc@PC1:~$ ping 192.168.30.10

PING 192.168.30.10 (192.168.30.10): 56 data bytes64 bytes from 192.168.30.10: seq=0 ,l=63 time=9.720 ms64 bytes from 192.168.30.10: seq=1 ,l=63 time=4.125 ms64 bytes from 192.168.30.10: seq=2 ,l=63 time=4.920 ms^C


29 de 48 02/10/2013 08:01 a.m.

— 192.168.30.10 ping statistics —3 packets transmi,ed, 3 packets received, 0% packet lossround-trip min/avg/max = 4.125/6.255/9.720 ms

tc@PC1:~$ ping 192.168.40.10


2. Check a routing table of Core switch

Check if redundant routes are presented in kernel routing table of Core1. Non-redundant routes such as10.10.10.4/30, 10.10.10.8/30, 10.10.10.16/30 and 127.0.0.1 are directly connected networks. Two equal-costpath should exist to other network learned by OSPF routing protocol.

tc@Core1:~$ ip route show

10.10.10.0/30 proto zebra metric 20nexthop via 10.10.10.9 dev eth1 weight 1nexthop via 10.10.10.17 dev eth2 weight 110.10.10.4/30 dev eth0 proto kernel scope link src 10.10.10.510.10.10.8/30 dev eth1 proto kernel scope link src 10.10.10.1010.10.10.12/30 proto zebra metric 20nexthop via 10.10.10.6 dev eth0 weight 1nexthop via 10.10.10.17 dev eth2 weight 110.10.10.16/30 dev eth2 proto kernel scope link src 10.10.10.1810.10.10.20/30 proto zebra metric 20nexthop via 10.10.10.6 dev eth0 weight 1nexthop via 10.10.10.9 dev eth1 weight 1127.0.0.1 dev lo scope link192.168.10.0/24 proto zebra metric 20nexthop via 10.10.10.9 dev eth1 weight 1nexthop via 10.10.10.17 dev eth2 weight 1192.168.20.0/24 proto zebra metric 20nexthop via 10.10.10.9 dev eth1 weight 1nexthop via 10.10.10.17 dev eth2 weight 1192.168.30.0/24 proto zebra metric 20nexthop via 10.10.10.9 dev eth1 weight 1nexthop via 10.10.10.17 dev eth2 weight 1192.168.40.0/24 proto zebra metric 20nexthop via 10.10.10.9 dev eth1 weight 1nexthop via 10.10.10.17 dev eth2 weight 1

3. Check connectivity between Access and Core layer


30 de 48 02/10/2013 08:01 a.m.

Issue ping from host PC1 in VLAN10 to eth0 interface of Core2.

tc@PC1:~$ ping 10.10.10.6


4. VRRP testing

Issue ping from PC1 to IP address of VLAN10 interfaces – 192.168.10.2 and 192.168.10.3 on bothDistribution switches. Stop ping sequence and ping VRRP Virtual IP address 192.168.10.1.

Now check ARP cache of PC1.

tc@PC1:~$ arp

? (192.168.10.2) at 00:23:20:bc:57:67 [ether] on eth0? (192.168.10.1) at 00:23:20:bc:57:67 [ether] on eth0? (192.168.10.3) at 00:23:20:8c:1d:cf [ether] on eth

Virtual IP address 192.168.10.1 has assigned MAC address of VLAN10 interface – 00:23:20:bc:57:67 onDistrib1 switch. Packets destined outside VLAN10 will leave subnet 192.168.10.0/24 via vlan10 interfaceon Distrib1. We have expected it because Distrib1 is MASTER router for VRRP group 10.

Now start to ping IP address of eth0 interface of Core2 – 10.10.10.6/30 from PC1. We are going to killkeepalived proccess on Distrib1 switch to check if Disrib2 will transit to MASTER state for VRRP group10 and 40.

List the keepalived proccess on Distrib1 switch

root@Distrib1:~# ps -ef | grep keepalived

2549 root /usr/local/sbin/keepalived -P -f /usr/local/etc/keepalived/keepalived.conf2550 root /usr/local/sbin/keepalived -P -f /usr/local/etc/keepalived/keepalived.conf4937 root grep keepalived

Now, kill Keepalived process.

root@Distrib1:~# kill 2549

Keepalived: Terminating on signalKeepalived: Stopping Keepalived v1.2.2 (06/28,2011)Keepalived_vrrp: Terminating VRRP child process on signal

Check the output of console of Distrib2 switch. Information messages tell us about transition of Distrib2switch to MASTER state for VRRP group 10 and 40.


31 de 48 02/10/2013 08:01 a.m.

Keepalived_vrrp: VRRP_Instance(10) Transition to MASTER STATEKeepalived_vrrp: VRRP_Instance(40) Transition to MASTER STATEKeepalived_vrrp: VRRP_Instance(10) Entering MASTER STATEKeepalived_vrrp: VRRP_Instance(40) Entering MASTER STATE

Check ARP cache PC1 again. It is not big surprise that Virtual IP address 192.168.10.1 has assigned MACaddress 00:23:20:8c:1d:cf of VLAN10 interface of Distrib2 switch.

tc@PC1:~$ arp

? (192.168.10.2) at 00:23:20:bc:57:67 [ether] on eth0? (192.168.10.1) at 00:23:20:8c:1d:cf [ether] on eth0? (192.168.10.3) at 00:23:20:8c:1d:cf [ether] on eth0p

It seems that series of ping requests from PC1 to 10.10.10.6 has not been broken by transition.

— 10.10.10.6 ping statistics —206 packets transmi,ed, 206 packets received, 0% packet lossround-trip min/avg/max = 1.664/4.343/15.437 ms.

Start keepalived daemon on Distrib1 switch again. Distrib1 is going immediately to state MASTER forVRRP group 10 and 40 and to BACKUP for VRRP group 20 and 30.


Keepalived: Starting VRRP child process, pid=5134Keepalived_vrrp: Registering Kernel netlink reflectorKeepalived_vrrp: Registering Kernel netlink command channelKeepalived_vrrp: Registering gratutious ARP shared channelKeepalived_vrrp: Opening file ‘/usr/local/etc/keepalived/keepalived.conf’.

Keepalived_vrrp: Configuration is using : 45592 BytesKeepalived_vrrp: Using LinkWatch kernel netlink reflector…Keepalived_vrrp: VRRP_Instance(20) Entering BACKUP STATEKeepalived_vrrp: VRRP_Instance(30) Entering BACKUP STATEKeepalived_vrrp: VRRP_Instance(10) Transition to MASTER STATEKeepalived_vrrp: VRRP_Instance(40) Transition to MASTER STATEKeepalived_vrrp: VRRP_Instance(10) Entering MASTER STATEKeepalived_vrrp: VRRP_Instance(40) Entering MASTER STATE

Distrib2 received VRRP Advertisiment with priority higher than its own and transit to BACKUP state forVRRP instance 10 and 40.

Keepalived_vrrp: VRRP_Instance(10) Received higher prio advertKeepalived_vrrp: VRRP_Instance(10) Entering BACKUP STATEKeepalived_vrrp: VRRP_Instance(40) Received higher prio advertKeepalived_vrrp: VRRP_Instance(40) Entering BACKUP STATE.

Also in this case ping started on PC1 was not interrupted by transition back to MASTER state on Distrib1for VRRP group 10.


32 de 48 02/10/2013 08:01 a.m.

END.

h,p://en.wikipedia.org/wiki/Virtual_Router_Redundancy_Protocol (h,p://en.wikipedia.org/wiki/Virtual_Router_Redundancy_Protocol)h,p://www.estoile.com/links/vrrp.htm (h,p://www.estoile.com/links/vrrp.htm)h,p://osdir.com/ml/linux.keepalived.devel/2005-01/msg00000.html (h,p://osdir.com/ml/linux.keepalived.devel/2005-01/msg00000.html)

FILED UNDER OPENVSWITCH TAGGED WITH ACCESS LAYER, CAMPUS MODEL, CORELAYER, CORE LINUX, DISTRIBUTION LAYER, KEEPALIVED, MICROCORE, OPENVSWITCH,QEMU, QUAGGA

Part1 – OPENVSWICH – Creating and SubmittingOpenvswitch Extension To Tinycore Upstream

SEPTEMBER 3, 2011 8 COMMENTS (HTTP://BREZULAR.WORDPRESS.COM/2011/09/03/PART1-OPENVSWICH-CREATING-AND-SUBMITTING-OPENVSWITCH-EXTENSION-TO-MICROCORE-UPSTREAM/#COMMENTS)

i2 Votes

In May 2011, I read a request (h,p://www.gns3.net/phpBB/topic3549.html?sid=9da8d8dc809d413978e92277b9205d90) for installation Openvswitch(h,p://openvswitch.org/) on Qemu image. I started to play with Openvswitch and finally became fan ofthis project. I realized how powerful can be Openvswitch, offering many features listed here(h,p://openvswitch.org/). Spending my time playing with Microcore and Openvswitch after several days Iwas able to create Microcore Qemu image with 8021q (h,p://brezular.wordpress.com/2011/06/06/linux-microcore-kernel-compilation-for-802-1q-support/) support and install Openvswitch on the top of it.

In the tutorial I would like to show how to create Openvswitch extension (h,p://wiki.tinycorelinux.net/wiki:creating_extensions) and make it ready for submit to the Tinycore upstream. I chose Core Linux,because I am familiar with this minimal Linux distribution and the Core is incredibly small.

The following steps describe installation Openvswitch on Qemu image with pre-installed Microcore Linux.

I also created three labs using Openvswitch. I tested how can Openvswitch works with VLANs, 802.1qtrunk ports, if it was capable of creating L3 VLAN interfaces and Inter VLAN routing was working. Labsare available here (h,p://brezular.wordpress.com/2011/06/25/part2-openvswich-vlans-trunks-l3-vlan-


33 de 48 02/10/2013 08:01 a.m.

interface-intervlan-routing-configuration-and-testing/):

1. Start Qemu Microcore image

Assuming your Microcore Qemu image with console support (h,p://brezular.wordpress.com/2011/01/26/how-to-setup-linux-microcore-3-x-router-qemu-image-in-fedora-linux-part1/) has been created andsupports 8021q VLAN tagging (h,p://brezular.wordpress.com/2011/06/06/linux-microcore-kernel-compilation-for-802-1q-support/), start the image:

qemu-kvm /home/brezular/linux-microcore-4.7.7.img -m 512

2. Download Openvswitch and Extract it

wget hAp://openvswitch.org/releases/openvswitch-1.11.0.tar.gz (hAp://openvswitch.org/releases/openvswitch-1.11.0.tar.gz)

tar zxvf ./openvswitch-1.11.0.tar.gz

cd ./openvswitch-1.1.0

3. Install Necessary Dependences for Openvswitch Compilation

tce-load -w -i compiletc.tcz python.tcz perl5.tcz openssl-1.0.0-dev.tcz tunctl.tcz bridge-utils.tcz linux-headers-3.0.21-tinycore.tcz

To compile openvswitch with the ovsdmonitor tool, the machine must also have the following installedsoftware:

tce-load -w -i python_twisted-2.7.tcz python-simplejson.tcz python_zope.interface-2.7 pyqt.tcz

Note: The kernel source will be saved in /lib/modules/`uname -r`/build

It is recommended to check the list of necessary applications here (h,p://git.openvswitch.org/cgi-bin/gitweb.cgi?p=openvswitch;a=blob_plain;f=INSTALL;hb=HEAD):

4. Openvswitch Installation

It is recommended to start here (h,p://wiki.tinycorelinux.net/wiki:creating_extensions) and continue here(h,p://git.openvswitch.org/cgi-bin/gitweb.cgi?p=openvswitch;a=blob_plain;f=INSTALL;hb=HEAD) :-)After that you can type following:

export CFLAGS="-march=i486 -mtune=i686 -Os -pipe"

export CXXFLAGS="-march=i486 -mtune=i686 -Os -pipe"

export LDFLAGS="-Wl,-O1"

cd ./openvswitch-1.11.0

./configure --prefix=/usr/local --with-linux=/lib/modules/`uname -r`/build

make -j2

sudo su


34 de 48 02/10/2013 08:01 a.m.

cd /home/tc/openvswitch-1.11.0/

make DESTDIR=/tmp/openvswitch install

5. BACKUP and LOAD Module openvswitch.ko

a) BACKUP module openvswitch.ko

After building, a kernel module “openvswitch.ko” will be saved in a ./datapath/linux/ directory. We’vebuilt both userspace and kernel module as well. The performance is be,er when a kernel module is used.Create a directory in a structure of the openvswitch extension where the kernel module openvswitch.kowill be saved.

sudo mkdir -p /tmp/openvswitch/usr/local/lib/modules/3.0.21-tinycore/kernel/openvswitch/

Copy the kernel module to the directory.

sudo cp /home/tc/openvswitch-1.11.0/datapath/linux/openvswitch.ko /tmp/openvswitch/usr/local/lib/modules/3.0.21-tinycore/kernel/openvswitch/

b) Check if module can be loaded correctly

sudo mkdir -p /usr/local/lib/modules/3.0.21-tinycore/kernel/openvswitch/

sudo cp /home/tc/openvswitch-1.11.0/datapath/linux/openvswitch.ko /usr/local/lib/modules/3.0.21-tinycore/kernel/openvswitch/

depmod -a

sudo modprobe openvswitch

Check if module openvswitch_mod.ko is loaded to the kernel with lsmod command.

lsmod | grep openvswitchopenvswitch 49152 0

6. Backup /home/tc/openvswitch-1.11.0/vswitchd/vswitch.ovsschema

In order to initialize the configuration database using ovsdb-tool later, the file /home/tc/openvswitch-1.11.0/vswitchd/vswitch.ovsschema is needed. You need to copy it to /tmp/openvswitch/usr/local/etc/openvswitch/vswitchd/ to become part of the extension.

sudo mkdir -p /tmp/openvswitch/usr/local/etc/openvswitch/vswitchd/

sudo cp /home/tc/openvswitch-1.11.0/vswitchd/vswitch.ovsschema /tmp/openvswitch/usr/local/etc/openvswitch/vswitchd/

7. Create Openvswitch Extension

a) Remove unneccessary files

sudo rm -rf /tmp/openvswitch/usr/local/share/man/


35 de 48 02/10/2013 08:01 a.m.

b) Install squashfs and create openvswitch-3.0.21-tinycore.tcz extension

tce-load -w -i squashfs-tools-4.x

sudo sucd /tmp/mksquashfs openvswitch/ openvswitch-3.0.21-tinycore.tcz

c) Create a list of files presented in extension

sudo su cd /tmp/openvswitch/find usr -not -type d > ../openvswitch-3.0.21-tinycore.tcz.listcd ..

d) Create md5 check sum of openvswitch-3.0.21-tinycore.tcz

md5sum openvswitch-3.0.21-tinycore.tcz > openvswitch-3.0.21-tinycore.tcz.md5.txt

e) Create openvswitch-3.0.21-tinycore.tcz.info

An info file describing its contents (.tcz.info) – this content is standardized. Check repository(h,p://distro.ibiblio.org/tinycorelinux/4.x/x86/tcz/) for examples.

f) Create openvswitch-3.0.21-tinycore.tcz.build-dep file

Additional build instructions in a plain text file for future reference, mentioning such things as whichextensions are required to build the package and what compile flags were used.

g) Create the dependency list openvswitch-3.0.21-tinycore.tcz.dep

List of the extensions that have to be presented to run openvswitch extension correctly.

h) Install openssh.tcz an extension and copy all openvswitch-3.0.21-tinycore.tcz.* files and a sourcefile to the guest system

tce-load -wi openssh

cp -rv /home/tc/openvswitch-1.11.0.tar.gz /tmp

scp -rv /tmp/openvswitch-3.0.21-tinycore.* [email protected]:/home/brezular/scp -rv /tmp/openvswitch-1.11.0.tar.gz [email protected]:/home/brezular/

8. Test and Submit Openvswitch Extension

a) Create the new clean Microcore Qemu image without any extensions installed

Assuming the image is ready to start, run it:

qemu-kvm /home/brezular/linux-microcore-4.7.7-clean.img -m 512

b) Install submitqc.tcz and openssh.tcz


36 de 48 02/10/2013 08:01 a.m.

tce-load -wi submitqc4.tcz openssh.tcz

c) Copy the openvswitch files you have created in steps 1 – 7

(.tcz, .list, .md5.txt, .info, .dep, build-dep) from old Qemu image to the clean Qemu image, to the directory /tmp/All.

mkdir /tmp/Allcd /tmp/All

scp -rv [email protected]:/home/brezular/linux-core-nemaz/extensions/openvswitch/* .

Run an extension testing script:

sudo submitqc4

The script checks all openvswitch files in /tmp/All and creates the directory /tmp/submitqc/ with log files.

d) Send extension

sudo mv /tmp/submitqc /tmp/Allcd /tmp/All

sudo tar zcvf openvswitch-1.11.0-extension.tar.gz *

Send openvswitch-1.11.0-extension.tar.gz to [email protected]

e) Copy new created files to the guest

sudo scp -rv submitqc/ [email protected]:/home/brezular/sudo scp -rv openvswitch-3.0.21-tinycore.tcz.zsync [email protected]:/home/brezular/sudo scp -rv openvswitch-1.11.0-extension.tar.gz [email protected]:/home/brezular/

9. Openvswitch After-Install Configuration

a) Make openvswitch, 8021q, ipv6 modules to be loaded to a kernel during boot of Microcore

echo "modprobe openvswitch" >> /opt/bootlocal.sh

echo "modprobe 8021q" >> /opt/bootlocal.sh

echo "modprobe ipv6" >> /opt/bootlocal.sh

sudo modprobe openvswitch_modsudo modprobe 8021qsudo modprobe ipv6

b) Initialize the configuration database using ovsdb-tool

Check if a directory /usr/local/etc/openvswitch/ exists, if not create it.

sudo mkdir -p /usr/local/etc/openvswitch/

Create conf.db configuration file.


37 de 48 02/10/2013 08:01 a.m.

sudo ovsdb-tool create /usr/local/etc/openvswitch/conf.db /usr/local/etc/openvswitch/vswitchd/vswitch.ovsschema

Add /usr/local/etc/openvswitch/ to the list of after restart kept files.

echo "/usr/local/etc/openvswitch/" >> /opt/.filetool.lst

c) Make ovsdb-server to be started after start of the Microcore

vi /opt/bootlocal.sh

/usr/local/sbin/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \

--remote=db:Open_vSwitch,Open_vSwitch,manager_options \

--private-key=db:Open_vSwitch,SSL,private_key \

--certificate=db:Open_vSwitch,SSL,certificate \

--bootstrap-ca-cert=db:Open_vSwitch,SSL,ca_cert \

--pidfile --detach

:wq!

Note: If you are not familiar with vi editor, use the reference below, please:

h,p://nemesis.lonestar.org/reference/docs/vi.html (h,p://nemesis.lonestar.org/reference/docs/vi.html)

d) Make the database initialialization using ovs-vsctl

This is only necessary the first time after you create the database with ovsdb-tool (but running it at anytime is harmless).

echo "/usr/local/bin/ovs-vsctl --no-wait init" >> /opt/bootlocal.sh

e) Make the main Open vSwitch daemon being started, telling it to connect to the same Unix domainsocket

echo "/usr/local/sbin/ovs-vswitchd --pidfile --detach" >> /opt/bootlocal.sh

f) Enable IPv4 and IPV6 packets forwarding between interfaces

Although not directly connected with Openvswitch configuration we need to enable ipv4 and ipv6packets forwarding between interfaces for Microcore. This option is disabled in kernel by default.

sudo sysctl -w net.ipv4.ip_forward=1sudo sysctl -w net.ipv6.conf.all.forwarding=1

echo "sysctl -w net.ipv4.ip_forward=1" >> /opt/bootlocal.sh

echo "sysctl -w net.ipv6.conf.all.forwarding=1" >> /opt/bootlocal.sh

g) Run commands you have entered into /opt/bootlocal.sh and make them persistent after restart ofthe Microcore

sudo /opt/bootlocal.sh


38 de 48 02/10/2013 08:01 a.m.

Optionally: It is recommended to delete the history of used commands.

echo "" > /home/tc/.ash_history

Save changes made for files and directories which are listed in /opt/.filetool.lst

/usr/bin/filetool.sh -b

10. Configuration example

Now you may use ovs-vsctl to set up bridges and other Open vSwitch features. For example, to create abridge named br0 and add ports eth0, eth1 and eth2 to it:

sudo ovs-vsctl add-br br0

sudo ovs-vsctl add-port br0 eth0sudo ovs-vsctl add-port br0 eth1sudo ovs-vsctl add-port br0 eth2

Before shutdown you always force Core to save configuration changes in the openvswitch database file – /usr/local/etc/openvswitch/conf.db. Use the command:


Referenceh,p://openvswitch.org/ (h,p://openvswitch.org/)h,p://openvswitch.org/cgi-bin/gitweb.cgi?p=openvswitch;a=blob_plain;f=INSTALL.userspace;hb=HEAD(h,p://openvswitch.org/cgi-bin/gitweb.cgi?p=openvswitch;a=blob_plain;f=INSTALL.userspace;hb=HEAD)h,p://openvswitch.org/?page_id=14 (h,p://openvswitch.org/?page_id=14)h,p://openvswitch.org/cgi-bin/gitweb.cgi?p=openvswitch;a=blob_plain;f=INSTALL.Linux;hb=HEAD(h,p://openvswitch.org/cgi-bin/gitweb.cgi?p=openvswitch;a=blob_plain;f=INSTALL.Linux;hb=HEAD)

FILED UNDER OPENVSWITCH TAGGED WITH CORE LINUX, EXTENSION, LINUX SWITCH,MICROCORE

Part2 – OPENVSWICH – VLANs, Trunks, L3 VLANinterface, InterVLAN Routing – Configuration And Testing

JUNE 25, 2011 17 COMMENTS (HTTP://BREZULAR.WORDPRESS.COM/2011/06/25/PART2-OPENVSWICH-VLANS-TRUNKS-L3-VLAN-INTERFACE-INTERVLAN-ROUTING-CONFIGURATION-AND-TESTING/#COMMENTS)


39 de 48 02/10/2013 08:01 a.m.

i1 Vote

In a previous tutorial we showed how to install Openvswitch on Qemu image with Microcore Linux. Atthe end of tutorial we created Openvswitch extension (h,p://brezular.wordpress.com/2011/06/25/part1-openvswich-creating-and-submi,ing-extension-to-microcore-upstream/) and submi,ed it to Microcoreupstream. Assuming that Openvswitch is configured and functional, we are ready to make three labswhich helps us to test features such as VLANs, trunks and interVLAN routing.

LAB 1 Access VLAN Configuration And Testing

In this lab we are going to test connectivity between PCs which reside in same VLAN. PC1 and PC2 areassigned to VLAN 10 and PC3 and PC4 reside in VLAN 20. According to theory, each VLAN should haveits own separate subnet. In our case, we are going to test not only connectivity between computers in thesame VLANs but also to test connectivity between PCs placed in different VLAN.

For this reason I chose one subnet 192.168.1.0/24 for all the computers so L3 connectivity exists betweene.g. PC1 (VLAN 10) and PC3 (VLAN20). Of course, the computers assigned to different VLANs areseparated on Layer 2 so we do not expect any connectivity between those computers.

(h,p://brezular.files.wordpress.com/2011/06/access_vlans.jpeg)

(h,p://brezular.files.wordpress.com/2011/06/access_vlan1.jpeg)

Figure 1 Openvswitch with configured VLANs on its access ports – click image to enlarge

1) Openvswitch configuration

Login is tc without password set.


40 de 48 02/10/2013 08:01 a.m.

a) Let’s create bridge br0

sudo ovs-vsctl add-br br0

b) Add access port eth0 to the bridge br0 and assign it to VLAN 10

sudo ovs-vsctl add-port br0 br0 eth0 tag=10

c) Assign remaining ports to the br0

sudo ovs-vsctl add-port br0 eth2 tag=10sudo ovs-vsctl add-port br0 eth1 tag=20sudo ovs-vsctl add-port br0 eth3 tag=20

d) Assign hostname to Microcore

sudo hostname openvswitchecho "hostname openvswitch" >> /opt/bootlocal.sh

e) Save conf.db file to keep it persistent after the next reboot of Microcore Linux


2) PC1 configuration

Login is tc without password set.

Assign IP address 192.168.1.1/24 to eth0 and make it persistent after next reboot of Microcore

sudo hostname PC1sudo ifconfig eth0 192.168.1.1 netmask 255.255.255.0

echo "hostname PC1" >> /opt/bootlocal.sh

echo "ifconfig eth0 192.168.1.1 netmask 255.255.255.0" >> /opt/bootlocal.sh


3) PC2 to PC4 configuration

Configure all remaining computers with correct IP paramters, similary as it was done in point 2)..

4) Access VLAN connectivity test

a) Connectivity test between PC which are residing in the same VLAN

Issue ping from PC1 (VLAN10) to PC2 (VLAN10).

tc@bPC1:~$ ping 192.168.1.2PING 192.168.1.2 (192.168.1.2): 56 data bytes64 bytes from 192.168.1.2: seq=0 ,l=64 time=26.666 ms64 bytes from 192.168.1.2: seq=1 ,l=64 time=0.000 ms64 bytes from 192.168.1.2: seq=2 ,l=64 time=3.334 ms64 bytes from 192.168.1.2: seq=3 ,l=64 time=0.000 ms


41 de 48 02/10/2013 08:01 a.m.

— 192.168.1.2 ping statistics —4 packets transmi,ed, 4 packets received, 0% packet lossround-trip min/avg/max = 0.000/6.000/26.666 mstc@PC1:~$


tc@PC3:~$ ping 192.168.1.4PING 192.168.1.4 (192.168.1.4): 56 data bytes64 bytes from 192.168.1.4: seq=0 ,l=64 time=3.334 ms64 bytes from 192.168.1.4: seq=1 ,l=64 time=3.333 ms64 bytes from 192.168.1.4: seq=2 ,l=64 time=3.333 ms64 bytes from 192.168.1.4: seq=3 ,l=64 time=0.000 ms


As we have expected ping should have worked in both cases.

b) Test connectivity between computers placed to different VLANs


tc@PC1:~$ ping 192.168.1.4PING 192.168.1.4 (192.168.1.4): 56 data bytes

— 192.168.1.4 ping statistics —11 packets transmi,ed, 0 packets received, 100% packet loss

We can see that ping is not working between PC1 and PC3. We confirm that there is not connectivitybetween computers placed in to different VLANs and openvswitch is working correctly.

LAB 2 VLAN Trunk Configuration And Testing

In previous LAB we proved, that openvswitch only forwards traffic between computers which reside inthe same VLAN. In this LAB we are going to add additional switch to the topology – openvswitch2 andconnect switches with link configured as 8021.1q trunk. The access VLAN 10, 20, 30 is added toopenvswitch2 and VLAN 30 is added to openvswitch. There is only VLAN 10 and 30 allowed on bothsides of the trunk.

Our goal is to show that only traffic from VLAN 10 and VLAN 30 is forwarded between switches.


42 de 48 02/10/2013 08:01 a.m.

(h,p://brezular.files.wordpress.com/2011/06/vlan_trunk2.jpeg)

Figure 2 Two openvswitches connected with a link configured as 8021.q trunk – click image to enlarge


Assuming that the previous configuration of openvswitch has been stored, we only add followingcommands.

a) Add access port eth4 to the bridge br0 and assign it to VLAN 30

sudo ovs-vsctl add-port br0 eth4 tag=30

b) Add trunk port eth5 to bridge br0 and allow VLAN 10 and 30 on the trunk

sudo ovs-vsctl add-port br0 eth5 trunks=10,30

c) Save configuration of Microcore


2) Openvswitch2 configuration

As it was explained every single step of configuration of openvswitch in previous examples I am going toskip it now.

sudo hostname openvswitch2echo "hostname openvswitch2" >> /opt/bootlocal.sh

sudo ovs-vsctl add-br br0sudo ovs-vsctl add-port br0 eth5 trunks=10,30sudo ovs-vsctl add-port br0 eth0 tag=30sudo ovs-vsctl add-port br0 eth1 tag=20sudo ovs-vsctl add-port br0 eth2 tag=10


Similarity, configure IP parameters for the PC5, PC6 to PC8 according to the topology.

3) Trunk VLAN test


43 de 48 02/10/2013 08:01 a.m.

a) Issue the ping from PC1 (VLAN10) to PC8 (VLAN10)

tc@PC1:~$ ping 192.168.1.8PING 192.168.1.8 (192.168.1.8): 56 data bytes64 bytes from 192.168.1.8: seq=0 ,l=64 time=6.667 ms64 bytes from 192.168.1.8: seq=1 ,l=64 time=3.333 ms64 bytes from 192.168.1.8: seq=2 ,l=64 time=0.000 ms64 bytes from 192.168.1.8: seq=3 ,l=64 time=3.334 ms^C— 192.168.1.8 ping statistics —4 packets transmi,ed, 4 packets received, 0% packet lossround-trip min/avg/max = 0.000/3.333/6.667 mstc@PC1:~$

We can see that traffic in VLAN 10 is succesfully transferred through trunk. If the same VLAN exists ontwo switches we call it end-to-end VLAN.

b) Issue the ping from PC5 (VLAN30) to PC6 (VLAN30)

tc@PC5:~$ ping 192.168.1.6PING 192.168.1.6 (192.168.1.6): 56 data bytes64 bytes from 192.168.1.6: seq=0 ,l=64 time=6.667 ms64 bytes from 192.168.1.6: seq=1 ,l=64 time=3.334 ms64 bytes from 192.168.1.6: seq=2 ,l=64 time=3.333 ms64 bytes from 192.168.1.6: seq=3 ,l=64 time=3.333 ms^C— 192.168.1.6 ping statistics —4 packets transmi,ed, 4 packets received, 0% packet lossround-trip min/avg/max = 3.333/4.166/6.667 mstc@PC5:~$

From what we see, trunk is successfully carrying the traffic marked with tag 30 between switches.

c) Issue the ping from PC3 (VLAN20) to PC7 (VLAN20)


— 192.168.1.7 ping statistics —6 packets transmi,ed, 0 packets received, 100% packet losstc@PC3:~$

Traffic from vlan 20 is not forwarded between switches. It is because VLAN 20 is not allowed on thetrunk. If there would be need to allow all the VLANs on the trunk, only parameter trunks (withoutspecific VLAN) should be presented.

LAB 3 L3 VLAN interfaces, InterVLAN routing – Confi guration


44 de 48 02/10/2013 08:01 a.m.

And Testing

In this last LAB we are going to create L3 VLAN interface for VLAN 10, 20, 30. After that we will check ifMicrocore can forward traffic between different VLANs. We assume that particular ports were added to VLANs in previous LABs and only additional configuration will be shown in this LAB. I have alsochanged a topology to make it more simple and create the new IP plan for the devices.

(h,p://brezular.files.wordpress.com/2011/06/l3_routing.jpeg)

Figure 3 Topology for InterVLAN routing testing – click image to enlarge


a) Add an internal port vlan10 to bridge br0 as a VLAN access port for VLAN 10

sudo ovs-vsctl add-port br0 vlan10 tag=10 -- set interface vlan10 type=internal

b) Create vlan20 and vlan30 interfaces



c) Add access port eth4 and eth5 to the bridge br0 and assigned them to VLAN 30

First, We need to delete configuration for eth4 and eth5 that was done for LAB1and LAB2.

sudo ovs-vsctl del-port br0 eth4sudo ovs-vsctl del-port br0 eth5

Now we can assign ports to the VLAN 30.

sudo ovs-vsctl add-port br0 eth4 tag=30sudo ovs-vsctl add-port br0 eth5 tag=30

d) Assign IP adresses to the VLAN interfaces

sudo ifconfig vlan10 192.168.10.254 netmask 255.255.255.0


45 de 48 02/10/2013 08:01 a.m.

sudo ifconfig vlan20 192.168.20.254 netmask 255.255.255.0sudo ifconfig vlan30 192.168.30.254 netmask 255.255.255.0

echo "ifconfig vlan10 192.168.10.254 netmask 255.255.255.0" >> /opt/bootlocal.sh




e) Enable IPv4 and IPV6 packets forwarding between interfaces

Forwarding between network interfaces is disabled by default. To activate ipv4and ipv6 forwarding for Microcore you need to enable it:

sudo sysctl -w net.ipv4.ip_forward=1sudo sysctl -w net.ipv6.conf.all.forwarding=1

echo "sysctl -w net.ipv4.ip_forward=1" >> /opt/bootlocal.sh

echo "sysctl -w net.ipv6.conf.all.forwarding=1" >> /opt/bootlocal.sh


2) PC1 to PC6 – IP address, netmask, default gateway – configuration

a) Delete old IP addresses from the computers

We should split IP address space in such way when each VLAN would have assigned separate subnet.First, for each computer delete a row with old IP address from /opt/bootlocal.sh which was previouslyassigned in LAB1 (use dd command for delete of particular row).

b) Assign IP address and default GW to PC1 to PC6

Example for PC1:

sudo ifconfig eth0 192.168.10.1 netmask 255.255.255.0sudo route add default gw 192.168.10.254

echo "ifconfig eth0 192.168.10.1 netmask 255.255.255.0" >> /opt/bootlocal.sh

echo "route add default gw 192.168.10.254" >> /opt/bootlocal.sh


Similiary, do it for all the PCs, according to topology and IP address plan.

3) InterVLAN routing testing

As long as PCs have had correct IP address, netmask and default gateway assigned, we can either ping PCin the same VLAN or any other vlan interface.

a) Issue ping from PC1 (VLAN10) to PC2 (VLAN10)



46 de 48 02/10/2013 08:01 a.m.

64 bytes from 192.168.10.2: seq=0 ,l=64 time=3.333 ms64 bytes from 192.168.10.2: seq=1 ,l=64 time=3.333 ms64 bytes from 192.168.10.2: seq=2 ,l=64 time=3.334 ms64 bytes from 192.168.10.2: seq=3 ,l=64 time=3.333 ms


b) Issue ping from PC1 (VLAN10) to interface vlan20



We can see that in both cases, ping was succesfull.

c) Issue the ping from PC1 (VLAN10) to PC3 (VLAN20)



Thanks to enabled forwarding between interfaces in kernel, ping is working between different VLANS.Microcore with Openvswitch is acting as the Layer3 switch.

End.

FILED UNDER OPENVSWITCH TAGGED WITH 8021Q LINUX, CORE LINUX, GNS3, LINUXROUTER, LINUX SWITCH, MICROCORE, OPENVSWITCH, TRUNKS LINUX, VLAN LINUX

Blog at WordPress.com.


47 de 48 02/10/2013 08:01 a.m.

The Enterprise Theme.

Follow

Follow “Brezular's Technical Blog”

Powered by WordPress.com


48 de 48 02/10/2013 08:01 a.m.

openvswitch _ brezular's technical blog

Documents

openvswitch h

microcore linux h

gns3 h

option h

andqemu h

microc h

andquagga h

ifenslave h