ha nam

Palo Alto Networks TechConnect

High AvailabilityVon Nguyen

Webinar Agenda

Active/Passive HA Overview Configuration

Active/Active HA Overview Configuration HA Monitoring Troubleshooting

•

Active/Passive HA

© 2012 Palo Alto Networks. Proprietary and Confidential

Active/Passive HA Overview

• Supported Modes:- Layer 2, Layer 3, Virtual Wire

• Links:- HA1, HA2

• Device States:- Initial, Active, Passive, Non-functional, Suspend

• Synchronization of:- State-full sessions, Certificates, Response Pages, Configuration

- Not synchronized: Admin accounts, HA configuration

• 2 Unit cluster, same model

•

Active/Passive HA Operation

Primary Path

Secondary PathH

A2

HA

1

Control Plane

Sync Configuration

Data Plane

Sync Active Sessions

•

HA Configuration

Different from Mgt IP

Enables stateful synchronization across HA2 link

Only encrypts HA1 link info

Group ID for HA pair

Device with lower Priority will be elected Active

Device can resume Active after recovery

Ping across HA1 link

“Auto” (for L3 interfaces) or “Shutdown”

• © 2012 Palo Alto Networks. Proprietary and Confidential.

|

• HA1 • ethernet1/1

• HA2 • ethernet1/2

Control & Data link backup

Gateway specification

Configurable link support

Control/Data Link

|

• <Heartbeat/Hello>

• Redundant path• DP status confirmation• Supported on full product line

• <Heartbeat/Hello>

Heartbeat Backup – Split Brain Protection

Active/Active HA

A/A Agenda

• Overview

• Packet Handling

• Deployments

• Configuration

• Monitoring

• Troubleshooting

• Special Case, Wrap-Up

|

Active/Active HA Overview

What is High Availability Active/Active?

• With A/A deployment, both HA peers are active and processing traffic.

• A/A HA is supported only in the virtual-wire and Layer 3 modes beginning with PAN-OS 4.0.

• Such deployments are most suited for scenarios involving asymmetric routing.

• Deployment also can be to allow dynamic routing protocols (OSPF, BGP) to maintain active status across both peers.

• In addition to the HA1 and HA2 links used in A/P, A/A deployments require a dedicated HA3 link. HA3 link is used as packet forwarding link for session setup and asymmetric traffic handling.

|

Which to use - A/P or A/A?

What Active/Active is NOT designed for:

• A/A does NOT load balance. Load sharing can be done via sending of traffic across each peer, but there is no load-balancing mechanism.

• A/A will not increase performance or allow greater capacity. At no point should traffic loads go beyond capacity of a single stand-alone system as failover could cause single system to become overloaded causing possible outage.

Note: Unless Active/Active asymmetric flow or dynamic routing capability is a requirement, for most deployments Active/Passive is better option as it is more simple to deploy.

|

HA Peer Connection

• Same HA1 and HA2 links as A/P.

• Add HA3, any free dataplane port with interface mode ‘HA’.- All packet forwarding between the two devices uses HA3 link.

|

• HA1

• HA2• HA3

Agenda

• Overview

• Packet Handling

• Deployments

• Configuration

• Monitoring

• Troubleshooting


|

Active/Active Packet Handling

In Active/Active cluster, the packet handling can be distributed between the two peers. There are two important functions that are handled by devices in a cluster

• Session ownership

• Session setup

|

Session Ownership

• Session owner device can be either the firewall that receives the first packet of a new session or the device in an ACTIVE-PRIMARY state.

• This device is responsible for all layer 7 processing, i.e. app-id, content-id, and threat scanning for this session.

• This device is also responsible for generating all traffic logs for the session.

|

Session Setup

• Session setup device is responsible for layer2 through layer4 processing required for setting up a new session.

• Address translation is performed by session setup device.

• Session setup device is determined by configuring “session setup load sharing” options.

• Separation of session owner and session setup devices is necessary to avoid race conditions that can occur in asymmetrically routed environments

|

Packet Flow

In order to understand packet flow within a cluster, we will discuss three different scenarios

1. New session

2. Established session

3. Asymmetric packet flow

|

Session Setup

1.Packet arrives at one of the devices

2.Receiving device has no session for the packet, and assumes ownership of the session

3.Computed hash/modulo determines device is not responsible for session-setup, and forwards packet to peer device over HA3 link

4.Session is setup and session info and packet are returned to session owner

5.Original device forwards packet out appropriate interface

0010100010101001001

Session owner

Will be L7 ownerSession setup device

|

Packet Flow: New Session

The sequence of steps involved in setting up a session is listed below

1. End host sends packet to device-A.

2. Firewall examines the contents of the packet to match it to an existing session.

3. If there is no session match, Dev-A determines that it has received the first packet for a new session. Therefore Dev-A becomes the session owner.

4. Dev-A uses the configured session setup load sharing options to identify the session setup device. In this example we assume the setup function is performed by Dev-B

5. Using the HA-3 link, Dev-A sends the first packet it received to Dev-B.

6. Dev-B sets up the session and returns the packet to Dev-A for layer 7 processing if any.

7. Dev-A then forwards the packet out via the egress interface to the destination

|

Established session


2.Receiving device has session for the packet and owns the session

3. Packet is processed and sent out via the appropriate egress interface

0010100010101001001

Session owner

Layer 7 processing

|

Packet Flow: Existing Session

The sequence of steps for an existing session is listed below

1. End host sends packet to Dev-A.

2. Firewall examines the contents of the packet to match the packet to an existing session.

3. If there is a session match, Dev-A processes the packet and sends the packet out via the egress interface to the destination

|

Established Session – Packet Arriving at non session owner device


2.Receiving device has a session for the packet but it is owned by peer device

3.Receiving device forwards packet over the HA3 link to the owner for processing

4.Owner processes packet1. In vwire packet is sent back to

receiving device

2. In L3 if owner has route to destination, packet is forwarded out

0010100010101001001

Session owner

Layer 7 processing

|

Packet Flow: Asymmetric Flow - L3

The sequence of steps for an assymetric packet flow

1. Dev-B receives a packet.

2. Receiving device has a session for the packet but it is owned by peer device, Dev-A.

3. Dev-B forwards packet over the HA3 link to the Dev-A for processing.

4. In layer3 deployment , Dev-A processes packet and forwards it to destination if it has the route.

|

Packet Flow: Asymmetric Flow – V-Wire

The sequence of steps for an assymetric packet flow

1. Dev-B receives a packet.

2. Receiving device has a session for the packet but it is owned by peer device, Dev-A.

3. Dev-B forwards packet over the HA3 link to the Dev-A for processing.

4. In Vwire deployment in order to preserve the forwarding path, Dev-A processes the packet and returns to Dev-B, to be transmitted out the egress interface to the destination.

|

Agenda

• Overview

• Packet Handling

• Deployments

• Configuration

• Monitoring

• Troubleshooting


|

Deployment: V-Wire

• Simplest solution to implement high availability

• Firewalls are installed between L3 devices. These are often used in conjunction with dynamic routing protocols which will fail traffic over to the other cluster member if needed.

Note: Implementing A/A HA in v-wire mode in a layer2 sandwich will result in switching loops if Spanning Tree Protocol is not enabled on the switches. It is recommended to deploy A/A in v-wire in a layer3 topology.

|

Deployment: Layer 3

Layer3 deployment supports virtual IP addressing, NAT, and use of dynamic routing protocols for redundancy. Active/Active cluster can be deployed in several different scenarios in layer3 mode as described below

• Floating IP

• ARP load sharing

• Mixed mode (combine both floating IP and ARP load share)

|

Deployment: L3 Floating IP• Floating IP can move between HA devices when a link

failure or device failure occurs.

• Interface on device in cluster that owns floating IP responds to ARP requests with a virtual MAC.

• Floating IPs are recommended when VRRP-like functionality is required.

• Floating IPs can be used for VPNs and source NAT allowing for persistent connections when a failure occurs.

• Each interface on firewall has its own IP and a floating IP. Interface IP remains local to the device but floating IP address can move between the devices.

• End hosts are configured to use floating IP as default gateway allowing traffic to be load balanced within the cluster.

• External load balancers can also be used to load balance traffic between firewalls within the cluster.

• If failover occurs, gratuitous ARP is sent out by the functional device. Once device recovers, floating IP and VMAC will move back to the original device.

|

Deployment: L3 ARP Load Sharing• HA pair to share an IP address and provide gateway

services.

• All hosts are configured with single gateway IP. ARP requests for gateway IP are responded to with a virtual MAC address from a single device in the pair.

• Each device will have unique virtual MAC address generated for the shared IP.

• The device that responds to ARP request is determined by computing hash or modulo of source IP of the ARP request.

• Once end host receives ARP response from device, it caches the MAC address and all traffic from host is routed via the firewall that responded with VMAC. Life time of ARP cache is dependent on end host OS.

• ARP load-sharing should be used only when a Layer 2 separation exists between firewalls and end hosts.

• If link or device failure, floating IP and VMAC moves over to the functional device. Gratuitous ARP is sent out by the functional device.

|

Deployment: L3 Mixed Mode• It is possible to have some of interfaces configured with

floating IPs and some with shared IPs for ARP loading sharing.

• Cluster can be configured with ARP load sharing IPs, configured for hosts on the LAN segment, and floating IP configured on upstream WAN edge routers.

|

Agenda

• Overview

• Packet Handling

• Deployments

• HA States

• Configuration

• Monitoring

• Troubleshooting


|

Active/Active Configuration

• First step, set the HA mode to active-active. Device > High Availability; Setup

• ID: HA group ID. Both devices must have the same group ID. HA group-ID is used to calculate virtual MAC.

• Mode: Choose active-active from the drop down list.

• Device-id: Select unique device from drop down list (0 or 1). Device-ID remains local to the device and does not transition between devices during failover. This field is also used to calculate VMAC.

• Peer HA IP Address: IP address of HA1 control link on peer device.

• Backup Peer HA IP Address: IP address of backup control link on peer device. This field is optional.

• Enable Config Sync: Enabled by default, required to synchronize configuration between devices in cluster.

|

HA Control and Data Links

• Same as Active/Passive

|

• Data

Link

• ControlLink

• PA-1 • PA-2

HA3 Link

Used for packet forwarding between session owner and session setup device.

• HA3 link is L2 link and uses MAC-in-MAC encapsulation.

• Aggregate interfaces can be configured as HA3 link (4000 and 5000 series only) for redundancy of HA3 link.

• Interface mode must be HA to use as HA3 link.Note: Because of overhead associated with encapsulation on HA3 link, switch ports connecting HA3 link must be configured to support jumbo frames.

|

Configuring ARP Load Sharing

Device > High Availability > Virtual Address

• Click on “Add” to add a new virtual address.

• From interface drop down list choose appropriate interface, and click “Add”.

• Choose Type to “arp-load-sharing”. In this example we choose “ip-modulo” as ARP Load Sharing Type.

|

Configuring Floating IP

Device > High Availability > Virtual Address

• Click “Add” to add a new virtual address.

• From interface drop down list choose appropriate interface, and click “Add”.

• Choose Type to be “floating”. Device priority determines which device will own the floating IP address.

• Configure two floating IP address, one for each device, with different priorities as shown above. Address with lower numeric value will have highest priority.

|

Monitoring

Settings are same for Active/Passive and Active/Active:

• Heartbeat polling

• Link monitoring

• Path monitoring

|

Configuring Link Monitoring

• Device > High Availability; Link Monitoring

|

“Any” or “All” failure conditions will cause failover

Configuring Path Monitoring

• Device > High Availability; Path Monitoring

|

“Vwire”, “VLAN”, “VR”

“Any” or “All” failure conditions will cause failover

Agenda

• Overview

• Packet Handling

• Deployments

• Configuration

• Troubleshooting


|

Troubleshooting

• CLI show commands:admin@PA-2(active-primary)> show high-availability ?> all Show high-availability information> control-link Show control-link statistic information> dataplane-status Show dataplane runtime status> flap-statistics Show high-availability preemptive/non-functional flap statistics> interface Show high-availability interface information> link-monitoring Show link-monitoring state> path-monitoring Show path-monitoring statistics> state Show high-availability state information> state-synchronization Show state synchronization statistics> transitions Show high-availability transition statistic information> virtual-address Show Active-Active virtual address status

• Logs:- less mp-log ha_agent.log

- show log system

Note: For HA issues, be sure to always get data from BOTH peers as issues may be on either device.

|

HA CLI Commands

• Force configuration and session synchronization to peeradmin@student1> request high-availability sync-to-remote

• Fail HA master to peer and make system ineligible to be masteradmin@student1> request high-availability state suspend

• Re-enable HA on suspended systemadmin@student1> request high-availability state functional

• Show HA statusadmin@student1> show high-availability state

admin@student1> show high-availability link / path -monitoring

Troubleshooting Sessions

Session flow from host 172.35.2.4 to host 10.1.1.250.

admin@PA-2(active-primary)> show session all filter destination-port 23 -------------------------------------------------------------------------------- ID Application State Type Flag Src[Sport]/Zone/Proto (translated IP[Port]) Vsys Dst[Dport]/Zone (translated IP[Port]) -------------------------------------------------------------------------------- 19485 telnet ACTIVE FLOW NS 172.35.2.4[56484]/trust-l3/6 (10.1.1.101[57558]) vsys1 10.1.1.250[23]/untrust-l3 (10.1.1.250[23])

From session table, we see that host 172.35.2.4 is translated to IP 10.1.1.101, floating IP on PA-2 which is device-id 1

admin@PA-2(active-primary)> show session id 19485 | match HA session synced from HA peer : False session owned by local HA A/A : True

PA-2 is session owner.

|

Global Counter

Show counter global for Active/Active related packets.

admin@PA-2(active-primary)> show counter global filter aspect aa delta yes Global counters: Elapsed time since last sampling: 24.406 seconds

name value rate severity category aspect description -------------------------------------------------------------------------------- ha_aa_session_setup_peer 1 0 info ha aa Active/Active: setup session on peer device ha_aa_pktfwd_rcv 1 0 info ha aa Active/Active: packets received from peer device ha_aa_pktfwd_xmt 1 0 info ha aa Active/Active: packets forwarded to peer device -------------------------------------------------------------------------------- Total counters shown: 3 --------------------------------------------------------------------------------

|

Viewing Floating IPs

• “show high-availability virtual-address” can be used to view all configured floating IP addresses.

admin@PA-1(active-primary)> show high-availability virtual-address

Total interfaces with virtual address configured: 2

Total virtual addresses configured: 4

-----------------------------------------------------------------------------

Interface: ethernet1/2 Virtual MAC: 00:1b:17:00:01:11

10.1.1.100 Active:yes Type:floating

10.1.1.101 Active:no Type:floating

-----------------------------------------------------------------------------

Interface: ethernet1/1 Virtual MAC: 00:1b:17:00:01:10

172.35.2.100 Active:yes Type:arp-load-sharing

-----------------------------------------------------------------------------

|

Manual failover

Same as A/P except will determine Primary/Secondary.

• GUI:

• CLI (on active peer):request high-availability state suspend

request high-availability state functional

|

Logs and Packet Captures

• All traffic logs are logged by session owner.

• When session owner fails, peer device will become session owner and will handle logging.

• If preempt is enabled and should failed device recover before session ends, it will take back ownership of the session and handle logging.

|

Agenda

• Overview

• Packet Handling

• Deployments

• Configuration

• Monitoring

• Troubleshooting


|

PA-200 – A/P HA-Lite

|

Supports limited A/P functionality “HA-Lite” Uses MGMT port as HA1 link for heartbeats and config sync No HA2 or HA3 link supported, no session sync

|

For More Information

• Active/Passive HA Tech Note: https://live.paloaltonetworks.com/docs/DOC-1160

• Active/Active HA Tech Note: https://live.paloaltonetworks.com/docs/DOC-1756

• Designing Networks with Palo Alto Networks firewalls: https://live.paloaltonetworks.com/docs/DOC-2561

https://live.paloaltonetworks.com/docs/DOC-1160






|

THANK YOU !!

• Upcoming TechConnect Webinars:

• Go to www.paloaltonetworks.com/partner site to register.

http://www.paloaltonetworks.com/partner

ha nam

Technology

session setup

session info

session match

existing session

session owner5

session setup1

session setup devices

separation of session