© 2018

HAProxy ResolverVNS3 Plugin Container Guide 2018

© 2018

Table of Contents

2

Introduction 3

Configurable HAProxy Plugin 7

Customizing HAProxy Plugin 17

Running HAProxy Plugin 26

Resources 28

© 2018

Introduction

3

© 2018

VNS3 provides container based network services

4

waf content caching nids proxy load balancing custom

router switch firewall vpn concentrator protocolredistributor

dynamic &srciptable sdn

VNS3 Core Components

firewall vpn concentrator protocol redistributor extensible nfv

VNS3 Core Components

router switch

Isolated Linux containers within VNS3 allow partners and customers to embed features and functions safely and securely into their cloud network.

© 2018

Requirements

5

You have a cloud or virtual infrastructure account that Cohesive Networks can use for enabling your access to the VNS3 Controller Images.

You have the ability to configure a client (whether desktop based or cloud based) to use OpenVPN client software.

You have agreed to the VNS3 Terms and Conditions.

Basic knowledge of Linux software installation and use of command line tools.

© 2018

Getting Help with VNS3

6

This document assumes you have a VNS3 Controller instance launched and running in a security group, network or similar that has the appropriate access rules included for normal VNS3 operations. For any support issues, email us at support@cohesive.net

Please review the VNS3 Support Plans and Contacts before sending support inquiries. If you need specific help with project planning, POCs, or audits, contact our professional services team via sales@cohesive.net for details.

© 2018

Configurable HAProxy Resolver Plugin

7

© 2018

Getting Started with VNS3 Plugin System

8

The following HAProxy DNS Resolution functionality is deployed to VNS3 as a plugin using the container system. These instructions cover customization of the container image that will be used so that customer keys and rule sets can be employed.

Please be familiar with the VNS3 plug-in configuration guide: https://cohesive.net/dnld/Cohesive-Networks_VNS3-3.5-Container-System.pdf

© 2018

HAProxy Resolver Plugin - What does it do?

9

Simply stated HAProxy Resolver Plugin acts as a layer 4 forward proxy and resolves DNS at runtime.

Many VNS3 users run application topologies in 3rd party controlled environments like public clouds where they have limited access, control and visibility to the underlying infrastructure.

In the cloud underlying IPs can changes due to reboot or normal operations of services like Load Balancers (e.g. AWS Classic Elastic Load Balancers and Application Load Balancers). Many Layer 4 - 7 Network services can be configured using DNS names but resolution is usually done at startup and the resulting IP is cached for the length of the process.

This container plugin is designed to process a server or service's name resolution at runtime to honor shorter TTL in the case of Load Balancers and allow for infrastructure modifications at the IP level without reloading configurations to update the cached DNS resolution.

© 2018

Example Architecture: Simple Cloud Application with Load Balancers

10

AZ1a App Subnet

host 1

host n

AZ2a App Subnet

host 1

host n

AZ1a VNS3 Subnet

HA Proxy Plugin

VNS3AWS Route 53Users ALB

1

2

3

1. DNS advertisement of URLs to non-RFC1918 IP addresses included in remote IPsec tunnel encryption domain "behind" VNS3 Controller.

2. VNS3 Netmaps traffic to non-RFC1918 IP addresses to HA Proxy Plugin.

3. HAProxy Plugin performs layer 4 proxy forward to AWS ALB DNS name. It also performs DNS Resolution at runtime (configurable but default every 10s) to handle the ALB floating private IP.

1

2

3

© 2018

HAProxy Plugin - How does it work?

11

HAProxy DNS Resolver plugin performs name resolution in two places:

1. Process Startup - The plugin parses the configuration and resolves any DNS name included via the configured name server.

2. Runtime Success - Periodic resolutions are performed for any servers using DNS name in the configuration.

3. Runtime Failed Health Check - If a health check results in a connection timeout, a resolution is triggered.

This periodic and configurable resolution allows VNS3 users to take advantage of highly available cloud servers that periodically change the underlaying IP associated with a DNS name. The most common example of such a service is AWS Elastic Load Balancers and Applications Load Balancers.

This Plugin is compatible with VNS3 Peering Meshes, Highly Available IPsec VPN connections via BGP and the HA Plugin (both cloud and on-premises version) for a highly available/fault tolerant architecture.

© 2018

Getting the HAProxy Resolver Plugin

12

The Linux-based (Ubuntu 14.04) HAProxy Resolver Plugin is accessible at the following URL:

https://vns3-containers-read-all.s3.amazonaws.com/HAproxy_Resolver/HAproxy_Resolver-20180709.tar.gz

This is a read-only Amazon S3 storage location. Only Cohesive Networks can update or modify files stored in this location.

This URL can be used directly in a VNS3 Controller via the Web UI or API to import the container for use into that controller. (General screenshot walkthrough and help available in the plug-in configuration document.)

© 2018

Uploading the Container Image to the VNS3 Plugin System

13

From the Container —> Images menu item, choose Upload Image.

To use the pre-configured plugin paste the URL into the Image File URL box.

© 2018

Allocating a Container from the HA Image

14

When the Image has imported it will say Ready in the Status Column.

To then launch a running HA On Premise container, choose Allocate from the Action menu.

© 2018

Launching the HA On Premise Plugin

15

After selecting Allocate from the Actions menu you then name your container, provide a description and the command used to execute the container.

The name and description should be something meaningful within the context of your organization and its policies.

For the HA Plugin the command used to run plugin containers will be:/usr/bin/supervisord

© 2018

Confirming the HA Plugin is running

16

After executing the Allocate operation you will be taken to the Container Display page.

You should see your HA Plugin with the name you specified. The Status should be Running and it should have been given an IP address on your internal plug-in subnet (in this case 198.51.100.2).

© 2018

Customizing the HAProxy Resolver Plugin

17

© 2018

Accessing the HAProxy Plugin

18

Accessing a Plugin from the Public Internet or your internal subnets will require additions to the inbound hypervisor firewall rules with the VNS3 Controller as well as VNS3 Firewall.

The following example shows how to access an SSH server running in a Plugin listening on port 22.

Network Firewall/Security Group Rule Allow port 22 from your source IP or subnets.

VNS3 FirewallEnter rules to port forward incoming traffic to the Container Network and Masquerade outgoing traffic off the VNS3 Controller’s outer network interface.

#Let the Container Subnet Access the Internet or Local Network Via the VNS3 Controller’s Outer or Public IPPOSTROUTING_CUST -o eth0 -s <HA Container Network IP> -j MASQUERADE

#Port forward port 44 to the BGP HA Plugin port 22PREROUTING_CUST -i eth0 -p tcp -s 0.0.0.0/0 --dport 44 -j DNAT --to <HA Container Network IP>:22

© 2018

Securing the HA container

19

By default the container has the following accounts, configured as described.

“root” - The root account is locked. The root account is not allowed to remote shell into the container. This is our recommended approach. However, if you wish to, you can use the “container_admin” account to unlock root, provide a root password, and edit /etc/ssh/sshd_config to allow remote login by root.

“container_admin” - The default password is “container_admin_123!” The default demo public key is also installed in the /home/container_admin/.ssh/authorized_keys. PLEASE change this password and this key when configuring, or create a new default image as your base for future use, following your authentication procedures. The account “container_admin” has “sudo” or superuser privileges, and is allowed to remote shell into the container.

© 2018

Primary files for customization - Securing SSH

20

There are two significant files for securing the BGP HA container:

• /etc/ssh/sshd_config Please ensure this file is configured to your organization’s best practices.

• /home/container_admin/.ssh/authorized_keysThe base container comes with an example public key installed, and private key for use in VNS3 documentation. Please remove after initial use or programmatic configuration.

© 2018

Primary files for customization - Supervisor

21

/etc/supervisor/conf.d/supervisord.conf

This file defines what services are started when the container is started. Looking at the default you will see Rsyslogd, SSH, and the HAContainer script.

© 2018

Primary file for customization - HAProxy

22

The HA Proxy configuration file, /etc/haproxy/haproxy.cfg, is the primary file to control both the DNS resolution periodicity and target DNS name server and port for the Layer 4 forward proxy.

The following pages provide summary information about the relevant configuration sections to setup the resolve and forwarding behavior for your use-case.

Detailed documentation for configuration of the HAProxy Server DNS Resolution can be found here - http://cbonte.github.io/haproxy-dconv/1.9/configuration.html#5.3.

© 2018

haproxy.cfg - Resolvers Section

23

The resolvers section begins with resolvers <resolvers id> and includes information related to name resolution. There can be as many resolvers sections as needed but this plugin comes with a single section. NOTE: The default configuration uses a loopback address for the name server which allows the VNS3 Plugin system provided by Docker to use the VNS3 host name server. This is the recommended approach but other name servers can be specified.

resolvers aws nameserver aws1 127.0.0.11:53 resolve_retries 3 timeout resolve 1s timeout retry 1s hold valid 30s hold other 30s hold refused 30s hold nx 30s hold timeout 30s hold valid 10s hold obsolete 30s

default settings resolve DNS every 10 seconds when receiving valid responses and 30 seconds when receiving error to avoid flapping.

© 2018

haproxy.cfg - Proxies

24

The default Proxy configuration is included in the frontend <name> and backend <name> sections.

The "frontend" section describes a set of listening sockets that accept client connections in.

The "backend" section describes a set of servers to which the proxy will connect to forward incoming connections. This is where you specify the DNS name (in our example an AWS ALB).

frontend www bind 198.51.100.2:443 default_backend alb-backend

backend alb-backend balance roundrobin mode tcp server alb-1 <alb name>:<alb listenerport> check resolvers aws

© 2018

Forwarding Traffic to the HAProxy Plugin

25

Forwarding traffic to the container uses the same technique as was shown for accessing the container via Remote Shell.

VNS3 Firewall

Add a destination NAT rule to the VNS3 firewall to take traffic coming into the VNS3 controller primary network interface. This will NAT traffic with specific destination IP and destination port to a specific listening port on the HAproxy Resolver Container's IP.

#Destination NAT inbound traffic to a <public or private IP> on a <destination port>and send it to the <HAproxy Listening Port> on the <HAproxy Container IP> PREROUTING_CUST -i eth0 -p tcp -d <public or private IP>/32 --dport <destination port> -j DNAT --to <HAproxy Container IP>:<HAproxy Listening Port>

Here is an example rule: PREROUTING_CUST -i eth0 -p tcp -d 54.232.104.168/32 --dport 443 -j DNAT --to 198.51.100.2:443

© 2018

Running the HAProxy Resolver Plugin

26

© 2018

Run the HAProxy Resolver Plugin

27

Once the container is running on your VNS3 controller, SSH in and update the /etc/haproxy/haproxy.cfg file as specified previously. Save the file. Stop and start the container via the VNS3 UI Containers page.

© 2018

VNS3 Configuration Document Links

28

VNS3 Product Resources - Documentation | Add-ons

VNS3 Configuration Instructions Instructions and screenshots for configuring a VNS3 Controller in a single or multiple Controller topology. Specific steps include, initializing a new Controller, generating clientpack keys, setting up peering, building IPsec tunnels, and connecting client servers to the Overlay Network.

VNS3 Administration Document Covers the administration and operation of a configured VNS3 Controller. Additional detail is provided around the VNS3 Firewall, all administration menu items, upgrade licenses, other routes and SNMP traps.

VNS3 Troubleshooting Troubleshooting document that provides explanation issues that are more commonly experienced with VNS3.