astricon 2017 - docker swarm & asterisk
TRANSCRIPT
Putting the
in
Docker Swarm
What I do: am a human phone assistant
What I do: assist phones.
Who am I
+
• Lightweight and distributable containers• Containers should be single-process, no durable storage, ephemeral little
creatures• Scalability due to built in resilience and orchestration layers• Higher packing density than virtual machines due to shared libs
What is Docker
Docker Swarm mode
• Orchestration layer (Swarm) is built into Docker Engine• Easily scale containers across multiple nodes• Easy, multi-host VXLAN routing• Built in state reconciliation, health checks, and designed for resilient deployments• Mesh networking allows ingress from any node to reach any other node
• Auto-load balancing of inbound requests utilizes in-kernel Linux IPVS• Built-in DNS-based service discovery• Rolling Updates• Ease of Scaling• A free basket of puppies (Docker EE only)
Deploying Your First Swarm
• Spin up a few machines with your favorite OS • We’ll be using Debian 9
Install Docker CE
Deploying Your First Swarm
sudo apt-get updatesudo apt-get install \ apt-transport-https \ ca-certificates \ curl \ gnupg2 \ software-properties-commoncurl -fsSL https://download.docker.com/linux/$(. /etc/os-release; echo "$ID")/gpg | sudo apt-key add -sudo add-apt-repository \ "deb [arch=amd64] https://download.docker.com/linux/$(. /etc/os-release; echo "$ID") \ $(lsb_release -cs) \ stable"sudo apt-get updatesudo apt-get install docker-ce
Voila!root@astricon-sfo-1:~# docker -vDocker version 17.09.0-ce, build afdb6d4
Initialize the Swarm• TCP 2377 for cluster management communications • TCP & UDP port 7946 for communication among nodes • UDP port 4789 for overlay network traffic
Deploying Your First Swarm
root@astricon-sfo-1:~# docker swarm init —advertise-addr 107.170.205.26
Now run that join line on each worker root@astricon-london-1:~# docker swarm join --token SWMTKN-1-4n2nbpa5lntnbpomq8h412xliz9orjcxlrdjwhrg2v756nzfg0-0q4061v448gvcnqckd8t1z75k 107.170.205.26:2377
This node joined a swarm as a worker.
Swarm initialized: current node (uri5s1oes546txryvdv5hyu69) is now a manager.
To add a worker to this swarm, run the following command:
docker swarm join --token SWMTKN-1-4n2nbpa5lntnbpomq8h412xliz9orjcxlrdjwhrg2v756nzfg0-0q4061v448gvcnqckd8t1z75k 107.170.205.26:2377
To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.
Success
Your Swarm Is Ready!
root@astricon-sfo-1:~# docker node lsID HOSTNAME STATUS AVAILABILITY MANAGER STATUSzaecqxqlgj5n1jtxwi34r08cg astricon-london-1 Ready Activegq45t7og425dmanur0b4fz0p6 astricon-nyc-1 Ready Activeuri5s1oes546txryvdv5hyu69 * astricon-sfo-1 Ready Active Leader
• Only a single Manager • Managers are also workers with a little more overhead • All nodes report ready
Let’s Get Our Asterisk Swarm On
Swarm mode• Build your own Asterisk or utilize an existing image from Docker Hub
• Docker Hub is built into all engines: • We’ll be using:
• Now we need to define our services • We need to build a Stack file• A stack file is simply a YAML file in Compose format that lists the various
services run by the swarm
docker pull <IMAGE>
respoke/asterisk:14
Let’s Break For
version: "3.4"
services:
### To demonstrate mesh networking, let’s use Nginx nginx: image: nginx ports: - 80:80 networks: - astricon-is-radical
networks: astricon-is-radical: attachable: true
Stack file for Nginx
root@astricon-sfo-1:~# docker stack deploy -c nginx.yml astricon_swarmCreating network astricon_swarm_astricon-is-radicalCreating service astricon_swarm_nginx
• Deploy the Stack!
• Check the Stack! root@astricon-sfo-1:~# docker service lsID NAME MODE REPLICAS IMAGE PORTS7a1292n9v49n astricon_swarm_nginx replicated 1/1 nginx:latest *:80->80/tcp
• Scale the Stack! root@astricon-sfo-1:~# docker service scale —-detach=true astricon_swarm_nginx=2astricon_swarm_nginx scaled to 2
ID NAME MODE REPLICAS IMAGE PORTS7a1292n9v49n astricon_swarm_nginx replicated 2/2 nginx:latest *:80->80/tcp
NYC-1 SFO-1 LONDON-1
NYC-1 SFO-1 LONDON-1
astriconswarm.io• All requests pointed at IP addresses
associated with the swarm will be load-balanced to machines with that exposed port
• Even machines that aren’t running anything! Every node becomes and ingress node and will automatically address the correct service.
• You can ping by service name!
Overlay Network
172.18.0.2/16
Back to
version: "3.4"
services:
### The Big Kahuna! asterisk: image: respoke/asterisk:14 networks: - host_mode deploy: placement: constraints: [node.labels.service == asterisk]
networks: host_mode: external: name: 'host'
Asterisk Stack FileThe Paralysis of Choice
version: "3.4"
services:
### The Big Kahuna! asterisk: image: respoke/asterisk:14 networks: - astricon-is-radical deploy: placement: constraints: [node.labels.service == asterisk]
networks: astricon-is-radical: attachable: true
Host Mode Overlay Network
Choosing a Network Mode• The issue: opening ports in Docker uses iptables
• Opening many ports (*caugh RTP caugh*) results in a severe memory crunch• Each UDP port = ~1MB in RAM (Debian 9.1, DO)
• Overlay Mode• Has full access to service discovery and mesh networking• Can’t run many open ports — perhaps a few hundred at most.
• Host Mode• No access to overlay or service discovery network• Has full access to the underlying networking subsystems• Unlimited ports
SECRET TRICKCIP=$(sudo docker inspect --format='{{.NetworkSettings.IPAddress}}' $CID)
sudo iptables -A DOCKER -t nat -p udp -m udp ! -i docker0 --dport 10000:65535 -j DNAT --to-destination $CIP:10000-65535sudo iptables -A DOCKER -p udp -m udp -d $CIP/32 ! -i docker0 -o docker0 --dport 10000:65535 -j ACCEPTsudo iptables -A POSTROUTING -t nat -p udp -m udp -s $CIP/32 -d $CIP/32 --dport 10000:65535 -j MASQUERADE
Shout out to BetterVoice
• Allows for a large port range to be opened with a single iptables rule• Must be run on any machine that is running Asterisk and can’t be run from
inside the container (Swarm doesn’t allow privilege escalation)
version: "3.4"
services:
### The Big Kahuna! asterisk: image: respoke/asterisk:14 networks: - host_mode deploy: placement: constraints: [node.labels.service == asterisk]
networks: host_mode: external: name: 'host'
Asterisk Stack Fileversion: "3.4"
services:
### The Big Kahuna! asterisk: image: respoke/asterisk:14 networks: - astricon-is-radical deploy: placement: constraints: [node.labels.service == asterisk]
networks: astricon-is-radical: attachable: true
Revisited
• Better for small business deployments • Simpler to manage, requires no extra
DevOps (with small port ranges) • One at a time
• Works more like a traditional Asterisk installation
• Still functions with built-in HA / health checks / swarm deployment options
• Many at a time
Additional Pieces
DNS - Route 53
• Setup Geolocation to route to closest server
• SRV records point to multiple datacenters
• Health checks or API calls to verify uptime/downtime
NYC-1 SFO-1 LONDON-1
External DNS
Docker DNS
Docker Healthchecks
version: "3.4"
services:
### The Big Kahuna! asterisk: image: respoke/asterisk:14 networks: - host_mode deploy: placement: constraints: [node.labels.service == asterisk] healthcheck:
test: ["CMD", "sipsak", "-s", "sip:[email protected]"] interval: 30s timeout: 15s retries: 3
networks: host_mode: external: name: 'host'
Docker Healthcheck
• Send an OPTIONS ping every 30s • If failure occurs, retry 3 times with a wait
period of 15 seconds
Data Persistence
Secrets Management docker secrets create
Node labels docker node update --label-add
Private Registry docker service update --with-registry-auth
Final Tips• Make your images generic
• E.G. RingPlus abstracted away all business logic to API calls. This made our B2BUAs functionally interchangeable.
• Cows, not kittens.• Leverage all the pieces of Swarm that you can.
• It can replace a bunch of auxiliary tools and cron scripts• It forces you to think of things as components in a larger system and how
they all fit together. • No single VM running 15 different components installed by a consultant
or someone who left the company a year ago.• Your deployments become far less brittle and are easier to test. • Development / Production Parity
