Container Cluster Management with AWS ECS

Anshul PatelTalentica DevOps Engineering Team

What is Docker container ?

Open platform to build, ship and run distributed applications

Groups application with all the dependencies, shared libs

Compared to Traditional Virtualized system, which encapsulates

OS + Kernel, Docker has same Kernel Space

Isolation between different containers is done through

Namespaces, Cgroups, Network Interfaces

Current Docker Engine Stable version 1.11.2 (2016-05-31)

Default Engine Execution Driver : libcontainer

Docker uses libcontainer to manipulate Namespaces, Cgroups,

Network Interfaces

Why Docker ?

Lightweight, Open and Secure

Portable and efficient in comparison to VM

Accelerates Developer onboarding

Empower Developer creativity

Eliminates Environmental Inconsistencies

Ability to scale quickly

Reduces time to market of your application

Why Docker ?

Why Container Cluster Management System ?

Provides clustering layer for controlling the deployment of your

containers onto the underlying hosts

Manages container lifecycle within the cluster

Scheduling Containers across the cluster

Abstracting developers away from underlying machines

Scaling containers

Increases robustness and resilience of distributed containerized

application

What is AWS ECS (EC2 Container Service) ?

Highly scalable, fast, container management service from Amazon

Web Services.

Easily run, stop and manage Docker containers on cluster of

Amazon EC2 instances

Schedules the placement of Docker containers across your

cluster based on resource needs, availability and requirements

Eliminates need for you to operate your own cluster management

systems or write wrappers/logic for scaling your underlying

infrastructure

Components of ECS

Cluster - Logical group of container instances

Container Instance - Runs ECS agents and is registered to

cluster

Task Definition - Description of application to be deployed

Scheduler - Method of placing task on container instance

Task - An instantiation of task definition running on container

instance

Service - Runs and maintains predefined tasks simultaneously

Container - Docker Container created during task instantiation

ECS Architecture

ECS Architecture

Key Components

Agent Communication Service - Gateway between ECS agents

and ECS backend cluster management engine

API - Provides cluster state information

Cluster Management Engine - Provides cluster coordination and

state management

Key/Value Store - It is used to storing cluster state information

ECS Architecture

It isn’t a blackbox, runs on your own EC2(Container) instances

ECS cluster is collection of EC2(Container) Instances

ECS agent is installed on each of EC2(Container) Instances

ECS agent registers instance to centralised ECS service

ECS agent handles incoming requests for container deployment

ECS agent handles the lifecycle of container

Creating ECS Cluster

Cluster can be created using AWS Console, alternatively you can

create using aws ecs CLI

AWS Container Instance Requirement

Linux Kernel > = 3.10

Docker > = 1.5.0

ECS container agent

Nanny process to monitor ECS agent (For eg: ecs-init )

Amazon ECS optimized AMI is preconfigured with these

requirements

Launching EC2 container Instances

Points to remember

ec2InstanceRole must be created

If the EC2 instance is required to be part of pre-defined cluster,

following should be added in user data#/bin/bash

echo ECS_CLUSTER=your_cluster_name >> /etc/ecs/ecs.config

During the launch of AMI, it will consist of two EBS, one for OS

and another for Docker’s use.

ECS Task Definition

Task definition is 1 or more container definitions

It defines

Docker Images to use

Port and Drive Volume Mapping

CPU and memory to use with container

Whether containers are linked

Environmental variable which is required to be passed to

container

Whether task should continue if container finishes or fails

ECS Scheduler

By default, AWS supports two ways for scheduling

Running Tasks Services

Apart from the above, StartTask API of ECS can be used to write

custom scheduler. Currently Amazon has developed PoC scheduler

for Mesos

ECS Running Tasks

Instantiation of Task Definition

Task can be either short lived or long lived. (For eg: event-driven-

data task or batch jobs)

Architecturally all containers belonging to same task, have to run

on the same machine. In order to run container on different

machine, new task should be created.

Co-location of containers can be achieved by grouping them in the

same task

ECS services

Allows you to run and maintain a specified/desired number of

instances ( For eg : webservers )

If any tasks should fail or stop for any reason, ECS service

scheduler launches another instance of your task definition to

maintain desired count

Deploying ECS Cluster

Create a Load Balancer

Create a Launch configuration

Create a Autoscaling group, which specifies the desired number of

instances

Create a task definition

Create a service

ECS Container Instance Autoscaling

ECS provides cluster-level parameters which can give the cluster

utilization statistics

MemoryReservation - Current % of reserved memory by

cluster

MemoryUtilization - Current % of utilized memory by cluster

CPUReservation - Current % of reserved CPU by cluster

CPUUtilization - Current % of utilized CPU by cluster

CloudWatch Alarms on the above parameters enables to Scale

Up/Down the ECS cluster

ECS Task Autoscaling

ECS also provides the facility to scale up the tasks in the service.

Tasks can be autoscaled on following ECS service parameters

CPUUtilization - Current % CPU utilization by ECS service

MemoryUtilization - Current % Memory Utilization by ECS

service

EC2 Container Registry

AWS managed Docker container registry

Stores and Manages Docker Images

Hosts images in a highly available and scalable architecture

It is integrated with ECS

No upfront fee, pay only for the data stored

Key Advantages of ECS Service

ECS is monitoring status of Docker containers, so if it goes down

it will be deployed automatically

ASG is monitoring your EC2 instances, if one instance goes

down, it will spawn new instance

ECS can do zero-downtime deployments of new version ( Blue -

Green deployments )

Updation requires two things

Create new revision of task

Update the revision in the service

Key Challenges

No direct relation between EC2 Autoscaling and Task Autoscaling

ELB works with 1:1 port mapping, so if the number of tasks is

equal to the number of instances, scheduler will not attempt to

deploy another task

Service Discovery

ECS Limits

Resource Default LimitNumber of clusters per region, per account 1000Number of container instances per cluster 1000Number of services per cluster 500

ECS Limits

Resource Default LimitNumber of load balancers per service 1Number of tasks per service (the desired count)

1000

Number of tasks launched (count) per run-task

10

Number of container instances per start-task

10

Throttle on container instance registration rate

1 per second / 60 max per minute

Task definition size limit 32 KiBTask definition max containers 10Throttle on task definition registration rate 1 per second / 60 max per minute

Thanks & Questions ?