Amazon EC2 Container Service: Manage Docker-Enabled Apps in EC2

Chris BarclayPrincipal Product Manager

Running Batch Jobs on Amazon EC2 Container Service

2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

AgendaContainersEC2 Container ServiceCommon patternsDemoQ&A

2

Containers

What are Containers?

OS virtualizationProcess isolationImagesAutomation ServerGuest OSBins/LibsBins/LibsApp2App1

4

Container Advantages

Portable

ServerGuest OSBins/LibsBins/LibsApp2App1

5

Container Advantages

Flexible

ServerGuest OSBins/LibsBins/LibsApp2App1

6

Container Advantages

Fast

ServerGuest OSBins/LibsBins/LibsApp2App1

7

Container Advantages

Efficient

ServerGuest OSBins/LibsBins/LibsApp2App1

8

ServerGuest OSBins/LibsBins/LibsApp2App1

9

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

ServerGuest OS

10

EC2 Container Service Benefits

11

Easily Manage Clusters for Any ScaleNothing to runComplete stateControl and monitoringScale

12

Flexible Container PlacementApplicationsBatch jobsMultiple schedulers

13

Designed for Use with Other AWS ServicesElastic Load BalancingAmazon Elastic Block StoreAmazon Virtual Private CloudAWS Identity and Access ManagementAWS CloudTrail

14

ExtensibleComprehensive APIsOpen source agentCustom schedulers

15

EC2 Container Service Terminology

16

Amazon EC2 instances

Docker daemon

Amazon ECS agentKey Components: Container Instances

17

Regional

Resource pool

Grouping of Container Instances

Start empty, dynamically scalableKey Components: Clusters

18

Key Components: Task Definitions

Volume DefinitionsContainer Definitions

19

Key Components: Task Definitions

Shared Data VolumePHP AppTime of day App

20

Key Components: Task Definitions { "environment": [], "name": "simple-demo", "image": "my-demo", "cpu": 10, "memory": 500, "portMappings": [ { "containerPort": 80, "hostPort": 80 } ], "mountPoints": [ { "sourceVolume": "my-vol", "containerPath": "/var/www/my-vol" } ], "entryPoint": [ "/usr/sbin/apache2", "-D", "FOREGROUND" ], "essential": true },{ "name": "busybox", "image": "busybox", "cpu": 10, "memory": 500, "volumesFrom": [ { "sourceContainer": "simple-demo" } ], "entryPoint": [ "sh", "-c" ], "command": [ "/bin/sh -c \"while true; do /bin/date > /var/www/my-vol/date; sleep 1; done\"" ], "essential": false }

{ "environment": [], "name": "simple-demo", "image": "my-demo", "cpu": 10, "memory": 500, "portMappings": [ { "containerPort": 80, "hostPort": 80 } ], "mountPoints": [ { "sourceVolume": "my-vol", "containerPath": "/var/www/my-vol" } ], "entryPoint": [ "/usr/sbin/apache2", "-D", "FOREGROUND" ], "essential": true },Key Components: Task Definitions[ { "image": "mysql", "name": "db", "cpu": 10, "memory": 500, "essential": true, "entryPoint": [ "/entrypoint.sh" ], "environment": [ { "name": "MYSQL_ROOT_PASSWORD", "value": "pass" } ], "portMappings": [] }]

Essential to our TaskCreate and mount volumesExpose port 80 in containerto port 80 on host10 CPU Units (1024 is full CPU),500 Megabytes of Memory

{ "name": "busybox", "image": "busybox", "cpu": 10, "memory": 500, "volumesFrom": [ { "sourceContainer": "simple-demo" } ], "entryPoint": [ "sh", "-c" ], "command": [ "/bin/sh -c \"while true; do /bin/date > /var/www/my-vol/date; sleep 1; done\"" ], "essential": false }Key Components: Task Definitions[ { "image": "tutum/wordpress-stackable", "name": "wordpress", "cpu": 10, "memory": 500, "essential": true, "links": [ "db" ], "entryPoint": [ "/bin/sh", "-c" ], "environment": [ ], "portMappings": [ { "containerPort": 80, "hostPort": 80 } ] }, ]

From Docker HubMount volume from other containerCommand to exec

Key Components: Tasks

Container InstanceSchedule

Shared Data VolumePHP AppTime of day App

Shared Data VolumePHP AppTime of day App

24

Unit of work

Grouping of related Containers

Run on Container InstancesKey Components: Tasks

25

Key Components: Run a Task

Good for short-lived containers, e.g. batch jobs

26

Key Components: Create a Service

Good for long-running applications and services

27

Key Components: Create ServiceLoad Balance traffic across containersAutomatically recover unhealthy containersDiscover services

Shared Data VolumePHP AppTime of day App

Shared Data VolumePHP AppTime of day App

Shared Data VolumePHP AppTime of day App

Elastic Load Balancing

28

Key Components: Update ServiceScale upScale down

Elastic Load Balancing

Shared Data VolumePHP AppTime of day App

Shared Data VolumePHP AppTime of day App

Shared Data VolumePHP AppTime of day App

Shared Data VolumePHP AppTime of day App

Key Components: Update ServiceDeploy new versionDrain connections

Elastic Load Balancing

Shared Data VolumePHP AppTime of day App

Shared Data VolumePHP AppTime of day App

Shared Data VolumePHP AppTime of day App

Key Components: Update ServiceDeploy new versionDrain connections

Shared Data VolumePHP AppTime of day App

Shared Data VolumePHP AppTime of day App

Shared Data VolumePHP AppTime of day App

Elastic Load Balancing

Key Components: Update ServiceDeploy new versionDrain connections

Shared Data VolumePHP AppTime of day App

Shared Data VolumePHP AppTime of day App

Elastic Load Balancing

Shared Data VolumePHP AppTime of day App

Batch jobs

33

Containers and Batch JobsEasy to updateAny app, any languageIncludes all dependenciesImage is the versionTest & deploy same artifact

When to Use ECS and LambdaECSLong-running jobsManage your event triggeringAny language, any dependencyResources are your own - use Spot, RIsLambdaShort-lived jobsTriggered on specific eventsSupports specific environmentsNo infrastructure to manage

35

Demo: Batch Jobs with ECS

.zip

s3://bucket/scene.zip

ecs:RunTaskECS Cluster

.png

36

Demo

37

Q&A

AWS re:Invent 2015 October 6-9AWS re:Invent is the largest annual gathering of the global cloud community. Whether you are an existing customer or new to the cloud, AWS re:Invent will provide you with the knowledge and skills to refine your cloud strategy, improve developer productivity, increase application performance and security, and reduce infrastructure costs.

Though AWS re:Invent tickets are sold out, you can still register to view the Live Stream Broadcasts of the keynote addresses and select technical sessions on October 7 and October 8. Register now.

Details:Wednesday, October 79:00am - 10:30am PT: Andrew Jassy, Sr. Vice President, AWS11:00am - 5:15pm PT: 5 of the most popular breakout sessions (to be announced)

Thursday, October 89:00am - 10:30am PT: Dr. Werner Vogels, CTO, Amazon11:00am - 6:15pm PT: 6 of the most popular breakout sessions (to be announced)

Register now for the Live Stream Broadcast by submitting your email where prompted on the AWS re:Invent home page.

Stay Connected: Follow event activities on Twitter @awsreinvent (#reinvent), or like us on Facebook.

39

Thank you!