ufo ship for aws ecs

19
UFO Tung Nguyen Jan 2017

Upload: tung-nguyen

Post on 16-Apr-2017

160 views

Category:

Engineering


0 download

TRANSCRIPT

Page 1: Ufo Ship for AWS ECS

UFOTung Nguyen

Jan 2017

Page 2: Ufo Ship for AWS ECS

About Me

I’ve done a little bit of everything. Both dev and ops.

@tongueroo - Twitter.https://medium.com/@tongueroo - Blog.tongueroo.com - Personal Site.

Page 3: Ufo Ship for AWS ECS

lono - CloudFormation template generator.

jack - Manage ElasticBeanstalk environments.

thor_template - used this to generate this tool.

ufo - ship docker images to ecs. What this talk is about.

These tools are in Ruby.More: http://tongueroo.com/projects/

Other Tools

Page 4: Ufo Ship for AWS ECS

Why?Wanted get my hands dirty with ECS.Wanted more control over the ECS task definition.

Wanted to automated the building of the docker image as part of building task definition.

Ending up using it on my side projects.

Page 5: Ufo Ship for AWS ECS

ECS Introduction - TermsTask definition - blueprint for docker container.

Task - Actual running container.Service - Long running tasks.Cluster - Logic group of EC2 Container instances.

Container Instance - EC2 instance that is part of an ECS Cluster.

Page 6: Ufo Ship for AWS ECS

ECS Introduction - Terms

Page 7: Ufo Ship for AWS ECS

Steps for Deploying on ECS1.Build and push docker image2.Build and register task definition 3.Update the ECS service (create the service if needed)

Page 8: Ufo Ship for AWS ECS

Google Search: “deploying to ecs”silinternational/ecs-deploy - pulls down existing task definition.

CircleCI deploy.sh example - nice and simple easy to build from.

Empire - heroku like opensource SaaS from Remind engineering. A little more than I need but cool project.

Programmer Virtue #1: Laziness -> Research

Page 9: Ufo Ship for AWS ECS

Wanted more control over the task definition template.

Would be nice if the task definition could be reused for common and similar processes: web, clock, worker.

More Control

Page 10: Ufo Ship for AWS ECS

Task Definition Similarities$ cat ufo/output/hi-web.json { "family": "hi-web", "containerDefinitions": [ { "name": "web", "image": "tongueroo/hi:ufo-2016-12-04T17-54-23-2d1634d", "cpu": 128, "memoryReservation": 256, "portMappings": [ { "containerPort": "3000", "protocol": "tcp" } ], "command": [ "bin/web" ], "environment": [ { "name": "ADMIN_PASSWORD", "value": "secret" } ], "essential": true } ]}$

$ cat ufo/output/hi-worker.json { "family": "hi-worker", "containerDefinitions": [ { "name": "worker", "image": "tongueroo/hi:ufo-2016-12-04T17-54-23-2d1634d", "cpu": 128, "memoryReservation": 256, "command": [ "bin/worker" ], "environment": [ { "name": "ADMIN_PASSWORD", "value": "secret" } ], "essential": true } ]}$

Page 11: Ufo Ship for AWS ECS

UFO Template DSLUFO will use a ERB template to generate the task definition for all the processes you need: web, clock, worker, etc.

Allows sharing of code and settings.Let’s see what it looks like.

Page 12: Ufo Ship for AWS ECS

UFO Task Definition Template DSL$ cat ufo/templates/main.json.erb { "family": "<%= @family %>", "containerDefinitions": [ { "name": "<%= @name %>", "image": "<%= @image %>", "cpu": <%= @cpu %>, <% if @memory %> "memory": <%= @memory %>, <% end %> <% if @memory_reservation %> "memoryReservation": <%= @memory_reservation %>, <% end %> <% if @container_port %> "portMappings": [ { "containerPort": "<%= @container_port %>", "protocol": "tcp" } ], <% end %> "command": <%= @command.to_json %>, <% if @environment %> "environment": <%= @environment.to_json %>, <% end %> <% if @awslogs_group %> "logConfiguration": { "logDriver": "awslogs", "options": { "awslogs-group": "<%= @awslogs_group %>", "awslogs-region": "<%= @awslogs_region || 'us-east-1' %>", "awslogs-stream-prefix": "<%= @awslogs_stream_prefix %>" } }, <% end %> "essential": true } ]}$

$ cat ufo/task_definitions.rb common = { image: helper.full_image_name, cpu: 128, memory_reservation: 256, environment: env_file(".env")}

task_definition "hi-web" do source "main" variables(common.dup.deep_merge( family: task_definition_name, name: "web", container_port: helper.dockerfile_port, command: ["bin/web"] ))end

task_definition "hi-worker" do source "main" variables(common.dup.deep_merge( family: task_definition_name, name: "worker", command: ["bin/worker"] ))end

task_definition "hi-clock" do source "main" # will use ufo/templates/main.json.erb variables(common.dup.deep_merge( family: task_definition_name, name: "clock", command: ["bin/clock"] ))end$

Page 13: Ufo Ship for AWS ECS

UFO Task Definition Output$ cat ufo/output/hi-web.json { "family": "hi-web", "containerDefinitions": [ { "name": "web", "image": "tongueroo/hi:ufo-2016-12-04T17-54-23-2d1634d", "cpu": 128, "memoryReservation": 256, "portMappings": [ { "containerPort": "3000", "protocol": "tcp" } ], "command": [ "bin/web" ], "environment": [ { "name": "ADMIN_PASSWORD", "value": "secret" } ], "essential": true } ]}$

$ cat ufo/output/hi-worker.json { "family": "hi-worker", "containerDefinitions": [ { "name": "worker", "image": "tongueroo/hi:ufo-2016-12-04T17-54-23-2d1634d", "cpu": 128, "memoryReservation": 256, "command": [ "bin/worker" ], "environment": [ { "name": "ADMIN_PASSWORD", "value": "secret" } ], "essential": true } ]}$

Page 14: Ufo Ship for AWS ECS

ufo ship: whole point of this tool

NOTE: The ECS Cluster and a Container Instance must already exist. I created Container Instances with the lono tool.

1.Build and push docker image

2.Build and register task definition

3.Update the ECS service (create the service automatically if needed)

Page 15: Ufo Ship for AWS ECS

UFO Demo

ufo ship demo

Page 16: Ufo Ship for AWS ECS

What Happened Review Summaryufo init - generates the skeleton ufo files.

Mainly ufo/task_definitons.rb and ufo/templates/main.json.erb - most configurations and changes go in here.

ufo/settings.yml - is nice to know about a useful setting in here.

bin/deploy - wrapper script.

Page 17: Ufo Ship for AWS ECS

In ufo/task_definitions.rb there are some helper methods.

helper.full_image_name - this is the generated docker image name.

helper.dockerfile_port - parsed from the Dockerfile.

env_file - useful helper method to translate a env list to the json formatted list for an ECS Task definition.

Review tasks_definitions.rb Helpers

Page 18: Ufo Ship for AWS ECS

ufo docker build - only build the docker image

ufo tasks build - only build the task definitions

ufo scale hi-web 1 - scale up and down tasks

Misc ufo commands