introduction to amazon ec2 container service and setting up build pipeline with ecs and jenkins
TRANSCRIPT
Introduction to Amazon EC2 Container Service and setting up build pipeline with
ECS and Jenkins
Swapnil DahiphaleDevOps Engineer, WhiteHedge Technologies
Agenda
• Containers• Orchestration • Why ECS?• What is ECS?• Key Components• Typical user workflow• Build pipeline with Jenkins and ECS
Containers
“autom ates the deployment of any appl ication as a l ightweight, portable, self-sufficient container that will run virtual ly anywhere”
What are containers?
OS virtualization
Process isolation
Automation
ImagesServer
Guest OS
Bins/Libs Bins/Libs
App2App1
Why use Docker?
With Docker we can solve many problems
• “it works on my machine”• reducing build & deploy time• Infrastructure configuration – automation!• Libs dependency hell• Cost control and granularity
A container pipelineIT Operations
BaseImage
PatchesUtilities
A container pipelineIT Operations
BaseImage
PatchesUtilities
Ruby
Redis
Logger
A container pipelineIT Operations Developer
BaseImage
PatchesUtilities
Ruby
Redis
Logger
App
A container pipeline
BaseImage
PatchesUtilities
IT Operations Developer
Ruby
Redis
Logger
App
Guest OS
Server
Bins/Libs Bins/Libs
App2App1
$ docker run myimage
Orchestration!
Orchestration is a mean to Automate Manual Process
Orchestration
Why Orchestration?
• Microservice Architectures• Fault tolerance / High Availability• Scalability• Resource Utilization
Orchestration Tools
• Kubernetes• Docker Swarm• Amazon EC2 Container Service
Why EC2 Container Service ?
What is ECS?
Building Block Service
What is ECS?
Easily Manage Clusters for Any Scale
• Nothing to run• Complete state• Control and monitoring• Scale
ECS List* and Describe* API actions
What is ECS?
Flexible Container Placement
• Applications• Batch jobs• Multiple schedulers
What is ECS?
Designed for use with other AWS services
• Virtual Private Cloud• Elastic Load Balancing• Elastic Block Store• IAM• CloudTrail
What is ECS?
Performance at scale
What is ECS?
Secure
Your containers,
Your instances
What is ECS?
Extensible
• Comprehensive APIs• Open source agent• Custom schedulers
Key Components
Tasks
Containers
Clusters
Container Instances
Key Components: Container Instances
Amazon EC2 instances
Docker daemon
Amazon ECS agent
Key Components: Clusters
Regional
Resource pool
Grouping of Container Instances
Start empty, dynamically scalable
Key Components: Tasks
Unit of work
Grouping of related containers
Run on Container Instances
Key components: task definitions{
"environment": [],"name": "simple-demo","image": "my-demo", "cpu": 10,"memory": 500, "portMappings": [
{"containerPort": 80,"hostPort": 80
}],"mountPoints": [
{"sourceVolume": "my-vol", "containerPath": "/var/www/my-
vol"}
],"entryPoint": [
"/usr/sbin/apache2", "-D",
"FOREGROUND"],"essential": true
},
{"name": "busybox","image": "busybox", "cpu": 10,"memory": 500, "volumesFrom": [{
"sourceContainer": "simple-demo"
}],"entryPoint"
: [ "sh",
"-c"],"command": [
"/bin/sh -c \"while true; do
/bin/date > /var/www/my-vol/date; sleep 1; done\""
],"essential":
false}
{"environment": [],"name": "simple-demo","image": "amazon/amazon-ecs-sample", "cpu": 10,"memory": 500, "portMappings": [
{"containerPort": 80,"hostPort": 80
}],"mountPoints": [
{"sourceVolume": "my-vol", "containerPath": "/var/www/my-
vol"}
],"entryPoint": [
"/usr/sbin/apache2", "-D",
"FOREGROUND"],"essential": true
},
Key components: task definitions
Essential to our task
10 CPU units (1024 is full CPU), 500 megabytes of memory
Expose port 80 in containerto port 80 on host
Create and mount volumes
{
Key components: task definitions"name": "busybox","image": "busybox", "cpu": 10,"memory": 500, "volumesFrom": [{"sourceContainer":
"simple-demo"}],
"entryPoint": [ "sh","-c"
],"command": [
"/bin/sh -c \"while true; do
/bin/date > /var/www/my-vol/date; sleep 1; done\""
],"essential": false
}
[{
"image": "tutum/wordpress-stackable", "name": "wordpress","cpu": 10,"memory": 500,"essential": true, "links": [
"db"],"entryPoint": [
"/bin/sh","-c"
],"environment": [
…],"portMappings": [
{"containerPor
t": 80,"hostPort":
80}
]},]
From Docker Hub
Mount volume from other container
Command to exec
Typical user workflowI want to run a service
Typical user workflow
Run Instances AmazonEC2
Use custom AMI with Docker support and ECS Agent. Instances will register with default cluster.
Typical user workflow
Create Task Definition
Declare resource requirements for containers
Shared Data Volume
Node.js App Time of day App
Typical user workflow
Create Service
Declare resource requirements for service
Shared Data Volume
Node.js App Time of day App
Elastic Load
Balancing
X 3
Typical user workflow
Describe Service
Demo
Build Pipeline with ECS and Jenkins
Continuous delivery with Jenkins
Build image
Push image
Update service
Q&A
Thank You
github.com/swapnildahiphale
Swapnil Dahiphale
@Swapnil2233