microservices and amazon ecs
TRANSCRIPT
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Johan Broman, Solutions Architect AWSJude D´Souza, Architect Wrapp
2016-05-04
Deep Dive on Microservices and Amazon ECS
What to Expect from this Session
Microservices: What are They?
Challenges of Microservices
Microservices on Amazon ECS
Containers @ Wrapp, presented by Jude D´Souza
What are Microservices?
What are Microservices?
“is a software architecture style in which complex applications are composed of small, independent
processes communicating with each other using language-agnostic APIs. These services are small, highly decoupled and focus on doing a small task, facilitating a
modular approach to system-building.” - Wikipedia
https://en.wikipedia.org/wiki/Microservices
Monolithic vs. SOA vs. Microservices
SOACoarse-grained
MicroservicesFine-grained
MonolithicSingle unit
Order UI User UI Shipping UI
Order Service
User Service
Shipping Service
Data Access
Monolithic Architecture
Monolithic Architecture – Scaling
Order UI User UI Shipping UI
Order Service
User Service
Shipping Service
Microservices Architecture
Order UI User UI UI
Order Service Service Shipping
Service
Order UIOrder UI
User UI UIShipping UI
Order ServiceOrder
ServiceService
ServiceService
ServiceUser
Service
Shipping Service
Microservices Architecture – Scaling
What are Microservices Challenges?
Resource and state management
Data management
Monitoring
Service discovery
Deployment
Containers are Natural for Services
Simple to model
Any app, any language
Image is the version
Test & deploy same artifact
Stateless servers decrease change risk
What are containers?
Self contained
Process isolation
OS virtualization
AutomationServer
Guest OS
Bins/Libs Bins/Libs
App2App1
Container advantages
Portable
Server
Guest OS
Bins/Libs Bins/Libs
App2App1
Container advantages
Fast
Server
Guest OS
Bins/Libs Bins/Libs
App2App1
Container advantages
Efficient
Server
Guest OS
Bins/Libs Bins/Libs
App2App1
Managing One Host is Straightforward
Server
Guest OS
Bins/Libs Bins/Libs
App2App1
Managing a Fleet is Hard
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
Server
Guest OS
AZ 1 AZ 2
AZ 3
What are Microservices Challenges?
Resource and state management
Data management
Monitoring
Service discovery
Deployment
Amazon EC2 Container Service
Scalable
Designed for use with other AWS services
Elastic Load BalancingAmazon Elastic Block StoreAmazon Virtual Private CloudAWS Identity and Access ManagementAWS CloudTrail
What are Microservices Challenges?
Resource and state management
Data management
Monitoring
Service discovery
Deployment
Scheduling Containers on ECS
Batch jobs
ECS task schedulerRun tasks once
Batch jobsRunTask (random) StartTask (placed)
Long-running apps
ECS service schedulerHealth managementScale-up and scale-downZone-awareGrouped containersELB (optional)
Scheduling Containers: Long-running App
Optionally run your service behind a load balancer.ELB currently supports a fixed relationship between the load balancer port and the container instance port.If a task fails the ELB health check, the task is killed and restarted (until service reaches desired capacity).
Scheduling Containers: Long-running app
Deploy using the least space: minimumHealthyPercent = 50%, maximumPercent = 100%
Scheduling Containers: Long-running App
Deploy quickly without reducing service capacity: minimumHealthyPercent = 100%, maximumPercent = 200%
Scheduling Containers: Long-running App
Blue-Green Deployments
• Define two ECS services• Each service is associated w/ ELB• Both ELBs in Route 53 record set
with weighted routing policy, 100% Primary, 0% Secondary
• Deploy to blue or green service and switch weights
TaskTask
Route 53 record set
with weighted routing policy
0%100%
What are Microservices Challenges?
Resource and state management
Data management
Monitoring
Service discovery
Deployment
Monitoring with Amazon CloudWatch
Metric data sent to CloudWatch in 1-minute periods and recorded for a period of two weeksAvailable metrics: CPUReservation, MemoryReservation, CPUUtilization, MemoryUtilizationAvailable dimensions: ClusterName, ServiceName
Monitoring with Amazon CloudWatch
Monitoring with Amazon CloudWatch
Use the CloudWatch monitoring scripts to monitor additional metrics: e.g., disk space
# Edit crontab
> crontab -e
# Add command to report disk space utilization to CloudWatch every five minutes
*/5 * * * * <path_to>/mon-put-instance-data.pl --disk-space-util --disk-space-used --disk-space-avail --disk-path=/ --from-cron
Monitoring Amazon ECS with Datadog
Monitoring Amazon ECS with Sysdig Cloud
Containers @ WrappPowered by AWS EC2 Container Service
JudeD’SouzaSolutionsArchitect@WrappEmail:[email protected]
AboutMe…
q JudeD’Souza– Stockholm,Swedenß Karachi,[email protected]
q MS– DistributedSystemsKTHRoyalInstituteofTechnology,Stockholm,Sweden
q Passionforbuildinglargescaledistributedsystems
AboutWrapp …Wrapp deliverspersonaloffersandrewardsbasedonwhereyoushop,dineandthebrandsyoufollow.
q Microservice Architecture
q Infrastructureq 13ECSClustersq 50+Instancesq 90+Services
q TechnologyStackq OperatingSystem- Ubuntuq Consul,Registrator,AWS(EC2,ECS,Lambda),Docker,HAProxyq Monitoring– Sensu,Riemannq Logs- Logentries andSumoq Languages- Go,Python,Java
Agenda§ ArchitecturebeforeECS
– ServiceOrchestration– ServiceDiscovery– ServiceDeployments– ServiceAvailability&Scalability– ContainerRegistry
§ TransitiontowardsECS– Microservice Architecture@Wrapp
§ WhyECS?§ NextChallenges§ QA
Howwedidit…BeforeECS(1/3)ServiceOrchestrationq Via runlist configs
Oninstanceboot:q Determineautoscaling groupq Pullrunlist forautoscaling groupq Runcontainersspecifiedinconfigq Addthesetosupervisor
misc ops api
miscAutoscaling group
opsAutoscaling group
api Autoscaling group
PULL RUNLIST CONFIGSSampleRunlist configurationfile
ü ServiceOrchestrationServiceDeploymentsServiceAvailability&ScalabilityContainerRegistry
ServiceDeployments
q Findinstanceswhereserviceisrunning(viaserf)
q SSHintotheseinstances
q Dorollingdeploy(Terminatecontainer,Launchcontainerwithnewversion)
misc ops api
miscAutoscaling group
- misc-i-abc 10.0.0.1 rewards/http/9090- misc-i-def 10.0.1.1 rewards/http/9090- misc-i-ghi 10.0.2.1 rewards/http/9090- ....- ....
Sample serf data for services
SERF CLUSTER
opsAutoscaling group
apiAutoscalinggroup
Howwedidit…BeforeECS(2/3)ü ServiceOrchestrationü ServiceDeploymentsServiceAvailability&ScalabilityContainerRegistry
ServiceAvailabilityq Supervisord
ServiceScalabilityq Serviceitselfcan'tscaleq Scaleonly byscalingautoscaling group
Howwedidit…BeforeECS(3/3)
ContainerRegistryq Hostingourowncontainerregistryq Backedbys3
misc ops api
miscAutoscaling group
SERF CLUSTER
opsAutoscaling group
apiAutoscalinggroup
misc
ü ServiceOrchestrationü ServiceDeploymentsü ServiceAvailability&Scalabilityü ContainerRegistry
ServiceOrchestrationq Runlistsq SupervisorServiceDeploymentsq SupervisorServiceAvailabilityq SupervisorServiceScalabilityq EC2AutoscalingContainerRegistryq Self-hosting
ServiceDiscoveryq Serfq HAProxy
ServiceDiscoveryq Consulq Registratorq HAProxy
ServiceOrchestrationServiceDeploymentsServiceAvailabilityServiceScalability (withlambda)
AWSContainerRegistry
+Servicestats(cpu +memory)
+Servicemonitoring+ServiceProvisioning
TransitiontoECS
Wrapp Microservice Architecture
PUBLICSUBNET
PRIVATESUBNET
ElasticLoadBalancer
api api api
Microservices
ZONEA ZONEB ZONEC
api.wrapp.com DNS
Opsweb – ToolingaroundcontrollingourInfrastructure
ClusterOverview
Opsweb – ToolingaroundcontrollingourInfrastructure
ServicesOverview
ServiceScalability(ConceptfromAmazon)
misc
miscAutoscaling group
ScalingAmazonECSServicesAutomaticallyUsingAmazonCloudWatchandAWSLambda:https://aws.amazon.com/blogs/compute/scaling-amazon-ecs-services-automatically-using-amazon-cloudwatch-and-aws-lambda/
CloudWatch AlarmSNSTopic AWS
Lambda
EC2ContainerService
ECSInstances
TriggerCloudWatch alarmfromECSMetrics(CPU/Memory)
Trigger Lambda function from SNS Topic
+1 DesiredCount in Service
Service (CPU/Memory)Provisioning
api
emailsauth
rewards
notifications
eventbus
TypicalECSInstance
ü MemoryallocationbyService
ü AllocateCPUUnitsbyService
ü EfficientResourceutilization(Containerplacement)
WhyECS?q OurExperience
q Generallygoodservice (ECSisoK)
q Greatsupport
q Wedon’t needtohosttheseservices ourselves onAWS
q Alwaysimprovingtomaketheirservices betterandbetter
ü Supports built-in servicediscovery, loadbalancing androuting
ü WeaveFlux:Supportsbuilt-in servicediscovery,loadbalancing androuting
ü DockerSwarm:Supportsorchestrating containers
ü Fleet- Container Orchestration
api
8080
Consulserver-b
Consul server-a
Consulserver-c
users
8082
auth
8081
users
8082
Registrator Registrator Registrator
ConsulAgent
ConsulAgent
ConsulAgent
Key/value store
Servicecatalog
10.0.0.1 10.0.0.2 10.0.0.3
ServiceDiscovery@Wrapp (1/2)
HAConsulCluster
NextChallenges(1/2)– Shortcomingsperceived…
Built-in(likekube-proxy,weave-proxy)?q ServiceDiscoveryq ServiceRoutingq ServiceLoadbalancing
Approach1:ELBPerservice?q Costly- 90+services,70+ELBs
Approach 2:SingleELBforallservices?ü ServiceRoutingworksü ServiceLoadbalancingworksq Nohealthcheckperserviceq NowaytoreturnHTTP503
(butpossibleviahaproxy httpcheckoption)
Benefits:q RemovestoolslikeConsulandRegistrator
Elastic Load Balancer
8080 8081 8082
ECS Boto3:create_service()
Route53: ELB.dns = elb-services
frontend users-inbind users:80mode httpdefault_backend users
backend usersserver users elb-services:8080 …
####
>> curl users/ping
SINGLE ELB FOR MULTIPLE SERVICES
NextChallenges(2/2)– Shortcomingsperceived…
Ø Cron schedulingq Setupecs taskstorunoncron schedule
Ø ECSEventstreamq Actionables oncertainevents(whendeploycompletes)
Ø ECSContainerLogging
Ø OptionallyIntegrateECSClusterwithEC2Autoscaling Group
Summary
q TheWrapp ArchitecturebeforeECS
q TransitiontoECS
q ServiceDiscovery@Wrapp withConsul,Registrator andHAProxy
q Ourperceived shortcomingsfromECSq Built-inServiceDiscoveryq Cron schedulingonTasksq Eventstream
What are Microservices Challenges?
Resource and state management
Data management
Monitoring
Service discovery
Deployment
Continuous Delivery to ECS with Jenkins
4. Push image to Docker registry
2. Build image from sources 3. Run test on image
1. Code push triggers build
5. Update service
6. Pull image
ECS CI/CD Partners
Continuous Delivery to ECS with Shippable
Thank you!