aws re:invent 2016: from ec2 to ecs: how capital one uses application load balancer features to...
TRANSCRIPT
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Jeff Storey, Engineering Tools Lead, Capital One
Kit Ewbank, Lead Cloud Engineer, Capital One
Peter Dalbhanjan, Solutions Architect, AWS
12/01/2016
From Amazon EC2 to Amazon ECS
How Capital One Uses Application Load
Balancer Features to Serve Traffic at ScaleNET203
What to Expect from the Session
• Classic Load Balancer / Application Load Balancer
features
• Load balancers at Capital One
• Capital One’s container journey
ELB/ALB features
Elastic Load Balancing automatically distributes
incoming application traffic across multiple
Amazon EC2 instances.
EC2
Instance
Single instance
Classic Load Balancer
ELB
EC2
Instance
EC2
Instance
EC2
Instance
HTTP/HTTPS
TCP/SSL
ELB
EC2
Instance
EC2
Instance
EC2
Instance
example.com/app2
example.com/app1
app1
app2
app1/
app2
Application Load Balancer
• Health checks
• Idle timeouts
• Cross-zone
• Path-based routing
• Integration with AWS services
• Security
• Monitoring
• Logging
Features – at a glance
Support for TCP and HTTP health checks.
Customize the frequency and failure
thresholds.
Must return a 2xx response.
Consider the depth and accuracy of your
health checks.
Health Checks
Length of time that an idle connection should be kept open.
For both client and back-end connections.
Defaults to 60 seconds but can be set between 1 and 3,600
seconds.
Timeouts should decrease as you go
up the stack.
Idle Timeouts
15s
3s
3sELB
15sEC2
Instances
Amazon S3
Amazon RDS
Amazon SWF
3s
9s
Idle Timeouts
Load balancer absorbs impact of DNS caching.
Eliminates imbalances in back-end instance utilization.
Requests distributed evenly across multiple
Availability Zones.
Check connection limits before enabling.
No additional bandwidth charge for
cross-zone traffic.
Cross-Zone Load Balancing
• Health Checks
• Idle Timeouts
• Cross-Zone
• Path-based routing
• Integration with AWS Services
• Security
• Monitoring
• Logging
Features – At a Glance
Load balancing at Capital One
Why are load balancers so important to Capital
One?
• REST APIs everywhere
• Numerous API implementation technologies
• Load balancing as a service
Load balancers at Capital One
• Load balancers everywhere
• ELBs in front of Auto Scaling groups
• ELBs in front of Amazon ECS services
• ALBs in front of Auto Scaling groups
• ALBs in front of Amazon ECS services
Classic Load Balancer
ELB
EC2 in ASG
Apache mod_proxy
EC2 in ASGELB
/credit-card
/mortgage
What we manage for a “simple” proxy
• Classic LB
• Auto Scaling group
• EC2 instances
• Provisioning
• Logging/monitoring/security
• Apache proxy
• Versions/patching
• Configuration/rules
Application Load Balancer
ELB
EC2 in ASG
/credit-card
/mortgage
What we manage for an ALB proxy
• ALB target groups
Target groups
What are target groups?
• A set of targets
• Targets dynamically register with target groups
• Different routes for each target group
Target Group
Path-based routing
Load Balancer
ListenerRules
Target Target
Health
Check Target Group
ListenerRules
Target Target
Health
Check
Port 80 Port 8080
/credit-card /mortgage
Capital One’s container journey
Container orchestration – the problem
Container Instance
Container
Container
Container
?
ECS containers with Classic ELBs
• Auto Scaling with EC2 instances and ELBs just works
• For ECS, not so much
Fixed port mapping
Container Instance
Service 1
Container Port 80
Host Port 8080
Service 2
Container Port 80
Host Port 8081
Container Instance
Service 1
Container Port 80
Host Port 8080
Service 2
Container Port 80
Host Port 8081
Load Balancer
Service 1
Port 80
Load Balancer
Service 2
Port 80
Container Instance
nginx + registrator + consul + consul-template
Container
registratorContainer
Container
Consul Clusternginx
consul-template
Container Instance
nginx + registrator + confd + DynamoDB
Container
registratorContainer
Container
confd
DynamoDB
Lambda TTL
function
nginx
Container Instance
Container orchestration with ALB
Container
Container
Target Group
Container
Target Group
Auto Scaling
• Use Amazon CloudWatch alarms to trigger Auto Scaling
• Container host Auto Scaling
• Service-level Auto Scaling
ALB health checking
• Health checks defined for each target group
• Layer 7 health checks only
• Customizable range of “healthy” status codes – not just
200
Deployments
Capital One deployments
AWS Region
Container
Container
Container
Container
Container
Container
AWS Region
Container
Container
Container
Container
Container
Container
Amazon
Route 53
Built-in ALB deployments with ECS
Container Container Container
Container Container Container
X X XTarget Group
ALB deployments with ECS
Container
Container
Target Group
Container
Container
Target Group
Operating ALBs at scale
Transport Security
• Layer 7
• HTTPS security
• Integration with Amazon Certificate Manager (ACM)
• TLS terminated at ALB
• May be re-wrapped to target
Logging
• ALB access logs
type | timestamp | elb | client:port | target:port | request_processing_time | target_processing_time | response_processing_time | elb_status_code | target_status_code | received_bytes | sent_bytes | "request" | "user_agent" | ssl_cipher | ssl_protocol | target_group_arn
Monitoring
• CloudWatch metrics provided for each target group
• Includes most metrics that were available in ELBs
• New notable metrics:
• ActiveConnectionCount
• NewConnectionCount
• ProcessedBytes
Cost
• Cost model differs for Classic LB vs. ALB
• Classic: Base price + Data transfer charges
• ALB: Base price + Load Balancer Capacity Unit (LCU)
charges
• LCU – Combination of new connections + active connections
+ bandwidth
• Base price 10% less for ALB
Product feedback
ALB desired features
• Blue/green deployments with weighted routing built into
ECS
• Host-based routing vs. path-based routing
• Support for two-way TLS
Conclusion
• Capital One teams continue to move to Application Load
Balancers for newly developed services
• New features drastically simplify ECS integration
• Like every AWS service, we expect it to only get better
as it matures
Thank you!
Remember to complete
your evaluations!