![Page 1: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/1.jpg)
The Evolution of MicroservicesAdrian Cockcroft @adrianco
Technology Fellow - Battery Ventures Craft Conference April 2016
![Page 2: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/2.jpg)
What does @adrianco do?
@adrianco
Technology Due Diligence on Deals
Presentations at Conferences
Presentations at Companies
Technical Advice for Portfolio
Companies
Program Committee for Conferences
Networking with Interesting PeopleTinkering with
Technologies
Maintain Relationship with Cloud Vendors
Previously: Netflix, eBay, Sun Microsystems, CCL, TCU London BSc Applied Physics
![Page 3: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/3.jpg)
Agenda
Why now? Microservice Architectures
What’s Missing? Migration and Simulation
What’s Next?
![Page 4: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/4.jpg)
Key Goals of the CIO? Align IT with the business Develop products faster Try not to get breached
![Page 5: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/5.jpg)
Security Blanket Failure
Insecure applications hidden behind firewalls make you feel safe until the breach happens…
http://peanuts.wikia.com/wiki/Linus'_security_blanket
![Page 6: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/6.jpg)
“Web scale” vs.
“Enterprise”
![Page 7: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/7.jpg)
“Webscale”
Freedom and responsibility High trust
![Page 8: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/8.jpg)
“Enterprise”
Bureaucracy and blame Low trust
![Page 9: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/9.jpg)
How can everyone get speed, low cost, and better
usability?
![Page 10: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/10.jpg)
Mixed methods: Disaggregation into microservices helps!
![Page 11: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/11.jpg)
@adrianco
Example Monolith:
Sign Up Login Home
Page
Payment Method
Personal Data Reports
Monolithic “kitchen sink”
database
Monolithic application
Complex mix of queries
User
Because one part of the monolithic application and database holds sensitive data all of it is subject to the most rigorous policies
![Page 12: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/12.jpg)
@adrianco
Microservices version:
Sign Up Login Home
Page
Payment Method
Personal Data ReportsOptimized
datastores
Microservices separation of concerns
Isolated single purpose connections
User
Because each microservice can conform to the appropriate policy, demands for agility can be separated from requirements for security
Segregated team owns secure data sources and
infrequent updates
Segregated team owns rapid improvement of
most common use cases
![Page 13: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/13.jpg)
@adrianco
In Search of Segmentation
Ops
Dev
AWS Accounts / Datacenters IAM/AD/LDAP Roles VPC/VLAN Networks
Security Groups/Hypervisor IPtables/Calico Policy
Docker Links/Weave Overlay
![Page 14: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/14.jpg)
@adrianco
Hierarchical Segmentation
B CA B C E FD E F
Homepage Team Security Group Reports Team Security GroupVPC Z - Manage a small number of large network spaces
D
An AWS oriented example…
AWS Account - Manage across multiple accounts
containers and links
![Page 15: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/15.jpg)
“You build it, you run it.” Werner Vogels 2006
![Page 16: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/16.jpg)
Developer responsibilities: Faster, cheaper, safer
![Page 17: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/17.jpg)
Speeding Up The Platform
Datacenter Snowflakes• Deploy in months• Live for years
![Page 18: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/18.jpg)
Speeding Up The Platform
Datacenter Snowflakes• Deploy in months• Live for years
Virtualized and Cloud• Deploy in minutes• Live for weeks
![Page 19: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/19.jpg)
Speeding Up The Platform
Datacenter Snowflakes• Deploy in months• Live for years
Virtualized and Cloud• Deploy in minutes• Live for weeks
Container Deployments• Deploy in seconds• Live for minutes/hours
![Page 20: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/20.jpg)
Speeding Up The Platform
Datacenter Snowflakes• Deploy in months• Live for years
Virtualized and Cloud• Deploy in minutes• Live for weeks
Container Deployments• Deploy in seconds• Live for minutes/hours
Lambda Deployments• Deploy in milliseconds• Live for seconds
![Page 21: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/21.jpg)
Speeding Up The Platform
AWS Lambda is leading exploration of serverless architectures in 2016
Datacenter Snowflakes• Deploy in months• Live for years
Virtualized and Cloud• Deploy in minutes• Live for weeks
Container Deployments• Deploy in seconds• Live for minutes/hours
Lambda Deployments• Deploy in milliseconds• Live for seconds
![Page 22: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/22.jpg)
What Happened?Rate of change
increased
Cost and size and risk of change
reduced
![Page 23: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/23.jpg)
Microservices
![Page 24: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/24.jpg)
A Microservice Definition
Loosely coupled service oriented architecture with bounded contexts
![Page 25: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/25.jpg)
A Microservice Definition
Loosely coupled service oriented architecture with bounded contexts
If every service has to be updated at the same time it’s not loosely coupled
![Page 26: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/26.jpg)
A Microservice Definition
Loosely coupled service oriented architecture with bounded contexts
If every service has to be updated at the same time it’s not loosely coupled
If you have to know too much about surrounding services you don’t have a bounded context. See the Domain Driven Design book by Eric Evans.
![Page 27: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/27.jpg)
Microservice Architectures
ConfigurationTooling Discovery Routing Observability
Development: Languages and Container
Operational: Orchestration and Deployment Infrastructure
Datastores
Policy: Architectural and Security Compliance
![Page 28: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/28.jpg)
Next Generation ApplicationsFill in the gaps, rapidly evolving ecosystem choices
ArchaiusLaunchDarkly
Configuration
Docker CaaSSpinnaker
Tooling
EtcdEurekaConsul
Discovery
ComposeCalicoWeave
Routing
ZipkinPrometheus
Hystrix
Observability
Development: Assembled from components stored in public Docker Hub and in private hubs
Operational: Mesos, Kubernetes, Swarm, ECS etc. across public and private clouds
Datastores: Distributed Ephemeral, Orchestrated or DBaaS
Policy: Architectural and security compliance, Cloud Foundry/Apcera for low trust teams
![Page 29: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/29.jpg)
What could go wrong?
![Page 30: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/30.jpg)
@adrianco
Timeouts and Retries
Edge Service
Good Service
Good Service
Bad config: Every service defaults to 2 second timeout, two retries
![Page 31: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/31.jpg)
@adrianco
Timeouts and Retries
Edge Service
Good Service
Good Service
Bad config: Every service defaults to 2 second timeout, two retries
![Page 32: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/32.jpg)
@adrianco
Timeouts and Retries
Edge Service
Good Service
Good Service
Bad config: Every service defaults to 2 second timeout, two retries
Edge Service not responding
Overloaded service not responding
Failed Service
If anything breaks, everything upstream stops responding
![Page 33: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/33.jpg)
@adrianco
Timeouts and Retries
Edge Service
Good Service
Good Service
Bad config: Every service defaults to 2 second timeout, two retries
Edge Service not responding
Overloaded service not responding
Failed Service
If anything breaks, everything upstream stops responding
Retries add unproductive work
![Page 34: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/34.jpg)
@adrianco
Timeouts and Retries
Edge Service
Good Service
Budgeted timeout, one retry
Failed Service
![Page 35: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/35.jpg)
@adrianco
Timeouts and Retries
Edge Service
Good Service
Budgeted timeout, one retry
Failed Service
3s1s1s
Fast failresponseafter 2s
Upstream timeout must always be longer than total downstream timeout * retries delay
No unproductive work while fast failing
![Page 36: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/36.jpg)
@adrianco
Timeouts and Retries
Edge Service
Good Service
Budgeted timeout, failover retry
Failed Service
For replicated services with multiple instances never retry against a failed instance
No extra retries or unproductive work
Good Service
![Page 37: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/37.jpg)
@adrianco
Timeouts and Retries
Edge Service
Good Service
Budgeted timeout, failover retry
Failed Service3s 1s
For replicated services with multiple instances never retry against a failed instance
No extra retries or unproductive work
Good Service
Successfulresponsedelayed 1s
![Page 38: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/38.jpg)
Cloud Native Monitoring and Microservices
![Page 39: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/39.jpg)
Interesting architectures have a lot of microservices! Flow visualization is
a big challenge.
See http://www.slideshare.net/LappleApple/gilt-from-monolith-ruby-app-to-micro-service-scala-service-architecture
![Page 40: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/40.jpg)
Simulated MicroservicesModel and visualize microservices Simulate interesting architectures Generate large scale configurations Eventually stress test real tools
Code: github.com/adrianco/spigo Simulate Protocol Interactions in Go Visualize with D3 See for yourself: http://simianviz.surge.sh Follow @simianviz for updates
ELB Load Balancer
ZuulAPI Proxy
KaryonBusiness Logic
StaashData Access Layer
PriamCassandra Datastore
ThreeAvailabilityZones
DenominatorDNS Endpoint
![Page 41: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/41.jpg)
Definition of an architecture
{ "arch": "lamp", "description":"Simple LAMP stack", "version": "arch-0.0", "victim": "webserver", "services": [ { "name": "rds-mysql", "package": "store", "count": 2, "regions": 1, "dependencies": [] }, { "name": "memcache", "package": "store", "count": 1, "regions": 1, "dependencies": [] }, { "name": "webserver", "package": "monolith", "count": 18, "regions": 1, "dependencies": ["memcache", "rds-mysql"] }, { "name": "webserver-elb", "package": "elb", "count": 0, "regions": 1, "dependencies": ["webserver"] }, { "name": "www", "package": "denominator", "count": 0, "regions": 0, "dependencies": ["webserver-elb"] } ] }
Header includeschaos monkey victim
New tier name
Tier package
0 = non Regional
Node count
List of tier dependencies
See for yourself: http://simianviz.surge.sh/lamp
![Page 42: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/42.jpg)
Running Spigo$ ./spigo -a lamp -j -d 2 2016/01/26 23:04:05 Loading architecture from json_arch/lamp_arch.json 2016/01/26 23:04:05 lamp.edda: starting 2016/01/26 23:04:05 Architecture: lamp Simple LAMP stack 2016/01/26 23:04:05 architecture: scaling to 100% 2016/01/26 23:04:05 lamp.us-east-1.zoneB.eureka01....eureka.eureka: starting 2016/01/26 23:04:05 lamp.us-east-1.zoneA.eureka00....eureka.eureka: starting 2016/01/26 23:04:05 lamp.us-east-1.zoneC.eureka02....eureka.eureka: starting 2016/01/26 23:04:05 Starting: {rds-mysql store 1 2 []} 2016/01/26 23:04:05 Starting: {memcache store 1 1 []} 2016/01/26 23:04:05 Starting: {webserver monolith 1 18 [memcache rds-mysql]} 2016/01/26 23:04:05 Starting: {webserver-elb elb 1 0 [webserver]} 2016/01/26 23:04:05 Starting: {www denominator 0 0 [webserver-elb]} 2016/01/26 23:04:05 lamp.*.*.www00....www.denominator activity rate 10ms 2016/01/26 23:04:06 chaosmonkey delete: lamp.us-east-1.zoneC.webserver02....webserver.monolith 2016/01/26 23:04:07 asgard: Shutdown 2016/01/26 23:04:07 lamp.us-east-1.zoneB.eureka01....eureka.eureka: closing 2016/01/26 23:04:07 lamp.us-east-1.zoneA.eureka00....eureka.eureka: closing 2016/01/26 23:04:07 lamp.us-east-1.zoneC.eureka02....eureka.eureka: closing 2016/01/26 23:04:07 spigo: complete 2016/01/26 23:04:07 lamp.edda: closing
-a architecture lamp-j graph json/lamp.json-d run for 2 seconds
![Page 43: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/43.jpg)
Open Zipkin
A common format for trace annotations A Java tool for visualizing traces Standardization effort to fold in other formats Driven by Adrian Cole (currently at Pivotal) Extended to load Spigo generated trace files
![Page 44: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/44.jpg)
Trace for one Spigo Flow
![Page 45: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/45.jpg)
Migrating to MicroservicesSee for yourself: http://simianviz.surge.sh/migration
Endpoint
ELB
PHPMySQL
MySQL
Next step Controls node placement distance
Select models
![Page 46: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/46.jpg)
Migrating to MicroservicesSee for yourself: http://simianviz.surge.sh/migration
Step 1 - Add Memcache Step 2 - Add Web Proxy Service
![Page 47: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/47.jpg)
Migrating to MicroservicesSee for yourself: http://simianviz.surge.sh/migration
Step 3 - Add Data Access Layer Step 4 - Add Microservices
Data Access
node.js
memcache per zone
![Page 48: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/48.jpg)
Migrating to MicroservicesSee for yourself: http://simianviz.surge.sh/migration
Step 5 - Add Cassandra Step 6 - Remove MySQL
12 node cross zone Cassandra cluster
MySQL
![Page 49: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/49.jpg)
Migrating to MicroservicesSee for yourself: http://simianviz.surge.sh/migration
Step 7 - Add Second Region Step 8 - Connect Cassandra RegionsEndpoint with location routed DNS
![Page 50: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/50.jpg)
Migrating to MicroservicesSee for yourself: http://simianviz.surge.sh/migration
Step 9 - Add Third Region
Endpoint with location routed DNS
![Page 51: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/51.jpg)
@adrianco
Simple Architecture Principles
SymmetryInvariants
Stable assertionsNo special cases
![Page 52: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/52.jpg)
What’s Next?
![Page 53: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/53.jpg)
Trends to watch for 2016:
Serverless Architectures - AWS Lambda
Teraservices - using terabytes of memory
![Page 54: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/54.jpg)
Serverless Architectures
AWS Lambda getting some early wins
Google Cloud Functions, Azure Functions alpha launched
IBM OpenWhisk - open sourced
Startup activity: iron.io , serverless.com, apex.run toolkit
![Page 55: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/55.jpg)
Teraservices
![Page 56: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/56.jpg)
Terabyte Memory Directions
Engulf dataset in memory for analytics
Balanced config for memory intensive workloads
Replace high end systems at commodity cost point
Explore non-volatile memory implications
![Page 57: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/57.jpg)
Terabyte Memory Options
Flash based Diablo 64/128/256GB DDR4 DIMM Shipping now as volatile memory, future non-volatile
Intel 3D XPoint - new non-volatile technology on the way
Announced but not shipped for 2016 AWS X1 Instance Type - over 2TB RAM Easy availability should drive innovation
![Page 58: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/58.jpg)
Learn More…
![Page 59: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/59.jpg)
@adrianco
“We see the world as increasingly more complex and chaotic because we use inadequate concepts to explain it. When we understand something, we no longer see it as chaotic or complex.”
Jamshid Gharajedaghi - 2011Systems Thinking: Managing Chaos and Complexity: A Platform for Designing Business Architecture
![Page 60: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/60.jpg)
Q&AAdrian Cockcroft @adrianco
http://slideshare.com/adriancockcroftTechnology Fellow - Battery Ventures
See www.battery.com for a list of portfolio investments
![Page 61: Evolution of Microservices - Craft Conference](https://reader031.vdocument.in/reader031/viewer/2022030317/586e88571a28aba0038b7473/html5/thumbnails/61.jpg)
Security
Visit http://www.battery.com/our-companies/ for a full list of all portfolio companies in which all Battery Funds have invested.
Palo Alto Networks
Enterprise IT
Operations & Management
Big DataCompute
Networking
Storage