Download - Building a microservice ecosystem
Building a microservice ecosystem
Daniel Bryant@danielbryantuk
OpencRedo
Today• What is a ‘microservice ecosystem’?
• Local dev of services becomes challenging
• Think about your build pipeline(s)
• Testing requires a paradigm shift
• Push to production as early as possible
24/11/2016 @danielbryantuk
@danielbryantuk• Chief Scientist at OpenCredo, CTO at SpectoLabs
ü Transforming organisations through technology and teams
ü Agile, Lean, Architecture, CI/CD, DevOps
ü Microservices, cloud, Containers, Java, Go, Docker, Kubernetes
• London Java Community Associate
• Adopt OpenJDK and JSR
• InfoQ Editor, DZone MVB, VOXXED, O'Reilly
24/11/2016 @danielbryantuk
My Biggest Claim to Fame in Microservices?
24/11/2016 @danielbryantuk
Seriously Though...…
24/11/2016 @danielbryantuk
skillsmatter.com/skillscasts/7004-our-journey-to-world-gifting-domination-how-notonthehighstreet-com-embraced-docker
So, What is a microservice?
“Loosely coupled service oriented architecture with bounded contexts”
Adrian Cockcroft
“Applications that fit in your head”James Lewis
24/11/2016 @danielbryantuk
Why? The move towards Cloud-native
• Hypothesis-driven business (and development)
• Microservice architecture (and 'Micro' teams)
• Devops mindset/culture (shared, learning/feedback & Mechanical sympathy)
• Continuous delivery
• Automated, self-service platform
Hat tip @caseywest
24/11/2016 @danielbryantuk
Exploring the Ecosystem
24/11/2016 @danielbryantuk
Adrian Cockcroft’s Thoughts
24/11/2016 @danielbryantuk
www.slideshare.net/adriancockcroft/microxchg-microservices
Alexis Richardson’s Thoughts
24/11/2016 @danielbryantuk
gotocon.com/goto-london-2015/#!#schedulePopupExtras-7011
wikibon.com/wp-content/uploads/container_implementations.png
Technology Choices
24/11/2016 @danielbryantuk
24/11/2016
What do I mean by ‘ecosystem’?• Build– Local development, pipelines and integration
• Test– From local integration to End-to-end
• Deploy
• Operate
• Observe– Monitoring/logging/alerting
24/11/2016 @danielbryantuk
Don’t forget about (RE)ArchitecturE Complexity
24/11/2016 @danielbryantuk
24/11/2016 @danielbryantuk
www.infoq.com/news/2015/04/raffi-krikorian-rearchitecting
What do I mean by ‘ecosystem’?• Build– Local development, pipelines and integration
• Test– From local integration to End-to-end
• Deploy
• Operate
• Observe– Monitoring/logging/alerting
24/11/2016 @danielbryantuk
Build
24/11/2016 @danielbryantuk
Developing Locally: The Basics
• GitHub’s Boxen (Puppet)
• Pivotal’s Sprout (Chef)
• Mac-dev-playbook (Ansible)
• Hashicorp Vagrant
• Docker Compose (Docker machine/native)
24/11/2016 @danielbryantuk
Developing Locally
• The naive approach
– Replication of env per service
– …and dependencies and data stores and...
– Soon gets crazy
• Local profiles + mocking/stubbing
– Spring profiles + Mockito etc
24/11/2016 @danielbryantuk
Developing Locally
• Service virtualisation– Hoverfly, Mountebank, Wiremock (Saboteur)
• ‘Production-in-a-box’ (IFTTT)– Docker Compose, Vagrant, cf_nise_installer
• Environment leasing– Create your own env (e.g. Hailo)
24/11/2016 @danielbryantuk
Developing Locally
www.opencredo.com/2015/09/20/working-locally-with-microservices/24/11/2016 @danielbryantuk
Create a Pipeline
24/11/2016 @danielbryantuk
Key lesson learned (the Hard Way)
• Push through to production as early as possible!
• Detect and fix issues
– Technology
– Process
– People
24/11/2016 @danielbryantuk
Test
24/11/2016 @danielbryantuk
Always Remember...
24/11/2016 @danielbryantukblog.bbv.ch/2012/06/13/acceptance-test-driven-development/
Microservice Testing Basics
• Toby Clemson’s article
martinfowler.com/articles/microservice-testing
24/11/2016 @danielbryantuk
Microservice Testing Basics
• My article
https://www.specto.io/blog/recipe-for-designing-building-
testing-microservices.html
24/11/2016 @danielbryantuk
(multiple) Pipelines• Big-bang release
– Beware of rubber stamping (distributed monolith)
– ‘semver’ if you must (semver.org) e.g. 1.2.1
• Gated release– Critical path testing in Stage
• Single service Continuous delivery
– Consumer-based contracts
– Backwards compatibility
24/11/2016 @danielbryantuk
Service-level Integration Testing
• Contracts
– Pact-JVM github.com/DiUS/pact-jvm
– PACT broker
github.com/bethesque/pact_broker
– Examples:
github.com/mstine/microservices-pact
24/11/2016 @danielbryantuk
My Opinions• BDD critical paths throughout application
– Including API journey
• Contract tests (failure is a conversation)
• BDD services API (e.g. Serenity BDD)
• Component test and unit test (as normal)– Maven surefire/failsafe
24/11/2016 @danielbryantuk
Final Words on Testing• Don’t forget the ‘ilities”
• Security / reliability– ZAP (from the OWASP team)
– github.com/continuumsecurity/bdd-security
• Performance / scalability
– Jmeter (Jenkins Performance plugin)
– Gatling
– flood.io
24/11/2016 @danielbryantuk
Deploy
24/11/2016 @danielbryantuk
Separate Deploy and Release
• Feature flags– Difficult at scale (and distribution)
– Enable at ingress
• Incremental (phased) rollout
• Canary vs blue/green
• Avoid datastore migrations (if possible)
24/11/2016 @danielbryantuk
Centralise Configuration
• Consul & consul-template
• Etcd/ZK & confd
• Netflix Archaius
• Spring Cloud config
• Watch for application rollback!
24/11/2016 @danielbryantuk
Operate
24/11/2016 @danielbryantuk
Building Blocks
• HashiCorp Terraform
– VMWare? cloud-init and vcloud-tools
– “Boot my secure government cloud”
• “CAPS”– Chef, Ansible, Puppet, SaltStack
– Automated sysadmin
• Docker / Ami / RPM / JAR
– Machine vs Os/lang artifact
• Standardise on an OS
– Amazon Linux vs mainstream distros
24/11/2016 @danielbryantuk
DevOps and Programmable Infrastructure
• Devops involves programming…
• Introduce SOLID principles
• Good CI/CD principles
– Gitflow etc
– Testing
24/11/2016 @danielbryantuk
Service Discovery
• External
– HAProxy / nginx / ELB etc
• Client-side
– Netflix Ribbon (with Prana)
– ‘Baker Street’ (extending SmartStack)
– srv-router
• Kubernetes and CF are good to go
24/11/2016 @danielbryantuk
External
24/11/2016 @danielbryantuk
Client-side
24/11/2016 @danielbryantuk
Observe
24/11/2016 @danielbryantuk
When bad things happen, people are always involved
24/11/2016 @danielbryantuk|@oakinger
Monitoring and People
24/11/2016 @danielbryantukwww.infoq.com/news/2015/06/too-big-to-fail
Start with the Basics
• Health checks– Coda Hale (DropWizard) Metrics
– Spring Boot actuator
• KPIs for apps (and business)– Assertions / invariants
– Throughput
– Queue length
24/11/2016 @danielbryantuk
Logging
• What every engineer should know
• 10 Tips for Proper Application Logging
• ElasticSearch-Logstash-Kibana (ELK)
– Buffer/proxy log sending or…
– Mount directory into container
24/11/2016 @danielbryantuk
Monitoring• Push
– Spring Boot actuator e.g. InfluxDbExporter
• Pull – E.g. Telegraf (shout to Tareq Abedrabbo)
• InfluxDB vs prometheus vs graphite vsopentsdb
• Information radiators– Aggregate vs individual
24/11/2016 @danielbryantuk
Aggregation: Sick Cattle, Not Sick Pets
Opentracing & OpenZipkin
24/11/2016 @danielbryantuk
Problems?
• Rob Ewaschuk’s “Philosophy on Alerting”
• Brendan Gregg’s USE method
– “check utilization, saturation, and errors.”
• “DevOps Troubleshooting”– Kyle Rankin
24/11/2016 @danielbryantuk
Let’s wrap this up...
24/11/2016 @danielbryantuk
Summary• development of 3+ services is challenging
• Think about your build pipeline(s)
• Testing requires a paradigm shift
• Operations requires much more automation
• Push through to prod as early as possible
24/11/2016 @danielbryantuk
Bedtime reading
24/11/2016 @danielbryantuk
THANKS...
@danielbryantuk
http://muservicesweekly.com/
(Credit to Tareq Abedrabbo for inspiration/guidance)
24/11/2016 @danielbryantuk
Logging vs monitoring
• Logging
– Primarily used post-incident
– Machine readable
– Hard to do right
– Noise/signal
• Monitoring
– Useful in Real time during incident
– Trend analysis
– Easier to do
24/11/2016 @danielbryantuk