continuous integration and delivery at shapeways (matt boyle)
TRANSCRIPT
Continuous Integration and Delivery at Shapeways
Matt Boyle NordicAPIs Platform SummitStockholm, Sweden25 October 2016
What is Shapeways?The worlds largest 3d printing service and marketplace
Marketplace, service hub, and community hosted at Shapeways.com
REST API provided at api.shapeways.com
Why CI/CD?
Idea
Exploration
Definition
Execution
Release
Evaluation
Consider the SDLC
…except this one
All stages represent active, valuable work by software team members.
Releasing Software Starts Easy
Everything Working?
Releasing Software Gets Harder
Git pull Build FrontEnd?
Manage Symlinks?
Bounce Memcache
?
Restart Apache?
Releasing Software Gets REALLY hardBuild Artifact
Run tests
Deploy to multiple
datacenters
Intra-DC deployment to
servers
Manage artifacts on
serverRotate through load balancer
Generate documentation
Restart services
Detect regressions
Monitor for Breakages Deployments like this take time and
resources
If you don’t automate, you’ll spend as much time running deployments as you do writing code
The Shapeways CI Story
In The Beginning…
60 minute deployments No API documentation Frustrating Manual Testing
Baby Steps
Continuous Integration with Jenkins
Integration + Functional Testing with
Selenium
New tests required to ship features
Pro’ing up - JenkinsActive-Active Jenkins instances – one per datacenter
Use Jenkins API for remote execution of deployments and test jobs
Flow-based deployment using Jenkins Pipeline DDL
Pro’ing up - Coyote FrameworkIn-house FOSS testing framework for integration and functional testing
Thoughtfully developed testing infrastructure to ensure consistent, reliable, and maintainable tests
Two-level API testing: contract enforcement and functional flow
Pro’ing up - ACMEIn-house developed test management application
Schedule test runs on-demand for developers
Test analysis and intelligence
Pro’ing up - ChatOps via HubotGitHub-developed extensible chat bot written in CoffeeScript
Manage deployments, development/QA infrastructure, permission management
Acts as interface between our different tools: jenkins, coyote, acme, slack
Looking Ahead – Staged RolloutsPresent deployment is fast but requires us to be all-in on new code
Ability to use infrastructure for new feature A/B testing rather than user groups
Increased deployment time, but faster recovery should something go wrong
Looking Ahead - Swagger for all APIs, int+extEasier documentation generation
Client generation/update for multiple languages
Source of truth for service-based applications – contract enforcement, test generation
Looking Ahead - Improved Testing ToolingGenerate testing profiles based on changes made in codebase
Code coverage for functional testing via URL mapping
Performance and capacity measurement based on previous test runs and environments