using docker in ci process
TRANSCRIPT
![Page 1: Using Docker in CI process](https://reader031.vdocument.in/reader031/viewer/2022021814/58e6816c1a28ab6a5b8b4f07/html5/thumbnails/1.jpg)
Introduction The pipeline The environment The images The problems Finish
Using Docker in CI processApplied Docker
G. Godlewski
March 16, 2016
![Page 2: Using Docker in CI process](https://reader031.vdocument.in/reader031/viewer/2022021814/58e6816c1a28ab6a5b8b4f07/html5/thumbnails/2.jpg)
Introduction The pipeline The environment The images The problems Finish
Author
About me
Programming, karate, music and cheese making.
Currently working for SMT Software Services
![Page 3: Using Docker in CI process](https://reader031.vdocument.in/reader031/viewer/2022021814/58e6816c1a28ab6a5b8b4f07/html5/thumbnails/3.jpg)
Introduction The pipeline The environment The images The problems Finish
The background
Long time ago, in a galaxy far, far away...
![Page 4: Using Docker in CI process](https://reader031.vdocument.in/reader031/viewer/2022021814/58e6816c1a28ab6a5b8b4f07/html5/thumbnails/4.jpg)
Introduction The pipeline The environment The images The problems Finish
The background
![Page 5: Using Docker in CI process](https://reader031.vdocument.in/reader031/viewer/2022021814/58e6816c1a28ab6a5b8b4f07/html5/thumbnails/5.jpg)
Introduction The pipeline The environment The images The problems Finish
The background
Scope of automation
Unit tests
Functional tests
Integration tests
Quality checks (nightly build, code coverage, phpmd, phpcs,jshint, jslint, ...)
Deployments
One should constantly search for spots where time could be saved
![Page 6: Using Docker in CI process](https://reader031.vdocument.in/reader031/viewer/2022021814/58e6816c1a28ab6a5b8b4f07/html5/thumbnails/6.jpg)
Introduction The pipeline The environment The images The problems Finish
Definitions
Continuous Integration
Martin Fowler:
Continuous Integration is a software developmentpractice where members of a team integrate theirwork frequently, usually each person integrates at leastdaily - leading to multiple integrations per day. Eachintegration is verified by an automated build (includingtest) to detect integration errors as quickly as possible.
![Page 7: Using Docker in CI process](https://reader031.vdocument.in/reader031/viewer/2022021814/58e6816c1a28ab6a5b8b4f07/html5/thumbnails/7.jpg)
Introduction The pipeline The environment The images The problems Finish
Definitions
Continuous Delivery
Martin Fowler:
A common question we hear is “what is the differencebetween Continuous Delivery and ContinuousDeployment?” Both terms are similar and were coinedaround the same time. I see the difference as a businessdecision about frequency of deployment into production.Continuous Delivery is about keeping your applicationin a state where it is always able to deploy intoproduction. Continuous Deployment is actuallydeploying every change into production, every day ormore frequently.
![Page 8: Using Docker in CI process](https://reader031.vdocument.in/reader031/viewer/2022021814/58e6816c1a28ab6a5b8b4f07/html5/thumbnails/8.jpg)
Introduction The pipeline The environment The images The problems Finish
Product
The point of view
![Page 9: Using Docker in CI process](https://reader031.vdocument.in/reader031/viewer/2022021814/58e6816c1a28ab6a5b8b4f07/html5/thumbnails/9.jpg)
Introduction The pipeline The environment The images The problems Finish
Process
The process
Building - parts put together, diagnostics and verification,removal of unnecessary parts
Packaging - a ready product is being put into a package(container)
Delivery - the whole package is shipped to the client in aclearly defined manner
Each of the stages has a clear I/O definition
![Page 10: Using Docker in CI process](https://reader031.vdocument.in/reader031/viewer/2022021814/58e6816c1a28ab6a5b8b4f07/html5/thumbnails/10.jpg)
Introduction The pipeline The environment The images The problems Finish
Process
Building up a complete solution
In case of a product composed from several components (eg.complex system) we could use the analogy of buying kitchenfurniture.
All packages contain components that fulfill a defined set ofcriteria
We know how the components should be connected
We know how the components should be placed withinclient’s infrastructure (kitchen)
Don’t dismiss the power of analogy!
![Page 11: Using Docker in CI process](https://reader031.vdocument.in/reader031/viewer/2022021814/58e6816c1a28ab6a5b8b4f07/html5/thumbnails/11.jpg)
Introduction The pipeline The environment The images The problems Finish
Warming up!
What will we use Docker for
Performing CI builds within isolated containers (docker indocker, sic!)
Providing the package for the product in which will it befurther distributed
Running our applications
![Page 12: Using Docker in CI process](https://reader031.vdocument.in/reader031/viewer/2022021814/58e6816c1a28ab6a5b8b4f07/html5/thumbnails/12.jpg)
Introduction The pipeline The environment The images The problems Finish
Warming up!
![Page 13: Using Docker in CI process](https://reader031.vdocument.in/reader031/viewer/2022021814/58e6816c1a28ab6a5b8b4f07/html5/thumbnails/13.jpg)
Introduction The pipeline The environment The images The problems Finish
Warming up!
Getting ready
1 Ensure the right kernel version (starting from 3.10!)
2 Ensure you have all required kernel modules (aufs,devicemapper etc - depends on the distribution)
3 Latest docker-engine installed
4 Latest docker-compose installed
![Page 14: Using Docker in CI process](https://reader031.vdocument.in/reader031/viewer/2022021814/58e6816c1a28ab6a5b8b4f07/html5/thumbnails/14.jpg)
Introduction The pipeline The environment The images The problems Finish
Additional tools
Ansible
What for?
Describe how the application should be delivered (deploymentto server)
Configuration management
Why?
Great documentation
Huge amount of modules - I didn’t have to build my own yet!
![Page 15: Using Docker in CI process](https://reader031.vdocument.in/reader031/viewer/2022021814/58e6816c1a28ab6a5b8b4f07/html5/thumbnails/15.jpg)
Introduction The pipeline The environment The images The problems Finish
Own Docker Hub
Docker Hub
hub:
restart: always
image: registry:2
ports:
- "5000:5000"
environment:
TERM: linux
REGISTRY_HTTP_TLS_CERTIFICATE: :)
REGISTRY_HTTP_TLS_KEY: :)
REGISTRY_AUTH: htpasswd
REGISTRY_AUTH_HTPASSWD_PATH: /auth/htpasswd
REGISTRY_AUTH_HTPASSWD_REALM: Registry Realm
volumes:
- "/home/docker-distro/registry/data:/var/lib/registry"
- "/etc/ssl/certs:/certs"
- "/etc/ssl/private:/keys"
- "/home/docker-distro/registry/auth:/auth"
Hint
Keep security in mind!
![Page 16: Using Docker in CI process](https://reader031.vdocument.in/reader031/viewer/2022021814/58e6816c1a28ab6a5b8b4f07/html5/thumbnails/16.jpg)
Introduction The pipeline The environment The images The problems Finish
The CI Server
TeamCity
Free version provides:
3 build agents
20 build configurations
Broad configuration capabilities (build parameters,configuration templates, defining dependencies etc)
![Page 17: Using Docker in CI process](https://reader031.vdocument.in/reader031/viewer/2022021814/58e6816c1a28ab6a5b8b4f07/html5/thumbnails/17.jpg)
Introduction The pipeline The environment The images The problems Finish
Understanding CI
Build Server and Build Agents
![Page 18: Using Docker in CI process](https://reader031.vdocument.in/reader031/viewer/2022021814/58e6816c1a28ab6a5b8b4f07/html5/thumbnails/18.jpg)
Introduction The pipeline The environment The images The problems Finish
Customs
Special forces
![Page 19: Using Docker in CI process](https://reader031.vdocument.in/reader031/viewer/2022021814/58e6816c1a28ab6a5b8b4f07/html5/thumbnails/19.jpg)
Introduction The pipeline The environment The images The problems Finish
Customs
Custom Agent
FROM sjoerdmulder/teamcity-agent:latest
MAINTAINER Grzegorz Godlewski <[email protected]>
RUN apt-get -y install software-properties-common
RUN apt-add-repository ppa:ansible/ansible
RUN apt-get update
RUN apt-get -y install ansible
COPY ./keys/id_rsa /home/teamcity/.ssh/id_rsa
RUN chown -R teamcity:teamcity /home/teamcity/.ssh
ADD docker-entrypoint.sh /docker-entrypoint.sh
RUN chmod a+x docker-entrypoint.sh
![Page 20: Using Docker in CI process](https://reader031.vdocument.in/reader031/viewer/2022021814/58e6816c1a28ab6a5b8b4f07/html5/thumbnails/20.jpg)
Introduction The pipeline The environment The images The problems Finish
Customs
Extending the images
![Page 21: Using Docker in CI process](https://reader031.vdocument.in/reader031/viewer/2022021814/58e6816c1a28ab6a5b8b4f07/html5/thumbnails/21.jpg)
Introduction The pipeline The environment The images The problems Finish
Implementation
TeamCity Server
server:
image: "sjoerdmulder/teamcity:latest"
ports:
- "8111:8111"
volumes:
- "/home/teamcity/server/data:/var/lib/teamcity"
environment:
TERM: linux
![Page 22: Using Docker in CI process](https://reader031.vdocument.in/reader031/viewer/2022021814/58e6816c1a28ab6a5b8b4f07/html5/thumbnails/22.jpg)
Introduction The pipeline The environment The images The problems Finish
Implementation
TeamCity Agent
agent:
image: "my-own-hub/teamcity-agent:latest"
ports:
- "9090:9090"
volumes:
- "/var/run/docker.sock:/var/run/docker.sock" # docker in docker
- "/usr/bin/docker:/usr/bin/docker"
- "/usr/bin/docker:/usr/local/bin/docker"
- "/home/teamcity/agent/work:/opt/buildAgent/work" # work directories
- "/home/teamcity/agent/composer:/opt/composer/cache" # composer cache
- "/home/teamcity/agent/docker:/home/teamcity/.docker" # hub auth keys
environment:
TERM: linux
TEAMCITY_SERVER: "http://teamcity_server:8111" # your server
TEAMCITY_AGENT_NAME: "Alpha" # readable name
AGENT_HOME_ON_HOST: "/home/teamcity/agent" # custom ENV for runtime
![Page 23: Using Docker in CI process](https://reader031.vdocument.in/reader031/viewer/2022021814/58e6816c1a28ab6a5b8b4f07/html5/thumbnails/23.jpg)
Introduction The pipeline The environment The images The problems Finish
Implementation
The flow
1 Run build in dev image
2 Package using dist image
3 Distribute using Ansible and docker-compose
![Page 24: Using Docker in CI process](https://reader031.vdocument.in/reader031/viewer/2022021814/58e6816c1a28ab6a5b8b4f07/html5/thumbnails/24.jpg)
Introduction The pipeline The environment The images The problems Finish
It’s never that easy
Not so easy...
Cleaning up
CI process performance
Proper kernel modules
Dynamically linked modules
Race conditions
![Page 25: Using Docker in CI process](https://reader031.vdocument.in/reader031/viewer/2022021814/58e6816c1a28ab6a5b8b4f07/html5/thumbnails/25.jpg)
Introduction The pipeline The environment The images The problems Finish
It’s never that easy
https://github.com/docker/docker/issues/4036
![Page 26: Using Docker in CI process](https://reader031.vdocument.in/reader031/viewer/2022021814/58e6816c1a28ab6a5b8b4f07/html5/thumbnails/26.jpg)
Introduction The pipeline The environment The images The problems Finish
It’s never that easy
https://github.com/docker/docker/issues/4036
If you’re using the devicemapper diver, make sure that Udev SyncSupported is set to true.
![Page 27: Using Docker in CI process](https://reader031.vdocument.in/reader031/viewer/2022021814/58e6816c1a28ab6a5b8b4f07/html5/thumbnails/27.jpg)
Introduction The pipeline The environment The images The problems Finish
Questions?
![Page 28: Using Docker in CI process](https://reader031.vdocument.in/reader031/viewer/2022021814/58e6816c1a28ab6a5b8b4f07/html5/thumbnails/28.jpg)
Introduction The pipeline The environment The images The problems Finish
Thank you!
http://linkedin.com/in/[email protected]
@GGodlewski