docker to the rescue of an ops team
TRANSCRIPT
![Page 2: Docker to the Rescue of an Ops Team](https://reader030.vdocument.in/reader030/viewer/2022020314/5a64776e7f8b9afc4d8b478d/html5/thumbnails/2.jpg)
AgendaThis talk is about:
● Monitoring (a bit)● Private Registry● CI/CD● Security● Docker experience
![Page 3: Docker to the Rescue of an Ops Team](https://reader030.vdocument.in/reader030/viewer/2022020314/5a64776e7f8b9afc4d8b478d/html5/thumbnails/3.jpg)
Once upon a time An ops team starts a new project
![Page 4: Docker to the Rescue of an Ops Team](https://reader030.vdocument.in/reader030/viewer/2022020314/5a64776e7f8b9afc4d8b478d/html5/thumbnails/4.jpg)
Rebuild everything !June 2014A monitoring system from scratch“Microservices” orientedReplaceable parts
![Page 5: Docker to the Rescue of an Ops Team](https://reader030.vdocument.in/reader030/viewer/2022020314/5a64776e7f8b9afc4d8b478d/html5/thumbnails/5.jpg)
Some rules first !SimpleEfficientExtendable Python based
![Page 6: Docker to the Rescue of an Ops Team](https://reader030.vdocument.in/reader030/viewer/2022020314/5a64776e7f8b9afc4d8b478d/html5/thumbnails/6.jpg)
Components Collectd
Collectd proxy
Graphite
Grafana
Cabot (alerting)
![Page 7: Docker to the Rescue of an Ops Team](https://reader030.vdocument.in/reader030/viewer/2022020314/5a64776e7f8b9afc4d8b478d/html5/thumbnails/7.jpg)
Test your might !Python 2.6 along with python 2.7Different version of “some” librariesWhisper backend (I/O storm)Upstream repositories issues
![Page 8: Docker to the Rescue of an Ops Team](https://reader030.vdocument.in/reader030/viewer/2022020314/5a64776e7f8b9afc4d8b478d/html5/thumbnails/8.jpg)
Docker to the rescueSave our project
![Page 9: Docker to the Rescue of an Ops Team](https://reader030.vdocument.in/reader030/viewer/2022020314/5a64776e7f8b9afc4d8b478d/html5/thumbnails/9.jpg)
Docker ? way too soon !Barely no skillsUsed only to do some testingPretty serious concerns
![Page 10: Docker to the Rescue of an Ops Team](https://reader030.vdocument.in/reader030/viewer/2022020314/5a64776e7f8b9afc4d8b478d/html5/thumbnails/10.jpg)
Ok let’s gamble !Grow our docker fuWrite some DockerfilesBuild images locallySpawn PoC platform
![Page 11: Docker to the Rescue of an Ops Team](https://reader030.vdocument.in/reader030/viewer/2022020314/5a64776e7f8b9afc4d8b478d/html5/thumbnails/11.jpg)
…..
RUN echo "deb http://mirror.debian.ikoula.com/debian wheezy-backports main" >> /etc/apt/sources.listRUN apt-get -qq updateRUN apt-get -qqy dist-upgrade
RUN apt-get -qqy --force-yes install vim python-cairo gunicorn supervisor (...)RUN pip install whitenoise txamqp whisper==0.9.13 carbonateRUN pip install --install-option="--prefix=/var/lib/graphite" --install-option="--install-lib=/var/lib/graphite/lib" carbon==0.9.13RUN pip install --install-option="--prefix=/var/lib/graphite" --install-option="--install-lib=/var/lib/graphite/webapp" graphite-web==0.9.13
ADD conf/nginx.conf /etc/nginx/nginx.confADD conf/supervisord.conf /etc/supervisor/conf.d/grafana.conf
ADD initial_data.json /var/lib/graphite/webapp/graphite/initial_data.jsonADD conf/local_settings.py /var/lib/graphite/webapp/graphite/local_settings.pyADD conf/carbon.conf /var/lib/graphite/conf/carbon.confADD conf/storage-schemas.conf /var/lib/graphite/conf/storage-schemas.confRUN mkdir -p /var/lib/graphite/storage/whisperRUN touch /var/lib/graphite/storage/graphite.db /var/lib/graphite/storage/indexRUN chmod 0775 /var/lib/graphite/storage /var/lib/graphite/storage/whisperRUN python /var/lib/graphite/webapp/graphite/manage.py syncdb --noinput --pythonpath=/var/lib/graphite/webapp/graphite --settings=settingsRUN chmod 0664 /var/lib/graphite/storage/graphite.dbRUN chown -R www-data /var/lib/graphite/storage
…..
![Page 12: Docker to the Rescue of an Ops Team](https://reader030.vdocument.in/reader030/viewer/2022020314/5a64776e7f8b9afc4d8b478d/html5/thumbnails/12.jpg)
WHAT ???
![Page 13: Docker to the Rescue of an Ops Team](https://reader030.vdocument.in/reader030/viewer/2022020314/5a64776e7f8b9afc4d8b478d/html5/thumbnails/13.jpg)
Container = OS …. Wait !
Too many layers ( 121+ layer issue)
Build time …. (20 to 30 minutes at best)
Huge Images (800+ Mo)
Unnecessary tools and libs
![Page 14: Docker to the Rescue of an Ops Team](https://reader030.vdocument.in/reader030/viewer/2022020314/5a64776e7f8b9afc4d8b478d/html5/thumbnails/14.jpg)
Bye Bye !!!!
![Page 15: Docker to the Rescue of an Ops Team](https://reader030.vdocument.in/reader030/viewer/2022020314/5a64776e7f8b9afc4d8b478d/html5/thumbnails/15.jpg)
We can do better !Apply best Practices (@abbyfuller)
Implement simple CI/CD
Dockerfile Linting
Build a private registry
Deal with security concerns
![Page 16: Docker to the Rescue of an Ops Team](https://reader030.vdocument.in/reader030/viewer/2022020314/5a64776e7f8b9afc4d8b478d/html5/thumbnails/16.jpg)
…
RUN echo "APT::Install-Recommends false;" >> /etc/apt/apt.conf.d/00recommends \ && echo "APT::Install-Suggests false;" >> /etc/apt/apt.conf.d/00recommends \ && echo "APT::AutoRemove::RecommendsImportant false;" >> /etc/apt/apt.conf.d/00recommends \ && echo "APT::AutoRemove::SuggestsImportant false;" >> /etc/apt/apt.conf.d/00recommends
ENV DEBIAN_FRONTEND noninteractiveENV GRAPHITE_VERS 0.9.13
RUN apt-get -qqy update \ && apt-get -qqy install python-cairo gunicorn git python2.7-dev wget ca-certificates python-flup expect sqlite3 libcairo2 libcairo2-dev pkg-config nodejs sqlite3 memcached python-ldap make gcc libffi-dev
RUN wget https://bootstrap.pypa.io/get-pip.py \ && python get-pip.py \ && pip install --no-cache-dir --upgrade setuptools \ && pip install --no-cache-dir django django-admin-tools \ && pip install --no-cache-dir whitenoise txamqp whisper==${GRAPHITE_VERS} carbonate \
&& pip install --no-cache-dir --install-option="--prefix=/var/lib/graphite" --install-option="--install-lib=/var/lib/graphite/lib" carbon==${GRAPHITE_VERS} \
&& pip install --no-cache-dir --install-option="--prefix=/var/lib/graphite" --install-option="--install-lib=/var/lib/graphite/webapp" graphite-web==${GRAPHITE_VERS}
RUN apt-get purge gcc make python2.7-dev libcairo2-dev libffi-dev python2.7-dev pkg-config -qqy \ && apt-get clean\ && apt-get autoremove -qqy \ && rm -rf /root/.cache /var/lib/apt/lists/* /tmp/* /var/tmp/*
...
![Page 17: Docker to the Rescue of an Ops Team](https://reader030.vdocument.in/reader030/viewer/2022020314/5a64776e7f8b9afc4d8b478d/html5/thumbnails/17.jpg)
Best practices :)Few image layers ( < 20 )Small image (~ 400Mo)Lower footprint (100Mo)Faster build time (~5Min)
![Page 18: Docker to the Rescue of an Ops Team](https://reader030.vdocument.in/reader030/viewer/2022020314/5a64776e7f8b9afc4d8b478d/html5/thumbnails/18.jpg)
We did it , we did it Yeah !
![Page 19: Docker to the Rescue of an Ops Team](https://reader030.vdocument.in/reader030/viewer/2022020314/5a64776e7f8b9afc4d8b478d/html5/thumbnails/19.jpg)
CI/CD Diagram
![Page 20: Docker to the Rescue of an Ops Team](https://reader030.vdocument.in/reader030/viewer/2022020314/5a64776e7f8b9afc4d8b478d/html5/thumbnails/20.jpg)
CI/CD Recipe
build: image: registry.synolia.com/synolia/dockerunitest:latest
publish: docker: repo: synomon_datastor tag: $${BRANCH/master/latest} file: Dockerfile insecure: true when: repo: synolia/systeam-monitoring_datastor branch: [develop, master]
notify: hipchat: from: "synoci" room_id_or_name: "$$ROOM_ID" auth_token: "$$AUTH_TOKEN" notify: true when: success: false failure: true
Dockerfile Linting
Build
Push
Notify (fail only)
![Page 21: Docker to the Rescue of an Ops Team](https://reader030.vdocument.in/reader030/viewer/2022020314/5a64776e7f8b9afc4d8b478d/html5/thumbnails/21.jpg)
Docker to the rescue IIThe return of the hero moby
![Page 22: Docker to the Rescue of an Ops Team](https://reader030.vdocument.in/reader030/viewer/2022020314/5a64776e7f8b9afc4d8b478d/html5/thumbnails/22.jpg)
This isn’t over yet !Docker udp issuesTricky iptable filteringUnstable data volumeConfiguration management
![Page 23: Docker to the Rescue of an Ops Team](https://reader030.vdocument.in/reader030/viewer/2022020314/5a64776e7f8b9afc4d8b478d/html5/thumbnails/23.jpg)
Round 2: FIGHT !Metric proxy (Collectd) on the host!!Simplify iptables rulesMount directories (metrics)
![Page 24: Docker to the Rescue of an Ops Team](https://reader030.vdocument.in/reader030/viewer/2022020314/5a64776e7f8b9afc4d8b478d/html5/thumbnails/24.jpg)
A new path opensTo a brighter future
![Page 25: Docker to the Rescue of an Ops Team](https://reader030.vdocument.in/reader030/viewer/2022020314/5a64776e7f8b9afc4d8b478d/html5/thumbnails/25.jpg)
Epic loots !No more dependency issuesReplaceable and movable partsGreater security levelClustering ready (Swarm/K8S)
![Page 26: Docker to the Rescue of an Ops Team](https://reader030.vdocument.in/reader030/viewer/2022020314/5a64776e7f8b9afc4d8b478d/html5/thumbnails/26.jpg)
Lessons learnedCaution when using udp IPv4Config files out of the containerDon’t use env variables (security)Use (abuse) automation
![Page 27: Docker to the Rescue of an Ops Team](https://reader030.vdocument.in/reader030/viewer/2022020314/5a64776e7f8b9afc4d8b478d/html5/thumbnails/27.jpg)
What’s next ?Greater Docker challenges
![Page 28: Docker to the Rescue of an Ops Team](https://reader030.vdocument.in/reader030/viewer/2022020314/5a64776e7f8b9afc4d8b478d/html5/thumbnails/28.jpg)
Many rooms to growReduce (even more) Image sizeSign Images (notary to the rescue)Vulnerability scanningImplement rolling upgrades
![Page 29: Docker to the Rescue of an Ops Team](https://reader030.vdocument.in/reader030/viewer/2022020314/5a64776e7f8b9afc4d8b478d/html5/thumbnails/29.jpg)
New docker based projectsMigrate Development platform (2015) Swarm clustering (*)Full scale Docker (*)(*) Work In Progress
![Page 30: Docker to the Rescue of an Ops Team](https://reader030.vdocument.in/reader030/viewer/2022020314/5a64776e7f8b9afc4d8b478d/html5/thumbnails/30.jpg)
Thank You DockerCon !PS: don’t forget to rate my talk :)