9th docker meetup 2016.07.13

Our experiences with Docker!

Dr. Piyush Harshhttp://piyush-harsh.info/

Senior Researcher, ICCLabZurich University of Applied Sciences

What is Cyclops➔ Fully open-source (ASL2.0) framework for rating-charging-billing

◆ Highly modular - micro-services architecture with well defined APIs◆ Generic, model-driven processing logic (enables a very reactive billing strategy)

➔ Pluggable collectors◆ You can write your own collector module◆ Already available usage data / billing events collectors for popular IaaS frameworks

● OpenStack and CloudStack◆ Many more under planning - hadoop, mesos, docker, openshift, cloud foundry etc.◆ Don’t wait for it to be made available by ICCLab - write your own!

➔ Highly scalable◆ Autoscaling logic included◆ Framework scales easily to thousands of billable events / minute

List of microservices

Package DependenciesWith numerous micro-services controlling the build and test environment is difficult

Specific requirements -

➠ Maven 3.0.5 or higher➠ Java 8➠ InfluxDB 0.9.3 or higher➠ RabbitMQ➠ PostgresSQL➠ And there are others ...

End to end testing nightmare!!➔ We work a lot with students and interns, after all ZHAW

is an educational institution◆ They get excited very easily with some *cool* feature in a new

release without thinking about the implications of it!◆ Many are still learning concepts of software development.◆ They are eager to learn new things and we don’t STOP them!

➔ But this flexibility creates a testing and debugging nightmare!!◆ Cases where an intern spends 1 day fixing something which was

NOT broken at all is more common than anyone would care to admit!

Cyclops with Docker➔ Streamlined testing environment regardless of people’s own base system

configuration➔ We divided all containers into 2 groups

◆ Static - where the base image is fixed, these we published on hub.docker.com◆ On the fly - containers with untested, under-development code modules

● The images for these are build locally

➔ Faster & predictable environment setup for each test run!

Easy composition with docker-composeDocker-compose allows an easy way to stitch together all containers that make up a service!

➔ Containers are created in order!➔ Linking containers makes the services available

by name◆ Allows configuration file stability

Developers and testers can now create and execute the complete Cyclops framework in one step!

Why not use cloud (or VM) based techniques?We could have used Heat or AWS CloudFormation, or something similar!

❖ Clouds for testing is costly, docker is free!

Why not use cloud based techniques?Even with this highly improbable scenario ...

Why not use cloud based techniques?Even if I had gazillions of gold coins, docker allows me full control and the response time is orders of magnitude faster than any VM based deployment strategies.

Specific challenges we faced!!➔ Port conflicts are with external host environments (using Docker for mac)

◆ Easy to fix - change the external mapped port value◆ Services in containers that are linked are accessible at the internal ports

● Could be useful when working with container images developed by someone else

➔ Order of container start does not ensure that services start in that order too!◆ Also easy to fix: create a script to check service dependencies before starting self

➔ Logging …◆ Either send all logs to console - docker-compose picks up and color-codes log outputs◆ Or send all logs to a logging server

● ELK stack is useful

Specific challenges we faced!!➔ Port conflicts are with external host environments (using Docker for mac)

◆ Easy to fix - change the external mapped port value◆ Services in containers that are linked are accessible at the internal ports

● Could be useful when working with container images developed by someone else

➔ Order of container start does not ensure that services start in that order too!◆ Also easy to fix: create a script to check service dependencies before starting self

➔ Logging …◆ Either send all logs to console - docker-compose picks up and color-codes log outputs◆ Or send all logs to a logging server

● ELK stack is useful

vs

Specific challenges we faced!!➔ Port conflicts are with external host environments (using Docker for mac)

◆ Easy to fix - change the external mapped port value◆ Services in containers that are linked are accessible at the internal ports

● Could be useful when working with container images developed by someone else

➔ Order of container start does not ensure that services start in that order too!◆ Also easy to fix: create a script to check service dependencies before starting self

➔ Logging …◆ Either send all logs to console - docker-compose picks up and color-codes log outputs◆ Or send all logs to a logging server

● ELK stack is useful

vs

Cyclops demo: TNova use-caseMarketplace and brokerage for VNF developers, users and operators.➔ Actors

◆ Service providers◆ Function developers◆ Service users

➔ Requirements on billing◆ Revenue sharing reports

● Between SP and FP

● Negotiated billing & SLA models

● SLA Penalty models◆ Billing reports for users

Cyclops demo: TNova use-caseMarketplace and brokerage for VNF developers, users and operators.➔ Cyclops interfaces with

external components◆ Accounting module

● Developed by ATOS

➔ Cyclops µServices used◆ Udr, rc and billing, rating◆ Scheduler

● Periodicity: 2 minutes

LET’S SEE THE DEMO NOW!!

Time for a quick hands-on session now ...Using the TNova use-case -

Steps:

➔ Start cyclops framework with docker-compose➔ Start a service with 2 constituent VNFs via ATOS accounting module➔ Start their respective SLA enforcements via ATOS SLA manager➔ Create a bill for customer c1 that is consuming the service

◆ What is owes to provider p1

➔ Create a revenue sharing report for function provider f1◆ What is should get from various “customers” - the service providers

Conclusion wrt Cyclops & DockerDocker is sweeeet!

➔ Much more control over development and testing◆ Core service provisioning is much faster

➔ Better developer experience in the community◆ The core system just works!

With docker orchestration and clustering frameworks such as docker-swarm or kubernetes, it should be easy to test Cyclops under load-balancers and see the effects of elasticity on latencies and consistencies.

Further information ...Cyclops Page: http://icclab.github.io/cyclops/

Gatekeeper: http://icclab.github.io/gatekeeper/

ICCLab RCB initiative page: http://bit.ly/1WYC27l

@rcb_cyclops - follow us on twitter.

Cyclops has been proudly funded by these organizations / projects