containerdayvietnam2016: docker at scale with mesos

Click here to load reader

Post on 20-Mar-2017

187 views

Category:

Technology

4 download

Embed Size (px)

TRANSCRIPT

  • Docker at scale with Mesos

    Phm Tun Anh Lead engineer @ VCCloud Platform Services

  • $ whoami

    Lead engineer at VCCloud Platform Services

    Build & work with large-scale/production platform services for about 6 years

    Services: Storage, DNS, CDN, DDoS Protection, Logging, Private PaaS

  • Agenda

    1. What is Mesos?

    2. How to run docker containers on Mesos

    3. Lessons learned

    4. Q&A

  • 1. What is Mesos?

    2. How to run docker containers on Mesos

    3. Lessons learned

    4. Q&A

  • https://blog.twitter.com/2016/overview-of-the-twitter-cloud-platform-compute

    https://blog.twitter.com/2016/overview-of-the-twitter-cloud-platform-compute

  • https://mesosphere.com/blog/2015/04/23/apple-details-j-a-r-v-i-s-the-mesos-framework-that-runs-siri/

    https://mesosphere.com/blog/2015/04/23/apple-details-j-a-r-v-i-s-the-mesos-framework-that-runs-siri/

  • Powered by Mesos: Apple

    Atlassian

    CERN

    Cisco

    Cloudflare

    Foursquare

    Groupon

    HubSpot

    Netflix

    Opera

    Paypal

    Shopee

    Shopify

    Saleforce

    Twitter

    Uber

    Vimeo

    Verizon

    Weibo

    Yelp

    Xiaomi

    http://mesos.apache.org/documentation/latest/powered-by-mesos/

    http://mesos.apache.org/documentation/latest/powered-by-mesos/

  • Solomon Hykes, Founder & CTO of Docker, said at DockerCon EU 12/2014:

    Mesos is the gold standard for large-scale production clusters running containers

  • Mesos is:

    Open source, top-level Apache project

    Proven at massive scale

    Multi-resource (CPU, RAM, Disk, GPU) scheduling

    Simplified operational model

    Cross platform

    Native Docker support

  • PaaS: Marathon, Aurora, Kubernetes, Swarm

    Big Data: Hadoop, Spark, Storm

    Database: Cassandra, ArangoDB,

    ElasticSearch, Chronos, Jenkins, Kafka

    Mesos Frameworks:

  • 1. What is Mesos?

    2. How to run docker containers on Mesos

    3. Lessons learned

    4. Q&A

  • Apache Aurora

    Google Kubernetes

    HubSpot Singularity

    Mesosphere Marathon

  • Features:

    Start, stop, scale, update, rollback apps

    Rolling deploy / restart

    Constraints

    Health checks

    App dependencies

    Highly available, no SPoF

  • 1. What is Mesos?

    2. How to run docker containers on mesos

    3. Lessons learned

    4. Q&A

  • Immutable Infrastructure / CodeShip

    Some of the major challenges today when building infrastructure are predictability, scalability and automated recovery.

  • Container style

    Service discovery

    Networking

    Lessons Learned

    Managing secrets

    App deployment

    Docker annoyances

  • Choose a container style:

    "thin" single-process container

    "fat" container

    sshd in container

  • Container style

    Service discovery

    Networking

    Lessons Learned

    Managing secrets

    App deployment

    Docker annoyances

  • Services Discovery

    Mesos-DNS:

    ..marathon.mesos

    record types: A / SRV

    simple & stateless

  • Container style

    Service discovery

    Networking

    Lessons Learned

    Managing secrets

    App deployment

    Docker annoyances

  • Networking

    Project Calico:

    IP-per-container

    Pure Layer 3

  • Container style

    Service discovery

    Networking

    Lessons Learned

    Managing secrets

    App deployment

    Docker annoyances

  • Managing Secrets

    ansible-vault

  • Container style

    Service discovery

    Networking

    Lessons Learned

    Managing secrets

    App deployment

    Docker annoyances

  • App deployment

    ansible-playbook

    marathon.json

    docker tag name = git commit hash

    1 mesos cluster (3 VMs) per dev

    deploy to servers deploy to marathon

  • marathon.json

  • https://flask-hello.marathon.mesos.vn/

    https://flask-hello.marathon.mesos.vn/

  • Monitoring

    Containers RAM used

  • Stateful Containers

    Local persistent volumes

  • Container style

    Service discovery

    Networking

    Lessons Learned

    Managing secrets

    App deployment

    Docker annoyances

  • Docker

    restart / upgrade docker without stopping the running containers

    old images / tags cleanup

    containers stdout / stderr log files

    docker pull

    docker hub

  • "Normal" container vs cloud-native container:

    No manual steps

    Service discovery friendly

    Config files should be optional

    Using env vars for config

  • 1. What is Mesos?

    2. How to run docker containers on Mesos

    3. Lessons learned

    4. Q&A

  • Links

    Mesos: http://mesos.apache.org/

    Marathon: https://mesosphere.github.io/marathon/

    Project Calico: https://www.projectcalico.org/

    Mesos-DNS: https://github.com/mesosphere/mesos-dns

    A Healthy Platform Checklist: https://gist.github.com/andy-pham/04dedff872b33c07a31277afb900950b

    http://mesos.apache.org/https://mesosphere.github.io/marathon/https://www.projectcalico.org/https://github.com/mesosphere/mesos-dnshttps://gist.github.com/andy-pham/04dedff872b33c07a31277afb900950b