www.huawei.com

HUAWEI TECHNOLOGIES CO., LTD.

Introduction to Mesos Bay

Hongbin Lu

HUAWEI TECHNOLOGIES CO., LTD. 2

Content

Introduction to Mesos

What is Mesos?

Why Mesos?

Architecture

Ecosystem

Marathon

Mesos Bay

Provisioning

Usage

Features

HUAWEI TECHNOLOGIES CO., LTD. 3

What is Mesos?

Apache Mesos is an open source distributed resource

management system to support fine grained resource

sharing among multiple distributed applications.

Roughly speaking, Mesos is the opposite of

virtualization.

Virtualization splits signal machine into multiple virtual

machines.

Mesos put multiple machines together to make them

behave like a signal machine.

HUAWEI TECHNOLOGIES CO., LTD. 4

Why Mesos?

HUAWEI TECHNOLOGIES CO., LTD. 5

Architecture

HUAWEI TECHNOLOGIES CO., LTD. 6

Ecosystem

Frameworks

Long running service: Aurora, Marathon, …

Big data processing: Hadoop, Spark, Storm, MPI, …

Batch scheduling: Jenkins, Chronos, …

Data Storage: Cassandra, ElasticSearch, …

Others: Kubernetes, …

Containerizers

Docker

Cgroups

HUAWEI TECHNOLOGIES CO., LTD. 7

Marathon

Marathon is a Mesos framework for managing applications.

Each application includes one or multiple instances.

Marathon ensures that the specified number of instances are running at any

time (like k8s ReplicationController).

Each instance is scheduled to an executor.

Support several scheduling constraints (i.e. affinity).

The number of instances can be dynamically scaled up or down.

Mesos Marathon

Executor Resources

Tasks

Tasks

Executor

Status

HUAWEI TECHNOLOGIES CO., LTD. 8

Content

Introduction to Mesos

What is Mesos?

Why Mesos?

Architecture

Ecosystem

Marathon

Mesos Bay

Provisioning

Usage

Features

HUAWEI TECHNOLOGIES CO., LTD. 9

Mesos Bay - Provisioning

$ magnum baymodel-create \

--name mesosbaymodel \

--image-id ubuntu-mesos \

--keypair-id testkey \

--external-network-id $NIC_ID \

--dns-nameserver 8.8.8.8 \

--flavor-id m1.small \

--coe mesos

$ magnum bay-create \

--name mesosbay \

--baymodel mesosbaymodel \

--node-count 2

mesos-master

Master Node

marathon

Docker

Slave Node

mesos-slave

Slave Node

docker

zookeeper

HUAWEI TECHNOLOGIES CO., LTD. 10

Mesos Bay - Usage

Use Marathon to launch applications

WebUI

Restful API

CLI or language binding (3rd party)

Example

$ curl -X POST -H "Content-type: application/json" http://$MASTER_IP:8080/v2/apps -d

'{

"container": { "type": "DOCKER", "docker": { "image": "cirros" } },

"id": "ubuntu", "instances": 1, "cpus": 0.5, "mem": 512,

"cmd": "ping 8.8.8.8",

……

}'

HUAWEI TECHNOLOGIES CO., LTD. 11

Mesos Bay – Fault Tolerance

If a slave fails, containers running on that salve will be automatically

relocate to other slaves.

Similar to HDFS.

If a master fails, another master will back up.

Support multiple master nodes, one active and several standby (work in process).

ZooKeeper is used to elect an active master.

On an event that the current elected master fails, ZooKeeper will elect another

master.

HUAWEI TECHNOLOGIES CO., LTD. 12

Mesos Bay – Scheduling

Unique constraints

$ curl -X POST -H "Content-type: application/json" localhost:8080/v2/apps -d '{

"id": "sleep-unique",

"cmd": "sleep 60",

"instances": 3,

"constraints": [["hostname", "UNIQUE"]]

}'

Cluster constraints

$ curl -X POST -H "Content-type: application/json" localhost:8080/v2/apps -d '{

"id": "sleep-cluster",

"cmd": "sleep 60",

"instances": 3,

"constraints": [["rack_id", "CLUSTER", "rack-1"]]

}'

Others

…

HUAWEI TECHNOLOGIES CO., LTD. 13

Mesos Bay – Health Checks

Ensure that the specified number of instances are running at any time (like k8s

ReplicationController).

$ curl -X POST -H "Content-type: application/json" localhost:8080/v2/apps -d '{

"id": "sleep-unique",

"cmd": "sleep 60",

"instances": 3,

"healthChecks": [{

"protocol": "COMMAND",

"command": { "value": "curl -f -X GET http://$HOST:$PORT0/health" },

"maxConsecutiveFailures": 3

}],

}'

HUAWEI TECHNOLOGIES CO., LTD. 14

Mesos Bay – Other Features

Service discovery

Integrable with Mesos-DNS, a DNS-based service-discovery system

Load balancing

Through a HAProxy

No external load balancer support

Event notification

Allow users to provide an URL to receive event notifications

WebUI

Metrics

Authentication

Basic HTTP authentication and SSL

Thank you www.huawei.com

Copyright©2011 Huawei Technologies Co., Ltd. All Rights Reserved.

The information in this document may contain predictive statements including, without limitation, statements regarding the future

financial and operating results, future product portfolio, new technology, etc. There are a number of factors that could cause actual

results and developments to differ materially from those expressed or implied in the predictive statements. Therefore, such information

is provided for reference purpose only and constitutes neither an offer nor an acceptance. Huawei may change the information at any

time without notice.