introduction to mesos bay
Post on 21-Aug-2015
42 Views
Preview:
TRANSCRIPT
www.huawei.com
HUAWEI TECHNOLOGIES CO., LTD.
Introduction to Mesos Bay
Hongbin Lu
HUAWEI TECHNOLOGIES CO., LTD. 2
Content
Introduction to Mesos
What is Mesos?
Why Mesos?
Architecture
Ecosystem
Marathon
Mesos Bay
Provisioning
Usage
Features
HUAWEI TECHNOLOGIES CO., LTD. 3
What is Mesos?
Apache Mesos is an open source distributed resource
management system to support fine grained resource
sharing among multiple distributed applications.
Roughly speaking, Mesos is the opposite of
virtualization.
Virtualization splits signal machine into multiple virtual
machines.
Mesos put multiple machines together to make them
behave like a signal machine.
HUAWEI TECHNOLOGIES CO., LTD. 4
Why Mesos?
HUAWEI TECHNOLOGIES CO., LTD. 5
Architecture
HUAWEI TECHNOLOGIES CO., LTD. 6
Ecosystem
Frameworks
Long running service: Aurora, Marathon, …
Big data processing: Hadoop, Spark, Storm, MPI, …
Batch scheduling: Jenkins, Chronos, …
Data Storage: Cassandra, ElasticSearch, …
Others: Kubernetes, …
Containerizers
Docker
Cgroups
HUAWEI TECHNOLOGIES CO., LTD. 7
Marathon
Marathon is a Mesos framework for managing applications.
Each application includes one or multiple instances.
Marathon ensures that the specified number of instances are running at any
time (like k8s ReplicationController).
Each instance is scheduled to an executor.
Support several scheduling constraints (i.e. affinity).
The number of instances can be dynamically scaled up or down.
Mesos Marathon
Executor Resources
Tasks
Tasks
Executor
Status
HUAWEI TECHNOLOGIES CO., LTD. 8
Content
Introduction to Mesos
What is Mesos?
Why Mesos?
Architecture
Ecosystem
Marathon
Mesos Bay
Provisioning
Usage
Features
HUAWEI TECHNOLOGIES CO., LTD. 9
Mesos Bay - Provisioning
$ magnum baymodel-create \
--name mesosbaymodel \
--image-id ubuntu-mesos \
--keypair-id testkey \
--external-network-id $NIC_ID \
--dns-nameserver 8.8.8.8 \
--flavor-id m1.small \
--coe mesos
$ magnum bay-create \
--name mesosbay \
--baymodel mesosbaymodel \
--node-count 2
mesos-master
Master Node
marathon
Docker
Slave Node
mesos-slave
Slave Node
docker
zookeeper
HUAWEI TECHNOLOGIES CO., LTD. 10
Mesos Bay - Usage
Use Marathon to launch applications
WebUI
Restful API
CLI or language binding (3rd party)
Example
$ curl -X POST -H "Content-type: application/json" http://$MASTER_IP:8080/v2/apps -d
'{
"container": { "type": "DOCKER", "docker": { "image": "cirros" } },
"id": "ubuntu", "instances": 1, "cpus": 0.5, "mem": 512,
"cmd": "ping 8.8.8.8",
……
}'
HUAWEI TECHNOLOGIES CO., LTD. 11
Mesos Bay – Fault Tolerance
If a slave fails, containers running on that salve will be automatically
relocate to other slaves.
Similar to HDFS.
If a master fails, another master will back up.
Support multiple master nodes, one active and several standby (work in process).
ZooKeeper is used to elect an active master.
On an event that the current elected master fails, ZooKeeper will elect another
master.
HUAWEI TECHNOLOGIES CO., LTD. 12
Mesos Bay – Scheduling
Unique constraints
$ curl -X POST -H "Content-type: application/json" localhost:8080/v2/apps -d '{
"id": "sleep-unique",
"cmd": "sleep 60",
"instances": 3,
"constraints": [["hostname", "UNIQUE"]]
}'
Cluster constraints
$ curl -X POST -H "Content-type: application/json" localhost:8080/v2/apps -d '{
"id": "sleep-cluster",
"cmd": "sleep 60",
"instances": 3,
"constraints": [["rack_id", "CLUSTER", "rack-1"]]
}'
Others
…
HUAWEI TECHNOLOGIES CO., LTD. 13
Mesos Bay – Health Checks
Ensure that the specified number of instances are running at any time (like k8s
ReplicationController).
$ curl -X POST -H "Content-type: application/json" localhost:8080/v2/apps -d '{
"id": "sleep-unique",
"cmd": "sleep 60",
"instances": 3,
"healthChecks": [{
"protocol": "COMMAND",
"command": { "value": "curl -f -X GET http://$HOST:$PORT0/health" },
"maxConsecutiveFailures": 3
}],
}'
HUAWEI TECHNOLOGIES CO., LTD. 14
Mesos Bay – Other Features
Service discovery
Integrable with Mesos-DNS, a DNS-based service-discovery system
Load balancing
Through a HAProxy
No external load balancer support
Event notification
Allow users to provide an URL to receive event notifications
WebUI
Metrics
Authentication
Basic HTTP authentication and SSL
Thank you www.huawei.com
Copyright©2011 Huawei Technologies Co., Ltd. All Rights Reserved.
The information in this document may contain predictive statements including, without limitation, statements regarding the future
financial and operating results, future product portfolio, new technology, etc. There are a number of factors that could cause actual
results and developments to differ materially from those expressed or implied in the predictive statements. Therefore, such information
is provided for reference purpose only and constitutes neither an offer nor an acceptance. Huawei may change the information at any
time without notice.
top related