building a dev/test cloud with apache cloudstack
Post on 16-Apr-2017
797 Views
Preview:
TRANSCRIPT
Building a Test/Dev Cloud with Apache CloudStack
David Nalleyke4qqq@apache.org@ke4qqq
#whoami
Sysadmin of a decade
Apache Software Foundation Member
Apache CloudStack Committer & PMC Member
Fedora Project Contributor
Employed by Citrix in the Open Source Business Office
Why use cloud?
From a dev POV the process normally looks like:Start new project
File ticket for resources....wait....wait....wait
Get resources, that aren't configured....wait...
Get network access.....get permission....wait
Get things done.
Why use cloud?
What IT Operations provides is not what a developer wants.
Dev wants to work - IT has process, and tickets, etc.
Get rid of the waiting
Remove the constraints - developers empowered to get things done.
Agility
Enforce automated process instead of manual ones
What does a dev/test cloud look like?
Self-service - developers can provision their own environments
Usage measurement - we worry about VM sprawl
Isolated networks - must not let dev/test interfere with the real world.
Commodity - as cheap as practical
May also house production workloads
Talk a lot about the general architecture of CloudStack
Self service
Provisioning manually doesn't add value
Can be completely automated
Do they need full control or just pushing pre-configured environments?
Self-service
UI?
API?
Some external tool?
What does self service really mean?
IS this really your idea of enabling developers?
It might be nice to have, but it shouldn't be the real interface - even with defaults this takes 6 clicks.....which is better than filing a ticket, but doesn't scale.
API or Commandline?
cloudmonkey> deploy virtualmachine serviceofferingid=d8611d07-acf5-4cd4-a630-5c4d937ef043 templateid=081358ff-2427-44f8-adcc-1bb002fab361 zoneid=d06193b2-7980-4ad1-bd8-7b2f2eda63c3curl 'http://localhost:8096/client/api?command=listUsers'
Config-mgmt based deployment
{
"name": "hadoop_cluster_a",
"description": "A small hadoop cluster with hbase",
"version": "1.0",
"environment": "production",
"servers": [
{
"name": "zookeeper-a, zookeeper-b, zookeeper-c",
"description": "Zookeeper nodes",
"template": "rhel-5.6-base",
"service": "small",
"port_rules": "2181",
"run_list": "role[cluster_a], role[zookeeper_server]",
"actions": [
{ "knife_ssh": ["role:zookeeper_server", "sudo chef-client"]
}
]
},
{
"name": "hadoop-master",
"description": "Hadoop master node",
"template": "rhel-5.6-base",
"service": "large",
"networks": "app-net, storage-net",
"port_rules": "50070, 50030, 60010",
"run_list": "role[cluster_a], role[hadoop_master],
role[hbase_master]"
},
{
"name": "hadoop-worker-a hadoop-worker-b hadoop-worker-c",
"description": "Hadoop worker nodes",
"template": "rhel-5.6-base",
"service": "medium",
"port_rules": "50075, 50060, 60030",
"run_list": "role[cluster_a], role[hadoop_worker],
role[hbase_regionserver]",
"actions": [
{ "knife_ssh": ["role:hadoop_master", "sudo chef-client"] },
{ "http_request": "http://${hadoop-master}:50070/index.jsp" }
]
}
}
Use a tool
Usage
Jevons Paradox
Plenty of waste possible as well - will developers always destroy a machine when they are done with it?
Important to show what projects and groups are consuming resources as well as how they are using those resources
Commodity - Storage
Commodity storage - this is dev/test environment - high performance, resilient storage isn't needed.
Local storage tends to be the best mix of cheap and performant
No failover, but it's dev/test - do you need it?
Commodity - Networking
Layer 3 isolation - (aka Security Groups)
VLANs - (not as commodity, but still relatively cheap on a small scale, but not at scale)
Virtual routers (provide DHCP, DNS, LB, Firewall, PF, NAT, etc)
Commodity Hypervisor
KVM is my personal choice in this space.
Easiest to consume - completely open source
Limiting resources
Limit the number of VMs, snapshots, IP addresses, etc.
Use 'projects' to share resources
This means most folks will never have problems, but heaviest users will not be able to interrupt service for others.
Elasticity is only imagined - look at default limits placed on AWS users.
Questions?
Resources
http://cloudstack.apache.org
http://cloudstack.jp
#cloudstack on irc.freenode.net
Mailing lists/google group
top related