on-demand continuous integration with jenkins, jclouds, and cloudstack

Jenkins, jclouds, and CloudStackDavid Nalley @ke4qqq [email protected]

#whoami

• Recovering sysadmin• Former server hugger, now cloud

addict• Contributor to a few open source

projects:– Fedora, Zenoss, CloudStack, jclouds

• VP, Infrastructure for the Apache Software Foundation

• Employed by Citrix in the Open Source Business Office

Jenkins + jclouds + Centos + CloudStack

Background: Jenkins

‘Continuous Integration Server’

Formerly Hudson – community forked

Written in Java

MIT-LicensedLots of plugins


Background: Apache jclouds

Cloud Abstraction Library

Written in Java

ASLv2 License


Background: Apache CloudStack

Formerly from Cloud.comCompute- focused IaaS management solution.

Started in 2008

Now a top level project at the ASF

Written in Java

ASLv2


The Problem

Lots of different testing needs – multiple platforms, multiple versions, multiple versions of software.


The Problem – at scale

We have nearly 200 software projects each with their own set of needs, distinct platforms. Some want CentOS 5, 6, or 7; others want OSX, Win 2012, Ubuntu, FreeBSD, or Solaris.

Demand varies widely – projects getting closer to release test more; some days we had hundreds of jobs in the queue, other days we had 20 or 30


The transition for monolith to scalable.


Where we were

Dedicated Jenkins master

Esoteric things had VMs

~30 dedicated machines, strewn across multiple datacenters.


How we started our move

• Defining what a build slave needs

• Make sure that Jenkins is working efficiently(Misconfigured jobs are a huge drain)

• Adopted a LTS version of Jenkins


Building images

Packer – building images

Start with JeOS – declare everything else in puppet, run puppet as a job.

Create a jenkins job for this – everytime a job changes – rebuild, reupload.


How you plan to use Cloud

Server huggers wanted to use fixed VMs

(deploy fixed machines)


How you want to use cloud

Fresh, dynamically provisioned VM for each build


The right answer


The right answer(for us)

Spin up build slaves in response to demand.

Leave the machine up for 30 minutes after demand subsides to keep from flapping.


Make it all work


Install the jclouds plugin


Choose a cloud provider


Configure


Profit?


Bonus


Get Involved

Web: http://cloudstack.apache.org/

Mailing Lists: cloudstack.apache.org/mailing-lists.html

IRC: irc.freenode.net: 6667 #cloudstack

Twitter: @cloudstack

LinkedIn: www.linkedin.com/groups/CloudStack-Users-Group-3144859

If it didn’t happen on the mailing list, it didn’t happen.


on-demand continuous integration with jenkins, jclouds, and cloudstack

Technology