Download - 7 Ways to Optimize Hudson in Production
7 Ways To Optimize Hudson/Jenkins for Production Installation
Kohsuke Kawaguchi Creator of the Hudson project
©2011 Cloud Bees, Inc. All Rights Reserved
2 ©2011 Cloud Bees, Inc. All Rights Reserved
About CloudBees Our Mission
Strategy
Become the leading Java™ Pla3orm as a Service (PaaS)
Why We’re Different
CloudBees services the complete lifecycle of Cloud applica>on development and deployment. No Servers. No Virtual Machines. No IT.
§ DEV@cloud – Cloud Services for Developers
§ RUN@cloud – Fric@onless run@me PaaS for Java apps
3
Benefits of DEV@cloud Hudson Service: • Scale your Hudson environment with the power of the Cloud
• Ease your Hudson management overhead • Speed your Hudson builds • Save money with on-‐demand Hudson Service.
©2011 Cloud Bees, Inc. All Rights Reserved
Continuous Integration - Hudson
0
5,000
10,000
15,000
20,000
25,000
Hudson Adoption, by Builds
Source: Hudson Labs
Nectar – On-‐Premise Enterprise Hudson • Support from the Hudson experts. • VMware scale your Hudson environment. • Enterprise Features extend Hudson for large environments.
• Integrate with the Cloud integra@on with DEV@Cloud and RUN@Cloud coming
4 ©2011 CloudBees, Inc. All Rights Reserved
• 7 relatively easy techniques you can apply today
• It helps to spend some time setting it up “right” – Even if Hudson is easy enough to deploy – Especially for serious production use
• It’s not too late even if you are already running Hudson
Idea Behind This Webinar
©2011 CloudBees, Inc. All Rights Reserved
5 ©2011 CloudBees, Inc. All Rights Reserved
• Better late than never • Useful for
– Disaster recovery – Accidental configuration changes
#1: Backup
6 ©2011 CloudBees, Inc. All Rights Reserved
• Everything is under $HUDSON_HOME – Check system config to see where it is – No need to backup slaves
• Live vs offline – Live backup is OK – Configuration writes are atomic
#1: Backup Planning
7 ©2011 CloudBees, Inc. All Rights Reserved
• /war (exploded war) • /cache (downloaded tools) • /tools (extracted tools) • /jobs/*
– builds (build records) – builds/*/archive (archived artifacts) – workspace
#1: Subsetting backup
8 ©2011 CloudBees, Inc. All Rights Reserved
• File system-level snapshot for maximum consistency – LVM – Other storage-level snapshots
• Don’t forget fsfreeze – ZFS, if you are on Solaris
• Incremental, too
• Nectar can help you
#1: More on backup
9 ©2011 CloudBees, Inc. All Rights Reserved
• Just copy it somewhere and “java -jar hudson.war --httpPort=9999” – HUDSON_HOME is relocateable
#1: Testing restore
10 ©2011 CloudBees, Inc. All Rights Reserved
• Prepare for disk usage growth – Especially when you start to host jobs from
people who aren’t close to you – Make sure you can throw more disks at the
problem later
• No need to waste money on 15000rpm SCSI disks – But bigger disk is nice
#2: Disk
11 ©2011 CloudBees, Inc. All Rights Reserved
• “Spanned volume” on Windows • LVM on Linux
– This needs the most upfront planning • ZFS
– Shame there’s no viable free Solaris flavor – $HUDSON_HOME should be its own FS
• If it’s too late, symlink is your friend
#2: Expandable volume
12 ©2011 CloudBees, Inc. All Rights Reserved
• Comes in .deb/.rpm – Much easier to install/upgrade – Reproducible – init script & daemon user
• Configuration in – /etc/default/hudson – /etc/sysconfig/hudson
• Windows --- work in progress • Existing $HUDSON_HOME can be
migrated
#3: Native packages
13 ©2011 CloudBees, Inc. All Rights Reserved
• You will grow beyond a single system – Load is not the only factor
• Better isolation between builds • Diversity in the test/build platform
– Cheaper to scale horizontally
• Let Hudson use multiple computers
#4: Distributed builds
Demo
15 ©2011 CloudBees, Inc. All Rights Reserved
• Let master launch slaves (if you can) – Via SSH or DCOM – Easier to keep the cluster up & running – It’s amazing how easily a computer gets sick
• Set up SSH public key authentication – Login to systems without typing password – Must-have! Inexcusable! – Only takes 5 minutes or so to set up
#4: Slave reconnectivity
16 ©2011 CloudBees, Inc. All Rights Reserved
• Treat build machines like livestock, not like pets – Don’t tie builds to specific build machine – Build machines should be interchangeable
• Labels, instead of names – Tag slaves by their capabilities/environments – Jobs can be tied to boolean expression of
labels • Better resource utilization, manageability
#5: Labels
Demo
18 ©2011 CloudBees, Inc. All Rights Reserved
• If your users can’t see Hudson, much of the benefit is lost – Make the URL easier to remember
#6: Invest in good URL
hVp://sca14-‐3530.sca.cloudbees.com:8080/hudson/
hVp://hudson.cloudbees.com/
19 ©2011 CloudBees, Inc. All Rights Reserved
• Get the host alias, instead of using primary machine name – If your IT isn’t helping, use dynamic DNS – This also makes your service relocatable
#6: Use service name
20 ©2011 CloudBees, Inc. All Rights Reserved
• Apache reverse proxy – Let you run Hudson in non-root, too
• For Windows – IIS7+URL Rewrite+Application Request
Routing
#6: Share port 80 with other apps
Apache Hudson browser
21 ©2011 CloudBees, Inc. All Rights Reserved
• Use virtual host to distinguish multiple apps, not context path
#6: No “/hudson”
Apache / IIS
Hudson
browser
Redmine hVp://hudson.cloudbees.com/ hVp://redmine.cloudbees.com/
22 ©2011 CloudBees, Inc. All Rights Reserved
• Discard old build records if you can – Helps start-up time, memory usage – Often feasible, for example CI builds
• Setting is per-project basis
• The point is to avoid unbounded consumption. Don’t go overboard
#7: Keeping build records under control
Demo
24 ©2011 CloudBees, Inc. All Rights Reserved
• 7 tips – Backup – Disk planning – Native packages – Distributed builds – Labels – Reverse proxy – Discard build records
Conclusion
25 ©2011 CloudBees, Inc. All Rights Reserved
• CloudBees – http://www.cloudbees.com/
• Nectar – http://nectar.cloudbees.com/
• Try Dev@Cloud – https://grandcentral.cloudbees.com/account/signup
• Register for news from CloudBees – http://www.cloudbees.com/company.cb
• CloudBees on YouTube: new videos soon – http://www.youtube.com/user/CloudBeesTV
CloudBees Resources
26 ©2011 CloudBees, Inc. All Rights Reserved
• Instructor-led • More comprehensive, all-round • London, New York, and Tokyo
• To receive updates: – http://www.cloudbees.com/company.cb
Coming Soon: 1 Day Hudson Training
Q&A
©2011 CloudBees, Inc. All Rights Reserved