jenkins state of union 2013
DESCRIPTION
Current state of the Jenkins projectTRANSCRIPT
![Page 1: Jenkins State of union 2013](https://reader035.vdocument.in/reader035/viewer/2022062513/554f41e6b4c905cd048b54b1/html5/thumbnails/1.jpg)
Jenkins User Conference Shefayim, Jun 2013 #jenkinsci
Jenkins: State of Union
#jenkinsci
Kohsuke KawaguchiJenkins Creator
[email protected] / @kohsukekawa
![Page 2: Jenkins State of union 2013](https://reader035.vdocument.in/reader035/viewer/2022062513/554f41e6b4c905cd048b54b1/html5/thumbnails/2.jpg)
Jenkins User Conference Shefayim, Jun 2013 #jenkinsci
In Average Week of Jenkins
![Page 3: Jenkins State of union 2013](https://reader035.vdocument.in/reader035/viewer/2022062513/554f41e6b4c905cd048b54b1/html5/thumbnails/3.jpg)
Jenkins User Conference Shefayim, Jun 2013 #jenkinsci
1 new release comes out
Gets downloaded 48,000 times
![Page 4: Jenkins State of union 2013](https://reader035.vdocument.in/reader035/viewer/2022062513/554f41e6b4c905cd048b54b1/html5/thumbnails/4.jpg)
Jenkins User Conference Shefayim, Jun 2013 #jenkinsci
That fills up 513 DVDs
![Page 5: Jenkins State of union 2013](https://reader035.vdocument.in/reader035/viewer/2022062513/554f41e6b4c905cd048b54b1/html5/thumbnails/5.jpg)
Jenkins User Conference Shefayim, Jun 2013 #jenkinsci
3.5 Brand-new Plugins Released
![Page 6: Jenkins State of union 2013](https://reader035.vdocument.in/reader035/viewer/2022062513/554f41e6b4c905cd048b54b1/html5/thumbnails/6.jpg)
Jenkins User Conference Shefayim, Jun 2013 #jenkinsci
25 New Versions of Plugins Released
Gets downloaded 500,000 times
![Page 7: Jenkins State of union 2013](https://reader035.vdocument.in/reader035/viewer/2022062513/554f41e6b4c905cd048b54b1/html5/thumbnails/7.jpg)
Jenkins User Conference Shefayim, Jun 2013 #jenkinsci
80 Tickets Filed
![Page 8: Jenkins State of union 2013](https://reader035.vdocument.in/reader035/viewer/2022062513/554f41e6b4c905cd048b54b1/html5/thumbnails/8.jpg)
Jenkins User Conference Shefayim, Jun 2013 #jenkinsci
50 Tickets Solved
![Page 9: Jenkins State of union 2013](https://reader035.vdocument.in/reader035/viewer/2022062513/554f41e6b4c905cd048b54b1/html5/thumbnails/9.jpg)
Jenkins User Conference Shefayim, Jun 2013 #jenkinsci
145 Commits Are Made
![Page 10: Jenkins State of union 2013](https://reader035.vdocument.in/reader035/viewer/2022062513/554f41e6b4c905cd048b54b1/html5/thumbnails/10.jpg)
Jenkins User Conference Shefayim, Jun 2013 #jenkinsci
400 Jenkins Installations Pops Up
![Page 11: Jenkins State of union 2013](https://reader035.vdocument.in/reader035/viewer/2022062513/554f41e6b4c905cd048b54b1/html5/thumbnails/11.jpg)
Jenkins User Conference Shefayim, Jun 2013 #jenkinsci
1200 New Slaves Are Added
![Page 12: Jenkins State of union 2013](https://reader035.vdocument.in/reader035/viewer/2022062513/554f41e6b4c905cd048b54b1/html5/thumbnails/12.jpg)
Jenkins User Conference Shefayim, Jun 2013 #jenkinsci
19,000 New Jobs Are Added
![Page 13: Jenkins State of union 2013](https://reader035.vdocument.in/reader035/viewer/2022062513/554f41e6b4c905cd048b54b1/html5/thumbnails/13.jpg)
Jenkins User Conference Shefayim, Jun 2013 #jenkinsci
# of plugins
Jan-07 Jan-08 Jan-09 Jan-10 Jan-11 Jan-12 Jan-130
100
200
300
400
500
600
700
800
767
# of unique plugins
releases/weeknew plugins/week
![Page 14: Jenkins State of union 2013](https://reader035.vdocument.in/reader035/viewer/2022062513/554f41e6b4c905cd048b54b1/html5/thumbnails/14.jpg)
Jenkins User Conference Shefayim, Jun 2013 #jenkinsci
Ticket Activities
Jan-07 Jan-08 Jan-09 Jan-10 Jan-11 Jan-12 Jan-130
2000
4000
6000
8000
10000
12000
14000
16000
18000
17500
Total tickets
Resolved tickets
![Page 15: Jenkins State of union 2013](https://reader035.vdocument.in/reader035/viewer/2022062513/554f41e6b4c905cd048b54b1/html5/thumbnails/15.jpg)
Jenkins User Conference Shefayim, Jun 2013 #jenkinsci
Tracked Installations (conservative)
Jan-07 Jan-08 Jan-09 Jan-10 Jan-11 Jan-12 Jan-130
10000
20000
30000
40000
50000
60000
64000
# of masters
![Page 16: Jenkins State of union 2013](https://reader035.vdocument.in/reader035/viewer/2022062513/554f41e6b4c905cd048b54b1/html5/thumbnails/16.jpg)
Jenkins User Conference Shefayim, Jun 2013 #jenkinsci
![Page 17: Jenkins State of union 2013](https://reader035.vdocument.in/reader035/viewer/2022062513/554f41e6b4c905cd048b54b1/html5/thumbnails/17.jpg)
Jenkins User Conference Shefayim, Jun 2013 #jenkinsci
Recently Introduced Featureshttp://www.flickr.com/photos/randar/3879926596/
![Page 18: Jenkins State of union 2013](https://reader035.vdocument.in/reader035/viewer/2022062513/554f41e6b4c905cd048b54b1/html5/thumbnails/18.jpg)
Jenkins User Conference Shefayim, Jun 2013 #jenkinsci
Lazy loading of build records
• Jenkins slow to start up?– It was loading build records upon boot
• This is finally fixed, including fallouts– Loaded on demand– In weak references
• Now on LTS: 1.509.1
![Page 19: Jenkins State of union 2013](https://reader035.vdocument.in/reader035/viewer/2022062513/554f41e6b4c905cd048b54b1/html5/thumbnails/19.jpg)
Jenkins User Conference Shefayim, Jun 2013 #jenkinsci
Master/slave performance
• Better sustained data transfer rate– Copy large files, test reports, etc– Now on par with scp
• Classloading improvements– Local cache & prefetching– Will help ephemeral/elastic slaves & maven jobs
![Page 20: Jenkins State of union 2013](https://reader035.vdocument.in/reader035/viewer/2022062513/554f41e6b4c905cd048b54b1/html5/thumbnails/20.jpg)
Jenkins User Conference Shefayim, Jun 2013 #jenkinsci
Performance/Scalability Improvements
• Lock strength reduction• Fingerprint scalability improvements• Better caching of images & scripts in UI• The “people” page• Handling large test result
![Page 21: Jenkins State of union 2013](https://reader035.vdocument.in/reader035/viewer/2022062513/554f41e6b4c905cd048b54b1/html5/thumbnails/21.jpg)
Jenkins User Conference Shefayim, Jun 2013 #jenkinsci
Testing Improvements
• Previous testing scheme– Embedded Jetty in Maven– Code from ~/.m2/repository and resources from
war– The setup was different from how you run it
![Page 22: Jenkins State of union 2013](https://reader035.vdocument.in/reader035/viewer/2022062513/554f41e6b4c905cd048b54b1/html5/thumbnails/22.jpg)
Jenkins User Conference Shefayim, Jun 2013 #jenkinsci
New Blackbox Acceptance Test
• Tests the final deliverables– jenkins.war, .deb, .rpm, etc– With Tomcat, JBoss, or “java -jar jenkins.war”
• In different OSes– In VirtualBox (via vagrant)– Hopefully with EC2 (upcoming)
• Tests via Selenium– Capybara + Cucumber– Sauce OnDemand to test different browsers
![Page 23: Jenkins State of union 2013](https://reader035.vdocument.in/reader035/viewer/2022062513/554f41e6b4c905cd048b54b1/html5/thumbnails/23.jpg)
Jenkins User Conference Shefayim, Jun 2013 #jenkinsci
Database Plugin
• For other plugins, not for users– But large data set in RDBMS should result in tangible
benefits for users• Provides unified database connectivity– UI for admin to select the backend– Expose DataSource for other plugins– JPA support, per-job database
• Looking for plugins to start using this
![Page 24: Jenkins State of union 2013](https://reader035.vdocument.in/reader035/viewer/2022062513/554f41e6b4c905cd048b54b1/html5/thumbnails/24.jpg)
Jenkins User Conference Shefayim, Jun 2013 #jenkinsci
Plugins that generate jobs
• Job DSL plugin– Groovy DSL for describing job definitions– DSL can be evaluated from within Jenkins– E.g., automatically generate new jobs for new
branches• Job Generator plugin– Create a new ste of jobs from “template jobs” by
eagerly evaluating some build parameters
![Page 25: Jenkins State of union 2013](https://reader035.vdocument.in/reader035/viewer/2022062513/554f41e6b4c905cd048b54b1/html5/thumbnails/25.jpg)
Jenkins User Conference Shefayim, Jun 2013 #jenkinsci
Build Flow Plugin
• Groovy DSL for kicking builds– High-level primitives– Ability to define abstractions
b = build(“acme-build”)guard {
parallel ({ build(“acme-test1”,
param1:b.number) },{ build(“acme-test2”,
param1:b.number) })
} rescue {build(“acme-teardown”)
}
![Page 26: Jenkins State of union 2013](https://reader035.vdocument.in/reader035/viewer/2022062513/554f41e6b4c905cd048b54b1/html5/thumbnails/26.jpg)
Jenkins User Conference Shefayim, Jun 2013 #jenkinsci
Jenkow Plugin
• Embed BPMN workflow engine in Jenkins– Timeout, fork, join, …
• Workflow is version controlled in Git– Push to Jenkins to load them up
![Page 27: Jenkins State of union 2013](https://reader035.vdocument.in/reader035/viewer/2022062513/554f41e6b4c905cd048b54b1/html5/thumbnails/27.jpg)
Jenkins User Conference Shefayim, Jun 2013 #jenkinsci
Recipe Plugin
• Sharing a set of jobs and other settings in Jenkins with the community
• Convenient for…– Sample setup / quick start for plugins– Show cases
• Simplify the first “getting started” phase
![Page 28: Jenkins State of union 2013](https://reader035.vdocument.in/reader035/viewer/2022062513/554f41e6b4c905cd048b54b1/html5/thumbnails/28.jpg)
Jenkins User Conference Shefayim, Jun 2013 #jenkinsci
Parallel Test Executor Plugin
• Split tests N-way, run them in parallel• Create even splits by using timing information
Test Group #1 Test Group #2 Test Group #3
![Page 29: Jenkins State of union 2013](https://reader035.vdocument.in/reader035/viewer/2022062513/554f41e6b4c905cd048b54b1/html5/thumbnails/29.jpg)
Jenkins User Conference Shefayim, Jun 2013 #jenkinsci
Demo
![Page 30: Jenkins State of union 2013](https://reader035.vdocument.in/reader035/viewer/2022062513/554f41e6b4c905cd048b54b1/html5/thumbnails/30.jpg)
Jenkins User Conference Shefayim, Jun 2013 #jenkinsci
Upcoming Features
![Page 31: Jenkins State of union 2013](https://reader035.vdocument.in/reader035/viewer/2022062513/554f41e6b4c905cd048b54b1/html5/thumbnails/31.jpg)
Jenkins User Conference Shefayim, Jun 2013 #jenkinsci
Future Direction: Which Way?
http://www.flickr.com/photos/quinnanya/3500158343/
![Page 32: Jenkins State of union 2013](https://reader035.vdocument.in/reader035/viewer/2022062513/554f41e6b4c905cd048b54b1/html5/thumbnails/32.jpg)
Jenkins User Conference Shefayim, Jun 2013 #jenkinsci
Make Complex Things Doable
• Running big, complex deployment should be easier– Master-to-master communication channel– Per-job database storage, open for plugins
• More automatic slave managements– Better diagnostics
• Access control for running builds
![Page 33: Jenkins State of union 2013](https://reader035.vdocument.in/reader035/viewer/2022062513/554f41e6b4c905cd048b54b1/html5/thumbnails/33.jpg)
Jenkins User Conference Shefayim, Jun 2013 #jenkinsci
Make Complex Things Doable
• Better Workflow/pipeline support– Already a lot going on in plugins!– Slice up build histories– Shared context between related builds
• Modeling external systems– Databases, test environments, …– Remembering who’s using what– Remember what was done to them
![Page 34: Jenkins State of union 2013](https://reader035.vdocument.in/reader035/viewer/2022062513/554f41e6b4c905cd048b54b1/html5/thumbnails/34.jpg)
Jenkins User Conference Shefayim, Jun 2013 #jenkinsci
Make Simple Things Simpler
• More spying, less asking– E.g., “where are unit test reports?”
• Simpler project creation ala BuildHive– Only needs to be 70% accurate– Sharable archetypes, repository sniffing
• Encourage sharing of recipes
![Page 35: Jenkins State of union 2013](https://reader035.vdocument.in/reader035/viewer/2022062513/554f41e6b4c905cd048b54b1/html5/thumbnails/35.jpg)
Jenkins User Conference Shefayim, Jun 2013 #jenkinsci
Help You Find Plugins You Need
• Revisit bundled plugins– Why Git plugin isn’t in there?
• Plugin packs (aka iTunes playlist)– Follow the lead of the Jenkins PHP website
• Recommendation based on what you have• Reviews, feedbacks
![Page 36: Jenkins State of union 2013](https://reader035.vdocument.in/reader035/viewer/2022062513/554f41e6b4c905cd048b54b1/html5/thumbnails/36.jpg)
Jenkins User Conference Shefayim, Jun 2013 #jenkinsci
More UI Improvements
• FOSDEM-driven on-going effort
• TODO in this round:– Hunt on stale data in HTML– Performance on loading config page– More compact UI controls
![Page 37: Jenkins State of union 2013](https://reader035.vdocument.in/reader035/viewer/2022062513/554f41e6b4c905cd048b54b1/html5/thumbnails/37.jpg)
Jenkins User Conference Shefayim, Jun 2013 #jenkinsci
Improving Plugin Development Experience
• More extension points• Better Plugin Development Environment– More Wiki articles/samples?– Training course?
• Better development mode– Skeleton generation– Checking common pitfalls in the dev mode
• Tell us what you need!
![Page 38: Jenkins State of union 2013](https://reader035.vdocument.in/reader035/viewer/2022062513/554f41e6b4c905cd048b54b1/html5/thumbnails/38.jpg)
Jenkins User Conference Shefayim, Jun 2013 #jenkinsci
Conclusions
• A lot has been done, not just in code– And a lot more to come!
• Please help the project• Please enjoy the day!
![Page 39: Jenkins State of union 2013](https://reader035.vdocument.in/reader035/viewer/2022062513/554f41e6b4c905cd048b54b1/html5/thumbnails/39.jpg)
Jenkins User Conference Shefayim, Jun 2013 #jenkinsci