time to build and test results 3x faster - how we did it
DESCRIPTION
After one year of regular work on our build mechanism and Continuous Integration infrastructure, we are able to provide RCP products, along with their complete test suite, more than 3 times faster than before. And guess what, we are using less physical resources. The path to this result was quite long and sometimes tedious, but we finally managed it. I will share our experience and provide tips so you can speed up your own build. I will also mention which improvements are coming. I will explain the methods used to identify bottlenecks and which technical solutions we applied (and the ones we dropped). This talk will go through the usage and possibilities brought by Tycho, Target Platforms, Hudson/Jenkins with some of its plugins and a Maven Repository ManagerTRANSCRIPT
Time to Build and Test Results
3x faster – How We Did It
Aurélien Pupier R&D Engineer, Studio Development Leader
@apupier
©2014 Bonitasoft
Agenda
• Initial context
• Actual status
• Methodology and Technology
• What’s next?
©2013 Bonitasoft 2
Initial context
Once upon a time…
©2013 Bonitasoft 3
What?
©2013 Bonitasoft 4
What it means?
©2013 Bonitasoft 5
Global Infrastructure
©2013 Bonitasoft 6
φ Linux
SSD
VM
- Hudson slave
- RCP builds
φ Mac
- Hudson slave
φ Linux
VM
- Hudson Master
- Other Bonita
components
Execution time
©2013 Bonitasoft 7
Actual status
What do we gain?
©2013 Bonitasoft 8
3x faster!
©2013 Bonitasoft 9
Simplicity!
©2013 Bonitasoft 10
φ Linux
SSD
- Other Bonita components
- RCP builds
Methodology and Technology
Through the path
©2013 Bonitasoft 11
Workforce
• Along the way
• 1 year
• Dedicated part-time “Build engineer” at the
end
©2013 Bonitasoft 12
Tycho
• PDE build Tycho
• Easier Modularization
• Help from non-Eclipse RCP developer
– 2 20+
• Easier for Bonita Community
©2013 Bonitasoft 13
Modularization
• Easier to split
• On-demand/nightly build:
– Translations
– Patched plugins
– Generated code
– Installers
©2013 Bonitasoft 14
Mirrored Target Platform
• Avoid network latency
• Remove external dependencies
– LTS
– Stay the master
©2013 Bonitasoft 15
Remove virtualization
• I/O access
• Virtualization killed SSD performance
• Tried
– mounted partition
– Logical Unit
©2013 Bonitasoft 16
Backup
©2013 Bonitasoft 17
• Environment and CI config stored on
– Less down time
– Easier to modify environment
Jenkins Scriptler plugin
• Allow “complex” build flow
• Easier to test a subpart
• Rejected:
– build-pipeline-plugin
– join-plugin
– Downstream/Upstream
©2013 Bonitasoft 18
Don’t sleep while testing!
• bot.sleep() bot.waitUntil()
• Gain:
– Stability
– Speed
©2013 Bonitasoft 19
Genisoimage
• Generate .dmg
©2013 Bonitasoft 20
φ Mac
- Hudson slave
The key of success
©2013 Bonitasoft 21
The key of success
©2013 Bonitasoft 22
• Fail Fast
– Easy roundtrip
– Subpart testing
What’s next?
Harder, Better, Faster
©2013 Bonitasoft 23
Others
• Tycho
– TP resolution
– Young project
• Test split
– Slow/fast
– Enough confidence on a smaller set?
• Translations
– BND use
– Avoid TP resolution
• Workflow Integration in Hudson/Jenkins
©2013 Bonitasoft 24
©2013 Bonitasoft 25
Evaluate This Session
Sign-in: www.eclipsecon.org
Select session from schedule
Evaluate:
1
2
3