![Page 1: Presentation 1 open source tools in continuous integration environment v1.0](https://reader035.vdocument.in/reader035/viewer/2022081412/5400f2e98d7f72c8628b45f4/html5/thumbnails/1.jpg)
Agille Development| Open Source Tools in CI | © 2011 by K. DJAAFAR Jasmine Conseil ; made available under the EPL v1.0
Open Source Tools in Continuous Integration Environment
March 2011Santa Clara
Karim DJAAFAR
CO of Jasmine [email protected]
![Page 2: Presentation 1 open source tools in continuous integration environment v1.0](https://reader035.vdocument.in/reader035/viewer/2022081412/5400f2e98d7f72c8628b45f4/html5/thumbnails/2.jpg)
Agille Development| Open Source Tools in CI | © 2011 by K. DJAAFAR ; made available under the EPL v1.0
Who am I ?
• A JEE Evangelist over 15 years of experience in enterprise and Web technologies
• Founder and CO of Jasmine Consulting a French and International company focused in JAVA/JEE agile development
• Author of several books around Eclipse Development• Gold Member of Eclipse Foundation, we promote JEE
Open Source incubation projects around agile development and JEE Expertise (see JasForge project)
![Page 3: Presentation 1 open source tools in continuous integration environment v1.0](https://reader035.vdocument.in/reader035/viewer/2022081412/5400f2e98d7f72c8628b45f4/html5/thumbnails/3.jpg)
Agille Development| Open Source Tools in CI | © 2011 by K. DJAAFAR ; made available under the EPL v1.0
About you
• Ant users? Ivy ?
• Maven users ?
• Continuous integration ?
![Page 4: Presentation 1 open source tools in continuous integration environment v1.0](https://reader035.vdocument.in/reader035/viewer/2022081412/5400f2e98d7f72c8628b45f4/html5/thumbnails/4.jpg)
Agille Development| Open Source Tools in CI | © 2011 by K. DJAAFAR ; made available under the EPL v1.0
Outline
• Towards a better build process using agile tools• Build Managements and repository• Continuous Integration, Principles and Tools • Workflow• Benefits• Jasforge: the unified agile tools• Demos
![Page 5: Presentation 1 open source tools in continuous integration environment v1.0](https://reader035.vdocument.in/reader035/viewer/2022081412/5400f2e98d7f72c8628b45f4/html5/thumbnails/5.jpg)
Agille Development| Open Source Tools in CI | © 2011 by K. DJAAFAR ; made available under the EPL v1.0
TOWARDS A BETTER BUILD PROCESS USING AGILE TOOLS
![Page 6: Presentation 1 open source tools in continuous integration environment v1.0](https://reader035.vdocument.in/reader035/viewer/2022081412/5400f2e98d7f72c8628b45f4/html5/thumbnails/6.jpg)
Agille Development| Open Source Tools in CI | © 2011 by K. DJAAFAR ; made available under the EPL v1.0
What Is a Build?
• A build is much more than a compile (or its dynamic language variations)
• A build may consist of the compilation, testing, inspection, and deployment—among other things
• A build acts as the process for putting source code together and verifying that the software works as a cohesive unit.
![Page 7: Presentation 1 open source tools in continuous integration environment v1.0](https://reader035.vdocument.in/reader035/viewer/2022081412/5400f2e98d7f72c8628b45f4/html5/thumbnails/7.jpg)
Agille Development| Open Source Tools in CI | © 2011 by K. DJAAFAR ; made available under the EPL v1.0
Build Tools Automation in a agile scenario
CI Build Server
Automated testing
Automated code quality
Build Tool Agile and collaborative
tools
![Page 8: Presentation 1 open source tools in continuous integration environment v1.0](https://reader035.vdocument.in/reader035/viewer/2022081412/5400f2e98d7f72c8628b45f4/html5/thumbnails/8.jpg)
Agille Development| Open Source Tools in CI | © 2011 by K. DJAAFAR ; made available under the EPL v1.0
Build Tool
• Collect input
• Process inputs
• Generate final outputs
![Page 9: Presentation 1 open source tools in continuous integration environment v1.0](https://reader035.vdocument.in/reader035/viewer/2022081412/5400f2e98d7f72c8628b45f4/html5/thumbnails/9.jpg)
Agille Development| Open Source Tools in CI | © 2011 by K. DJAAFAR ; made available under the EPL v1.0
Build Tool - Inputs
• Source code
Java, SQL
• Resources
Properties, XML
• Dependencies
Libraries, components, frameworks
![Page 10: Presentation 1 open source tools in continuous integration environment v1.0](https://reader035.vdocument.in/reader035/viewer/2022081412/5400f2e98d7f72c8628b45f4/html5/thumbnails/10.jpg)
Agille Development| Open Source Tools in CI | © 2011 by K. DJAAFAR ; made available under the EPL v1.0
Build Tool - Processing
• Compile
• Run tests
• Copy files
• Replace keywords
• Package
• Install / deploy
![Page 11: Presentation 1 open source tools in continuous integration environment v1.0](https://reader035.vdocument.in/reader035/viewer/2022081412/5400f2e98d7f72c8628b45f4/html5/thumbnails/11.jpg)
Agille Development| Open Source Tools in CI | © 2011 by K. DJAAFAR ; made available under the EPL v1.0
Build Tool - Outputs
• JAR / WAR / EAR files
• Zip files
• Reports
![Page 12: Presentation 1 open source tools in continuous integration environment v1.0](https://reader035.vdocument.in/reader035/viewer/2022081412/5400f2e98d7f72c8628b45f4/html5/thumbnails/12.jpg)
Agille Development| Open Source Tools in CI | © 2011 by K. DJAAFAR ; made available under the EPL v1.0
Ant vs Maven
Ant: Procedural1. Set path2. Compile jars3. Run unit tests4. Create reports5. Copy files6. Assemble WAR
Maven: Declarative1. Define name &
version2. Define as WAR
project3. Specify
dependencies4. Specify unit test
plugin
![Page 13: Presentation 1 open source tools in continuous integration environment v1.0](https://reader035.vdocument.in/reader035/viewer/2022081412/5400f2e98d7f72c8628b45f4/html5/thumbnails/13.jpg)
Agille Development| Open Source Tools in CI | © 2011 by K. DJAAFAR ; made available under the EPL v1.0
Ant Multi-Project Builds
• Option 1 – Master build.xml
• Option 2 – Cascading build.xml's
![Page 14: Presentation 1 open source tools in continuous integration environment v1.0](https://reader035.vdocument.in/reader035/viewer/2022081412/5400f2e98d7f72c8628b45f4/html5/thumbnails/14.jpg)
Agille Development| Open Source Tools in CI | © 2011 by K. DJAAFAR ; made available under the EPL v1.0
Maven
• Project and artifact-based build platform
• Uses repositories to manage artifacts and third party
libraries
• Customized by modifying the behavior of the life-cycle
via plugins.
![Page 15: Presentation 1 open source tools in continuous integration environment v1.0](https://reader035.vdocument.in/reader035/viewer/2022081412/5400f2e98d7f72c8628b45f4/html5/thumbnails/15.jpg)
Agille Development| Open Source Tools in CI | © 2011 by K. DJAAFAR ; made available under the EPL v1.0
Maven and the Development Build Process
• Maven the source build management tool for enterprise Java projects
• Use declarative approche rather then the task-based approach used in Ant or in traditional Make files or shell scripts
• Promotes the use of standard directory structures and a well-defined build lifecycle• Support quality metric reporting and documentation generation tool throw the use of plugins
![Page 16: Presentation 1 open source tools in continuous integration environment v1.0](https://reader035.vdocument.in/reader035/viewer/2022081412/5400f2e98d7f72c8628b45f4/html5/thumbnails/16.jpg)
Agille Development| Open Source Tools in CI | © 2011 by K. DJAAFAR ; made available under the EPL v1.0
Phases: build life cycle
• The standard goals for building:
compile & test - compile and test the code
package – build and test the artifact
install – package the artifact and install to local repository
deploy – package the artifact and copy to the remote
repository (release time)
• Release management
![Page 17: Presentation 1 open source tools in continuous integration environment v1.0](https://reader035.vdocument.in/reader035/viewer/2022081412/5400f2e98d7f72c8628b45f4/html5/thumbnails/17.jpg)
Agille Development| Open Source Tools in CI | © 2011 by K. DJAAFAR ; made available under the EPL v1.0
Reports
• Centralize your project information and documentation for developers and their managers
• Integrate project information such as source code, inspection reports, etc…
Reports are used to show the state of the project
Can be integrated with little to no configuration on an existing
project
Test coverage (eg. Clover), test results, code style (eg
Checkstyle, PMD), and many more
![Page 18: Presentation 1 open source tools in continuous integration environment v1.0](https://reader035.vdocument.in/reader035/viewer/2022081412/5400f2e98d7f72c8628b45f4/html5/thumbnails/18.jpg)
Agille Development| Open Source Tools in CI | © 2011 by K. DJAAFAR ; made available under the EPL v1.0
Maven main strengh
• A Standard way of identifying artifacts Each Maven artifact has a unique identifier, or “coordinates All project interactions go through the repository No more relative paths! Declarative Dependency Management Easy to share between team
<repositories> <repository> <id>central</id> <name>Maven Repository Switchboard</name> <layout>default</layout> <url>http://repo1.maven.org/maven2</url> <snapshots> <enabled>false</enabled> </snapshots> </repository> </repositories>
![Page 19: Presentation 1 open source tools in continuous integration environment v1.0](https://reader035.vdocument.in/reader035/viewer/2022081412/5400f2e98d7f72c8628b45f4/html5/thumbnails/19.jpg)
Agille Development| Open Source Tools in CI | © 2011 by K. DJAAFAR ; made available under the EPL v1.0
Maven in the Eclipse Ecosystem• Eclipse IAM (
http://www.eclipse.org/iam/) Integration for Apache Maven and
direct import of Maven 2 projects continuation of Q for Eclipse Provide a rich interface over Maven Wizard for creation of new projects
using the archetype mechanism of Maven
• Using m2Eclipse plugin Setting up Maven projects can be
done in two ways: using Maven-eclipse-plugin (mvn eclipse:eclipse) or using M2eclipse plugin (see http://m2eclipse.sonatype.org/)
![Page 20: Presentation 1 open source tools in continuous integration environment v1.0](https://reader035.vdocument.in/reader035/viewer/2022081412/5400f2e98d7f72c8628b45f4/html5/thumbnails/20.jpg)
Agille Development| Open Source Tools in CI | © 2011 by K. DJAAFAR ; made available under the EPL v1.0
Automated web deployment• Automating the deployment
process using the Cargo Maven plugin
Start, stop and install application
servers
Deploy to different application
servers
Run a stand-alone instance
Or deploy to an existing server
Deploy to different Application server
Deploying to an embedded Jetty instance
![Page 21: Presentation 1 open source tools in continuous integration environment v1.0](https://reader035.vdocument.in/reader035/viewer/2022081412/5400f2e98d7f72c8628b45f4/html5/thumbnails/21.jpg)
Agille Development| Open Source Tools in CI | © 2011 by K. DJAAFAR ; made available under the EPL v1.0
BUILD MANAGEMENT AND REPOSITORY
![Page 22: Presentation 1 open source tools in continuous integration environment v1.0](https://reader035.vdocument.in/reader035/viewer/2022081412/5400f2e98d7f72c8628b45f4/html5/thumbnails/22.jpg)
Agille Development| Open Source Tools in CI | © 2011 by K. DJAAFAR ; made available under the EPL v1.0
Artifact Repository Manager
• Artifact Search• Remote Proxying Cache• Artifact Upload• Graphical Administration• Virtual Repositories• RSS Feeds• Role-based Security• Integrity Reports• Maintenance
![Page 23: Presentation 1 open source tools in continuous integration environment v1.0](https://reader035.vdocument.in/reader035/viewer/2022081412/5400f2e98d7f72c8628b45f4/html5/thumbnails/23.jpg)
Agille Development| Open Source Tools in CI | © 2011 by K. DJAAFAR ; made available under the EPL v1.0
Archiva
• Archiva http://archiva.apache.org
the build artifact repository for use
with build tools such as Maven
and Ant
• Proxy and cache
• Search and repository browser
• Reporting features
• User interface
• Uploading/Deleting artefacts
![Page 24: Presentation 1 open source tools in continuous integration environment v1.0](https://reader035.vdocument.in/reader035/viewer/2022081412/5400f2e98d7f72c8628b45f4/html5/thumbnails/24.jpg)
Agille Development| Open Source Tools in CI | © 2011 by K. DJAAFAR ; made available under the EPL v1.0
CONTINUOUS INTEGRATIONPRINCIPLES AND TOOLS
![Page 25: Presentation 1 open source tools in continuous integration environment v1.0](https://reader035.vdocument.in/reader035/viewer/2022081412/5400f2e98d7f72c8628b45f4/html5/thumbnails/25.jpg)
Agille Development| Open Source Tools in CI | © 2011 by K. DJAAFAR ; made available under the EPL v1.0
Continuous Integration (CI) scenario
• A CI scenario starts with the developer committing source code to the repository.
• On a typical project, people in many project roles may commit changes that trigger a CI cycle:
Developers change source code,
database administrators (DBAs) change table definitions,
build and deployment teams change configuration files,
interface teams change DTD/XSD specifications,
and so on…
![Page 26: Presentation 1 open source tools in continuous integration environment v1.0](https://reader035.vdocument.in/reader035/viewer/2022081412/5400f2e98d7f72c8628b45f4/html5/thumbnails/26.jpg)
Agille Development| Open Source Tools in CI | © 2011 by K. DJAAFAR ; made available under the EPL v1.0
Why do we need CI ?
• Replace big (and long) integration phases with small and frequent ones
Ideally instantaneous Think of « incremental compilation » in Eclipse
• Minimize integration effort Keep the development process running …
• CI was introduced with XP principles and by Martin Fowler
![Page 27: Presentation 1 open source tools in continuous integration environment v1.0](https://reader035.vdocument.in/reader035/viewer/2022081412/5400f2e98d7f72c8628b45f4/html5/thumbnails/27.jpg)
Agille Development| Open Source Tools in CI | © 2011 by K. DJAAFAR ; made available under the EPL v1.0
Workflow1. Checkout from SCM
2. Code a new feature
3. Run automated build on my machine Repeat 2 and 3 until test pass !
4. Merge with latest changes from SCM
Fix and rebuild until test pass
5. Commit6. Run a build on a clean machine
Immediately fix bugs and integration issues
![Page 28: Presentation 1 open source tools in continuous integration environment v1.0](https://reader035.vdocument.in/reader035/viewer/2022081412/5400f2e98d7f72c8628b45f4/html5/thumbnails/28.jpg)
Agille Development| Open Source Tools in CI | © 2011 by K. DJAAFAR ; made available under the EPL v1.0
Components of a Continuous Integration system
CI Repository CI Server Deployment Platform
Build Manager
Feedbackmecanism
1.Checkout: get all existing source from the repository
12
2.Commit: commit all local modifications in the repository
3
3.Pool: See if there is a commit in the repository
4
4.Build: Build construction in the JEE lifecycle.
5
5.Deploy: Artefact deployment in a target platform
6
6.Feedback: feedback mecanism like email notification and groupware.
![Page 29: Presentation 1 open source tools in continuous integration environment v1.0](https://reader035.vdocument.in/reader035/viewer/2022081412/5400f2e98d7f72c8628b45f4/html5/thumbnails/29.jpg)
Agille Development| Open Source Tools in CI | © 2011 by K. DJAAFAR ; made available under the EPL v1.0
Version Control Repository• Simply put, you must use a version control repository in
order to perform CI• Even if you don’t use CI, a version control repository
should be standard for your project• The purpose of a version control repository is to manage
changes to source code and other software assets (such as documentation) using a controlled access repository
• There are different types of version control systems you can use too
• We use Subversion for most of the examples in this seminar
![Page 30: Presentation 1 open source tools in continuous integration environment v1.0](https://reader035.vdocument.in/reader035/viewer/2022081412/5400f2e98d7f72c8628b45f4/html5/thumbnails/30.jpg)
Agille Development| Open Source Tools in CI | © 2011 by K. DJAAFAR ; made available under the EPL v1.0
Continuous Integration not exist without TDD !• A software development practice where
members of a team integrate their work frequently usually each person integrates at least daily (Martin Fowler)
• CI leads to significantly reduced integration problems and allow a team to develop cohesive software more rapidly and reduce risk
• To complete this definition Automated, self-testing and fast builds Make it easy for everyone to get the last
executable Automate deployment Everyone can see what’s happening
progress, statistics on code coverage tests, etc …
Reduced risks
![Page 31: Presentation 1 open source tools in continuous integration environment v1.0](https://reader035.vdocument.in/reader035/viewer/2022081412/5400f2e98d7f72c8628b45f4/html5/thumbnails/31.jpg)
Agille Development| Open Source Tools in CI | © 2011 by K. DJAAFAR ; made available under the EPL v1.0
Continuous Integration
• Major tools Hudson
Writen in java (https://hudson.dev.java.net/ ) No install required (put a war in a web container) Easy to use Powerful Default support for CVS and SVN Great support and active community
Cruise Control Continuum
Java and Ant/Maven based Open Source …
![Page 32: Presentation 1 open source tools in continuous integration environment v1.0](https://reader035.vdocument.in/reader035/viewer/2022081412/5400f2e98d7f72c8628b45f4/html5/thumbnails/32.jpg)
Agille Development| Open Source Tools in CI | © 2011 by K. DJAAFAR ; made available under the EPL v1.0
Every commit much triggers a build !
• Commit at least daily… As soon as you have completed an independent
functionnality
• A full build on another empty machine Not on your own !
• With hudson SCM polling Cron-like scheduller Trigger with http request (from maven or svn commit hook)
![Page 33: Presentation 1 open source tools in continuous integration environment v1.0](https://reader035.vdocument.in/reader035/viewer/2022081412/5400f2e98d7f72c8628b45f4/html5/thumbnails/33.jpg)
Agille Development| Open Source Tools in CI | © 2011 by K. DJAAFAR ; made available under the EPL v1.0
Practicing CI or TDD does not make you agile !• Although it is recommended, a CI server isn’t required to
perform continuous integration• You can write your own custom scripts• Moreover, you can manually run an integration build
whenever a change is applied to the repository• Using a CI server can reduce the number of custom scripts
that you would otherwise need to write• Many CI servers are freely available and open source: Cruise
Control, Hudson are the most well known • Without active engagement of the customer an organization
will NEVER be agile, regardless of the other development techniques used.
![Page 34: Presentation 1 open source tools in continuous integration environment v1.0](https://reader035.vdocument.in/reader035/viewer/2022081412/5400f2e98d7f72c8628b45f4/html5/thumbnails/34.jpg)
Agille Development| Open Source Tools in CI | © 2011 by K. DJAAFAR ; made available under the EPL v1.0
THE INTEGRATED AGILE INFRASTRUCTURE
![Page 35: Presentation 1 open source tools in continuous integration environment v1.0](https://reader035.vdocument.in/reader035/viewer/2022081412/5400f2e98d7f72c8628b45f4/html5/thumbnails/35.jpg)
Agille Development| Open Source Tools in CI | © 2011 by K. DJAAFAR ; made available under the EPL v1.0
Managing all the agile tools is a complex process
Today, managing and taking control of your software process is a major challenge for maintaining is capacity to be innovative and competitive
Ho can we manage efficiently all these toolls in an unified way ?
We must manage the quality of our software development en meet the requirements and delays in a predictive mode
This is the aim of JasForge© Project (http://www.jasforge.com) : Taking the control of your development process using an “agile “ approache
![Page 36: Presentation 1 open source tools in continuous integration environment v1.0](https://reader035.vdocument.in/reader035/viewer/2022081412/5400f2e98d7f72c8628b45f4/html5/thumbnails/36.jpg)
Agille Development| Open Source Tools in CI | © 2011 by K. DJAAFAR ; made available under the EPL v1.0
Some “little” screen casts and demos
Managing your projects
Managing users and roles
Managing you servers