15.06.07

Java development infrastructure at FMI

Pekka Rantala

FMI

15.06.07

ComponentsFour Open Source software components:

• Eclipse IDE for developers

• Subversion version control (used by developers through Eclipse's Subclipse plugin)

• Maven 2 for builds (mostly hidden from developers)

• Plus Maven proxy for querying and caching internet and local repos

• Continuum for continuous integration building

15.06.07

General architecture

Developer

Maven

Eclipse

Infra server

Subversion

Maven

Continuum

Local Maven repo

Subclipse

Maven proxy

Internet

Maven repo 1

Maven repo n

Local Maven repo

FMI App metadb

15.06.07

Day-to-day usage: developer1) Developer commits code into SVN

2) Continuum detects a change in the SVN repo and

3) uses Maven to build the project; as a side effect

4) Maven installs the newly built component in the main FMI component repo, thus making the changed artifact visible for all other projects and webstart apps.

The developer only sees the infra via Eclipse: Subclipse is used to access SVN, and Maven is used locally as an “external tool”. Developer receives emails from Continuum if (s)he breaks the build (and subsequently another mail when the build is fixed)

15.06.07

Motivation: the Great Refactoring• A large-ish workstation project (on average five

developers, maybe about a thousand java classes) was built as a monolithic Ant build → builds were taking long and were very often broken

• The project was split up to consist of manageable chunks of code (about forty projects remained)

• Building 40 projects as separate Ant builds is not feasible

• Maven-Continuum-Subversion interplay makes it fairly flexible to build components (“artifacts”) independently of each other

• Nowadays we have a little over 100 projects including “common” reusable components easily incorporable into any in-house project

15.06.07

Benefits• Maven strong points:

• Easy management of dependencies (in-house built libraries and third party libraries)

• Powerful extension mechanism for inheriting dependencies and other configuration from a project to another

• Build lifecycle is easily extended by creating one's own plugins (mojos)

• Various reports (javadocs, source Xrefs etc.) are easily generated automatically

15.06.07

Benefits• For the developer

• ideally, the developer needs not be bothered about a project's build internals; declaring dependencies is usually sufficient

• Building artifacts continuously and enforcing a “commit early, commit often” policy greatly reduces the risk of components running out of sync with each other

15.06.07

Artifact versioning• An artifact is the end product of a build: a library,

component, XML file or such. Conventions state that one project should produce one artifact

• An artifact under development is labeled as a SNAPSHOT and is by definition “unstable” (API is allowed to change at will); SNAPSHOTs are used “at own risk”.

• 0.7-SNAPSHOT becomes 0.7 during a disciplined release process (other projects may declare their dependencies to which ever version they like). Releases are final; contract determines that a release of an artifact will not change

15.06.07

Releases1) When release criteria (API documentation, release notes

in wiki, Unit tests run, ...) is met:

2) Release Manager tells Maven to prepare and perform the release. Maven:

3) checks out all code from SVN,

4) builds the project (running Unit tests),

5) tags it under SVN as a release

6) and upgrades the project's version number.

7) The new release and the new SNAPSHOT are available to other projects via the Maven proxy

15.06.07

FMI utils: Application metadb• Stores metadata about FMI-built artifacts (available

versions, bindings between FMI Service API components etc.) Metadata is communicated to the DB by way of an FMI-built Maven plugin during build

• Signs and stores FMI-built SNAPSHOT artifacts during build-time and other artifacts per request (implemented as a Servlet wrapper around Sun's jarsigner utility)

• Based on the first two points: offers dynamic composition of .jnlp files (Java Webstart) resolving all dependencies and checking the signing status of needed components (for instance the SmartMet II workstation is primarily started via javaws)

15.06.07

FMI utils: POMBrowser• Resolves projects'

dependencies and dependency paths, evaluating formal release readiness.

• Draws a dependency graph of the current artifact and a dependency element for use in other projects

15.06.07

FMI utils: Repository browser• Searches the local Maven

repo for FMI projects

• Composes URLs to Maven built sites, Subversion views, APIs , wiki and the POMBrowser

• Offers rudimentary quality checks of project settings

15.06.07

FMI utils: Project creator• Creates directories,

Project Object Model file pom.xml and other skeleton files

• Imports the new project into SVN

• Notifies Continuum about a new project

15.06.07

To come: Open sourcing / collaborations• It is planned that FMI may open for public access:

• a part of the Subversion repository

• similarly, a public Maven repository for FMI built artifacts in the public domain.

• Common data and metadata handling components are good candidates for open sourcing

• The tools are in wide use; Open sourcing issues should be easily solved.

• If the infra (including FMI utils) gathers enough interest, we may feel enough pressure to “productise” and release the utils and documentation publically.

15.06.07

Resources• Eclipse: http://www.eclipse.org/

• Subversion: http://subversion.tigris.org/

• Maven: http://maven.apache.org/

• http://mvnrepository.com/ for searching publically available artifacts

• Continuum: http://maven.apache.org/continuum/