jaxmag-2010-03

Issue November 2010 | presented by

www.jaxenter.com

Java, Enterprise Architecture, Agile and EclipseBuild tools are essential for todays busy developer, allowing the scripting of a variety of repetitive everyday tasks. Build tools have come a long way from when they were used to call compilers and linkers from inside a build script, and developers can now choose from a wide variety of open source and proprietary solutions. In this issue of JAXmag, we take a look at some of the most widely-used build tools, including the recently-released Maven 3 and Ant, and some new tools that use programming languages such as Groovy and Ruby.

Build ToolsApache Maven Turns 3.0Maven Goes Parallel

#3

Web Development With Apache AntRecipes for Cooking With Ant

Build meets GroovyGradle your Build like a Baby

A Brief Introduction ToGant and Pulse

Editorial

Build Automation Moves Into The 21st Century

Build automation has come a long way, as the make scripting language and, later, GNU Make, heralded in a new era of build tools that didn't just call compilers and linkers from inside a build script, but automated the calls to the compilers and linkers with extra functionality for pre and post actions. Today, the idea of build automation typically includes managing both the pre and post compile and link activities, as well as the compile and link activities themselves. With the wide range of open source and proprietary solutions currently available, developers can select tools that perform tasks such as streamlining compile and linker calls, support remote agent processing for distributed builds, or are written in socalled 'real' programming languages. The latter is a recent development, as some developers move away from using XML to define build logic and towards programming languages, such as Groovy and Ruby. The driving argument, has been that code written in a more familiar programming language, is easier to understand and more concise. In this issue of JAXmag, we take a look at some of the most widely-recognised build tools. Maven 3.0 was the release that landed with the biggest splash, and 'Apache Maven 2: Effective Implementation' co-author Brett Porter takes a look at what's new in this release, by placing it in the broader Maven timeline. We take a look at the other side of the frequent Maven vs. Ant flame-war, with some Ant recipes from 'Apache Ant Recipes for Web Developers' author, Jim Fuller. If you want to get your hands dirty with Apache Ant, then this is the article to get you coding! As already mentioned, the world of the build tool is moving in new and exciting directions, and we wind up this issue by offering an introductory article to help you get acquainted with these new tools including Gradle, Pulse and Gant, which uses Groovy instead of XML. Despite the flame-wars that break out in the community about which build tool is 'best' (most often, it's Ant and Maven fans

at loggerheads) ultimately the community can only benefit from the diversity of build solutions, regardless of whether they prefer Ant, or Maven, or Buildr, or another tool. No one really wants to write build scripts, after all. So, read on to get coding with Ant, learn the ins-and-outs of the new Maven release, and expand your build tool horizons!

Jessica Thornsby Editor JAXenter.com and JAXmag

Jessica Thornsby

Sebastian Meyen

Claudia Frhling

Editorial Team JAXmag

Apache Maven

How has Maven arrived at version 3.0?

Maven Turns 3.0This month, the Apache Maven project [1] has shipped the Maven 3.0 release, just short of the 5 year anniversary of its previous major release. In that time, Maven has become a leading choice for building software on Java and other platforms, changing the way developers think about builds, and establishing a growing ecosystem of tools that automate more of the development process.

By Brett PorterI first encountered Maven at the very beginning of 2003. I was working together with a number of Java development teams, and wed gone through 3 different build systems in the preceding years - from Make, to a series of home-grown shell scripts, and finally to a number of Ant scripts. We were rapidly starting new development projects, establishing shared libraries, setting up shared development infrastructure such as issue tracking and continuous integration, and attempting to share development practices. Keeping consistency between projects was becoming increasingly difficult. It was in the search for a better way to manage common Ant script fragments and each projects dependencies that I found an early beta of Maven 1.0. In those days, it was very rough around the edges and painfully slow, but it was still very powerful and showed a lot of promise. The solutions to our problems were quickly within reach. New projects and libraries were fast and easy to set up, and completely standardized. By pushing functionality into plugins, there was no duplication, and build system improvements could be rolled out to all projects simultaneously. We were able to generate a CruiseControl configuration file from any project to have it quickly up and running in the continuous integration environment, and all of the dependencies were moved into the common repository, retrieved through Mavens dependency mechanism. We could generate IDE project files to have a project set up on a new developers machine instantly, and could codify aspects of development practices to share across different teams. Further enhancements to the build processes were then made possible through Mavens standardization. For examp-

le, we established release and source control automation a new plugin, to update and tag projects before pushing them to the remote Maven repository to be picked up by the operations team for later deployment.

Maven Past, Maven PresentMaven has certainly come a long way in the 7 years since, but the fundamental principles that are at the core of its success have remained unchanged. All of the above have become the expected basis for a development environment, rather than something to stretch for. Through the standardization of build and development practices, Maven reduces the cost of everything it touches that might ever need to be done twice. Projects have a low set up and maintenance cost. Ramping up new developers onto a project comes much more cheaply. New development tools can be integrated consistently across a number of teams, with minimal configuration. Sharing code and development practices become easier and more natural. Best practices can be built into projects from the beginning and maintained. Additional development infrastructure can be established and reused easily by multiple members of a team, or multiple teams within an organization. In this article, well look at the influence Maven has had, and where it is heading in the future.

Shaping Development InfrastructureThough Maven runs on a single machine at a time, often through direct interaction with individual developers, it is clear that its strengths lie in team environments. This isnt surprising, as developers rarely work in a vacuum and they need a way to effectively collaborate with others on the same codebase. Common infrastructure for project teams is nothing new, but what we have seen in

www.JAXenter.com | November 2010

3

Apache Maven

conjunction with Maven adoption is that it has become a path towards wider adoption of standardized build infrastructure. No longer do teams acquire a handful of their own resources, but can more readily share the same infrastructure across teams. Mavens natural fit with continuous integration and source control servers make it easier to set up new projects. The use of artifact repositories have given rise to a new piece of infrastructure, the repository manager. Servers such as Archiva, Artifactory or Nexus can be used to manage all aspects of the artifact repository, beyond being a place to easily deploy builds. This includes features such as search and indexing, acquisition of third-party artifacts, clean up operations such as removing old development builds, access permissions, trend analysis, integrity issue reporting, and additional release management. Such a repository can commonly act as the distribution point for other consumers of build artifacts, such as a QA team or operations staff that will run a new version of an application. These are sometimes even shared externally for all Maven users to access the redistributable artifacts of a project. This infrastructure, along with Mavens ease of automating other testing and analysis tools, is likely to see it at the center of Java projects looking to learn from the DevOps trend. Combined, all of these tools work together to provide teams with an instant shared development environment - further extending Mavens benefits including reuse, project set up and maintenance, and increased consistency and predictability of builds.

taining complete backwards compatibility (with a few clearly documented exceptions).

What to ExpectIf youre switching from a recent release of Maven 2, there should be very few surprises. You will likely notice that youre running the new version due to the slightly different output format, and some different error messages that are discussed below. There should be fewer general annoyances at build time than in previous versions. In terms of the output of the build and whether it will succeed, hopefully you wont notice any difference at all! The main new feature is performance and memory usage, which are enhanced through: optimized performance and caching in the Maven core and APIs reduced network usage where updates are not likely to be needed the availability of optional parallel module builds to make use of multiple CPUs/cores Everyone will have a different experience on the gains in each of these areas, from mild to significant improvements - but given the frequency with which Maven builds are run both by developers and continuous integration servers, performance may be reason to upgrade. The remaining improvements in Maven 3.0 are to do with user experience and avoiding certain classes of bugs, including: improved POM validation and inspections improved error handling and reporting better tracking of repository metadata in the local repository, to ensure build results match the repositories declared in the project and to handle the improved caching classloader separation for plugins improved reactor project management, no longer requiring a full mvn install command to build a project in some situations, and more consistency when building sub-modules either independently or as part of a reactor

Establishing a Pattern of StandardizationMaven hasnt always enjoyed popularity with some developers. In fact, it tends to polarize opinions between those that cant live without it, and those that cant stand to work with it. The benefits of standardization it brings, which in places are enforced rigidly, come at the cost of lost flexibility. However, even the alternative tools such as Apache Buildr, Apache Ivy, and Gradle that have emerged during its era, build on Mavens successful concepts. Many of the same ideas have been applied in different ways, and conventions such as the standard directory structure are reused. Tools providing dependency management inevitably interoperate with Maven repositories, its metadata and dependency resolution mechanism. In particular they rely on the Maven central repository as a source for existing components. They go on to reuse the same tools such as existing repository managers. Despite the naysayers, Maven adoption has grown rapidly and shows no signs of retreat.

What not to ExpectFirst, dont expect the hassles of the Maven 1.0 to Maven 2.0 migration all over again! As is probably obvious from the previous description, you should also not expect significant new features or changes in behavior in this release. Some widely reported features that had at times been under consideration are not included, such as: POM mix-ins A revised POM format, perhaps using attributes Ability to write POMs in Groovy or other languages (there is a separate project [3] for this) An OSGi based runtime Significant changes or new features with regard to dependency resolution

Maven 3.0The road from Maven 2.0 to 3.0 has been a surprisingly long one for a number of reasons, but it is partly because Maven is a victim of its own success - with so many projects depending on it, and a large number of custom plugins written for it - the need to retain compatibility even through significant internal changes became paramount. However it has now arrived [2], and so far has achieved its mission - to improve the internals and test suite to facilitate future changes, while improving user experience and main-


4

Apache Maven

How to Prepare

As compatible as this release strives to be, there are still some potential bumps in the road that it is worth preparing for before upgrading. These and more can be seen in the Maven 3 Compatibility Notes [4] as well.

[ERROR] Failed to execute goal on project archiva: Could not resolve dependencies for project ...: Failure to find esymock:easymockclassextension:jar:1.2 ...

POM ValidationYou should be watching the start of your builds carefully the first few times you run Maven 3 to see if it has picked up on any improvements you need to make. In some cases the validation may be serious enough that the build wont start. For example, if you havent specified a plugin version (in the same way as the enforcer rule that can be used in Maven 2), youd see:[WARNING] 'build.plugins.plugin.version' for org.codehaus.mojo:exec-maven-plugin is missing. @ line 145, column 15

was cached in the local repository, resolution will not be reattempted until the update interval of archiva-internal has elapsed or updates are forced -> [Help 1] To improve build performance, Maven 3 now strictly follows the update policy of a repository as set in the POM or settings file, including when an error occurs. To force an update, you can run Maven with the -U option:mvn -U clean install

Output ChangesAs noted earlier, there have been several changes to the output format in Maven 3 that are worth familiarizing yourself with. The first that you are likely to see is that goal execution has changed its format to provide more information. In Maven 2, youd have seen the following:[remote-resources:process]

Other common problems are duplicated dependencies and deprecated expression notations. Even if these are only warnings, it is worth addressing them straight away since they are generally simple to fix, and of benefit to the build in the long run.

In Maven 3, the output is more detailed:[INFO] --- maven-remote-resources-plugin:1.1:process (default) @ maven-parent ---

Site PluginThrough the technical changes to the plugin manager, it has now become possible to separate the reporting engine from the core Maven distribution. A result of this is that while the existing Site plugin may work to some extent, two features do not work: inherited site descriptors, as the site:attach-descriptor is no longer configured by default and needs to be added to parent POMs where appropriate reports will not be generated and included in the navigation To regain reporting functionality, a beta release of the site plugin for Maven 3 is already available. This can be used instead of the previous version, though note that if you want your build to continue to work on Maven 2.2.1 and below as well, you should include the different versions in a profile.

This is particularly helpful for understanding the output of a parallel build since the output during the build is not necessarily in sequential order. Probably the most notable change in output is to the error messages. If something goes wrong during the build, it will now always be reported as the last thing in the build output, and provide a link to a relevant wiki page for the exception that occurred. For example:[ERROR] No plugin found for prefix 'foo' in the current project and in the plugin groups [org.apache.maven.plugins, org.codehaus.mojo] ... -> [Help 1] ... [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ NoPluginFoundForPrefixException

Non-HTTP RepositoriesEven though repository managers are growing in use and provide a simple HTTP interface to deploy artifacts to, its still common to see projects that need to deploy to repositories or sites using scp:, ftp:, or other protocols. In Maven 3, only file:, http: and https: are supported by default, so projects needing the other protocols will need to add the required Wagon extension into their POM.

Finally, for those building a multi-module project, the reactor summary is now presented more consistently if an error occurs. Again, this can be helpful when the parallel build is in use. Any module that happens not to be built due to a previous error will be shown as skipped:[INFO] ----------------------------------[INFO] Reactor Summary: [INFO] [INFO] Apache Maven 3.x . FAILURE [0.394s] [INFO] Maven Model ...... SKIPPED

Metadata UpdatesIf you have become used to encountering remote repository problems in Maven 2 and correcting them and then running the build again, under Maven 3 you might encounter a message like the following:

The downside to this is that in some cases, the important output (such as test results) can scroll off the screen due to a large number of modules being present that need to be summarized.


5

Apache Maven

These output changes can take some getting used to, but in the long run are worthwhile for the greater level of detail.

gins to ensure they are upgraded to thread-safe versions and marked accordingly.

Using Parallel BuildsWhile the parallel builds feature is still technically labeled experimental in the Maven 3.0 release, it has become stable enough to use on day-to-day builds and to evaluate whether the performance gains warrant enabling it more regularly. It is very straightforward to use with the new -T argument. This argument specifies the number of threads to use, and if greater than 1 it will enable the parallel build mode of operation:mvn -T 4 clean install

Technical ChangesThe Maven 3.0 release is more notable for the raft of technical changes made under the hood, even though by design they may not directly affect end users. They will, however, be of some interest to those writing Maven plugins or seeking to embed Maven technologies in other applications.

Google GuiceMaven 2 was based on the Plexus container, which was developed with features specifically designed to achieve some of Mavens needs. In the years since it was originally released, other dependency injection frameworks have risen to prominence. A late addition to the Maven 3 betas was a switch to a compatibility layer that uses Google Guice as the dependency injection framework, but allows existing Plexus descriptors and annotations to work as before. In future releases, it can be expected that Maven and its plugins will move towards standard JSR-330 annotations for dependency injection.

There are two ways that you can specify the number of threads: either as an integer as above, or as a multiple of the number of CPU cores the current machine has (such as -T 1C or -T 1.5C). It is also possible to append W to the argument to enable weave mode. This is theoretically more performant than the other method of parallelization, however is not stable enough for general production use in this release. It may take some experimentation to find the right settings for the particular properties of your hardware and project. While the default is for 1 thread per CPU core, you may still get better performance by exceeding that. Builds are also bound by other factors, in particular disk operations, that may limit the benefits of parallelization. The structure of the build will have a significant impact as well. Tightly intertwined dependency trees will not be able to be effectively parallelized. Large, single operations will often dwarf the rest of the build. This might be a large compile or packaging operation, but is also common when some test sets are much larger or longer than others. For the best results, spread work out as much as possible and reduce the coupling between modules. As it happens, this is another instance where Maven encourages best practices! If you still have large test sets that cannot be further broken up, you might consider using JUnit or TestNGs parallel test execution capability as well (which is also available via Maven 2). There are still some catches to running build threads in parallel. As mentioned previously, the output ordering may not necessarily be sequential. If you need to do any mid-build output forensics, it would be best to run it in a single thread. If a build is using external resources that might not handle being accessed concurrently, this will also need to be adjusted. The classic example is multiple modules starting a server on a single specified port for testing. In such cases you may need to dynamically allocate the port instead, which can be done using the build helper plugin [5]. Aside from being required for parallelization, this also helps increase build portability. Finally, plugins need to be thread-safe. The latest versions of the key Maven plugins have been updated to ensure this is the case and marked with the new @threadSafe annotation. If your build is using older versions, third-party plugins or if you have written your own plugins, you will receive a series of warnings during the build. It is worth reviewing these plu-

Dependency Resolution and AetherAnother late addition to the Maven 3 betas was to externalize Mavens dependency resolution logic. Aether is a library rewritten to reproduce the behavior of the Maven 2 dependency resolution mechanism as closely as possible, but architected to allow further customization and to be more easily embedded. This is unlikely to affect users or plugin developers, as the library is still exposed through the existing APIs under Maven and requires no changes. However, those seeking to use Mavens resolution logic in their own application will find it of interest.

Embeddable Project BuilderThe responsibility of the project builder in Maven is to read a POM and process the required interpolation of properties and inheritance of parent models to construct the final, effective POM. This is one of the most desirable features for other applications to embed to ensure that the results are identical to Maven itself. However, it has been difficult historically due to a number of implementations details - it was tightly tied to Mavens resolution mechanism for retrieving parent POMs, and it cached projects in a way impractical for long-running applications (such as repository managers). The Maven 3 implementation is provided in a new artifact, maven-model-builder, and provides a new API that avoids these issues, for instance by allowing you to provide your own model resolver.

Reusable Plugin ManagerLike other aspects of Maven in the new release, the plugin manager can also be embedded. The plugin manager is responsible for both resolving and executing Maven plugins.


6

Apache Maven

VIDEO

The first challenge in performing these changes, which has prevented them in the past, is defining a way that the POMs uploaded to a remote repository remain usable by previous versions of Maven and the larger ecosystem of other tools that use Maven repositories and may not understand the new format. Aside from the POM, the most likely change in Maven 3.1 would be an extended plugin API that more clearly sets out what the suitable method of accessing Maven APIs is. This would help to prevent the kind of tight coupling that has occurred in the past, which limited the ability for Maven to evolve internally.

A Good Time to Get InvolvedThis capability is used by the new version of the site plugin to be able to resolve and execute reporting plugins after that functionality was removed from the core application. With Maven now on a clearly defined path forward, more accessible code base, and renewed interest in new features, now is the perfect time to get involved. The project is always looking for more people to fix bugs, make improvements, improve tests and documentation, or help triage bug reports. One of the great advantages of open source development is that there often arent any special requirements - anyone can subscribe to the development mailing list, start small on the things that are important to you (as I did), and see where you end up.

API Changes and Compatibility LayerSeveral of the public APIs have changed as part of the release that will also be of interest to plugin authors. It is important to note that plugins that depend on the Maven 2 versions of libraries will mostly continue to work unchanged as the old APIs remain exposed and are mapped onto the new code. However, some APIs that were not intended to be used publicly have been hidden by a change in package. If this affects your plugins, you may need to change the way they behave. As long as a plugin intends to continue supporting Maven 2, it is best to also continue depending on the API versions of the lowest supported version of Maven. When the time eventually comes to upgrade to Maven 3 versions of the libraries, you will find that several API classes have moved into the maven-compat artifact instead of where they previously resided. Rather than changing the plugin to depend on this artifact, it would be best to move over to the newer APIs defined in the other Maven 3 libraries.

Looking ahead: Maven 3.1Maven 3.0 may not contain the features that many were expecting to see, but it was intended as a starting point rather than an ending. Some plans have already emerged for the new features that may appear in the next release - again following in the pattern of backwards compatibility put in place for this release. First, an appropriate disclaimer: these features are by no means set in stone, nor has any timeframe been set for Maven 3.1! The discussion to date has emphasized making long awaited changes to the POM. This will most likely include the addition of new elements, and the ability to use mix-ins to reduce duplication in scenarios where inheritance is not appropriate. More significant changes to the format may also be possible.

Brett Porter has been a member of the Apache Maven project for the last 7 years, and is a co-author of the book Apache Maven 2: Effective Implementation. He works as the vice president of product development at MaestroDev [6], directing development on Maestro 3, which applies the key concepts of Maven to the wider scope of automating DevOps tasks on an included private cloud environment.

Links[1] http://maven.apache.org/ [2] http://maven.apache.org/docs/3.0/release-notes.html [3] http://polyglot.sonatype.org/ [4] https://cwiki.apache.org/confluence/display/MAVEN/Maven+3. x+Compatibility+Notes [5] http://mojo.codehaus.org/build-helper-maven-plugin/ reserve-network-port-mojo.html [6] http://www.maestrodev.com/


7

Apache Ant

How to apply Apache Ant to everyday development tasks

Web Development With Apache AntMost Java developers will be aware of Apache Ant, the defacto utility for building and deploying Java software. Apache Ant also finds itself being applied to doing a lot more than just 'building' Java software. This article will provide a few examples of how to apply Apache Ant throughout all phases of Web Development.

By Jim FullerMost of the material for this article is drawn from a modest little book I just wrote and published called 'Apache Ant Recipes for Web Developers' (http://www.antrecipes.com). Before we start showing some of the 'recipes' from the book, let's remind ourselves of some of the reasons why Ant is a compelling software utility: Cross Platform: Solutions developed in Ant, to a certain extent, will run wherever Ant will run. Easy to understand: There are many Java developers who dislike the usage of XML (too verbose, etc) as the scripting format for Apache Ant; I have found that Apache Ant's declarative XML markup makes it highly readable by developers 'outside' of the Java world. A high level of comprehension seems to translate into a willingness by Web Developers to run such scripts. Integration: Ants extensible suite of Tasks is an impressive software architectural feat e.g. it's very hard to balance off extensibility and core functionality but Apache Ant does this very well. With this extensibility Apache Ant integrates with most of the important Java technologies providing a uniform faade. The ease with which you can invoke from Ant the multiplicity of API's can make using Ant as a prototyping tool very valuable indeed. Large User Base: Ant has been around for a while and is used in lots of popular Open Source projects. The knock on effect of this wide spread adoption is support of Ant in most

development environments, IDE's and continuous integration environments. In the book, I created around 80 recipes, each of which contain a standalone solution to some common problem encountered when developing for the Web, grouped into Text, XML, Database, and Web categories. I have listed a few of the recipes from each category below.

Text RecipesLearn how to find and replace text Substitute tokens found in text using Ant native filter functionality Create reusable sets of filters that perform token replacement Create sophisticated find and replace pipelines Find and replace text using property expansion Transform a set of source text files into a range of language variants

Database RecipesManage import of csv delimited data into MySQL Use the result of a SQL query as properties within your Ant build Execute SQL statements in Ant


8

Apache Ant

XML RecipesLoad external XML into Ant Properties Simple XSLT Processing with Ant Generate documentation for your XSLT using XSLTDoc Validate XML using Ants built in facilities Use Jing to perform all major schema validation Create sophisticated documentation using Docbook XML

Web RecipesCheck that a site is live and accessible Perform a HTTP Get with Ant Perform a HTTP Post Perform a HTTP Put Clean up X/HTML with TagSoup Compress a set of HTML files Optimising css with yui.compressor Measuring website performance with Apache Benchmark I will present an example recipe from each category. Note that you can access code examples for the recipes at http:// www.antrecipes.com.

consistent set of attributes being progressively added to all Tasks by their authors. Wouldnt it be easier to leave Task attributes alone and implement some common mechanism with which to implement piping? Enter Ant which the reader should imagine as an ordered chain of filterreaders (similar to ) each of which perform a process with an input and hand over the output to the next filters input. Ant defines several native filters that work with , which are in my opinion inconveniently called FilterReaders (probably mirroring their intention in Java terms). In Apache Ant, one can create their own FilterReader or choose from the built-in FilterReaders listed below. FilterReader: generic filterreader ClassConstants: outputs any found Java constants and outputs them into name=value format EscapeUnicode: convert input into Unicode format ExpandProperties: expand any property declarations into their values HeadFilter: output a number of lines at the beginning of a file TailFilter: output a number of lines at the end of a file LineContains: outputs a line which contains a certain string LineContainsRegExp: outputs a line which matches regular expression PrefixLines: prefixes each line with a defined string ReplaceTokens: similar to StripJavaComments: remove Java comments StripLineBreaks: remove line breaks StripLineComments: remove line comments TabsToSpaces: convert tabs to spaces DeleteCharacters: delete specific characters ConcatFilter: prepend or append the contents of a file with another file A element will contain an ordered list of these FilterReaders. The only drawback with FilterChains, is its inability to define a reusable FilterChain (e.g. using a reference id attribute). In the filterchain Target, I emulate previous examples, by using the Filter- Reader.

Working with TextI choose the following recipe because it's widely applicable, either in emulating bash shell scripts or when you want to perform a series of 'data munging' tasks on text Recipe: create sophisticated find and replace pipelines A pipeline, set in the context of UNIX commandline, is a sequence of one or more shell commands separated by the pipe character e.g. | . The following example shows how such a pipe may be con- structed from the command line:> tail n 100 /var/log/system.log | grep root

This command sequence will take the last 100 lines of a log file and print out just lines containing the term root. Pipes makes it possible to implement sophisticated workflows of text processing; it is one of the reasons why many developers like myself spend most of their time within a command line shell environment. Ants are not exactly like unix pipes but were created in the same spirit. Before v1.5, Apache Ant had no established facility for the piping of the output of an operation to be supplied as the input of another operation. This shortcoming limited Ants native textual processing capabilities as I couldnt perfectly emulate command line recipes that had been honed and tweaked over years. Not having a standard mechanism for passing data around also meant confusion with respect to Ants own development, that is new functionality would commonly show up as a new attribute on a Task ... leading to an in-

The embedded contains a single FilterReader in a , which doesnt quite illustrate the power of this approach. In my next recipe, I move onto an example of FilterReaders processing text and handing off their output to the next FilterReader in series.


9

Apache Ant

The performs textual substitution then strips line breaks with the filter. A useful variant of is the TokenFilter, which tokenizes an input stream into discrete line, string, or file tokens. Each token, in turn, can then be passed to other filters. I am specifically interested in the string tokenizers as one can process data on a string by string token basis using the following string filters. Here is a list of string filters native to Ant. ReplaceString: replace a substring in the string token ContainsString: output when string token contains a substring ReplaceRegex: output string token with regular expression substitution ContainsRegex: output string token which matches regular expression Trim: trim whitespace in string token IgnoreBlank: strips empty string token DeleteCharacters: delete characters from a string token For ease of use, most stringreaders can be used directly in a filterchain, without having to explicitly declare a tokenfilter element. Tip: You can use a which will execute a script in an Apache BSF supported language, just like the Task.

You will need to supply the example script with your own database details. The above invocation of the Task should access a local mysql user table, which should be installed with your default MySQL installation. The SQL statement I intend to run against this table is contained in a nested element. The SQL Task has many options and attributes, so please refer to the Ant manual. One other item of note, is the ability to instruct the task to look for a Java jar or set of class files to provide the JDBC driver for your flavor of database. Our next example, builds on the previous Ant script, showing how I can have a series of transactions, supplied from external files containing SQL statements, as well as mixing in SQL statements embedded in the task itself, as before. The use of the element can be used to specify embedded SQL statements or load in SQL statements from an external file. select * from myTable;

Interacting with DatabasesWorking directly with databases is not Apache Ant's forte but does exist as a possibility and somthing that may need to be done from time to time. I choose the following recipe because it illustrates the built in SQL Task that Apache Ant ships with. Recipe: Execute SQL statements in Ant Ant comes with a standard SQL task from which you may execute SQL statements embedded within your build scripts or provide source files that contain SQL statements to be processed. You must have a JDBC connector installed and available for to work properly. I will be using com.mysql.jdbc. driver from MySQL's official website for use with a MySQL database for our examples shown here. The proceeding example shows how to invoke the task with database specific details such as a username and password to access database, driver class name, if I would like to save the output to a file, etc... select * from user;

This example tries to connect to a Microsoft SQL server and then attempts to execute SQL statements from a source file named somefile.sql, then it performs the embedded SELECT statement. Finally, another files SQL statements would be processed. The statement does support the concept of Ant , though it is recommended to use the element, which guarantees the order of processing with each set of SQL statements.

XML TechnologiesThe book has a wide range of recipes working with XML. I decided to show a more exotic example, using my favourite validator for validating XML, schematron. Recipe: validate using Schematron Schematron is another popular schema technology, details can be found at the website http://www.schematron.com/. There exists a 3rd party schematron Ant task, which will invoke schematron processor against your xml. The example1 Target does this as follows.

Just a matter of supplying the schema with the schema attribute and using to select xml documents to validate.


10

Apache Ant

Running the example1 Target will output this from the console.Buildfile: /4_validate_xml/schematron/build.xml 66 example1: [schematron] [schematron] version outside [schematron] version outside [schematron] version outside [schematron] Compiler warnings: jar:file:/lib/ant-schematron.jar!/iso_schematron_skeleton.xsl: line of element. jar:file:/lib/ant-schematron.jar!/iso_schematron_skeleton.xsl: line of element. jar:file:/lib/ant-schematron.jar!/iso_schematron_skeleton.xsl: line of element. 376: Attribute 399: Attribute 414: Attribute 3 file(s) have been successfully validated. BUILD SUCCESSFUL Total time: 1 second

flags=g replace= />

XML RecipesA successful build will mean that all xml successfully validated, the task also generates a report in result.xml.Tip: There are a few 3rd party tasks for validation in existence, one of which (untested by the Author) can be found here http://iso-relax.sourceforge.net/ JARV/antTask.html

The reader will spot the use of a element which instructs the filterchain to pass the contents of an entire file as a single token, as opposed to creating tokens for each string. Without the element Ant normally treats each string as a token; the effect of applying the regular expression to each separate token can dramatically slow things down. As long as your files are not gigantic, then you should see better performance using . Running the example2 Target, will copy the contents of the /data directory and compact file contents, placing the output into the /results directory. Test Sed ut perspiciatis unde omnis iste natus error sit voluptatem accusantium doloremque laudantium, totam rem aperiam, eaque ipsa quae ab illo inventore veritatis et quasi architecto beatae vitae dicta sunt explicabo. Nemo enim ipsam voluptatem quia voluptas sit aspernatur aut odit aut fugit, sed quia consequuntur magni dolores eos qui ratione voluptatem sequi nesciunt. Neque porro quisquam est, qui dolorem ipsum quia dolor sit amet, consectetur, adipisci velit, sed quia non numquam eius modi tempora incidunt ut labore et dolore magnam aliquam quaerat voluptatem. Ut enim ad minima veniam, quis nostrum exercitationem ullam corporis suscipit laboriosam, nisi ut aliquid ex ea commodi consequatur? Quis autem vel eum iure reprehenderit qui in ea voluptate velit esse quam nihil molestiae consequatur, vel illum qui dolorem eum fugiat quo voluptas nulla pariatur?

Web TechnologiesRecipe: Compress a set of HTML files The following short regular expression is popular with the Perl language crowd for compressing text and html files.>\s+

This simple expression normalises all occurrences of whitespace found in text, by matching all contiguous strings of whitespace and then replacing each match with a single whitespace character. To perform this operation in Ant, I use the native Task which was designed find and replace using regular expression matching to find. In the next listing, I would like to demonstrate how I can optimize a set of HTML files using a regular expression to strip out extraneous whitespace. The example1 Target will apply this regular expression against the text contained in each file defined by a . If the example1 Target is executed then it will process the actual files in /data directory. As this is an irreversible action I would propose using a and Task which means I retain the originals and generate a new set of compressed files. For now I have commented out the Task in the code example so you dont permanently compress the file examples under / data directory. What I really would like to do is apply our handy regular expression to a copy of the /data directory, which is what the following provides through the use of a element within a Task. size + file.size() } outputFile.text = totalSize } }

Incremental BuildTasks should not be executed redundantly between consecutive runs of the build. This means that if a tasks input (properties, files, and folders) and output (files and folders) have not changed since the last run, the execution of the task can and should be skipped. Gradle provides this functionality out-of-

Listing 2: Gradle script that declares a task to calculate the size of all Java production sources. The custom task type is annotated in order to participate in incremental builds.apply plugin: java task size(type: Size) { inputDir = sourceSets.main.allJava outputFile = file($buildDir/tmp/size.txt) } class Size extends DefaultTask { @InputFiles FileTree inputDir @OutputFile File outputFile @TaskAction void generate() { def totalSize = inputDir.files.inject(0) { def size, File file -> size + file.size() } outputFile.text = totalSize } }

The concept of input and output values is simple yet powerful. With Gradle, there is no need for task-specific (error-prone) work-arounds in order to achieve an incremental build. Gradle provides a generic solution for both out-of-the-box and custom task types.

2-Phase BuildRunning a Gradle build consists of two phases: the configuration phase and the execution phase. During the configuration phase, Gradle parses the build script, instantiates and configures the tasks and creates the DAG. During this phase, we have an opportunity to, for example, set the version number based on whether it is a production build or not, disable certain tasks, or even dynamically create new tasks. During the execution phase, the tasks are executed in the order as defined by the DAG. More precisely, the tasks actions are executed. The 2-Phase concept gives us a lot of flexibility to make build decisions at the proper point in time. There are also various possibilities for getting notified about build lifecycle events. For example, we can register a callback that gets invoked when the DAG has been populated just before the execution phase starts. There are also hooks for receiving notification just before a task is executed or just after it has finished executing.

Multi-Project BuildEnterprise software projects are typically modularized into sub-projects with well-defined project dependencies. Gradle provides extensive and smart build support for such multiproject setups. It maintains a virtual hierarchical project structure with a root project and an arbitrarily deep tree of


15

Build meets Groovy

sub-projects. Gradle places no restriction on the physical project layout, but always maps this to the virtual structure. Like tasks, projects are first-class citizens that can be accessed across the entire build. It is possible to define the evaluation order of the projects and to establish task dependencies across sub-projects. A critical feature of every large multi-project build is partial builds. Since Gradle has a deep understanding of task and project dependencies, it is able to offer true partial builds. For example, we can build a project and the projects it depends on, or build a project and all the projects that depend on it. Gradle makes it very easy to apply a common configuration from the root project to all sub-projects. In the sample below, we apply the java plugin and set the project version on all sub-projects.

subprojects { apply plugin: java version = 1.0 }

Migration from Ant or MavenGradle permits Ant builds to be imported and run as Gradle tasks. Ant targets can depend on Gradle tasks and vice versa. This smooths the transition from Ant to Gradle, especially when the Ant scripts are still undergoing modification whilst the migration to Gradle is in progress. In other words, we always have a running build. In the sample below, we import an Ant build script. All the Ant targets are then available as Gradle tasks. The deploy task can depend on such an Ant-originated package task as if it were a native Gradle task.

Listing 3: Gradle script with configured tasks to build, test, package and deploy a Java project.apply plugin: java apply plugin: maven docsDirName = documentation sourceSets { main { java.srcDirs = [src/prod/java] resources.srcDirs = [src/prod/resources] } test { java.srcDirs = [src/test/java, src/test-criticial] } } repositories { mavenCentral() } dependencies { testCompile junit:junit:4.8.2 } test { forkEvery = 50 maxParallelForks = 3 } jar { baseName = project.name.toUpperCase() manifest { attributes(Specification-Title: $project.name project)

} dependsOn javadoc } jar { long startTime doFirst { startTime = System.nanoTime() } doLast { long endTime = System.nanoTime() long duration = (endTime - startTime) / 1e6 println Packaging took $duration ms } } uploadArchives { configure(install.repositories.mavenInstaller) { pom.project { groupId jaxmag artifactId project.name.toLowerCase() url http://www.canoo.com } } } task zip(type: Zip) { baseName = project.name.toUpperCase() classifier = all from sourceSets.all.allSource from jar.outputs.files } task wrapper(type: Wrapper) { gradleVersion = 0.9-rc-2 }


16

Build meets Groovy

ant.importBuild build.xml task deploy(dependsOn: package)

A similar approach is planned for migration from Maven to Gradle.

builds and properly add third-party dependencies with sources and javadoc, if available. IntelliJ IDEA envisions a deep integration for Gradle build scripts, including code analysis and code completion.

Gradle WrapperEvery project is faced with a bootstrap problem: We first need the build system before we can build the project. However, installing the build system on a Continuous Integration server requires admin rights; and developers trying to install the build system on their own machine typically require support. Gradle provides a solution to this problem. With the Gradle Wrapper, we can set up a project such that every developer and CI server can build the project, even if they dont have Gradle installed. If we run the wrapper task shown below, the task generates a batch file gradlew.bat, a shell script gradlew.sh, and some more files that we need to check in together with the project. The command gradlew downloads the configured Gradle distribution and then delegates the call to the Gradle executable of that distribution. For example, the command gradlew build is forwarded as gradle build to the specified Gradle distribution once the distribution has been transparently downloaded.task wrapper(type: Wrapper) { gradleVersion = 0.9-rc-2 }

ConclusionGradle is a highly extensible and configurable build toolkit. It combines the declarative concepts of Maven with the flexibility of Ant. Its rich domain model enables intelligent build decisions to be made automatically. The Gradle DSL is extremely expressive and, in the case of simple projects, permits the writing of build scripts with just a few lines of code. Gradle also scales elegantly to meet the needs of complex enterprise builds and it is here where its conciseness, simplicity and maintainability lead to significant time and cost savings.

Etienne Studer currently works as a Senior Software Developer and Project Manager at Canoo Engineering AG in Basel. Previously, he worked as a Software Architect at Navis LLC in California, creating software for shipping terminals. His focus is on building and delivering high-quality software solutions that inspire the customer. Etienne is a JetBrains Academy Expert and maintains the blog practicalgradle.org.

Links & Literature[1] http://www.gradle.org [2] http://community.jboss.org/wiki/Gradlewhy [3] http://static.springsource.org/spring-security/site/build.html [4] http://grails.org/Developer+Guidelines [5] http://www.gradle.org/userguide.html [6] http://repo1.maven.org/maven2/

IDE-IntegrationGradle provides plugins for the generation of IDE project files for Eclipse and IDEA. The plugins support multi-project

ImprintPublisher Software & Support Media GmbH Editorial Office Address Geleitsstrae 14 60599 Frankfurt am Main Germany www.jaxenter.com Editor in Chief: Editors: Authors: Copy Editor: Creative Director: Layout: Sebastian Meyen Jessica Thornsby, Claudia Frhling James Fuller, Brett Porter, Jason Sankey, Etienne Studer, Russel Winder Nicole Bechtel Jens Mainz Karolina Gaspar, Melanie Hahn, Patricia Schwesinger Sales Clerk: Mark Hazell +44 (0)20 7401 4845 [email protected] Entire contents copyright 2010 Software & Support Media GmbH. All rights reserved. No part of this publication may be reproduced, redistributed, posted online, or reused by any means in any form, including print, electronic, photocopy, internal network, Web or any other method, without prior written permission of Software & Support Media GmbH The views expressed are solely those of the authors and do not reflect the views or position of their firm, any of their clients, or Publisher. Regarding the information, Publisher disclaims all warranties as to the accuracy, completeness, or adequacy of any information, and is not responsible for any errors, omissions, inadequacies, misuse, or the consequences of using any information provided by Publisher. Rights of disposal of rewarded articles belong to Publisher. All mentioned trademarks and service marks are copyrighted by their respective owners.


17

More to build

A brief look beyond the mainstream

Gant and PulseThe build tools story doesn't end with Ant and Maven! Here, we take a quick look at some additional build tools: Gant, for scripting Ant tasks using Groovy instead of XML; and the Pulse continuous integration server.Russel WinderGant was created because I hated using XML to describe my Ant-based builds. Groovy, the dynamic symbiote of Java, has a builder called AntBuilder that makes scripting Ant tasks using Groovy extremely easy. As a trivial example:ant = new AntBuilder ( ) ant.mkdir ( dir : 'Stuff' ) ant.javac ( srddir : '.' , destdir : 'Stuff' , debug : 'true' ) { classpath { ( new File ( 'libs' ) ).eachDir { pathelement ( location : it ) } } }

Picture 1: Pulse 2.2 administration

The meta-object protocol of Groovy makes these sort of builders very straightforward to implement. However, alone, the AntBuilder needs significant infrastructure to be useful as a build framework. To avoid everyone having to replicate that infrastructure for themselves, there is Gant. Gant provides command line processing, infrastructure for defining targets, a set of useful tools and target sets, and the ability to easily create new tools and target sets. So for example:destination = 'Stuff' source = 'src/main/java' includeTargets

jaxmag-2010-03

Documents