overview of the automated build & deployment process johnita beasley tuesday, april 29, 2008
TRANSCRIPT
![Page 1: Overview of the Automated Build & Deployment Process Johnita Beasley Tuesday, April 29, 2008](https://reader038.vdocument.in/reader038/viewer/2022100508/56649f525503460f94c76d2b/html5/thumbnails/1.jpg)
Overview of the Automated Build & Deployment Process
Johnita BeasleyTuesday, April 29, 2008
![Page 2: Overview of the Automated Build & Deployment Process Johnita Beasley Tuesday, April 29, 2008](https://reader038.vdocument.in/reader038/viewer/2022100508/56649f525503460f94c76d2b/html5/thumbnails/2.jpg)
Overview of Deployment Tiers
Software builds and deploymentsperformed by the QA Team
Software builds and deploymentsperformed by the Systems Team
Software builds and deploymentsperformed by the Systems Team
Inside the NIH Firewall
Outside the NIH Firewall
DEV
QA
STAGE
PROD
![Page 3: Overview of the Automated Build & Deployment Process Johnita Beasley Tuesday, April 29, 2008](https://reader038.vdocument.in/reader038/viewer/2022100508/56649f525503460f94c76d2b/html5/thumbnails/3.jpg)
Build/Deployment Process (Current)
1. Generate & Test the Ant build script.
AntBuild Script
DEV
2. Test the Ant build scriptand Deployment on a Production-like server
Desktop
3. Generate Build & Deployment Instructions for QA
Build Request &DeploymentInstructions
QA
4. Test the Build &Deployment
STAGE
PROD
5. Tests app. Sends Build & Deployment To Systems Team
Build Request &DeploymentInstructions
6. Deploys ApplicationBased on Build & Deployment Instructions
6A
6B
![Page 4: Overview of the Automated Build & Deployment Process Johnita Beasley Tuesday, April 29, 2008](https://reader038.vdocument.in/reader038/viewer/2022100508/56649f525503460f94c76d2b/html5/thumbnails/4.jpg)
Guiding Principles (Automated)
One process to rule them all. The same build and deployment process will be used on all tiers, without exception. This enables ongoing validation and will reveal problems more quickly. In a broader sense, this principle also means that all teams will use the same approach.
Simplicity. Where multiple alternatives existed, the team favored the one that seemed simpler and more direct.
Standardization. This involved standardizing on a directory structure, creating a master set of environment properties, and designing a standard build script. This approach benefits the Center as a whole, because developers who work on more than one project, and open source customers who implement more than one of our products locally, only have to learn one way of doing things.
Tool-neutrality.Build scripts must be executable from the command line in a properly configured environment. The Center uses AnthillPro as a Build Management Server, but open source users of our software may not have access to it. Therefore, we will not inject any dependencies on AnthillPro into our process.
![Page 5: Overview of the Automated Build & Deployment Process Johnita Beasley Tuesday, April 29, 2008](https://reader038.vdocument.in/reader038/viewer/2022100508/56649f525503460f94c76d2b/html5/thumbnails/5.jpg)
Guiding Principles (Automated)
Tier-neutral builds.
The output of the build step should be a tier-neutral deployment unit, and a set of external property files,
which implies that a build script should not have tier specific build targets.
Appropriate error handling.
Build scripts should adopt a ‘fail early’ approach to detecting and reporting errors. Database scripts should
perform extensive error checking and should be able to rollback to the previous state if any error is encountered during execution. Development teams are responsible for providing for both confirmation of a successful deployment, and rollback of failed deployments.
![Page 6: Overview of the Automated Build & Deployment Process Johnita Beasley Tuesday, April 29, 2008](https://reader038.vdocument.in/reader038/viewer/2022100508/56649f525503460f94c76d2b/html5/thumbnails/6.jpg)
Central Build Server
Used as a ‘reference’ build machine
Supports isolation of build and deployment configuration parameters
Configured with the necessary external dependencies
Anthill Pro build management server will fulfill build and deployment requests on all tiers
Training will be provided to teams in use of the Anthill Pro tool
![Page 7: Overview of the Automated Build & Deployment Process Johnita Beasley Tuesday, April 29, 2008](https://reader038.vdocument.in/reader038/viewer/2022100508/56649f525503460f94c76d2b/html5/thumbnails/7.jpg)
Hot Deployments
NCICB uses shared application containers
Applications must be hot-deployable to be considered eligible for deployment at arbitrary time without disrupting operation of other applications
Applications that are NOT hot-deployable (require a container restart) will only be deployed during regularly scheduled deployment windows.
![Page 8: Overview of the Automated Build & Deployment Process Johnita Beasley Tuesday, April 29, 2008](https://reader038.vdocument.in/reader038/viewer/2022100508/56649f525503460f94c76d2b/html5/thumbnails/8.jpg)
Master set of tier-specific properties
Systems team will provide to all developers a standard set of environment properties and their appropriate values.
This file will describe the deployment environment to which their applications must conform.
A version of this file will be maintained on each tier and the appropriate version will be supplied to the build process
All tier-specific properties will be included in the master file. Ensures all projects will use a consistent naming scheme.
![Page 9: Overview of the Automated Build & Deployment Process Johnita Beasley Tuesday, April 29, 2008](https://reader038.vdocument.in/reader038/viewer/2022100508/56649f525503460f94c76d2b/html5/thumbnails/9.jpg)
Directory Structure Standardization
Influenced by the Eclipse web application defaults (since Eclipse is the tool of choice for most development teams)
Detailed description provided in the Automated Build and Deployment Handbook
![Page 10: Overview of the Automated Build & Deployment Process Johnita Beasley Tuesday, April 29, 2008](https://reader038.vdocument.in/reader038/viewer/2022100508/56649f525503460f94c76d2b/html5/thumbnails/10.jpg)
Build Script Standardization
Two primary build targets recommended are:– build-all: creates the (tier-neutral) deployment unit, and copies all
output to the deployment-artifacts directory– deploy: first checks to see that the deployment-artifacts directory
exists and the deployment unit is there before proceeding with the deployment
These targets are the primary interface points for all users of the script
There should be no direct link between these targets
![Page 11: Overview of the Automated Build & Deployment Process Johnita Beasley Tuesday, April 29, 2008](https://reader038.vdocument.in/reader038/viewer/2022100508/56649f525503460f94c76d2b/html5/thumbnails/11.jpg)
Standard for reading build property files
A build script using this approach can be invoked in one of two ways: a. >ant build-all
– Load build properties from a local file called build.properties b. >ant -DPROP.FILE=/path/to/my.properties build-all
– Overloads the default value of PROP.FILE and loads properties from the specified file The benefits of this approach are:
– The build script can be run with an arbitrary set of parameters– Allows the Systems Team to maintain protected versions of properties files for
STAGE & PROD– Keeps all tier-specific properties in easily accessible place
<property name="PROP.FILE" location="build.properties" /><property file="${PROP.FILE}" />
![Page 12: Overview of the Automated Build & Deployment Process Johnita Beasley Tuesday, April 29, 2008](https://reader038.vdocument.in/reader038/viewer/2022100508/56649f525503460f94c76d2b/html5/thumbnails/12.jpg)
Naming Conventions
The file prefix “template.” indicates build and deploy substitution variables are present.The expanded file will have the same file name without the “template” prefix, e.g.template.web.xml will be used to create web.xml after the substitution variables areexpanded, using Ant's filtering capabilities.
Property names should be all UPPER CASE for external build.xml properties and alllower case for internal build.xml properties. This is to make it easier to determine where aproperty’s definition can be found.
Properties that are directory or path references should include “.DIR” or “.dir” in theirnames.
Properties that refer to files should include “.FILE” or “.file” in their names.
Properties that refer to environment variables should use “ENV.” in their names.
Target names begin with a dash (‘‘) for internal build.xml targets.
In addition to the conventions outlined here, the project team has adopted the recommendations of The Elements of Ant Style, available at http://wiki.apache.org/ant/TheElementsOfAntStyle.
![Page 13: Overview of the Automated Build & Deployment Process Johnita Beasley Tuesday, April 29, 2008](https://reader038.vdocument.in/reader038/viewer/2022100508/56649f525503460f94c76d2b/html5/thumbnails/13.jpg)
Property validation
Each build script is responsible for validating it’s properties
For most cases, it will only be possible to validate that the property actually has a value
Validating of meaningful values should be performed if possible – Example: Verifying the correct versions of Java and Ant are being used
Both the JAVA_HOME and ANT_HOME environment variables will be available during script execution and should be checked for the appropriate values.
If any validation step fails, an appropriate error message should be generated and the script halted immediately.
![Page 14: Overview of the Automated Build & Deployment Process Johnita Beasley Tuesday, April 29, 2008](https://reader038.vdocument.in/reader038/viewer/2022100508/56649f525503460f94c76d2b/html5/thumbnails/14.jpg)
Property Validation Example
The 1st segment of the following code snippet validates all properties, and sets the check.props property if a required property is not set. It is at the top of the script and immediately evaluated before any targets are invoked.
In the 2nd segment, the ‘–prop-check’ target is a dependent of every other target and will cause the script to fail if check.props is set.
<condition property="check.props"> <and> <isset property="PROPERTY1" /> <isset property="PROPERTY2" /> <isset property= ... /> </and></condition>
<target name="-prop-check" unless="check.props"> <fail>Failure Message</fail></target>
![Page 15: Overview of the Automated Build & Deployment Process Johnita Beasley Tuesday, April 29, 2008](https://reader038.vdocument.in/reader038/viewer/2022100508/56649f525503460f94c76d2b/html5/thumbnails/15.jpg)
Summary of Build Step
Build script reads a tier-specific set of properties passed in on the command line.
Build script verifies that all required properties have values. If possible, the script may also check to ensure that the properties have meaningful values. This is the required approach for all teams implementing auto deployment.
Build script creates tier-neutral build artifacts, and copies them all to the deployment artifacts subdirectory.
![Page 16: Overview of the Automated Build & Deployment Process Johnita Beasley Tuesday, April 29, 2008](https://reader038.vdocument.in/reader038/viewer/2022100508/56649f525503460f94c76d2b/html5/thumbnails/16.jpg)
Summary of Deploy Step
Based on the tier-specific properties selected at deploy time, the script should copy the deployment units to the specified directory on the deployment machine.
After the files are successfully copied over, the build script should move them to their final destination on the target machine. This can be accomplished this by using the Ant SSHEXEC task to execute remote commands on the target. The build machine has been configured with SSH keys to enable appropriate access on the target machine.
![Page 17: Overview of the Automated Build & Deployment Process Johnita Beasley Tuesday, April 29, 2008](https://reader038.vdocument.in/reader038/viewer/2022100508/56649f525503460f94c76d2b/html5/thumbnails/17.jpg)
Example: Execution of Remote Commands
The example code is reusable as is, without modification, to any project that follows the recommended directory structure.
<scp todir="${SCP.USER}@${SCP.HOST}:${SCP.PATH}" port="${SCP.PORT}" passphrase="${SCP.PASSPHRASE}" keyfile="${SCP.KEYFILE}" trust="true" verbose="true"> <fileset dir="${artifacts.dir}" includes="**/*" /></scp>
![Page 18: Overview of the Automated Build & Deployment Process Johnita Beasley Tuesday, April 29, 2008](https://reader038.vdocument.in/reader038/viewer/2022100508/56649f525503460f94c76d2b/html5/thumbnails/18.jpg)
Responsibilities
Systems/SCM Team– Maintains a master file of properties (provided to development
teams) that describes the build and deployment environment– Provides implementation support for teams automating their build
scripts through all tiers– Providing access to the Anthill Build Server
Development Teams– Writing and maintaining build scripts– Addressing build failures
![Page 19: Overview of the Automated Build & Deployment Process Johnita Beasley Tuesday, April 29, 2008](https://reader038.vdocument.in/reader038/viewer/2022100508/56649f525503460f94c76d2b/html5/thumbnails/19.jpg)
Related Documents and References
Deployment Handbook Automated Build and Deployment Handbook
You can get these documents from:http://gforge.nci.nih.gov/docman/index.php?group_id=27&selected_doc_group_id=84&language_id=1
![Page 20: Overview of the Automated Build & Deployment Process Johnita Beasley Tuesday, April 29, 2008](https://reader038.vdocument.in/reader038/viewer/2022100508/56649f525503460f94c76d2b/html5/thumbnails/20.jpg)
Questions