rapid distribution deployment: a goal-oriented framework for successful deployment of free and open...

8/14/2019 Rapid Distribution Deployment: a goal-oriented framework for successful deployment of free and open source op…

http://slidepdf.com/reader/full/rapid-distribution-deployment-a-goal-oriented-framework-for-successful-deployment 1/10

Rapid Distribution Deployment: a goal-oriented

framework for successful deployment of free and

open source operating systems

José Miguel Parrella Romero

[email protected]

November 22, 2007

1 Introduction

Most organizations tend to buy proprietary software based on the cost analysisassumption that money spent on software now will become company revenuelater. Furthermore, IT vendors are starting to spread the word: technology isnot business support, it is the business. Nowadays, medium and small sizedcompanies realize that they’re spending more in software than what they’reearning in their core business.

Free and open source software (FOSS ) is often regarded as a solution to thismoney bucket as its TCO is significantly lower than proprietary software [4].

While FOSS TCO is never zero, proprietary software companies force users intobuying licenses, training and maintenance contracts which usually drive moneyaway from the company’s main business. Therefore, one of the most effec-tive ways to reduce costs on software expenditure is deploying FOSS massivelyaround the organization [1] [3] [5]

Cost saving is not the only reason driving companies away from proprietarysoftware. Most IT units often feel compelled to use free and open source op-erating systems in order to either diversify their product portfolio or ensuresecurity through the peer-based review approach native to FOSS developments.Sooner or later, deploying a FOSS-based operating system becomes a main fieldof interest in the organization, and ad-hoc fear, uncertainty and doubt appearsto take over most critical decisions.

2 Course of action

Unplanned deployments are expensive on the long-term; trying to acceleratethe deployment of software that involves a change of mind in an organizationusually arises urgent needs, and some leaders address this needs spending lotsof money. Deploying free and open source software is not an exception: it has

1



to be thoroughly planned. Agile work teams know the difference between being

fast and being cautious.

2.1 Rapid Distribution Deployment

Rapid Distribution Deployment (RADID) provides a framework for fast, lo-calized distribution deployment. It aims to provide systematic, well definedand distribution-agnostic guidelines to goal oriented deployment of FOSS basedoperating systems. Actions expected from the IT department include:

• Build an agile, committed and able work team

• Define organization requirements in a SLA

• Profile users and select applications

• Prepare a good infrastructure

• Select a base distribution

• Deploy fast, deploy once

• Use a configuration manager

• Always give back

2.2 Understanding FOSS development

First and foremost, the deployment team needs to be prepared to assume in a

positive way the differences between FOSS and proprietary courses of action.Shell scripting, by means of the mighty sh language or multipurpose lan-

guages such as Perl, Ruby and Python is a must for any deployment team.Version control with modern tools such as Subversion and Bazaar will proveuseful as well. Organizations will benefit from having an able staff, which willsolve most problems using elegant and reusable methods. Finally, documen-tation should be endorsed by means of open standards, such as XML-basedDocBook, and strict documentation policies.

FOSS developments are usually fast-paced. Developers tend to group them-selves in communities, where they provide spaces of interaction by means of mailing lists, forums, wikis, blogs and instant messaging. Organizations mustprovide access to this resources for their development teams. The most technical

activities usually happens on public means, through the means of meritocratic,ad-hoc formed development groups.Since FOSS developments happen on communities that are more or less phisi-

cally isolated, some people work on the integration of these separate portions of software, often spending considerable amount of resources in such task. Whensomebody selects given versions of publicly-available FOSS and performs certainconfiguration on it (such as corporate art or optional features) it is said that

2



a distribution is built. These distributions are offered to the public, and most

FOSS users are actually using one of the thousands of available distributions.Organizations should be aware of the FOSS workflow, and it should adaptbusiness methods to a changing environment: applications are developed incommunities, released to the public, and then packaged and integrated intodistributions. Next, end users install distributions and give feedback to eitherthe distributors or the original developers. Finally, feedback results on patches,bug reports, documentation and other goodies, which are integrated into themain development and then goes downstream again to the distribution.

2.3 Selecting a base distribution

A common, if not main, reason for organizations choosing FOSS as a field of interest is to optimize their expenditures. In more than 20 years of fast-paced

development, most FOSS projects are now stable, with a huge amount of man-power invested to provide stable products. Existing FOSS distributions havealready achieved the same reliability point, and organizations willing to deployFOSS-based operating systems ought to choose an existing distribution to buildthings upon it and remain cost-effective.

Most distributions use the Linux kernel, an stable operating system availableto the public in [1]. There are distributions based on other kernels, mainlyFreeBSD, which might be of interest when organizations have special needssuch as enhanced security or real-time processing. This methodology focus onLinux-based operating systems, which are popular between end-users due to ahuge support for tenths of hardware architectures and accessories.

Distributions are usually grouped by their packaging methods. A package

is some sort of binary, compressed file which contains software elements (thedata) and information regarding what is needed to interact with other softwareelements (the metadata)

The most extended packaging method is the Advanced Packaging Tool (APT)which uses binary based, ar-compressed files whose metadata can express fine-grained, atomic relationships between independent software elements. APT isused by Debian, Ubuntu and hundreds of derivatives. Extensive documentationis available regarding elegant derivation of FOSS-based operating systems fromDebian [2] but this article does not provide the technical details since RADIDremains command- and distribution- agnostic whenever possible.

RPM, the Red Hat Package Manager, is a mature binary-based packag-ing system featuring gzipped-archives, with strengthened signature and errorcorrection elements, and currently in use by Red Hat Enterprise Linux, Fedora

Core, SuSe and hundreds of derivatives. Software vendors used to prefer RPM astheir packaging method of choice when they needed to deliver software packagesfor open source operating systems, however things have been mostly reversednowadays.

Other packaging methods are mostly source-based. The user downloads asource package, usually compressed, from the original developers, then addsa set of patches provided by the distributor and a general metadata file can

3



describe how the package is built and installed. Gentoo Linux provides a great

tool called emerge, and Arch Linux provides a source package manager calledpacman. Non-Linux based operating systems also feature both binary- andsource-based packaging systems.

In general, packaging allows distributors to integrate upstream software intheir final products, allowing some degree of cooperation between independentsoftware packages. Selecting a base distribution usually means selecting a pack-age manager, which will sign the organization experience with FOSS-based op-erating systems for years to come. Most organizations would choose a binary-based packaging method, such as APT or RPM, since it saves time when asecurity patch needs to be deployed.

There are, however, three next-generation packaging systems that are worthto talk about. Those are Conary, Smart and Klik. The first one is a distributedpackage management system which presents a completely new way of looking

into software distribution in Linux. Currently there is a lack of documentationand a strange situation regarding Conary’s licensing. Smart provides new algo-rithms for the main packaging methods, in two great interfaces. Klik providesbundles that contains everything needed to run an application.

All of these technologies ought to be tried nowadays, since they provide newways to do things and can ease distribution deployment when correctly used.Packaging technologies become critical on the early phases of FOSS operatingsystems adoption, since they provide uniform ways of installing, updating andremoving software packages from the end user work environments.

2.4 Setting up infrastructure

Infrastructure is critical for the deployment process, and it is highly recom-mended to invest most resources on building reliable support systems. Whilesmall organizations with less than a thousand users can handle the whole de-ployment with a single, workstation-grade server, geographically dispersed andbigger organizations will need several servers and support technologies. Ba-sically, the infrastructure will be comprised of a repository, a version controlsystem, a configuration manager and probably user support elements such asan online helpdesk, mailing lists or forums.

Repositories are software reservoirs which are key for any distribution de-ployment. Most packaging methods allow repositories being served over HTTPand FTP. HTTP is a cost-effective choice since most organizations feature somesort of reliability for this protocol, such as caching, proxying and clustering.

Both APT and RPM provide tools for building repositories, but they often

lack documentation regarding how they work. Packaging methods embed theinternals of their HTTP negotiations inside their source code, so there are twomethods for finding out how a repository works: looking at the code, or lookingat a static repository. The latter is easier to accomplish browsing anyone’sfavorite distribution installation CD.

Most organizations would like to integrate their software repositories in theirexisting web-based services. Repositories can be cached, proxied, virtualized and

4



clustered, and it’s a good thing to do so, as it guarantees load balancing and

high availability for the entire distribution deployment.Since any change to the software repository will inflect changes on end-user machines, software repositories should be put under version control, andmonitoring (using Webalizer, Munin or Awstats, for instance) will provide greatstatistics and control over the deployment.

IT-related organizations will probably setup several repositories in order tostage development (i.e., development, consolidation, production) split commonpackages from specialized applications, or even due to license restrictions. Orga-nizations will probably benefit in the short-term from having several repositories,since it allows them to perform deploy and rollback tests.

Under RADID, applications should be thought of as packages. An appli-cation is composed of at least one package, and one package does not alwaysmake an application out of its own. Big applications such as the office suite

OpenOffice.org can be split in tenths of packages in order to allow the user toselect only the desired functionality and reduce download times. On the otherhand, installing xserver-xorg-core in Debian will not get the X.org server.

Packages are atomic elements of software which are tightly related to eachother. They contain data (the files actually used) and metadata (what they needto be run, how they are installed, etc.) Software repositories hold packages, andpackaging utilities on the end-user side contact the repository to get and installpackages.

2.5 User profiling

Most organizations already have well-defined user profiles, such as office worker,

systems administrator, graphic designs, etc. Researchers should select applica-tions that fit every profile needs from the vast amount of applications releasedand available under FOSS developments, and determine their package names.All users will share a common base of packages, which is called the base system.

User profiles should be written in a document which should be put under ver-sion control, as per auditing methodologies. User profiles won’t usually changein time if user needs are taken into account properly and a good application se-lection is performed. Later it will serve a good purpose when the configurationmanager is modelled to the user base.

2.6 Localization

In this methodology, localization stands for the configuration done to any base

distribution in order to accomplish the needs of any given organization. Require-ments range from corporate art to LDAP integration, and even if each particularrequirement needs different expertise to solve it, the biggest distributions oftenprovide methods to integrate all changes in a single way.

Debian-based distributions use the debconf management system. Some pack-ages provide bindings to the debconf system and perform automatic setup basedon the information given to debconf. For example, LDAP integration can be

5



configured completely by using debconf, and the Debian Installer allows users

to provide a remote file containing answers to debconf question. This method,called preseeding, provides an elegant way for localization of some distributions.RPM disregards preseeding in favor of other configuration methods which

will be explained later. So, when the distribution methods don’t solve thelocalization requirements, it might be needed to further modify the packageusing available distribution methods.

Being binary-based package management systems, APT and RPM are usu-ally thought of as read-only: users can only modify the behavior of an appli-cation after it has been installed. However, both systems also manage sourcepackages, which are source-based atomic elements of software where the devel-oper can specify what it’s needed to build the package in a particular way.

Organizations might be tempted to modify lots (or all!) of the availablesource packages in order to deploy a uniform virtual corporate image. This

becomes time-consuming on the short term, and it is not cost effective. Intel-ligent deployments modify few source packages (< 5%) and do most of theirlocalization using configuration management.

It is important that the deployment team knows what to modify in whichway. Again, the deployment is a planned activity, and nothing should be unex-pected. On the long term, well-done localizations don’t affect further securityupdates, which is a real concern when the team is handling lots of source pack-ages.

3 Deploying

There’s one great thing about deployment: it only has to be done once. And,

in most cases, it should only be done once, since the organization will be biggertomorrow than it is today, and in situ staff visits are not cost effective.

Rapid distribution deployment is about being able to follow the fast pace of the FOSS community, select and develop an operating system (a distribution)in a cost-effective manner and provide guarantees for further developments onthe long-term.

3.1 Massive installation

Massive installations are the easiest way to deploy new software. Applicationssuch as System Imager and FAI provide a great path for physical installations onbig volumes of computers. System Imager provides the easiest way for massive

installations, and it’s one of the most important applications on this area, thoughFAI is probably best adapted for Debian-based environments, since it’s able tohandle most

The deployment team will need to closely cooperate with logistics staff inorder to physically move the computers to the installation site. A small net-work will need to be setup in order to prevent installation procedures from thedevelopment environment to affect users prematurely. The networking staff will

6



need to either setup an isolated network using IEEE 802.1Q or NAT, or build

a small network from scratch.Fast, flash memory based media should be preferred over optical media suchas compact discs. Optical media tend to degrade in some cases and will notprovide a trustworthy installation service, being only useful in cases where themedia is < 200 MB. In most cases DVD images won’t be needed, since the initialinstallation should be as small as possible and most heavyweight applicationswill be sent over the network.

3.2 Remote installation

Remote installations work for small, well-defined networks where the availableinfrastructure works reliably enough to push software installations and updatesthrough the network. Again, System Imager and FAI provide greats means for

remote installations, and the organization can always choose to deploy theirown PXE environment for network-based bootup and posterior operation.

Some organizations already using network-based storage for profiles anduser’s data might also like the idea of booting a clean environment on eachstartup. This approach is mostly found on public access hardware such as In-ternet Kiosks, but it lately proves quite unreliable for big organizations mostlybecause of its harsh use of the network and the incompatibility with the currentenergy-wasting user behavior of leaving hardware on for long times.

3.3 In situ installation

The organization’s culture or internal structure might require in-situ installa-tions. Of course, from all three deployment methods studied, this is the slowestand most expensive method, since it requires manpower and logistics (e.g., travelcosts for that office in Middle of Nowhere Ave.) That said, this process can beheavily optimized by good documentation, strong, uniform process enforcementand prepared installation media.

Currently, most base distributions offer methods to prepare installation me-dia. Organizations should take profit from those mechanisms and elegantlymodel their installations in order to minimize the installation times when doingin situ deployments.

4 Maintenance

Keeping up with the organization’s recent deployed distribution is the most es-sential part of the deployment process. Scaffolding activities with RADID willensure organizations have already profiled users, setup a useful infrastructureand massively deployed a FOSS-based operating system in the most efficientway for each case. Maintenance should be painless when everything has beensetup using common sense, and little effort should be needed to keep the wholething running.

7



Even if the most elegant way to do things in a particular base distribution

was chosen, it is highly recommended to use a configuration manager from thepoint the operating system is installed onwards.

4.1 Configuration management

Configuration management is one of the most important elements for the main-tenance of any distribution deployment. It consists on being able to handle setsof configuration for the network workstations. Currently, there are two reliableconfiguration managers: Puppet and cfengine.

Puppet features a declarative language which abstracts the systems admin-istrator from complicate tasks. It is a young development, raising rapidly andwritten in Ruby. Cfengine, on the other hand, is a stable, mature agent whoselearning curve is steeper than Puppet’s. Both of them allow the distribution

manager to model the target system and get every node to become like themodel in an elegant way.

The selected configuration manager should have support for classes, andhaving a declarative language that abstracts the system administrator fromdistribution-specific ways is a plus. Classes should be tightly mapped to userprofiles, which should have been already done at this point.

Organizations will find that having a configuration manager in place savesthem from maintaining more source packages, and make it easier to deploychanges on all machines without visiting in situ, while keeping a consistentsystem model which integrates seamlessly with the distribution methodology.

5 Change management

RADID is about ordered, fast deployment of FOSS for non-IT-based organi-zations. By any means, change control becomes crucial, and organizations areadvised to maintain control of the changes made from the very beginning of the deployment process. Examples of items that should be kept under changecontrol are:

• Design documents stating the general deployment layout and expectedrange of action for the project

• User profiles stating the expected level of service, applications and versionsused, application priority and attention to accessibility

• Automated installations schemas, specification files and preseeding files

• Configuration files

• Source packages

• Written deployment procedures

8



Change management can be handled in the version control software which

should be already setup with the rest of the platform, and the whole conceptfits in a greater quality management system which hopefully will measure theclient’s satisfaction with the deployed product.

6 Giving back

One of the main items of an RADID-inspired deployment is the give back ele-ment. Strategically speaking, most organizations will take over freely availablesoftware and documentation which will help them to reduce costs, own theirtechnology, improve security or all of the above. Therefore, it’s usually saneto assume that giving back will turn into benefits to the FOSS ecosystem inthe short term, and that organizations play an active role in the use, feedback,

improve cycle of FOSS developments.Furthermore, giving back is a matter of staff development, since it will boostknowledge, provide areas of specialization for the staff, improve morale and gen-erally establish a productive work environment. These are the most commonactivities organizations should participate in when taking into account the col-laboration nature of FOSS-based operating systems:

• Report bugs in a sane way, providing generic patches when applicable

• Provide well-written, generic documentation or improve the existing one

• Participate on community-based support such as mailing lists and forums

• Donate, either with money, hardware or software, to the development

project

• Participate in periodic developer meetings

• Invest around ten (10) hours a week to community-related activities

7 Afterword

FOSS has grown horizontally in a distributed manner, and the fragile ecosys-tem holding things together features a plethora of concepts, relationships and jargon which is usually frightening for the modern IT department. FOSS op-erating systems, and specifically Linux distributions, are a matter of different

methodologies, techniques and ultimately styles of work which sometimes gen-erate uncertainty about the right thing to do.Rapid distribution deployment is about providing non-IT-oriented organi-

zations with a generic way of doing FOSS operating system deployment in avery efficient way. RADID looks forward to integrate well with a quality man-agement system, make sense inside an organization IT plan and boost staff participation. The bottomline is: deploy once in a lifetime, maintain once in a

9



while, always give back . This approach proves to be cost effective on the medium

term and guarantees governance on the deployed platform.It is the author’s interest to provide an insightful, down-to-earth methodol-ogy which allows organizations to fastly achieve their deployment goals, reusingcomponents whenever possible and participating in the bigger FOSS ecosys-tem, thus providing enough momentum to improve the current software baseand sharing benefits with thousand of organizations which have chosen free andopen source software for most IT operations.

References

[1] Jesús González Barahona. Impact of open source in the total cost of owner-ship. 2000.

[2] Ernesto Hernández-Novich. Construyendo una distribución debian adap-tada. 2007.

[3] Forrester Research. Linux crosses into mission-critical apps. 2004.

[4] Soreon Research. Saving cash: A comparison of open source and proprietarysoftware. 2004.

[5] David Wheeler. Why open source software / free software (oss/fs, floss, orfoss)? look at the numbers! 2007.

10

rapid distribution deployment: a goal-oriented framework for successful deployment of free and open...

Documents