ASSESSING SOFTWARE EXCELLENCE:A MODEL AND AN EMPIRICAL TEST

by

S. DUTrA*and

L. N. VAN WASSENHOVE**

96/69/TM

* Associate Professor of Information Systems at INSEAD, Boulevard de Constance, Fontainebleau 77305Cedex, France.

** Professor of Operations Management and Operations Research at INSEAD, Boulevard de Constance,Fontainebleau 77305 Cede; France.

A working paper in the INSEAD Working Paper Series is intended as a means whereby a faculty researcher'sthoughts and findings may be communicated to interested readers. The paper should be consideredpreliminary in nature and may require revision.

Printed at INSEAD, Fontainebleau, France.

Assessing Software Excellence:A Model and An Empirical Test

Soumitra Dutta and Luk N. Van Wassenhove*Research Initiative in Software Excellence (RISE)

Technology Management AreaINSEAD

FontainebleauFrance 77305

Abstract

The strategic importance of software has long been recognised by both

practitioners and academics. Organizations have faced two challenges in

leveraging the strategic potential of software. First, there is a need for increased

maturity in software development processes. Second, many organisations have

struggled to derive adequate business value from software to key stakeholders.

Several models for assessing and improving the maturity of software processes

have been proposed in the literature. Taking guidance from recent developments

in the domain of Total Quality Management, this research proposes a model of

Software Excellence which extends the narrow focus of current software

maturity models on the "software factory" to the broader organisational context.

The Software Excellence Model defines the degree to which an organisation is

succeeding in both creating the enabling conditions and also in obtaining results

for leveraging software to create value for all key stakeholders including

shareholders, end-users, employees and the parent business unit at large. The

validity and usefulness of the Software Excellence Model is demonstrated by an

empirical test - a questionnaire-based survey of European organisations.

1 Introduction

This section introduces the need for excellence in software and outlines the focus and

structure of the paper.

1.1 Strategic Importance of Software

Software forms the "back-bone" of major industries such as banking, airlines and

publishing, and is an increasingly important value-adding component of consumer

products such as television sets, cameras, cars and mobile phone sets. Software is

* Author names are listed in alphabetical order. Both authors contributed equally to the paper.

1

today a dominant force in enabling companies to exploit new distribution channels,

create new products and deliver differentiated value-added services to customers. In

reality, there is often little difference between an organisation's software strategy and its

business strategy [7,44].

In addition to the ubiquitous nature of software, the amount of software code in most

consumer products and systems is doubling every two to three years. This increase is

being driven both by escalating demands placed on the functionality of software

systems and the rapid pace of progress in the enabling hardware technology.

Consequently, software developers are scrambling to cope with the pressures of

developing systems which are not only a couple of orders of magnitude bigger and

more complex than those developed a few years ago, but also which need to meet ever-

increasing demands for higher quality and superior performance.

The strategic importance of software has been long understood by practitioners and

researchers [33,40,42,41,47]. However, organisations face two major challenges in

leveraging the strategic potential of software. First, there is a need for a better execution

of software projects. Stories of dramatic time and cost overruns of software projects are

legendary. For example, Gibbs [16] notes that: "for every six new large-scale software

systems that are put into operation, two others are cancelled. The average software

project overshoots its schedule by half; larger projects generally do worse" (pp. 72-73).

Second, there is a need to derive greater business value for key organisational

stakeholders from software. Several researchers [1,31] have found little or no benefits

to organisations from recurrent investments in information technology. This

"productivity paradox" [6] is particularly significant when one notes that the delivered

computing power in many developed economies has increased by more than two orders

of magnitude over the past two decades [6]. Given the technological promise and

potential of software, Brynjolfsson [6] states that "..disillusionment and even

frustration with the technology is increasingly evident.." (p. 67). Indeed, there is a

growing concern that there is often a fundamental mis-alignment between the software

and business strategies of many organisations [4,13,19].

Over the last decade, several models such as the Capability Maturity Model (CMM)

[23,38,39], have been proposed and used within industry for assessing and improving

software development processes. The focus in CMM and similar models is on

addressing chronic problems in software development processes. They typically

address issues such as whether an organisation has appropriate software project

2

management procedures in place and whether the right metrics are being collected and

utilised for managing software processes.

Given the hurdles in obtaining adequate returns from rapidly escalating software

investments, and in aligning software strategy with an organisation's business strategy,

there is a need to include the broader organisational context of the application of

software within models for assessing and improving the strategic leverage of software.

Guidance can be obtained from the domain of Total Quality Management (TQM) where

research over the past four decades has incrementally extended the focus of TQM

models from a narrow "quality control" perspective to a company-wide strategic

"quality of management" perspective which is synonymous with overall business

excellence.

The contribution of this research lies in the proposal of a model of Software Excellence

(SE) which integrates the strengths of software process assessment models with the

overall business-wide scope of TQM models. Inspired by the European Quality Award

model [29], the proposed SE model defines the degree to which an organisation is

succeeding in . both creating the enabling conditions and also in obtaining results for

leveraging software to create value for all -key stakeholders. We also demonstrate the

validity and usefulness of the SE model by means of an empirical test - a questionnaire-

based survey of European organisations.

1.2 Structure of Paper

There are seven additional sections of the paper. The next section describes prior

research in assessing software maturity and TQM and outlines potential synergies

between the two domains. Section 3 outlines the integrated model for assessing SE

proposed in this paper. The research design used for this study is the subject of the next

section. The next three sections describe the empirical validation of our research.

Section 5 describes general results and overall scores of SE among the surveyed

organisations. This is followed by an analysis of the discriminatory and explanatory

power of the SE model in Section 6. Section 7 presents a discussion of the overall

results of the study and provides relevant comparisons with the literature. The last

section concludes the paper with notes on the limitations of the study and directions for

further research.

3

2 Software Maturity and Total Quality Management

This section describes prior research in TQM and software process maturity

assessments.

2.1 Models of Total Quality Management

The roots of the quality movement can be traced back to more than four decades ago.

The Deming Prize was launched in Japan in 1951 with the declared purpose of

"Awarding Prizes to those companies recognised as having applied Company Wide

Quality Control based on statistical quality control" [29, p. 152]. For the first decade,

the focus of the Deming Prize was limited to the application of statistical techniques in

the factory. Since 1964, the scope was expanded to include company-wide quality

control.

Established in the USA in 1987, the Malcolm Baldrige National Quality Award [36], is

based on the implementation of a company-wide system of Total Quality Management

(TQM). The Baldrige Award framework consists of several categories: Leadership,

Process Management, Human Resource Development and Management, Strategic

Planning, Information and Analysis, Customer Focus and Satisfaction and Business

Results. Over the years, the Baldrige Award categories have evolved from a

product/service quality focus to a wider definition of business excellence. For example

the categories "Process Management" and "Business Results" were previously titled

"Quality Assurance of Products and Services" and "Quality Results" respectively. Also

after criticism that the award emphasised investments in quality efforts at the expense of

bottom-line financial results, the award criteria were adjusted to give more importance

to customer satisfaction and bottom-line business results [15].

The European Quality Award was created in 1991 by the European Foundation for

Quality Management (EFQM). While influenced by the Baldrige Award framework,

the European Quality Award emphasises a holistic view of overall business excellence

[29]. It splits business excellence into two categories: Enablers (cause) and Results

(effect), with equal importance being assigned to both categories. There are five major

dimensions of Enablers: Leadership, People Management, Policy and Strategy,

Resources and Processes. Both financial and non-financial results for the key

stakeholders of an organisation are included in the Results category: People

Satisfaction, Customer Satisfaction, Impact on Society and Business Results. The

European Quality Award has been widely accepted within different European countries

as a basis for self-assessment and performance improvement.

4

Thus, over the years, quality models have progressively expanded in two directions.

First, there has been an increase in their scope from the "factory" to the entire

organisation. Second, the focus has shifted from a pre-dominant "product/service

quality" perspective to a notion of overall business excellence which delivers value to all

key stakeholders of the organisation - including customers, employees, shareholders

and society at large.

2.2 Software Maturity Models

The roots of research in software maturity assessment can be traced back to a decade

ago when the SEI in collaboration with the MITRE Corporation, began developing a

framework for assessing the maturity of software processes within organisations. From

a first description in 1987 [22] the framework has evolved over the years into the

widely accepted Capability Maturity Model (CMM) model for software maturity

assessment [38]. A recent article [39] provides a comprehensive review of the evolution

of the CMM model.

Since the pioneering work of the SEI, a number of initiatives in modelling and

assessing software process maturity have been started in parallel across the globe.

Some initiatives were started at the pan-European level such as BOOTSTRAP [28].

Other initiatives were developed with a focus on specific industries such as the Trillium

model [2] for the telecom industry. A common denominator in these alternate initiatives

is that they are all strongly influenced by the CMM model and essentially represent

different variations of the same themes.

The core concept underlying the CMM model [38] is that of five maturity levels - Initial,

Repeatable, Defined, Managed and Optimized - which define an ordinal scale for

assessing the maturity of an organisation's software processes. At the initial level, an

organisation's software processes are ad hoc and occasionally even chaotic. In contrast,

at the highest level, continuous improvement procedures enabled by the appropriate use

of metrics are institutionalised within the organisation's software processes. Table 1

provides a summary of the key process areas considered within the CMM [39].

Table 1 about here

The proliferation of different process assessment standards coupled with their increased

use in industry lead to the creation of the SPICE (Software Process Improvement and

5

Capability dEtermination) initiative [12,46], an attempt to create an international

standard for process assessment. While an initial architecture for the SPICE framework

has been proposed, the validation of this framework is ongoing within selected

organisations across the world [46].

The SPICE framework has the twin objectives of facilitating both process improvement

and capability determination [46]. Towards these ends, it distinguishes between two

types of practices: base practices and generic practices. Base practices cover the core

processes related to software development and are grouped into the following five

process categories: customer-supplier activity; engineering; project management;

support, and organisational (see Table 2). Generic practices refer to the implementation

and institutionalisation of processes within an organisation and help in the determination

of the appropriate capability level for the organisation. SPICE recognises six capability

levels: Initial, Performed, Managed, Defined, Measured and Optimized.

Table 2 about here

The evolution of software maturity models mirrors that of quality models. This is not

surprising given the fact that research in software maturity assessment was and

continues to be influenced strongly by research in total quality management. For

example, the CMM maturity framework was influenced by the work of Crosby [10]

and resulted from the adaptation of Crosby's quality management maturity grid to

software processes [43].

However, there is a lag in the evolution of software maturity models relative to quality

models. Current software maturity models emphasise the quality of the "software

factory" as opposed to measuring the impact of software on the entire organisation.

Also, the focus is primarily "software-product-related" as opposed to overall excellence

in delivering value from software to all major stakeholders of the organisation. A clear

indication of this lag is the fact that none of the existing software maturity models

include an emphasis on bottom-line business results.

3 Software Excellence

This section introduces a comprehensive model of SE which incorporates the strengths

of software maturity models and extends their scope to a wider organisational context.

6

3.1 A Model of Software Excellence

Software maturity models such as the CMM and SPICE largely focus on the superior

execution of software projects. For example, the CMM model prescribes specific

actions such as effective software project management, product assurance and change

control, for enabling an organisation to move its software processes from Level 1 to

Level 2. There is a need to expand the scope of software maturity models in order to

address the other important challenge mentioned in Section 1.1: the effective leverage of

software to derive value for all key organisational stakeholders. If experience in the

application of TQM models is to be taken as a guide, a narrow focus on improving the

maturity of software processes alone will not deliver the required organisational value.

The continuing quest for business value has been a major driver in the expansion of the

scope of TQM models from narrow "quality control" issues to a wider "quality of

management" perspective [11]. We propose below a model of Software Excellence

which integrates a software process perspective with the larger organisational context of

the application of software and focuses on the derivation of business value from

software.

In line with the notion of "business excellence" inherent within the European Quality

Model [29], we define "Software Excellence" (SE) as follows.

In any Software Producing Unit(SPU), leadership driving policy & strategy,

SPU people management, end-user management, resource management and

processes lead to Software Excellence, i.e., excellence in all aspects of the

creation and application of software in the broad organisational context.

Software Excellence in turn leads to end-user satisfaction, SPU people

satisfaction, a positive impact on the organisation and effective bottom-line

business results for the organisation.

In the above defmition, we use the term SPU (Software Producing Unit) to refer to

either an independent information technology organisation within a larger parent

organisation or an information technology division or department within an

organisation.

Figure 1 about here

Adapted from the European Quality Award model, the framework used for defming SE

is shown in Figure 1. There are ten aspects to be considered in total, which are

7

organised into two categories: Enablers and Results. Enablers are more concerned with

how things are done. Results are more focused on what has been achieved. A brief

description of the emphasis of each Enabler/Result category is as follows:

Enablers

• Leadership: The role of senior managers in creating and driving a culture of software

excellence throughout the organisation (including the SPU).

• SPU Policy and Strategy: The role of software excellence in the organisation's values,

vision, strategic direction and the implementation of its policy and strategy.

• SPU People Management How the organisation manages the SPU employees and

releases their potential to continuously improve the business.

• End-user Management: The nature, extent and effectiveness of partnerships created

between the SPU and its customers.

• Resource Management: The management, utilisation and preservation of financial and

non-financial resources by the SPU in a planned manner.

• Processes: The identification, management and continuous improvement of all key

processes related to the development of software.

Results

• SPU People Satisfaction: The organisation's success in meeting the needs and

expectations of the SPU employees.

• End-user Satisfaction: The degree of success in satisfying the needs and expectations

of the customers of the SPU.

• Impact on Organisation: The perception of the SPU within the organisation at large;

the degree of success of the SPU in satisfying the needs and expectations of the

organisation at large.

• Business Results: The success of the SPU in making the appropriate contribution to

the financial success and other business targets of the organisation.

There are two major changes in the SE model as compared to the European Quality

Award model. First, End-user Management has been introduced as a key element of the

Enablers. This is in recognition of the importance of partnerships with end-users for the

success of software projects [18,25]. All software maturity models have a special focus

on processes/activities interfacing with end-users. Second, the element "Impact on

Society" has been changed to "Impact on Organisation" in recognition of the fact that a

SPU exists within the context of the parent organisation analogous to an organisation

existing within society at large.

8

Analogous to the European Quality Award, a total of 1000 points has been divided

equally between the Enablers and the Results. As the assignment of points within the

European Quality Award was established "following a wide-ranging exercise to collect

views from business leaders, practitioners, academics and consultants" [29, p. 15], we

used it as a guide to determine the proportion of points assigned to each category of

Enablers and Results in the SE model. The proportion of points assigned to each Result

category is the same as in the European Quality Award. Due to the additional Enabler -

End-user Management - in the SE model, the points for this category were obtained by

proportionately reducing the points for the other Enablers. Table 3 summarises the

number of points assigned to and the salient aspects considered within each category of

the SE model.

Table 3 about here

3.2 Distinguishing .Features of the Software Excellence Model

Software maturity models such as CMM and SPICE primarily emphasise issues related

to software project management, software quality management, and the management

and improvement of software processes (see Tables 1 and 2). Within the SE model, the

Enabler category Processes includes aspects related to the management of software

processes which have been identified as important within CMM and SPICE. However,

there are nine other categories in the SE model which provides it with several unique

features relative to CMM and SPICE.

Perhaps the most striking feature is that the SE model emphasises the achievement of

business results, something which is ignored in software maturity models. As.

mentioned in Section 1.1, the derivation of business value from software is a key

challenge faced by organisations. The SE model includes both non-fmancial and

financial indicators of business results for all major stakeholders of the SPU - end-

users, SPU employees, shareholders and the parent business unit. Due to the equal

weight assigned to the Enabler and Result categories, it is impossible to achieve SE

without being good at both of them. Thus, the SE model explicitly includes the degree

to which an organisation has been successful in obtaining all-round business value from

software.

The SPICE framework recognises that software process improvement occurs in a

business context and, to be successful, must address business goals. However, within

9

SPICE, only process area ORG.1 (see Table 2) deals with selected organisational

aspects as detailed below:

• (ORG 1.1) Establish a strategic vision for the SPU;

• (ORG 1.2) Deploy the organisation's strategic vision to all employees;

• (ORG 1.3) Establish a culture which focuses on customer satisfaction;

• (ORG 1.4) Build integrated teams to satisfy customers;

• (ORG 1.5) Provide incentives to team members to accomplish team goals; and

• (ORG 1.6) Define appropriate career plans for employees.

In contrast, the SE model includes a significantly more comprehensive set of features of

the rich organisational context in which software is developed and applied to benefit the

major stakeholders of an organisation (see Table 3). Aspects related to software

Processes, which constitute the bulk of the content of CMM and SPICE, only account

for 12% of the total score assigned within the SE model. The importance of effective

software Processes is recognised by the fact that it accounts for the single largest

contribution (24%) within the Enablers. However, the SE model also includes five

other Enabler categories which cumulatively account for the bulk (76%) of the total

Enabler score, and cover the broad organisational context in which software is

developed and applied.

More specifically, the SE model recognises the important role -of top management

leadership in influencing the success of the SPU. Research [13,21,26,49] has indicated

that leadership and involvement of senior business managers are critical for the success

of software divisions within organisations. Issues related to top management leadership

and their involvement , in creating . a culture of software excellence within the

organisation remain largely unrecognised in software maturity models. The- SE model

also includes an elaborate treatment of issues related to SPU people management. The

literature [17,24] recognises that while software-related professionals constitute an

important and distinct group of employees within organisations, little effort has been

made to deal with their specific concerns and to better integrate them with the rest of the

organisation [13].

4 Empirical Validation of the Software Excellence Model

The research reported in this paper is based on the results of a questionnaire-based

survey of European organisations in late 1995. The research and survey described

below was conducted in collaboration with the European Software Institute, Bilbao,

Spain. This section describes the research design used in the study.

10

4.1 Design of Questionnaire

Several sources of information were used for the design of a questionnaire based on the

SE model. First, the existing body of literature on Software Maturity Assessment and

Quality Models was scrutinised carefully. In particular, aspects used to assess business

excellence within the European Quality Award model [29] and software maturity within

the CMM and SPICE models were found to be particularly useful for the design of the

questionnaire. Second, the authors conducted several discussions with managers to

elicit the key aspects of software excellence as perceived by practitioners. Finally, the

resulting questionnaire was pre-tested with selected INSEAD MBA participants and

evaluated by experts both at INSEAD and the European Software Institute for clarity

and relevance of the individual questions.

The SE questionnaire was made up of four sections. Section A had 9 general questions

about the SPU and its management, areas of operation and type of development

activities undertaken.

Section B and C contained 73 questions related to the Enablers and Results of the SE

model respectively. As the questionnaire represented a first exploratory attempt to test

our model of SE, we did not have pre-tested constructs representing the constituent

categories of the SE model. The use of a single item scale was discarded in favour of a

multi-item scale as research in measurement theory has "established beyond doubt...the

superiority of multi-item scales in respect of such matters as reliability,

unidimensionality and freedom from specific wording bias" [35, p. 182]. We used a

Likert scale [45] with multiple statements for each category, each statement covering a

particular facet of the respective dimension. Table 4 illustrates the nature of questions in

the SE questionnaire with a list of the precise questions used for the Enabler category

"SPU People Management".

Table 4 about here

Section D contained 6 questions to gather information on international software

development and the major obstacles in global software operations. In addition,

questions were included to ascertain the:

• Current and future competitive priorities for the responding SPUs (18 different

priorities were assessed);

11

• Use of formal methods by which the performance of the SPU was assessed, audited

or certified (9 different methods were included);

• Quantitative estimations of performance improvement in 28 key areas over the last two

years;

• Development cost structure for the SPU (3 questions); and

• Overall financial results of the SPU (12 questions).

4.2 Data Collection

The SE questionnaire was sent to around 3,000 companies all over Europe. Companies

were chosen at random from the INSEAD database as well as from the ESI database.

The data sample was distributed over all the European countries with at least 200

companies from each of the top 10 countries in order of GDP, and at least 100

companies each for the rest of the countries. Companies were selected from a number

of industrial sectors including information technology, manufacturing, business and

services.

A total of 85 duly completed questionnaires were received from companies (see

Appendix A for a profile of the respondents). This represents a response rate of 2.84%.

The low response rate can be attributed to the length of the questionnaire (20 pages), the

timing of the mailing (during the summer month of August 1995) and a lack of follow-

up with telephone or other direct contact after the mailing.

4.3 Analysis Outline

Very few responses were received from respondents for the data related to Section D

(global operations), estimations of performance improvement, development cost

structures and overall financial results. These questions were omitted entirely from the

analyses. The lack of responses on the above questions can be attributed to the

reluctance of respondents to part with financial information, the low usage of

quantitative measures of SPU performance and the lack of global operations on the part

of the respondents.

There were a few missing data for statements related to the various categories of

Enablers and Results of the SE model. The literature on replacing missing data [45] is

extensive and inconclusive. Given the limited missing data in the relevant part (Sections

B and C) of our survey and the exploratory nature of the research we decided to replace

the missing data with the "worst-case" assumption of "Absent" (see Response

categories in Table 4).

12

The reliability of the multi-item scale used in the SE questionnaire was measured with

the inter-item correlation matrix [34]. Cronbach' s alphas for each category of Enablers

and Results in the SE model was seen to be higher than 0.9. This suggests that the

scales used to measure the various categories of Enablers and Results are reliable. The

correlation matrices for the different Enabler and Result categories are shown in

Appendix B. The correlations are on the higher side, specially among the Enabler

categories.

The research results presented below consist of three parts. The focus in the first part

(Section 5) is on determining overall scores of SE for all responding SPUs and on

identifying their performance along each Enabler/Result category. Scores for both the

current and desired future levels of SE are presented. In . the next part (Section 6), we

focus on the explanatory power of the SE model. The group of respondents are divided

into two categories - those with high and low overall SE scores and the discriminatory

power of each Enabler and Result category is tested via discriminatory function

analysis. Also included in this section is a validation of the SE model through an

analysis of the variance in the Result categories which can be explained by the Enabler

categories. Section 7 provides a discussion of our findings and relates it to prior

research. Finally, the last part (Section 8) contains some concluding comments.

All statistical analyses were performed using Statistica [48].

5 General Results

This section describes the overall scores of SE determined in the survey and also

presents an analysis of the scores for the Enabler and Result categories for all

respondents.

5.1 Scores of Overall Software Excellence

An overall score of SE was computed for each company out of a maximum of 1000

points assigned across all dimensions of Enablers and Results (see Table 3 for the

distribution of points across the various Enabler and Result categories). This score was

calculated by normalising the scores for each Enabler/Result category and then

aggregating them with appropriate weights (as indicated in Table 3) to arrive at the final

overall score. The average SE score across all respondents is 473 out of a maximum of

1000 points (with a standard deviation of 125). The average scores for the Enablers and

13

Results for all respondents are 256 and 217 respectively out of a maximum of 500

points each (with standard deviations of 62 and 70 respectively).

It is important to note that the SE score (similar to scores computed using other

Software Maturity and Total Quality Models) is an assessment and not an accurate

measurement of the level of SE within an organisation. Thus it would not be right to

directly rank the performance of organisations based on the scores. Rather, the scores

give a good indication of the relative strengths and weaknesses of an organisation and

outline areas where further improvements are possible [9,20].

It is too early to establish the appropriate threshold for truly world-class SE in

companies. However some guidance can be obtained from the profiles of scores

achieved by companies winning major Quality Awards' . Allowing for variations in

assessor ratings and in the underlying model, Lascelles and Peacock [29, p.77] note

that "...anything between 700 and 800 points represents genuine world class. Anything

above 600 points represents a level of excellence to which few companies can aspire. A

typical score for an organisation that is being run competently and has a set of good

results is 450 points."

The distribution of the overall SE scores is depicted in Figure 2. Focusing on the

current SE scores, five companies have obtained scores in the range of 700 - 799 and

can claim to have achieved a measure of SE. About 64% of all respondents score below

500 and this gives an indication of the degree to which they can further improve their

respective levels of SE.

Figure 2 about here

The "future" SE scores of Figure 2 are computed on the basis of the importance

assigned by the responding SPUs to each dimension of the SE model over the next

three years (also see Table 4). Thus they cannot be directly compared to the current SE

scores. However, a distinct shift towards a "higher score" can be observed by

comparing the current and future "scores" in Figure 2. The average for the future score

is 657 with a standard deviation of 106. The average future scores for the Enablers and

Results for all respondents are 350 and 307 respectively out of a maximum of 500

points each (with standard deviations of 48 and 68 respectively). Clearly, all

1 One should keep in mind that the method used for computing the SE score is not the same as thatused for computing the overall scores for the European Quality Award. The latter does not use a strictquestionnaire format and leaves considerable freedom of judgement to the individual assessors inapplying the award criteria to specific organizations.

14

responding SPUs wish to improve their overall levels of SE in the near future with a

stronger emphasis on the Enablers.

Most organisations score below 500 in assessments using the European Quality Award

model. It is also a fact that software process maturity assessments have demonstrated

fairly low levels of software process capability in most organisations. Gibbs [16] notes

that as of 1994, of all the 261 organisations world-wide assessed using the CMM

model, a staggering 75% were at level 1 - the Initial level with no formal processes, no

measurements and no way to track their progress/failure; only two organisations world-

wide were found to be at the highest (Optimized) level of the CMM model. In another

recent survey, Brodman and Johnson [5] found that 64% of the companies in their

sample were at the lowest two levels of the CMM model and only 11% in the two

highest levels of the CMM model.

Against this background, the overall SE Scores tend to seem a bit optimistic - about

36% of the respondents score above 500. This could be caused by an upward bias due

to the limitations of self-assessment by the questionnaire format of the current survey.

The European Quality Award and CMM/Bootstrap assessments use a more rigorous

approach comprising several field visits and in-depth interviews by trained external

assessors [29] - aspects which are difficult to replicate in the current SE survey.

5.2 Overall Scores for Individual Enabler and Result Categories

Normalising each dimension of the SE model to a scale of 100, the average scores for

each category of Enablers and Results are shown in Figures 3 and 4 respectively.

Analogous to the future overall SE scores, the future scores for each Enabler and Result

category in Figures 3 and 4 are computed on the basis of the assigned importance to

each item constituting the scale for the respective category.

Figures 3 and 4 about here

Comparing the current and future "scores" it can be observed that the responding SPUs

assign a higher level of importance to all dimensions of Enablers and Results. More

importance is given in general to the Enablers as compared to the Results categories.

Also, all scores are clustered within relatively narrow ranges. This indicates a possible

lack of discriminatory focus in investments for enhancing SE - SPUs may not

necessarily be obtaining the right balance in their efforts along different categories of

Enablers and Results.

15

A more detailed discussion of our findings related to the salient management practices

comprising the Enabler and Result categories is presented in Section 7.

6 Validation of the Software Excellence Model

For the next phase of analysis, two subsets were selected from the responding

companies - twenty companies with the highest and lowest overall SE scores -

henceforth termed as the High Scorers and Low Scorers. Table 5 compares the average

future and current scores for the High and Low Scorers with all responding SPUs. The

Mann-Whitney U Test' [32] was used to confirm that there is a statistically significant

difference (p 0.001) in the overall SE scores between the High and Low Scoring

groups. A computation of the Mahalanobis distances [45,48] showed a very clear

distinction between the two groups with no incorrect classifications.

Table 5 about here

6.1 Discrimination by the Result Categories

Figure 5 depicts the scores for the different categories of Results for the High and Low

Scorers. A large difference can be observed between the two groups along all of the

Result categories. This indicates that the High Scorers have been more successful in

translating their investments in the Enablers to financial and non-financial business

results. The High Scorers have avoided falling in the trap which ailed many of the early

investments in TQM [11].

Figure 5 about here

Multiple discriminant function analysis was used to find the discriminatory power of

each Result category for distinguishing between the High and Low Scorers: Table 6

summarises the results. Overall, the discrimination is highly significant (Wilks' Lambda

= 0.091; F(4,35) = 87.25; p <0.000). As a Wilks' Lambda of 0 indicates perfect

discriminatory power, it can be concluded that the Result categories provide a robust

discrimination between the High and Low Scorers.

Table 6 about here

2 The Mann-Whitney U test is the most powerful non-parametric alternative to the west for independentsamples and is well suited to the sample sizes considered in our data set.

16

The Wilks' Partial Lambdas in Table 6 give the unique contribution of the respective

categories to the discrimination between the High and Low Scorers. We note that End-

user Satisfaction has the maximum discriminatory power between the two groups

(Wilks' Partial Lambda = 0.7; p = 0.0004). Thus if end-users of a SPU have a high

degree of satisfaction, then it is likely that the SPU is a High Scorer. The category

Business Results has the next highest discriminatory power (Wilks' Partial Lambda =

0.843; p = 0.01). This emphasises the importance of including bottom-line business

results in the SE model. It is interesting to note that SPU People Satisfaction also has

some discriminatory power (Wilks' Partial Lambda = 0.896; p = 0.05) between the

High and Low Scorers.

6.2 Discrimination by the Enabler Categories

Figure 6 depicts the scores for the different categories of Enablers for the High and

Low Scorers. A large difference can be observed between the two groups along all

Enabler categories.

Figure 6 about here

Multiple discriminant function analysis was also used to find the discriminatory power

of each Enabler category for distinguishing between the High and Low Scorers. Table 7

summarises the results. Overall, the discrimination is again highly significant (Wilks'

Lambda = 0.0828; F = 60.898; p <0.000) and the Enablers provide a robust

discrimination between the High and Low Scorers

Table 7 about here

Table 7 shows that the category SPU People Management has the maximum

discriminatory power (Wilks' Partial Lambda = 0.621; p = 0.000) between the High

and Low Scorers. The category End-user Management has a marginally lower

discriminatory power (Wilks' Partial Lambda = 0.657; p = 0.000) between the two

groups. The High Scorers are investing and succeeding in keeping both the developers

and customers of software satisfied to a higher degree. The categories of Leadership

(Wilks' Partial Lambda = 0.882; p = 0.043) and SPU Policy and Strategy (Wilks'

Partial Lambda = 0.891; p = 0.052) also have some discriminatory power between the

High and Low Scorers. These results show that senior management's role and actions

are relevant for influencing the level of SE.

17

It is interesting to note that the categories of Resource Management and Processes do

not discriminate between the High and Low Scorers. Thus the question can be raised

whether a narrow focus on Software Processes, as is common with software maturity

models such as CMM and SPICE, can lead to high levels of overall SE.

6.3 Analysis of Variance Based on Individual Enabler Categories

6.3.1 SPU People Satisfaction

Table 8 summarises the variance in SPU People Satisfaction explained by the different

Enabler categories individually. Not surprisingly, SPU People Management explains

the greatest amount of variance (46%) in SPU People Satisfaction when all respondents

are considered. All other Enabler categories also explain a significant amount of the

variance in SPU People Satisfaction - ranging from a high of 38% for Leadership to a

low of 31% for Resource Management.

Table 8 about here

The situation is different when one considers the High Scorers: although SPU People

Management explains a large proportion (71%) of the variance, Leadership now

explains the greatest amount (79%) of variance in SPU People Satisfaction. This

highlights the importance and need for effective leadership in guiding a SPU towards

higher levels of SPU People Satisfaction. The amount of variance explained by all other

Enabler categories also increases, though the increase is the smallest for the category

Processes (from 36% to 49%). It is interesting to note that the Enablers do not explain

much of the variation in SPU People Satisfaction for the Low Scorers.

While considering all responding SPUs and the High Scorers, the B values are all

positive and of a reasonably large magnitude. This indicates a direct positive impact of

each Enabler category on SPU People Satisfaction. Note that the B values are much

higher for the High Scorers than for all respondents (and the Low Scorers).

6.3.2 End-user Satisfaction

The variances in End-user Satisfaction explained by the different Enabler categories

individually are summarised in Table 9. Taking all responding SPUs into consideration,

Processes and End-user Management explain 39% and 37% of the variance in End-user

18

Satisfaction respectively. Next to these two Enablers, Leadership explains 24% of the

variance for all respondents. The proportion of variance explained by all Enabler

categories is low as compared to Table 8. While the low figures can be partly explained

by the diversity in the respondents (see Appendix A), it also indicates that the SE

questionnaire does not fully capture all causal factors for End-user Satisfaction

satisfactorily.

Table 9 about here

When only the High Scorers are considered, the amount of variance explained by

Processes, End-user Management and Leadership declines. However, all Enabler

categories (except for Resource Management) now explain a similar amount of variance

which ranges from a low of 20% for Leadership to a high of 27% for SPU Policy and

Strategy. Analogous to SPU People Satisfaction, the Enabler categories do not explain

any significant variance in End-user Satisfaction for the Low Scorers.

The B values are positive for all SPUs and the High Scorers. This indicates a direct

positive impact of each Enabler category on End-user Satisfaction. The B values for

Processes are the largest for all SPUs (0.63) and High Scorers (0.54). End-user

Management has the next largest B values, 0.62 for all SPUs and 0.51 for High

Scorers.

6.3.3 Impact on Organisation

Table 10 summarises the variances in Impact on Organisation as explained by the

different Enabler categories. When all respondents are included, SPU Policy and

Strategy explains 63% of the variance. SPU People Management and Resource

Management come in next explaining 52% and 50% of the variance respectively.

Table 10 about here

Similar to SPU People Satisfaction, the percentage of variance explained by all Enabler

categories increases when the High Scorers are considered. SPU People Management

and SPU Policy and Strategy now explain a very large part of the variance: 93% and

90% respectively. It is interesting to note that while the first five Enabler categories each

explain 72% or more of the variance in Impact on Organisation, the last category of

Enablers, Processes, only explains 47% of the variance in Impact on Organisation

(even though this is a relatively high figure for the diverse sample considered in this

19

research). These results indicate that overall excellence in the different Enablers do have

a beneficial impact on the organisation. While none of the Enablers explain a major part

of the variance in the Low Scorers for SPU People Satisfaction and End-user

Satisfaction, a significant part of the variance in Impact on Organisation for the Low

Scorers is explained by the different Enablers (except for Processes).

The B values are all positive and thus indicate a positive direct impact of each Enabler

category on Impact on Organisation. Note that the B values are much higher for the

High Scorers than for all respondents - ranging from a high of 0.96 for SPU People

Management to a low of 0.71 for Processes.

6.3.4 Business Results

Table 11 summarises the variances in Business Results as explained by the different

Enabler categories. End-user Management explains the largest (45%) variance in

Business Results for all respondents. Processes and Leadership explain 38% and 35%

of the variance in Business Results respectively.

Table 11 about here.

The proportion of variance explained increases for all Enablers when one considers

only the High Scorers - the figures range from a low of 44.% for Processes to a high of

65% for Resource Management. The uniformly high proportion of variance explained

by each Enabler shows that all Enabler categories have a significant impact on Business

Results. The Enablers do not explain any of the variance in the business results of the

Low Scorers.

• Again, the B values are all positive and thus indicate a positive direct impact of each

Enabler category on Business Results. Note that the B values are much higher for the

High Scorers than for all respondents and the Low Scorers.

6.4 Analysis of Variance Based on Combinations of Enablers

The variance explained in each Result category by a combination of two Enabler

categories and all Enabler categories was also analysed. The results of this analysis are

summarised in Table 12 for all respondents. There is no significant change in the

proportion of variance explained by either considering two Enabler categories at a time

or all Enabler categories at once. This is due to the relatively high correlations between

20

the different Enabler categories (see Appendix B). This stresses the need to further

ref= the SE model and the multiple-item scales used to measure each category in future

editions of the SE survey.

Table 12 about here

7 Discussion of Results

This section discusses the managerial implications of our research and relates our

findings to prior research.

7.1 Validity of the Software Excellence Model

A core assumption underlying the SE model is that a narrow focus on software

processes alone is insufficient to enable organisations derive business value from

software. The argument was made, that analogous to the evolution of TQM models, it

is necessary to expand the scope of software maturity models to the broader

organisational context. Thus, the range of Enablers was expanded to include five other

categories (beside Processes) which cumulatively account for 76% of the total

contribution of the Enablers to the SE score.

In this context, it is interesting to note that the Enabler category - Processes, which

represents the dominant emphasis within software maturity models, does not

discriminate between the High and Low Scorers (see Table 7). In contrast, the other

Enabler categories of End-user Management, SPU People Management, Leadership and

SPU Policy and Strategy all discriminate (to varying degrees) between the High and

Low Scorers. The benefits of using software maturity models to improve levels of

software maturity have been documented in the literature [9,20]. However, the results

of this study show that other aspects of the general organisational context are more

important in determining the overall level of software excellence achieved by an

organisation.

Relative to software maturity models, the SE model is unique in explicitly including the

value accrued to key organisational stakeholders. The underlying theory is that due

attention to the setting up of the right organisational context for the development and

application of software will lead to good business results as measured by the different

Result categories. The analysis of variance results summarised in Tables 8 through 12

show that a large proportion of the variance in the responding SPUs is explained by the

21

different Enabler categories, either individually or jointly. The proportion of variance

explained by the Enablers goes up significantly (except for End-user Satisfaction) when

only the High Scorers are considered. These results demonstrate a strong association

between investments in Enablers and the derivation of relevant business value and

justify the inclusion of both Enabler and Result categories in the SE model.

Figure 7 about here

Figure 7 provides a graphical summary of the Enabler categories which individually

explain the largest and least amount of variance in the Result categories for the High

Scorers. It is striking to note that the Enabler category Processes, explains the least

amount of variance in three Result categories: SPU People Satisfaction, Impact on

Organisation and Business Results. This again raises questions about the usefulness of

the narrow focus on software processes prevalent within software maturity models.

Figure 7 also shows that the other Enabler categories, Leadership, SPU Policy and

Strategy, SPU People Management and Resource Management play important roles in

explaining the variance in the Results. This reinforces the utility of including aspects of

the general organisational context within the SE model.

7.2 Robustness of the Results

The results of our research show strong support for the SE model. The utility of

including the general .organisational context in which software is developed and applied

has been demonstrated. Also, a strong association between the Enabler and Result

categories has been observed.

The SE model has been adapted from the European Quality Award model and it is likely

that the model will need further refinement. While the results of this research do not

support any obvious additions or deletions of Enabler/Result categories, the constituent

elements of the different categories will need additional careful attention. In particular,

the results of Table 9 suggest that aspects which explain End-user Satisfaction may not

be adequately captured within the current SE model/questionnaire.

The proportion of variance explained in the Result categories (see Tables 7 through 11)

is not very high when all respondents are considered. This could be attributed to the

diversity inherent in the sample of respondents (see Appendix A). However, it is

interesting to note that the explained variance increases significantly when only the High

Scorers are considered. This indicates that there are some common distinct features of

22

organisations with high levels of SE regardless of their underlying differences in terms

of sectors, countries, sizes and business foci.

7.3 Salient Management Practices

An analysis of the responses to the SE survey highlighted several salient aspects of

current management practices within the respondents. We restrict the discussion below

to key aspects of management practices related to the role of senior management,

partnerships with end-users, management of SPU personnel and software processes.

The quantitative numbers mentioned below within brackets following particular

statements are in either of the following formats: (AR) or (HS;AR;LS), where HS, AR

and LS are the average scores on a scale of 1 to 5 (see Table 4) for the High Scorers,

All Respondents and Low Scorers respectively for•the corresponding statement in the

SE survey.

7.3.1 Role of Senior Management

In only about half the responding SPUs, the strategy and mission of the SPU is set

personally by senior management, (2.7). A large number of SPUs said that progress

towards achieving software excellence is retarded by the fact, that senior management

display a lack of commitment and do not "walk the talk" with appropriate follow-up

actions (2.5). Few respondents felt that there is a method for evaluating whether the

SPU goals are attainable and if they fit the strategy of the parent business unit (2.2).

The above fmdings are consistent with these observations from the literature which has

documented the lack of adequate involvement of senior management in the planning and

execution of technology strategy. Business managers frequently do not consider

technology to be an area in which they needed to get involved personally [13].

Jarvenpaa and Ives [26] mention that "few nostrums have been prescribed so

religiously and ignored as regularly as executive support in the development and

implementation of management information systems" (p. 205). Lederer and Mendelow

[30] note that their research has shown that "top management still needs to be convinced

of the potential strategic impact of information systems" (p. 525).

However, a difference can also be observed between the High and Low Scorers in the

more intimate involvement of the senior management in setting the basic strategy and

mission of the SPU (3.3;2.7;2.1). Within the high scoring SPUs, senior management

take more active steps to instil a culture of SE within the SPU and the parent business

unit (3.2;2.7;2.2), and show their own commitment to it by their actions (3.1;2.5;1.9).

23

These views are echoed by Jarvenpaa and Ives [26], who found from a survey of fifty

five CEOs that those CEOs who participated in the management of IT were more

involved in it and that this in turn led to their firm being more progressive in the use of

IT. While senior management is more involved in the High Scorers, there is room for

improvement - they do not score higher than 3.5 for any management practice

7.3.2 Partnerships with End-users

The importance and utility of having end-users participate in the software development

process for defining requirements and specifications are well documented in the

literature [8,18,25]. Keil and Cannel [27] note that the issue "is not whether customers

should participate in the development process, but how they should participate" (p. 34).

Beyer and Holtzblatt [3] have emphasised the importance of a close collaborative

relationship between the software design team and the customer. A number of

customer-centred software development approaches such as participatory design [8]

have been proposed in the literature [3] in recent years.

Thus, it is not surprising to find that on average, the responding SPUs have created

procedures to create partnerships with end-users in order to learn about their needs and

concerns (2.9). However these partnerships are, in general, not very effective.

Procedures for obtaining regular feedback from end-users (2.5) 'and for reviewing the

scope and coverage of relationships with them are weak (2.3). As a result, SPUs do not

have a realistic overview of all customer complaints (2.5). While SPUs collect customer

data, they have neither established the relevance of these measures for customer

satisfaction (1.9) nor benchmarked them against comparable external organisations

(1.7).

The High Scorers fare much better than the average for all surveyed management

practices related to end-users. Their partnerships with end-users are more effective

overall (3.9;2.9;2.3). This is partially because personnel from all levels and functions

are actively involved in partnerships with end-users (3.4;2.6;2.0). This helps to create

multiple links between the SPU and its user community. Keil and Cannel [27] have

found that more successful projects have more links with end-users/customers than less

successful projects. The greater the number of indirect and direct links with end-

users/customers, the greater the exchange of information between developers and end-

users and the more successful the projects. In addition, the High Scorers have formal

processes for obtaining regular feedback from end-users (3.5;2.5;1.7) and for

systematic reviews and updates of the scope and coverage of the partnerships

(3.2;2.3;1.5). They have a better overview of all customer complaints and are more in

24

tune with the needs of their customers (3.2;2.5;1.8). It is useful to note that while the

High Scorers fare much higher than the average for all end-user related management

practices, their scores for each practice is less than 4.

7.3.3 Management of SPU Personnel

The effective management and satisfaction of SPU personnel is an area of general

weakness for the respondents. There is little progress in making SPU personnel more

empowered to act and take responsibility without increasing business risk (2.2). Most

SPUs do not regularly measure factors (such as staff turnover) which influence or

predict their personnel satisfaction (2.1). The career development plans for SPU

personnel are not adequately linked to the business plans of the SPU (2.5) and the

recognition and reward of their efforts are performed informally (2.3). Few SPUs have

succeeded in involving their employees in generating ideas for continuous

improvement, either individually or in groups (2.6). It is also interesting to note that in

most SPUs, management does not publicise results of SPU personnel perceptions and

act on them accordingly (1.9).

The High Scorers fare better, but not significantly more, than the average with respect

to SPU people management. They emphasise the empowerment of their personnel

(3.0;2.2;1.6) and involve them both individually and within groups in generating

improvements (3.3;2.6;2). They tend to have a more open organisation in which

management seeks out the perceptions of SPU personnel (3.4;2.6;1.9). However, they

do not score more than 3.5 for any people-management practice.

The above findings are again supported by the literature. While the concepts of

empowerment and career development have been researched extensively by

organisational researchers, "little attention has been devoted to exploring job

involvement and its relationship to the work experiences and job attitudes of IS

personnel" [24, pp. 176-177]. Prior research [17] has demonstrated that overall job

satisfaction is the primary motivator for SPU personnel to be organisationally

committed. SPU personnel have evolved over the years as a large and distinct group of

organisational employees who through the implementation of new technologies directly

and indirectly impact the consciousness and practices of other organisational employees

[24,37]. Without their commitment, it is unlikely that an organisation can leverage

technology successfully. Thus, a number of researchers have "identified effective

human resource management as a critical issue facing the IS field and have called

attention to the need for improving human resource planning, recruitment, and

development of IS employees" [24, p. 175].

25

7.3.4 Management of Software Processes

Relative to the other categories of Enablers, the responding SPUs score higher along

the Processes dimension. This reflects both the process-focus within software maturity

models and their increasing use within industry [23,38,39]. While project management

procedures are commonly adopted in most surveyed SPUs (3.0), few report having

systematic processes to evaluate and manage project related risks (2.5). The use of

metrics for managing software processes is poor (2.2) and this is confirmed in a recent

study on the adoption of software management best practices within Europe [14]. The

responding SPUs emphasise the structuring of processes to code and test software

(2.9) and the creation of a detailed software design is a routine part of the development

process (3.0). However, there is. little emphasis on the systematic reuse of software

components (1.9). This is supported by the literature [14].

The High and Low Scorers are similar with respect to coding, unit testing and software

integration practices. Project management practices are also equally emphasised in both

high and low scoring SPUs except with respect to risk assessment and the use of

metrics. High Scorers have established processes by which project risks are evaluated

and take steps to manage these risks continuously (3.0;2.5;1.9); they also collect and

analyse a more extensive set of software-related metrics to improve project performance

(3.0;2.2;1.7).

8 Conclusion

Software is an important enabler for organisations to exploit new distribution channels,

create new products and deliver differentiated value-adding services to customers.

Given the rapid progress of technology and the increasing inter-dependencies between

an organisation's business and software strategies, the ability to successfully develop

and leverage software is critical for the competitiveness of organisations.

Organizations have traditionally faced two major challenges with respect to software.

First, software development processes have been notoriously immature and examples

of dramatic cost and time overruns in software projects are common in the literature

[16]. Second, there has often been a fundamental mis-alignment between an

organisation's software and business strategies [4,13,19] leading to discontent and

frustration with inadequate returns from investments in technology.

26

Prior research in the domain of software engineering has largely focused on the former

challenge. Consequently, several software maturity models such as the CMM [39] and

SPICE [46], have been proposed with the aims of assessing and improving the maturity

of software development processes. The literature [9,20] provides examples of how

these software maturity models have been used by organisations to the identify

strengths and limitations of, and then improve software development processes.

Little has been done in the literature to define models which extend the software process

focus of software maturity models to the broader organisational context with the

objective of helping organisations obtain adequate business value from software.

Guidance in this direction can be taken from the domain of Total Quality Management,

where over the last four decades, quality models have progressively expanded in two

directions. First, they have increased their scope from the "factory" to the entire

organisation. Second, their emphases have shifted from a dominant "quality of product"

perspective to a "quality of management" perspective leading to overall business

excellence.

This research has put forward the concept of "Software Excellence" to capture the

ability of an organisation to (a) create the right organisational context for developing and

applying software and (b) derive appropriate business value to all key stakeholders -

shareholders, end-users, software-related employees and the organisation at large.

Adapted from the European Quality Award model [29], the SE model includes ten

different categories which have been grouped into two groups: Enablers and Results.

Enablers focus on how an organisation creates the appropriate context for Software

Excellence and consist of the following categories: Leadership, SPU Policy and

Strategy, SPU People Management, End-user Management, Resource Management and

Processes. Results describe the value accrued from software to key organisational

stakeholders and comprise SPU People Satisfaction, End-user Satisfaction, Impact on

Organisation and Business Results.

The SE model extends current research in software maturity models in the following

manner. First, it includes an explicit focus on the achievement of both financial and

non-financial business results for the organisation from software. Current software

maturity models ignore business results and thus can lead organisations into the pitfall

of inadequate returns faced by organisations investing in quality about a decade ago

[11,15]. The Result and Enabler categories have equal importance within the SE model

and thus it is impossible for an organisation to achieve a high level of SE without good

results from software-related activities. Second, the SE model includes far more of the

rich organisational context in which software is developed and applied than current

27

software maturity models. Software Processes, the pre-dominant focus of software

maturity models such as the CMM, account for only 12% of the entire SE score.

We have performed an empirical validation of the SE model through a questionnaire-

based survey of European organisations. Our survey results show that there is a strong

association between the Enablers and Results. A significant proportion of the variance

in the individual Result categories can be explained by the different Enabler categories

(see Section 6.3). These results show that investment in the Enabler categories is

associated positively with increased value in the Result categories. The necessity of

increasing the scope of the SE model from a narrow software process focus to the

broad organisational context is validated by the observation that the Enabler category,

Processes, does not discriminate between organisations with high and low levels of

overall SE. In contrast, the other Enabler categories of End-user Management, SPU

People Management, Leadership and SPU Policy and Strategy all discriminate between

the High and Low Scorers.

The overall scores of SE show that "Software Excellence", i.e., excellence in all aspects

of the creation and application of software in the general organisational context, remains

a distant objective for SPUs. Despite the lack of rigorous assessment in the

questionnaire format of the current survey, a large majority of the surveyed SPUs score

well below 500 (see Figure 2). In particular, the scores of the Result categories are

lower than for those for the Enablers. This shows that additional efforts are needed to

create more value for the key stakeholders of a SPU.

The results of the SE survey has also identified strengths and limitations of the

surveyed SPUs. Issues relating to senior management leadership, SPU personnel

management and end-user management need more attention. Senior management need

to get more involved in setting and communicating SPU strategy, and should reflect

their commitment to SE by their own actions. The aspirations and careers of SPU

personnel need to be managed more actively in order to increase their level of job

satisfaction and dedication to the organisation. While many SPUs are taking steps to

create partnerships with end-users, it is important the increase the degree of

effectiveness of these partnerships.

When the SE model is used to differentiate between SPUs with high and low overall SE

scores, it is clear that the high scoring SPUs score higher for all management practices

than their low scoring counterparts. However, their scores for all management practices

are relatively low (rarely above 3.5 and never above 4) and this highlights the large

scope for further improvement in the surveyed SPUs. It is also interesting to note that

28

aspects related to end-user management, SPU personnel satisfaction and senior

management leadership discriminate between the High and Low Scorers to a much

higher degree than the maturity of their respective software processes.

While the SE model shows promise in setting a new benchmark for assessing SE and

identifying areas for improvement for SPUs, some caution is necessary in interpreting

the results. The survey sample is relatively diverse - from many different countries and

sectors. One should note that the results presented in this report are for the aggregate

sample and thus may only be partially valid for a particular country or sector. Also, as

this was the first time that the SE model was proposed and used, the management

practices used to assess the different Enabler and Result categories may need to be

refined further. It is planned to conduct annual SE surveys in collaboration with the

European Software Institute to give a better feel for both the constituent elements of the

SE model and the true level of SE in European organisations.

Acknowledgements

The authors would like to thank Ashis Bhattacharya and Amit Pathare for conducting

the analyses for this research, and Santiago Rementeria and other colleagues at the

European Software Institute, Bilbao for help with the design and implementation of the

Software Excellence Questionnaire.

29

References

1. A. Barua, C. Kriebel, and T. Mukhopadhyay, "Information Technology and

Business Value: An Analytic and Empirical Investigation", University of Texas at

Austin Working Paper, Austin, TX, May 1991.

2. Bell Canada, "Trillium - Telecom Software Product Development Capability

Assessment Model", Draft 2.1, July 1992.

3. H.G. Beyer and K. Holtzblatt, "Apprenticing with the Customer", Communications

of the ACM, Vol. 38, No. 5, pp. 45-52, May 1995.

4. M. Broadbent and P. Weill, "Improving Business and Information Strategy

Alignment: Learning from the Banking Industry", IBM Systems Journal, Vol. 32,

No. 1, 1993, pp. 162-179.

5. J.G. Brodman and D.L. Johnson, "Return on Investment (ROI) from Software

Process Improvement as Measured by US Industry, Software Process

Improvement and Practice", Pilot Issue, pp. 35-47, 1995.

6. E. Brynjolfsson, "The Productivity Paradox of Information Technology",

Communications of the ACM, 35, pp. 66-77, 1993.

7. J.L. Cecil and E.A. Hall, 'When IT Really Matters to Business Strategy", The

McKinsey Quarterly, 2-26, Autumn 1988.

8. A. Clement and P. Van den Besselaar, "A Retrospective Look at PD Projects",

Communications of the ACM, Vol. 36, No.4., pp. 29-37, June 1993.

9. M. Craigmyle and I. Fletcher, "Improving IT Effectiveness through Software

Process Assessment", Software Quality Journal, 2, pp. 257-264, 1993.

10. P.B. Crosby, Quality is Free, Mc-Graw Hill, New York, 1979.

11. X. de Groote, C.H. Loch, L. Van der Heyden, Luk N. Wassenhove and E.

Yucesan, "Measuring Management Quality in the Factory", forthcoming European

Management Journal, 1996.

12. A. Dorling, "SPICE: Software Process Improvement and Capability

determination", Software Quality Journal, 2, pp. 209-224, 1993.

13. S. Dutta, "Linking IT and Business Strategy: The Role and Responsibility of

Senior Management", European Management Journal, 14:3, pp.255-268, 1996.

14. S. Dutta, S. Kulandaiswamy and L. N. Van Wassenhove, "Benchmarking

European Software Management Best Practices", INSEAD Working Paper

96/45/TM, 1996.

15. D.A. Garvin, "How the Baldrige Award Really Works", Harvard Business

Review, pp. 80-95, Nov-Dec 1991.

16. W.W. Gibbs, "Software's Chronic Crisis", Scientific American, pp.72-81, Sep.

1994.

30

17. T. Guimaraes and M. Igbaria, "Determinants of Turnover Intentions: Comparing IC

and IS Personnel, Information Systems Research", 3:3, pp. 273-303, Sep. 1992.

18. J. Hartwick and H. Barki, "Explaining the Role of User Participation in

Information System Use", Management Science, 40:4, pp. 440-465, April 1994.

19. J. Henderson and N. Venkatraman, "Strategic Alignment: Leveraging Information

Technology for Transforming Organizations", IBM Systems Journal, 32:1, pp. 4-

16, 1993.

20. J. Herbsleb, A. Carleton, J. Rozum, J. Seigel and D. Zubrow, "Benefits of CMM

Based Software Process Improvement: Initial Results", Report CMU/SEI-94-TR-

13, Software Engineering Institute, Pittsburgh, 1994.

21. R. Heygate, "IT for CEOs", The McKinsey Quarterly, pp. 54-63, Winter 1990.

22. W.S. Humphrey and W.L. Sweet, "A Method for Assessing the Software

Engineering Capability of Contractors", Software Engineering Institute, CMU/SEI-

87-TR-23, DTIC Number ADA187320, 1987.

23. W.S. Humphrey, "Managing the Software Process", Reading, MA: Addison-

Wesley, 1989.

24. M. Igbaria, S. Parasuraman and M.K. Badawy, "Work Experiences, Job

Involvement, and Quality of Work Life Among Information Systems Personnel",

MIS Quarterly, pp. 175-201, June 1994.

25. B. Ives and M.H. Olson, "User Involvement and MIS Success: A Review of

Research", Management Science, Vol. 30, No. 5, pp. 586-603, May 1984.

26. S.L. Jarvenpaa and B. Ives, "Executive Involvement and Participation in the

Management of Information Technology", MIS Quarterly, 15:2, pp. 205-227, June

1991

27. M. Keil and E. Carmel, "Customer-Developer Links in Software. Development",

Communications of the ACM, Vol. 38, No. 5, pp. 33-44, May 1995.

28. P. Kuvaja, J. Simila, I. Krzanik, A. Bicego, G. Koch and S. Saukkonen,

Software Process Assessment and Improvement: The BOOTSTRAP Approach,

Blackwell Business, Oxford, UK and Cambridge, MA, 1994.

29. D. Lascelles and R. Peacock, Self-Assessment for Business Excellence, London,

Mc-Graw Hill, 1996.

30. A.L. Lederer and A.L. Mendelow, "Convincing Top Management of the Strategic

Potential of Information Systems", MIS Quarterly, 12:4, pp. 525-534, December

1988.

31. G.W. Loveman, "An Assessment of the Productivity Impact on Information

Technologies", in T.J. Allen and M.S. Scott-Morton (Eds.), Information

Technology and the Corporation of the 1990s: Research Studies, MIT Press,

Cambridge, MA, 1994.

31

32. H.B. Mann and Whitney, D.R., "On a Test of Whether One of Two Random

Variables is Stochastically Larger than the Other", Annals of Mathematical Statistics,

18 (1947), pp. 50-60.

33. F.W. McFarlan, "Information Technology Changes the Way you Compete",

Harvard Business Review, pp. 98-103, May-June 1984.

34. J.P. McIver and E.G. Carmines, Unidimensional Scaling, No. 24 in Quantitative

Applications in the Social Sciences, Beverly Hills, Sage Publications, 1981.

35. A.C. McKennel, "Attitude Scale Construction", in C.A. O'Muircheartaigh and C.

Payne (Eds.), The Analysis of Survey Data, Vol. I, New York: Wiley, 1977.

36. National Institute of Standards and Technology, "The Malcolm Baldrige National

Quality Award 1995 Award Criteria", NIST, Gaithersburg, 1995.

37.W.J. Orlikowski and J.J. Baroudi, "The Information Systems Profession: Myth or

Reality?", Office, Technology and People, 4:1, pp. 13-30, January 1989.

38. M.C. Paulk, B. Curtis, M.B. Chrissis and C.V. Weber, Capability Maturity Model

for Software, Version 1.1, Software Engineering Institute, CMU/SEI-93-TR-24,

DTIC Number ADA262403, 1993.

39. M.C. Paulk, "The Evolution of the SEI's Capability Maturity Model for Software",

Software Process-Improvement and Practice, Pilot Issue, 3-15, 1995.

40. M.E. Porter and V.E. Millar, "How Information Gives You Competitive

Advantage", Harvard Business Review, pp. 149-160, July-August 1985.

41. J.B. Quinn, J.J. Baruch and K.A. Zien, "Software-Based Innovation", Sloan

Management Review, pp. 11-24, Summer 1996.

42. N. Rackoff, C. Wiseman and W.A. Ullrich, "Information Systems for Competitive

Advantage: Implementation of a Planning Process", MIS Quarterly, pp. 285-294,

December 1985.

43. R.A. Radice, J.T. Harding, P.E. Munnis and R.W. Phillips, "A Programming

Process Study", IBM Systems Journal, 24:2, 1985.

44. J.F. Rayport and J.J. Sviokla, "Managing in the Marketspace", Harvard Business

Review, pp. 141-150, Nov.-Dec. 1994.

45. P.H. Rossi, J.D. Wright and A.B. Anderson, Handbook of Survey Research,

Academic Press, 1983.

46. T.P. Rout, "SPICE: A Framework for Software Process Assessment", Software

Process - Improvement and Practice, Pilot Issue, 57-66, 1995.

47. M. Scott Morton, (Ed.), The Corporation of the 1990s: Information Technology

and Organizational Transformation, Oxford University Press, New York, 1991.

48. Statistica Reference Manuals, StatSoft Inc., Tulsa, OK, 1994.

49. "The End of Delegation? Information Technology and the CEO", Harvard Business

Review, pp. 161-172, Sep. - Oct. 1995.

32

List of Key Phrases

Software Maturity Assessment; Software Process Maturity; Software Excellence;Business Value from Software; European Software Survey

33

Leadership

■■■

PeopleManagement

End-userManagement

SPU Policy& Strategy

Resources

Processes

People

Satisfaction

End-user

Satisfaction

Impact on

Organization

Business

Results

ENABLERS RESULTS

Figure 1: The Software Excellence Model

34

35In'V 300-0g 25ci.: 20ce

",5 15

L0 1 012E=z

cr) 0) alcy) C) C)`■' m nrO 6 6o o oNI m nr

Overall Software

Figure 2: Distribution of Overall Software Excellence Scores (Current and Future)

35

■ FutureD Current

SoftwareProcesses

Resourceeu Management

End-user•4./ Management

SPU PeopleManagement

W SPU Policy andu.,Strategy

Leadership

0 20 40 60

80Scores (Out of 100)

Figure 3: Average Scores each Category of Enablers

36

BusinessResults

N•

O Impact on• Organization

End-userSatisfaction

cc

SPU PeopleSatisfaction

■ Future▪ Current

0 20 40 60

80

Scores (Out of 100)

Figure 4: Average Scores each Category of Results

37

■ Low Scorers• High Scorers

BusinessResults

Impact onva OrganizationiaS.flien

°C End-userSatisfaction

SPU PeopleSatisfaction

0

20 40 60

80Scores (Out of 100)

Figure 5: Current Scores of Result Categories for High and Low Scorers

38

■ Low ScorersO High Scorers

SoftwareProcesses

ResourceManagement

End-userCA....

O Management7:11(VC SPU People

ILIManagement

SPU Policy andStrategy

Leadership

0 20 40 60

80

Scores (Out of 100)

Figure 6: Current Scores of Enabler Categories for High and Low Scorers

39

SPU PeopleSatisfaction

(EC

SI3U PeopleManagement

nd-userManagement

93

16°6'). / I

End-userSatisfaction

Impact onOrganization

(Resource. Management

CProcesses

47°/,65° Business

Results44%

Leadership 79%

Largest amount of variance (%) explainedSecond largest amount of variance (%) explained

ON- Smallest amount of variance (%) explained

Figure 7: Summary of Variances Explained for Results by Individual EnablerCategories for High Scorers

40

Maturity Level Key Process Areas

Level 5 • Defect prevention• Technology change management• Process change management

Level 4 • Quantitative process management• Software quality management

Level 3 • Organisation process focus• Organisation process definition• Training program• Integrated software management• Software product engineering• Intergroup coordination• Peer reviews

Level 2 • Requirements management• Software project planning• Software project tracking and oversight• Software subcontract management• Software quality assurance• Software configuration management

Level 1 None

Table 1: Key Process Areas within Different Maturity Levels of the CMM Framework

41

Process Area List of Processes

Customer-Supplier CUS.1 Acquire Software Product and/or ServiceCUS.2 Establish ContractCUS.3 Manage Customer Requirements and RequestsCUS.4 Perform Joint Audits and ReviewsCUS.5 Package, Deliver, and Install the SoftwareCUS.6 Support Operation of SoftwareCUS.7 Provide Customer ServiceCUS.8 Assess Customer Satisfaction

Engineering ENG.1 Develop System Requirements and DesignENG.2 Develop Software RequirementsENG.3 Develop Software DesignENG.4 Implement Software DesignENG.5 Integrate and Test SoftwareENG.6 Integrate and Test SystemENG.7 Maintain System and SoftwareENG.8 Perform Peer Reviews

Project PRO.1 Plan Project Life CyclePRO.2 Establish Project PlanPRO.3 Build Project TeamsPRO.4 Coordinate Project TeamsPRO.5 Manage RequirementsPRO.6 Manage QualityPRO.7 Manage RisksPRO.8 Manage Resources and SchedulePRO.9 Manage Subcontractors

Support SUP.1 Develop DocumentationSUP.2 Perform Configuration ManagementSUP.3 Perform Quality AssuranceSUP.4 Perform Problem Resolution

Organisation ORG.1 Engineer the BusinessORG.2 Define the ProcessORG.3 Improve the ProcessORG.4 Perform TrainingORG.5 Enable ReuseORG.6 Provide Development EnvironmentORG.7 Provide Work Facilities

Table 2: List of Process Categories and Processes within the SPICE Framework

42

Category(Points assigned) _

Examples of Aspects Considered

Leadership • Role of senior management in setting and communicating strategy and(90) mission of SPU;

• Steps taken by senior management to create a culture of SE;• Review of progress towards SE by senior management; and• Commitment of senior management towards SE with specific actions.

SPU Policy and • Role of SE in setting mission, values and strategy of SPU;Strategy • Use of information about internal and supplier performance in(60) planning SPU operations;

•Methods for evaluating the attainability and degree of fit of SPU'sgoals with those of the parent business unit; and• Knowledge of SPU's goals and plans among SPU employees.

SPU People • Established processes for employee appraisal and career development;Management • Involvement of employees in continuous improvement;(70) • Empowerment of employees;

• • Effective communication between employees and management; and• Perceptions of employees about the value, of their oyinions.

End-user • Steps to create active partnerships with end-users;Management . • Involvement of different personnel in partnerships with end-users;(90) • Formal processes to obtain feedback on partnerships with end-users;

• Review of scope and coverage of partnerships with end-users; and• Appropriate action on feedback from end-users.

Resource • Allocation of financial resources based on SE goals and strategy;Management • Reliability and availability of all information for decision making;(70) • Efficient use of sub-contractors; and

• Identification and appropriate use of emerging technologies.Software Processes • Definition of and management of software processes;(120) • Formal processes for obtaining software from external agencies;

• Conversion of customer requirements into a software architecture;• Formal project management process with estimates and schedules;

. • Evaluation of and management of project-related risks;• Formal process for change management;• Collection and analyses of software metrics;• Software quality control.

I i

SPU People •Measurement of factors influencing people satisfaction;Satisfaction • Process for employees to give their perceptions and feedback; and(90) • Actions by management upon feedback from employees.End-user Satisfaction • Realistic overview of all end-user complaints from all areas;(200) • Regular evaluation of key measures of end-user satisfaction; and

• Benchmarked superiority of results vis-a-vis competitors.Impact on • Evaluation of impact of SPU on parent business unit;Organisation • Use of SPU's capabilities to enhance parent unit's performance; and(60) • Recognition of SPU's contribution by management of parent unit.

, Business Results • Measurement of impact of SPU on parent unit's financial results;(150) •Measurement of impact of SPU on parent unit's non-financial results;

• Benchmarked superiority of results vis-a-vis competitors; and_ • Communication of financial and non-financial results to all personnel.

Table 3: Selected Aspects of Enablers and Results

43

Instructions (common for all questions of the Enablers categories)

In the following list we identify a number of factors which are enablers of the SoftwareExcellence Strategy of the SPU. On the left-hand side, please indicate the number thatbest indicates the current practice levels of your SPU. The numbers representing thecapability levels should be interpreted in the following manner:

1- Absent not performed; perhaps some good ideas but not much progress onimplementation.

2 - Performed Informally: not rigorously performed; performance depending onthe skill and effort of individuals

3 - Institutionalised: performance according to well-defined procedures isformalised across the organisation;

4 - Controlled and Managed: subject is well understood; performance measuresare systematically collected and analysed; performance is objectivelymanaged for improvements.

5 - Optimised: an outstanding result that is universally implemented and servesas a role model achievement.

On the right hand side, please indicate the number that best indicates the importance ofeach item to your SPU over the next three years. On a scale from 1 to 5, 1 representsthe lowest degree of importance and 5 the highest degree.

SPU People Satisfaction

1 2 3 4

2 3 4

1 2 3 4

2 3 4

2 3 4

1 2 3 4

2 3 4

1 2 3 4

1 2 3 4

1 2 3 4

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

1 2 3 4 5

5 Does the SPU derive its peopleplans directly and formally from itsbusiness plans?

5 Is there an established and generallyaccepted process in the SPU forperiodic employee appraisalsincluding career development andtraining?

5 Are all SPU personnel both asindividuals and groups involved ingenerating improvements?

5 Can the SPU show that SPUpersonnel are becoming moreempowered to act and takeresponsibility without increasingbusiness risk?

5 Does the SPU recognise and rewardeffort towards software excellence atthe same level as other factors likequalifications or service?

5 Has the SPU management achievedeffective two-way communicationwith the SPU personnel?

5 Do the SPU personnel feel that theyare informed about activities in theSPU and do they feel that theiropinions are valued?

Table 4: Sample questions from the Software Excellence Questionnaire

44

Current Scores

All Respondents High Scorers Low ScorersMean Standard

deviationMean Standard

deviationMean Standard

deviationOverall SEScore(1000)

473 125 653 56 321 35

Enablers(500)

256 62 335 37 181 24

Results(500)

217 70 317 34 139 23

Future Scores

Overall SEScore(1000)

657 106 741 67 589 106

Enablers(500)

350 48 381 43 322 50

Results(500)

307 68 359 36 266 65

Table 5: Software Excellence Scores Summary

45

Wilks' Lambda for model = 0.09115F(4,35) = 87.250 N = 40 p < 0.0000

Variables Wilks' Lambda Wilks' PartialLambda

p-level

SPU PeopleSatisfaction

0.101 0.896 0.0516

End-userSatisfaction

0.130 0.700 0.0004

Impact onOrganisation

0.107 0.854 0.7674

Business Results 0.108 0.843 0.0154

Table 6: Discriminant Function Analysis Summary for Results Between High andLow Scorers

46

Wilks' Lamda for model = 0.08283F(6,33) = 60.898 N = 40 p < 0.0000

Variables Wilks' Lambda Wilks' PartialLambda

p-level

Leadership 0.094 0.882 0.043

SPU Policy &Strategy

0.092 0.891 s 0.052

SPU PeopleManagement

0.133 0.621.

0.000.

End-userManagement

0.126 0.657 0.000

ResourceManagement

0.084 0.987 0.513

Processes 0.085 0.973 0.352

Table 7: Discriminant Function Analysis Summary for Enablers Between High' andLow Scorers

47

Adjusted R2 B values

AllSPUs

HighScorers

LowScorers

AllSPUs

HighScorers

LowScorers

Leadership 0.38 0.79 0.25** 0.63 0.90 0.53

SPU Policy and Strategy 0.34 0.70 0.19** 0.59 0.85 0.48

SPU People Mngt. 0.46 0.76 0.14* 0.68 0.88 0.42

End-user Mngt. 0.35 0.64 -0.03 0.60 0.81 0.12

Resource Mngt. 0.31 0.71 0.05 0.56 0.85 0.31

Processes 0.36 0.49 -0.04 0.61 0.72 -0.12

*p5_ 0.05; ** p�. .01; all others p 0.001

Table 8: Variance explained in SPU People Satisfaction

48

Adjusted le B values

AllSPUs

HighScorers

LowScorers

AllSPUs

HighScorers

LowScorers

Leadership 0.24 0.20 0.11* 0.45 0.49 -0.4

SPU Policy and Strategy 0.17 0.27 0.01 0.43 0.51 -0.25

SPU People Mngt. 0.21 0.22 0.13 0.47 0.51 -0.42

End-user Mngt. 0.37 0.22 -0.04 0.62 0.51 0.11

Resource Mngt. 0.15 0.16

'

-0.05 0.40 0.44 -0.05

Processes 0.39 0.26 -0.05 0.63 0.54 0.07

*p.� 0.05; ** 1; all others p 0.

Table 9: Variance explained in End-user Satisfaction

49

Adjusted R2 B values

AllSPUs

HighScorers

LowScorers

AllSPUs

HighScorers

LowScorers

Leadership 0.37 0.84 0.32** 0.62 0.92 0.60

SPU Policy and Strategy 0.63 0.90 0.75** 0.79 0.95 0.88

SPU People Mngt. 0.52 0.93 0.67** 0.72 0.96 0.83

End-user Mngt 0.41 0.81 0.36** 0.65 0.90 0.63

Resource Mngt 0.50 0.72 0.57** 0.71 0.86 0.77

Processes 0.27 0.47 0.01 0.53 0.71 0.24

*p5_ 0.05; ** p5.0.01; all others p 0.001

Table 10: Variance explained in Impact on Organisation

50

Adjusted R2 B values

AllSPUs

HighScorers

LowScorers

AllSPUs

HighScorers

LowScorers

Leadership 0.35 ' 0.52 -0.04 0.60 0.74 0.13

SPU Policy and Strategy 0.31 0.56 -0.02 0.57 0.76 0.17

SPU People Mngt. 0.32 0.54 -0.02 0.57 0.75 0.18

End-user Mngt. 0.45 0.54 0.06 0.67 0.75 0.32

Resource Mngt. 0.30 0.65 0.05 0.55 0.82 0.31

Processes 0.38 0.44 0.003 0.63 0.68 0.23

*p� 0.05; ** p5. .01; all others p 0.00

Table 11: Variance explained in Business Results

51

Result Category Enabler Category(ies) Proportion of VarianceExplained

Best Models with two Enabler Categories (All Respondents)

SPU People Satisfaction SPU People ManagementProcesses

47.45%

End-user Satisfaction End-user ManagementProcesses

42.51%

Impact on Organisation SPU Policy and StrategySPU People Management

63.67%

Business Results End-user ManagementProcesses

46.84%

Models Considering all Enabler Categories (All Respondents)

SPU People Satisfaction All Enablers 46.2%

End-user Satisfaction All Enablers 41.5%

Impact on Organisation All Enablers 64.3%

Business Results All Enablers 47.3%

Note: p5. 0.01 for all numbers

Table 12: Variance Explained in Result Categories by Combinations of Enablers

52

Appendix A: Profile of Respondents

A breakdown of the home-countries of the respondents is as follows: France (12),Ireland (12), Germany (10), Finland (8), Switzerland (8), Great Britain (7), Spain (6),Sweden (4), Belgium (3), Croatia (3), Italy (2), Netherlands (2), USA (2), Austria (1),Denmark (1), Iceland (1), Mexico (1), Portugal (1) and Russia (1).

A sectorial breakdown of the responding SPUs is as follows: IT activities (58%),Business (16%) and Manufacturing (12%). Individual SIC codes were not taken as thebase unit of differentiation among companies because of the small sample size.

Companies were asked to classify themselves as either a company, a department, adivision or as an "other" category. About 60% of the respondents were SPUs whichwere a division or a department within a larger organisational unit while 40% wereautonomous companies.

Fifty two out of the 85 respondents (60%) had less than 100 employees. At the otherend of the spectrum, there were a few very large firms (only 8 companies have morethan 10,000 employees). Twenty seven companies out of 78 (almost 35%) had lessthan 10% of the total number of employees in software-related jobs.

With respect to the development categories for the respondents (each respondent couldtick multiple categories if applicable), the largest number were from the Electronic DataProcessing/Management Information Systems domain (50), followed by TransactionProcessing (40), Decision Support Systems (32), Control Systems (27),Telecommunications (25), Production Systems (20) and Software DevelopmentProducts (16). This shows that a majority of surveyed companies were involved in -developing software applications to support business needs. In fact, 71% of therespondents were working in at least one of the top three categories.

Customised Software Development was the largest percentage (38%) of the portfolio of.software activities for the responding SPUs, although this number was expected todecline slightly (to 33%) in the next year. The next three major categories of softwareactivities for the responding SPUs were Software Product Development (28%),Maintenance (28%) and Embedded Applications (25%).

An analysis was done of the distribution of project work among the various softwarelife cycle activities. The maximum amount of time (27%) was being spent by therespondents on coding and unit testing. Upstream activities such as Requirements(11%) and Planning/Specifications (13%) were getting less emphasis while moreemphasis was being given to activities like System Testing (17%) and Maintenance(16%).

53

Appendix B

Correlation Matrices for Enablers and Results (All Respondents)

Leader- -ship ,

SPU Policy '& Strategy

SPU People 'Mngt. ,

End-user -Mngt.

ResourceMngt.

Processes

Leader-ship

1 0.78 0.74 0.66 0.60 0.70

SPUPolicy &Strategy

1 0.83 0.72 0.82 0.66

SPUPeopleMngt.

1 0.72 0.75 0.74

End-userMngt.

1 0.70 0.77,ResourceMngt.

1 0.64

Processes 1

Correlations Among Enabler Categories for All Respondents

SPU PeopleSatisfaction

End-userSatisfaction

Impact onOrganisation

BusinessResults

SPU PeopleSatisfaction

1 0.55 0.64 0.55

End-userSatisfaction

1 0.44 0.69

Impact onOrganisation

1 0.60

BusinessResults

1

Correlations Among Result Categories for all Respondents

54