using dempster-shafer theory and real options theory

Decision Support Systems 46 (2008) 344–355

Contents lists available at ScienceDirect

Decision Support Systems

j ourna l homepage: www.e lsev ie r.com/ locate /dss

Using Dempster–Shafer theory and real options theory to assess competing strategiesfor implementing IT infrastructures: A case study

Cokky Hilhorst a,⁎, Piet Ribbers a, Eric van Heck b, Martin Smits a

a Tilburg University, P.O. Box 90153, 5000 LE Tilburg, The Netherlandsb RSM Erasmus University, P.O. Box 1738, 3000 DR Rotterdam, The Netherlands

⁎ Corresponding author.E-mail addresses: [email protected] (C. Hil

(P. Ribbers), [email protected] (E. van Heck), m.t.smits@

0167-9236/$ – see front matter © 2008 Elsevier B.V. Aldoi:10.1016/j.dss.2008.07.006

a b s t r a c t
a r t i c l e i n f o
Article history:
This paper discusses the sel Received 23 August 2006Received in revised form 29 June 2008Accepted 6 July 2008Available online 17 July 2008
Keywords:Multi-attribute decision analysisRiskBelief functionsReal options theoryIT infrastructure implementation strategy

ection of a preferred strategy for implementing an IT infrastructure from a rangeof competing alternatives. The model presented here combines the use of an evidential reasoning approachbased on the Dempster–Shafer theory of belief functions with real options analysis. We discuss the combineduse of both theories and show that combining the Dempster–Shafer theory with real options analysisprovides flexible support that takes account of the multi-dimensional nature of implementation decisions.We also go into the fundamental requirements that need to be met when selecting a strategy for imple-menting an IT infrastructure. We conclude by outlining a number of the model's limitations.

© 2008 Elsevier B.V. All rights reserved.

1. Introduction

The consolidation of IT infrastructures in large organisations canresult in development and implementation projects inwhich separateorganisational units are required to work with the same system. Thismay result in complex technology development and implementationprojects costing millions of euros. Since significant organisationalchange is required across all organisational units, and since suchprojects often involve the use of complex technology, they face seriousrisks thatmay result in unrealised benefits, high costs or overruns. Thepreferred strategy for developing and implementing such projectstherefore needs to reduce the degree of uncertainty and risk, and alsocreate managerial flexibility so as to maximise the project's benefits.

Multi-attribute decision analysis is a field of research in whichvarious techniques have been developed to make ‘preferencedecisions (such as evaluation, prioritization, selection and so on)over the available alternatives that are characterised by multiple,usually conflicting, criteria’ [4]. One of the more recent developmentsin multi-attribute decision analysis is the use of an evidentialreasoning approach based on the Dempster–Shafer theory of belieffunctions [20,22,28]. The Dempster–Shafer theory models risk byusing the notion of the plausibility of a negative outcome, and bycapturing both precise data and various types of uncertainties [23].

Where the degree of uncertainty is high and the costs are irre-versible, there is a consensus that real options theory can be applied to

horst), [email protected] (M. Smits).

l rights reserved.

capture the financial value of managerial flexibility in IT infrastructureprojects. By giving managers the ability to ‘wait and see’ in the event ofuncertainty, real options analysis can help to identify the mostfavourable staging of investments, and can also create scope for addi-tional learning about future payoffs before a final decision is made [16].

In recent years, a growing volume of research has been performedinto IT investments and real options [2,3,14,21] and IT investments andthe use of multi-attribute decision analysis to account for uncertainties[9,17,18,23]. Nevertheless, to our knowledge, no attempt has beenmadeto date, as far as we are aware, to synthesise the findings of research onreal options andmulti-attribute decision analysis to support complex ITinfrastructure investment decisions in conditions of uncertainty.

A vital issue in determining the practical value of both analyticalmethods is how to combine real options analysis with multi-attributedecision analysis, so as to incorporate different types of uncertainties andrisks, as well as quantitative and qualitative information. Consequently,thegoal of thispaper is todevelop formal support fordefininga favourablestrategy for implementing an IT infrastructure that takes different typesof uncertainty and risk into account. We combine real options theory andthe Dempster–Shafer theory of belief functions to model risk. This allowsus to analyse a fundamental problem in the implementation of ITinfrastructure systems: what is the best strategy for balancing the risksand benefits from both financial and non-financial perspectives?

We compare the combined use of real options theory andDempster–Shafer theory with the use of:

(1) traditional Net Present Value (NPV) analysis,(2) real options analysis,(3) Dempster–Shafer theory and NPV analysis.

mailto:[email protected]




http://dx.doi.org/10.1016/j.dss.2008.07.006

http://www.sciencedirect.com/science/journal/01679236

345C. Hilhorst et al. / Decision Support Systems 46 (2008) 344–355

We discuss the theoretical assumptions underlying and thelimitations of combining real options theory and an evidentialreasoning approach based on the Dempster–Shafer theory. Usingreal data from a large European-based service-provider, we use thiscombined model to define a favourable multi-stage strategy fordeveloping and implementing a human resource managementapplication. We show that the combined use of these two approachescan generate important information for making a selection fromcompeting strategies, and that this combined model takes fullaccount of the multi-dimensional nature of IT investment decisions.We also offer a set of minimum decision-making criteria for IT in-frastructure implementation projects that is consistent with priorwork.

The proposed combined model covers a range of IT infrastructureprojects in which a selection needs to be made from competingimplementation scenarios in a setting where there is a project risk.The model provides flexible decision support that can be adapted tothe specific domain in question.

The rest of the paper is organised as follows. In the next section,we take a closer look at decisions which can be modelled with the aidof an evidential reasoning approach based on the Dempster–Shafertheory of belief functions. We also introduce real options theory,present a combination of the two theories and discuss the underlyingtheoretical assumptions. We then describe the background to theproblem and outline the decision attributes that are relevant indefining an IT infrastructure implementation strategy. The applica-tion of the model in a case study is then presented, in which we usedata from a large European-based service-provider to define afavourable strategy for implementing a human resource managementsystem. Next we compare the decision-making models and discusstheir limitations and we conclude with a summary of the study'smain findings and suggest a number of possible topics of furtherresearch.

2. Theoretical background

A project for developing and implementing an IT infrastructurestands more chance of being successful if it is structured to fit thedemands imposed by the risk inherent to the project [11]. Bothfinancial and non-financial aspects play an important role in theinvestment process. In order to make a selection from competingimplementation strategies in a situation where there is a project risk,both qualitative and quantitative decision attributes and differenttypes of risk have to be taken into account.

One means of structuring decision-making is presented by anevidential reasoning approach based on the Dempster–Shafer theoryof belief functions [28,29], which models both quantitative andqualitative information in a situation of risk. After discussing thisapproach, we then introduce the Net Present Value (NPV) method andthe valuation of multi-stage options using real options theory. Tocalculate the option value of alternative strategies, we use thebinomial options model proposed by Cox et al. [8]. This model hasfrequently been used in prior research to value multi-stagedinvestment scenarios. However, researchers have not paid muchattention to the way in which non-financial aspects of the investmentscenario affect the choice of a strategy for implementing an IT infra-structure. We show how we combined the use of real options analysisand the Dempster–Shafer theory and discuss their underlying theo-retical assumptions.

2.1. The evidential reasoning approach based on the Dempster–Shafertheory of belief functions

The evidential reasoning approach based on the Dempster–Shafertheory of belief functions can be used to deal with multi-attributedecision-making problems of both a quantitative and qualitative

nature, in conditions of risk. There is growing support for the use ofthe Dempster–Shafer theory of belief functions for analysing multi-attribute decisions [23,29,30]. The approach has a number ofimportant characteristics, which we discuss here.

In its traditional definition, risk is measured as the probability of anegative outcome of an event or situation [23]. However, risk may alsobe conceptualised as an uncertain condition or event that has either anegative or a positive effect on the achievement of an objective [9].The Dempster–Shafer theory of belief functions allows us toincorporate both these above notions of risk. Support for a hypothesisindicates the amount of belief that directly supports a givenhypothesis, because there is no evidence that would contradict thehypothesis. Using belief functions we can withhold belief from aproposition without according that belief to the negation of the sameproposition. This allows us to model the level of residual uncertaintyor ambiguity that remains after the available evidence has beenconsidered. Using belief functions allows us to model a lack of data,probabilities or vagueness in subjective judgments.

Using the Dempster–Shafer theory of belief functions, risk can bemodelled by applying the notion of the plausibility (i.e. risk) of anegative outcome. To illustrate this, consider the situation where wehave belief that the implementation time of a proposed IT implementa-tion strategy (which may be subject to considerable uncertainty)could be 30% good and 20% not good, and 50% ignorance indicating wedo not know whether the implementation time will be good or bad,based on what we know about the presence of threats and availablecountermeasures. In this case, ‘Good’ and ‘Not Good’ denotedistinctive evaluation grades, and the percentage values of 30 and20 are degrees of belief, indicating the extents to which thecorresponding grades are assessed. This may be expressed as:

S Implementation timeð Þ= Good;0:3ð Þ; Not Good;0:2ð Þf g ð1Þ

where S(Implementation time) is the implementation time of theimplementation strategy and the figures 0.3 and 0.2 stand for thedegrees of belief. In this case, the plausibility that the implementa-tion time is not good is 70%, based on the available information. So,we have 70% risk that the implementation time is not good. Asillustrated, belief in a statement represents the total belief thatthe statement is true. Belief of zero in a statement means lack ofevidence in support of the statement, unlike representing impossi-bility in probability.

The evidential reasoning approach uses an extended decisionmatrix, in which each attribute of an alternative is described by adistributed assessment. It employs a belief function based on theDempster–Shafer theory of belief functions to represent an assess-ment of an attribute as a distribution of a set of evaluation grades.Suppose there are K alternatives, Oj (j=1,…, K), to choose from and Mattributes, Ai (i=1,…, M), to consider. Using a set H={Hi | i=1,…, N} ofevaluation grades, we can represent the assessment of an attribute A1

on an alternative O1, denoted by the expectation S(A1(O1)), using thefollowing belief structure:

S A1 O1ð Þð Þ= H1;β1;1� �

; H2;β2;1� �

;…; HN ;βN;1� ��

; where 0VΣβn;1V1 for n=1;…N

ð2Þ

where βn,1 denotes the degree of belief that attribute A1 is assessed toevaluation grade Hn. The expectation S(A1(O1)) reads that attribute A1

at an alternative O1 is assessed to grade Hn to a degree of βn,1×100%(n = 1,…, N). An assessment may be regarded as being complete if thesum of the degrees of belief for all n is 100%. In the evidentialreasoning framework, a multi-attribute decision analysis problemwith M attributes Ai (i=1,…, M), K alternatives Oj (j=1,…, K) and Nevaluation grades Hn (n=1,…, N) for each attribute is representedusing an extended decision matrix with S(Ai(Oj)) as its element in theith row and jth column.

346 C. Hilhorst et al. / Decision Support Systems 46 (2008) 344–355

In a utility-based evidential reasoning approach to multi-attribute decision analysis, an attribute can have its own set ofevaluation grades that may differ from those of other attributes [28].Rule-based or utility-based techniques can be used to devise asystematic procedure for transforming various types of informationinto a unified format, so that there is consistency between qualitativeand quantitative information. It differs in this respect fromtraditional multi-attribute decision analysis approaches, most ofwhich aggregate average scores. Instead of aggregating averagescores, an evidential reasoning algorithm uses decision theory andthe evidence combination rule of the Dempster–Shafer theory [20] toaggregate belief degrees. The logic behind the algorithm is that, if anobject has a good (or bad) attribute, then that object must be good (orbad) to a certain extent. The extent is measured both by the relativeweight, denoted by ω, that decision-makers assign to the attributeand by the degree to which the attribute belongs to the good (or bad)category. Appendix A provides details on the evidential reasoningalgorithm.

The evidential reasoning approach is capable of accommodatingnumerical data and subjective judgments of various formats, aswell as incomplete and imprecise information. In the presentedanalysis in Section 4, we use a window-based software tool calledIntelligent Decision System (IDS) [28] to support the evidentialreasoning approach based on the Dempster–Shafer theory of belieffunctions.

2.2. The valuation of multi-stage IT investments using the binomialoptions model

Researchers have used real options analysis as a capital budgetingapproach that takes explicit account of the value of flexibility in ITinvestment decisions [2,5,24–26]. It assumes that decision-makerscan intervene if the projected level of cash flow is not realised due toresolved uncertainty, and that managers can actively maximise theupside potential of the investment or limit the downside loss. From anoptions perspective, risk is defined as the upward and downwardvariation in expected outcomes.

Options analysis allows us to decide which implementationstrategy results in the greatest managerial flexibility, by calculatingthe optimum outcome for project risk and financial return. Thetraditional NPV approach discounts the estimated cash flow of aninvestment to its present value, using a time value of money as thediscount rate commensurate with the market expectations of theproject risk. It assumes that initiation of a project entails a completecommitment to the cash flow specified. This assumption is not correctwhenever the probability distribution of the returns is asymmetric, asis usually the case [16].

However, IT infrastructure development and implementationprojects may involve a different range of options. This paper focuseson stage options. Stage options create value by providing anopportunity to alter or terminate a project before each new stageof funding, based on updated information about costs and benefits,thus enabling projectmanagers to learn or gather information duringthe investment process. As a stage is completed, the ambiguitiesabout the net payoffs from subsequent stages may be resolved andvalue is created by only pursuing subsequent stages with positivepayoffs.

In order to calculate the optional value of the different IT imple-mentation alternatives, we use the binomial options model devised byCox et al. [8]. This values the real options in discrete time using abinomial lattice and allows for the transparent and custom-tailoredmodelling of multi-stage investment decisions [5]. The binomialoptions model assumes that, starting at t0=0, the value of therisky underlying asset V may increase to uV or decrease to dV by timet1= t0+Δt. In this case V is the value of an IT infrastructure investment.The probability that value V will rise is assumed to be q and the

probability that value V will fall is 1−q, where d b 1, u N 1. The up anddown movements in the lattice follow the equations

u= exp σ√t=nÞ d= exp −σ√t=n� �� ð3Þ

where n is the number of steps in the binomial lattice and one timeperiod Δt is defined as the time to expiration of the option divided bythe number of steps in the binomial lattice. In the option calculation,the volatility σ represents an estimate of the variance of projectreturns. It measures the risk associated with the project in terms offluctuations in the cash flow during the course of the project.

For an explanation of how to calculate a call option using abinomial lattice, see extensive descriptions of the model in priorliterature [5,8]. We can use the binomial options model to optimisethe balance between implementation risk and benefits from a finan-cial perspective.

2.2.1. Real option analysis and the Dempster–Shafer theory of belieffunctions

In the context of the evidential reasoning approach, quantitativedata can be transformed to a unified format using utilities which canbe estimated explicitly using the decision-maker's preferences. In thisway, a quantitative basic attribute can be transformed to an equivalentexpectation so that the quantitative attribute can be aggregated inconjunction with other qualitative attributes. In order to incorporatethe generated option value in the cost attribute of the evidentialreasoning framework, we assume a linear marginal utility function.This assumption cannot be relaxed, as is explained below. The utilityof the financial attribute is normalised as follows. As the option valueC is a cost and benefit attribute, the highest option value is preferred.Given a linear utility function, we assign u(Cmax)=1 and u(Cmin)=0.Then a value Cj on an attribute Ai may be represented using thefollowing equivalent expectation:

Si Cj� �

= hn;i;βn; j� �

; for n=1;…;Ni ð4Þ

where

βn; j= u hn+1;i� �

−u hj� ��

= u hn+1;i� �

−u hn;i� ��

and βn+1; j=1−βn; j ð5Þ

if u(hn,i)≤u(hj)≤u(hn +1,i). This transformation from a quantitativeassessment to Eq. (4) is equivalent with regard to the underlyingutility and rational in terms of preserving the features of the originalassessment [28]. In particular, as is shown in [28] the completeness ofan assessment is preserved in the transformation process. We can usethis quantitative data transformation technique to incorporate thefinancial valuation in the evidential reasoning framework.

2.3. Combining the use of real options with the Dempster–Shafer theoryof belief functions

Combining real options with the Dempster–Shafer theory of belieffunctions appears attractive from a practical viewpoint, since it allowsus to make a selection from competing implementation strategies in asituation where both qualitative and quantitative decision attributesand different types of risk have to be taken into account. However,both theories rest on certain assumptions that lead to a lack oftheoretical elegance.

In the first place, Goodwin and Wright [12] have already pointedout that converting NPV to utilities requires clear assumptions aboutthe decision-maker's preferences. For example, the NPV assumes aconstant rate of trade-off for sums of money between the same pair ofyears, irrespective of the amount of money which is transferred fromnext year to this [12]. From the viewpoint of the evidential reasoningapproach, if this assumption is seriously violated, the NPV will notaccurately represent the decision-maker's preference between sumsof money arriving at different points in time.


In addition, real options analysis uses the volatility of a portfolio ofsecurities that exactly replicates the project's payoffs as if they weretraded on the market, assuming complete and perfect markets. Thevalue of the project is indicated by the market value of this replicatingportfolio. Furthermore, real options analysis assumes risk-neutralinvestors. Clearly, the decision-maker's probabilities and utilityfunction for describing its preference for cash flow over time maywell be different from this perspective. According to [22] however, theoption pricing method is a simpler and more direct way of computingthe project value and determining the best project managementstrategy than trying to define the decision-maker's probabilities andutility function for the project's NPV. Based on these theoreticalassumptions of risk neutrality and perfect markets it is not meaningfulfrom a real options perspective to use a non-linear utility functionto impose the decision-maker's risk attitude into the cost-benefitattribute when translating the option value into an evidentialreasoning framework.

Despite these objections to the combined use of theories, whenestimates of future cash flows and volatility are carefully considered,an identification of the options and an understanding of the value offlexibility can help managers to identify and abandon failing ITinvestments before they escalate out of control.

Themain advantage of the combinedmodel is that it effectively andcomprehensibly combines both option analysis and the evidentialreasoning approach into a practical means of accurately valuingprojects. A thorough sensitivity analysis – both of theweights assignedto the criteria in the evidential reasoning approach and of the volatilityin the real options analysis – generates information from a number ofdifferent perspectives. This gives us a practical advantage over therealistic alternatives of either not valuing or systematically under-valuing the different investment options associatedwith newproductsand projects [16].

Table 1Attributes for defining an IT infrastructure implementation strategy

Attribute label Description

(e1) Costs and benefits The costs of the implementation related to(e1a) Costs of the implementation The costs of the system implementation(e1b) Benefits of the implementation The monetary benefits of the system imp

(e2) Project implementation risk Reduction of exposure to organisational or(e2a) Project size The size of the project, measured in num

(e2a1) Implementation time Estimated project implementation time(e2a2) Number of departments involved Number of departments involved with

(e2b) Experience with technology The project team and organisations' fami(e2b1) New hardware The extent to which the hardware is ne(e2b2) New software The extent to which the software is ne(e2b3) User IT knowledge The extent to which the user is knowle(e2b4) Project team knowledge The extent to which the project team i

(e2c) Project structure The nature of the task complexity the procommitment to the system

(e2c1) Replaced functions The percentage of existing functions th(e2c2) Procedural changes The severity of user-department proce(e2c3) Structural changes The degree of needed user-organisatio(e2c4) User attitude The general attitude of the user toward(e2c5) Management commitment Commitment of upper-level user mana

(e3) Business change The possibility of the IT implementation to(e3a) Improvement in consumer service Changes associated with the IT which po(e3b) IT aligned with business strategy Level to which the IT supports the strateg(e3c) Improvement of organisational image The extent to which the IT use will positi(e3d) Improvement of strategic positioning The impact of the IT to open up opportun(e3e) Improved efficiency and control of

internal processesThe impact of the IT to foster monitoring

(e4) Learning effects The possibility of the IT implementation str(e4a) Functional flexibility The extent to which learning is supported

processes(e4b) Technical scalability The extent to which learning is supporte(e4c) Technical compatibility The extent to which learning is supported

different departmental infrastructures(e4d) Improved implementation processes The quality of the implementation proces

3. Attributes for IT infrastructure strategies

In deciding on the most favourable implementation strategy,managers seek to learn about uncertainties to reduce risks andincrease benefits. For example, a system may need functionalflexibility to support differently organised business processes, or theorganisation may need to respond to organisational changes duringthe course of implementation. The implementation strategy may alsobe a source of risk in itself; obviously, a big bang-type implementationacross all organisational units is a riskier undertaking than a projectstaged in multiple phases. An important step in the evidentialreasoning approach is defining the relevant decision-making attributehierarchy. The set of measurable attributes for modelling benefits andrisks is obtained from prior research on IT project risk and on thejustification of IT investments.

In an attempt to deal with the continuing problem of informationsystem failures, researchers have tried to systematically organisecritical risk factors in IT projects [3,13] and have identified IT projectimplementation risk factors [1]. Borenstein and Betencourt [6]identify a minimum set of attributes for the justification of generalIT investments with operational, tactical and strategic attributes oncosts, business change and risks, based on previous research studies[7,12,19]. These attributes and risk factors lead to the formulation ofthree basic criteria:

(1) costs and benefits: the costs of implementation in relation to themonetary benefits,

(2) project implementation risk: the reduction of exposure toorganisational or system failure or to budget overshoots ortime overruns,

(3) business change: the ability of the IT infrastructure to createopportunities for business transformation.

the monetary benefits

lementationsystem failure, or to higher-than-expected costs or timeber of departments involved and the estimated project implementation time

implementing the systemliarity with the systems technologyw to the organisationw to the organisationdgeable in the area of ITs knowledgeable in the proposed application areaject faces, measured in changes that are needed to implement the system and the

at are replaced on a one-to-one basisdural changes caused by the proposed systemn structural change to meet requirements of the new systems the IT solutiongement to the systemprovide opportunities for business transformationsitively influence the relationship with clientsy of the businessvely influence the image of the organisation from the clients point-of-viewities for new business changesof internal processes

ategy to support learning about the system implementationabout the functional flexibility of the system to support differently organised business

d about the technical scalability of the system to support different user groupsabout the technical compatibility of the system to be successfully implemented in the

s by having less implementation problems


To these three main attributes we can add the learning effectsattribute, which is the ability of the implementation strategy tosupport learning about the implementation of the system. With theexception of the cost and benefit attribute, all attributes addressevidence for the competing implementation strategies that cannot bequantified financially at the time of making a decision. In total, theattribute hierarchy consists of four first-level attributes, fourteensecond-level attributes and eleven third-level attributes. Table 1summarises the attribute hierarchy, including a brief description ofthe attributes. The set of attributes to cover the minimum dimensionsrelevant to the selection of an implementation strategy may beexpanded, depending on the characteristics of the specific situation.

4. The case: implementation of a human resourcemanagement system

We used the combined model comprising the Dempster–Shafertheory and real options theory presented above to analyse a de-velopment and implementation investment decision of an IT infra-structure using data from a European-based service-provider. Anorganisation employing over 30,000 FTEs1 developed a humanresource management system (HRMS) to help the organisationmatch its staff complement to changing external and internaldemands as effectively and efficiently as possible. The organisationconsisted of 18 autonomous departments, ranging in size from smallto large. The HRMS needed to support strategic high-level planning tojustify capital expenditure, tactical management of human resourcesto meet demand, and operational scheduling and individual timeregistration. The HRMS implementation was a large and complexproject affecting all the organisation's employees to a lesser or greaterdegree. After the development of the HRMS, 18 separate applicationimplementation projects were planned for the 18 different depart-ments. The business objectivewas cost reduction; the benefits were toconsist of internal organisational benefits derived from the deploy-ment of the system.

4.1. The HRMS project risks

The HRMS implementation project was associated with a highdegree of risk due to organisational, technological and functionaluncertainties. The project was affected by several organisation-specific and IT project implementation risks. These were due toendogenous factors and affect the organisation's ability to success-fully implement the system. Monetary risks were caused by reason-able doubts as to whether the estimated financial benefits of thedevelopment and implementation of the HRMS were valid andrealisable, given that the organisation had a long history of failed ordelayed IT projects. There was uncertainty about the clarity of scope,since the autonomous departments differed in their organisationalstructures and in the way they had organised their business pro-cesses. The new IT application needed to contain the featuresrequired to support all the various departments. Risks concerninginfrastructural stability and compliance, as well as adequate infra-structural support, arose from the fact that the departmentalinfrastructures used different technical platforms, different sharedapplications and different types of data. The application needed to bescalable to the larger departments and had to be compatible withdepartmental IT infrastructures. If the risks relating to the clarity ofscope and infrastructural stability were not adequately addressed,the projected monetary benefits and opportunities for businesstransformation might not be realised in all the departments. Projectimplementation risks, such as those relating to project size and

1 Due to reasons of confidentiality we do not provide the exact number of employeesin the organisation. We also adjusted, without loss of generality, the number ofdepartments of the service-provider and the name of the information system.

project structure, arose from the absence of sufficient project capa-bilities such as supportive senior management and access to keyresources to effectively deploy the system during and after imple-mentation. Insufficient project capabilities resulted from the largenumber of diverse departments that had to be supported duringimplementation. These project implementation risks could affect thesuccess of implementation and hence the organisation's ability toachieve business change in all departments, eventually leading to alack of monetary benefits.

4.2. Application of the evidential reasoning approach based on theDempster–Shafer theory

The case study analysis was carried out from February to October2005. The analyses included in-depth interviews with various peopleworking on the project, namely the IT project manager, line managers,interviews with employees from the 18 departments (both large andsmall) and interviews with project team members. We used openquestions to roughly guide the interviews. The data provided by theseinterviews plus archival data based on internal documents and otherwrittenmaterial was used to define and analyse the research problem.The data needed for the cost-benefit analysis was based on a systemdevelopment and implementation estimation supplied by externalconsultants working on the project and was also obtained frominternal project documents.

The process of defining a preferred implementation strategy usingthe evidential reasoning approach based on the Dempster–Shafertheory consisted of five stages:

(1) specifying the decision-making goal and the alternatives takeninto consideration

(2) specifying the decision-making attributes (as presented inSection 3)

(3) assessing the attributes, including the cost-benefit analysis ofthe alternatives, and assigning weights to the attributes

(4) aggregating the individual assessments into anoverall assessment(5) performing a sensitivity analysis of the weights assigned.

Having identified the specific risks affecting the project, we willnow present our findings relating to the individual stages.

4.3. Outline of the alternative implementation strategies for the HRMSproject

The first stage involved specifying the decision-making goal andthe alternatives. The goal of the decision is to define the best possiblestrategy for implementing an IT infrastructure that balances the risksand benefits from both a financial and a non-financial viewpoint.

In order to define the alternative implementation strategies, weneeded to make certain assumptions. First, the development of theHRMS would last 1 year. This assumption was based on the projectplanning documents for the software development phase. Second,each implementation stage would take 1 year. Moreover, due to thescarcity of resources, each implementation stage had to be completedbefore a new one could start. Third, the maximum duration of theentire HRMS implementation project was 3 years. If this deadline wasnot observed, the departments might diverge too widely in theirorganisational directions. We therefore decided that there should be amaximum of three one-year implementation stages. Although thecosts and benefits of the HRMS could vary from one department toanother, we assumed that the implementation had fixed costs andbenefits in all departments.

Another important assumption was that the department imple-mentation projects did not have correlated risk profiles in the realoptions analysis. The risks associated with the development andimplementation of one IT project may impact other projects as well.


This may result in a significant level of positive correlation of theorganisational, technological and functional risks associated withimplementation of projects with similar capabilities and dependenton similar skills. As a consequence, the valuation of a portfolio in-corporating risk may be different from the sum of the valuations ofindividual projects if the firm is risk averse [2]. Although systemimplementation risks could change during the course of implementa-tion as a result of learning (for example, whether the system supportsdifferent ways of working or whether it is technically scalable), weassumed that uncertainty about the organisational benefits dependedon departmental efforts and therefore are not correlated. Additionally,we did not have data on the correlation between risks in this case.Instead, we added the learning effects attribute, which is the ability ofthe implementation strategy to support learning about the imple-mentation of the system. The assumption on the correlation of riskprofiles in the real options analysis could be loosened if data isavailable. Depending on the source of the correlated risk, this mayinfluence the relevance of the added learning effects attribute, whichcan easily be adjusted.

In the light of these assumptions, we defined four alternativedevelopment and implementation strategies. These were as follows:

(1) a one-stage implementation,where, after a development stage att = 0, the organisation implements the HRMS in all 18 depart-ments at once (i.e. in the form of a ‘big bang’) at t = 1;

(2) a two-stage, nine–nine implementation, where, after thedevelopment stage, the implementation is divided into twostages with nine departments in each stage;

(3) a two-stage, six–twelve implementation, where the imple-mentation is divided into two stages at t=1, with sixdepartments in the first stage and twelve departments in thesecond stage;

(4) a three-stage implementation, where implementation isdivided into three stages after development, with six depart-ments in each stage (see Fig. 1).

Thus far, we have defined the decision-making goal, distinguishedthe alternatives and specified the decision-making attributes (seeSection 3). The next section assesses and represents the evidencestrength, including a cost-benefit analysis of the alternatives.

Fig. 1. The different development and implementation s

4.4. Assessment of the different implementation strategies

To assess the attributes, we started by performing a cost-benefitanalysis of the alternatives.We evaluated the cost-benefit attribute forthe four development and implementation strategies. First, wecalculated the NPVs of the implementation scenarios withoutflexibility (Section 4.4.1). We then determined the best implementa-tion strategy using the real options theory (Section 4.4.2). The nextstep was to assess the non-financial attributes and assign weights tothe attributes (Section 4.4.3). We incorporated the results of thetraditional NPV and real options theory into a multi-attribute decisionanalysis framework for evaluating the implementation strategies.After aggregating the individual assessments into an overall assess-ment for the four strategies, we performed a sensitivity analysis on theassigned weights. We then compared the combined use of Dempster–Shafer theory and real options theory with the use of:

(1) NPV analysis,(2) real options theory,(3) Dempster–Shafer theory and NPV analysis.

4.4.1. Implementation without flexibility: the NPV analysisWe conducted a standard NPV analysis, based on the cash flows

estimation during development and implementation. We calculatedthe NPVs of all four strategies, i.e. one-stage, the two-stage nine–nine,two-stage six–twelve and three-stage implementation. These valueswere all different, since implementing the project in stages defersboth costs and savings.

The project costs consist of personnel and non-personnelexpenses, i.e. the costs pertaining to the project developmentenvironment (such as housing, workstations, hardware and applica-tion packages). The training costs comprise the investments made intraining staff who were to work with the system. Management andmaintenance costs consist of ongoing investments in IT infrastructureand maintenance. Since the organisation is a government body, weestimated a relatively low cost of capital (k=10%), involving a riskpremium of 6% above the risk-free rate (r=4%).

As we have already mentioned, the project was designed to reducecosts. This means that the benefits consist of internal organisationalbenefits derived from using the system. There are two sources of

cenarios: from ‘Big Bang’ to staged implementation.

Table 2aSummary of cash-flows (cost and benefits) for the development of the two-stage nine–nine implementation of the HRMS (×1000 €)

Development and two-stage nine–nine implementation

Year 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014

CostsInternal staff 800 1092 1092External staff 3512 1714 1714Exploitationand maintenance

1510 3020 3020 3020 3020 3020 3020 3020 1510

Training 361 361Non-personnel 100 23 45 45 45 45 45 45 45 23Subtotal 4412 4699 6232 3065 3065 3065 3065 3065 3065 1533

BenefitsReduction oflabour costs

438 1313 2188 3063 3500 3500 3500 1750

Productivityimprovement

625 1875 3125 4375 5000 5000 5000 2500

Subtotal 0 0 1063 3188 5313 7438 8500 8500 8500 4250Net cash flow 4412 4699 5169 122 2247 4372 5435 5435 5435 2717


financial benefits: lower labour costs and productivity gains. Labourcosts can be lowered as the new HRMS supports the pro-activemanagement of variable personnel expenses, leading to a reduction inovertime, picket costs, special duty costs, etc. We assumed that thereduction in labour costs consisted of an average of seven FTEs worth€40,000 a year in each department, resulting in an average reductionof approximately 0.2%/annum in the cost of labour. The total benefitsof the lower labour costs, once all departments are operational, wouldbe €5 million/annum.

The new HRMS would also improve staff productivity, as fewerplanning mistakes would be made. The assumption here was thatcomputerised support of capacity planning would lead to a produc-tivity gain of one and a half percent per annum. The total benefits ofproductivity improvement once all departments were operationalwould be €3.5 million/annum. In both cases, we incorporated alearning effect for the estimated project benefits, assuming that 25% ofthe total estimated benefits would be realised in each department inthe first deployment year, 50% in the second year, 75% in the third yearand 100% as from the fourth year.

Table 2 shows the NPV analysis for the one-stage implementationproject over the next 8 years (i.e. until 2013), which is the expectedlifespan of the HRMS. Including the development stage and a one-stepimplementation in all departments, the project results in a positiveNPV of €1,183,678. We then performed the same calculation for theother implementation strategies, leading to an NPV of €929,329 forthe two-stage nine–nine implementation, €844,546 for the two-stagesix–twelve implementation and €690,395 for the three-stage imple-mentation. In the case of the different two-stage and three-stageimplementations, whilst implementing the system in a large numberof departments earlier augments the NPV because of the earlierpayback of larger benefits, the NPV is nonetheless not higher than inthe case of a one-stage implementation strategy. Table 2a shows theNPV analysis for the two-stage nine–nine implementation as anexample.

Traditional investment valuationmethodswould therefore suggestthat the project is an attractive investment and that the system shouldpreferably be implemented in a single stage. Of course, this analysisdoes not include risk and does not take accurate account of theembedded option value of a staged implementation.

4.4.2. The real options analysis: using the binomial real options modelWe also performed a real options analysis using the binomial real

options model including a sensitivity analysis. In order to calculate thevalues of the different implementation scenarios, we needed to know,for each call option in the lattice, the expected present value of the

Table 2Summary of cash-flows (cost and benefits) for the development of the one-stageimplementation of the HRMS (×1000 €)

Development and one-step implementation

Year 2005 2006 2007 2008 2009 2010 2011 2012 2013

CostsInternal staff 800 2184External staff 3512 3427Exploitation andmaintenance

3020 3020 3020 3020 3020 3020 3020 3020

Training 721Non-personnel 100 45 45 45 45 45 45 45 45Subtotal 4412 9398 3065 3065 3065 3065 3065 3065 3065

BenefitsReduction oflabour costs

875 1750 2625 3500 3500 3500 3500

Productivityimprovement

1250 2500 3750 5000 5000 5000 5000

Subtotal 0 0 2125 4250 6375 8500 8500 8500 8500Net cash flow 4412 9398 940 1185 3310 5435 5435 5435 5435

incremental cash flows, Vt and the investment It made to acquire theoption, where t is the expiration time of the option. We also needed toknow the volatility σ at each step.

In the option calculation, the volatility σ represents an estimate ofthe variance of project returns. A representation of the projectvolatility is the standard deviation of the rate of change in projectreturns over one time period [10,25]. It measures the risk associatedwith the project in terms of fluctuations in the project's cash flowduring the course of the project. Although a good estimate of volatilityσmay be based on historical data [10], in our case the IT infrastructurewas implemented in 18 autonomous, decentral departments, whereno historical data were available. Instead, as in [2,25], different cashflow scenarios for the investment (i.e. worst-case, base-case and best-case scenarios) were produced for the various project stages and thevariance was computed using the percentile estimate for the normaldistribution. As we have previously explained, the base-case scenariois based on an average labour reduction of seven FTEs per departmentper annum and a productivity gain of one and a half percent perannum. In the worst-case scenario, the reduction in labour costs isworth five FTEs per department per annum, whereas the best-casereduction is nine FTEs per department per annum. In the worst-case scenario, the productivity gain is half that achieved in the base-case scenario, whereas the best-case gain is double that in the base-case scenario. We then calculated the standard deviation of the rate ofchange in project returns in the worst-case, base-case and best-casescenario over one time period, representing the volatility values forthe different implementations stages. We subsequently calculatedoption values for each project implementation strategy as sum-marised in Table 3.

Table 3Volatility values for the different implementation stages (in %) and option values for thedifferent implementation strategies (×€1000)

Implementation strategy Volatility per stage Option value perimplementation strategy

One-stage implementation 4,411,823Stage one (18 departments) 18.41

Two-stage nine–nine implementation 2,566,928Stage one (nine departments) 6.66Stage two (twelve departments) 6.05

Two-stage six–twelve implementation 2,487,099Stage one (six departments) 4.44Stage two (twelve departments) 8.07

Three-stage implementation 2,298,951Stage one (six departments) 4.44Stage two (six departments) 4.03Stage three (six departments) 3.67

Table 4Option values at varying volatility values for the different implementation strategies ofthe HRMS (×1000 €)

Option values of the different implementation strategies

Volatility One-stage Two-stagenine–nine

Two-stagesix–twelve

Three-stage Preferredstrategy

σ0: 0.1 3043 2704 2591 2386 One-stageσ0: 0.2 4833 4412 4272 4018 One-stageσ0: 0.3 8676 8081 7882 7522 One-stageσ0: 0.4 14,653 13,786 13,498 12,792 One-stageσ0: 0.5 22,925 21,683 21,268 20,515 One-stageσ0: 0.6 33,736 32,002 31,424 30,373 One-stageσ0: 0.7 46,897 44,828 43,963 42,709 One-stageσ0: 0.8 62,664 60,499 59,188 57,876 Two-stage

nine–nineσ0: 0.9 81,949 79,712 77,840 76,485 One-stage

nine–nine


4.4.2.1. Sensitivity analysis. We conducted a sensitivity analysis for awide range of volatility values to measure its impact on the selectionof the most suitable strategy. We let σ range from σ=0.10 to σ=0.90and used a constant volatility for each implementation stage. Theoption values for the different implementations at different volatilityvalues are summarised in Table 4. The value of σ does not influencethe final decision on the most favourable implementation strategy.

Relaxing the assumption that the volatility σ remains constant, wethen lowered the volatility value at each new project implementationstage by 50%. As expected, this only strengthened the preference for aone-stage implementation. In this case, at all different volatility values(σ=0.10 to σ=0.90), a one-stage implementation is the preferredimplementation strategy.

We also conducted a sensitivity analysis after augmenting thevalue of the organisational benefits while keeping the volatilityconstant across all stages. Raising the organisational benefits does notaffect the choice of the most favourable form of project staging.Lowering the organisational benefits for all departments by 25%results in a negative NPV and a positive option value for all scenarios

Table 5The assessment for the different implementation strategies of the HRMS

Attribute Weight One-stage

(e1) Costs and benefits 0.15 {(E, 1.0)}(e2) Project implementation risk 0.35(e2a) Project size 0.4

(e2a1) Implementation time 0.4 {(G, 0.5), (E, 0.5)}(e2a2) Number of departments 0.6 {(B, 1.0)}

(e2b) Experience with technology 0.3(e2b1) New hardware 0.2 {(I, 0.5), (A, 0.5)}(e2b2) New software 0.2 {(B, 1.0)}(e2b3) User IT knowledge 0.2 {(A, 0.5), (G, 0.5)}(e2b4) Project team knowledge 0.4 {(B, 1.0)}

(e2c) Project structure 0.3(e2c1) Replaced functions 0.1 {(A, 0.5), (G, 0.5)}(e2c2) Procedural changes 0.2 {(A, 0.5), (G, 0.5)}(e2c3) Structural changes 0.2 {(A, 0.5), (G, 0.5)}(e2c4) User attitude 0.25 {(B, 0.8), (G, 0.2)}(e2c5) Management commitment 0.25 {(B, 0.8), (G, 0.2)}

(e3) Business change 0.35(e3a) Improvement in consumer service 0.2 {(A, 0.6), (G, 0.4)}(e3b) IT aligned with business strategy 0.1 {(G, 0.8), (E, 0.2)}(e3c) Improvement of

organisational image0.2 {(E, 1.0)}

(e3d) Improvement of strategic positioning 0.2 {(E, 1.0)}(e3e) Improved

efficiency and control of internal processes0.3 {(E, 1.0)}

(e4) Learning effects 0.15(e4a) Functional flexibility 0.25 {(B, 1.0)}(e4b) Technical scalability 0.25 {(B, 1.0)}(e4c) Technical compatibility 0.25 {(B, 1.0)}(e4d) Improved implementation processes 0.25 {(B, 1.0)}

only if σ is higher than 0.50. In these conditions, the three-stageimplementation is preferable to the two-stage nine–nine, the two-stage six–twelve and the one-stage implementation in this order. Thusfrom an options perspective, if it is likely that the organisationalbenefits will turn out to be lower, and if there is a high degree ofuncertainty as to whether these benefits can be realised, a differentimplementation scenario then becomes more favourable.

4.4.3. Combining the use of real options analysis and Dempster–Shafertheory

We have already discussed the optimum implementation strategyfrom a financial viewpoint using the NPVmethod and, when includingrisk and the value of embedded managerial flexibility, using optionanalysis. The most favourable implementation strategy from afinancial viewpoint is the one-stage implementation strategy usingthe NPV method. The same strategy was also found to be best whenwe included risk and the value of embedded managerial flexibilityusing option analysis.

Using the framework presented above, we combine optionsanalysis and the rule-based evidential reasoning approach. Theattributes for assessing the various implementation scenarios havealready been presented in Section 3. There are four thematic topics,consisting of both qualitative and quantitative attributes.

This stage involves the users assessing the strength of evidence foreach attribute. This indicates the level of support for each attribute. Inthe evidential reasoning process, the decision-makers have to assignweightsωi to the various attributes. In our case, we assessed attributesand assigned weights to the attributes using information on theproject obtained both from documentation and from interviews. Weverified the assessment of attributes and weights with the HRMSproject manager, in terms of their plausibility with regard to theproposed implementation strategies, leading to the final attributeassessment and weights set out in Table 5.

To this end, we used a window-based software tool called Intelli-gent Decision System (IDS) [28] to apply the evidential reasoningapproach based on the Dempster–Shafer theory of belief functions.

Two-stage nine–nine Two-stage six–twelve Three-stage

{(I, 1.0)} {(I, 0.3), (B, 0.7)} {(B, 1.0)}

{(A, 0.5), (G, 0.5)} {(A, 0.5), (G, 0.5)} {(I, 0.7), (A, 0.3)}{(I, 0.4), (A, 0.6)} {(I, 0.6), (A, 0.4)} {(G, 0.5), (E, 0.5)}

{(A,1.0)} {(A, 1.0)} {(A, 0.5), (G, 0.5)}{(A, 0.6), (G, 0.4)} {(A, 0.4), (G, 0.6)} {(G, 0.4), (E, 0.6)}{(A, 0.5), (G, 0.5)} {(A, 0.5), (G, 0.5)} {(A, 0.5), (G, 0.5)}{(A, 0.6), (G, 0.4)} {(A, 0.4), (G, 0.6)} {(G, 0.4), (E, 0.6)}

{(A, 0.5), (G, 0.5)} {(A, 0.5), (G, 0.5)} {(A, 0.5), (G, 0.5)}{(A, 0.5), (G, 0.5)} {(A, 0.5), (G, 0.5)} {(A, 0.5), (G, 0.5)}{(A, 0.5), (G, 0.5)} {(A, 0.5), (G, 0.5)} {(A, 0.5), (G, 0.5)}{(B, 0.6), (G, 0.4)} {(B, 0.4), (G, 0.6)} {(B, 0.3), (G, 0.7)}{(B, 0.6), (A, 0.2), (G, 0.2)} {(B, 0.4), (A, 0.4), (G, 0.2)} {(B, 0.4), (A, 0.2), (G, 0.4)}

{(A, 0.8), (G, 0.2)} {(A, 0.8), (G, 0.2)} {(A, 1.0)}{(G, 0.6), (A, 0.4)} {(A, 0.6), (G, 0.4)} {(A, 1.0)}{(G, 0.5), (E, 0.5)} {(G, 0.4), (E, 0.6)} {(G, 0.5), (A, 0.5)}

{(G, 0.6), (A, 0.4)} {(A, 0.6), (G, 0.4)} {(A, 1.0)}{(G, 0.6), (A, 0.4)} {(A, 0.6), (G, 0.4)} {(A, 1.0)}

{(B, 0.6), (A, 0.4)} {(A, 1.0)} {(E, 0.2), (G, 0.8)}{(B, 0.6), (A, 0.4)} {(A, 1.0)} {(E, 0.2), (G,0.8)}{(B, 0.6), (A, 0.4)} {(A, 1.0)} {(E, 0.2), (G, 0.8)}{(B, 0.6), (A, 0.4)} {(A, 1.0)} {(E, 0.2), (G,0.8)}

Table 6Quantified assessment of implementation scenarios

Implementation strategy Assessment Ranking

One-stage implementation 0.514 2Two-stage nine–nine implementation 0.491 4Two-stage six–twelve implementation 0.521 1Three-stage implementation 0.511 3


We defined a set H of evaluation grades for assessing all theattributes of each alternative implementation strategy:

H= hj; j=1;…;5� �

= Excellent Eð Þ;Good Gð Þ;Average Að Þ; Indifferent Ið Þ;Bad Bð Þf g:ð6Þ

All attributes, apart from quantitative ones, can be assessed usingthese evaluation grades. We only considered complete assessments. Ifcertain qualitative assessment values do not refer to this set, we canuse the rule-based information transformation technique to assess all

Fig. 2. Summary of the combined approach of real

attributes with reference to this set. Because of the limited amount ofspace available to us, wewill not demonstrate this technique here andrefer the reader instead to a detailed description of this technique in[28].

We then incorporated the option values into a multi-attributedecision analysis framework. To estimate the utilities of the costattribute we assume a linear marginal utility function. For the optionvaluation, we normalised the option value for each alternative (seeTable 3) by assigning the highest option value (one-stage implemen-tation) u(3,411,470)=1 and the lowest option value (three-stageimplementation) u(2,298,951)=0. As we assume a linear utilityfunction, u(3,133,340)=0.75, u(2,855,211)=0.5 and u(2,577,081)=0.25. Let

h1;1=3;411;470; h2;1=3;133;340; h3;1=2;855;211;

h4;1=2;577;081; h5;1=2;298;951:

The option value of the two-stage six–twelve alternative can berepresented using Eq. (4). The option value of the two-stage six–twelve alternative, h1=2,487,099. Since h5,1 b h1 b h4,1, we can

options analysis and Dempster–Shafer theory.

Table 7Comparison of the different methods

Comparison of methods

NPV Real optionsanalysis

EvidentialReasoning based onDempster–Shaferand NPV

EvidentialReasoning based onDempster–Shaferand real options

Evaluation criteriaFinancial Yes Yes Yes YesNon-financial

No No Yes Yes

Risks Risk-freerate ofreturn

Variability of theprojects'financial valueand risk-freerate of return

Uncertainty insubjectivejudgments andrisk-free rateof return

Uncertainty insubjectivejudgments andvariability of theprojects' financialvalue and risk-freerate of return

Process supportParticipatoryplanning

No No Yes Yes

Impact of methodLearning No Yes Yes YesFlexibility Does not

takeflexibilityintoaccount

Takesmanagerialflexibility from afinancialperspective intoaccount

Takes multipledefinitions offlexibility intoaccount

Takes managerialflexibility from afinancial perspectiveinto account andmultiple definitionsof flexibility


represent the option value as S3(2,487,099)={(h4,1, β4,1), (h5,1, β5,1)},where

β4;1= h5;1−h1� �

= h5;1−h4;1� �

= 2;298;951−2;487;099ð Þ= 2;298;951−2;577;081ð Þ=0;7 and

β5;1=1−β4;1=0:3:

So, S3(2,487,099)={(h4,1, 0.7), (h5,1, 0.3)}={(I, 0.3), (B, 0.7)}. In asimilar way we calculate the two-stage nine–nine alternative S2(2,566,928)={(h4,1, 1.0)}={(I, 1.0)}.

Using a similar calculation, we calculated the utilities assuming alinear utility function. We normalised the NPVs for each alternative byassigning the highest NPV (one-stage implementation) u(1,183,678)=1 and the lowest NPV (three-stage implementation) u(690,395)=0.The NPVs may be represented as:

S1 1;183;678ð Þ= E;1:0ð Þf g;S2 929;329ð Þ= E;0:2ð Þ; G;0:8ð Þf g;S3 844;546ð Þ= G;0:1ð Þ; A;0:9ð Þf g; andS4 690;395ð Þ= B;1:0ð Þf g:

Using the IDS software tool mentioned above, which is based onthe evidential reasoning algorithm referred to in [28], we were ableto calculate the overall degrees of belief. We aggregated all theattributes, leading to a final assessment of the proposed implementa-tion strategies.

Combining real options analysis with theDempster–Shafer theory ofbelief functions results in a preference for the two-stage six–twelveimplementation (see Table 6). This is what onewould intuitively expect,since more attributes are used that support phase-wide implementa-tion. The one-stage implementation is ranked second, followed by thethree-stage implementation and the two-stage nine–nine implementa-tion. Fig. 2 represents a stepwise summary of our overall approach.

4.4.3.1. Sensitivity analysis. As the final stage of the assessmentprocess, we performed a sensitivity analysis to see how the preferredalternative changes if weights are assigned differently. If the weightfor cost and benefits changes from 0.15 to 0.24, the two-stage nine–nine implementation becomes the preferred implementation strategy.If the weight for the ‘costs and benefits’ attribute is changed to 0.53,the preferred strategy changes to one-stage implementation. Theseoutcomes support what one would intuitively expect, since the cost-benefit attribute supports earlier implementation of the system in alarge number of departments because of the earlier payback ofbenefits. If we use the NPV calculation in the multi-attribute decisionanalysismodel, we get the same outcome, but the aggregated value foreach alternative varies.

5. Discussion and conclusion

In this paper, we have defined a theoretical model for selecting themost favourable strategy for implementing an IT infrastructure inconditions of uncertainty. The aim of this paper is to help decision-makers to opt for the best implementation strategy by combiningthe Dempster–Shafer theory with real options analysis. We draw onDempster–Shafer belief functions to evaluate decision-making attri-butes that cannot be easily quantified. This combined model takes fullaccount of the multi-dimensional nature of the decision that needs tobe taken. We applied the model in a case study, inwhich we used datafrom a large European-based service-provider to define a favourablestrategy for implementing a human resource management system.

Decisions on IT investments are complex decisions based onmultiple goals and values. It may well be because of this complexitythat organisations often fail in practice to follow a well-structured,accountable and reproducible decision-making process for assessing

the profitability of IT investments. We have shown that our model foranalysing investments in IT infrastructures, which takes account of themulti-dimensional nature of such investments, generates vitalinformation for selecting the most appropriate strategy. Whilstprevious research studies have valued multi-stage investmentsusing real options analysis, they have ignored the multi-dimensionalnature of IT infrastructure investment decisions. Although optionsanalysis generates valuable insights into the trade-off between riskand benefits from a financial perspective, as we have shown in thisarticle, adding non-financial attributes that support phase-wideimplementation can lead to very different outcomes in terms of themost suitable implementation strategy. Combining the Dempster–Shafer theory with real options analysis supports both qualitative andquantitative decision attributes, as well as different types of risk forboth quantitative and qualitative attributes. This is shown in Table 7.

The case study shows that a combined analysis can generatevaluable information that can help decision-makers to select the mostsuitable strategy from a range of competing strategies. This is because,as both the multi-attribute decision analysis and the real optionscalculation make it easy to perform a sensitivity analysis of the statedpreferences, the various scenarios can be evaluated for differentparameter ranges. The advantage of using the options model lies inidentifying the trade-offs between risk and benefits from a financialviewpoint. Since the combined analysis also facilitates transparentgroup decision-making, it can help to overcome one of the mainobjections to the use of real options analysis, namely its lack oftransparency for decision-makers. However, the difficulty of obtainingan accurate estimate of the volatility of the investment benefitsremains to be a source of criticismwhen using options valuation [28].

Our approach covers a wide range of competing strategies for ITinfrastructure projects where there is a project risk. These include, forexample, the implementation of group-wide computer platforms,application platforms (e.g. ERP) and data warehouses. The analysisprovides flexible decision support that can be adapted to the specificdomain in question.

Aswe have seen, amultiple attribute approach results in a differentdecision thanwhen aNPVor real options analysis is used. Theremay betwo reasons for this: firstly, the decision-maker's objectives may


diverge from those of capital market players. Alternatively, the NPV orreal options valuation may not have been correctly calculated. Toelaborate on the first reason, the normative decision models assumethat a firm pursues the sole objective of stockholder wealth max-imisation. However, IT infrastructure projects undertaken by largeorganisations are complex and involve interactions among a widevariety of stakeholders. The latter may have their own interpretationsof wealth maximisation, subject to individual concerns about risk,liquidity, responsibility and so forth. For example, as is shown in thesensitivity analysis in the real options calculation, raising the degree ofuncertainty leads to a higher real options value and therefore to ahigher preference for the one-stage implementation strategy. How-ever, the decision-maker in question may well have a preference for alower project risk, resulting in a different implementation strategy. Asregards the risks in relation to non-financial aspects such as learning,these are difficult to quantify in practice. This is easy to model bycombining the Dempster–Shafer theory with real options analysis.Consequently, certain criteria that are not quantified in an NPV or realoptions analysismay bemade explicit in amultiple attribute approach,thus resulting in a different decision.

The second possible reason for a different decision is if the NPV orreal options valuation is not correct. This may be the case if cash flowscannot be accurately projected at the time when the decision is taken.For example, in our case, the results of the application of theimprovement of strategic positioning attribute cannot be incorporatedinto the cash flow calculation for the project since it is not yet possibleto place a monetary value on the attribute.

As for the limitations of our combined model, firstly real optionsand decision analysis rest on assumptions that lead to a lack oftheoretical elegance when used in combination. Converting NPV toutilities requires clear assumptions about the decision-maker'spreferences as described earlier. Also, the decision-maker's probabil-ities and utility function for describing its preference for cash flowover time may well be different from assumptions underlying realoptions theory. Violation of the theoretical assumptions underlyingreal options theory has been extensively discussed in literature, see forexample [5], and these arguments on the validity of using real optionsanalysis for IT infrastructure investment decisions can be extended tothe combined model as presented in this paper.

Secondly, in the case we presented we assumed no projectinterdependencies between the 18 different HRMS implementations.Project interdependencies between two projects exist when acapability developed for one project is required by one or moreother project(s) or when a capability developed for one projectsupports capabilities required by other projects [2]. Recently, a realoptions approach has been developed to prioritize a portfolio of ITprojects that have interdependencies [2]. There are also examples ofmodelling project interdependencies using programming techniquesin a multi-attribute decision analysis as in [15]. The specific goal ofour approach was to define an implementation strategy for anisolated application in autonomous departments which can bemodelled as a stage option. Of course, also in this case project inter-dependencies may exist. For example, benefit interdependencies canoccur due to a synergy effect when the HRMS is implemented in alldepartments.

A vital issue for further research is how to combine real optionsanalysis with multi-attribute decision analysis to model these projectinterdependencies. In the presented combined model, project inter-dependencies from a financial perspective can bemade transparent byusing the real options approach that deals with project interdepen-dencies mentioned above [2]. Non-financial project interdependen-cies can be made transparent by introducing attributes to the modelthat represent the interdependencies. For example, a specific attributefor resource interdependencies may be added to account for theextent to which software resources can be shared among the variousimplementation projects.

When dealing with project interdependencies to prioritize aportfolio of IT projects, uncertainty about the benefits or a lack ofinformation on other characteristics of future projects may exist. Thismakes the combined use of real options and the Dempster–Shafertheory of belief functions a strong candidate for the support of theseinvestment decisions.

The model can be further refined with the aid of validation tech-niques such as user assessments to further extend the attributehierarchy. It can also be extended to deal with project interdepen-dencies as discussed above. Furthermore, for the purpose of ouroption analysis, we assumed that uncertainties surrounding differentprojects in different departments are not related. This may of coursenot be the case in real life. A future study should take into accountthe correlation of risk between projects. Also, we used the model toidentify a multi-stage implementation strategy for an IT applicationthe aim of which is to cut costs. It would also be interesting to analysean IT infrastructure investment in which growth options form part ofthe investment proposal, given that there will be a greater disparitybetween the different implementation scenarios.

Appendix A. The evidential reasoning algorithm

The evidential reasoning algorithm uses the concepts in set theoryand probability theory for aggregating multiple attributes [28,29].Suppose there is a simple two level evaluation hierarchy with ageneral attribute Ai at the upper level and a set of L basic attributes atthe lower level. The assessments represented in Eq. (2) for an alter-native Oi can be aggregated using the following evidential reasoningalgorithm.

Suppose ωi is the relative weight of the attribute Ai and ωi and isnormalised, so that 0≤ωi≤1 and ∑ωi=1 for n=1,…, L. Without loss ofgenerality, we present the evidential reasoning algorithm forcombining two attribute assessments given by the assessment S(A1

(O1)) as presented in Eq. (1) and the assessment S(A2(O1)), which isgiven by

S A2 O1ð Þð Þ= H1;β1;2� �

; H2; ;β2;2� �

;…; HN ;βN;2� ��

; where 0VΣβn;1V1 for n=1;…;N:

We need to aggregate the two assessments S(A1(O1)) and S(A2(O1))to generate a combined assessment. Suppose S(A1(O1)) and S(A2(O1))are both complete. Let

mn;1=ω1βn;1 and mH;1=1−ω1Σβn;1=1−ω1 for n=1;…;Nmn;2=ω2βn;2 and mH;2=1−ω2Σβn;2=1−ω2 for n=1;…;N

wheremn,1 andmn,2 are referred to as basic probability mass and eachmH,i (for j=1, 2) is the remaining belief for attribute j unassigned to anyof the Hn (where n=1,…, N). The evidential reasoning algorithm isused to aggregate the basic probability masses to generate combinedprobability masses, denoted by mn (where n=1,…, N) and mH usingthe following equations:

Hnf g : mn=k mn;1mn;2+mH;1mn;2+mn;1mH;2� �

for n=1;…;NHf g : mH=k mH;1mH;2

� �

where

k= 1−ΣΣmn;1mp;2� �−1 for n=1;…;N and p=1;…;N and n≠p:

Now the combined probability masses can be aggregated withanotherassessment in thesamewayuntil all assessments are aggregated.

If there are only two assessments, the combined degrees of beliefβn,1 for n=1,…, N are generated by:

βn=mn=1−mH for n=1;…;N:


The combined assessment for the alternative O1 can then berepresented as follows:

S O1ð Þ= H1;β1ð Þ; H2; ;β2ð Þ;…; HN ;βNð Þf g where 0VΣβnV1 for n=1;…;N:

Since the Dempster's rule of combination proved to be commu-tative and associative [20], evidence can be combined in any order. Incase of multiple belief structures, the combination of evidence can becarried out in a pairwise way [27].

References

[1] L.M. Applegate, F. WarrenMcFarlan, J.L. McKenney, Corporate Information SystemsManagement, Irwin/McGraw-Hill, New York, 2005.

[2] I. Bardhan, S. Bagchi, R. Sougstad, Prioritizing a portfolio of information technologyinvestment projects, Journal of Management Information Systems 21 (2) (2004).

[3] H. Barki, S. Rivard, J. Talbot, An integrative contingency model of software projectrisk management, Journal of Management Information Systems 17 (4) (2001).

[4] V. Belton, T.J. Stewart, Multiple Criteria Decision Analysis: An Integrated Approach,Kluwer Academic Publishers, 2001.

[5] M. Benaroch, R.J. Kauffman, A case for using real options pricing analysis toevaluate information technology project investments, Information SystemsResearch 10 (1) (1999).

[6] D. Borenstein, P.R.B. Betencourt, A multi-criteria model for the justification of ITinvestments INFOR, Information Systems and Operational Research 43 (1) (2005).

[7] E.K. Clemons, Strategic and competitive information systems, Journal of Manage-ment Information Systems 8 (2) (1991).

[8] J. Cox, S. Ross, M. Rubinstein, Option pricing: a simplified approach, Journal ofFinancial Economics 7 (3) (1991).

[9] Cutter Consortium, Exactly what is risk management? Cutter Consortium PressRelease, Arlington, MA. (June 2002).

[10] B.L. Dos Santos, Justifying investments in new information technologies, Journal ofManagement Information Systems 7 (4) (1991).

[11] J.R. Galbraith, Organizational design: an information processing view, Interfaces 4 (3)(1974).

[12] P. Goodwin, G. Wright, Decision Analysis for Management Judgment, John Wileyand Sons, Ltd, 2005.

[13] M. Keil, P.E. Cule, K. Lyytinen, R.C. Schmidt, A framework for identifying softwareproject risk, Communications of the ACM 41 (11) (1998).

[14] Y.J. Kim, G. Lawrence Sanders, Strategic actions in information technologyinvestment based on real option theory, Decision Support Systems 33 (1) (2002).

[15] J.W. Lee, S.H. Kim, An integrated approach for interdependent information systemproject selection, International Journal of Project Management 19 (2001).

[16] J.E. Neely, R. de Neufville, Hybrid real option valuation of risky product developmentprojects, International Journal of Technology, Policy and Management 1 (1) (2001).

[17] D. Petkov, O. Petkova, T. Andrew, T. Nepal, Mixing multiple criteria decision makingwith soft systems thinking techniques for decision support in complex situations,Decision Support Systems 43 (4) (2007).

[18] K.B. Salling, S. Leleur, A. Vestergaard Jensen, Modelling decision support anduncertainty for large transport infrastructure projects: the CLG-DSS model of theØresund fixed link, Decision Support Systems 43 (4) (2007).

[19] A.H. Segars, V. Grover, Strategic information systems planning success: aninvestigation of the construct and its measurement, Management InformationSystems Quarterly 22 (2) (1998).

[20] G. Shafer, A Mathematical Theory of Evidence, Princeton University Press, 1976.[21] C. Singh, R. Shelor, J. Jiang, G. Klein, Rental software valuation in IT investment

decisions, Decision Support Systems 38 (1) (2004).[22] J.E. Smith, R.F. Nau, Valuing risky projects: option pricing theory and decision

analysis, Management Science 41 (5) (1995).[23] L. Sun, P.R. Srivastava, T.J., An information systems security risk assessment model

under the Dempster–Shafer theory of belief functions, Journal of ManagementInformation Systems 22 (4) (2006).

[24] A. Taudes, Software growth options, Journal of Management Information Systems15 (1) (1998).

[25] A. Taudes, M. Feurstein, A. Mild, Options analysis of software platform decisions: acase study, MIS Quarterly 24 (2) (2000).

[26] L. Trigeorgis, Making use of real options simple: an overview and applications inflexible-modular decision making, The Engineering Economist 50 (1) (2005).

[27] Y.M. Wang, J.B. Yang, D.L. Xu, Environmental impact assessment using the evidentialreasoning approach, European Journal of Operational Research 174 (3) (2006).

[28] J.B. Yang, Rule and utility based evidential reasoning approach for multi-attributedecision analysis under uncertainties, European Journal of Operational Research131 (1) (2001).

[29] J.B. Yang, M.G. Singh, An evidential reasoning approach for multiple attributedecision making with uncertainty, IEEE, Transactions on Systems, Man andCybernetics 24 (1) (1994).

[30] E. Wilkins, S.H. Lavington, Belief functions and the possible words paradigm,Journal of Logic and Computation 12 (3) (2002).

F U R T H E R R E A D I N G

[1] K. Roberts, M. Weitzman, Funding criteria for research, development and explo-ration projects, Econometrica 49 (5) (1981).

Cokky Hilhorst is a doctoral student at the Department ofInformation Management at Tilburg University. Her researchproject aims to understand how managers can evaluateflexibility in IT investment decision-making when factoringrisks. She is a senior manager IT Effectiveness at Pricewa-terhouseCoopers Advisory N.V. in The Netherlands. Sheholds BSc and MSc degrees in artificial intelligence fromUtrecht University, was a research student at CambridgeUniversity, England, and holds a masters degree in informa-tion management from the TIAS Business School.

Pieter Ribbers is Professor of Information Systems and Chairof the Department of Information Management at theUniversity of Tilburg. He is the Director of the TIAS BusinessSchool's master's programme in information management.From 1991 to 1995, he was Affiliate Professor of InformationManagement atWashington University in St. Louis, USA. As aresearcher, teacher and consultant, his main areas of interestare the management challenges posed by IT, e-commerceand the strategic and organisational impact of IT. He has co-authored a number of books.

Eric van Heck is Professor of Information Management andMarkets at RSM Erasmus University, where he combinesresearch into and teaching on the strategic and operationaluses of information technology. He is best known for hiswork on how companies can use on-line auctions to createvalue. He has co-authored a number of books and articlesthat have been published in journals including the CaliforniaManagement Review, Communications of the ACM, DecisionSupport Systems and Information Systems Research. Heobtained both his MSc and his PhD degrees at WageningenUniversity.

Martin Smits is Associate Professor of Information Systemsand Management at the School of Economics and BusinessAdministration at Tilburg University. His research focuses onstrategic information systems planning, healthcare informa-tion systems, group decision support systems, inter-organi-sational systems, network organisations and knowledgemanagement. He teaches at various universities on topicsincluding information management, accounting and control,the strategic management of educational institutions, andhealthcare management.

using dempster-shafer theory and real options theory

Business

projects real options

todempstershafer theory

preference decision

infrastructure systems

nal decision

implementation projects

uncertainties projects

projects benets