multi criteria decision analysis with minimum information combining dea with mavt original research...

7/31/2019 Multi Criteria Decision Analysis With Minimum Information Combining DEA With MAVT Original Research Article

http://slidepdf.com/reader/full/multi-criteria-decision-analysis-with-minimum-information-combining-dea-with 1/16

Computers & Operations Research 33 (2006) 2083–2098

www.elsevier.com/locate/cor

Multicriteria decision analysis with minimum information:combining DEA with MAVT

George Mavrotas∗, Panagiotis Trifillis

Laboratory of Industrial & Energy Economics, School of Chemical Engineering, National Technical University of Athens,

Zografou Campus, Athens 15780, Greece

Available online 1 January 2005

Abstract

In this paper we use some basic principles from data envelopment analysis (DEA) in order to extract the neces-

sary information for solving a multicriteria decision analysis (MCDA) problem. The proposed method (enhanced

alternative cross-evaluation, ACE+) is appropriate when either the decision maker is unwilling (or hardly available)

to provide information, or there are several decision makers, each one supporting his/her own option. It is similar

to the AXE method of Doyle, but it goes one step further: each alternative uses its most favourable weights (as in

AXE) and its most favourable value functions in order to perform a self evaluation, according to multi attributevalue theory (MAVT). These self-evaluations are averaged in order to derive the overall peer-evaluation for each

alternative. The minimum information required from the decision maker is to define the weight interval for each

criterion. Beside the peer evaluation and the final rating of the alternatives, the method provides useful conclusions

for the sensitivity analysis of the results.

2004 Elsevier Ltd. All rights reserved.

Keywords: Multiple criteria; DEA; Cross-efficiency; Value functions

1. Introduction

In multiple criteria decision analysis (MCDA) problems a number of alternatives have to be evaluated

and compared using several criteria. The aim of MCDA is to provide support to the decision maker(DM) in the process of making the choice between alternatives. In other words, MCDA techniques helpthe DM to articulate his/her preferences in a complex decision making environment. A pre-requisite in

∗ Corresponding author. Tel.: +30 210 7723283; fax: +30 210 7723155.

E-mail address: [email protected] (G. Mavrotas).

0305-0548/$ - see front matter 2004 Elsevier Ltd. All rights reserved.

doi:10.1016/j.cor.2004.11.023

http://www.elsevier.com/locate/cor

mailto:[email protected]

mailto:[email protected]

http://www.elsevier.com/locate/cor



2084 G. Mavrotas, P. Trifillis / Computers & Operations Research 33 (2006) 2083–2098

most MCDA methods is that the DM is able to provide the necessary information. However, in severalsituations the DM is hardly available or unwilling to express his/her preferences or, furthermore, thereis not an unequivocal DM but a group of stakeholders that each one supports his/her own alternative.

In these situations the available information in order to perform the multicriteria analysis is limited orambiguous. In order to fill the information gap, or to reach a consensus in the latter case, informationthat emerges from the data set itself can be utilized. In the present work, the suggested way to extract this

information is by using the basic ideas of Data Envelopment Analysis (DEA).DEA origins are found in the late 70s [1]. DEA deals with the evaluation of the performance of objects

(Decision Making Units or DMU in DEA’s terminology) using the transformation process of severalinputs into several outputs. Relying on a technique based on Linear Programming and without having tointroduce any subjective or economic prices (weights, costs, etc), DEA provides a measure of efficiency

of each object. Its basic aim is to separate efficient from non-efficient DMUs and to indicate for eachnon-efficient DMU its efficient peers. During the last 25 years thousands of papers related to DEA have

been published in OR/MS journals.MCDA and DEA have been receiving considerable attention in the OR/MS literature. The problem

tackled by DEA is one which may equally well be approached using MCDA. Indeed, in common withmany MCDA approaches, DEA incorporates a process of assigning weights to criteria. However, despite

having much in common, the two fields have developed almost entirely independently to each other[2–5]. It is only in the last decade the success of DEA in the area of performance evaluation together

with the formal analogies existing between DEA and MCDA (which become clear replacing DMU withalternatives, outputs with maximization criteria and inputs with minimization criteria) have lead someauthors to propose to use DEA as a tool for MCDA [6–8]. Among the first steps towards this co-operation,

was the notion of cross-efficiency in DEA, originated by Sexton et al. [9] and further developed by Doyleand Green [10]. The basic aim was to increase the discriminatory power of DEA. In cross-efficiency eachone of the alternatives is evaluated using the most favourable set of weights (self evaluation). In the same

time, the rest of the alternatives are evaluated using the above set of weights. This process is repeatedfor all the alternatives resulting in a square matrix of evaluations with the diagonal elements being the

self evaluation scores and the off-diagonal elements the peer evaluation scores of the alternatives. Thefinal score for each alternative is obtained as the corresponding column average of this matrix. As Doyleand Green state “. . . cross efficiency, with its intuitive interpretation as peer appraisal, has less of the

arbitrariness of additional constraints, and has more of the right connotations of a democratic process,

as opposed to authoritarianism (externally imposed weights) or egoism (self-appraisal)” [10]. Theseresults where further elaborated by Doyle [11] in order to develop a MCDA method called alternative

cross-evaluation (AXE) in its work with the intriguing title “Multiattribute Choice for the Lazy Decision

Maker: Let the Alternatives Decide!”.The notion of cross-efficiency has been emerged as an alternative DEA approach in order to increase

the relative discriminating power. As a consequence, it uses DEA’s mathematical representation and itis confined in linear relationships. Hence, in MCDA terminology, the value functions of the criteria

are assumed linear and the only variables in the Linear Programming (LP) formulation are the relativeweights. In the current work we extend the notion of cross efficiency in order to allow the value functionsin each criterion to be non-linear. It is accomplished with the incorporation in the model of a specially

developed non-linear value functions with one parameter. The model turns inevitably to non-linear withthe parameters of each criterion’s value function being the second set of decision variables (the first is the

set of weights). The proposed method is named enhanced alternative cross-evaluation (ACE+) and allows

http://-/?-

http://-/?-

http://-/?-

http://-/?-



G. Mavrotas, P. Trifillis / Computers & Operations Research 33 (2006) 2083–2098 2085

for a fruitful insight in the problem as it can extract intra-criterion (value functions) and inter-criterion(weights) information. Moreover, it maintains the objective and democratic character of cross-efficiencythat makes it attractive in minimum information situations.

The method can be used when the information from the DM in a MCDA problem is rather limited andwe want to obtain a better insight in the problem or, even more, to solve the problem adequately underthese minimum information conditions. It must be noted that in relative decision situations the usual,

naïve approach is to assign equal weights to the criteria, assume linear value functions and use an additiveaggregation function. Using more sophisticated methods like ACE+ the underlying information in the

alternatives and criteria can be better exploited leading to more robust decisions. In other words, ACE+is not only used for the evaluation of the alternatives under minimum information conditions, but it also“produces” information that help the decision maker to better perceive the potential of each alternative

in the specific decision context.The rest of the paper is organized as follows: In Section 2 the main topics of DEA and cross-efficiency

are briefly reminded. In Section 3, we describe the suggested method (ACE+) in detail. In Section 4we present the application of the method with an educational example making comparisons with otherapproaches. Namely, 14 countries of the European Union are compared by means of seven eco-efficiencyindicators. Finally, in Section 5 the most important conclusions and issues for further research are

presented.

2. Data envelopment analysis and cross-efficiency

The efficiency of a many-input, many-output decision making unit (DMU) may be defined as a weightedsum of its outputs divided by a weighted sum of its inputs (Eq. (1)).

Ek =u1y1k + u2y2k + · · · + unynk

1x1k + 2x2k + · · · + mxmk

, (1)

where yrk is the r th output of the DMU k , xik is the ith input of the DMU k , ur is the weight of the r th

output i is the weight of the ith input and Ek the efficiency of k th DMU. This, so called “engineeringratio” is the most popular of a number of alternative measures of efficiency.

DEA was initiated in 1978 when Charnes, Cooper and Rhodes demonstrated how to transform this

fractional linear measure of efficiency into a LP model. DEA formulates choice of these weights as alinear program, which allows each DMU to appear in the best possible light (maximize its own measured

efficiency relative to the other DMUs). The original problem that seeks for the maximum efficiency of

DMU k is formulated as follows:

Ek = max

sr=1 ur yrkmi=1 i xik

,

ST sr=1 ur yrjmi=1 i xij

1, j = 1 . . . n ,

ur , i0, (2)




where s is the number of outputs, m is the number of inputs and n is the number of DMU to be evaluated.In their seminal work Charnes, Cooper and Rhodes transform the above fractional optimization probleminto conventional LP by assigning the denominator of the objective function to unity. The obtained model

became

Ek = max

nr=1

ur yrk ,

ST

mi=1

i xij −

nr=1

ur yrj0 j = 1 . . . n , (3)

mi=1

i xik = 1,

ur , i0.

The necessary condition for a DMU to be characterized efficient is that the optimal value of model (3) isunity. Model (3) is solved n times, one for each DMU, in order to discover which DMU are efficient and

which are not. Because the number of DMUs is usually greater than the number of inputs and outputsit is usually more convenient to solve the dual of model (3) which results in less constraints. Furtherinformation for the non-efficient DMU can be also extracted (e.g., which DMU or combination of DMUs

dominates a specific non-efficient DMU). A number of alternative models have been developed, allowing

for economies of scale [12], constrained weights etc. The interested reader may refer among others toThanassoulis [13] and Cooper et al. [14] in order to read more about the theory and applications of DEA.

The main focus of applications of DEA has been in performance evaluation of non-profit organizationsand in estimating non-parametric production function. The initial aim of DEA was rather to identify those

DMUs which are not efficient in some sense, and to assess where the inefficiencies arise and not to rank the DMUs according to their efficiency. In essence, DEA can be described as trying to find a set of weightswhich present each DMU “in the best possible light”. The major advantage of this technique is that it is

strongly objective, in the sense that it does not need any preference information. On the other hand, itsmain limitation is its small discriminating power when used for evaluation purposes.

In order to overcome the latter drawback Sexton et al. in 1986 introduce the notion of cross-efficiency.

Since then several authors contribute in this idea (see e.g., [6,10,15]). The cross-efficiency method simplycalculates the efficiency score of each DMU n times, using the optimal weights evaluated by the n LPs.

The results of all the DEA cross-efficiency scores can be summarized in a cross-efficiency matrix asshown in Eq. (4):

hkj =

sr=1 urk yrjmi=1 ik xij

f or k, j = 1 . . . n . (4)

Thus, hkj represents the score given to DMU j when optimizing for DMU k , i.e. DMU j is evaluatedby the optimal weights of DMU k . Note that the diagonal elements of the cross-efficiency matrix (hkk )

represent the standard DEA efficiency scores (see Table 1).

http://-/?-

http://-/?-




Table 1

Cross-efficiency square matrix

Evaluating DMU Evaluated DMU

1 2 · · · n

1 h11 h12 · · · h1n

2 h21 h22 · · · h2n

· · · · · · · · · · · · · · ·

n hn1 hn2 · · · hnn

Column average e1 e2 · · · en

Furthermore, if the weights of the LP for DMU k are not unique, a secondary goal (either aggressive

or benevolent) can be activated. In the aggressive context, DMU k chooses among the optimal weights

such that it maximizes self efficiency and at a secondary level minimizes the other DMUs cross-efficiencylevels. The secondary goal can be incorporated in the objective function as follows:

max hkk = hkk −

nj =1,j =k hkj

n − 1, (5)

where >1. On the other hand, in the benevolent context, DMU k chooses among the optimal weights

such that it maximizes self-efficiency and at a secondary level maximizes the other DMUs cross efficiencylevels [10]. Thus, the benevolent context is implemented by reversing the sign of in Eq. (5).

The cross-efficiency ranking method utilizes the results of the above matrix in order to rank the DMUs.

Firstwecalculate ek astheaveragescoreofthe ncross-efficiencyscoresofDMU k (seeTable1,lastrow).Sub-sequently, we rank the DMUs according to the average efficiency score e

kof each unit. It is argued that e

kis more representative than hkk as an efficiency score, since all the elements of the cross-efficiency matrix

are included. Furthermore, in the calculation of ek all the units are evaluated with the same set of weights.The notion of cross-efficiency was further extended by Doyle in order to develop theAXE method [11].

AXE can be considered as an attempt to apply DEA in MCDA, so it was properly adjusted to MCDAterminology. In AXE the DMUs correspond to the alternatives which have to be evaluated. The inputsand outputs correspond to the criteria to be minimized and maximized respectively. The weights are the

scaling factors of the additive aggregation function as in DEA.Let an MCDA problem with n alternatives (i = 1 . . . n ) and m criteria (s = 1 . . . m ). Assume that the

DM defines value functions V s ( ) for each criterion in order to map the performance of each alternative

i (Xis ) to a value for the DM, expressed as V s (Xi ). It is recommended to normalize V s ( ) in [0, 1] or

[0,100] in order to have a common scale. In order to perform self-evaluation of alternative i the followingLP problem is solved:

max hii =

ms=1

wis V s (Xi ),

ST (6)

hij1, j = 1 . . . n ,

wis 0, s = 1 . . . m .




The essence of the DEA approach is in the challenge to each alternative to choose its own weights inorder to become as desirable as possible (maximize hii ) under the certain restrictions governing all thealternatives (the hij1 condition). After solving (6), one can evaluate alternative j (j = i) under ith

alternative’s view (with the optimal weights for i) using the following formula:

hij =

ms=1

woptis V s (Xj ). (7)

In this way one can obtain the cross-evaluation matrix, similar to Table 1. The column average for eachalternative represents the so-called peer-evaluation score according to which the rating of alternatives is

performed.

3. The suggested method: ACE+

3.1. General description and differences with AXE

ACE+ can be considered as an attempt to enhance the capabilities of AXE. It is designed to be usedwith even less information than AXE. In AXE the value function V s () of each criterion is considered tobe fixed and given by the DM. In ACE+ the value functions are adjustable, letting each alternative use the

most favourable value function during its self appraisal. In other words, the self-evaluation procedure,which is the cornerstone of cross-evaluation, is extended to comprise not only weight adjustments butalso value function adjustments. The motivation is to achieve the best case scenario for each alternative

during its self appraisal in the MCDA context. While AXE exploits mainly the inter-criteria information(weights of criteria) ACE+ exploits also the intra-criteria information (value function in each criterion).

As a consequence, ACE+ is characterized by more degrees of freedom than AXE, and results in a widerrange of performances for the alternatives.

Another modification regards the objective function of the self-evaluation model. Instead of using the

aggressive approach of AXE (i.e. use as primary goal the maximization of the score hii of alternative i

and if there are alternative optima use as secondary goal the minimization of the average score of the restalternatives) we suggest a somewhat different idea: rather than seeking for the combinations of weights

and value functions that give to ith alternative the higher possible score, we seek those combinations of weights and value functions that give to the ith alternative the greatest possible advantage in relation to the

other alternatives. In the self-appraisal model it is translated as maximization of the difference betweenthe ith alternative’s score and the average score of the rest alternatives. Belton and Stewart in an alternativeapproach they propose the maximization of the minimum difference in order to identify potentially optimal

alternatives [19]. In the present approach the weight and the value function parameter of a criterion inthe self evaluation model of alternative i, are decided taking into account the performances of the otheralternatives in the specific criterion and not only the performances of alternative i in the other criteria (as

it is done in AXE). As it will be shown in Section 4, this approach leads to more meaningful results.

3.2. The calculation procedure

3.2.1. Normalization

The first step in the calculation procedure is the normalization of the alternative scores in the criteria.The common scale in the criteria is necessary in order to have comparable weights in the aggregation




model. The performance of the alternatives in each criterion is scaled to the interval [0,1] according tothe following linear formulas (see e.g., [16]):

For a minimization criterion j the normalized score for alternative i is given by

N ij =XU

j − Xij

XU j − XL

j

. (8)

Similarly, for a maximization criterion j the normalized score for alternative i is given by

N ij =Xij − XL

j

XU j − XL

j

. (9)

The values of XLj and XU

j are the lower and upper anchor values respectively of the criterion j. Thesevalues can be set exogenously but they are usually taken as the minimum and the maximum of eachcriterion. However, there are cases where it is more convenient to take anchor values different from the

minimum and the maximum of the criterion, especially in the case of outliers. The outliers are isolatedvalues which are away from the main body of the data, and can be identified by specific statistical tests.The use of outliers as anchor values tend to “squeeze” the rest of the data in the [0,1] scale reducing the

discriminating power of the criterion. In order to avoid this inadequacy we propose that in the case of outliers to use as lower and upper anchor values the percentiles 5% and 95% instead of the minimumand maximum value, respectively. The normalized scores of the outliers that inevitably fall outside the

interval [0,1] are adjusted to the most nearest bound. With the above procedure we alleviate the effect of

outliers in the criterion scale.

3.2.2. Value functions

The value functions that are used in the proposed method are the typical concave and convex functions

from decision analysis, along with the linear function [17–19]. They are formulated as one-parameterfunction with a common formula as given below:

Y ij = V j (i) =1 − ecj N ij

1 − ecj. (10)

The denominator in (10) is necessary in order to keep the values of the value function in [0,1]. Theparameter cj defines whether the function will be concave, convex or linear as it is shown in Fig. 1. Forlinear functions we set cj = 0.001, instead of zero in order to avoid the disappearance of the denominator.Negative and positive values of cj characterize concave and convex functions respectively, while cj is

taken to vary from −5 to 5 giving enough curvature to the value functions.The concave and convex functions indicate Decreasing and Increasing Returns to Scale respectively,

in production theory terminology. In the decision theory they indicate risk averse and risk prone decision

makers and they are widely used. The general notion is that in concave functions the difference betweentwo alternatives in the beginning of the scale counts more than the same difference at the end of the scale.The opposite holds for the convex functions.

http://-/?-

http://-/?-

http://-/?-




Fig. 1. Examples of value functions used in ACE+.

3.2.3. Model for self-evaluation and the cross-evaluation matrix

In a MCDA problem with n alternatives and m criteria the self-evaluation model for each one alternativeis obtained through model (11)

hkk = max

mj =1

wj

1 − ecj N kj

1 − ecj−

ni=1,i=k

mj =1 wj

1−ecj N ij

1−ecj

n − 1

ST m

j =1

wj = 1,

wj U j , j = 1 . . . m ,

wj Lj , j = 1 . . . m ,

cjV j , j = 1 . . . m ,

cjP j , j = 1 . . . m ,

wj 0. (11)

It is a non-linear programming (NLP) model with decision variables the weights (wj ) and the valuefunction parameters (cj ). The parameters of the model are the normalized scores N ij , the bounds on the

weights U j and Lj and the bounds on the value function parameters V j and P j . The objective functionexpresses the maximization of the difference between the score of the self-evaluating alternative i and theaverage score of the rest alternatives. Although it looks complicated, the objective function is smooth and

can be adequately tackled by a conventional NLP solver like the GRG2 code which is used in MicrosoftExcel’s Solver module. The constraints of the model are the restriction on the sum of weights to equal tounity and the corresponding bounds on the weights and the value function parameters. The recommended




Fig. 2. Flowchart of the method.

upper and lower bounds of the value function parameters V j and P j are 5 and −5, that are considered togive enough curvature to the value functions. The upper and lower bounds on the weights depend on thenumber of criteria. A rule of thumb is to set these bounds to +/−50% of the uniform weight value 1/ m (i.e.

the weight interval becomes [l/2m, 3/2m]). Solving model (11) for all n alternatives the cross-evaluationmatrix is obtained. The flowchart of the procedure is shown in Fig. 2.

Fruitful information is carried by the cross-evaluation matrix (Table 1). Except from the self-evaluation

score hkk and the peer evaluation score (column average ek),onecanobtaintherangeofscoresforeachone

of the alternatives. The range of scores for each alternative is an important piece of information that can betaken into account in the final decision. For example between two alternatives with the same average scoreone may select the one with the smaller range of scores (risk averse DM). In general, alternatives whichare characterized by more extreme performances in the criteria result in greater range than alternatives

with mediocre performances. With ACE+ this characteristic can be identified and encountered in the finaldecision.

In addition, one can examine the weight and the value function parameter distributions per criterion as it

is obtained by the n self-evaluation problems. From these two distributions one can use the correspondingaverage values as representative weights and value function parameters. The weights can be consideredas inter-criteria information while the value function parameters as intra-criteria information.




3.2.4. Optional capabilities

Besides the basic framework that was described in the previous paragraphs, two optional modificationscan also be incorporated in the ACE+ method. First, it is the way that the representative score is obtained

from the efficiency score matrix. The average of the efficiency scores for each alternative, although areasonable metric for the distribution, it may lead to violations of monotonicity as it was shown byBouyssou [20], especially when we have only few alternatives. The violation of monotonicity is the main

criticism against the cross-efficiency method and occurs when the position of one alternative may improvealthough its evaluation in one criterion deteriorates. In order to avoid this phenomenon, the midrange

(instead of the average) of the distribution of the efficiency scores can be used as a representative scorefor each alternative. As it can be shown in the example of Bouyssou in [20], the midrange metric doesnot violate the monotonicity principle even in the extreme case of the specific example, as it encounters

only the extreme values of the cross-efficiency distribution.Another modification regards the formulation of the non-linear model. In order to avoid the solution of a

non-linear programming problem (which is always vulnerable to local optimal solutions) we may separatethe process into two independent steps. First, we can calculate for each alternative and for each criterionthe best value function regarding the rest of the other alternatives. This implies the calculation of thecorresponding parameter c that maximizes the difference between the score of the optimized alternative

i and the average score of the rest alternatives in the criterion j and is defined as c∗ij . This task requires

n × m one-parameter non-linear optimizations of the form

c∗ij = max

cj ∈[V j ,P j ]

1 − ecj N ij

1 − ecj−

1 − ecj N

avgj

1 − ecj

for i = 1 . . . n , j = 1 . . . m , (12)

where n is the number of alternatives, m the number of criteria, N

avg

j is the average score of the criterionexcluding the ith alternative and V j and P j are the bounds for cj . In other words, c∗ij

is the optimal value

of the parameter c that is derived from the self-evaluation of the ith alternative in the j th criterion as itis compared with a virtual alternative having the average performance of the j th criterion. The optimalscore of the ith alternative in the j th criterion is subsequently calculated as

s∗ij =

1 − ec∗

ij N ij

1 − ecjfor i = 1 . . . n , j = 1 . . . m . (13)

The second step is the maximization of the multicriterion score for each alternative. The already calculated“optimal” scores in each criterion are used and the only decision variables for the model are the weights

of the criteria. For alternative i it holds

hii = maxwj ∈[Lj ,U j ]

mj =1

wj s∗ij for i = 1 . . . n , (14)

where Lj and U j are the bounds on the weights for criterion j .

The optimal parameters c∗ij

and weights w(i)j

for alternative i are then used for the calculation of thecross-efficiencies of the other alternatives k in order to obtain the scores hik for k = 1 to n. In this way

the non-linear optimization procedure is divided into two (more easy to solve) independent parts: Firstthe calculation of the optimal parameters for each alternative and each criterion (n × m one parameter




non-linear optimizations) and then the calculation of the optimal weights for each alternative ( n linearoptimizations). The results obtained from the two procedures in a few examples were almost identicalregarding the ranking of the alternatives and the value of the parameters (weights and value function

parameters).

4. Numerical example

The example that is used in order to demonstrate the method concerns the environmental evaluation

of 14 countries of the European Union (we exclude Luxemburg— in most cross-national comparisons inEU—cause it acts as an outlier in the data set). The cross-country comparison based on multiple indices

is a common task that requires an objective evaluation procedure with little information by the decisionmaker. Moreover, the relative information must be extracted by the data, as usually there are no decision

makers (or they are hardly available) willing to assign numbers for the required parameters. This is thedecision context where ACE+ can really help. The evaluation is performed according to seven indicatorsthat are widely used to express eco-efficiency in a country. The indicators which are used as criteria inthe multicriteria evaluation problem are:

1. energy intensity (ENINT) defined as the Energy to GDP ratio (toe/million ¥),2. emission intensity (GHGINT) defined as the Green House Gas (GHG) emissions to GDP ratio (kg

CO2/¥),3. acidifying gases intensity (ACGAS) defined as the ratio of acidifying gases (SO2, NOx , etc) to GDP

(kg/ ¥),

4. renewable energy sources (RES) share in the energy mix excluding big hydro projects (%),5. share of population with public sewage treatment (PUBSEW) measured as %,

6. share of farming areas cultivated without chemical substances (organic farming, ORGFAR) measuredas %,

7. share of paper that comes from recycle (RECPAP) measured as %.

The performance of the countries in the seven criteria is shown in Table 2 and are taken from the official

site of the EU (http://europa.eu.int).From the above matrix we obtain the matrix of the normalized scores (Table 3) according to Eqs. (8)

and (9) and taking as XU j

and XLj

the 95% and 5% percentile respectively, in order to reduce the outlier

effect.

The problem is first solved withACE+ with cj ∈ [−5, 5] and wj ∈ [0.05, 0.3] and the cross-evaluationmatrix is shown in Table 4. The calculation procedures of the method have been developed in VisualBasic for Applications under the platform of Microsoft Excel using the Excel’s Solver for the non-linearoptimization module (the GRG2 algorithm incorporated in Solver found to be adequate for such type of

NLP problems). Cell a(i,j) denotes the score of alternative j if it is calculated with the weights andvalue function parameters that result from the self-evaluation of alternative i. Therefore the figures in thediagonal are the self-evaluation scores for each country. The last row includes the column averages as the

representative value of the multicriteria score for each alternative.As it was mentioned in the previous section an important piece of information is the range of scores

for each alternative. Fig. 3 illustrates the range of sores along with the average score for each country,

http://europa.eu.int/

http://europa.eu.int/




Table 2

Performances of countries in the eco-efficiency criteria

Code ENINT GHGINT ACGAS RES PUBSEW ORGFAR RECPAP

Austria A 182 0.52 0.060 10.5% 81.5 8.4 69

Belgium B 30 0.81 0.101 0.8% 38.05 1.4 38

Germany D 216 0.65 0.065 2.8% 91.5 2.6 70

Denmark DK 167 0.57 0.094 11.6% 89.01 5.4 50

Spain E 231 1.90 0.514 4.5% 48.3 1.4 42

Greece EL 345 1.57 0.376 0.5% 56.2 0.3 29

France F 224 0.50 0.094 2.3% 81 1.0 41

Finland FIN 266 0.62 0.082 11.2% 80 5.2 57

Italy I 170 0.56 0.091 4.7% 75 5.5 31

Ireland IRL 191 0.93 0.212 1.7% 57.6 0.7 12

Holland NL 292 0.82 0.080 3.7% 97.9 1.1 62

Portugal P 319 1.17 0.381 4.4% 46.1 1.2 40Sweden S 254 0.35 0.054 5.1% 93 4.5 62

United Kingdom UK 255 0.72 0.102 1.2% 84 2.5 40

Criterion direction min min min max max max max

Table 3

Normalized scores

Code ENINT GHGINT ACGAS RES PUBSEW ORGFAR RECPAP

A0.917 0.944 0.995 0.921 0.743 1.000 0.992B 0.184 0.712 0.883 0.010 0.000 0.141 0.323

D 0.703 0.842 0.982 0.198 0.937 0.340 1.000

DK 1.000 0.903 0.904 1.000 0.889 0.808 0.582

E 0.614 0.000 0.000 0.357 0.098 0.138 0.409

EL 0.000 0.093 0.140 0.000 0.251 0.000 0.129

F 0.655 0.957 0.902 0.151 0.733 0.077 0.388

FIN 0.394 0.866 0.935 0.987 0.714 0.782 0.733

I 0.993 0.908 0.910 0.376 0.617 0.832 0.172

IRL 0.861 0.615 0.584 0.094 0.278 0.025 0.000

NL 0.231 0.699 0.942 0.282 1.000 0.085 0.841

P 0.057 0.415 0.126 0.348 0.055 0.104 0.366

S 0.468 1.000 1.000 0.414 0.967 0.660 0.841

UK 0.458 0.785 0.881 0.047 0.792 0.316 0.366

as they are obtained from the 14 self evaluation model runs (column ranges and column averages fromTable 4).

It is obvious that some alternatives display wider range than others. These alternatives are characterized

by dispersed performances along the criteria and so they are more sensitive to the modification of weights.On the other hand the alternatives with more narrow range (e.g., A and EL) are characterized by moreaccumulated performances in the criteria as it can be verified from the relative data in Table 3.




Table 4

Cross-evaluation matrix from method ACE+

Cross-evaluated alternative

Code A B D DK E EL F FIN I IRL NL P S UK

Self evaluating alternative

A 0.94 0.05 0.48 0.51 0.10 0.03 0.17 0.49 0.34 0.09 0.28 0.07 0.41 0.13

B 0.93 0.74 0.85 0.93 0.18 0.29 0.78 0.86 0.78 0.55 0.84 0.51 0.86 0.77

D 0.81 0.21 0.89 0.56 0.13 0.03 0.39 0.52 0.40 0.15 0.71 0.11 0.78 0.40

DK 0.78 0.16 0.35 0.96 0.14 0.04 0.25 0.63 0.62 0.22 0.21 0.11 0.39 0.26

E 0.95 0.39 0.79 0.90 0.71 0.10 0.70 0.84 0.76 0.42 0.72 0.49 0.88 0.56

EL 0.90 0.07 0.44 0.75 0.13 0.24 0.40 0.67 0.55 0.31 0.39 0.11 0.49 0.38

F 0.91 0.26 0.75 0.84 0.36 0.21 0.83 0.76 0.80 0.48 0.62 0.20 0.92 0.66

FIN 0.90 0.29 0.55 0.91 0.27 0.09 0.35 0.90 0.53 0.15 0.41 0.23 0.64 0.44

I 0.87 0.21 0.52 0.87 0.16 0.05 0.39 0.57 0.85 0.32 0.29 0.15 0.61 0.37

IRL 0.95 0.54 0.80 0.92 0.19 0.26 0.73 0.73 0.87 0.77 0.63 0.35 0.71 0.67

NL 0.78 0.33 0.84 0.67 0.21 0.07 0.47 0.62 0.42 0.20 0.85 0.21 0.85 0.47

P 0.94 0.50 0.79 0.93 0.52 0.24 0.69 0.87 0.76 0.38 0.82 0.70 0.87 0.59

S 0.70 0.25 0.70 0.65 0.13 0.01 0.55 0.54 0.51 0.14 0.64 0.10 0.93 0.42

UK 0.95 0.51 0.91 0.97 0.26 0.26 0.76 0.94 0.87 0.51 0.80 0.29 0.94 0.86

Avg 0.88 0.32 0.69 0.81 0.25 0.14 0.53 0.71 0.65 0.34 0.59 0.26 0.73 0.50

Fig. 3. Ranges and average values for the multicriteria scores of the alternatives.

Subsequently, in order to examine the effect of the different objective function in the self-evaluation

model between the AXE and the ACE+ method we use the following procedure: We first solve theproblem with AXE and then we solve the problem with ACE+ but restricting the value functions to belinear (hereafter named as ACE+LIN). It is interesting to note that the objective function of ACE+LIN

leads to more balanced weights than AXE, as it is shown in Table 5. This is explained by the fol-lowing reasoning: in AXE, if a criterion has the majority of alternatives with high performances (pos-itive skew), the corresponding self-evaluation problems will attribute this criterion with relative high




Table 5

Weight distribution from AXE and ACE+LIN

A B D DK E EL F FIN I IRL NL P S UK Avg

Weights from AXE

w1 0.05 0.05 0.05 0.30 0.30 0.05 0.05 0.05 0.30 0.30 0.05 0.05 0.05 0.05 0.12

w2 0.05 0.30 0.05 0.05 0.05 0.05 0.30 0.20 0.20 0.30 0.05 0.30 0.30 0.20 0.17

w3 0.30 0.30 0.30 0.20 0.05 0.30 0.30 0.30 0.30 0.20 0.30 0.05 0.30 0.30 0.25

w4 0.05 0.05 0.05 0.30 0.20 0.05 0.05 0.30 0.05 0.05 0.05 0.20 0.05 0.05 0.11

w5 0.05 0.05 0.20 0.05 0.05 0.30 0.20 0.05 0.05 0.05 0.30 0.05 0.20 0.30 0.14

w6 0.30 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.05 0.07

w7 0.20 0.20 0.30 0.05 0.30 0.20 0.05 0.05 0.05 0.05 0.20 0.30 0.05 0.05 0.15

Weights from ACE+LIN

w10.05 0.05 0.05 0.30 0.30 0.05 0.05 0.05 0.30 0.30 0.05 0.05 0.05 0.05 0.12w2 0.05 0.30 0.05 0.05 0.05 0.05 0.30 0.05 0.20 0.30 0.05 0.20 0.20 0.20 0.15

w3 0.05 0.30 0.20 0.05 0.05 0.05 0.30 0.05 0.05 0.20 0.20 0.05 0.05 0.30 0.14

w4 0.30 0.05 0.05 0.30 0.30 0.30 0.05 0.30 0.05 0.05 0.05 0.30 0.05 0.05 0.16

w5 0.05 0.05 0.30 0.05 0.05 0.30 0.20 0.05 0.05 0.05 0.30 0.05 0.30 0.30 0.15

w6 0.30 0.05 0.05 0.20 0.05 0.05 0.05 0.30 0.30 0.05 0.05 0.05 0.05 0.05 0.11

w7 0.20 0.20 0.30 0.05 0.20 0.20 0.05 0.20 0.05 0.05 0.30 0.30 0.30 0.05 0.18

weight (e.g., criterion 3 in our example). Therefore, the specific criterion will be characterized by highweights among the 14 set of weights and thus relative high average weight. In the ACE+LIN method

this does not happen because the objective function does not promote alternatives with high perfor-mances independently of the other alternatives. In other words, the high performance of alternative i

in criterion j (in relation to the other criteria) it is not enough to attribute high weight to the spe-cific criterion according to the self-evaluation model (as it happens in AXE). The performance of theother alternatives in this criterion could be in average higher than the performance of the ith alterna-

tive, driving the self-evaluation model of the ith alternative not to choose a relatively high score forcriterion j .

Regarding the comparison between ACE+LIN and ACE+ the basic difference is that the scores of each

alternative resulted from ACE+ vary in a wider range than the respective scores from ACE+LIN. This is

explained by the greater number of degrees of freedom inACE+ (adjustable and not fixed value functions)which allows for more variability in the calculated scores.

Conclusively, some general remarks concerning the results are outlined. It is obvious that solely theaverage score obtained from the cross-evaluation matrix is not enough to judge the alternatives. However,

the judgment can be significantly enriched with information concerning the corresponding score-rangesthat is provided by the cross-evaluation matrix. As it was expected, the countries which present widerange of performances across the criteria result in a wide range of scores in the cross-evaluation matrix

(e.g., B, F, IRL, UK). These countries are very sensitive to the weight set that is used for their evaluation.On the contrary countries with more accumulated scores in the criteria (e.g., A, EL) result in a narrowermulticriteria score range and thus they are less sensitive to weights’ modification.




5. Concluding remarks

The situations where a multicriteria evaluation of alternatives has to be performed based on limited

information about the preferences of the DM are not rare. The self and peer evaluation of the alternatives,as it was inspired by DEA methodology, is a good remedy in such situations where the relevant informationmust be extracted from the data. One of the first approaches in this direction was AXE method that uses

the cross-evaluation of alternatives. In the suggested method ACE+ we are going one step further byallowing each alternative to use its own weights and value functions for the criteria. The price for the

more realistic representation of the decision situation is that a more difficult problem has to be solved inthe self evaluation phase of each alternative: Instead of an LP problem with the weights of criteria beingthe only decision variables, now we have an NLP problem with two sets of decision variables, namely

the weights of criteria and the value function parameters.The use of a different objective function in the self-evaluation model of ACE+ seems to lead to more

meaningful results, as it aims to optimize the relative position of the respective alternative in relation tothe others, and not an absolute measure of its performance.

The method ACE+ requires only a minimum of information in order to be applied and it provides awealth of decision making relevant information. In ACE+ the DM is not limited solely to average values

but he/she is encouraged to develop an “information warehouse” (ranges of scores, information aboutthe weights and the value functions, pairwise comparisons) that is very useful for the final decision.

Consequently, ACE+ inherently performs a parallel sensitivity analysis of the results, leading to morerobust decisions.

Another attractive issue of ACE+ is that its baselines can be easily understood by non-experts. For

example, ACE+ can be very useful in decision situations where each alternative is represented by astakeholder. It is a method that easily can be agreed upon as it provides the results in an objectiveand democratic manner. Each stakeholder assigns its most favourable weights to the criteria in order to

promote his/her alternative. The final judgment is then emerges from the cross-evaluation (self and peerevaluation) of each one of the alternatives.

Some ideas for future research regarding ACE+ include the following: Different objective functions canbe tested in the self evaluation model. More types of value functions, besides the usual concave and convexvaluefunctions, maybe incorporatedin the model(e.g., S-curves) in order to make themodel more general.

Finally, it will be a good idea if the method is adjusted so that it can be used iteratively, in the case that theDM is available. The DM exploits the obtained information and expresses his/her preferences which areincorporated in the model through an interactive process, which hopefully converges to the final decision.

Acknowledgements

The authors should like to thank the two anonymous referees for their stimulating comments that enrichthe paper.

References

[1] Charnes AW, Cooper WW, Rhodes E. Measuring efficiency of decision making units. European Journal of Operational

Research 1978;2:429–44.




[2] Stewart T. Data envelopment analysis and multiple criteria decision making: a response. Omega 1994;22(2):205–6.

[3] Stewart T. Relationships between data envelopment analysis and multicriteria decision analysis. Journal of the Operational

Research Society 1996;47:654–65.

[4] Doyle J, Green R. Data envelopment analysis and multiple criteria decision making. Omega 1993;21(6):713–5.[5] Belton V, Vickers S. Demystifying DEA—A visual ineractive approach based on multiple criteria analysis. Journal of the

Operational Research Society 1993;44(9):883–96.

[6] Oral M, Kettani O, Lang P. A methodology for collective evaluation and selection of industrial R&D projects. Management

Science 1991;37(7):871–85.

[7] Sarkis J. A comparative analysis of DEA as a discrete alternative multiple criteria decision tool. European Journal of

Operational Research 2000;123:543–57.

[8] Green R, Doyle J, Cook W. Preference voting and project ranking using DEA and cross evaluation. European Journal of

Operational Research 1996;90:461–72.

[9] Sexton T, Silkman R, Hogan A. Data envelopment analysis: critique and extensions. In: Silkman R, editor. Measuring

efficiency: an assessment of data envelopment analysis. San Francisco: Jossey-Bass; 1986.

[10] Doyle J, Green R. Efficiency and cross-efficiency in DEA: derivations, meanings and uses. Operational Research Society

1994;45(5):567–78.

[11] Doyle J. Multiattribute choice for the lazy decision maker: let the alternatives decide!. Organizational Behavior and Human

Decision Processes 1995;62:87–100.

[12] Banker R, Charnes A, Cooper W. Some models for estimating technical and scale inefficiencies in data envelopment

analysis. Management Science 1984;30:1078–92.

[13] Thanassoulis E. Introduction to the theory and application of data envelopment analysis. Massachusetts: Kluwer Academic

Publishers; 2001.

[14] Cooper WW, Seiford L, Tone K. Data envelopment analysis: a comprehensive text with models applications, references

and DEA-solver software. Massachusetts: Kluwer Academic Publishers; 1999.

[15] Adler N, Friedman L, Sinuany-Stern Z. Review of ranking methods in the data envelopment analysis context. European

Journal of Operational Research 2002;140:249–65.

[16] Zeleny M. Multiple criteria decision making. New York: McGraw-Hill; 1982.

[17] Keeney RL, Raiffa H. Decisions with multiple objectives: preferences and value tradeoffs. New York: Wiley; 1976.

[18] Von Winterfeldt D, Edwards W. Decision analysis and behavioral research. New York: Cambridge University Press; 1986.[19] Belton V, Stewart T. Multiple criteria decision analysis: an integrated approach. Massachusetts: Kluwer Academic

Publishers; 2002.

[20] Bouyssou D. Using DEA as a tool for MCDM: some remarks. Journal of the Operational Research Society 1999;50:

974–8.

multi criteria decision analysis with minimum information combining dea with mavt original research...

Documents