FROM AGGREGATING BY RULES TO THE INTEGRATION OF EXPERT SYSTEMS IN MULTICRITERIA DSSs

J.-Ch. POMEROL LAFORIA / IBP

P. and M. CURIE University 4 Place Jussieu, 75252 PARIS 5, France

Abstract The aims of this paper relies on tradeoffs between more or less contradictory firstly we present a survey of criteria. We can directly express these tradeoffs by rules. Let systems or works developed to merge Multicriteria Decision Making (MCDM) and Artificial Intelligence (AI), . in the second part of the paper, we examine the properties of the knowledge-based or "Intelligent" Multicriteria Decision Support Systems (IMDSS).

I-NIRODUCTION

Papers relating AI and MCDM appear here and there, but it is quite diffiult to exactly map the border betweenlthe two fields and to organize the various contributions in a understandable framework. It is such a comprehensible framework that we would like to introduce here, reviewing, by the way, a large number of papers. To our knowledge, four previous attemps were made with almost similar aims, the first unpublished one [ 11 and the three others, limited to expert systems [2], [3] and [4].

We will distinguish three main interactions between MCDM and AI. The first one, developed in section 11, is concerned with the aggregation process.

The third section surveys the various supports that expert system (ES) technology may offer at different steps of the multicriteria decision process. Section IV is devoted to the direct use of AI methods to make a multicriteria choice, while the fifth section, more prospective, tries to evaluate the possible applications of MCDM to AI. Finally the last section is concerned with multicriteria DSSs.

11-AGGREGATION AND RULES

2.1.Aggregation by rules

Let us think of the multicriteria decision making process as a reasoning process. The decision maker's (DM) choice

us give an example, drawn from [5] concerning a car:

If SECURITY is high and COMFORT is very good then TECHNICAL-CHARCTERISTICS are very good.

It is clear that the preceding rule aggregates the two criteria "SECURITY" and "COMFORT" under the name "TECHNICAL CHARACTERISTICS". Fundamentally a rule i s an aggregating process. When talking of a rule as an aggregating process we can imagine, at least, two situations. Firstly the rules are interpreted as a not clearly modeled aggregation process depending on the alternatives and on the satisfaction levels. In this case the final evaluation of an alternative depends on a variable unstructured collection of facts. We are in a classical ES process in which each rule is a "local" mental aggregating process.

Different is the case where each criterion is defined as an aggregation of hierarchically lower criteria. For example the programming quality of a microcomputer may be divided into langage and quality of the user environment, and the user environment is further divided into hardware and operating system utilities ( example drawn from [6]).

Then each rule gives the values of the "functional schema" [7], e.g. for the functional schema (langage, user environment)--->quality of the microcomputer, two rules would be:

If the LANGAGE is acceptable and if the QUALITY

Then the QUALITY OF THE MICRO is good OF THE USER ENVIRONMENT is good

If the LANGAGE is poor and if the QUALITY OF

Then the QUALITY OF THE MICRO is bad THE USER ENVIRONMENT is good

The semantic schema tree sustaining the reasoning of the ES is thus an aggregating net with the advantage of legibility.

2.2.Rules and functions

As already noted in [7] a schema expresses a discrete functional relation which may be restated as a decision table [8]. The problem is that being discrete, this function presents jumps which are regarded as drawbacks in many applications. By giving values in [O, l l and replacing the table by a continuous function (e.g. QUALITY OF THE MICRO = 0.4 LANGAGE + 0.6 ENVIRONMENT), Saaty's method [9] avoids the gap effect inherent to the discrete world. From MYCIN certainty factors, to fuzzy logic, many ideas were introduced in the ES field for the same reasons.

Rather than replacing the rules by continuous functions, an alternative consists of smoothing the discrete function defined by a table. References [ 101, [ 1 11 and [ 121 have thus used various interpolation functions to pass from a discrete to a continuous function.

2.3. Criticisms

The aggregation process based on rules suffers some drawbacks. Firstly, the construction of the rules is generally considered as being too empirical to be reliable. At least the rules in their symbolic form are readable for the decision maker (DM). However, it is true that if the rules are unorganized in a messy file of multiple "local ideas", the readibility disappears and the result of the aggregation is unpredictable. When the aggregation is organized along a semantic tree, as explained above, the process is much more readable. Moreover, the rules associated with a schema like (LANGAGE, ENVIRONMENT)--->QUALITY contains more information than a function such that: QUALITY OF THE MICRO = 0.4 LANGAGE + 0.6 E"lMENT in which the parameters 0.4 and 0.6 are fixed.

Another criticism pointed out in [3] and [13] and emphazised in [14] is that the DMs preferences are not explicitly considered because the rules are generally not centered on the DMs objectives. This criticism is certainly true and important in the general setting of ES. As pointed out in [ 131, the explicit distinction between choice (or user preferences) and descriptive facts is fundamental both from decision theory and practice points of view. We will retum to this question in the next section. But, in the MCDM context, the situation is somewhat different, because the

limit between facts and objectives depends heavily on the DM's interpretation. More precisely, it is the DM who draws the frontier between objectives and factual alternatives.

III-EXPERT SYSTEMS AND M O M

3.1. Evaluating the alternatives or constructing the utility functions

It has been frequently reported by practioners that DMs are often upset by the evaluation of the alternatives according to their various criteria. The problem is equivalent to that of building a utility function (numerical or symbolic) starting from a given set of alternatives. Contrary to what was examined in Section II, the output of the system is now not a choice, i.e. an ordering of the alternatives, but an evaluation of the alternatives according to several attributes. ,

To build a symbolic utility function (i.e. whose values belong to an ordered set of "strings") we can imagine the use of an ES. This idea appears in [ 151.

The simplest structure consists of an ES processing various data to assess the value of different attributes. The main criticism of this technique is, as above, that it merges choice and descriptive items. To overcome this diffficulty and to separate the choice (preference) from the descriptive (factual) part of the alternatives, Uvine et al. [16] have imagined a structure in which each alternative is characterized by a fact base and each criterion by a rule base. The rule base is organized as a semantic tree of criteria. The evaluation of an alternative is made by applying the rules to the facts characterizing each alternative. The result of the system is a decision matrix.

It is clear that the semantic tree of criteria in the above method is not very far from the hierarchical tree in Saaty's method [9]. The only difference is that the aggregation is made by rules discussed with the DM rather than by number crunching.

Another idea for evaluating the alternatives consists of using some non-classical logics to describe the preferences of the DM. Tsoukias [17] and Mainka et al. [18] have designed such a system which moreover uses the "Truth Maintenance System" of de Kleer [ 191.

Finally, it is possible to use a neural network to evaluate the alternatives [20].

484

I

3.2.Supporting the decision maker's choice

The rules can also be used either to direct the exploration process in the altemative set (or possibly in the aspiration level set) or to support the choice in a multicriteria decision making method. Reference [2] provides such an example. In the same vein, [21] uses rules to choose between basic

method during the choice among multicriteria methods.

in [241, [251 and [261. The ES can alsohe used either to determine the weights

according to the context [27,28] or modify them according to the "satisfaction levels" [29].

3.3. Constructing the alternatives

In many real situations, the first step is not even to assess the altematives but to understand what an altemative is. In many economical or strategical decisions, the altematives are actually scenarios . Let us give an example. The problem is to assess the robustness of railway timetables [30]. Given a railway network (RN) and the machines (M) we can define theoretical timetables (TT). Now, when an incident occurs, the train schedules are perturbed and it is evident that the importance of the perturbation depends on RN, M, and TT. An incident generally delays many trains, but the importance of the total delay depends on the network and-the timetable. Assume that you want to assess an investment devoted to improving the situation in case of incident. It means that you are obliged to evaluate the robustness of the alternatives of the form (RN, M, TT) when an incident occurs. It is a typical case where human reasonning is unable to construct, without help, what is called in [30] the fully expanded alternative ( F E A ) . The fully expanded altemative associated with an alternative (RN, M, TT) is the real timetable incorporating all the delays caused by various incidents. To "propagate" the incidents you need support. In [30], an Expert System plays the role of the dispatchers, i.e. it makes the decisions, resulting from an incident. Here the E.S. or more generally the DSS is "simply" used to bridge the gap between

alternatives and 'yully expanded alternatives ". We believe that this type of problem will become more and more frequent as people a d m s more complex problems.

Another example of the use of an ES in order to build an altemative is presented in [3 13 where a frame-based system determines the possible pathology of a patient. The plausibility of these pathologies are then evaluated by a multicriteria method (PROMEITHEE).

IV-DIRECT USE OF AI PDR MuLTlCRllERM DECISION

/ 4.1.Exploration of the alternative set

The altemative set may be regarded as a state space in Newel1 and Simon sense [32]. Thus, after defining the operators, the decision maker can perform a k d s t i c search in this space. This idea was exploited in PRIAM [33,34]. A resulting software named MULTIDEKISION was developed involving PRIAM algorithm [35]. In PFUAM, each state essentially contains the "satisfaction levels" resulting from the real or fictive altemative(s) on the spot. It is the reason why the method is somewhat reminescent of the elimination by aspects [36]. PFUAM methodology has been exploited again in some new packages [37]. The same idea of interactive search is also used in AIM [38] at one of the steps of the the method.

4.2.Lexicographic order and PROLOG

Assume that you write a rule in PROLOG style asserting that quality depends on comfort and technical characteristics

Comfort(X)ATech. Charact.(X) ---> Quality(X)

Assume that PROLOG seeks to instanciate the variable X, then the order in which the rules are written influences the order in which the instanciations will be obtained. This therefore produces a kind of lexicographical order. This idea was, with some consequences about classification, developed in [39].

4.2. Rules replace weights

Another direct use of AI programming is developed in EXTRA [40]. Let J be the set of criteria and 2 the set of J

J subsets of J. Pasche defines on 2 a relation > with some J extra reasonable properties. The order resulting from the relation > replaces the weights. J

V-APPLICATION OF MULTICRllFNA To AI

This section is presently more prospective than the others because it seems that most AI scientists are not aware of multicriteria methods.

5.I.Planning

One may distinguish two phases in planning : searching the actions to be implemented in order to perform certain tasks and ordering these actions in a "possible order" providing the "best" plan. It is clear that the second operation is related to ordering and could take advantage of multicriteria techniques. Considering two actions A and B we will say that A --> B if B is feasible after A, in other words if the conclusion of A does not inhibit the precondition of B. The relation --> has some resemblance with a preorder. The possibilities of multicriteria methods in such a context have not yet be explored.

5.2. Distributed AI

Distributed AI is interested in creating "computer agents" that interactively act in order to perform some tasks. The performances of these agents are evaluated to make decision: who is allowed to work, who is momentanetaly inhibited for example, the aim being to enable the agent who performs best on the particular task.

This evaluation preceeding the self-decision concerning an agent is generally made along one criterion (a kind of metrics), it would gain in sophistication to make this decision along many criteria evaluating more precisely the performance of each agent.

5.3. Evaluation by several criteria in AI search

Finally AI has introduced the notion of heuristic search. Some of the algorithms performing these heuristic searches are well-known such that A , A 0 and some few others. At each step in a heuristic search, the system evaluates newly developed states (nodes). Up to now, it was done according

* *

to only one criterion (evaluation function). Recently, in [41], [42] and [43] the idea of making a multicriteria evaluation of the nodes was explored and a generalized version of A , dealing with multicriteria evaluation, was

*

proposed.

VI -MULTICRlTEiIUA DSSS

6.1 .Desirablefunctions for the choice

Following Simon [44] there are four phases in a decision process: intelligence, design, choice and review. In most cases MCDM people have focused on the choice. This is a shortcoming because the decision is strongly constrained by the fust two phases [MI. Anyway, as regards the choice, the main functions are:

- changing, adding and deleting criteria, - changing, adding and deleting altematives, - modifying the importance of the criteria (e.g.

weights) and making sensitivity analyses, - using various types of criteria (fuzzy, pseudo, etc), - visualization capabilities of the actions according to

- modifying the parameters of the aggregating process, - the possibility of changing the aggregation procedure, - the possibility of finding the weights to obtain a

Various multicriterion packages offer some of the above capabilities (see a complete review in [35]. Among them: AIM [38], DECISION PAD [46], EXPERT CHOICE [9], ELECTRE 1s [47], TRIPLE C [45], DEX [ l l ] , PROMCALC [48,49]. Moreover we have seen (3.2) that in some cases an ES may be useful to aid the DM to make his choice.

the attributes,

given ordering (TRIPLE C, [45]).

6.2 .Multicriteria DSSs

The many possible applications of multicriteria DSSs are surveyed in [50]. In fact there are many possible ways to support the DM on many crucial points others than choice. Some of them were already suggested in [22] and [51]. The first one is to build the alternatives (see 3.3). Other examples of using an ES for building the alternatives are [52, p. 249 et seq.] and [53]. Often the multicriteria module is connected to a spreadsheet which aids the management of the alternatives [28,54] or to a Data Base Management

System [%I. More generally the purpose of the designers is to provide a tool addressing all the functionalities needed for properly dealing with a given task, see e.g. [30,56,57,58,59,60,61]. In many of these systems the model contains the heuristics used by the experts who previously did the task.

We have already examined the possibility of helping the decision maker to evaluate the criteria (section 3.1). Finally a few papers examine learning and reviewing. Learning from examples is addressed via case-based reasoning in [62].

Several of the above systems merge ES modeling for qualitive data and O.R. modeling for quantitative data, paving the way to "Intelligent" DSSs which were advocated in several books [52,63,64].

W-CONCLUSION

Too often MCDM has compensated the triviality of the studied situations, by the sophistication of the aggregation methods. I believe that it has somewhat discredited the MCDM field to the practioners' eyes. By introducing AI and building "intelligent" DSSs, it becomes possible to address much more involved situations along at least two directions:

-enlarging the vision of the decision maker, firstly by supporting the enlargement of the alternatives set with regards to the values, secondly by an automatic design of the alternatives.

-using ES and others AI methods to really deal with "fully extended alternatives" which are much more realistic than the poor alternatives generally considered in MCDM.

[I] J.-Ch. Pomerol, "MCDM and AI : Heuristics, Expert and Multi- Expert Systems," Presented at EURO IX, Paris, July, 1988.

[2] J Reyes and S. Barba-Romero, "Expert Systems and multicriteria decision making," presented at EURO VIII Lisbon, Sept. 1986.

[3] C.C. White Ill, "A survey on the integration of Decision Analysis and Expert Systems for Decision Support," IEEE Transactions Syst. Man Cybern., vol SMC20 n02, pp.358-364,

[4] V.M. Ozernoy and A. B. Bordetski, "Using Expert System technology for Multiple Criteria Decision Making: a Survey," Proceedings of the 10th conference on Multiple Criteria Decision Making, Taipei, pp.27 1-280, August 1992.

[ 5 ] M. Bohanec. I . Bratko and V. Rajkovic, "An expert system for decision making." in Processes and Tools for Decision Support, H.G. Sol Ed., North Holland, 1983, pp.235-248.

1 9 9 0 .

161 J. Efstathiou, V. Rajkovic and M. Bohanec. "Expert Systems and Rules Based Decision Support Systems," in Computer Assisted Decision Making, G. Mitra Ed., Elsevier Science Publishers,

[7] P. Uvine and J.-Ch. Pomerol, "Knowledge representation by schemata in financial expert systems," Theory and Decision 27 n01/2, pp.147-161, 1989.

[8] J.-Ch. Pomerol, "A comprehensive view of the relationships between MCDM and AI," presented to EURO XIYTIMS XXXI, Helsinki, June 1992.

[9] T.L. Saaty, The analytic hierarchy process, New-York: Mc Graw- Hill, 1980.

[lo] M. Bohanec, B. Urh. and V. Rajkovic, "Evaluating Options by Combined Qualitative and Quantitative Methods," 13th Conference on Subjective Probability, Utility and Decision Making, Fribourg, 199 1.

[ l l ] M. Bohanec and V. Rajkovic, "DEX : An Expert System Shell for Decision Support ." Sistkmica vol.1 nO1, pp. 145-158.. 1990

[I21 V. Rajkovic, M. Bohanec and V. Batagelj. "Knowledge Engineering Techniques for Utility Identification," Acta Psychologica 68, pp.271-286, 1988.

[13] R.L. Keeney, "Value-Driven Expert Systems or Decision Support," Decision Support Systems 4 n04. pp. 405-412. 1988.

[ 141 R.L. Keeney, Value-Focused Thinking, Cambridge: Harvard University Press. 1992

[15] D.E. O'Leary, "Multiple criteria decision making in accounting expert systems," Comptes rendus des 6iemes journQs internationales sur les systemes experts et leurs applications, Avignon, pp.1017-1035, May 1986.

[I61 P. Uvine, M.4. Pomerol and R. Saneh, "Rules Integrate Data in a Multicriteria Decision Support System," IEEE transaction on Man, Machine and Systems v01.20 n03, pp.678-686, May 1990.

[17] A. Tsoukias, "Preference modeling as a reasoning process: a new way to face uncertainty in Multiple Criteria Decision Support Systems," Eur. J. of Opl. Res. ~01.55, pp. 309-318, 1991.

(181 J. Mainka, C. Rosenthal-Sabroux and A. Tsoukias, "AI techniques and Multicriteria aid: a new approach in dealing with incoherence and inconsistency in a preference model," Presented at EURO XI1 / TlMS XXXI, Helsinki, June 1992.

[I91 J. De Kleer, "An assumption-based Truth maintenance system," Artificial Intelligence

[20] J. Wang and B. Malakooti, "A feedforward network for multiple criteria decision Making," Computers Ops. Res. ~01.19, pp. 151- 167, 1992.

[21] N. Honda and T. Mimaki, "Multiobjective Decision Method Using Heuristic Rules," in Artificial Intelligence in Economics

and Management, L.F. Pau Ed., pp.157-165, 1986. [22] G. Balestra and A. Tsoukihs A., "Multicriteria Analysis

Represented by Artificial Intelligence Techniques," J. Opl. Res. Soc. vol. 41 n"5, pp.419-430, 1990.

[23] T. Jelassi, "MCDM : from "Stand-Alone" Methods to Integrated and Intelligent DSS," in Toward Interactive and Intelligent Decision Support Systems, Y. Sawaragi , K. lnoue and H. Nakayama Eds., Heidelberg: Springer, pp.90-99, 1986.

[24] J. Teghem, C. Delhaye and P.L Kunsh, "An Interactive Decision Support System (IDSS) for Multicriteria Decision Aid," Mathl. Comput. Modelling. v01.12, nO1O/ll, pp.1311-1320. 1989.

[25] I.B. Hong and D.R. Vogel. "Data and Model Management in a Generalized MCDM-DSS," Decision Sciences 22, pp. 1-25. 1991.

1986, pp.165-174.

vol. 28, pp.127-162. 1986.

[261 V.M. Ozemoy, "Developing an interactive decision support for discrete alternative MCDM method selection," in Multiple Criteria Decision Support, P. Korhonen, A. Lewandowski and J. Wallenius Eds., Heidelberg: Springer, 1991, pp. 242-257.

[27] J.-M. Labat and M. Futtersack, "Un SIAD multicrithre pour la planification des Aides aux ch6meurs de longue durk," Decision Systems Journal vol.1 nO1, pp. 79-92, 1992.

[28] J.-Ch. Pomerol, M. Futtersack and J.-M. Labat, "Decision Support System in an Administrative context: The case of the Reinsertion of Long-Term Unemployed in France," in Decision Support in Public Administration, P. W. G. .Bots, H. G. Sol and R. Traunmiiller, Eds, Amsterdam:North-Holland, 1993,

[29] N.J. Mandic and E.H. Mamdani, "A multiple attribute decision making model with fuzzy rule-based modification of priorities," in Fuzzy Sets and Decision Analysis, E.J. Zimmerman, L.A. Zadeh and B.R. Gaines Eds, 1984, pp.285-306.

[30] J.-Ch. Pomerol , B. Roy, C. Rosenthal-Sabroux and A. Saad, "An "Intelligent" DSS for the Multicriteria Evaluation of Railway Timetables," EURO XI11 , Lisbon, July 1993.

[31] Ph. Du Bois, J.-P. Brans, F. Cantraine and B. Mareschal, "Medicis : An expert system for computer-aided diagnosis using the PROMETHEE multicriteria method," Eur. J. Opl. Res. vo1.39,

[32] A. Newell and H. A. Simon, Human Problem Solving, Englewood Cliffs: Prentice-Hall, 1972.

[33] P. Uvine and J.-Ch. Pomerol, "An interactive program for the discrete multiobjective problem," Methods of Operation Res.

[34] P. Uvine P. and J.-Ch. Pomerol, "PRIAM, an interactive program for choosing among multiple attribute alternatives," Eur. J. Opl. Res. vol 25, n02, pp. 272-280, 1986.

[35] J.-Ch. Pomerol and S. Barba-Romero, Choix multicrit&re dans I'entreprise, Hermks: Paris., 1993

[36] A. Tversky, "Elimination by Aspects," Psy Review vo1.79, pp.

. [37] P. Fiala,, "Artificial Intelligence methods in Multicriteria Analysis," in , Multicriteria Decision ,Making, Methods, Algorithms, Applications, M. Cerny, D. Gliickaufova agd D: Loula, Eds>kzechoslovak Academy of Sciences, Prague, 1992;

[38] V. Lotfi and S. Zionts, "AIM, Aspiration-Level Interactivg Method for Multiple Criteria Decision Making," User's Guidk, State University of New York at Buffalo, 1990:

[39] Y. Rommel, "Une proctdure interactive reposant sur-, des . techniques d'apprentissage symbolique: 'Bibliographje' sur

I'appren-tissage symbolique par induction partir d'exenfples et prtsentation de 18 proctdure interactive PLEXIGLAS? Cahier LAMSADE .n067iZ:Paris: Universitt de Paris Dauphine.

[40] C. ,J'asche,*"EXTRA ? An expert system for multicrityfia decision making," Eur. J. Opl.

[41] Y. Lirov, "Algorithmic multi-objective heuristics construction 'in

he A search," Decisbn Support Systems ~01.7, 'pp.1591167,

[42f.C.C. White Ill, "'The 'AUOR Interface,"lEEE Conference SMC,

[43] C.C. White 111. BS. Stewart and R.L. Carraway, "Multiobjective, ' preference-based seareh in acyclic OR-graphs,'' Eur: J. Qpl. Res. "~01.56, p~.157-36$ 1992.

pp. 133- 147.

pp.284-292, 1989.

~01.45, pp.479-480, 1983.

. ' ;,281-299. 1971.

pp. 58-65.

L '

'r

I

Res. vo1.52, pp. 224-234, 1991.

, * ;

1 9 9 1 . . .

Chicago, October ./ 1992: pp, xv-xvii.

0

[44] H.A. Simon, The New Science of Management Decision, Englewood Cliffs: Prentice-Hall, 1977.

[45] A.A. Angehm, "TRIPLE C, Overview and How to Use it," User's Guide, INSEAD, 1990.

[46] C. Kepner and J. Tregoe, The New Rational Manager, Princeton: Charles Kepner Associates, 198 1

[47] B. Roy and J.M. Skalka, "ELECTRE 1s. Aspects dthodologiques et guide d'utilisation," Cahier du LAMSADE n030, Paris: Universite de Paris-Dauphine, 1985.

[48] J.-P. Brans and B. Mareschal, "The PROMCALC and GAIA Decision Support System for Multicriteria Decision Aid,"

IFORS SPCl, Bruges, Mars 1991. [49] B. Mareschal and J.-P. Brans, "Geometrical Representations

for MCDA," Eur. J. Opl. Res. vo1.34, pp. 69-77. 1988. [50] B.H. Eom, "The Current State of Multiple Criteria Decision

Support Systems," Human Systems Man. vo1.8, 113-119, 1989. [51]T. Jelassi, M. Jarke and E.A. Stohr, "Designing a Generalized

Multiple Criteria Decision Support System," J. of Management. infor. Sys., vol.1 n04, 24-43.

[52] P. Uvine and J.-Ch. Pomerol, Systhmes Interactifs #Aide ?i la Wcision et Systhmes experts, Paris: Hermhs, 1989.

[53] R. Bayad et J.-Ch. Pomerol, "An "Intelligent" DSS for the Renforcement of Urban Electrical Power Networks," In decision support systems: Experiences and Expectations, T. Jelassi, M. Klein and W.M. Mayon-White Eds., Amsterdam: Elsevier, 1992,

[54] F.J. Vasko, M.L. Cregger, L.G. Lutsko, D.D. Newhaxt and D.J. Renn, "A Multiple-Attribute-Based Expert Support system for Customer/Order Evaluation," J. Opl. Res. Soc. vol 41. nO1l,

[55] Y. Siskos and N. Matsatsinis, "A DSS for market Analysis and new product Design," Journal of Decision Systems vol. 2 nO1,

[56] A. Beurrier, Ph. Demota, P. Uvine and J.-Ch. Pomerol, "Distribution of Empty Railcars by an Expert System: a Case Study with Comparison with O.R. Approaches," J. Opl Res. Soc. vol 4, n06, pp. 527-537,

[57] P. Uvine and J.-Ch. Pomerol, "Railcar Distribution at the French Railway," IEEE Expert, vo1.5 n05, October 1990.

[58] Wu De-Shih and M.T. Tabucanon, "Multiple Criteria Decision Support System for Production Management," Engineering Costs and production Economics v01.20, pp.203-213, 1990.

[59] T.J. Stewart, "A Multi-Criteria Decision Support System for R&D Project Selection," J. Opl. Res. Soc. vol. 42 nO1, pp.17-26,

[60] B.Mareschal and J.-P. Brans, "BANKADVISER: an Industrial Evaluation System," Eur. J. Opl. Res. vo1.54, pp. 318-324, 1991

[61] R.R. Weitz and T. Jelassi, "Assigning students to groups: a Multi-Criteria Decision Support System Approach," to appear in Decision Sciences, 1992.

[62] A.A. Angehrn and S. Dutta, "Integraging Case-Based Reasoning in Multicriteria Decision Support Systems," In decision support systems: Experiences and Expectations, T. Jelassi, M. Klein and W.M. Mayon-White Eds., Amsterdam: Elsevier,

[63] E. Turban, Decision Support and Expert Systems, Reading:

[64] M. Klein and L. B. Methlie, Expert Systems: a Decision Support

pp. 153-165.

pp.1077-1080, 1990.

pp.35-60, 1993.

1990.

1 9 9 1 .

1992, pp.133-150

Addison-Wesley, 1988.

Approach, 1990.