Report Evaluation Model for Analysts in Mutual Fund

Companies

Hsin-Yuan Chang*,

Insurance and Financial Management Department,

Takming University of Science and Technology, 56, Sec.1, Huanshan Rd., Nei Hu,

Taipei 11451, Taiwan, R.O.C.

E-mail: hychang@mail.takming.edu.tw

Yu Ching Ho

Logistics Management Department, Dahan Institute of Technology, No.1, Shjen

Street, Dahan ,Sincheng Hualien 971 Taiwan , R.O.C.

E-mail:bessie@ms01.dahan.edu.tw

Abstract

This study develops an evaluation and selection model of brokerage firm’s

research reports by modified Delphi method and AHP. This model provides a distinct

approach to examine brokerage firm’s research reports. We establish four main-

criteria and nineteen sub-criteria to evaluate brokerage firm’s research reports. The

primary criterion is the objective of brokerage firm’s research reports. No bias

statement, no serious omission and detail description of information source are key

sub-criteria to reach objective. Consistent evaluation of target firm’s industry is also

important evaluation sub-criterion.

Keywords

Modified Delphi method, AHP, Brokerage firm’s research reports, Objective

1. Introduction

1

The major function of analysts in mutual fund companies is to provide investment

target selection suggestions to portfolio managers. With the boarder range of

investment area, to visit every company directly in the universe becomes a mission

impossible. In order to increase their understanding of the investment targets, analysts

must rely on research reports from all kinds of sources.

Although these reports play an important role in making investment decisions, the

purposes of issuing reports may differ from each other. For example, industrial

statistical data published by government is aim for increasing the transparency of

markets, but investment banks issue research papers may be in a willing to promote

their IPO stocks. Academic research shows that research reports from brokerage

firms are often optimistic or biased due to the brokerage firm analysts’ career

concerns and conflicts of interest inside the brokerage firm (e.g. Elton et al. ,1986;

Womack, 1996; Barber et al., 2001, 2003; Hong & Kubik, 2003; Azzi, Bird,

Ghiringhelli & Rossi, 2006; Cowen, Groysberg & Healy, 2006; Jacob, Rock & Weber,

2008). Therefore, carefully evaluation and selection between reports become a key

successful factor of creating superior investment performance.

Most related literatures focus on analyzing the information content of the brokerage

firm’s research reports or why and how those reports are biased. We lack an approach

to identify a most valuable brokerage firm’s research reports. To establish an

evaluation and selection model of brokerage firm’s research reports is the main

purpose of our study.

Analysts must evaluate the value of each report and decide a valuable report by

various selection criteria such as assumption of the research report, timely published,

conscientious, objective, etc., simultaneously. In this study, a model incorporated with

the Modified Delphi method and the Analytical Hierarchy Processing (AHP) method

2

to select the most valuable brokerage firm’s research reports is proposed. The

Modified Delphi method is adopted to extract the criteria from asking the opinions of

a panel of experts, and the AHP method is used to decide the priority of those criteria

gathered from the Modified Delphi method and compute the relative weight of each

alternative research reports.

To enhance investment performance of mutual funds, predictive suggestions of

investment targets from analysts are essential. Research reports with good quality help

analysts provide precise suggestions. Facing variety reports, this evaluation and

selection model supports analysts identify useful reports easily. This study attempts to

make a contribution to a better appreciation of brokerage firm’s research reports

through the viewpoint of analysts.

The remainder of this paper is organized as follows: Section 2 describes the

research methodology we use in this study. Section 3 develops the model. Section 4

discusses and concludes.

2. Research methodology

The purpose of this study is to establish a model to help investors to evaluate and

select the valuable brokerage firm’s research reports by the Modified Delphi method

and the AHP method. Before developing the model, we will introduce the Modified

Delphi method and AHP in general.

Modified Delphi method

The Delphi method is an approach to elicit experts’ opinion by an iterative process

without face to face grouping discussion. It is a systematic process that attempts to

obtain group consensus in much more open and in-depth research ( MacCarthy &

Atthirawong, 2003). Series stages of questionnaires are designed to elicit and refine

3

common opinions within a pre-selected panel of experts via mail. Murry and

Hammons (1995) proposed the Modified Delphi method which enables researchers to

shorten the determination process. The difference between these two methods is that

the Modified Delphi method needs to develop a structured questionnaire by literature

review or expert interview instead of open-ended questionnaire in the first stage. By

using structured questionnaire, the research horizon will be shortened and objective-

related criteria could be determined faster. Because the research burden of

ANALYSTS is heavy, that would be a tough work to have them discuss together by

face to face meeting. Therefore using the Modified Delphi method is an appropriate

approach to collect opinions of ANALYSTS about the criteria decisions they used to

evaluate and select brokerage firm’s research reports without disturbance.

Analytical hierarchy process

The Analytic Hierarchy Process developed by Satty is a kind of multi-criteria

decision making (MCDM) techniques and enables decision makers to represent the

simultaneous interaction of many factors in complex and unstructured situations. It

helps them to identify and set priorities on the basis of their objectives and their

knowledge and experience of each problem and provide a structured approach to

decision making (Saaty, 1999). The AHP is performed well to solve complex

decision-making problems in different areas, such as planning (Kwak & Lee, 2002;

Radash & Kwak, 1998), resources evaluation and allocation (Alphonce, 1997; Jaber

& Mohsen, 2001; Hsu, Wu & Li, 2008), measuring performance (Frei & Harker,

1999; Ahsan & Bartlema, 2004), choosing the best policy after finding a set of

alternatives (Poh and Ang, 1999; Chang et al., 2007), setting priorities (Schniederjans

and Wilson, 1991). The first step is to decompose a complex situation into relevant

4

main criteria and sub-criteria, then using these criteria to establish a hierarchy

structure. A basic hierarchy model of AHP including four levels (Figure 1). The top

level is the goal we want to achieve. The second and third levels are criteria and sub-

criteria respectively. Since human being could not compare too many elements

simultaneously, the elements in each main criteria and sub-criteria should not exceed

seven. Under this limitation, it may carry on the reasonable comparison and easier

ensure the consistency (Satty, 1980). The bottom level is the replacement alternatives.

Figure 1. A basic hierarchy model of AHP

To derive the priorities of main criteria and sub-criteria within the hierarchy

structure, AHP incorporates the comparison judgments of all decision makers into a

final decision, without having to elicit their utility functions on subjective and

objective criteria, by pair-wise comparisons of the alternatives and check their

consistency (Saaty, 1990). The various hierarchies’ elements weight computation are

shown as follow:

(1). Establish the pair - wise comparison matrix A

If there are n elements, it must carry out n(n-1)/2 elements pair-wise comparisons.

Let C1, C2, , Cn denote the set of elements, while aij represents a quantified

judgment on a pair of elements Ci, Cj. The relative importance of two elements is

rated using a scale with the values 1, 3, 5, 7, and 9, where 1 refers to “equally

5

important”, 3 denotes “slightly more important”, 5 equals “strongly more important”,

7 represents “demonstrably more important” and 9 denotes “absolutely more

important”. This yields an n-by-n matrix A as follows:

(1)

Where and In matrix A, the problem turn

into assign the n elements C1, C2, …, Cn a set of numerical weights W1, W2, …, Wn that

reflects the recorded judgments. If A is a consistency matrix, the relations between

weights Wi and judgments aij are simply given by (for )

and matrix A as follows:

(2)

(2) Eigenvalue and Eigenvector calculation

Matrix A multiply the elements weight vector ( ) equal to , that is ( - )

=0, the is the Eigenvalue ( ) of Eigenvector. Due to is the decision makers’

subjective judgment comparison and appraisal, the true value ( ) may be a little

different, so that is unable to set up. Satty (1990) suggested that the largest

eigenvalue be:

(3)

If A is a consistency matrix, eigenvector X can be calculated by

(4)

(3) Consistency test

The essential idea of AHP is that a matrix A of rank n is only consistent if it has

6

one positive eigenvalue n = λmax while all other eigenvalues are zero. Further, Saaty

developed the consistency index (CI) to measure the deviation from a consistent

matrix:

(5)

The consistency ratio (CR) is introduced to aid the decision on revising the

matrix or not. It is defined as the ratio of the CI to the so-called random index (RI)

which is a CI of randomly generated matrices:

(6)

For n = 3 the required consistency ratio (CR Goal ) should be less than 0.05, for n

= 4 it should be less than 0.08 and for n ≥ 5 it should be less than 0.10 to get a

sufficient consistent matrix. Otherwise the matrix should be revised (Saaty, 1994).

Once the priorities weights of each main criteria is established, then the relative

performance measures of the alternatives can be calculated in terms of each main

criteria and the best alternative is decided by relative performance scores.

3. Model development

A series steps to perform the AHP analysis are described as follows:

Step 1: Defining the evaluation criteria and sub-criteria used to select the valuable

brokerage firm’s research reports and establishing an AHP-based hierarchical

structure

The goal of our study is to select a valuable brokerage firm’s research reports. This

is the top level of the AHP-based model. Then we must break down our goal into

several elements as main criteria and sub-criteria, and arranges them hierarchically

through the Modified Delphi method. Although there are many literature discussed

brokerage firm analysts’ research reports, but there is few researches focused on the

topic we discuss in this study. By literature review, expert interview and some Taiwan

7

government regulations, the modified Delphi structure questionnaires are developed.

These regulations stipulate some norms that researchers of investment company or

securities company must obey before they publish research reports. After referring to

these literatures and in-depth expert interview, we select 28 indicators and organize

them into questionnaire to perform the Modified Delphi method. We sent the

Modified Delphi questionnaire to thirteen ANALYSTS, then analyze the opinion

feedback and extract decision elements, including four main criteria and nineteen sub-

criteria. All the criteria and denotation are summarized in Table 1. A hierarchy

structure shown as figure 2 is arranged by deep interview with three experts to ensure

the rationality of the hierarchy structure.

Table 1. The criteria extract by the Modified Delphi method

Criteria Definition

Main criteria 1.

assumption of research

reports (C1)

Sub-criteria for C1

value investing (SC1)

financial model (SC2)

consistent evaluation

of target firm’s

industry (SC3)

coherent evaluation

of target firm’s

industry (SC4)

(SC1)Ways to search for undervalued stocks

(SC2)Stock evaluation using common models

(SC3)consistent evaluation principals for all stocks

in the same industry

(SC4)coherent evaluation process for the whole

industry

Main criteria 2.

timely research (C2)

popular issues (SC5)

market trend (SC6)

predictions before

market shock (SC7)

(SC5)Address relevant opinions about popular

issues in today’s market

(SC6)Address relevant opinions about future

market trend

8

(SC7)Address relevant opinions before market

shock

Main criteria 3

conscientious (C3)

Honest stock

recommendation

(SC8)

work experience of

brokerage firm’s

research team (SC9)

industry experience

of brokerage firm’s

research team (SC10)

using financial model

correctly (SC11)

evaluating financial

accounting data

correctly (SC12)

careful examination

of every numeral and

statement published

(SC13)

(SC8)The recommendation are based on unbiased

and reasonable judgements

(SC9)Work experience of brokerage firm’s

research team in the investment related industries

(SC10)Work experience of brokerage firm’s

research team in the industries which they covered

(SC11)Choose suitable models and make sure the

methodology of the model using is correct

(SC12)Evaluate the inputs of models carefully

(SC13)Confirm the correctness of all the data

Objective (C4)

Applying

authoritative

information (SC14)

establishing multi-

information resources

(SC15)

detail description of

information source

(SC16)

true record (SC17)

no bias statement

(SC18)

no serious omission

(SC19)

(SC14) Refer to opinions of professionals or

information from authorities

(SC15)Confirm and compare data/ information

from different resources

(SC16)Clear description of each data/ information

sources

(SC17)Data is not fake or been changed

(SC18)Statements are unbiased and not related to

self interests

(SC19) no serious omission

9

10

11

Figure 2. The hierarchy structure of AHP-based model

Step 2: Establishing pair-wise comparison matrix of each factor

Based on the hierarchy structure, an AHP questionnaire is developed to make a

pair-wise comparisons in order to determine the relative priorities of each criteria. The

pair-wise comparisons are based on the scale of relative importance that assumes

values between 1 and 9. This scale can be applied with ease to criteria that can be

defined numerically as well as to those cannot be defined numerically. Relative

importance scale is presented. ANALYSTS is supposed to specify their judgments of

the relative importance of each contribution of every criterion towards achieving the

overall goal.

In this study, a purposive expert sampling is applied to sample ten respondents

from various ANALYSTS. The weights of level 2 criteria and level 3 sub-criteria are

then determined for a sample group of ten individuals matching the above

characteristics with each respondent making a pair-wise comparison of the decision

elements and assigning them relative scores. The relative scores provided by ten

experts are aggregated using the arithmetic mean method. Each decision maker in the

fund company makes a pair-wise comparison of the report evaluation under nineteen

subjective sub-criteria and, then, assigns those relative scores. We using the Eq. (1)

and (2) to calculated the aggregate pair-wise comparison matrix.

The results of the pair-wise comparison matrices about main-criteria and sub-

criteria are shown as table 2 and 3.

Step 3: Calculating the eigenvalue and eigenvector

The comparison in Tables 2 to 3 are used to calculate the eigenvectors using Eq.

(3) and (4). Table 4 summarizes the results of eigenvectors and weights for the main-

criteria and sub-criteria.

12

Step 4: Consistency test

According to Eq. (5) and (6), the consistency test of each criteria level is

calculated and the results are shown as table 2 and 3. The CR. of each comparison

matrices are all < 0.1, indicating “consistency”.

Step 5: Computing relative weight of each levels’ elements

Aggregate the related scores provided by all experts using simple additive

weighting and the results for each levels relative weight of the elements are shown as

Table 4. After sorting the four main-criteria by relative weights, the most important

main-criteria is objective(0.421), next are conscientious(0.237), assumption of

research(0.227) and timely research(0.116) separately.

The sub-criteria are sorted and analyzed based on relative weights under each

main-criteria as shown in Table 4. The results are summarized as follows:

(1) There are six sub-criteria under the most important main-criteria – objective. The

highest relative weight sub-criterion is no bias statement (0.225). We observe

that the relative weights of no serious omission (0.190), detail description of

information source (0.183) and true record (0.172) are also important criteria for

ANALYSTS to screen brokerage firm’s research reports. Applying authoritative

information (0.091) seems to be less important criterion under objective.

(2) With regard to the sub-criteria under conscientious, the most important criterion

is industry experience (0.296). The remainder criteria sorted by relative weights

are evaluating financial accounting data correctly (0.208), careful examination of

every numeral and statement published (0.169), Honest stock recommendation

(0.128), using financial model correctly (0.124) and work experience (0.075).

(3) According to the priority of relative weight, the sub-criteria under assumptions of

13

research reports ranked are consistent evaluation of target firm’s industry

(0.351), coherent evaluation target firm’s industry (0.282), value investing

(0.217) and financial model (0.149).

(4) For ANALYSTS, timely research reports mean address relevant opinions about

future market trend (0.470) and address relevant opinions before market shock

(0.416), addressing relevant opinions about popular issues in today’s market

(0.170) is not an important consideration for ANALYSTS.

Step 6. Computing global priority of each sub-criterion

Global priority of each sub-criterion is gathered by multiplying its relative-weight

by corresponding main-criterion’s relative-weight. The results are arranged in Table

5. The top five relative-weight sub-criterion are no bias statement (0.095), no serious

omission (0.080), consistent evaluation of target firm’s industry (0.080), detail

description of information source (0.077) and true record (0.072) respectively. The

bottom five relative-weight sub-criterion are address relevant opinions about popular

issues in today’s market (0.013), work experience of brokerage firm’s research team

(0.018), using financial model correctly (0.029), honest stock recommendation

(0.030) and financial model (0.034) respectively.

4. Discussion

As shown in Table 5, we discover that four of the top-five global priorities of sub-

criteria are of objective. This result is responding to the duty of ANALYSTS,

generating objective and valuable investment suggestions for mutual fund managers.

Brokerage firm’s analysts usually publish over-optimistic statements to lead investors

based on their own career concern or under top-management pressures. Investors like

ANALYSTS may suffer serious loss for adopting an over-optimistic opinion.

Therefore, they have to examine if research reports existing bias statement, serious

14

omission or false record and describing information source particularly before

accepting the investment recommendations and target price to make investment

decisions. One of top-five sub-criteria is of assumption of research reports. This sub-

criterion is consistent evaluation of target firm’s industry. Consistent evaluation

method makes valid and meaningful comparison with similar stocks for ANALYSTS.

The sixth global priority of sub-criterion is the industries’ domain knowledge or

experience of the researches’ target in the firm’s research team. But the importance of

work experience of brokerage firm’s research team is very low, the priority is 18. In

the viewpoint of ANALYSTS, the contribution to conscientious of industry

experience is greater than work experience. Consistent with Mikhail, Walther and

Willis (2003), they prove that analysts become more accurate with firm-specific

forecasting experience. Brokerage firm’s research team concentrates their attentions

in single industry will produce more valuable research reports.

Addressing relevant opinions about popular issues in today’s market is not

important considerations for ANALYSTS since stock market price has reflects those

popular issues; ANALYSTS is unable to acquire returns through such information.

The most important task of ANALYSTS is to find out under-valued stocks and invest

them now. Under-valued stocks mean their market price is lower than their real value

now and will go up in the future. Therefore ANALYSTS need an objective research

reports which can indicate what will happen in the future and how is the market trend.

Except future market trend, addressing opinions about possible market shock could

assist ANALYSTS to avoid loss. As for whether applying authoritative information or

not is not an important consideration for ANALYSTS.

5. Conclusion

Brokerage firm’s research reports provide investment information for institution

and individual investors to make investment decisions. Prior researches focus on

15

analyzing brokerage firm’s analyst or the relation between research reports and stock

market price, and so on. This study is aimed to provide a different approach to

evaluate brokerage firm’s research reports. A model with 4 main-criteria and 19 sub-

criteria is developed to assist investors to sieve out the most valuable brokerage firm’s

research reports by modified Delphi method and AHP. Investors, new employee of

securities investment trust company and brokerage firm’s research department can

benefit by our model.

Investors can judge which report is more valuable and worthy to refer through the

criteria in our model. In the case of new employee of securities investment trust

company, our model can help them to familiarize with research practices quickly. As

for brokerage firms research department, users of their research reports are existing

and potential clients. If those clients do not trust brokerage firm’s research reports,

then they will leave and brokerage firms will lose revenue. Our model is an important

impetus for brokerage firms when producing research reports.

We find the primary evaluation criterion of brokerage firm’s research reports is

objective. Brokerage firm’s analysts should make sure that there are no biases and

serious omissions in research reports published and declare information source to

ensure the objectivity. In the meanwhile, they must evaluate target firm’s industry

consistently. ANALYSTS do not care about the work experience of brokerage firm’s

research team when they evaluate the usefulness of research reports. They do care

about how brokerage firm’s analysts are familiar with the industries and the accuracy

of analysis process since these would affect the conscientious of brokerage firm’s

research reports.

Reference

Ahsan, M. K. and Bartlema, J. (2004), Monitoring healthcare performance by analytic

16

hierarchy process: a developing-country perspective, International Transactions

In Operational Research, 11(4), 465-478.

Azzi, Sarah and Bird, Ron (2005), Prophets during gloom and doom downunder,

Global Finance Journal, 15(3), 337–67.

Azzi, Sarah; Bird, Ron; Ghiringhelli, Paolo; Rossi, Emanuele (2006), Biases and

information in analysts' recommendations: The European experience, Journal of

Asset Management, 6(5), 345-380.

Barber, Brad; Lehavy, Reuven; McNichols, Maureen and Trueman, Brett (2001), Can

investors profit from the prophets? Security analyst recommendations and stock

returns, Journal of Finance, 56(2), 531-563.

Barber, B.M.; Lehavy, R.; McNichols, M. and Trueman, Brett (2003), Reassessing the

returns to analysts’ stock recommendations, Financial Analysts Journal, 59(2),

88–96.

Bjerring, J.H.; Lakonishok, J. and Vermaelan, T. (1983), Stock prices and financial

analysts’ recommendations, Journal of Finance, 38(1), 187–204.

Bradshaw, Mark t. (2002), The use of target prices to justify sell-side analysts’ stock

recommendations, Accounting Horizons, 16(1), 27-41.

Brav, Alon and Lehavy, Reuven (2003), An empirical analysis of analysts’ target

prices: Short-term informativeness and long-term dynamics, Journal of Finance,

58(5), 1933-1967.

Chang, Che-Wei; Wu, Cheng-Ru; Lin, Chin-Tsai and Chen, Huang-Chu (2007), An

application of AHP and sensitivity analysis for selecting the best slicing machine,

Computer & Industrial Engineering, 52, 296-307.

Chang, Che-Wei; Wu, Cheng-Ru; Lin and Chen, Huang-Chu (2008), Using expert

technology to select unstable slicing machine to control wafer slicing quality via

fuzzy AHP, Expert Systems with Applications, 34, 2210-2220.

17

Cowen, A.; Groysberg, B. and Healy, P.M. (2006), Which types of analyst firms are

more optimistic?, Journal of Accounting and Economics, 41(1/2), 146–199.

Elton, J. Edwin, Martin J. Gruber, and Seth Grossman (1986), Discrete expectational

data and portfolio performance, Journal of Finance, 41(3), 699-714.

Hong, Harrison and Kubik, Jeffrey D. (2003), Analyzing the analysts: Career

concerns and Biased Earnings forecasts, Journal of Finance, 58(1), 313-351.

Hsu, Pi-Fang; Wu, Cheng-Ru and Li, Zhao-Rong (2008), Optimizing resource-based

allocation for senior citizen housing to ensure a competitive advantage using the

analytic hierarchy process, Building and Environment, 43, 90-97.

Jacob, John; Rock, Steve and Weber, David P. (2008), Do non-investment bank

analysts make better earnings forecasts?, Journal of Accounting, Auditing &

Finance, 23(1), 23-61.

Jegadeesh, Narasimhan; Kim, Joonghyuk; Krische, Susan D. and Lee, Charles M. C.

(2004), Analyzing the analysts: When do recommendations add value?, Journal of

Finance, 59(3), 1083-1124.

MacCarthy, B. and Atthirawong, W. (2003), Factors affecting location decisions in

international operations: a Delphi study, International Journal of Operations and

production Management, 23(7), 794-818.

Michaely, Roni and Womack, Kent L. (1999), Conflict of interest and the credibility

of underwriter analyst recommendations, The Review of Financial Studies, 12(4),

653-686.

Mikhail, Michael B., Walther, Beverly R. and Willis, Richard H. (2003), The effect of

experience on security analyst underreaction, The Journal of Accounting and

Economics, 35, 101–116

Ryan, Paul and Taffler, Richard (2006), Do brokerage houses add value? The market

impact of UK sell-side analyst recommendation changes, The British Accounting

18

Review, 38, 371–386.

Saaty, T. (1999), Decision making for leaders: The analytic hierarchy process for

decisions in a complex world, RWS Publications, Pittsburgh.

Stickel, S.E. (1995), The anatomy of the performance of buy and sell

recommendations, Financial Analysts Journal, 51(5), 25–39.

Womack, Kent L. (1996), Do brokerage analysts’ recommendations have investment

value?, Journal of Finance, 51(1), 137-167.

19

Table 2 The pair-wise comparison matrix of the main-criteria

Goal C1 C2 C3 C4

C1 1.000 1.210 1.134 0.794

C2 0.826 1.000 0.266 0.306

C3 0.882 3.759 1.000 0.299

C4 1.259 3.268 3.344 1.000

4.271; CI = 0.09; RI = 0.90; CR = 0.1≦0.1

Table 3 The pair-wise comparison matrices of sub-criteria

C1 SC1 SC2 SC3 SC4

SC1 1.000 1.375 0.564 0.893

SC2 0.727 1.000 0.461 0.461

SC3 1.773 2.169 1.000 1.238

SC4 1.120 2.169 0.808 1.000

4.027; CI = 0.009; RI = 0.90; CR = 0.01≦0.1

C2 SC5 SC6 SC7

SC5 1.000 0.188 0.357

SC6 5.319 1.000 0.871

SC7 2.801 1.148 1.000

3.0696; CI = 0.0348; RI = 0.58; CR = 0.06≦0.1

C3 SC8 SC9 SC10 SC11 SC12 SC13

SC8 1.000 2.431 0.214 2.271 0.459 0.468

SC9 0.411 1.000 0.189 0.701 0.668 0.437

SC10 4.673 5.291 1.000 1.876 0.837 1.334

SC11 0.440 1.427 0.533 1.000 0.702 1.275

SC12 2.179 1.497 1.195 1.425 1.000 1.292

SC13 2.137 2.288 0.750 0.784 0.774 1.000

6.496; CI = 0.0992; RI = 1.24; CR = 0.08≦0.1

C4 SC14 SC15 SC16 SC17 SC18 SC19

SC14 1.000 0.223 0.439 0.776 0.777 0.433

SC15 4.484 1.000 0.492 0.454 0.454 0.475

SC16 2.278 2.033 1.000 0.981 0.940 0.717

SC17 1.289 2.203 1.019 1.000 0.605 1.063

20

SC18 1.287 2.203 1.064 1.653 1.000 1.772

SC19 2.309 2.105 1.395 0.941 0.564 1.000

6.496; CI = 0.0992; RI = 1.24; CR = 0.08≦0.1

Table 4 The eigenvectors and weights for the main-criteria and sub-criteria

Main-criteria Relative-weights Sub-criteria Relative-weights

C1 0.227

SC1

SC2

SC3

SC4

0.217

0.149

0.351

0.282

C2 0.116

SC5

SC6

SC7

0.114

0.470

0.416

C3 0.237

SC8

SC9

SC10

SC11

SC12

SC13

0.128

0.075

0.296

0.124

0.208

0.169

C4 0.421

SC14

SC15

SC16

SC17

SC18

SC19

0.091

0.139

0.183

0.172

0.225

0.190

21

Table 5 Global priority of sub-criteria

Main-criteria Sub-criteria Relative-weights Global priority

C1

SC1 0.049 10

SC2 0.034 15

SC3 0.080 2

SC4 0.064 7

C2

SC5 0.013 19

SC6 0.055 9

SC7 0.048 12

C3

SC8 0.030 16

SC9 0.018 18

SC10 0.070 6

SC11 0.029 17

SC12 0.049 10

SC13 0.040 13

C4

SC14 0.038 14

SC15 0.059 8

SC16 0.077 4

SC17 0.072 5

SC18 0.095 1

SC19 0.080 2

22