Farm sustainability assessment:

some procedural issues

M. Andreoli1, R. Rossi2 and V. Tellarini3

1 University of Pisa,Dip. Economia aziendale,

Via C. Ridolfi 10, 56124 Pisa, ItalyE-mail: mandreol@ec.unipi.it

2 Tuscany Region,Dip. delle Politiche Territoriali e Ambientali,

Area Tutela e valorizzazione delle risorse ambientaliVia di Novoli 26, 50127 Florence, ItalyE-mail:ro.rossi@mail.regione.toscana.it

3 University of Pisa,Dip. Economia dell'Agricoltura dell'Ambiente Agro-forestale e del Territorio,

Via del Borghetto 80, 56124, Pisa, ItalyE-mail: vittotel@vet.unipi.it

Abstract

This article discusses some procedural issues relating to a multicriterial assessment of farm sustainability,

based on the criteria proposed by the European Union Concerted Action on ‘The Landscape and Nature

Production Capacity of Sustainable/Organic Types of Agriculture’. Two main problems are stressed: 1)

the treatment of basic information used for evaluating farm performances as regards the criteria and 2) the

difficulties in evaluating a case study farm. Firstly, the problem of implementing multicriterial analyses

when using qualitative ordinal data and discrete quantitative data is faced, stressing the importance of

1

2

3

4

5

6

789

10

1112131415

16171819

20

21

22

23

24

25

26

clearly defining and applying procedures that can be transferred and repeated. This is due to the fact that

almost all research contributions describe in detail multicriterial methods and results, but give little space

to the problem of collecting and analysing basic information. Nevertheless, final results heavily depend on

the way basic information has been gathered and processed in order to obtain the indices that have been

used for the assessment. The lack of standards and of procedure description does not make it possible to

compare results of assessments and to judge their suitability to the aim of farm sustainability assessment.

Secondly, the problem of finding external points of reference for judging a case-study farm is confronted.

Case studies can be important as ‘models’ for other farms. Indeed, it is easier to persuade farmers to adopt

farming styles and decisions that somebody else has already successfully implemented rather than to adopt

unexplored ways of managing their farms. This asks for reliable methods to assess a single farm, but

almost all multicriterial methods only provide a tool for ranking a set of objects, e.g., farms, from the best

to the worst. Conclusions provide some comments on the usefulness of these approaches.

Keywords: sustainable farming, multicriterial analysis, Tuscany, landscape production, Italy, case-study

assessment

27

28

29

30

31

32

33

34

35

36

37

38

39

40

1. Introduction

The use of multidimensional approaches, e.g. based on multicriterial analysisl has been a major

improvement in respect to reductionistic approaches typical of a culture too much based on specialisation

(Tellarini et al., paper presented at the EU Concerted Action on Landscape and Nature production

Capacity meeting held in Wageningen, 1996). Studying phenomena from a holistic point of view means

taking into account all their relevant facets. Although a holistic approach consents to achieve a full

understanding of phenomena, it asks for tools capable to cope with multidimensional problems. From an

applied point of view the importance of a multidimensional approach in setting up interventions for

agriculture is apparent when considering that policies aiming to steer agricultural production or to

subsidise farms do not only affect economic and productive results but affect also, e.g., the quality of

environment and landscape. The effects of farming on environmental pollution and landscape quality have

been studied in Italy, e.g., by Pennacchi et al., 1994 and 1998, Accademia Agricoltura, 1991, Chiusoli,

1994. Policies having only one aim, such as supporting farmers’ income as the ‘old’ Common

Agricultural Policy (CAP), have often resulted not only in reaching, and sometimes only partially, the

intended goal, but they have caused other unforeseen ‘side-effects’. According to Croci-Angelini (1995),

CAP has resulted in deepening regional disparities, while Baldock and Beaufoy (1993) concluded that

rationalised intensive agriculture has been associated with damage and destruction of the environment,

natural and seminatural habitats and (visual) landscapes. The negative effects that can result from farming

have increased the need for sustainable farming practices. A review of the meaning and evolution of

sustainability in agriculture has been recently provided by Polinori (1998).

A checklist for ‘Sustainable Landscape Management’ has been produced as the final report of the EU

Concerted Action on ‘The Landscape and Nature Production Capacity of Sustainable/Organic Types of

Agriculture’ (van Mansvelt and van der Lubbe, 1999). This checklist provides an inventory of indices that

might be relevant when analysing farming activity impacts. These criteria, ‘using a unifying concept

derived from Maslow’s study on human motivation translated to the landscape and perceived as a

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

reflection of the priorities and motivations leading the actions of people’ (van Mansvelt and van der

Lubbe, 1999), have been organised in six main fields: a) Environment, b) Ecology, c) Economy, d)

Sociology, e) Psychology, and f) Physiognomy/Cultural Geography. Due to the variety of relevant fields,

sustainable farming has to be analysed using a multidimensional approach. This, however, implies the

need to cope with criteria expressed in different units of measurement and with data that are not

homogeneous as regard to the level of precision. This asks first of all for a very careful treatment of the

data used for building the indices on which to base the final assessment of farm performance, and

secondly for a rational choice of the methodology to be used for reaching an ‘overall judgement’ (Colorni

and Laniado, 1988, 1992). In this context, ‘overall judgement’ indicates a summary of all the performance

that the object of the analysis has shown for all the relevant criteria.

This article attempts to systemise a series of considerations relating to the above problems, which where

stimulated by some of the contributions of the members of the EU Concerted Action on Landscape and

Nature Production.

2. The importance of ‘a priori’ clarification of rules and procedures

According to Tellarini (1995), in social science empirical research it is possible to distinguish two

different phases: the first, called ‘private phase’, which concerns research organisation, data gathering,

data verification and data processing; and the second, called ‘public phase’, which involves summarising

and commenting results. The first phase is defined as private, because it is very seldom fully described by

the researcher, since this would take too much space, especially in the case of a multidisciplinary and

multicriterial approach. Thus, when presenting multicriterial analyses, quite often only the list of criteria

that have been used is provided, without giving any explanation on the way the basic data have been

gathered and transformed into indices (e.g., environmental impact criteria in Ciani et al., 1993). According

to Colorni and Laniado (1992), the Environmental Impact Assessments performed during the ‘80s “were,

in fact, more ‘surveys’ than assessments. Moreover, such ‘surveys’ were performed according to different

points of view, with no reference to a common standard: this makes comparison of different studies

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

difficult and, even, worse, means that it is often impossible for a public authority to really check the

adequacy of the impact study”. In the same way, the lack of a common standard and of the information

needed for fully understanding how criteria have been built does not allow a rational use of many of the

studies on the impact of farmers’ choices, especially on non-economic parameters. Consequently,

although the importance of ‘a priori’ clarification of rules relating to a scientific method may seem an

obvious concern, nevertheless, in our opinion, it is important to underline that:

A) The use of qualitative data requires greater attention in the description of hypotheses adopted and

of the procedure used for building criteria, since qualitative data are more difficult to be

interpreted objectively than quantitative data. In other words, in our opinion, it is easier to

evaluate the difference between a 1.000 and a 2.000 Euro monthly income than to judge how great

is the difference between a good or a normal level of ‘offer of sensory qualities, such as colours,

smells and sounds’;

B) Although it is very seldom possible to fully describe in an article the procedures leading to the

building of criteria used for an assessment, nevertheless it is necessary that before starting an

analysis researchers fully state the procedures for gathering and processing basic information.

These procedures should accommodate for the specific requirements of qualitative and

quantitative data processing. If during the analysis one or several procedures would demonstrate

not to be suitable, it is necessary to go back and start over again. Following a stated procedure

ensures consistency in data gathering and processing.

2.1. The problem of processing qualitative data

When facing a multicriterial analysis, researchers very often have to cope with qualitative variables. Many

of the parameters proposed by the EU Concerted Action members for evaluating farm performance (van

Mansvelt and van der Lubbe, 1999), such as landscape completeness or wholeness, are qualitative.

Moreover, in many cases the cost of quantitative information is so high that, although it might be possible

to measure a phenomenon exactly, it is preferable to use a ‘discrete scale’ (e.g., income classes) rather

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

than a continuous scale, or even to use qualitative data, provided that they can be ordered (Andreoli and

Tellarini, 1999). In the latter case, researchers have to translate qualitative ordinal information into

numerical codes due to the requirements of software for multicriterial analysis. However, researchers

should remember that only methods capable to cope with qualitative ordinal data, e.g., concordance

absolute index, would give correct results. For building concordance indices it is necessary to compare

every possible pair of objects for each criterion and to check if the first has a better, worse or equal

performance than the other ones (Colorni and Laniado, 1992). Consequently, this method can not be used

when the analysis is performed on one case study. The problem of dealing with only a single case-study

farm will be discussed later.

Let us take the case of erosion in the analysis of two case study farms by Rossi et al. (1997, 1999). The

erosion analysis was performed by using a five-step scale, since the quality of information was judged

insufficient for a finer scale, where each step was represented by a symbol that was associated to a real

situation. The observed situations and associated symbols were the following:

• Clear absence of erosion ++

• Absence of erosion with some uncertainty +

• Minimal erosion (without consequences) +/-

• Moderate erosion -

• Severe erosion --

When transforming qualitative ordinal data into numerical codes and processing them with multicriterial

methods, researchers should make sure that: a) numerical codes are attributed in a rational way, ranking

qualitative data, e.g., from the best to the worst and attributing to them decreasing, or increasing,

numerical codes, and b) the method used for performing multicriterial analysis is suitable for processing

qualitative ordinal information, as in the case of concordance absolute index method.

In the above described erosion case (Rossi et al., 1997, 1999), provided that data are considered

qualitative ordinal, the translation into numerical codes of the symbols can be done, e.g., as follows:

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

Symbol ++ + +/- - --

Value 1.00 0.75 0.50 0.25 0.00

In this case values have been obtained by giving score ‘1’ to the best situation and score ‘0’ to the worst

one and finding the three intermediate values in such a way that the scale has a ‘constant stepping’. This

method is very similar to normalisation procedures, which will be described later. However, when

transforming qualitative ordinal data, it is only important that numerical codes can allow ranking

situations from the best to the worst, independently from how much a situation differs from the next one.

Thus, any scale with decreasing or increasing values can be accepted, independently from the ‘stepping’.

2.2. Using continuous or discrete quantitative data

If in the case of erosion the above symbols represent a quantitative phenomenon expressed as a discrete

scale, the proposed conversion would not any longer be correct, in so far as the situation of clear absence

of erosion with some uncertainty is much closer to that of clear absence of erosion than to that of minimal

erosion (Andreoli et al., 1998). Again, this difference is smaller than that between moderate erosion and

severe erosion. In other words, the proposed numerical conversion is correct only if erosion data are

processed as qualitative ordinal data. If the initial information is processed as quantitative data, the scale

between clear absence of erosion and severe erosion must be divided in a way that more correctly reflects

the differences in the impact of the erosion levels (Andreoli and Tellarini, 1999). Fig. 1 provides a

graphical representation of a possible numerical conversion of the above symbols in the case of qualitative

ordinal information (graphic on the left-hand side) and quantitative information (graphic on the right hand

side).

Fig. 1 - Conversion of symbols relating to real situations into numerical codes, in the case of qualitative

ordinal and quantitative discrete data.

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

2.3. From indices expressed in physical units to indices expressed in terms of Utility

Performing a multicriterial analysis based on continuous quantitative data implies confronting the problem

that criteria are expressed in different units of measurement. Measurement units are not relevant if data are

qualitative ordinal because they are used only to compare, for each criterion, if one object of the analysis

has a better, worse or equal performance than another one. On the contrary, in the case of quantitative data

it should be taken into account how much a value differs from another one. If all values are transformed

into a common unit of measurement, by means of normalisation or other procedures, it is possible to reach

an ‘overall judgement’ for every object of analysis by summing up all the values it has scored for the

relevant criteria.

One of the most common ways for normalising the values of a criterion consists (Colorni and Laniado,

1988):

a) in giving score 0 to the lowest value observed in the analysis for that criterion;

b) in giving score 100 to the highest observed in the analysis for that criterion;

c) in calculating all the intermediate values by means of a linear transformation.

This kind of normalisation has the advantages of always obtaining, for each criterion, positive values

ranging from 0 to 100, but it is subject to two main critics. First of all, the normalised value given to an

object is strictly depending on which other objects are considered in the analysis; in other words,

normalised values for a group of objects of analysis could change if a new object is added or if one of the

previous is eliminated from the analysis (Colorni and Laniado, 1988). Secondly, as seen in the above

described example of erosion, very seldom a linear and automatic transformation of values consents to

adequately represent differences existing between ‘real situations’.

Conversion of data expressed in physical units into a common measurement unit can also be done by

transforming criteria into goals or ‘objectives’ (Colorni and Laniado, 1992). This means expressing

criteria in terms of ‘satisfaction’ or ‘utility’ resulting from the physical value of the criterion itself, e.g.

evaluating the satisfaction resulting from one, or several, levels of farm incomes or from varying levels of

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

pollutant concentration rather than measuring them in thousands of Euro or in p.p.m. Thus, rather than

transforming criteria in monetary terms, as in the case of Cost-Benefits-Analysis (Dasguta and Pearce,

1975), the common unit of measure chosen is ‘Utility’. The concept of Utility is often used in economic

analysis, e.g., for describing consumers’ behaviour. Indeed, while entrepreneurs are supposed to aim to

profit maximisation, consumers are supposed to aim at maximising the utility resulting from the

consumption of products and services (Samuelson and Nordhaus, 1993). When parameters are expressed

in terms of Utility, high values have always a ‘positive meaning’ and low values have a ‘negative

meaning’, while this is not true if working with physical units. From this point of view, working with

Utility values is easier because it is not necessary to remember how an index is defined (or calculated) for

knowing if a high value is desirable, or not.

When it is possible to set a target (e.g., an optimal share of fodder crops or a satisfactory level of income)

for every parameter, the transformation of conventional data into Utility can be done by giving score ‘1’

when the target is achieved and score ‘0’ when it is not. Since this method provides a too rough

measurement scale - only two values are allowed - it is usually necessary to find an alternative procedure.

When quantitative continuous physical data are available, it is possible to have a Utility function that is

continuous, rather than dichotomous. Given that the relationship between physical and utility values is

very seldom linear, it is necessary to define it case by case. Between the concentration of a pollutant in

p.p.m. (parameter in physical terms) and the Utility associated with it; e.g., there is an inverse relationship

so that as pollution increases Utility decreases. This relationship is not linear, since it is assumed that the

level of pollution has no negative effects on the environment, as long as it is very limited. As the pollutant

concentration increases, the quality of the environment worsens, at first quite slowly and then ever more

rapidly. In other cases, e.g., when the density of a natural population is involved, there is no consistently

positive or negative relationship between the physical parameter (e.g., expressed as number of

animal/hectares) and the Utility value. When the density is low its increase determines an increase in

Utility, in that the species is reaching optimum density levels; then there is a range of optimum density

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

within which the Utility function maintains its maximum level, but beyond which the satisfaction level

decreases again (Andreoli and Tellarini, 1999).

The use of Utility function could be criticised in so far as there could be subjectivity in building them. As

Bosshard (1997) states <<experiences in landscape planning, especially in the last few years, confirm

epistemological consideration, viz. that a model for evaluating cannot be ‘objective’ - in the sense of being

generally valid. Rather, every validation is individually dependent on at least the following three premises:

1. temporary, culturally dependent ideas of values;

2. the prevailing physical situation;

3. the personal standpoint of the participants, including that of the experts, with respect to the

presentation of the problem.>>

This statement does not only apply to the problem of building Utility functions for parameters, but above

all affects the problem of deciding the relative importance (weight) to be given to each criterion in

comparison with the other ones. Subjectivity in transforming physical values in satisfaction values - as the

importance given to each criterion - could be limited by applying a procedure capable to accommodate for

these causes of variability. In other words, in our opinion, a ‘satisfactory’ level of objectivity and

comparability of results might be reached, if, e.g., rational procedures and benchmarks for transforming

physical values in utility values are defined. In the same way, although weighting is a subjective process,

it is possible to limit its subjectivity by giving guidelines and a rational procedure for attributing weights.

A method for attributing weights taking into account the features of impacts (temporary/permanent,

local/national, short/long term) and impacted resources (renewable/not renewable, common/rare,

strategic/not strategic) is proposed in Schmidt di Friedberg (1987). Finally, only the exact knowledge of

the hypotheses on which data conversion in Utility values and weighting of criteria have been performed

can allow readers to judge on the reliability of an analysis. Indeed, the quality of results of an assessment

does not depend only on the methodology used for the evaluation, but it heavily depends also on the way

data used for the assessment have been obtained.

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

3. Analysing a single case-study

Analysing a single case study is in some way more difficult than analysing a set of objects, in so far as it is

not possible to perform comparisons between objects. Thus, analysing a single case study does not allow

using qualitative ordinal data because there is not any suitable object to compare data with. Moreover, this

means that it is not possible to normalise values, due to the lack of internal reference points. Indeed,

having only one value for each criterion (the one of the case study), the concepts of minimum, maximum

and average do no longer have any meaning. Consequently, when analysing only one case study, the

transformation of criteria into a common unit of measurement has to be done by means of Utility

functions. This because Utility functions are (or could be) based on external reference points. Due to the

fact that Utility data have to be used as quantitative ones, the conversion from physical to Utility units has

to be done very carefully. Thus, in our opinion, the conversion should start by defining a procedure that:

sets external points of reference for the minimum and maximum values of the scale, namely the

physical situations that correspond to value ‘0’ and value ‘1’ of the Utility function. This process is

similar to the one of calibrating a thermometer scale, where value 0 is given to the situation of melting

ice and value 100 is given to the situation of boiling water. Varying benchmarks should/could be used

for every region. Indeed, according to Hendriks et al. (in press), external reference values may or must

differ for different landscape types/regions; since an external point of reference can not be global, but it

must be filled in regionally (see also Rossi et al., 1997). A Utopic region is needed as guiding image

for farm development;

does not apply automatic conversions implying a linear transformation of data, but it tries to define

values that are representative of differences in satisfaction relating to real situations. From this point of

view, if it is not possible to reconstruct the whole Utility function, it is sufficient to be capable to find

the Utility level to be attributed to the case study.

It is important to note that what stated, as regards conversion procedures is not only valid for the analysis

of a single case-study farm, since the same principles can be adopted when a set of objects are analysed. In

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

fact, while when assessing one farm it is only necessary to place a single value in the range defined by the

0-1 external points of reference, in the case of a set of objects there is a number of values to be

transformed that corresponds to the number of objects. In both cases, in our opinion, it is important to

discuss the way external reference points could be chosen.

Individuating the values against which to calibrate the scale means deciding which situations to use as

references for the maximum and minimum points on the scale. An ‘objective procedure’ for individuating

reference points could be the one of taking the best situation achievable in the long term for each

parameter as a reference for the maximum Utility. In this case the term of comparison for judging a case

study would be a ‘Utopic farm’. The Utopic condition is not so much tied to the achievement of a

predetermined maximum target for a single parameter (which might actually be possible for real farms), as

to the possibility of reaching the maximum value of all indicators contemporarily. Indeed the concept of

Utopic Farm is similar to the one of ‘Ideal Point’ often used in the case of multiple criteria analysis (e.g.,

Romero and Rehman, 1989), which is characterised by the contemporary achievement of all the optimum

values (individually achievable) for conflicting objectives. Using Utopic values as reference points allows

for differences due to the specific region under examination in so far as it is possible to refer to a situation

that expresses the absolute maximum possible of that parameter, independently of the area where the case-

study is located (absolute or general Utopia), or to refer to a relative maximum, expressing the maximum

level actually possible in that particular region (relative or local Utopia). The choice of a local (or relative)

Utopia or a general (or absolute) Utopia conditions the reading of the results, as well as the possibilities of

comparison when evaluations of different situations are required. So whereas evaluations expressed

against the standard of a general Utopia are directly comparable, since they use the same scale, those

expressed according to the standard of a local Utopia indicate the position of the farm with respect to the

maximum result obtainable in the reference region, so that the scale is calibrated with a maximum value

that varies according to context.

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

285

286

Since it is the whole performance, and not that one regarding a single parameter, to indicate how much

the case-study farm differs from a Utopic farm, it is important to describe how this ‘overall judgement’ on

farm performance can be reached. The easiest way of doing it is to sum up all the Utility values scored by

each farm, after multiplying them for their weights. In this case each weight represents the relative

importance given to a criterion in comparison with the other ones. It should be noted that some researchers

are against weighting criteria because weights are the result of <<a subjective, uncertain and conflictual

operation>> (Colorni and Laniado, 1992) and consequently they might be unreliable. However, not using

weights when summing up the performance scored for criteria means giving to all of them the same

weight, i.e., weight 1; this is again a subjective decision and probably less correct than explicitly giving

weights. In this context, in our opinion, it is more suitable to try to control subjectivity, e.g., by giving

guidelines for weight attribution (e.g., as in Schmidt di Friedberg, 1987) or by checking how much the

results of the analysis are depending on the chosen set of weights, than avoiding using them. In other

words, if subjectivity is unavoidable, it is at least possible to try to control it and to explicitly state the

hypothesis that can be considered as subjective in order to make the analysis as ‘transparent’ as possible

(Colorni and Laniado, 1992). Since weights are strictly depending on the socio-economic and

environmental context where the analysis is placed, it is not possible to find a weighting system that could

be generally valid in every situation. It is apparent, e.g., that developing countries where people still suffer

for starvation are more concerned in productive problems of agriculture than in those of landscape

preservation. On the contrary, in ‘rich’ countries, environment and landscape are given an increasing

interest, in comparison with the problem of agricultural production, which nowadays is often higher than

needed. Thus, if a situation implies a level for a criterion which is below the minimum required, nobody

would be ready to compensate a decrease in this criterion with an increase in another one, which is less

important or which currently has a satisfactory level. Once that physical survival requirements, or needs

considered strictly necessary, have been met, it is possible to ‘trade’ between criteria, exchanging the

‘surplus’ of a criterion for an increase in another one. Thus, the trade-off between objectives (represented

by criteria) heavily depends on their initial values. Indeed, according to a marginalistic approach

287

288

289

290

291

292

293

294

295

296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

(Samuelson and Nordhaus, 1993) usually the importance of an improvement in a criterion is increasingly

lower when passing from a mere matching of requirements to increasing levels of surpluses. The above

statement is, in our opinion, perfectly coherent with the Maslow’s approach to human motivation used as

unifying concept in the EU Concerted Action on The Landscape and Nature Production Capacity of

Organic/Sustainable Types of Agriculture (van Mansvelt, 1997, van Mansvelt and van der Lubbe, 1999).

The use of weighted sum as method for assessing the overall farm performance and of score 1 as the

maximum Utility value results in giving to the Utopic farm an overall judgement of 1. This because

weights are recalculated in such a way that their sum is always 1. Thus the performance scored by a case-

study farm should be read taking into account that the maximum possible level of the overall judgement

(i.e., the one of the Utopic farm) is 1. In other words, if a real case-study farm would have an overall

judgement of 0.78, this would mean that its performance is 78% (0.78/1) of the maximum possible,

namely the overall judgement of the Utopic farm.

However, it should be remembered that, as shown above, exactly defining what Utopia is can be

problematic, especially as regards the choice of whether to take as reference the maximum values possible

for the various parameters (not always easy to establish) or those that can be considered maximum in the

examined context. Indeed, while the Utopic value for the erosion parameter might be objectively

generalised in ‘clear absence of erosion’, this is not the case for parameters such as farm income, where

Utopia might be characterised by extremely high values, completely incongruent with the context of the

farm under study. To set the external reference for 0 score could be still harder, since using a ‘too bad’

external reference point for score 0 might result in underestimating differences between the other

situations. Moreover, the distance between actual farm and Utopia depends on the units of measurement

adopted, or rather, on the weighting system used. In other words, using different vectors of weights, the

distance of a case-study farm from Utopia or ‘perfection’ may vary considerably.

Finally, it is important to remember that Utopia is, by definition, Pareto dominant on all the actual or

potential farm situations. <<A Pareto optimal solution is a feasible solution for which an increase in the

313

314

315

316

317

318

319

320

321

322

323

324

325

326

327

328

329

330

331

332

333

334

335

336

337

value of one criterion can only be achieved by degrading the value of at least one other criterion>>

(Romero and Rehman, 1989). Consequently, a situation is Pareto dominant when it is not worse for all

parameters and better for at least one. Since Utopia is characterised by scoring the maximum value for

each criterion, this means that real farms could match its performance but not perform better. Thus, Utopia

could not be used as a ‘second object of analysis’ for performing a multicriterial analysis based on

qualitative ordinal data.

Since farmers could consider the Utopic performance to be ‘out of reach’, researchers could consider

using a reference point that it is closer to the real case-study situation. From this point of view, another

way of calibrating the scale could be the one of using as reference points targets that could be achieved by

the case-study farm in the short or long run. In this way the judgement would consist in an assessment of

what the performance of the farm is in comparison with its potential performance in the long or short run.

In other words, with this kind of approach, it could be possible to judge how much efficient a farm is,

being the ‘inefficiency’ defined as distance between the case-study farm real situation and its potentiality.

Here too, it is essential to understand the type of reference to be used as external term of comparison, a

problem that, as in the previous case, brings us back to that of the calibration of the scale. The use of a

potential value rather than a Utopic one leads, however, to even greater problems of definition, depending

on which of the following courses is chosen:

To consider the case-study farm as a homogeneous part of the region in which it is located. In this case

the ‘local Utopia’ could be used as the term of external comparison, i.e. the best performance

theoretically obtainable in that context. This course is open to two main criticisms. Firstly, the

potentiality of the farm is not necessarily that of the surrounding territory. Indeed, with regard to

economic performance, e.g., if the size of the farm is atypical of the area, farm actual potentiality could

be quite different from that of the surrounding farms. Secondly, that reference is still made to a Utopic

rather than to a potential situation in that account is not taken of the fact that the various objectives are

conflicting. In other words, the maximum potential value obtainable for an individual parameter might

338

339

340

341

342

343

344

345

346

347

348

349

350

351

352

353

354

355

356

357

358

359

360

361

362

coincide with the Utopic one, in so far as Utopia refers to the contemporary achievement of the

maximum value for all parameters. Thus by trying to include in the ‘potentiality’ the concept of

conflicting objectives, it is much more difficult to individuate the set of maximum values for the

various objectives that may be contemporarily reached. When analysing a set of farms, a possible way

of calculating farm potential in a homogeneous context could be that of considering a selected case-

study as benchmark for the comparison, after checking that farms under study have the chance of

performing as well as the case-study farm. Although this ‘applied’ potentiality might solve the problem

of finding a set of reference values, nevertheless it might underestimate the ‘theoretical’ potentiality.

Despite the possible criticisms, the local Utopia approach is easy to apply and extend to other farms in

so far as it does not ask for repeating a double evaluation for them all, i.e. actual and potential

situations. However, the adjective ‘potential’ might be misleading since, as we have seen, it is more a

question of local Utopia (or of comparison with case-studies) rather than the specific potentials of the

farm under study.

To consider the real potentialities of the farm under examination, that need not necessarily coincide

with those of the surrounding territory for all parameters. The application of this type of approach

involves two rather difficult problems. First of all, it involves the need to carry out a double evaluation,

one of the actual situation and another of the potential situation of the farm. In other words, unlike for

Utopia with its common reference scale for the whole area, here the potentiality of the farm is

considered to be specific of the farm itself. Secondly, as in the previous case, the difficulty of defining

the potentialities of a farm with regard to a series of criteria relative to objectives that can not be

pursued contemporarily. So, unlike analyses in which only one parameter is evaluated, here there

might not be just one but many potential situations depending on the priority given to the achievement

of the various objectives. This results in great difficulty in the individuation of the potential situation to

be taken as referent. Moreover, unlike the previous case, it is not possible to use case studies as

external references in so far as farm features are not similar to the one of the context.

363

364

365

366

367

368

369

370

371

372

373

374

375

376

377

378

379

380

381

382

383

384

385

386

387

In conclusion, we believe that it is much more difficult to determine the margin of improvement of

overall farm efficiency by making use of targets potentially achievable by the actual farm than to

individuate the distance from a situation of local Utopia, even if the former method is formally more

correct. This due to the above mentioned fact that the main difference between a potential situation and

Utopia consists in not being capable to pursue and achieve contemporarily an excellent evaluation for

conflicting objectives.

4. Concluding remarks

Performing a farm sustainability assessment is not an easy task, especially if all the relevant effects of

farmers’ choices have to be taken into account. From this point of view, although the increasing interest of

researchers and the whole society are bringing about many studies on this topic, there is still a long way to

go. The checklist of criteria proposed by the EU Concerted Action on Landscape and Nature Production

constitutes a first step in this direction, providing an inventory of criteria that could be relevant for farm

sustainability assessment. Of course, since the checklist is supposed to be valid at European level,

researchers have to select every time which criteria to use and which ones are not suitable for an analysis

performed in a specific context. The second step that should be done is providing guidelines and standards

for using the criteria. This involves two different sets of problems. Firstly, the framework provided by the

EU Concerted Action members for sustainability assessment is quite complex. Thus, even if this approach

guarantees the reliability of results, nevertheless it asks for a very expensive and time consuming data

gathering. From this point of view it might be very interesting to have ‘shortcuts’, i.e., simplified

procedures for gathering information that guarantee a ‘satisfactory’, although not optimal, level of quality

of information while greatly reducing the effort needed for data collection. Secondly, it would be

important to dispose of surveys based on standard procedures, capable to provide researchers with the

reference points for calibrating criteria scales for a variety of contexts, characterised by specific socio-

economic, environmental, etc., features. This kind of research is not always very much appreciated, since

it asks for a lot of time and efforts and it only provides information for further research. In our opinion,

388

389

390

391

392

393

394

395

396

397

398

399

400

401

402

403

404

405

406

407

408

409

410

411

412

however, this kind of research is important since it allows to perform further analyses, whose results can

be compared. Moreover, since in the analysis of case-study farms researchers could be more easily biased

from their opinion on the farms they have selected, the use of standard procedures is to be strongly

advised. This article has confronted some of the issues relating to procedures that could be used for

implementing farms assessment using a multicriterial approach, and it has tried to stress the main

problems that can cause surveys and analyses not to be reliable or comparable. This with the aim of

promoting a discussion leading to the definition of standards that could be employed not only in

theoretical research, but also in applied research.

Acknowledgements

This research has been supported by National Research Council under contributions n. 94.00965.CT06

and n. 95.03251.CT06 and by the University of Pisa.

References

Accademia Nazionale di Agricoltura, 1991. Agricoltura e Ambiente, Edagricole, Bologna.

Andreoli, M., De Simone, A., Rossi, R. and Tellarini, V., 1998. Una proposta di percorso per la

valutazione di realtà aziendali o comprensoriali in base ad un set di criteri di tipo qualitativo, Il

Borghetto, Pisa.

Baldock, D. and Beaufoy, G., 1993. Nature conservation and new directions in the EC common

agriculture policy. Report for the Ministry of Agriculture, Nature Management and Fisheries, The

Netherlands. Arnhem and London: Institute for European Environmental Policy.

Bosshard, A., 1997. What does objectivity mean for analysis, valuation and implementation in agricultural

landscape planning? A practical and epistemological approach to the search for sustainability in

‘agri-culture’, Agric. Ecosyst. Environ., 63/2,3: 133-143.

413

414

415

416

417

418

419

420

421

422

423

424

425

426

427

428

429

430

431

432

433

434

Ciani, A., Boggia A. and Marinozzi, G., 1993. Metodologie di valutazione di alternative di parchi: il caso

del Parco del Nera, Genio Rurale, 11: 46-54.

Chiusoli, A., 1994. La rinaturalizzazione del paesaggio agrario: una esigenza ambientale, culturale e

civile, Genio Rurale, 4: 42-51.

Colorni, A. and Laniado, E., 1988. VISPA, Software Territoriale e Ambientale, CLUP, Milano.

Colorni, A. and Laniado, E., 1992. SILVIA: a decision support system for environmental impact

assessment, in Colombo, A.G. (Editor), Environmental Impact Assessment, ECSC, EEC, EAEC,

Brussels and Luxembourg, printed in the Netherlands, pp. 167-180.

Croci-Angelini, E., 1995. Effectiveness and redistribution of the regional policy in the European

Community, in Sotte, F. (Editor), The regional dimension in agricultural economics and policies,

proceedings of the 40th Seminar of the European Association of Agricultural Economists, Grafica

Tiburtina, Rome, pp. 251-274.

Dasguta, K. and Pearce, D.W., 1975. Analisi costi benefici, teoria e pratica, ISEDI, Milano.

Feliziani, R., 1997. Valutazione di alternative di gestione: il caso del Parco Nazionale dei Monti Sibillini,

Genio Rurale, 12: 25-32.

Hendriks, K., Stobbelaar D.J. and van Mansvelt J.D., 1999. The appearance of agriculture. Assessment of

landscape quality of (organic and conventional) horticultural farms in West-Friesland, Agric.

Ecosyst. Environ., in press.

Pennacchi, F., (Editor), 1994. La riforma Mac Sharry. Effetti nelle aziende R.I.C.A., CNR-RAISA n.

2224, Quaderni dell’Istituto di Economia e Politica Agraria, Tipografia dell’Università degli Studi

di Perugia, Perugia,.

Pennacchi, F., (Editor), 1998. Sostenibilità Efficienza e Successo Aziendale. Una valutazione nelle

aziende della R.I.C.A., CNR-RAISA, Quaderni dell’Istituto di Economia e Politica Agraria,

quaderno n. 24, Tipografia dell’Università degli Studi di Perugia, Perugia,

435

436

437

438

439

440

441

442

443

444

445

446

447

448

449

450

451

452

453

454

455

456

457

458

Polinori, P., 1998. Agricoltura e Sostenibilità, in Pennacchi, F., (Editor), Sostenibilità Efficienza e

Successo Aziendale. Una valutazione nelle aziende della R.I.C.A., CNR-RAISA, Quaderni

dell’Istituto di Economia e Politica Agraria, quaderno n. 24, Tipografia dell’Università degli Studi

di Perugia, Perugia, pp. 3-37.

Romero, C. and Rehman, T., 1989. Multiple Criteria Analysis for Agricultural Decision, Elsevier,

Amsterdam.

Rossi, R., Nota, D. and Fossi, F., 1997. Landscape and nature production capacity of organic types of

agriculture: examples of organic farms in two Tuscan landscapes, Agric. Ecosyst. Environ., 63/2,3:

159-171.

Rossi, R. and Nota, D., 1999. Nature and landscape production potentials of organic types of agriculture: a

check of evaluation criteria and parameters in two Tuscan farm-landscapes, Agric. Ecosyst.

Environ., in press.

Samuelson, P.A. and Nordhaus, W.D., 1993. Economia, italian edition of Economics, 14th edition,

Zanichelli, Bologna, 1993.

Schmidt di Friedberg, P., (Editor), 1987. Gli indicatori ambientali, Franco Angeli, Milano.

Stobbelaar, D.J. and van Mansvelt, J.D., 1999. The process of landscape evaluation. Introduction to

second special Agric. Ecosyst. Environ. issue of the Concerted Action: the Landscape and Nature

Production Capacity of Organic/Sustainable Types of Agriculture, Agric. Ecosyst. Environ., in

press.

Tellarini, V., 1995. Approcci metodologici, in Cannata, G., (Editor) Aziende e famiglie nella collina e

montagna appenniniche. Studi di casi, CNR-RAISA, FrancoAngeli, Milano, pp. 23-38.

van Mansvelt, J.D. 1997. An interdisciplinary approach to integrate a range of agro-landscape values as

proposed by representatives of various disciplines, in Agric. Ecosyst. Environ., 63/2,3: 233-250.

van Mansvelt, J.D., and van der Lubbe, M.J., 1999. Checklist for Sustainable Landscape Management,

Elsevier, Amsterdam.

459

460

461

462

463

464

465

466

467

468

469

470

471

472

473

474

475

476

477

478

479

480

481

482

483

Errata – corrige on print – proof

Page Row Errata Corrige1 8-9 from beginning abstract way basic information that has way how basic data has

1 10 from beginning abstract and their suitability and the possibility to judge their suitability

3 row 11 (right side) farms by (Rossi farms (see Rossi3 row 34 (right side) 1997, 1999 1997; Rossi and Nota, 19993 row 38 ish (right side) Symbol ++ + + - - - - Symbol ++ + + - - -

PROTO: the symbol before the last is a single dash “-“ and not a double one “- -“4 table caption qualitative ordinal qualitative ordinal (A)4 table caption quantitative discrete quantitative discrete (B)4 par. 2.2 row 5 of text closer to that closer to that PROTO:no extra space4 par. 2.3 row 22 (point b) highest observed highest value observed5 row 1 (left side) advantages advantage5 row 36 (left side) >From this point From this point5 row 8 (right side) with it; with it,5 row 36 (right side) (1) values values;5 row 37 (right side) (1) situation situation;5 row 40 (right side) (1) problem problem.7 row 10 (right side) than in those than with those7 row 36 (right side) 1999). 1999; Stobbelaar and van Mansvelt,

1999).8 row 40 (left side) >From this point From this point8 row 43 (left side) shorter or longer run short or long run9 row 26 (left side) to dispose off to have10 References: ELIMINATE: Feliziani, 199710 References: KEEP: Stobbelaar and van Mansvelt, 199910 References: ADD: Andreoli, M., Tellarini, V., 1999. Farm sustainability evaluation: methodology

and practice, Agric. Ecosys. Environm., in press

(1) It is a direct quotation

IN THE TEXT I have changed all case study and case studies in case-study and case-studies because in my Oxford dictionary is always spelt in this way

484

485

486487488489

490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521

522

Fig. 1 - Conversion of symbols relating to real situations into numerical codes, in the case of qualitative ordinal and quantitative discrete data.523

524

525