lable at ScienceDirect

Journal of Environmental Management 91 (2010) 2632e2638

Contents lists avai

Journal of Environmental Management

journal homepage: www.elsevier .com/locate/ jenvman

Does performance evaluation help public managers? A Balanced Scorecardapproach in urban waste services

Bernardo Guimarães, Pedro Simões, Rui Cunha Marques*

Center of Urban and Regional Systems (CESUR), DECivil, Technical University of Lisbon Avenida Rovisco Pais, 1049-001 Lisbon, Portugal

a r t i c l e i n f o

Article history:Received 9 December 2009Received in revised form16 June 2010Accepted 19 July 2010Available online 8 August 2010

Keywords:Balanced scorecardPerformancePortugalUrban waste

* Corresponding author. Tel.: þ351 218418305; fax:E-mail addresses: bernardo_guimaraes_@hotmail.c

civil.ist.utl.pt (P. Simões), rcmar@civil.ist.utl.pt (R.C. Ma

0301-4797/$ e see front matter � 2010 Elsevier Ltd.doi:10.1016/j.jenvman.2010.07.039

a b s t r a c t

The urban waste market has evolved significantly in the past decades, which among other changes, hasled to the creation of new utilities and new business models. However, very few things have changed forthe users. Urban waste collection remains mainly under the responsibility of local authorities and thecharges paid by the users in most countries are very low compared to the provision costs. This situationforces the injection of public money into the system, encouraging the ‘quiet-life’ within the utilities and,therefore, inefficiency. The present study intends to analyze the potential for the application of theBalanced Scorecard (BSc) methodology into the waste utilities. After a comprehensive revision of theurban waste sector in Portugal, the methodology of BSc and its application in local public services isdescribed and discussed. Focusing on implementation rather than on strategy, a set of performanceindicators is proposed to be utilized in the different management models of waste utilities in Portugal:the municipalities, semi-autonomous utilities, municipal companies and mixed companies. This imple-mentation is then exemplified through four case studies, one for each type of utility. This paper providesa flexible framework proposal to be applied to waste utilities operating both in Portugal and abroad.

� 2010 Elsevier Ltd. All rights reserved.

1. Introduction

Good living conditions require certain services to be provided inan appropriate way, even though they may be costly. The drinkingwater supply and wastewater collection and treatment, electricity,gas, heating or, right at the doorstep, urban waste collection aresome of them. The population sees these services so essential thatthey do not even realize the very large investments made by centralgovernments and municipalities. Along with the increase of wasteproduction and the stricter and stricter regulations of the EuropeanUnion (EU), especially concerning recycling (Bovea and Powell,2006), the principle of service costs recovery, the energy recoveryand the waste disposal became key issues in the European context,leading some countries (and Portugal in particular) to carry outmajor reforms over the whole waste sector (Massarutto, 2007).

Owing to the recognized importance of the waste sector asa public service, a huge reform has taken place in Portugal tomanage the huge infrastructure investment made in the sector(Magrinho et al., 2006). In this scope several concession

þ351 218409884.om (B. Guimarães), psimoes@rques).

All rights reserved.

arrangements were awarded to a public company (state ownedunder private law) and a sector-specific regulator for the wastesector was created. Although the regulator’s competences havefocused solely on concessionaire utilities (mainly disposal compa-nies e ‘wholesale’ market), its role which is atypical in the Euro-pean context (see Marques and Simões, 2008) is very important.This governance model requires specific standards of efficiency,price and quality of service, as well as environmental concerns, thatshould be met by the utilities to increase the quality of service.

However, there are some problems in the urbanwaste collectionservice (‘retail’ service), where the absence of rigorous regulation,the prevailing public management and the lack of expertise andaccountability cause inefficiency. The implementation of newregulations would lead to a more careful internal management.This context justifies the importance of benchmarking application,in general, and the performance evaluation of waste utilities, inparticular. This field, particularly in the Portuguese urban wastetreatment service (‘wholesale’ service), has been studied by someauthors (Gaiola, 2002; Marques and Simões, 2009), although ina regulatory perspective. This paper suggests and applies theBalanced Scorecard (hereafter, BSc) tool to four managementmodels of waste collection utilities in the operator’s perspective.These utilities have different management arrangements rangingfrom the direct service provided by the municipality to the

Table 1Urban waste market structure in mainland Portugal for the retail segment.

Arrangement Urban waste collection

Number Population

Directly by municipalities 220 7 565 058 (82.7%)Semi-autonomous utilities 5 490 674 (5.0%)Municipal companies 13 1 199 321 (12.2%)AM utilities 6 596 371 (6.1%)Total 244 9 851 424

B. Guimarães et al. / Journal of Environmental Management 91 (2010) 2632e2638 2633

publiceprivate partnerships (PPP). The BSc appears as a strategicsolution to help managers identifying themajor problems and to beaware of their success, leading to the fulfillment of future regula-tory requirements.

As the use of benchmarkingmight also be troublesome there aresome aspects that must be considered when applying it. Utilitiesmust be comparable (like with like); weighted variables must beused so as to avoid simplistic analysis; the results of benchmarkingimplementation are not immediate and easily tangible; and, finally,the outcomes might not be pleasant and can change the status quoin the organization (Marques and Witte, 2010).

The contribution of this paper is twofold. Firstly, as far as weknow, it represents one of the first attempts of using the BSc tool inthe urban waste sector and although in this research, there was noopportunity to formally carry out the BSc exercise in the wasteutilities, there was interaction with their managers. Secondly, it isalso one of the first applications of benchmarking in the wastesector under the perspective of the operator. Besides, it usesa sample of four utilities with different arrangements and comparestheir performance. In our opinion, BSc is, in fact, an important toolto reduce inefficiency. However, we also recognize the problems ofperformance evaluation tools and of benchmarking. The paperbased on a recent academic research (Guimarães, 2009) is orga-nized as follows. After this brief introduction, Section 2 providesa short description of the urban waste sector in Portugal. Themethodology of BSc is explained in Section 3. The results of itsapplication to some case studies are given in Section 4. Finally,Section 5 summarizes the main conclusions.

2. Institucional framework of the waste sector in Portugal

2.1. Waste market

Thewaste sector itself appearswith thedefinitionof urbanwaste.The Portuguese law refers to waste as ‘any substances or objectswhich the owner rejects, intends to reject or has the obligation toreject (.)’. This broad description is then narrowed into the urbanwaste scope through the same law with the following definition:‘domestic waste or similar (.) which do not exceed 1100 L per day’.

The urban waste sector is divided into ‘wholesale’ and ‘retail’services according to the lifecycle of waste. Moreover, the urbanwaste sector can also be disaggregated into three markets(Massarutto, 2006), namely, the primary market (collection of theurbanwaste), the secondarymarket (final treatment of urbanwaste,commonly in Portugal as in the majority of European countries, thedisposal in landfill) and, finally, the tertiary market (recycling).

Fig. 1. Lifecycles of urban waste.

2.2. ‘Wholesale’ service

‘Wholesale’ service is the part of the system which deals withthe waste after the collection service (when there is no transferstation) until the final treatment facility (commonly landfill). InPortugal, following the objectives established in the first Portu-guese Strategic Plan for UrbanWaste, it became necessary to definea national structure for the final destination of urban waste (Fariaet al., 1996). As such, it was decided that the creation ofa network of companies to operate the ‘wholesale’ infrastructureswould be the best option. These companies are used as regionalconcession arrangements directly from the central government. Atthis stage, in mainland Portugal, there are 29 regional systems splitinto public regional concessions, associations of municipalities(AM), which provide the services directly or through companies(with private partners or not) and municipal concessions (setbetween AM and private companies).

There are 15 regional concessions established between thecentral government and the municipalities. All of them belong,with a majority shareholding (at least 51%), to the Empresa Geral deFomento (EGF) the holding of the public company Águas dePortugal which is the state owned company responsible for envi-ronmental issues. As to the AM’s, there are 12 in charge of urbanwaste treatment, whereas only 5 of them operate on their ownbehalf. All the others operate through their companies and most ofthem correspond to the institutionalized PPP. The other manage-ment model used is the municipal concession which is a simplelong-term contract arrangement set by a group of municipalitieswith a private company. There are only two cases in Portugal.

2.3. ‘Retail’ service

The part of the system which deals directly with the user islabeled ‘retail’ service. Its main activity is the waste collection but itcan also include recyclable waste collection and street cleaning.

The ‘retail’ service corresponds to what is called the primarymarket as it concerns the first financial exchange in the lifecycle ofurban waste (user to waste collection utility). The waste utilitiesencompass mostly the municipalities themselves (direct provision)which account for about 83% of population served. Other slice of thepopulation is provided by semi-autonomous utilities, that is,financially and administratively autonomous utilities without legalresponsibility (only the municipality itself has legal responsibility)which have 5% of the sector. Finally, municipal companies, whichcover 12% of the population, can be split into two types: the onesfully owned by the municipality (66%) and the ones with mixedcapital (33%), that is, institutionalized PPPs. In both types the majorshareholders are themunicipalities and theyare always commercialcompanies. There are also some particular cases where the ‘whole-sale’ utility is responsible also for waste collection. The marketstructure of waste collection in Portugal can be seen in Table 1. It isalso worth noting the importance of contracting out for wastecollection in the waste market which represents about 42% of themarket, although in short-term contracts (1e3 years).

2.4. Life cycle of urban waste

Urbanwaste can also be split into two categories: recyclable andwaste, whose lifecycles are shown in Fig. 1. Recently, the waste hasstarted to be used for the production of organic compounds,although with poor economic viability (Renkow and Rubin, 1998),and energy (incineration). This brings economic value and reducesenvironmental impact of waste disposal. However, despite the State

Fig. 2. Distribution of rates paid by the users (households) for urban waste service.

B. Guimarães et al. / Journal of Environmental Management 91 (2010) 2632e26382634

efforts, the main final destination of urban waste continues to bethe landfill.

As to recyclable waste, in Portugal, sometimes it is also collectedby the ‘wholesale’ utilities. The ‘retail’ utilities that collect recy-clable waste themselves sell it to the ‘wholesale’ utility operating intheir area. The waste is then separated into the different types andis resold after being recycled (through washing or other chemicalprocesses). Trash is afterwards sent to the final destination.

Whereas in the Strategic Urban Waste Plan, for the year 2005,the total landfill part should have been 25%, the current result(2009) was about 60%. This was mainly due to the few other typesof equipment available and how they are distributed across thecountry. There are only two incinerators in mainland Portugallocated in Lisbon and Oporto. This situation is in line with thetheoretical idea that this service is only sustainable if encompassinga population of at least 1 million inhabitants (Hogland andMarques, 2007). The remainder of the country does not haveaccess to incineration. The same happens for organic recoveryequipments, which are also reduced at this stage.

2.5. Rates

Evaluating how much should be charged to each user is anespecially difficult problem. In Portugal, most rates are indexed tothe water bill, which definitely is not a fair system, although it doesnot have many complaints. The main reason for the lack ofcomplaints is related to the very low price paid by most users. It isestimated that 70% of the urban waste service cost is not directlypaid by the user (IRAR, 2008). Waste charges eventually end upbeing subsidized by the water service in the utilities which providethese activities together or directly by public money from themunicipality (taxpayers). Rates range from high levels to very lowlevels as Fig. 2 illustrates, but most of them are lower than 40V peryear and per household.

Fig. 3. BSc for non-profit organizations and governmental agencies.

2.6. Regulation of the waste market

In Portugal, diverging from the general context, there is a sector-specific regulator dedicated to the urban waste sector, the Institutefor the Regulation of Water and Solid Waste (IRAR). Urban wasteservice is seen as a service of general economic interest and due toits market failures it has few incentives to efficiency and innova-tion. In this line, IRAR was created in 1997 although it only startedto apply its regulatory powers effectively in 2002.

By using a ‘sunshine’ regulation, which relies on publicly dis-playing and comparing the utilities performance results ina scorecard, IRAR regulatory model has proved its success, mainlyin the quality of service improvement (Marques and Simões, 2008).Its model is based on a set of 20 performance indicators, which aredivided into three essential areas, i.e. user interests’ protection (4),utility sustainability (11) and environment sustainability (5). Also inthe benchmarking field, the results of performance indicatorsmeasurement are compared with reference values that IRARconsiders as suitable to be achieved by the utilities. Accordingly,IRAR classifies the performance of each utility as good, average orpoor, to which corresponds, respectively, a green, yellow or red ball.These scores are presented in a performance report at the end ofthe annual regulatory exercise (see IRAR, 2008).

Despite the absence of coercive power in this regulatory process,its success in the quality of service regulation has been supportedby the discomfort that managers feel when they get a red ball. Notonly the media but also the population starts to become moreinterested and aware of the importance of these matters. So, thewaste utilities get embarrassed and feel more and more pressure toprovide a better service.

Notice that the regulatory model of IRAR is only focused on thesupervision of quality of service provided. We suppose that some ofthese improvements are transferred to the users by increasing thetariffs and therefore the success of benchmarking can be put injeopardy. In the literature we found different examples of unsuc-cessful benchmarking, many of them with the problems alreadyreferred to (see Bowerman et al., 2001 or Helden and Tillema,2005). Even so, the positive examples are much more (see, forexample, the sound effect of benchmarking in a cross-countrystudy in the water sector by Witte and Marques, 2010).

Currently, IRAR is only entitled by law to regulate concessionarrangements, and as the majority of the ‘retail’ segment is notoperating in those terms, it is not regulated. This state of affairs ischanging as a new law was recently enacted, widening the scope ofintervention of IRAR. This will impose regulation to all the wasteutilities operating in this sector, affecting especially the ones in the‘retail’ service that are not so well prepared to be evaluated. In thisview, BSc could be a very useful tool to evaluate and enhanceefficiency over time.

3. Methodology

3.1. Introduction

In the beginning of the 1990s, Kaplan and Norton (1992) pub-lished an article introducing the BSc tool. This was created with thepurpose of integrating all former performance evaluation systemsinto a balanced method. The reasons were that if on the one handa financial analysis alone would lead to good financial results, onthe other hand these could be hiding a deficient organization andconsequently poor future results. Moreover, an exclusive focus onoperational improvements could be leading the company awayfrom profit or customer satisfaction. The method is based on thedefinition of a set of performance indicators under four perspec-tives (financial, customer, internal processes and learning and

Fig. 4. Proposed strategy map.

B. Guimarães et al. / Journal of Environmental Management 91 (2010) 2632e2638 2635

growth), creating a scorecard which enables the managers toimmediately gain an insight into the company’s performance witha balanced view.

The BSc came a long way to be implemented in a Europeanpublic service. The first barrier was related to the shift of themethodology itself. In the lead article referred to, the methodologywas introduced with no performance indicators or specific defini-tion of goals and objectives. Soon afterwards they were introduced(Kaplan and Norton, 1993), in 1995 the definition of strategy mapscame into place (Kaplan and Klein, 1995). After some criticism(Norreklit, 2000; Attadia et al., 2003) the authors edited a bookdescribing the implementation and providing several examples onhow to use strategy maps (Kaplan and Norton, 2004).

The evolution into a useful and suitable methodology for publiccompanies was also gradual, stressing the important role of Niven(2003). The BSc for non-profit organizations and governmentalagencies was launched with the customer perspective on top,replacing the financial perspective, as presented in Fig. 3. Thiscreated a framework for potential users of the BSc.

The current research follows the study of Kalagnanam (2004),which consisted in creating a BSc with the customer perspectiveand the financial perspective on top, balancing the interests of both.

3.2. Mission and strategy map

The definition of a mission is almost mandatory for the successof a BSc. It is the mission statement that allows the whole organi-zation to understand what it is working for. It is also from thisstatement that all the other goals are drawn.

In urbanwaste collection utilities, it is important that customersremain satisfied but without threatening the economic viability ofa utility and if possiblewith profits for the shareholders. Themission

Table 2Performance indicators for the customer perspective.

C01 Claims received C03 Relative quality of serviceC02 Results of inquiries C04 % of addressed claims

statement could be as follows: providing an effective urban wastecollection service to thepopulationandkeepinganappropriate rate-of-return for long-term shareholders. Other important aims of thiskind of utility, such as the environmental sustainability, if notimposed by law, can be included in some indicators or even ina category of its own (see Admiraal andHelden, 2003 for an exampleof a BSc with environmental aims at the wastewater sector).

From the mission, a strategic map is therefore drawn, definingobjectives in different hierarchical levels. Having the missionstatement as basis, two objectives are launched into the map permain perspective (in this case financial and customer perspectives).Given that both these perspectives are very important they shouldbe on top. Depending on the objective set, others derive from themaccording to what really matters. This leads to the creation ofa complex strategic map, which is shown in Fig. 4.

The lowest level of objectives is related to the goals. Goals mustbe achieved on a real measure. Here is where the performanceindicators are more useful. They provide measurable aspects of anorganization, carefully handpicked to be representative.

These performance indicators are usually very simple tocalculate from a reduced set of variables given the frequency withwhich they must be reviewed (quarterly, annually, etc.). Perfor-mance indicators must rely on a very accurate set of data so as tobe meaningful and should not be redundant in order to provideaccurate and concise information. A very important aspect ofperformance measurement through performance indicators isrelated to the variables used, which must univocally correspondto the information shown by the indicator. If this is not the case,the indicator will lose all its credibility as it is representingsomething with a mixed meaning. Notice, however, that somecare is required because very good results in a particular domainmight be detrimental to performance in other domains and it isimpossible to eliminate trade-offs or correlations between them.

Table 3Performance indicators for the financial perspective.

F01 Financial autonomy F03 LiquidityF02 Return on equity F04 Reduced liquidity

Table 4Performance indicators for the internal processes perspective.

P01 Price per ton P08 Staff per ton collectedP02 Payment method P09 Staff per area of municipalityP03 Collection coverage P10 AbsenteeismP04 Door-to-door coverage P11 Operational cost per tonP05 Service frequency P12 Operational cost per

area of municipalityP06 Service cleanliness P13 Disposal cost per tonP07 Quality of information system

B. Guimarães et al. / Journal of Environmental Management 91 (2010) 2632e26382636

A distinction could be made between leading and lagging indi-cators, or between performance drivers and outcome indicators,but for a question of space they were not considered (Kaplan andNorton, 1996).

The BSc methodology proposes a different set of indicators foreach of the four perspectives. Under the customer perspective, theanalysis should be different from that of a usual (for-profit)company. In this case, the customers are users of the system and donot have other choice, which makes their satisfaction degree a littledifferent from other cases. Also, the fact that there is no directcontact with the users makes their satisfaction degree measure-ment a difficult task. The four performance indicators proposed forthe customer perspective are presented in Table 2.

The only direct way to get to the user is by asking him hisopinion about the service. This performance indicator requiresaction from the utility and therefore it is not as easy to measure asthe conventional ones. However, given that there is no directcontact with the user, this measure would be a good way to inducethe utility to take action. The relative quality of service estimation issimilar to IRAR’s regulatory model. This is an aggregated indicatorwhich comprises mixed indicators mainly of environmental scope.It is based on a yet to come ranking of municipalities, but woulddefinitely increase the competition sense within the utility andbetween other utilities. The measure of claims received aims toanalyze the overall level of users’ dissatisfaction. Normally, themore claims, the more reasons there are to claim, which seems likea good proxy for users dissatisfaction. The last one, in thisperspective, is related to the customer service, addressing how theutilities are responding to their customers. If this measure is low,the consequence will be immediate. The immediate causeeeffectrelationship makes it belong to this perspective.

In the financial perspective, the performance indicators can relyon quantitative measures and are therefore, in most cases, moreaccurate. Table 3 displays these indicators.

The performance indicators are, in this circumstance, easilycomparable, although explanatory factors might exist. The mainproblems with these indicators are related to financial information,which is difficult to get when the waste service is directly providedby the municipality.

On the internal processes side there are performance indicatorsto measure efficiency levels throughout the variables. Not only thepopulation but also the employees and geographical aspects playan important role here. Therefore, same variables are crossed

Fig. 5. Implementation steps for the BSc.

several times with different ones. Table 4 shows the performanceindicators adopted in the internal processes perspective.

Considering price, communication, quality of service, humanresources and technology and taking into account the area and thepopulation of the municipality, the last perspective covers theefficiency in most aspects related to the waste utility. Thesemeasures are to be improved yearly if the focus on learning andgrowth is respected. Table 5 presents the performance indicatorsproposed.

The purpose of these last indicators is to keep the companyworking, encouraging the employees to take part in the project, tounderstand the BSc, to measure the managers’ concerns with theemployees and the evolution in terms of innovation and growth.

Computing performance indicators and applying the BSc can becostly and may require a lot of time. However, according to Fig. 5, itproves to be an achievable and possible process which might takea company to higher levels in management quality.

4. Case studies

In order to discuss the BSc methodology, it is important that anempirical example is given. In the present study four waste utilitieswere analyzed, each one with its own business model, specificallythe waste collection directly provided by a municipality (Munici-pality of Vila Franca de Xira e MVFX), a semi-autonomous utility(SMAS of Loures), a municipal company fully owned by themunicipality (EMARP in Portimão) and amunicipal mixed company(Tavira Verde in Tavira). To encompass different waste utilitiesarrangements allow not only to make some comparison betweenthem but also to increase the potential number of applicants of BScand demystify the idea that this type of techniques is only fit fora particular model of business (e.g. private company).

In all cases, being the BSc very new in this area, its imple-mentation is carried out under a merely introductory hypothesisrather than as an example or an observation of a BSc in action. Theapplication to these four waste utilities is illustrative as to where toput the values and how to read them. For each utility studied, thegeography, the business model, the demography and otherimportant informationmust be investigated and taken into accountas explanatory factors. Data were collected for a first year (in thiscase 2007) and subsequently the objectives could be set, as pre-sented in Table 6. Then, a fictitious next year’s observation wasdone on the 2008 data in order to assess changes and analyze theevolution. Table 7 shows some additional information regardingthe objectives and their level of fulfillment.

For Tavira Verde, as great amount of data was available, it waspossible to observe real data for the company and therefore it isa sound example. Tavira is a municipality with 25 542 inhabitantsspread over 611 km2. However, being located in a touristic area, thepopulation widely diverges due to the seasonal population flow.Table 6 enables us to observe that there was an evolution in theoverall user satisfaction, from 2.95 to 3.98, almost reaching thefictitious goal (4) previously set. Therewere also positive changes inprice and cost indicators. An increase in price is a positive result asthe intention is to allocate more of the collection costs to the user.

In the case ofMVFX, the Department of Environmental Quality isin charge of the urbanwaste collection for 142 290 inhabitants over

Table 5Performance indicators for the learning and growth perspective.

A01 Training actions A03 Continuous improvementA02 Budget % dedicated to innovation A04 Number of employee suggestions

A05 Average % of salary growth

Table 6Results and targets for the four case studies per performance indicator.

Aim ID Indicator Tavira Verde MVFX SMAS Loures EMARP

Real 2008 Target 2008 Real 2008 Target 2008 Real 2008 Target 2008 Real 2008 Target 2008

Customers C01 Claims received 119.0 80 142 126.4 1048 1050 169 150C02 Results of inquiries 3.98 4 N/A E N/A E N/A EC03 Relative service quality N/A N/A N/A N/A N/A N/A N/A N/AC04 % of addressed claims 92.4% 100% 90.1% 100% 89% 100% 100% 100%

Internal processes P01 Price per ton 113 105 24.9 30 81.6 65 115.4 122P02 Payment method 0 1 1 1 0 1 0 1P03 Collection coverage 95.8% 100% 100% 100% 100% 100% 100% 100%P04 Door-to-door coverage 0 20% 0 20% 16% 20% 0% 10%P05 Service frequency 18 20 55 60 286 300 63 70P06 Service cleanliness 1 2 0.7 1 2 3 1 2P07 Quality of information system 4 5 1 2 3 4 5 5P08 Staff per ton collected 3.1 2.5 1.1 1.1 1.7 2.5 2.1 1.5P09 Staff per municipality area 0.08 0.07 0.20 0.2 1.13 1 0.46 0.35P10 Absenteeism 15.7% 20% N/A E 15% 10% 5.6% 5%P11 Operation cost per ton N/A 30 25 22 28.1 25 N/A N/AP12 Operation cost per municipality area 169 145 N/A E N/A E 171 150P13 Disposal cost per ton 5191 4100 N/A E N/A E 38203 35000

Learning and growth A01 Training actions 5 4 4 5 2 3 5 5A02 Budget % dedicated to innovation N/A N/A N/A E N/A E N/A N/AA03 Continuous improvement 4 4 1 2 3 4 2 4A04 Number of employee suggestions 8 >1 N/A E 0 10 0 10A05 Average % of salary growth 4.92% 5% State N/A N/A E 0.3% 3%

Finances F01 Financial autonomy 0.17 0.2 N/A E N/A E 0.74 0.6F02 Return on equity 7% 1% N/A E N/A E 4% 1.73%F03 Liquidity 1.51 1.5 N/A E N/A E 5.31 3F04 Reduced liquidity 0.89 1 N/A E N/A E 5.53 3

E � evaluate; N/A e not available.

B. Guimarães et al. / Journal of Environmental Management 91 (2010) 2632e2638 2637

317.7 km2 of area. Being a large municipality, it would be expectedto find an organized department with all the values at hand.However, although functioning at a correct pace, all the data(especially financial) are centralised in the municipality and,therefore, it becomes nearly impossible to compute the perfor-mance indicators. The analysis was made with less than half of the

Table 7Example for two utilities with additional information and level of fulfillment.

Aim PI Tavira Verde

Information about the objectives

Customers C01 Reduce 20%C02 With door-to-doorC03C04 (IRAR, 2008)

Internal processes P01 Annual increase of 5%P02 Trend to be separatedP03 (IRAR, 2008)P04 Introduction in serviceP05 Avoid full containersP06 Containers cleanness twice a monthP07 Introduce a centralised systemP08 (IRAR, 2008)P09 Reduce with quality indicatorP10 Eliminate faultsP11 Increase recyclablesP12 Reduce 10%P13 P12 more quantity reduction

Learning and growth A01A02A03 Implement SGQ ISO 9001A04 Identify pioneersA05 Beat inflation in %

Finances F01 (IRAR, 2008)F02 Increase to 10%F03F04

N/A e not available; PI e performance indicator.

indicators, giving out very unrepresentative data and showingirrelevant changes from one year to another.

As far as the SMAS of Loures is concerned, the data set wascomplete. A very large shortfall is noticed for the first year and inthat case it becomes interesting to observe what would be thefollowing year in the BSc. From the utilities studied this is the one

EMARP

Fulfillment Information about the objectives Fulfillment

�69.6% Reduce claims of its own fault 66.7%98.1% Create surveys 0.00%N/A 0.00%13.5% (IRAR, 2008) 100%

214.1% Grow 10% 30.91%0.00% Reveal the consumes 0.00%0.00% (IRAR, 2008) 100%0.00% Start in urban areas 0.00%0.00% Reduce full containers in 10% 0.00%0.00% Reduce odours 0.00%0.00% Centralised system 100%�175.0% Introduce technological improvements �137.5%�491.6% Reduce with quality indicator �141.2%201.7% Back to other times 33.3%N/A N/A�52.3% Reduce 10% �7.64%�59.2% P12 more quantity reduction 11.4%

200.0% Plan training actions 100.0%N/A 0.00%100.0% Implement SGQ ISO:9001 0.00%100.0% Increase participation �100%95.9% �82.7%

400.0% Reduce capital 0.00%�247.3% Reduce costs �54.4%116.7% Instigate the investment �69.9%59.9% Instigate the investment �120%

B. Guimarães et al. / Journal of Environmental Management 91 (2010) 2632e26382638

which serves a major population (330 000), encompassing twomunicipalities, Loures and Odivelas, and covering 186.5 km2 ofarea.

As to EMARP, the public municipal company in Portimão is heldas a reference in the sector, responding promptly with most indi-cators, showing a high level of development and organization. Themunicipality is by the sea (like Tavira) in the Algarve and has a verylarge population during the summer months. Permanent residentsare only 49 881 in an area of 182.1 km2.

The BSc indicators measured do not show much relevantinformation. As the waste utilities studied are not doing the actualexercise of testing the BSc, but rather providing data for twodistinct years, it is possible to analyze their evolution and to assesshow the BSc should be accomplished.

5. Concluding remarks

This research is dealt with the BSc tool. It is a well-knownmethodology to the private sector, but not so common in the publicservices literature and particularly in the waste sector. So, the mainobjective of this research, focused on the urban waste collection, isto persuade the managers of waste utilities to use the BSc asa strategic tool that will allow them to increase their knowledgeabout the performance levels of their own utility, as well as toidentify the aspects that require urgent attention.

The forthcoming change in regulation in the urban waste sectorin Portugal will have special impact on the ‘retail’ segment. Thecollection segment has never been regulated and is characterizedby utilities that seldom are managed appropriately. This raises notonly environmental but also financial problems in the long-term.The prices will inevitably increase, even if against political will andthe public management will have to (at least) improve the effi-ciency of its operations. Such a changewill force utilities to improvetheir procedures, their communication policy and to manage theurbanwaste collection service as a business which, above all, servesits customers. The reduction of public money in the system ismandatory and has to come from the waste utilities themselves,with the help of valuable tools like the BSc. Measuring perfor-mance, whether or not the company is a profit seeking company,has to be done effectively in order to increase productivity andprovide value for money.

The BSc appears as a tool easy to use and simple to read that canbe implemented either by themanagement board alone or with thehelp of a dedicated team able to go across all the four perspectiveswithin the waste utility. However, the BSc success relies on gooddata quality and on how straightforward data is collected, as itmaintains the information updated.

There are no studies of performance evaluation of ‘retail’ urbanwaste service in Portugal with definitions of performance indica-tors or other kinds of measures. Utilities tend to act under theumbrella of the government participation and accommodatethemselves to this source of money. To strengthen the existingutilities, more evaluations, measurements and new studies willhave to be accomplished. The BSc could be an important anddecisive tool for this purpose. By using four different empirical case

studies, we highlight the unequivocal interest and usefulness ofthis strategic methodology.

References

Admiraal, R., Helden, G., 2003. Benchmarking in the Dutch wastewater treatmentsector. Public Money and Management 23 (2), 113e118.

Attadia, L., Canevarolo, M., Martins, R., 2003. Balanced Scorecard: uma análisecrítica. In: XXIII Encontro Nacional de Engenharia de Produção. Ouro Preto,Brazil.

Bovea, M., Powell, J., 2006. Alternative scenarios to meet the demands of sustain-able waste management. Journal of Environment Management 79 (2), 115e132.

Bowerman, M., Ball, A., Francis, G., 2001. Benchmarking as a tool for modernizationof local government. Financial Accountability and Management 17 (4),321e329.

Faria, A., Chinita, A., Ferreira, F., Presumido, M., Inácio, M., Gama, P., 1996. Planoestratégico para os resíduos sólidos urbanos (PERSU). Ministry for the Envi-ronment, Lisbon.

Gaiola, A., 2002. Efficiency evaluation in the urban solid waste systems of Portugalusing data envelopment analysis. In: DEA Symposium, Moscow.

Guimarães, B., 2009. O Balanced Scorecard aplicado aos serviços públicos locais. O casoda recolha de resíduos sólidos urbanos. MSc Dissertation, Technical University ofLisbon.

Helden, G., Tillema, S., 2005. In search of a benchmarking theory for the publicsector. Financial Accountability and Management 21 (3), 337e361.

Hogland, W., Marques, M., 2007. Sustainable waste management: internationalperspectives. In: International Conference on Sustainable Solid WasteManagement. Chennai, India.

IRAR, 2008. Relatório anual sobre o sector das águas e resíduos em Portugal 2007.Institute for the Regulation of Water and Solid Waste, Lisbon.

Kalagnanam, S., 2004. The adoption of the Balanced Scorecard in government-owned corporations. Studies in Managerial and Financial Accounting 14,407e425.

Kaplan, R., Norton, D., 2004. Strategy Maps: Converting Intangible Assets intoTangible Outcomes. Harvard Business School Press, Boston.

Kaplan, R., Klein, N., 1995. Chemical Bank: Implementing the Balanced Scorecard.Harvard Business School Press, Boston.

Kaplan, R., Norton, D., 1992. The Balanced Scorecard e measures that driveperformance. Harvard Business Review 70 (1), 70e79.

Kaplan, R., Norton, D., 1993. Putting the balanced scorecard to work. HarvardBusiness Review 71 (5), 134e147.

Kaplan, R., Norton, D., 1996. Linking the balanced scorecard to strategy. CaliforniaManagement Review 39 (1), 53e79.

Magrinho, A., Didelet, F., Semião, V., 2006. Municipal solid waste disposal inPortugal. Waste Management 26 (12), 1477e1489.

Marques, R., Simões, P., 2008. Does the sunshine regulation approach work?Governance and regulation model of the urban waste services in Portugal.Resources Conservation and Recycling 52 (8e9), 1040e1049.

Marques, R., Simões, P., 2009. Incentive regulation and performance measurementof the Portuguese solid waste management services. Waste Management andResearch 27 (2), 188e196.

Marques, R., Witte, K., 2010. Towards a benchmarking paradigm in the Europeanpublic water and sewerage services. Public Money and Management 30 (1),42e48.

Massarutto, A., 2006. Waste Management as a Service of General Economic Interest:Is the Self-sufficiency Principle Still Justified?. Working Paper Series inEconomics Universitá di Udine and Iefe, Udine.

Massarutto, A., 2007. Municipal waste management as a local utility: options forcompetition in an environmentally-regulated industry. Utilities Policy 15 (1),9e19.

Niven, P., 2003. Balanced Scorecard Step-by-step for Government and NonprofitAgencies. John Wiley & Sons, New Jersey.

Norreklit, H., 2000. The balance on the Balanced Scorecard e a critical analysis ofsome of its assumptions. Management Accounting Research 11 (1), 65e88.

Renkow, M., Rubin, A., 1998. Does municipal solid waste composting makeeconomic sense? Journal of Environmental Management 53 (4), 339e347.

Witte, K., Marques, R., 2010. Designing performance incentives, an internationalbenchmark study in the water sector. Central European Journal of OperationResearch 18 (2), 189e220.