Benchmarking of Indian Urban Water Sector: Performance Indicator System

versus Data Envelopment Analysis

By: Dr. Mamata Singh, Dr. Atul K. Mittal, and Dr. V. Upadhyay

Presented by:Dr. V. Upadhyay

ProfessorIndian Institute of Technology, Delhi, Hauz Khas, New Delhi – 110016, India

Problems of Water Sector• Exponential population growth • Industrialization and urbanization • Infrastructure inadequacies • Inadequate funds• Inefficient water use

Objective of the Paper• To demonstrate the benchmarking

approach using DEA and PIS methods• For the selected 12 Indian urban water

utilities (municipal bodies) of Maharashtra state/province

• Compare and analyze the results

Benchmarking Concept • A “benchmark” is a reference or

measurement standard used for comparison

• “Benchmarking” is the continuous activity of identifying, understanding and adapting best practice and processes that will lead to superior performance

Benchmarking Methods • Performance Indicator System - PIS (partial metric

method) • Performance Relative to a Model Company (engineering

approach –viz. DEA) • Performance Scores based on Production or Cost

Estimates (“total” methods)• Process Benchmarking (involving detailed analysis of

operating characteristics)• Customer Survey Benchmarking (identifying customer

perceptions); and • SWOT (strengths, weaknesses, opportunities and threats)

analysis

Performance Indicator System (PIS)• PIS comprises a set of performance indicators (PIs) and

related data elements which represents the real instances of utilities’ context

• Performance indicator (PI) is simply the ratio of an output to an input, or an input to an output

• Each PI reflects only one input and one output level, so difficult to view overall performance of a DMU

• Need to adopt a summary (or overall) measure of performance values

• Methods of aggregating PIs are generally subjective• PIs may be tailored for different objectives and priorities

Data Envelopment Analysis (DEA)• Charnes, Cooper and Rhodes (1978) first introduced the

term data envelopment analysis (DEA) • CCR considered constant returns to scale (CRS) model

with input orientation • DEA is a linear programming technique that incorporates

multiple inputs and outputs for assessing relative efficiencies

• The most efficient utilities are rated to have an efficiency score of one, while the less efficient utilities score between zero and one

• The utilities lying on efficient frontier are identified as best practice utilities by DEA

Inputs and Outputs in DEA• Resources utilized by the units or conditions

affecting their operation are typical inputs• Measurable benefits generated or service levels of

the utility constitute the outputs• Number of DMUs to be considered for DEA should

at least be three times the sum total number of inputs and outputs

• There should be positive correlation between inputs and outputs

• The basic CRS-DEA model with an input orientation has been considered and detailed below

Discussion: PIS versus DEA Technique

• DEA measures of performance are based on simultaneous consideration of multiple inputs and outputs, while each PI measures performance in relation to one input and one output only

• A unit offering unremarkable values on individual PIs can still be deemed a good performer in the context of DEA when its all-round performance is taken into account

• DEA is suitable for setting targets which would render a DMU relatively efficient but offers no indication of how relatively efficient DMUs might improve their performance

• PIs do give indications as to the specific aspects of performance that a DMU might strengthen

• DEA generally reflects overall, and PIs, factor-specific performance

Performance Indicators considered for the study

Performance Indicator System• Relative Performance Score (RPS) of

a DMU = Overall Performance Score of that DMU/ Highest Overall Performance Score

• The DMUs are then ranked on the basis of their RPS

DEA• Input Oriented CRS DEA Model• DEA is performed using free version

of DEAF software (by Joe Zhu) to obtain their relative efficiency scores

• DMUs are ranked on the basis of their relative efficiency scores

Inputs and Outputs for DEA

Relative Performance Score and Ranking using PIS

PIS Results • Wardha has a highest RPS of 1.0 (100%) and is ranked

first • Yavatmal is ranked the last with least RPS of 0.34• There are 3 DMUs with RPS < 50% and 4 DMUs with

RPS > 75% ≤ 100% • For 5 DMUs RPS varies between 50-75%• Minor difference in overall performance scores of better

performing utilities Wardha and Bhusawal, Bhusawal and Aurangabad, Aurangabad and Chandrapur.

TE Score and Ranking using DEA

DEA Results • 4 DMUs Aurangabad, Bhusawal, Parbhani

and Wardha have highest TE of 1.0 • Chandrapur has higher TE score of 0.850• Yavatmal has the least TE score of 0.487 and

is the only DMU with TE score < 50%• 2 DMUs with TE score < 75% and > 100%• 5 DMUs with TE score ranging between 50 -

75%

Potential for Input Reduction • Maximum for Yavatmal• Wardha, Bhusawal, Chandrapur and Prabhani deliver

higher output levels at relatively lower input usage• For most of the DMUs, potential for % reduction in total

expenditure and staff size are same (Amravati, Chandrapur, Kolhapur, Nanded Waghala, Solapur and Yavatmal) or closer (Nashik) except for a DMU Dhule

• For Dhule, potential for % reduction in staff size is lesser than that of total expenditure as also implied by its PI values

Note: Higher is the ranking – lower is the performance level.

PIS vs DEA Ranking • Same for 6 UWUs (Amravati, Dhule, Nanded - Waghala,

Solapur, Wardha and Yavatmal) • Differs maximum by six positions for the UWU Parbhani • Differs by one position for 2 UWUs (Bhusawal and

Chandrapur) • Differ by two to four positions for rest of the 3 UWUs• Rank correlation coefficient is 0.833. The high correlation

further indicates the fact that PIS and DEA methods agree strongly on the UWUs performance rankings

• UWUs with lower ranking positions (Yavatmal, Solapur, Nanded Waghala, Amravati and Nashik) under both PIS and DEA methods

Conclusion • Better performing UWUs under both PIS and DEA are - Wardha,

Aurangabad, Bhusawal and Chandrapur {except Prabhani)• Performance of Yavatmal is found to be lowest in both the methods• UWUs having lower performance levels under both PIS and DEA

are - Yavatmal, Solapur, Nanded Waghala, Amravati and Nashik • PIS and DEA results are complimentary

DEA efficient DMU Prabhani may focus on reducing its staff size but not expenditure level as can be observed from PI values

DEA efficient DMU Bhusawal indicates its superior performance as regards staff size but reasonable scope for reduction in total expenditure

Recommendation • DEA inefficient DMUs need to focus on reducing

their input (total expenditure and staff size) usage altogether

• DEA efficient DMUs need to explore the opportunity for reduction in specific input usage (total expenditure or staff size or both) by analyzing PIS outcomes as well

• Similar analysis may be performed by the state/independent regulator to devise suitable performance linked incentive mechanism

Thank You