Development of a Comprehensive Framework for the Efficiency Measurement of Road Maintenance Strategies using Data Envelopment Analysis

by Mehmet Egemen Ozbek (meozbek@vt.edu), Jesús M. de la Garza (chema@vt.edu), and Konstantinos Triantis (triantis@vt.edu), Virginia Tech

Sponsored By: Virginia Department of Transportation (VDOT)

1988: A survey performed on about 10% of all USA infrastructure by the National Council on Public Works Improvement revealed that the nation’s roads were in better than fair condition (Mirza 2006). 1998 2001 2003 2005

The Federal Highway Administration endorsed “asset management” to be the future approach of road maintenance for all state departments of transportation (DOTs) (JLARC 2002).

Currently: State DOTs are implementing a variety of performance measurement systems focusing mainly on the effectiveness of their road maintenance processes. Nonetheless, state DOTs need to and in fact seek to measure not only the effectiveness of their road maintenance processes but also the efficiency of- and value added through- such processes (TRB 2006).

1) Background

Similar surveys by American Society of Civil Engineers revealed that the nation’s roads were in poor condition (Mirza 2006).

Allocating resources to preserve, operate, and manage the nation’s transportation infrastructure. Calls for the utilization of management, engineering, and economic principles to help state departments of transportation (state DOTs) in making decisions as to how resources should be allocated. Requires, as an integral part, performance measurement. (Geiger 2005)

2) Significance of the Problem and the Proposed Research

Current Road Maintenance Performance Measurement Systems

Solely focus on “effectiveness” measures, e.g., level-of-service.Disregard the “efficiency” concept,e.g., the amount of resources utilized to achieve such level-of-service.

Do not investigate the effect of the environmental factors, e.g., climate and location.Do not investigate the effect of the operational factors, e.g., traffic, load, design-construction adequacy.

Not knowing how “efficient” state DOTs are in being “effective” can lead to excessive and unrealistic maintenance budget expectations.

For the cases in which comparative analyses are made, disregarding such external and uncontrollable factors and using pure effectiveness results may lead to unfair comparisons.

The findings of the research outlined herein will contribute new knowledge to the asset management field in the road maintenance domain by providing a framework that is able to differentiate effective and efficient maintenance strategies from effective and inefficient ones; as such, the impact of such framework will be broad, significant, and relevant to all transportation agencies.

3) Purpose, Objectives, and Hypothesis

The purpose of this research is to develop and implement a comprehensive framework that can: Measure the overall efficiency of road maintenance operations. Consider the effects of external and uncontrollable factors on such efficiency.

The specific objectives of this research are, through the use of real data, to identify: 1) The relative efficiency of different units in performing road maintenance services.2) The reasons of the efficiency differences between units.3) The effects of the external and uncontrollable factors on the efficiency of units.4) The benchmarks (peers) and best practices that pertain to the inefficient units. 5) The fundamental relationships between the maintenance levels of service and the budget requirements.

The hypotheses of this research are as follows:1) A significant portion of the observed efficiency differences between different units can be attributed to the effects of the external and uncontrollable factors.2) A unit that achieves the best road maintenance level-of-service, i.e., the most effective one, does not necessarily have to be the one that utilizes the most resources to achieve such level-of-service.

4) Literature Review

When there are multiple inputs/outputs

Partial Efficiency Measure Approach: Has a potential to result in serious misunderstandings about the overall efficiency of a process when only a single partial efficiency ratio is used (Craig and Harris 1973). Total Factor Efficiency Measure Approach: May result in subjectivity as the decision-maker prescribes weights to be assigned to each input and output variable (Cooper et al. 1999).System Dynamics:Requires the definition of the of mathematical relationships between key variables (Chasey et al. 1997). Regression Analysis: Compares the efficiency of units against a hypothetical average performance (Sexton 1986).

Data Envelopment Analysis (DEA)DEA can simultaneously deal with multiple outputs and multiple inputs.

DEA does not require the specification of a priori weights for the variables.

DEA is non-parametric.

DEA focuses on the best-practice frontiers.

0

1

2

3

4

5

0 1 2 3 4 5 6 7 8 9 10 11x1/y

x2/y

E

A B

C

D

F

B'

lOB'l lOBl

5) Framework Components 6) Contribution to the Body of Knowledge

1)Contribution to the Body of Knowledge in the Road Maintenance Literature Transportation Research Board identified that some topics related to maintenance management need more examination. This research addresses, to a certain extent, two of such topics as listed below (TRB 2006):

Fundamental relationships between road maintenance levels of service, budget, and labor requirements.

Best practices in specifying maintenance and operations performance, as used in contracting for these services.

This research, by taking the efficiency concept into account, will significantly improve the ways that are currently used to measure and model the performance of road maintenance.

2) Contribution to the Body of Knowledge in the Performance Measurement LiteratureEngineering is not a discipline in which research about performance measurement is performed as much as it is performed in disciplines such as operations research, management control systems, and economics. There has been limited amount of research that uses DEA in the engineering discipline (Rouse 1997, Triantis 2004). This research is believed to contribute to the literature of performance measurement (specifically DEA) by developing a generic framework that is based on engineering principles.

Input

OutputEfficiency=Q= (Cooper et al. 1999)

(Charnes et al. 1994, Rouse 1997, Ramanathan 2003)

1 2 3

4 5 6

In

Refine the comprehensive list of input-output variables and uncontrollable factors

Address the issue of uncontrollable factors

Prepare the data to be used in the DEA models Perform data mining

Clean the data

Allocate the data to the DMUs

Perform data conversion and data rearrangementChoose the type of DEA models to be run

Run the DEA models and obtain the efficiency score, targets, and peer(s) for each DMU; and the overall efficient frontiers

Derive overall conclusions (such as the reasons of inefficiency, benchmarks, best practices) that would help the decision-making process

Decide on the size of the DMU

Develop the comprehensive list of input-output variables and uncontrollable factors

Identify the effects of uncontrollable factors (i.e., environmental and operational factors) on the efficiency of the units

1

2

3

4

5

6

7

8

9