benchmarking supply chains

International Journal of Production Research,Vol. 44, No. 23, 1 December 2006, 50655087

Efficiency analysis of supply chain processes

G REINER*y and P. HOFMANNz

yDepartment of Information Systems and Operations, Vienna University of Economics andBusiness Administration, Nordbergstrae 15, 1090 Vienna, Austria

zSAP Research, SAP Labs Palo Alto, 3410 Hillnew Avenue, Palo Alto, CA, USA

(Revision received September 2005)

We present an integrated benchmarking approach. To analyse the performance ofinter-organizational (supply chain) processes at company level we combinedependency analysis and data envelopment analysis (DEA). DEA has beenproven to be a powerful tool for benchmarking processes and for identifying themost efficient ones. Before using DEA analysis the inputs and outputs as well asthe relevant dependencies have to be identified. To support the determination ofinput and output variables we propose to use dependency analysis. We illustratethe application of this integrated approach by analysing the results of anempirical benchmarking study of 65 European and North American companies.The study shows that make-to-stock is still the predominating manufacturingstrategy of the analysed industries. Therefore, we utilize different DEA modelswith weight restrictions only for companies with a make-to-stock strategy. Theresults support our basic hypotheses that efficient supply chains lead to highfinancial performance. Furthermore they indicate improvement potential for thebenchmarked supply chain processes.

Keywords: Supply chain performance measurement; Benchmarking;TETRAD; DEA

1. Introduction

The wealth of nations and companies depends on a continuous growth in

productivity. To sustain long-term growth and profitability in a competitive

environment, industrial enterprises must continuously improve their efficiency

(Sudit 1995). Therefore questions about the measurement of efficiency are

increasingly important. The search for potential improvement of efficiency has

also been spurred on by the realization that not only do single enterprises compete

against each other but also entire supply chains (Christopher 1992).The present study examines the entire supply chain processes of companies. We

intend to identify dependencies between operational performance and financial

success. We are also looking for dependencies between several key performance

measures of supply chain processes and the financial performance of a company or

division respectively.

*Corresponding author. Email: [email protected]

International Journal of Production Research

ISSN 00207543 print/ISSN 1366588X online 2006 Taylor & Francishttp://www.tandf.co.uk/journals

DOI: 10.1080/00207540500515123

In general, the partial efficiency of a supply chain is measured by different

performance indicators pertinent to specific aspects of the performance of an

enterprise. For the purpose of benchmarking supply chains against each other we

propose a composite measure of the total efficiency, the productivity measured by

data envelopment analysis (DEA), to identify the best practice supply chains.

In this context one needs to answer questions such as: Does enterprise X, having

twice as many inventory turns as enterprise Y, but having 2% lower delivery

performance to request date, perform better; i.e. does it have a higher productivity?

One has to establish which values of input and output indicators characterize an

efficient supply chain. Multiple input and output indicators have to be considered.We propose an integrated framework applying TETRAD (a dependency analysis

approach and software tool) and data envelopment analysis (DEA) to analyse the

results of an empirical benchmarking study of 65 European and US companies.

Benchmarking helps businesses to identify potential targets for performance

improvement. This is the initial step that a company must undertake in order to

plan and implement improvement methods for achieving better performance.

Although the literature proposes a lot of benchmarking methods, research in this

area lacks rigorous analytical tools for effective analysis of benchmarking results.

Several performance measures integrated with best practice processes must be

analysed (Talluri 2000).A review of the literature shows that there are not many applications of this kind

in the area of supply chain management. An interesting work in this context was

presented by Talluri and Baker (2002) who introduced a multi-phase mathematical

programming approach for supply chain design. In general, different elements of

supply chain processes have been analysed with DEA models for specific industries.

A lot of work has been done to analyse the suppliercustomer relationship. In this

context supplier evaluation is a core topic (Narasimhan et al. 2001). Other literature

deals with the efficiency/measurement of manufacturing processes (Talluri and

Sarkis 2002). The analysing of distribution systems is another core application area

of DEA (Ross and Droge 2004).In this paper we will show an application of DEA for the entire supply chain

processes of companies, i.e. the SCOR processes source, make and deliver. These

supply chain processes within one company we call a decision making unit (DMU).

We use a widely accepted industry standard, the Supply Chain Operations Reference

(SCOR) model (SCOR 2005). Only such a commonly used model offers the chance

that there are enough non-disclosure (intra-company) data available and the data is

comparable for different companies and industries. The SCOR model is a cross-

industry process reference model designed for supply chain management. The SCOR

model was designed with the intention to support better software systems,

benchmarking (use of common measurements and terms), as well as recognizing

and adopting best practices rapidly, regardless of their origin (Meyr et al. 2002). The

SCOR model is a tool to build standardized, comparable and measurable processes

for supply chains. Not all areas of supply chain management are included,

e.g. product development and marketing.In section 3 our integrated approach presented in section 2 will be illustrated by

analysing a benchmarking study of selected industries. We will analyse dependencies

between performance measures and we will also show how DEA can be used

5066 G Reiner and P. Hofmann

to identify reference sets, i.e. companies with high productivity. We will illustratehow to gain further managerial insight for the analysed supply chain processes.

2. Integrated approach of analysing benchmarking results

Benchmarking is a technique of finding potentials for improvement of processes.It can be used to determine best practice supply chain processes that result in bettersupply chain performance. The technique is also able to identify efficient supplychain processes as a reference for the improvement of inefficient processes.

The increased application of business excellence models (e.g. EuropeanFoundation for Quality Management Excellence Model, Malcolm BaldrigeNational Quality Award) during the last years has drawn significant attention tobenchmarking (EFQM 2004, MBNQA 2005). Also the SCOR model calls explicitlyfor the application of benchmarking (SCOR 2005).

Benchmarking methods for process improvements are mostly developed andintroduced by practitioners. Many practitioners use simple techniques rather thananalytical methods. Therefore there is strong demand for effective methods ofanalysing benchmarking results, which can be used for design, analysis andimprovement of processes (Talluri 2000).

DEA, a multifactor nonparametric productivity analysis technique, has beenproven to be a powerful tool for methodical benchmarking of processes andidentifying the most efficient ones. It has been widely used in a variety of settings(Tavares 2002).

The idea behind benchmarking is to transfer results (e.g. best practices) fromone company to others or to other industries irrespective of the characteristics(complexity, mechanics, demographics, etc.) of the company (Dahlgaard et al. 1998).Using a quantitative method such as DEA for recognition of best practices requirescomparable DMUs, i.e. supply chain processes in a company. The followingrequirements have to be fulfilled to apply DEA, i.e. to enable comparison of DMUsregardless of the industry in which the DMU operates:

. Processes have to be comparable with regard to a standardized processmodel. In our case we use the definition of processes of the SCOR model.

. All DMUs have to use measures with the same definition and calculation forcomparable processes. This is fulfilled by using the performance measuresof the SCOR model for benchmarking.

Productivity of the DMUs within the DEA model is measured by the weightedsum of outputs over the weighted sum of inputs. In order to employ DEA one mustdefine for each performance indicator (e.g. delivery performance) if it is input oroutput. Uncertainty as to which measure is used as input or output for a DMU oftenoccurs (Post 2001). The performance measures of the SCOR model are not definedwith respect to input or output vectors of efficiency analysis. The SCOR variables aredefined from a practitioners point of view. The SCOR best practices rely on theexperience of supply chain experts.

We propose a framework integrating dependency analysis and DEA to overcomethis difficulty. Figure 1 shows the proposed approach for analysing the results ofan empirical benchmarking study at enterprise level. A dependency analysis of the

Efficiency analysis of supply chain processes 5067

benchmarking data to identify the input and output variables for DEA is conductedby means of TETRAD. The result of TETRAD is a decision support for theclassification of performance indicators into input and output variables. The finaldecision about the selection has to be made by the user of DEA taking into accountDEA requirements. Then DEA is applied to the input and output variables toevaluate the performance of the supply chain processes the data of which has beencollected by the empirical study.

The results of DEA have to be further analysed before best practice DMU can beidentified and managerial implications derived. For example, one has to see ifthe DEA results deliver efficient and inefficient DMUs in a sensible mix. The DEAresults should be verified, e.g. by analysing the direction and strength of thecorrelation coefficients of the measures of the efficient and inefficient DMUs.

Figure 1. Integrated approach.


DEA suggests best practice DMUs (companies or divisions within a company)for benchmarking and shows quantitatively how much a performance indicatorcould improve in order to bring the underperforming process to the best practicelevel. The main advantage of using a productivity analysis like DEA forbenchmarking is its ability to deal with aggregate process information. This allowsevaluating and controlling the overall process performance without using detailedprocess information. However, comparing only aggregate inputoutput vectors ofDMUs does not provide sufficient advice to an inefficient DMU on how to improveits processes. Further detailed process analysis is necessary for concrete steps towhere and how to change the supply chain process. Our integrated approach (seefigure 1) only shows that there are best practice processes possible that are moreefficient. We consider the underlying physics only on an aggregate level; forexample, make-to-stock strategy, number of warehouses, number of suppliers, etc.,and look primarily at the processes from a performance point of view. Our integratedapproach does not point out a general limitation of DEA. In general, with DEA boththe aggregate and the detailed comparison are feasible. The latter would requiremodelling the relevant sub-processes explicitly (Homburg 2001).

Benchmarking yields little insight into best practices if a simplistic approach isused not considering that multiple inputs of supply chain processes can producemultiple outputs; e.g. inventory and resources contribute to delivery performance aswell as cash to cash cycle time. The effects of changes of each individual measure onthe overall efficiency would be difficult to evaluate without a quantitative methodlike DEA to generate a composite efficiency measure (Sudit 1995). DEA extends thetraditional concept of efficiency to make it suitable for the multi-input multi-outputcase. This is done without reducing several inputs to a simple one-dimensional value(e.g. cost). Instead, on the basis of the DMUs inputoutput vectors, a productionfrontier is established that can be viewed as best practice. In the context of activity-based management Homburg (2001) shows that a DMU needs only determine thetotal resource usage of all its activities because DEA does not require determiningthe physics of producing activities and using resources on a detailed level.

2.1 Dependency analysisTETRAD

This section describes a dependency analysis of the data to identify the inputoutputvariables for DEA. The method we propose is represented by the TETRAD-Project(2004). The standard way to define cause and effect for a statistical model isto apply theoretical knowledge for the assumptions of the model. In caseof insufficient theoretical knowledge for specification of a unique model depen-dency analysis can be a tool for searching the unknown model space (Castilloet al. 1997).

TETRAD is a computer program used for analysis of dependency. TETRADhelps to find statistical models that explain the data and are compatiblewith user-entered background knowledge. The program output is a (partially)directed acyclic graph (DAG) with nodes (e.g. performance measures) and directededges denoting direct influences between the nodes (Spirtes et al. 1993). The programis based on a set of tests of conditional independence between the measuredvariables. The basis for the TETRAD algorithm is a match of graphical and


probabilistic methods. There are several search algorithms differing in the assump-tions they make.

For discovering causal relationships we applied an algorithm that assumes thatthere is no latent (i.e. not measured) variable that contributes to the association oftwo or more measured variables (TETRAD-Project 2004). Figure 2 shows the basictypes of edges that could appear as result of the algorithm. In the case of a directededge the used algorithm deducted that A is a direct cause of B. An undirected edgemeans the algorithm cannot reveal if A causes B or if B causes A. The absence of anedge between any pair of nodes means that they are independent, or that the causaleffect of one node on the other is intermediate through other observed variables.Finally, a triplet of nodes with undirected edges (A and B are connected by anundirected edge, A and C are connected by an undirected edge whereas B and C arenot connected by an edge) means that B and C cannot be both cause of A. This leadsto three possible interpretations:

1. A is a cause of B and C.2. B is a direct cause of A, and A is a direct cause of C.3. A is a direct cause of B and C is a direct cause of A.

An illustrative example of TETRAD results could be found in Jammernegg andKischka (2005).

The TETRAD-Project has had many successful applications so far, e.g. inmedicine and econometrics. We propose to use it for support of the identification ofinput and output data for a DEA analysis. Furthermore it could be used to gaindeeper insight into the causal dependencies of the data set.

Figure 2. Dependency graph (cf. TETRAD-Project 2004).


2.2 Data envelope analysisDEA

Data envelopment analysis (DEA) is the most widely used programming method formeasuring the productivity of decision making units (DMU). DEA is a mathematicalprogramming technique that provides upper bound estimates of the actual totalfactor productivity of DMUs as well as estimates of production frontiers (Sudit1995). DMUs using multiple inputs to produce multiple outputs can be evaluatedand compared. The inputs and outputs may have different measurement units andthey can be under certain assumptions even qualitative (Cooper et al. 1999). DEA isused to calculate the efficiency of the units relative to each other as well as theoptimal weights of input and output. In our research the term decision-making unit(DMU) describes supply chain processes within one company, i.e. the SCORprocesses source, make and deliver.

The basic model for DEA is called the CCR (Charnes, Cooper and Rhodes)model and was developed by Charnes et al. (1978). Illustrated below is the dualmodel of the CCR, which is often preferred over the primal model for usage incalculations. is the aggregate efficiency score for DMUO, the supply chain underobservation. yr0 is the output r generated by DMUO . . . xiO is the input i used byDMUO. Xj (x1j, x2j, . . . , xmj) is the vector of actual inputs used by DMUO.Yj (y1j, y2j, . . . , ysj) is the vector of actual outputs generated by DMUO. si is theamount of slack in input i for DMUO. s

r is the amount of slack in output r for

DMUO. j is the dual multiplier (the weights assigned to the inputs and outputs atDMUj). s is the number of outputs. n is the number of DMUs and m is the number ofinputs.

minj,,s

i,sr

Xmi1

si Xsr1

sr

!1

s:t:Xnj1

Xjj si xi0, 8i 1, . . . ,m2

Xnj1

Yjj sr yr0, 8r 1, . . . , s 3

, j, si , s

r 0, 8i, r, j 4

The CCR model assumes constant returns to scale. In the context of supply chainprocesses the assumption is not appropriate that all DMUs are operating at optimalscale. For example, imperfect information flows, imperfect competition andconstraints on resources may cause a DMU to be not operating at optimal scale.Therefore we use the BCC (Banker, Charnes and Cooper) model assuming variablereturns to scale. Models with variable returns to scale differ from the CCR model bythe following additional constraint (Banker et al. 1984).

Xnj1

j 1 5


This constraint ensures that an inefficient DMU is only benchmarked againstDMUs of similar set up. Whereas using the CCR model could lead to benchmarkingDMUs against other DMUs with substantially different set up.

A DMU is called efficient if the following is true for the optimal solution:

. 1 and

. all slack variables si , sr equal zero.

The optimal solution shows for all input variables the fraction of their value thatis sufficient to generate the desired output. 1 is the necessary reduction of allinputs in order for a DMU to be efficient. Therefore the value of is a radialefficiency measure.

DEA is a valuable tool not only for classifying DMUs as efficient and inefficientbut for finding improvement potentials for inefficient units as well. Each inefficientDMU (

make-to-stock (MTS) and configure-to-order (CTO). About 45% of the companiesare in consumer goods, 38% operate globally, 67% have an MTS manufacturingenvironment and 60% have revenue generated by the supply chain of no more than$500M.

The team collected data from the various supply chains by sending questionnairesfocusing on supply chain planning and processes. We applied the widely acceptedindustry standard SCOR model to assess the current state of a supply chainsbusiness processes.

The SCOR model measures internal facing and customer facing businessperformance metrics. Internal facing metrics include:

. inventory days of supply,

. inventory carrying cost, and

. cash-to-cash cycle time.

Customer-facing metrics include:

. on time delivery to request date, and

. on time delivery to commit (promised) date.

Table 1 shows the relevant SCOR performance indicators selected for ouranalysis. For a detailed description of these indicators see SCOR (2005). To answerthe question which of these indicators should be used for the analysis of the empiricaldata we used the TETRAD dependency analysis as decision support. This way wealso determined input and output variables.

Figure 3. General information.


Table

1.

Perform

ance

Measures(cf.SCOR

(2005)).

No.

Perform

ance

measure

Definition

a1

Deliveryperform

ance

torequestdate

Ordersthatare

delivered

onthecustomersrequesteddate.

a2

Fillrate

byorder

Ordersshipped

from

stock

within

24hours

oforder

receipt.

a3

Fillrate

byline

a4

Accounts

receivable

Representsalesthathavenotyet

beencollectedascash.

a5

Totalinventory

Raw

materialsandwork

inprogress,finished

goods,fieldsamples,other.

a6

Cash-to-cash

cycletime(days)

Inventory

daysofsupplydayssalesoutstandingaveragepaymentperiodfor

materialsinventories/C

OGSaccounts

receivable/revenuevendorliabilities/material

costs(raw

materialsandpurchasedparts).

a7

Cost

ofgoodssold

(COGS)

Labour,materialandoverheadexpenses.

a8

Profitability(EBIT

)Revenueexpenses(excludingtaxandinterest).

a9

Expenses(sellinggoodsandadministration)

Includes

marketing,communication,customer

service,

salessalaries

andcommissions,

occupancy

expenses,unallocatedoverheads,etc.

Excludes

interest

ondebt,domesticor

foreignincometaxes,depreciationandamortization,extraordinary

item

s,equitygainsor

losses,gain

orloss

from

discontinued

operationsandextraordinary

item

s.

a10

Net

asset

Are

calculatedastotalassetstotalliabilities;wherethetotalassetsare

madeupoffixed

assets(plant,machineryandequipment)

andcurrentassetswhichisthetotalofstock,

debtors

andcash

(alsoincludes

accounts

receivable,inventory,prepaid

assets,deferred

assets,intangiblesandgoodwill).Thetotalliabilitiesare

madeupin

much

thesamewayof

long-term

liabilitiesandcurrentliabilities(includes

A/P,accrued

expenses,anddeferred

liabilities).


a11

FTEsin

productionandmanufacturing

Number

ofem

ployeesconvertedinto

fulltimeequivalents

(FTE).Figuresforthenumber

of

personsworkingless

thanthestandard

workingtimeofafull-yearfull-tim

eworker

should

beconvertedinto

fulltimeequivalents,withregard

totheworkingtimeofafull-tim

efull-yearem

ployee

intheunit.

a12

FTEsin

warehousingandtransportation

a13

FTEsin

purchasingandreceiving

a14

TotalFTEs

a15

Number

ofship-from

locations

Number

oftier

1suppliers.

a16

Number

ofproductionlocations

a17

Number

ofwarehousingfacilities

a18

Number

ofship-tolocations

Number

oftier

1customers.

a19

Inventory

carryingcost

Sum

ofopportunitycost,shrinkage,

insurance

andtaxes,totalobsolescence

forraw

material,

work

inprogress,andfinished

goodsinventory,channel

obsolescence

andfieldsample

obsolescence.

a20

Total,order

fulfilmentleadtime

(maketo

stock)

Customer

signature/authorizationto

order

receiptorder

receiptto

completionoforder

entrycompletionoforder

entryto

start

manufacturestart

manufacture

tocomplete

manufacturecomplete

manufacture

tocustomer

receiptofordercustomer

receiptof

order

toinstallationcomplete.

Formake-to-stock

products,thelead-tim

efrom

start

manufacture

tocomplete

manufacture

equals0.

a21

Revenue

Thetotalvalueofsalesmadeto

externalcustomersplusthetransfer

price

valuationof

intra-companyshipments,net

ofalldiscounts,coupons,allowances,andrebates.


3.1 Strategy selection

We decided to use DMUs with make-to-stock (MTS) manufacturing strategy sinceit is the prevalent strategy for most industries as one can see from figure 3. A mixtureof DMUs with different strategies would lead to wrong DEA results since DMUswith different strategies have different dependencies between their performancemeasures. Therefore, the TETRAD analysis was conducted only for DMUs withMTS scenario.

In supply chain processes the positioning of the customer order decoupling point(CODP) determines the trade-off between inventory costs and costs of resources.In the case of MTS production the decoupling point is at the finished goodsinventory, i.e. one puts up with higher inventory costs. In make-to-order (MTO)production the CODP is located at the raw material, i.e. higher cost of resources areaccepted (van der Vorst et al. 1998). Olhager (2003) identified two major factors thataffect the strategic positioning of the order decoupling point, the productionto delivery lead time ratio and the relative demand volatility (standard deviationof demand relative to the average demand). If the production lead time is longer thanthe delivery time for a customer order MTO production is not advisable becauseof poor customer service.

3.2 Process description

The definition and calculation of measures for comparable processes require astandardized process model. Therefore we use the standardized process definition ofthe SCOR model for MTS with the relevant process categories and process elements.In figure 4 we describe the relevant process categories.

P1. Plan supply chain: the development and establishment of courses of action overspecified time periods that represent a projected appropriation of supply chainresources to meet supply chain requirements.

P2. Plan source: the development and establishment of courses of action overspecified time periods that represent a projected appropriation of material resourcesto meet supply chain requirements.

P3. Plan make: the development and establishment of courses of action over specifiedtime periods that represent a projected appropriation of production resources tomeet production requirements.

P4. Plan deliver: the development and establishment of courses of action overspecified time periods that represent a projected appropriation of delivery resourcesto meet delivery requirements.

S1. Source stock product: the procurement, delivery, receipt and transfer of rawmaterial items, subassemblies, product and or services.

M1. Make-to-stock: the process of manufacturing in a make-to-stock environmentadds value to products through mixing, separating, forming, machining, andchemical processes. Make to stock products are intended to be shipped from finishedgoods or off the shelf, are completed prior to receipt of a customer order, andare generally produced in accordance with a sales forecast.


Figure

4.

SCOR-m

odelsselected

process

categories

andprocess

elem

ents

(SCOR

2005).


D1. Deliver stocked product: the process of delivering a product that is maintained

in a finished goods state prior to the receipt of a firm customer order.

3.3 Results of TETRAD analysis

We obtained 28 DMUs with MTS strategy and a complete data set. It is not

surprising that we found no DMU from the automotive industry and high-tech

industry in this data set. JIT concepts play an important role in the automotive

industry and construct to order is very important for the high-tech industry. The

selected sample consists of 18 DMUs from the consumer good industry (CG), four

from the chemical industry (CH), three from the electronic equipment manufacturer

(EE), two from the medical devices industry (MD) and one from the mill industry

(MI). Figure 5 shows the result of the search by dependency analysis for the causal

structure between the following quantitative variables (see table 1).The qualitative representation of dependencies in figure 5 supports the selection

of input and output variables. Using TETRAD avoids classification problems as in

the relevant DEA literature. The variable inventory for example is classified in some

supply chain management DEA research works as input variable (Gropietsch 2003)

and in other articles as output variable (Ross and Droge 2004).

Figure 5. Tetrad graphical results.


A level of significance () of 0.2 has been chosen for this TETRAD analysisbecause it gives the most reliable output for a sample size 100 or smaller (Scheines

et al. 1994). Since we want to analyse productivity the input variables have to act on

the output variables. Dependency among the input variables or dependency between

the output variables is not required.From figure 5 we can see that a8, a9 and a19 fulfil the conditions for output

variables whereas a3, a12, a15, a16, a17 and a18 fulfil the conditions for

input variables. The variables a1, a2, a6, a13 and a20 show dependencies to

other variables but these dependencies are not directed (no arrows). The other

variables show directed dependencies but can not be classified explicitly as input or

output variables for DEA without resorting to supply chain management knowledge.We use supply chain and DEA knowledge to finally pick the input and output

variables from the TETRAD results.The profitability (EBIT) a8 cannot be used for DEA as output as suggested by

TETRAD since it is calculated from input and output variables. Instead of a8 we

select a21 (revenue) as output variable. Instead of the suggested output variables

a9 (selling goods and administration) and a19 (inventory carrying cost) we use the

corresponding cost drivers as input variables since costs cannot be used as output

variable for a productivity measure such as DEA. As replacement for a19 we will

use a5 (total inventory) as input variable because it is also a driver of a19 and

shows a strong correlation with a21 (revenue). As the drivers of a9 (logistics cost)

we use the TETRAD suggestions a17 (number of warehousing facilities) and a18

(number of ship-to locations) as input variables. A further driver of logistic costs

a15 (number of ship-from locations) is also confirmed as input variable. We replace

the suggested input variable a16 (number of production locations) with the classical

input variable for DEA a11 (FTEs in production and manufacturing) because these

variables show a high correlation with each other.Besides revenue as output variable for DEA we look for a customer-oriented

output variable. We have to identify the relevant variable out of the group of SCOR

level one service measures, a1 (delivery performance to request date), a2 (fill rate

by order) or a3 (fill rate by line). The TETRAD results suggest a3 as input variable

therefore we cannot use it as output variable. a2 shows a dependency with a3 and

therefore cannot be used either. a1 was not classified explicitly as input or output

variable by TETRAD. Nevertheless we choose a1 as customer oriented output

variable in view of the fact that it is the most frequently used measure for customer

service in the SCOR model.For final determination of the input and output variables we checked if the

following precondition is fulfilled. The value of input variables should decrease

whereas the value of output variables should increase for the desired result

(Scheel 2001).Using TETRAD analyses as decision support we have determined the following

variables as input variables: the driver of production costs (a11 FTEs in production

and manufacturing), the driver of inventory costs (a5 total inventory) and the

drivers of logistics costs (a15 number of ship-from locations (Tier 1 suppliers), a17

number of warehousing facilities, a18 number of ship-to Locations (Tier 1

Customers)). As output variables we used Delivery Performance to Request Date

(a1) and revenue (a21). Table 2 lists the means and standard deviations.


3.4 Results of DEA analysis

The number of DMUs for DEA analysis confines the number of variables. As a rule

of thumb: numbers of DMUs 4 [ input variables output variables]. In somepublications a factor of 2 instead of 4 is considered sufficient (Dyson et al. 2001,Homburg 2002). Looking at make to stock as the primary manufacturing strategy

we have chosen 28 DMUs excluding the ones with incomplete data sets.We analysed several DEA models by varying different input variables with the

same set of output variables. Table 3 specifies these process models by listing which

inputs were included in the corresponding model. All models were analysed byassuming variable return to scale (BCC) and additional weight restrictions. The

choice of the appropriate model for benchmarking should be left to the ultimate

decision maker, the supply chain manager (Ross and Droge 2004). Model 1 includes

only the drivers of production and inventory costs. Model 2 includes in addition thedrivers of supplier and customer facing logistics. Whereas Model 3 includes,

in addition to Model 1, a driver of intra-company logistics costs.Empirical research results on the importance of performance measures show that

the weights used in a DEA model could not be unrestricted (Talluri and Sarkis 2002).

To choose appropriate bounds we applied a method suggested by Roll et al. (1991).We ran the BCC model in its unbounded version and looked at the weight matrix.

Then we eliminated the virtual zero weights so that the outcomes would not be

affected directly by bounding. We defined lower (0.1) and upper (10) bounds for the

marginal rate of substitution of variables. Each combination of ratios within a group

of input and output variables can be represented as a matrix. We allowed a relativelyhigh variation of weights since we compared DMUs with make to stock production

from different industries. The following matrix W of weight restrictions was applied

for the DEA calculation using the BCC model for process model 1:

W

1 0:1 0 01 10 0 00 0 1 0:10 0 1 10

0BBBB@

1CCCCA 7

Table 4 shows the results of the DEA analysis. The column Ind. indicates the

industry for each analysed DMU. The numbers in the columns BCCm i are the

efficiency indicators with the above weight restrictions. The factor (1 ) forj 1, . . . , 28 indicates the reduction of input needed for the DMU to reach theefficiency frontier. Assuming model 1 we find seven efficient DMUs. With model 2

we find four efficient DMUs. Model 3 shows eight efficient DMUs.

Table 2. Descriptive statistics.

Inputs Outputs

a5 a11 a15 a17 a18 a1 a21

Means 714 968 502 58 610 86 654Standard deviation 843 1227 1273 256 2308 14 874


The following example illustrates how to read table 4. We find an efficiencyindicator of 0.287 for the DMU 17 in model 1. The inputs are 71.3% (1 ) higherthan the inputs of the virtual DMU that is a combination of the dominating efficientDMUs 4, 16 and 22. In the column reference setmodel 1 of the table 4 we find thecoefficients of this combination, the variables j of the BCC model. For inefficientDMUs the reference DMUs with the corresponding j are shown in brackets in thecolumn, reference set, of table 4. For each efficient DMU the number of inefficientDMUs containing this DMU in their reference set is noted; e.g. 7 in the case ofthe efficient DMU 16.

At this point we are able to identify best practice companies as targets forbenchmarking. As mentioned, the reference set of an inefficient DMU contains theefficient DMUs that are efficient with the optimal weights for the inefficient DMU.In order to move to the efficient frontier inefficient DMUs should orient towards theefficient DMUs that are part of their reference set. A unit that is part of manyreference sets is a good performer among the efficient DMUs. It is an excellentbenchmarking partner for many inefficient DMUs. Such a DMU shows true efficientperformance (Boussofiane et al. 1991). We assert that it belongs to a company withbest practice supply chain processes. Efficient DMUs that are seldom found inreference sets of inefficient DMUs are probably not true efficient performers;at least in the sense of being targets for improvement by inefficient DMUs(Ramanathan 2003).

The distributions of the selected DEA models presented in table 5 allowadditional insight. We expect that the models 2 and 3 capture more operationalcharacteristics of the SC processes and are more comprehensive in measuringperformance due to more input variables compared to model 1. Ross and Droge(2004) stated for CCR and BCC models without weight restrictions that thedistributions of efficiency indices for models with more input variables should betighter. We used weight restrictions for our models. Therefore the oppositecharacteristic would be expected (Roll et al. 1991). Models with fewer inputvariables should show a tighter distribution, a higher mean and median efficiencyvalue. The standard deviation of model 1 and 2 is 0.258 and 0.324 respectively. Themean and median efficiency scores for model 2 are the lowest in all cases.Consequently the distributions of model 1 and model 2 are in line with ourconjecture above. Other than expected the distribution of model 3 has the highest

Table 3. Summary table of alternative process models.

Model 1 Model 2 Model 3

Inputsa5 x x xa11 x x xa15 xa17 xa18 x

Outputsa1 x x xa21 x x x


Table

4.

TheDEA

resultsforeach

DMU.

DMU

Ind.

BCCm1

Reference

set m

odel1

BCCm2

Reference

set m

odel2

BCCm3

Reference

set m

odel3

1CG

0.858

4(0.147)8(0.534)24(0.261)

27(0.058)

1.000

17

0.855

4(0.150)8(0.532)24(0.261)

27(0.057)

2CG

0.716

4(0.533)22(0.029)27(0.438)

0.679

1(0.447)16(0.095)27(0.458)

0.730

4(0.236)22(0.037)24(0.252)

27(0.475)

3CH

0.802

16(0.762)22(0.054)25(0.184)

0.188

16(0.799)25(0.201)

0.756

16(0.790)22(0.013)25(0.197)

4CG

1.000

16

0.697

1(0.901)16(0.099)

1.000

13

5CG

0.383

4(0.479)16(0.030)24(0.491)

0.544

1(0.986)16(0.014)

0.456

4(0.626)24(0.374)

6CG

0.901

4(0.216)22(0.007)27(0.777)

0.151

1(0.614)16(0.386)

0.899

4(0.218)22(0.007)27(0.774)

7CG

0.894

25(0.256)27(0.744)

0.485

16(0.202)25(0.230)27(0.568)

0.897

25(0.256)27(0.744)

8CG

1.000

20.805

1(1.000)

1.000

29

EE

0.384

4(0.403)16(0.054)24(0.543)

0.651

1(0.943)16(0.057)

0.418

24(0.968)25(0.020)28(0.012)

10

CH

0.427

24(0.999)27(0.001)

0.439

1(1.000)

0.428

24(0.999)27(0.001)

11

CG

0.651

4(0.335)22(0.321)27(0.344)

0.991

16(0.650)25(0.019)27(0.331)

0.657

22(0.332)24(0.290)27(0.378)

12

CG

0.684

4(0.210)22(0.043)25(0.396)

27(0.351)

0.576

16(0.354)25(0.353)27(0.293)

0.693

4(0.300)22(0.046)25(0.402)

27(0.252)

13

CG

0.248

4(0.874)16(0.079)22(0.047)

0.179

1(0.748)16(0.248)27(0.004)

0.255

4(0.312)22(0.063)24(0.601)

25(0.023)

14

CG

0.457

4(0.959)25(0.024)27(0.016)

0.588

1(0.700)16(0.154)27(0.146)

0.455

4(0.961)25(0.024)27(0.014)

15

CH

0.976

4(0.104)24(0.209)27(0.687)

0.017

1(0.690)16(0.310)

0.938

4(0.387)22(0.001)27(0.613)

16

CG

1.000

71.000

19

1.000

317

CG

0.287

4(0.813)16(0.141)22(0.046)

0.173

1(0.695)16(0.305)

0.294

4(0.454)16(0.002)24(0.486)

25(0.058)

18

EE

0.497

4(0.570)16(0.207)22(0.223)

0.008

1(0.253)16(0.747)

0.495

4(0.569)16(0.208)22(0.222)

19

CG

0.684

8(0.353)24(0.556)27(0.092)

0.711

1(1.000)

0.698

8(0.353)24(0.556)27(0.092)

20

CG

0.650

4(0.221)25(0.056)27(0.723)

0.675

16(0.146)25(0.020)27(0.834)

0.666

4(0.258)25(0.059)27(0.683)

21

MD

0.543

22(0.912)25(0.088)

0.434

25(0.442)27(0.558)

0.544

22(0.912)25(0.088)

22

CG

1.000

11

0.298

16(0.695)25(0.305)

1.000

11

23

CH

0.404

4(0.679)22(0.038)27(0.283)

0.697

1(0.653)16(0.196)27(0.151)

0.404

4(0.679)22(0.038)27(0.283)

24

CG

1.000

60.876

1(1.000)

1.000

925

CG

1.000

81.000

71.000

10

26

MP

0.461

4(0.843)22(0.066)25(0.038)

27(0.053)

0.675

1(0.542)16(0.396)27(0.062)

0.476

4(0.891)22(0.067)25(0.041)

27(0.001)

27

EE

1.000

13

1.000

10

1.000

13

28

MD

0.916

4(0.650)16(0.346)25(0.004)

0.043

1(0.570)16(0.430)

1.000

1


mean and median efficiency scores, as well as the lowest standard deviation (i.e. atighter distribution). Process model 3 includes number of warehouses as extension to

model 1. Our results suggest that the pooling method of warehouse centralization(i.e. a smaller input variable a17) has improved the calculated efficiency for mostcompanies and is no more a performance differentiator. In other words mostcompanies have already realized the potential benefits of pooling to reduce risks forthe supply chain processes. Our advice to the supply chain managers of the analysedDMUs to further improve efficiency would be, to put more emphasis on productcommunalities, postponement and transport processes. There it seems that hepotential for competitive differentiation is still big.

Table 6 shows further input for decision makers. We want to verify the DEAresults with our theoretical knowledge about supply chain processes. Good supplychain processes could be characterized by short cash-to-cash cycle time (a6) andhigher profitability (EBIT) as a percentage of revenue (a8/a21) (Hammel et al. 2002).The efficient DMUs of this empirical study should be characterized by highprofitability (EBIT) as a percentage of revenue (a8/a21) and short cash-to-cash cycletime. The results of model 1 and 3 (see table 6) verify this conjecture. The efficientDMUs exhibit shorter average cash-to-cash cycle time and higher averageprofitability (EBIT) as a percentage of revenue than the inefficient DMUs. Model

2 supports only partially the conjecture since the average EBIT as% of revenue forefficient DMUs is lower.

The correlation coefficient calculated for all relevant DMUs for a6 and a8/a21 is0.196; i.e. relatively weak. In addition, the coefficient is positive. This is the expectedresult for a make-to-stock strategy. An increase in inventory leads to an improvedservice level at the price of an increased cash-to-cash cycle time. The higher servicelevel has a positive impact on customer satisfaction and retention. This leadsgenerally to higher revenue and profitability.

On the other hand, detailed analysis of correlation shows for all efficient DMUsa strong negative correlation coefficient between a6 and a8/a21 irrespective of theDEA process model. This confirms clearly our main hypothesis, efficient supplychains (short cash-to-cash cycle time) lead to high financial performance.Considering the weak and positive correlation of average DMUs strongly supportsour assertion that efficient companies differentiate themselves from the rest byhaving best practice supply chain processes established. The correlation coefficientfor efficient DMUs is the most negative for model 2, the most comprehensive processmodel (number of input variables) that allows for major differentiation.

The results presented in table 6 have a limitation though. They are not statisticalsignificant, due to the small sample size of each group analysed.

Table 5. DEA efficiency results of alternative process models.

BCCm1 BCCm2 BCCm3

Mean 0.708 0.566 0.715Median 0.700 0.651 0.714Standard deviation 0.258 0.324 0.252Coefficient of variation 0.364 0.573 0.353


Table

6.

DEA

perform

ance

resultsofalternativeprocess

models.

Model

1Model

2Model

3

EfficientDMU

Non-efficientDMU

EfficientDMU

Non-efficientDMU

EfficientDMU

Non-efficientDMU

Key

perform

ance

measures

a6(M

ean)

69.4

105.0

78.4

99.0

88.9

98.0

a6(Standard

deviation)

49.8

69.3

70.3

66.4

72.0

65.2

a8/a21(M

ean)

0.16

0.13

0.11

0.14

0.15

0.13

a8/a21(Standard

deviation)

0.07

0.07

0.05

0.08

0.06

0.08

Coefficientofcorrelation

a6

a8/a21

0.47

0.37

0.78

0.30

0.47

0.42


4. Conclusions

In this article we illustrated an integrated approach by combining dependency

analysis and data envelopment analysis (DEA) to explore benchmarking results of an

empirical study of supply chain processes employed by companies from different

industries. The study used the process definition and performance measures of the

SCOR model, a widely accepted industry standard. A decision making unit (DMU)

was defined by the supply chain processes (source, make and deliver) within one

company.The analysed companies are part of different industries and geographies. They

generate a broad variety of annual revenues and follow a make-to-stock manufac-

turing strategy. Using a balanced supply chain performance scorecard the study

evaluated the supply chain performance.A dependency analysis of the data from the empirical study was conducted by

means of TETRAD to identify the input and output variables for DEA. The result of

TETRAD was used as decision support for the classification of performance

indicators into input and output variables.We used the DEA model with variable returns to scale (BCC) for analysing

DMUs with a make-to-stock production strategy. We illustrated how to suggest best

practice processes as targets for improvement by DEA analyses. Examples of best

practice performers among the efficient DMUs were shown. Supply chain processes

represented by outputinput relations of such DMUs belong to the best practice

companies.Besides suggesting best practice DMUs for benchmarking, DEA showed

quantitatively how much a performance indicator could improve in order to bring

the underperforming process to the best practice level.We showed the DEA results for three process models with different number

of input variables. In particular, the distributions of the efficiency scores were

presented. The results suggest that the pooling method warehouse centralization

improves the calculated efficiency similarly for most companies. In other words most

companies have already realized the potential benefits of pooling to reduce risks for

the supply chain processes. Our practical advice to the supply chain managers of the

analysed DMUs would be to put more emphasis on product communalities,

postponement and transport processes in order to further improve efficiency and

differentiate themselves from competitors.Finally we identified dependencies between operational performance and

financial success at company level. The analysis of correlation coefficients supports

our basic hypotheses that efficient supply chains lead to high financial performance.

For the efficient DMUs we found for all three process models a strong negative

coefficient of correlation between the DEA input variable cash-to-cash cycle time

and the EBIT as a percentage of revenue. Taking further into account that on the

other hand average DMUs show a weak and positive correlation strongly supports

our assertion that efficient companies differentiate themselves by having best practice

supply chain processes established. For the most comprehensive process model (most

number of input variables) the correlation coefficient for efficient DMUs is the most

negative. The latter suggests that the drivers of transport logistics costs are essential

for productivity analysis.


The diversity of results and possible analysis demonstrates the usefulness of ourintegrated approach.

A limitation of our approach is the restriction to make to stock productionstrategy. Further research activities could apply this approach also to differentmanufacturing strategies. Another constraint was the relatively small sample size dueto the limited access to non-disclosure intra-company data.

It would be interesting to integrate customer satisfaction measures (Reiner 2005)to interpret the relationship between the different cost drivers, customer satisfactionand profitability with DEA.

References

Banker, R., Charnes, A. and Cooper, W., Some models for estimating technical scaleinefficiencies in data envelopment analysis. Manage. Sci., 1984, 30(9), 10781092.

Boussofiane, A., Dyson, R.G. and Thanassoulis, E., Applied data envelopment analysis.Euro. J. Op. Res., 1991, 51(1), 115.

Castillo, E., Gutierrez, J.M. and Hadi, A.S., Expert Systems and Probabilistic NetworkModels, 1997 (Springer: New York).

Charnes, A., Cooper, W.W. and Rhodes, E., Measuring the efficiency of decision makingunits. Euro. J. Op. Res., 1978, 2(6), 429444.

Christopher, M., Logistics and Supply Chain ManagementStrategies for Reducing Costs andImproving Services, 1992 (Pitman Publishing: London).

Cooper, W.W., Park, K.S. and Yu, G., IDEA and AR-IDEA: Models for dealing withimprecise data in DEA. Manage. Sci., 1999, 45(4), 597607.

Cooper, W.W., Seiford, L.M. and Tone, K., Data Envelopment Analysis: A ComprehensiveText with Models, Applications, References and DEA-Solver Software, 2000 (KluwerAcademic: Boston).

Dahlgaard, J.J., Kristensen, K. and Kanji, G.K., Fundamentals of Total QualityManagementProcess Analysis and Improvement, 1998 (Chapman & Hall: London).

Dyson, R.G., Allen, R., Camanho, A.S., Podinovski, V.V., Sarrico, C.S. and Shale, E.A.,Pitfalls and protocols in DEA. Euro. J. Op. Res., 2001, 132(2), 245259.

EFQM, EFQM excellence model. Available online at: http://www.efqm.org (accessed17 December 2004).

Gropietsch, J., Erfolgsfaktoren im Supply Chain Management der Konsumguterindustrie.Doctoral thesis, University of Munster (Westphalia), 2003.

Hammel, T., Kuettner, D. and Phelps, T., The re-engineering of Hewlett-Packards CD-RWsupply chain. Supply Chain Manage.: An International Journal, 2002, 7(3), 113118.

Homburg, C., Using data envelopment analysis to benchmark activities. Int. J. Prod. Econ.,2001, 73(1), 5158.

Jammernegg, W. and Kischka, P., Dynamic, customer-oriented improvement of supplynetworks. Euro. J. Op. Res., 2005, 167(2), 413426.

MBNQA, Baldrige National Quality Program. Available online at: http://www.quality.nist.gov (accessed 5 January 2005).

Meyr, H., Rohde, J. and Stadtler, H., Basics for modeling. in Supply Chain Management andAdvanced PlanningConcepts, Models, Software and Case Studies, 2nd ed., edited byH. Stadtler and C. Kilger, pp. 4570, 2002 (Springer: Berlin).

Narasimhan, R., Talluri, S. and Mendez, D., Supplier evaluation and rationalisation viadata envelopment analysis: An empiric examination. J. Supply Chain Manage., 2001,37(3), 2837.

Olhager, J., Strategic positioning of the order penetration point. Int. J. Prod. Econ., 2003,85(3), 319329.

Post, T., Performance evaluation in stochastic environments using mean-variance dataenvelopment analysis. Op. Res., 2001, 49(2), 281292.


Ramanathan, R., An Introduction to Data Envelopment AnalysisA Tool for PerformanceMeasurement, 2003 (Sage: London).

Reiner, G., Customer-oriented improvement and evaluation of supply chain processessupported by simulation models. Int. J. Prod. Econ., 2005, 96(3), 381395.

Roll, Y., Cook, W.D. and Golany, B., Controlling factor weights in data envelopmentanalysis. IIE Transactions, 2004, 23(1), 29.

Ross, A. and Droge, C., An analysis of operations efficiency in large-scale distributionsystems. J. Op. Manage., 2004, 21(6), 673688.

SAP and PRTM, Pittiglio, Rabin, Todd and McGrath (PRTM)/SAP Benchmarking Study20022003: Supply Chain Planning. Available online at: http://www.sap.com/company/press/press.asp?pressID2425 and http://www.sap.com/company/press/pdf/PRTM_SCM_Benchmarking.pdf (accessed 6 April 2005).

Scheel, H., EMS: Efficiency Measurement System, Users Manual, Version 1.3, 2000(Dortmund).

Scheel, H., Undesirable outputs in efficiency valuations. Euro. J. Op. Res., 2001, 132(2),400410.

Scheines, R., Spirtes, P., Glymour, C. and Meek, C., TETRAD II: Tools for Discovery, 1994(Lawrence Erlbaum Associates: New York).

SCOR, Supply Chain Council. Available online at: http://www.supply-chain.org (accessed22 August 2005).

Spirtes, P., Scheines, R. and Glymour, C.N., Causation, Prediction and Search, 1993(Springer: New York).

Sudit, E.F., Productivity measurement in industrial operations. Euro. J. Op. Res., 1995, 85(3),435453.

TETRAD-Project, The TETRAD Project: Causal Models and Statistical Data.Available online at: http://www.phil.cmu.edu/projects/tetrad/index.html (accessed17 December 2004).

Talluri, S., A benchmarking method for business-process reengineering and improvement.Int. J. Flex. Manuf. Syst., 2000, 12(4), 291304.

Talluri, S. and Baker, R.C., A multi-phase mathematical programming approach for effectivesupply chain design. Euro. J. Op. Res., 2002, 141(3), 544558.

Talluri, S. and Sarkis, J., A methodology for monitoring system performance. Int. J. Prod.Res., 2002, 40(7), 15671582.

Tavares, G., A Bibliography of Data Envelopment Analysis (19782001). RUTCOR ResearchReport, RRR 0102, January, Piscataway, New Jersey, 2002.

Van der Vorst, J.G.A.J., Beulens, A.J.M., De Wit, W. and Van Beek, P., Supply chainmanagement in food chains: Improving performance by reducing uncertainty.Int. Trans. in Op. Res., 1998, 5(6), 487499.


benchmarking supply chains

Documents