theory for performance indicators

8/13/2019 Theory for performance Indicators

1/137

POLITECNICO DI TORINODept. of Production Systems and Business Economics

Performance indicators

by Domenico Maisano

Course of

Quality Engineering


2/137

PERFORMANCE INDICATORS by D.Maisano

Course of QUALITY ENGINEERING

2

Quality

(of a product or service) refers to the degree

to which a set of inherent characteristics fulfills (stated

or implied) requirements/needs (ISO-9000, 2000).In other words, Quality is the ability to fulfill different

types of requirements (e.g., productive, economical,social) with appropriate and measurable actions.

Quality and Indicators


3/137



3

to achieve Quality is therefore necessary:

1) To identify/observe these needs;2) To fulfill them using the appropriate resources.

Indicators are essential tools for observing

the

evolution of the process and its context.

Quality and Indicators


4/137



4

A Quality Management Systemis a set of tools for

driving and controlling an organization, considering all

different Quality aspects (ISO-9000, 2000):

human resources; know-how and technology; working practices, methodologies and procedures.

Quality guidelines


5/137



5

Typical requirements

are production, cost, time, ROI,

stakeholders exigencies or expectations.

Typical monitoring

activities are:

performance evaluation of the whole firm aspects(processes, suppliers, employees, Customer Satisfaction); market analysis (shares, development opportunities); productivity and competitors analysis; decisions about product innovation or new servicesprovided.

Quality guidelines


6/137



6

Modeling

a system means describing/representing

it,

considering the targets which should be met.A proper performance measurement system should be

arranged to verify if responses are consistent withrequirements.

A system of indicators should become an information system

for estimating the level of achievement of quality targets (UNI-11097, 2003).

Observation, representation, modelling


7/137



7

The Importance of Indicators

Indicators are supposed to be a representationmodel of a generic system/process.As a consequence they are the basis for

evaluations, judgments or decisions.


8/137



8

The importance of indicators

ISO 9004/2: 2009 standard explains how organizations can use a qualitymanagement approach to achieve sustained success.

8.3 MEASURING

ORGANIZATIONAL PERFORMANCE

8.3.1 GENERAL APPROACH TO ASSESSING PROGRESS

Assess your progress by measuring

your actual achievements against the results you plan to achieve.

Use a measurement

and analysis process to monitor your actual achievements against the results you

plan to achieve.

Select appropriate and practical methods for collecting information and monitoring key performanceindicators.

8.3.2 DEFINING KEY PERFORMANCE INDICATORSIdentify factors that are critical to your organizations success.

Select your organizations key performance indicators

(KPIs).

Implement KPIs

throughout your organization.

Use your KPIs to measure performance.


9/137



9

They may influence

and modify actions and decisions.

If a firm measures indicators a, b and c, neglecting x,y and z, then managers will pay more attention to thefirst ones. The firm gains core strengths in producing a,b and c.Firms become what they measure!(Hauser and Katz, 1998).

If maximizing a, b and c leads to long-term profit, theindicators are effective. If a, b and c lead tocounterproductive decisions and actions, then indicatorshave failed.

But even worse! Once the enterprise is committed to theseindicators, indicators provide tremendous inertia.

Indicators also have a deep normative effect


10/137



10

Measurements, Estimators,

Performance Indicators, Metrics...

Indicators are used within a wide range offields:

operations management (logistics, production,quality management, marketing, etc...);

Economy and finance;Social sciences;

Sport races (gymnastics, diving, figure skating,

etc...).


11/137



11

The customers perception on a new product;

The personal name of an individual;

The cycle time of a manufactured product.

Social status indicators (GDP):

- e.g. European countries with debt-to-GDPratio lower than 3% can adopt Euro currency;

- country inflation is running at 2.7 %;the air quality index value is 6 and so on

Some Examples


12/137



12

Open questions

Dealing with indicators, many questions arise:

How many indicators shall we use?

Is there an optimal set?

Is this set unique?

If not, what is the best one (if it exists)?

Can all these indicators be aggregated in a unique one?

What are the properties of indicators?

Are indicators the same as measurements?

Are there operative methods for defining and testing theindicators of a generic process?


13/137



13

Measurements (according to tradition)

According to physicists:

Measurement: is the process by which one canconvert physical parameters to meaningful numbers.

Instrument: may be defined as a device fordetermining the value or magnitude of a quantity orvariable.

The standard measure of each kind of physicalquantity is the unit; the number of times the unitoccurs in any given amount of the same quantity is the

number of measure. Without the unit, the number ofmeasure has no physical meaning.


14/137



14

Measurements (according to tradition)

The basic units are called fundamentals, while all

the others which can be expressed in terms offundamental units are called derived units, andformed by multiplying or dividingfundamental units.

E.g., the primary fundamental units which mostcommonly used are length, mass, and time, whilemeasurement of certain physical quantities in thermal,electrical, and illumination disciplines are alsorepresented by fundamental units.

Every derived unit originates from some physical law

defining that unit.


15/137



15SI:

InternationalS

ystemo

fUnits

(Sy

stmei

nternationald'units)


16/137



16

What about the measurement of non-physicalquantities?

e.g., raw scores of intelligence; grades of leather;

guidability of a car.WARNING! In many cases measurement

procedures are not as much well-defined andcannot be objective.

Any other types of Measurements?


17/137



17

A measurementis the assignment of numbers to properties of objectsor events in the real world, by means of an objectiveand empirical

operation, to describe them.objective

independence on the subjects (judgments may be

repeated by different subjects, obtaining the same result);

empirical it comes from reality (i.e., its not pure theory);objects/events and the corresponding properties/relations areobservable.

For example, an evaluation is not a measurement because of the lackof objectivity (it may depends on subjective perceptions).-

evaluations on the importance of customer needs according to

customers (within HoQ)

-

evaluations about the teaching activity of your teacher(s)

Some definitions according to S.S. Stevens


18/137



18

The Theory of Scales of Measurement by S.S. Stevens

Note: The columns listing the basic operations and the permissible statistics arecumulative: to an operation/statistic listed must be added all thoseoperations/statistics preceding it. Conversely, the column about

the

mathematical transformations which leave the scale-form invariant is inverselycumulative: each transformation in the column is contained in thetransformation immediately above it.

Scale

Type

Basic Empirical

Operations (relations

among objects)

Permissible scale-

transformations

Permissible Statistics Examples

Nominal Equivalence (equality) Permutation (one-to-

one substitution)

Mode, chi square Eye colour, place of

birth, etc

Ordinal Order (greater or less) Monotonic

increasing function

Median, percentiles Surface hardness,

military rank, etc

Interval Distance (addition orsubtraction)

Linear function(x=ax+b)

Mean, standard deviation,correlation, regression,

analysis of variance

Temperature in C,serial numbers, etc

Ratio Ratio (multiplication or

division)

Similarity (x=ax) geometric mean, harmonic

mean, coefficient of

variation, logarithms

Temperature in K,

weight, age, number

of children, etc

Tab. 1 Classification scheme of measurements/indicators depending on their scale types

CARDINAL

CATEGORICAL

No Oil In Rivers


19/137



19

Nominal ScaleThe numbers/symbols serve only as labels

or tags for identifying and

classifying objects.

e.g., classes concerning marital status: (A)-Married, (B)-Divorced,

(C)-Never married, (D)-Widowed.When used for identification, there is a strict one-to-one correspondence

between the numbers and the objects.e.g., numbering

of football players for the identification of the individuals.

Since the purpose is just as well served when any two designatingnumerals are interchanged, this scale form remains invariant under thegeneral substitution or permutation group.

Only a limited number of statistics, all of which are based on frequencycounts, are permissible, e.g., percentages, and mode.


20/137



20

Ordinal Scale

A ranking scale in which numbers are assigned to objects to indicatethe relative extent

to which the objects possess some characteristic.

Can determine whether an object has more or less of a characteristicthan some other object, but not how much more or less

e.g. T-shirt sizes: XS, S, M, L, XL and so on.

Any series of numbers can be assigned that preserves the ordered relationships between the objects (transitivityproperty of inequalitieshas to be satisfied too). Strictly (monotone) increasing transformationsof scale are permissible.

In addition to the counting operation allowable for nominal scale data,ordinal scales permit the use of statistics based on centiles, e.g.,percentile, quartile, median.


21/137



21

Interval ScaleNumerically equal distances on the scale represent equalvalues in the characteristic being measured.It permits comparison of the differences between objects.The location of the zero point is not fixed. Both the zeropoint and the units of measurement are arbitrary (e.g.altitude above sea level).

Any positive linear transformation of the form(x) = a + bx (with b>0) will preserve the properties ofthe scale.It is not meaningful to take ratios of scale values.

Statistical techniques that may be used include all of thosethat can be applied to nominal and ordinal data, and inaddition the arithmetic mean, standard deviation, andother statistics commonly used.


22/137



22

Ratio ScalePossesses all the properties of the nominal, ordinal, and intervalscales.

It has an absolute zero point (generally corresponding to theabsence of manifestation of the characteristic being measured).It is meaningful to compute ratios of scale values.

Only proportionate transformations of the form (x) = bx, where

b is a positive constant, are allowed.All statistical techniques can be applied to ratio data.

A scale with no arbitrary choice of a unit

or zero

to be made is

called absolute scale(e.g., the number of members of a givencollection of objects is determined uniquely, or the probability

of a

certain event [0,1]). The absolute scaleis a special variantofratio scales.

invariant just under the (trivial) identity transformation (x)=x.


23/137



23

The Mohs

scale of mineral hardness characterizes the scratch

resistance of various minerals through the ability of a harder materialto scratch a softer material.

The Mohs

scale is based on 10 standard minerals:

Link between observed relations and numerical relations

1. Talc

2. Gypsum3. Calcite

4. Fluorite

5. Apatite

6. Feldspar

7. Quartz8. Topaz

9. Corundum

10. Diamond

The Mohs

scale is strictly an ordinalscale (no interval property) and

uses half-numbers for in-between hardnesses. For instance, dolomite,which scratches calcite but not fluorite, has a Mohs

hardness of 3.5.

The ordinal relation between materials in terms of hardness is reflectedby the ordinal relation among the corresponding numbers(ISOMORPHISM = same structure).


24/137



241 kg 2 kg 5 kg 8 kg

A BC D

Weights, lengths and resistances are fundamental magnitudes defined onratio scales.

They satisfy the criterion of additivity, in fact they can be added in thephysical sense.

The corresponding relations (e.g., larger than, commutative and

associative property, transitive property of (in)equalities, etc) are reflectedby the relations among numbers.

We can make statements based on numbers, which reflect the relationsamong real objects

Link between observed relations and numerical relations

For example:

1)

C is heavier that B, which is heavierthan A

2)

A, B and C added together weigh asmuch as D


25/137



25

A

transformation of scale

is admissible

when it preserves the basic

relations among objects (represented on the scale).

-

see the 2nd

column of the previous table.

It can be demonstrated mathematically that every scale type admitsonly some specific transformations of scale (see Roberts).

Other definitions


26/137



26

A

statement/assertion using scales is called meaningful

if its truth (or

falsity) is unchanged whenever any scale is replaced by another

acceptable scale (i.e. obtained by an admissible scale transformation).In other words, the truth (or falsity) of the statement is independent onthe (acceptable) scale used.

General statements:-

for a ratio scale, it is meaningful to say that one thing is so-and-so

many times as big as another (e.g. C weights five times more than A);

- If f is an interval scale, then comparisons of intervals aremeaningful, e.g. f(a) f(b) is equal/greater than f(c) f(d).-

If f

is an ordinal scale, it is meaningful to say that f(a) is greater

than f(b).

Meaningfulness of a statement


27/137



27

Examples

-

the statement x1 x2 = 3(x3 x4

)

is meaningful for an intervalscale

(e.g. try with the linear transformation (x) = ax + b, being a>0),but not for an ordinalscale (e.g. try with the monotonic transformation(x) = x2);

- the statement x1 > x2 + x3 + x4 is meaningful just for a ratioscale(e.g. try with the similarity transformation (x) = ax, being a>0).

When a transformation of scale generates a paradox, it means that thestatement is not permissible (for that scale).

How to check the meaningfulness of a statement?


28/137



28

Try by yourself to check these:

-

the statement that your telephone number is twice as large as mine is

meaningless, because a change by one number in the last digit of

your

phone number

certainly an admissible transformation

could change

the statement from a true one to a false one.

- the statement that you weight less than the Statue of Liberty is true,independently of what scale is used to measure weight, and hence is a

meaningful statement.

Other Examples


29/137



29

(x) = (9/5)x + 32

yesterday today waters triple point

Temp. in C 5 10 0

Temp. in F 41 50 32

Today the temperature is twice as yesterday.

Today the difference between the air temperature and thetemperature of the waters triple point (i.e. =0C) is twice as

yesterday.


30/137



30

Using the previous checking mechanism, we can provide an indirect

demonstration of what reported in the Tab. 1, about the permissiblestatistics:

Example

Statement: the mean value of three objects is greater/smaller/equal toa certain quantity (defined on the scale of interest).

It is meaningful for an intervalscale (e.g. try with the lineartransformation (x) = ax

+ b, being a>0);

but it is not meaningful for an ordinalscale (e.g. try with themonotonic transformation (x) = x2). Instead, the median ismeaningful for ordinal scales

Meaningfulness of a statement about the use of(permissible) statistics

WARNING:It is not appropriate to make statements using non-permissible statistics


31/137



31

1

2

3

45

6

7

8

9

10

1112

0 1 2 3 4 5 6 7 8 9 10

Example 1

1 2 3 4 5 6 7 8

For which scale(s) is meaningful to say that ? 3x x

x1 x2 x3 x4 x5x

Data set: x1 = 1, x2 = 2, x3 = 4, x4 = 5, x5 = 8

1 2 3 4 5 6 7 10 11

x1 x2 x3 x4 x5

x

Lets try with a (monotonically increasing) transformation,

permissible for ordinal scales

(TRANSFORMED SCALE):

New data set: x1= 2, x2= 3, x3= 5, x4= 6, x5= 11

(x)


32/137



32

1

2

3

45

6

7

8

9

10

1112

0 1 2 3 4 5 6 7 8 9 10

Example 2

1 2 3 4 5 6 7

For which scale(s) is meaningful to say that ? 3x x

x1 x2 x3 x4 x5x

Data set: x1 = 1, x2 = 2, x3 = 4, x4 = 5, x5 = 6

1 2 3 4 5 6 7 8 9 10

x




x 3.6

x ' 5.2

x1 x2 x3 x4 x5


(x)


33/137



33

(Order) Categories C1 C2 C3 C4 C5

Occurrences 1 3 3 1 1

Scale 1 1 2 3 4 5

Scale 2 1 4 9 16 25

(x) = x2

MEDIAN MEAN

3 2.78

9 (= 32

) 9 (2.782

)

In detail (individual measurements): sum median mean

Scale 1 1, 2, 2, 2, 3, 3, 3, 4, 5 25 3 25/9 = 2.78

Scale 2 1, 4, 4, 4, 9, 9, 9, 16, 25 81 9 81/9 = 9

confirmation:

the mean is not an admissible statistic for ordinal scales

try yourself with other statistics/scales!

A similar case


34/137



34

What about the indicators of dispersion?

-5 -4 -3 -2 -1 0 1 2 3 4 5

N 28

x 0.5

x 1R 10

s 3.1xx

(SCALE 1)

1 2 3 4 5 6 7 8 9 10 11

N 28x ' 5.5 (x)

x ' 5 (x)

R' 10 R

s' 3.1 s

(SCALE 2)

Redefine the "zero point" on this interval scale:x = (x) = 1x + 6

sR

x 'x '

s'R'


35/137



35

What about the indicators of dispersion?

-5 -4 -3 -2 -1 0 1 2 3 4 5

N 28

x 0.5

x 1R 10

s 3.1

(SCALE 1)

-4 -2 0 2 4 6 8 10 12 14 16

N 28x '' 5 (x)

x '' 4 (x)

R'' 20 2 R

s'' 6.2 2 s

(SCALE 3)

Full

linear transformation (redefinition of the "zero point

and rescaling):

x = (x) = 2x + 6

xx

sR

x ''x ''

s''R''


36/137



36

1

2

3

45

6

7

8

9

10

1112

0 1 2 3 4 5 6 7 8 9 10

Example 3

1 2 3 4 5 6 7

For which scale type(s) is meaningful to say that ?1 1 2 3 2 4 5R (x ,x ,x ) R (x ,x )

x1 x2 x3 x4 x5x

Data set: x1 = 1, x2 = 2, x3 = 4, x4 = 5, x5 = 6

1 2 3 4 5 6 7 8 9 10

x




x1 x2 x3 x4 x5


(x)

1R 2R

1R ' 2R '


37/137



37

Example 4Provide an analytical demonstration of the meaningfulness of the

previous

statement for interval-

and ratio-scales.

Try the same with the statement 1 1 2 3 2 4 5(x ,x ,x ) (x ,x )


38/137



38

Show that the geometric mean, i.e.

is an admissible central tendencyindicator for ratio-scales

only.

Show that the harmonic mean, i.e.

is an admissible central tendencyindicator for ratio-scales

only.

Some additional exercises for you!1/ n

n

nGM i 1 2 n

i 1

x x x x ... x

HM n

1 2 ni 1 i

n nx

1 1 11...

x x xx


39/137



39

The concept of Measurement according to theRepresentational theory of Measurement (by Roberts)

We now study the concept of Measurement

from a more

theoretical and philosophical perspective.The modern theory of measurement is representational: numbersassigned to objects/events must represent the perceived relations

between the properties of those objects/events.The representational definition is based on four parts:

1) An empirical relational system;

2) A symbolic/numerical relational system,

3) A representation condition;

4) A non-uniqueness condition.


40/137



40

1) Empirical Relational System

Consider a specific characteristic/property/feature (for example thelength of an object) of a well-defined context/system.

A is the set of all possible manifestationsof the characteristic:

R is a family of empirical relationsamong the manifestations of A:

then is called empirical relational system

1

A ,..., ,...i

a a

1R ,..., mR R

A,RA


41/137



41

2) Numerical/Symbolic Relational System

Analogously, N is a set of symbols (generally numbers):

and P is a family of relationsamong the elements of N:

Then is called symbolic/numerical relationalsystem

1N ,..., ,... in n

1P ,..., mP P

,PZ


42/137



42

3) Representation ConditionIn general, according to the so called Symbolic RepresentationalTheory, a measurement is an objective empirical

function which

maps homomorphicallythe empirical relational system into thesymbolic/numerical relational system

P

NA

RF

A Z

a3a2

a1 n1

n2

R2

R1

P2

P1


43/137



43

2 mappings are defined:

1) M: A N Homomorphism (homo=alike, but not identical)its not

a one-to-one mapping (separate manifestations/entities may beindistinguishable from the viewpoint of the feature of interest,

e.g. the

height of an individual)

2)

F: R

P Isomorphism

(i.e., samestructure)

its a one-to-one

mapping.

Empirical relations should be preserved by symbolic/numerical relations

In other words, whatever inference can be made in the numericalrelational system MUST apply to the empirical one (e.g., you are tallerthan the queen of England

or your weight is twice as much as mine).

3) Representation Condition


44/137



44

The representation condition may be valid for more than one mappingfunction M (conversely, the second mapping (F) is unique).

There are admissible transformations from one scale to anotherwithout invalidating the representation condition.The uniqueness condition defines the class of transformations

for

which the representation condition is valid.

E.g., for ordinal scales, all monotone increasing functions areadmissible transformations.

E.g., regarding dimensional measurements (defined on ratio scales), a

plethora of scales based on different units are commonly used(meters, foots, inches, miles, (nautical) leagues, etc).

Pounds to grams: (x)= 453.6 x

4) Non-Uniqueness Condition


45/137



45

Definition of indicator according to theRepresentational Theory

An indicator ( I ), similarly to a measurement, can beconsidered as a map from an empirical system (the realworld) into a symbolic system (usually, a numerical system).However, the mapping between the empirical and symbolrelations (ISOMORPHISM), unlike measurement, is notrequired.

the result is that some relations among numbers MAY NOTreflect the empirical relations among manifestations.

In addition, the mapping between A and N should not

necessarily be objective.In this sense, the concept of Indicator is more relaxed.


46/137



46

The Representational Theory of Indicators

P

A

RF

IA Z

a3

a2

a1 n1

n2

R2

R1

P2

P1

: A N i i iI a I a n


47/137



47

No isomorphism among empirical andsymbolic/numerical relations

Ia1a2

a4

a5a3 n

1

n2

A N

In general, for indicators, the mapping of the empirical systeminto a symbolic may introduce new relations (not present in the

empirical system) or modify the existing ones.


48/137



48

INDICATORS MEASUREMENTS


49/137



49

A context

is a part of reality that we are focusing on (e.g. a

manufacturing process or a logistic process).

A representation-target is a specific aspect of a certain context that werepresent by one or more indicators (e.g., in the case of a logisticprocess, the classification of suppliers, the delivery time etc).

A model

is a conceptualization of the representation-target, obtained

by defining one or more indicators.

Some definitions

EmpiricalSystem

Model

Symbolic/NumericalSystem

INDICATOR(S)DEFINITIONINTERPRETATION AND DEFINITION OF

THE REPRESENTATION TARGET


50/137



50

Example 4.3 The wheelbase of a motor vehicle (the geometricaldistance between two car-axles) is a dimensional measurement, and

therefore also an indicator. The relationships among symbolicmanifestations (numerical distance values) are isomorphically

linked to

the relations among physical manifestations (physical distance).

Example 4.6 Let us consider the representation-target classification

of the students of a class, operationalized (i.e. modeled) by theindicator name of the student. This indicator associates each student(empirical manifestation) to the corresponding name (symbolicmanifestation). In nature there is no order relation among the empirical

manifestations (the students), which corresponds to the alphabeticalorder relation among the symbolic manifestations (names). So, thename of the student is only an indicator, not a measurement.

In other words: the order relation among symbolic manifestations doesnot correspond to any existing relation among real manifestations.

Indicators and measurements

PERFORMANCE INDICATORS b D M i


51/137



51

Given a representation-target, the related indicator (or set ofindicators) is not univocally defined.

The same representation-target can be represented by moreindependent indicators (or indicator sets).

Example 1: Let us consider 4 production lines .

The representation-target is identifying the line with the lower defectiveness. Atleast, two different indicators can be adopted:

1) The number of defective products in the daily production;

2) The number of defects detected during a full control of the daily production.

Of course, there is no mathematical transformation which biunivocally

links the

two indicators.

Example 2: different indicators about to evaluate job stress (e.g., NIOSH

equation, Strain Index, Ergonomitermometer, and many others).

Non-uniqueness for (single) indicators

, , ,



52/137



52

Non-uniqueness for (single) indicators

z''1

z''2

z'''1

z'''2

z'''3

I

I

I

Real manifestations ofthe empirical system

Manifestations of different

symbolic systems

a1

a2

a4

a5

a3

z'1

z'2 z'3

z'4

A

Z

Z

Z

Fig. 4.2.Schematic representation of the condition of non-uniqueness. The same

representation target is operationalized by three different indicators (I, I andI).Some empirical manifestations, indistinguishable according to one indicator,

can be distinguished by another one (for example the manifestations a3and a5 are

undistinguished byI, but distinguished byIandI)



53/137



53

Measurements the requirement of homomorphism for mappingempirical manifestations and isomorphism for mapping relations definesa class of equivalent scales. Each equivalent scale can be mappedinto another. All the possible transformations form the so called class

of admissible transformations.

Indicators

there can be different indicators (or sets of indicators)

which represent the same representation-target. However, there maynot necessarily be a transformation

from one indicator into another.

Analogous representation-targets might not be comparable, ifrepresented by different indicators.

Non-uniqueness: comparison between measurement andindicators

PERFORMANCE INDICATORS by D Maisano


54/137



54

Different indicators sets may refer to the same representation-target.What is the best way of selecting them?; When is the representation

exhaustive?.Even if there are many ways for representing the same process, thebest

one cannot be a-priori identified.

We will see some properties and rules to support the selection andaggregation of indicators and the verification of the representationmodel.

Non-uniqueness for Sets of indicators



55/137



55

Example 4.2 In a manufacturing company producing hydraulic valves,the purpose is to improve the quality of the produced goods. The

following indicator set is implemented to operationalize thisrepresentation target:- I1 number of units produced;

- I2

(monthly) number of units categorized as defective, and rejected.

A second possible indicators set is given by:

- I1

number of units produced by the first of 4 production lines;

- I2

average percentage of detective units: result of a spot check onthe 5% of total production.

Non-uniqueness for Sets of indicators



56/137



56

Objective indicators. They link empirical manifestations to symbolicmanifestations objectively. The mapping does not depend on the

subject who performs it.

Example 4.7 Let us consider the indicator: quantity of goodsproduced in a plant. The empirical manifestation (production) canobjectively be connected to a corresponding symbolic manifestation(number of goods produced). If there is no counting mistake, differentpeople (or automatic devices even) will determine the same value.

Objective indicators are not necessarily measurements (think ofExample_4.6 about the name of a student).

objectivity is a necessary but not sufficient condition to be a

measurement.

Classification: Objective and Subjective indicators



57/137



57

Subjective indicators. Empirical manifestations are subjectively mappedinto symbolic manifestations, depending on subjective perceptions or

personal opinions (different people can map the same empiricalmanifestation into different symbolic manifestations).

Example: the individual vote at the elections.

Example 4.8 The representation-target evaluation of the design of acarcan be operationalized

by the indicator quality of the design,

codified with a 5 level scale (1-very bad; 2-poor; 3-fair; 4-good; 5-

excellent).The indicator is subjective because the same empirical manifestation (aparticular car design) can be associated to different symbolicmanifestations (the 5 scale levels), depending on the subject.

Classification: Objective and Subjective indicators



58/137



58

Measurements as specific Objective indicators

reference for

comparison

object to be

measured

(metrological)traceability

chain



59/137



59

Traceability chain (it guarantees Objectivity)



60/137



60

Subjective indicators: the reference is in the mind of theevaluator

Lev.

reference

referenceperformance levels evaluation

scale

subjective

judgment

comparisonterm

object to be

evaluated

Lev. Lev. Lev.Lev.



61/137



61

Since subjective indicators provide essential information about theindividuals behaviour and perceptions, they are often used and studied

by many disciplines in the area of Social, Behavioural and CognitiveSciences.The numeric encoding is a common way to make the informationpractical for the user. However

when the relations among symbolic

manifestations do not correspond to relations among empirical ones this sort of conversion may distort the analysis results

Sometimes, the symbolic relations are (questionably) promoted,with reference to the empirical ones.

The importance of Subjective indicators



62/137



62

Importance levels

related to the Customer Needs within the HoQ.

they are supposed to be defined upon an ordinal

scale; however

their distance matters!Coefficients

of the relationship matrix (, , ) within HoQ.

Judgments

within AHP or Evaluations

within FMECA.

just to mention some indicators familiar to you.

Examples of promotions



63/137



63

Basic indicators. They are obtained from a direct observation of ageneric system.

Derived indicators. They are obtained combining the information of oneor more source

indicators (basic or derived). Synonyms: composite,

aggregatedindicators.

Example 4.9 Let consider the derived indicator:percentage ofdefectivesin a production line, given by:

Classification: Basic and Derived indicators

(number of defective units)1

3 2 (total number of produced units)

II

I



64/137

y


64

Example 4.10An organization for the environmental protection askstwo local Agencies -

A and B -

to estimate the pollution level of the

exhaust emissions of a motor vehicle, on the basis of four pollutantsconcentrations:

: the concentration of NOX

in the exhaust emissions [g/m3];

: the concentration of un-burnt hydrocarbons [g/m3];

: the concentration of CO [g/m3];

: the concentration of (PM10

) [g/m3].

Other examples of Derived indicators

XNOI

COI

HC

I

10PMI



65/137

y


65

Other examples of Derived indicators

BASIC Indicators DERIVED Indicators (I grade) DERIVED Indicator (II grade)

INOx

IHCICOIPM10

INOx (5 level scale)IHC (5 level scale)ICO (5 level scale)IPM10 (5 level scale)

Agency A:

IATOT = max {INOx,IHC,ICO ,IPM10 }

Agency B:

Real concentration of four airpollutants (expressed as g/m3) Mapping of the concentrationinto a 5 level (qualitative) scale Aggregations of the four derivedindicators into another one

10' ' ' '4

XNO HC CO PMB

TOT

I I I II

non uniqueness of theaggregation criteria



66/137

y


66

Derived indicators according to the Representationalapproach

Basic Indicators(I1, I2, I3)

Derived Indicator(I4)

Indicatorsaggregation

a1

a2a3

x1

x2

b1

b2

y1

y2

c1c2

c3

c4

z1

z2

z3

I1

I2

I3

A

B

C

X

Y

Z

w1

w2z3

w4

w5

I4d1:(x1, y1, z1)

d2:(x1, y1, z2)

d3: (x1, y1, z3)d4:(x2, y1, z1)

d5:(x2, y1, z2)

d6:(x2, y1, z3)

d7:(x1, y2, z1)

d8:(x1, y2, z2)

d9:(x1, y2, z3)

d10:(x2, y2, z1)

d11:(x2, y2, z2)

d12:(x2, y2, z3)

W

Sets of the empirical

manifestations of thesource indicators

Sets of the corresponding

symbolic manifestations ofthe source indicators

New set of empirical manifestations:

combination of the source indicatorssymbolic manifestations: D =(X x Y x Z)

Set of the symbolic

manifestations of thederived indicator



67/137


67

The problem of the aggregationThe aggregation of several indicators into a derived indicator is notalways easy to achieve, especially when the information to synthesize

is assorted.The aggregation of heterogeneous indicators is often complex and

based on questionable simplifications.Example: Analyze the derived indicator RPN (= SOD, from FMECA).



68/137


68

A Utopian schemeDerived indicators may be (in turn) aggregated into a derived indicatorof higher grade.

By extending this concept to the limit, we can imagine to define asuper-indicator, synthesizing all the aspects of the systeminvestigated.

I1

I2

I3

I4I5

BASIC Indicators DERIVED Indicators(I grade aggregation)

DERIVED Indicator(II grade aggregation)

Starting indicators

Super-indicator ofglobal performance



69/137


69

Problem: What is the best production line?

Example of aggregation

Table 3.1. Experimental data of four equivalent production lines for exhaust-

systems in a manufacturing plant

Production lines

Indicators

daily production [no. per day] 360 362 359 358

daily defectiveness [no. per day] 35 32 36 40

unavailability equipment ratio [%] 4.00% 5.50% 4.50% 5.00%

For each basic indicator we may establish the following rankings:

daily production:

daily defectiveness:

unavailability equipment ratio:

1 2 3 4



70/137


70

For each line, we can associate a Bordas

indicator IB

(x):

where Ii

(x)is the ranking obtained by a line xwith regard to i-th

basic

indicator and m is the number of indicators used (in this case, m=3).

The winner (the best line x*) is given by (Borda

1781):

where Sis the set of compared lines. In this example, .

Bordas indicator

1

m

B iiI x I x

* minB Bx S

I x I x

, , ,S



71/137


71

Some simplifying assumptions/drawbacks:

-

the (3) different criteria have the same weight;

- equal distance between two consecutive rank positions;-

Bordas

method is sensitive to irrelevant alternatives.

Bordas indicatorApplying Bordas method to data in Table 3.1, we obtain the following

results:

2 2 1 5BI 1 1 4 6BI

3 3 2 8BI

4 4 3 11BI According to Eq. 3.16, the final ranking is: .

The winner (i.e. the line with best overall performance) is line .


C f QUALITY ENGINEERING


72/137


72

Sensitivity to Irrelevant alternativesTable 3.3. Experimental data of three equivalent production lines for exhaust-systems in a manufacturing plant (1stcondition)

Production Lines

Indicators

daily production [no. per day] 367 350 354

daily defectiveness [no. per day] 35 30 37

1 2 3BI

3 1 4BI

2 3 5BI

Table 3.4. Experimental data of three equivalent production lines for exhaust-systems in a manufacturing plant (2

ndcondition)

Production Lines

Indicators daily production [no. per day] 367 350 345

daily defectiveness [no. per day] 35 30 33

1 3 4BI

2 1 3BI

3 2 5BI

Despite the marginal role played by , its change may influence the

winner




73/137


73

In Football

group tournaments, a popular ranking system is:

3-1-0 points systems for win-draw-loss.

In Formula 1

races: the winner receives 25 points, the second place

finisher 18 points, with 15, 12, 10, 8, 6, 4, 2 and 1 points for

positions

3 through 10.

Scoring system in Moto GP races:1st=25 points, 2nd=20 points, 3rd=16 points, 4th=13 points, 5th=11 points,6th=10 points, 7th=9 points, 8th=8 points, 9th=7 points, 10th=6 points,11th=5 points, 12th=4 point, 13th=3 points, 14th=2 points, 15th=1 point.

Scoring system is even more complex for Decathlon Competition.

all these rankings are questionable! A proof is that they (periodically) change over

time.

The problem of aggregation in Sports




74/137


74

How to select the best indicator or (set of indicators) for a specific goal?

God only knows

However analyzing the indicator properties may

help.

Properties can be classified into four groups:

1 - general properties,2 -

properties of sets

of indicators,

3 -

properties of derived

indicators,

4 - accessory properties.These properties can represent a useful tool to select and evaluate

performance indicators in different contexts.

Properties of indicators




75/137


75

Properties of indicatorsCategory Properties Short description

Consistency with

the representation-

target

The indicator should properly represent the

representation-target.

Level of detail The indicator should not provide more than therequired information.

Non counter-

productivity

Indicators should not create incentives for counter-

productive acts.

Economic impact Each indicator should be defined considering the

expenses to collect the information needed.

General

properties

Simplicity of use The indicator should be easy to understand anduse.

Exhaustiveness Indicators should properly represent all the system

dimensions, without omissions.

Non redundancy Indicators set should not include redundant

indicators.

Monotony The increase/decrease of one of the aggregatedindicators should be associated to a corresponding

increase/decrease of the derived indicator.

Properties of

sets of

indicators

S = {Ii, Ij, Ik}

Properties ofderived

indicators

(Ii, Ij, Ik)ITOT Compensation Changes of different aggregated indicators may

compensate each other, without making the

derived indicator change.

Long term goals Indicators should encourage the achievement ofprocess long-term goals.Accessory

properties Impact on the

stakeholders

For each indicator the impact on process

stakeholders should be carefully analysed.

1)

2)

3)

4)

sub-




76/137


76

1.1 Consistency with the representation-target

Every indicator should properly operationalize

a representation-target.

The mapping should be thoroughly verified before using the indicator.

Example 4.13 Referring to the representation-target sales of a

manufacturing company, the indicator IS

total number of goods soldin the whole year

is used to represent the process. Later, companymanagers realize that quarterly information on sales would be moreuseful for estimating the seasonal trend. Consequently, a new indicator

IS

representing the total number of quarterly sold goodsreplaces the

first one.

According to the representation-target, the second indicator is moreaccurate than the first one. It comprehends some important empirical

manifestations (quarterly information on sales), ignored by IS

.

1. General properties (referred to single indicators)




77/137


77

1.2 Level of detail (resolution)

If an indicator maps two empirical manifestations

not distinguished

according to a representation-target into different symbolicmanifestations, then the level of detail is excessive.

Excessive level could complicate the analysis and could be economicallywasteful.

Example 4.14A manufacturing company produces metal screws. Therepresentation-target is the production rate of the company. The indicator Irepresents the daily weight of produced screws. If the indicators accuracy is1 g/day

when 10 kg/day would be enough

then the level of detail is

excessive.

Example 4.15


your impression on the external design of car A is:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15VERY BAD EXCELLENT

* * * * *

your impression on the external design of car B is:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15VERY BAD EXCELLENT




78/137


78

1.3 Non counter-productivity

The idea of counter-productivity can be shown as follows. Some

indicators (Ik

, Ih

, Ii

, Il

, Im

, In

, Io

, Ip

) are aggregated in a derived

indicator (ITOT

), representing the global performance. If the increase of

a specific source indicator (Ik

) is associated with the decrease of one or

more indicators (for example Ih

, Ii

, Il

), determining a decrease of the

global performance (ITOT ) too, then Ik is counter-productive.


Ik Ih Ii Il Im In Io Ip

increasing indicator(counter-productive)

correlated indicatorswhich inevitably decrease

indicators whichdo not change

a'

a

global

performanceITOT(a) < ITOT(a)

= = = =

obviously, theseindicators should bedefined on scales with- at least- order relation




79/137


79


Example 4.16 The main purpose of a construction company is to reduce

the construction work time, in order to take a competitive advantage. Thispurpose may generate some counterproductive actions:

-

to save time, employees do not obey safety rules (i.e. they do not use the

safety helmets and harness);

- working vehicles, rushing around the building site to save time, becomedangerous for the public safety;-

customer satisfaction decreases, because the result of the work

is poor, due to

the excessive speed up.

In this case, focusing too much on a single dimension of the process can becounter-productive in general terms.





80/137


80


Example 4.17 To estimate the costumer satisfaction, a call-center uses

several indicators. Two of them are the following:I1

average number of rings before answering the phone;

I2

percentage of unanswered calls.

These two indicators can be counter-productive because employees can gamethe process answering the phone immediately and then putting the

call on hold

before starting the conversation.

Although that behaviour

increases the value of selected indicators, it is

absolutely counter-productive according to other indicators of customer

satisfaction. For example, the number of exhaustive answers, the

courtesy, the

number of queued calls etc.

In conclusion, the increase of I1

and I2

indicators could badly impact the proc-

ess, making the global customer satisfaction decrease.





81/137


81


Example.A car supplier (e.g. VW) offers an incentive contract to a dealerto encourage his effort in selling cars. The incentive is a bonusin case thenumber of cars sold in one month is larger than a threshold (e.g. 20 cars).

Possible counter-productive action: the dealer may tend to concentrate his effortin those months in which he realizes that there is a real chance

to get the bonus;

in the less promising

months

on the contrary

he may spare himself or even

postpone car sales.

The results are large fluctuations in the suppliers demand (this maybedangerous for bullwhip effect).


1 2 3 4 5 6 7 8 [month]

[cars sold]

20 threshold




82/137


82

1.4 Economic Impact

The economic impact of an indicator strictly depends on the nature of the

system investigated. The impact can be studied in relative terms, by com-paring two different indicators operationalizing

the same representation-

target.


Ia

Ib

Ic

mapping

Indicators operationalizing thesame representation-target

Economic Impact []




83/137


83

1.4 Economic Impact

Example 4.18A small company produces punched metal components. To

check the quality of the manufactured holes, two possible indicators can beused:

I1

)

diameter measurements, taken by using an accurate calliper. To

check each hole, the time needed is about 9 seconds.

I2

) the result of a (go - no go) manual testing, using a hard-gauge. Timeneeded is about 3 seconds.

Supposing that the cost for measurements is directly proportional to timespent, then indicator I2

can be considered three times more economical

than indicator I1

.





84/137


84

1.5 Simplicity of use

Indicators that are difficult to understand and interpret, because

reference to real data has been lost, are often rejected by potentialusers

This property, as the previous one, can be studied in relative terms, bycomparing two (or more) different indicators operationalizing

the same

representation-target.

The comparison concerns the aspects related to simplicity of use

(for ex-

ample, indicators should be easy to understand, easy to use, they shouldhave a clear meaning, they should be largely accepted, etc...).





85/137

Q

85

1.5 Simplicity of use

Example 4.19 Likewise Example 4.18, we set-up a mapping to evaluate the

simplicity of use of two indicators (I1

and I2

), considering the following criteria:

(a) technical difficulty in performing measurements;

(b) time required.


(a) technical skill (b) time required simplicity of use(sum of the two level no.)

I1 2 3 (2 + 3) = 5

I2 1 1 (1 + 1) = 2

(3 levels) (3 levels)




86/137

86

A set or family of indicators is composed by the indicators selected torepresent a generic process. These indicators can be grouped into sub-

sets, depending on their characteristics.

2. Properties of sets of indicators

2AI

Dimension A2: empiricalmanifestations

Symbolic manifestationsof dimension A2

a1a2

a4

a5

a3

Dimensions of theprocess represented

A2

Empirical manifestations that are undistinguishable, according

to the representation-target operationalized by the indicator2A

I

DimensionA2exploded

v1

v3

v2

Fig. 4.12. Schematic representation of the concept of indicators set. For each

process dimension (A1, A2, A3, ), it is possible to define one or more indicators.All the indicators form an indicators set or family. Indicator

2AI represents the

dimension A2




87/137

87

Example 4.20 Three indicators represent a companys sales:

I1

) number of products daily sold;

I2

) daily turnover;

I3

) daily takings (not including the credit given).

Two possible process states are:i-th

day:

I1

(i) = 203 pcs;

I2

(i) = 4820 ;

I3

(i) = 3600

j-th

day:

I1

(j) = 178 pcs;

I2

(j) = 5680 ;

I3

(j) = 3546

Each state is a snapshot

of the process condition in a particular day.

States of a process/system




88/137

88

2.1 Exhaustiveness

Indicators of the set must treat all parts or aspects of the system of

interest, without omission.


I

v1

v3

v2

b1b2

b4

b5

b3

Empirical manifestations of

the process dimension

Symbolicmanifestations


to the representation-target operationalized by the indicator

(a) Incorrect definition of one indicator (b) Missing indicator for a certain dimension

I (missing Indicator) v1v3

v2

b1

b2

b4

b5

b3

v4

Empirical manifestations of

the process dimension

Symbolicmanifestations


to the representation-target operationalized by the indicator




89/137

89

2.1 Exhaustiveness

Example 4.21 A manufacturing company producing metal components, uses

the following indicators:I1

) total number of units yearly produced;

I2

) manufacturing time;

I3

) lead times

(i.e. supply time, tool change time, etc...).

This set of indicators has been defined with the aim of differentiating the possiblesystem conditions. If two possible states, undistinguished by the previous indicators,are distinguished by a further indicator

which has previously been ignored (for

example I4

, the number of defective units produced)

then the set is not

exhaustive:


I1 I2 I3 I4

State 1 300000 pcs 160000 h 700 h 2.1%

State 2 300000 pcs 160000 h 700 h 3.5%

inexhaustive set of indicators

exhaustive set o f indicators




90/137

90

2.1 Exhaustiveness

In general, exhaustiveness is not fulfilled when

one or more indicators are missing;

one or more indicators are not enough specific (e.g. too low resolution).


Exhaustiveness is certainly the most important condition to guaranteeconsistency between indicators and the process/system represented.

A general rule to guarantee exhaustiveness is not available(non-uniqueness of process modeling by indicators).

Also, since every process is a dynamic system evolving over time,representation targets may change (for example, today the importance ofservices is much higher than 20 years ago).

Exhaustiveness has to be periodically checked.




91/137

91

2.2 Non redundancy

If a set (or family) of indicators (F) is exhaustive, and if it continues to

be exhaustive even when removing one indicator (Ik

), then the removedindicator is redundant.

Usually, indicators that can be deduced from other ones are redundant.The presence of redundant indicators does not provide additional

information on the process.


if I1 I2 I3 I4 I5is an exhaustive set of

indicators

and ifI1 I2 I4 I5 is a set which continuesto be exhaustive

then I3 is a redundant indicator




92/137

92

2.2 Non redundancy

Example 4.22 in a manufacturing company producing plastic component, theprocess is represented by four indicators:

I1

)

total number of units (yearly) produced;

I2

)

number of defective units (yearly) produced;

I3

)

manufacturing time;

I4

)

efficiency of the production, calculated as:

(term I5

refers to lead times

(supply time, tool change time, repairing time, etc..);

I5

) lead times.

Assuming that the set of indicators fulfils the property of exhaustiveness, theindicator I3

is removed from the set. If the residual set (I1

, I2

, I4

, I5

) continues

to be exhaustive, then the indicator I3

is categorized as redundant.


3 54

3

I III




93/137

93

In general these properties (Exhaustiveness and Non redundancy) areamong the most difficult to analyze.

An indirect proof is that the scientific literature includes several modelsto support the development of a system of performance indicators or

measurements.

Two of these are:

- the Balanced Scorecard Method;

- the EFQM

(European Foundation for Quality Management) model.





94/137

94

The performance of a complex system is related to severalinterconnected dimensions;

Indicators that make one dimension look good while deflating anotherare avoided (minimize negative competition).

The number of measures has to be limited to a vital few (not tocomplicate or to make the system representation too expensive).

The concept of balanceconsists in placing the right emphasis on all thesystems important dimensions

(e.g. often the economic/financial dimension

is favored).

Balanced Scorecard




95/137

95

Balanced Scorecard

CustomerHow well do we satisfy

our internal and externalcustomers needs?

FinancialHow do we look toour stakeholders?

Internal Business ProcessHow well do we perform at

key internal businessprocesses?

Learning and Growth

Are we able to sustain innovation,change, and continuous

improvement?

Secondary Influence on PerformancePrimary Driver of Performance

Fig. 5.6. The four perspectives of the Balanced Scorecard model (Kaplan and

Norton 1992). With permission




96/137

96

For each dimension/perspective, a (sub-)set of performance indicatorshave to be identified.

A way to help this task is to answer the following questions:

-

Financial

What are our strategic financial objectives?

- Customer What do we have to do for our customers in order toensure our financial success?-

Internal Business Process

Which of our business processes most im-

pact customer satisfaction?

-

Learning and Growth

What improvements can be made to ensure

sound business processes and satisfied customers?

Balanced Scorecard




97/137

97

Balanced Scorecard: visualization by the performance dashboardTemperature

HC

Fuel

FE

MaturityHow long has our organizationbeen on the road? How old isour measurement system? Is it

time for a check-up?

0 2 5 0 0 0

Odometer

Speed

110

RPM x 1000

70

Internal Business ProcessAre our internal business processesoperating efficiently and effectively?

In what gear are we operating?

FinancialDo we have the financial

resources and stability to reachour destination? Do we have the

backing of our stakeholders?

CustomerAre we addressing and meetingthe need of our customers? Arethey cold (not participating)? Are

they hot (complaining)?

Learning and GrowthAre we growing and improving at

a sustainable pace? Are wemoving too slow? Too fast?

Fig. 5.7. The Balanced Scorecard as a performance dashboard (Performance-

Based Management Special Interest Group 2001). With permission




98/137

98

The EFQM model can be used to assess an organizations progress to-wards excellence, independently of the organizations type, size,

structure, and maturity.

It is based on the premise that Excellence depends on the capacity ofconciliating the stakeholders

different exigencies and interests.

The model is based on nine criteria (dimensions). Five of these areclassified as Enablers

and four as Results. The Enabler

criteria

cover what an organization does; the Result

criteria cover what an

organization achieves. Feedback from Results

help to improve

Enablers.

EFQM method


99/137




100/137

100

3.1 Monotony

Let us consider a set of sub-indicators aggregated by a derived

indicator.

If the increase/decrease of (the performance of) one sub-indicator isnot associated to the increase/decrease of (the performance of) thederived indicator, then the derived indicator does not

fulfil

the condition

of monotony.

This definition implicitly entails that the symbolic manifestations of thesub-indicators are represented using a scale with (at least) order

relation.

3. Properties of derived indicators




101/137

101

3.1 Monotony

If a process is represented by different sub-indicators aggregated into aderived indicator (I

TOT

), and if the process skips from state S to state

S*, increasing/decreasing one sub-indicator Ik

, (not changing other

indicators

performance), then ITOT

should in-crease/decrease too.

Otherwise, ITOT

is not monotonous., then the derived indicator does not

fulfill the condition of monotony.


z1

z2

symbolic manifestations ofthe derived indicatorITOT

DERIVED Indicator

(ITOT)

S:(I1=x1, I2=x3, I3=x5)

S: (I1=x1, I2=x3, I3=x7)

manifestations of the sub-indicators aggregated byITOT

Fig. 4.20.Schematic representation of the condition of monotony. If process skips

from state Sto state S*, beingI1(S*)=I1(S), I2(S

*)=I2(S), and I3(S

*)>I3(S), then

the Monotony entails thatITOT(S*)>ITOT(S)

strictly

monotonic

weakly

monotonic

f(x)

x




102/137

102

3.1 Monotony

Example 4.24 The pollution level of the exhaust emissions of a motorvehicle is estimated using the following model:

The example shows that using a derived indicator which is notmonotonous, we may lose some information (according to , thereis no difference between state S and state S*).


10

' ' ' 'max , , ,X

A

TOT NO HC CO PM I I I I I

'

XNOI '

HCI '

COI 10'

PMI A

TOTI

state S 1 1 1 3 3

state S 2 3 2 3 3

ATOTI




103/137

103

3.2 Compensation

Given a derived (or aggregated) indicator,

- if the variation of a sub-indicator (always) makes the derived indicatorvalue change,-

and if there (always) exists another sub-indicator which variation may

cancel the variation in the derived indicator,

then the derived indicator fulfills the property of full compensation

and

a substitution rate

among sub-indicators can be calculated.





104/137

104

3.2 Compensation (in other words)

SI1

and SI2

are two sub-indicators

aggregated by a derived indicator D.

- If a variation SI1 (always) makes D change (D),-

and if there (always) exists a variation SI2

, so that D

is

compensated,

D

fulfills the property of full compensation,

and the function which relates SI1

and SI2

is defined as substitution

rate

between SI

1 and SI

2 .

Compensation is a typical property of additiveand multiplicativemodels.





105/137

105

3.2 Compensation

Example 4.25 The pollution level of motor vehicle exhaust emissions isestimated by :

As illustrated in the Table, the pollution level skips from state S to stateS*. The decreases of and are compensated by the increase of

value does not change.


10' ' ' '

4 X

NO HC CO PMB

TOTI I I II

XNO

I HCI COI 10PMI

BTOTI

state S 2 2 1 3 (2+2+1+3) / 4 = 2state S* 1 1 3 3 (1+1+3+3) / 4 = 2

XNO

IHC

ICO

I

BTOTI




106/137

106

3.2 Compensation

Substitution rate:

It is the portion x1 of one sub-indicator x1

that you should give up in

order to obtain an increase x2

in a second sub-indicator x2

, keeping the

aggregated indicator (y) constant.

Example: y = 2x1 + x2 + x3 x2

= -2x1

In general, if y is a continuous and derivable function with respect tothe sub-indicators, the substitution rate can be obtained from the

calculation of the infinitesimal increments (slope) of the indifferencecurve:


22 1

1

x2 x 2 x

x

x1

x2

x2

x1

y=constant




107/137

107

3.2 (Partial) Compensation

Example 4.24 The pollution level of the exhaust emissions of a motor

vehicle is estimated using the following model:

In case (b), no variation of other sub-indicators may compensate forthe resulting variation in the derived indicator.

A substitution rate cannot be defined univocally.


10

' ' ' 'max , , ,X

A

TOT NO HC CO PM I I I I I

'XNO

I 'HCI

'

COI 10'

PMI A

TOTI

state S 1 1 2 3 3

state S 4 1 2 3 4

'

XNOI '

HCI '

COI 10'

PMI A

TOTI

state S 1 1 2 3 3

state S 1 1 2 2 2

'

XNOI '

HCI '

COI 10'

PMI A

TOTI

state S 1 1 2 3 3

state S 4 1 2 3 4

(a) (b)




108/137

108

Two properties are introduced to help identifying representation

targets

which are consistent with the strategic objectives.

We underline that the properties are defined accessory because theyare helpful for testing process representation targets, rather thanindicators.

4.1 Long term goals:

Indicators should encourage the achievement of process

long-term

goals. Therefore, representation-targets should concern process

dimensions which are strictly linked to these goals.4.2 Impact on the Stakeholders/Customer orientation:

Many indicators focus on internal needs such as throughput, staffefficiency, cost reduction, and cycle time. While these needs are alllaudable, they usually have little direct impact on costumers needs. So,

it is important to identify process aspects with a strong impact oncustomer satisfaction.

4. Accessory properties




109/137

109

1. Replenishment lead time2. On-time performance3. Supply flexibility4. Delivery frequency5.

Supply quality

6. Inbound transportation cost

4.1 Long-term goals

Example: Supplier Scoring Assessment

a) Short-term indicator: focusing only on the quoted price.

b) Long-term indicator: it should take into account and aggregate

other important dimensions:

7. Pricing terms

8. Information coordinationcapability

9. Design collaboration capability

10. Exchange rates, taxes11. Supplier viability




110/137

110

Definition and testingof Indicators

Preliminary definition of indicators

Testing of the other indicators general properties

Indicators correctionor redefinition

Indicators validation

Do indicators fulfil theproperties? YESNO

Process identification

Representation-targetsidentification

Do representation-targetsfulfil the accessory

properties?YESNO

Correction ofrepresentation-targets

Check of the consistency with therepresentation-target (for each indicator)

Check of exhaustiveness and non redundancy

Check of the derived indicators properties

Several recursive steps arenecessary, before developing aproper model.




111/137

111

Measuring/evaluating the customer satisfactionis essential for Quality.

Understanding the customer behavior, attitude or perceptions is also

fundamental when designing a product/service.The fact that evaluations are very often subjective

does not imply that

they can be neglected.

they are studied by many disciplines in the area of Social, Behavioural

and

Cognitive Sciences.

The scientific literature shows plenty of techniques to design scales,

questionnaires, surveys, and analyze the relevant results (in a statisticallysound manner).

The importance of indicators for Quality




112/137

112

This activity is fundamental in marketing.

Understanding the attitude of customers is important to predict -

or

even influence! - their (future) behavior.From psychology:

-An attitudeis a residuum of experience, by which further activity is

conditioned and controlled. We may think of attitudes as acquiredtendencies to act in specific ways toward objects(Krueger & Reckless,1931).

-Attitude can be defined as readiness to respond to apsychological

objectwith some degree of (dis)favorableness(Thurstone, 1932).In general, the evaluative reaction of favor or disfavor can range from

extremely negative to extremely positive, through the neutral point, on a

dimension such as: good bad, pleasant unpleasant, or in favor opposed.

Measuring the Attitude




113/137

113

From psychology:

They are simply the things that psychologists take to be their properobjects (i.e., attributes, concepts) of investigation.

Examples of psychological objects: learning capability, motivation,intelligence, customer (un)satisfaction

with a service, your opinion

about smoking cigarettes, etc

Psychological Object

The attitude can be measured by generating items (i.e., statements orquestions) about the psychological object and analyzing theanswer/opinion

of the user.

basically, an indicator with a corresponding scale of measurement

How to measure the Attitude?




114/137

114

Psychological object:

Sarah

PalinOn a scale of 10 to +10, how would you rate Sarah Palin as a potentialPresident of US?

where:

10 = Strongly disapprove

+ 10 = Strongly approve

0 = Neutral; neither approve nor disapprove

On the same scale, how would you rate the Sarah Palins opinion onabortion?

General Question: How to define and organize these items properly?

Example of possible Items




115/137

115

Likert Scale (rating scale)

A Likert

scale is an ordinal scale format that asks respondents to

indicate the extent to which they agree or disagree with a series ofmental or behavioral belief statements about a given (psychological)object.

Example:

Scaling Techniques




116/137

116

Likert Scale (it continues)

-The analysis can be conducted on an item-by-item basis (profile

analysis), or a total (summated) score can be calculated.-When arriving at a total score, the categories assigned to the negativestatements by the respondents should be scored by reversing thescale.

Scaling Techniques




117/137

117

Semantic Differential scale (rating scale)

A semantic differential scale is unique bipolar ordinal scale format that

captures a persons attitudes and/or feelings about a given(psichological) object.Example:

Scaling Techniques




118/137

118

Semantic Differential scale (it continues)

-The negative adjective or phrase sometimes appears at the left side of

the scale and sometimes at the right.

This controls the tendency of some respondents

theory for performance indicators

Documents