an expert diagnosis system for the benchmarking of … · textual comments on practices and results...

Sylvain Delisle1 & Josée St-Pierre2

Institut de recherche sur les PMELaboratoire de recherche sur la performance des entreprises

Université du Québec à Trois Rivières1 : Département de mathématiques et d’informatique

2 : Département des sciences de la gestionTrois-Rivières, Québec, Canada

[email protected]

Business Excellence '03First International Conference on Performance Measures,

Benchmarking and Best Practices in New EconomyJune 10-13 2003, University of Minho

Guimaraes, Portugal

An Expert Diagnosis System for the Benchmarking of SMEs’ Performance

Disponible sur le site du Laboratoire de recherche sur la performance des entreprises (www.uqtr.ca/inrpme/larepe)Section Nos recherches / Sous-section Publications et communications

http://www.uqtr.ca/~delisle/ p.2 BEX 2003, Portugal, June 10-13, 2003

Benchmarking for SMEs: Why?

Facing increased competitive pressure due to globalization, as well as increased quality and service requirements from their customers, SME manufacturers SME manufacturers must increase their productivity and their must increase their productivity and their competitiveness in order to survive and prospercompetitiveness in order to survive and prosper.Benchmarking is a way to know how well a SME performs compared to others. Goal: compare one’s practices and Goal: compare one’s practices and results with those of other organizations operating in results with those of other organizations operating in a similar environment.a similar environment.Usefulness: identify possible sources of inefficiency in the firm in order to bring about appropriate corrective measures to insure optimal performance.Although fairly well-known in large enterprises, it is a strategic tool that remains relatively undersued in SMEs.


From SMEs to a Benchmarking Tool

SME: a unique enterprise, with limited resources, pursuing very diverse goals, subjected to the constraints of its environment ...Unicity of the firms and conviction of the impossibility of finding relevant information.Refusal of entrepreneurs to share perceived confidential strategic information.Difficulties in formulating strategic orientations beyond day-to-day activities.

So, how to approach SME benchmarking then!?Our answer: the PDG (Performance, Development, Growth) computerized benchmarking tool.www.pdg-sme.ca


The PDG System: Genesis and Goal

Developed by a multidisciplinary team Developed by a multidisciplinary team of researchers (in management, marketing, accounting, finance, human resources, engineering, innovation, information technology), SME entrepreneurs, and economic advisors.Has been in development / use / fine-tuning for 6 years. Most recent version has been in production for 2 years.Investment of 2M CAN$.The goal of the PDG system is to evaluate a SME, from an external perspective and on a comparative basis, in order to produce a diagnosis of its performance and potential, complemented with relevant recommendations.The PDG benefits to SME entrepreneurs, investors / lenders / bankers, and researchers (via its database).


The PDG System: A Bird’s Eye View

Entrepreneur(SME)

IntermediaryPartner

Lab

Oracle Database

PDG ExpertDiagnosisSystem

MultidisciplinaryTeam of Human

Expertsexpertise

information

ques

tionn

aire

,fin

anci

al d

ata

ques

tionn

aire

finan

cial

dat

a

repo

rt

repo

rt

repo

rt

data and results


The PDG’s Architecture(a “black-box” expert system)


The PDG System: Inputs & OutputsINPUT:preferred reference group + sections to be filled by owner-manager, human resources manager, controller or general manager, and production manager + complete financial statements of the last 5 years.Input data is computerized and fed into the PDG system. The evaluation is performed in 3 minutes by contrasting a particular SME with an appropriate reference group of SMEs for which we have already collected relevant data.OUTPUT (10-page report): 28 management practices(concerning human resources management, production systems and organization, market development activities, accounting, finance and control tools, effectiveness and vulnerability), 20 results indicators and 22 general information items are evaluated, leading to 14 recommendations on short term actions the evaluated SME could undertake to improve its overall performance.


An Excerpt from the Output Report

Management practices that Management practices that generate results (4 to 6 for generate results (4 to 6 for

each main business function)each main business function)

Results (2 to 5, depending on the Results (2 to 5, depending on the function) are commented upon and function) are commented upon and analyzed in relation to analyzed in relation to comparablecomparable

enterprises operating in a relatively enterprises operating in a relatively similar environmentsimilar environment

Comments and recommendations Comments and recommendations are formulated as to the practices are formulated as to the practices

adopted by theadopted by the SMEsSMEs in view of in view of their situation compared to the their situation compared to the

reference groupreference group


A Closer Look at some of the Results (1)

% of SMEsin the reference group

Evaluated SME

32%

Do you have a board of directors, a management committee or an advisory committee that meets more than twice a year?

Evaluated SME (green): yes.

Reference group (yellow): 32% said yes.

When production-related decisions are taken, production workers are generally: informed after, informed before, consulted, partners, or mandated?

Evaluated SME (red square): informed before.

Reference group: partners (average).



Certain financial informations, of historical nature, are presented differently.

40%

35%

30%

25%

20%

15%

10%

5%

0%

8.0%

7.0%

6.0%

5.0%

4.0%

3.0%

2.0%

1.0%

0.0%

Trend in profits

1996 1997 1998 1999

Net

mar

gin

Gro

ss m

argi

n

Marge brute - Entreprise Marge brute - GT

Marge nette - Entreprise Marge nette - GT

Evaluated SME Reference group



A one-page summary on each of the following topic is presented: human resources, production systems, production management, sales/marketing, management and control, effectiveness, and vulnerability. Illustrated with graphical representations.Example comment on practice evaluation of human resources: “Your human resources management practices are generally more advanced than those of your reference group. You could improve your performance, in particular by implementing participative management to involve employees in the growth of the business.”Example comment on human resources results: “Your overall effectiveness at managing human resources is comparable to that of your reference group. You should pay attention to why certain managerial jobs have a high rate of voluntary departure, with the objective of lowering hiring and training costs.”


The PDG’s 8 Main Steps of Processing (1)

1. The reference group is created according to a number of criteria usually after several iterations.

2. A number of statistical computations (averages, medians, distributions) related to the reference group are performed within the SAS statistical package, after data from the Oracle database has been fetched.

3. Statistical data is passed from SAS to Excel via an ODBC connection.

4. A bunch of automatic, numerical Excel computations take place at this stage in order to define the relative positioning of the evaluated SME to its reference group.

5. Graphics and various representations associated with the numerical values just computed are then produced.


The PDG’s 8 Main Steps of Processing (2)6. Textual comments on practices and results are

computed, in correlation with the computations completed in the previous steps.

7. An evaluation summary, to appear on the report’s first page, is produced for each of these seven main business functions:

i. Human resourcesii. Production systemsiii. Production managementiv. Sales / marketingv. Management and controlvi. Effectivenessvii. Vulnerability

8. The actual PDG output report, containing the benchmarking results, is then produced and saved on disk. The report is displayed on the screen by the Excel application.


The PDG System’s KnowledgeA good deal of multimulti--domain expertise and informal domain expertise and informal knowledge engineeringknowledge engineering was invested into the design of the PDG’s extended questionnaire, as well as into the benchmarking system. The current version of the PDG system, although not implemented with AI techniques (e.g. knowledge base of rules and facts, inference engine) qualifies as a qualifies as a ““blackblack--boxbox”” expert systemexpert system. A prototype expert system was developed for a subset of the PDG dealing only with human resources.The knowledge, information and algorithms it uses allow it to produce outputs that only several human experts in different domains would be able to produce in terms of diagnosis quality, including recommendations.Over 90% of the reports produced by the PDG system need no modification whatsoever before being returned to the evaluated SME.


The SMEs in the PDG’s DatabaseSo far, we have produced more than 600 reports and accumulated in the PDG’s database the evaluation results of 400 different manufacturing SMEs.These manufacturing SMEs:

Are mostly located throughout the province of Québec; Have between 20 and 400 employees;Have between 0.5 to 90M CAN$ in sales;Have been in business for 5 to 125 years;Are of all kinds: small, large, performing, in difficulty, etc.

Other SMEs are from the province of Ontario (Canada) and the USA. Recently, a deal was signed with a French businesspersons group for the production of 100 PDG reports per year.


Research and the PDG: An ExampleA study was made of 307 Canadian manufacturing SMEsthat have used the PDG report, including 49 that have done so more than once.The aim of that study was to show the importance of benchmarking for manufacturing SMEs in the new business environment and to assess the utility of the PDG report (and, implicitly, of the PDG system) to help businesses to identify management practices that needed to be adjusted.Our results show that this activity allows these organisations to improve their operational performance confirming the usefulness of benchmarking but also, the value of the recommendations included in the PDG report concerning short-term actions to be undertaken to modify management practices.


Conclusion

The PDG proves that if the benchmarking approach is tailored to SMEs’ characteristics, an adequate tool can be devised and used to help SMEs increase their performance.Benchmarking allows entrepreneurs to break the isolation in which they may be caught by comparing and positioning their SME to others (Monkhouse, 1995). The use of benchmarking first influences operational performance. It also influences the adoption of best management and production practices.Identification of the combinations of the most effective «good practices», adapted to the reality of SMEs.


Current & Future Work

We will very soon start feeding the PDG system with “international” data, i.e. with data coming from SMEslocated in Europe (or other countries).We are starting a migration to more efficient Web-based technologies (Java), thus allowing various improvements and making a further step toward a fully interactive online version of the PDG system.We are conducting an in-depth revision of the current PDG system in order to reengineer the system’s architecture from a data warehousing and data mining perspective, still within the context of an Oracle database.


A s s is ta n t

M o d u le 2A p p lic a t io n w e b

d e s q u e s t io n n a ire s

B a s e d e d o n n é e sd e s q u e s tio n n a ire s

S a is ie d 'u nq u e s t io n n a ire

S a u v e g a rd e d u q u e s t io n n a ire

C h a rg e m e n td e l 'e n tre p ô t

M o d u le 3A p p lic a t io n d e

c a lc u l s ta t is t iq u ee t d e d a ta m in in g

E x tra c t io n d e d o n n é e s

M o d u le 4A p p lic a t io n d ep ro d u c t io n d e s

ra p p o r ts

E x tra c t io n d e d o n n é e s

U tilis a t io n d e c a lc u lss ta t is t iq u e s e x te rn e s

B a s e d e d o n n é e sd e s ré s u lta ts

S a u v e g a rd ed u ra p p o r t

M o d u le 5A p p lic a t io n d e

g e s t io n d e sd o s s ie rs

A ffic h a g e e tm o d if ic a tio nd e s d o s s ie rs

R a p p o r ts l ié sa u x d o s s ie rs

A lg o r ith m e s d etra n s fo rm a tio nE n tre p r is e

A s s is ta n t

C h e rc h e u r / s ta t is t ic ie n

E x tra c tio n d e d o n n é e s

U til is a t io n d es ta t is t iq u e s e t

d a ta m in in g

A ff ic h a g e e tc o rre c tio n d u

ra p p o r tIm p re s s io n d u ra p p o r t

U tilis a t io n d uc o d e d e

l'e n tre p ô t

P ro fe s s io n n e l

A d m in is tra t io n d e l 'e n tre p ô t,d ic t io n n a ire d e v a r ia b le s ,

s é c u r ité , c h a rg e m e n t, e tc .

M o d u le 1E n tre p ô t

A s s is ta n t /P ro fe s s io n n e l

L é g e n d e : A p p lic a t io n P o in t d 'a c c è s a u s y s tè m e

B a s e d e d o n n é e s A lg o r ith m e s d 'u n e a p p lic a t io n

D o n n é e s

an expert diagnosis system for the benchmarking of … · textual comments on practices and results...

Documents