2008-ica_kpi reporting in the public sector

KPI REPORTING IN THE PUBLIC SECTOR

Keith T Linard1

Director, Ankie Consulting Pty Ltd GREENDALE VICTORIA 3341 AUSTRALIA

Tel: 0412-376-317 E-mail : linard.keith#@#gmail.com

(Remove ‘#’ for email)

ABSTRACT:

Performance indicators are key feedback drivers of organisation dynamics. However, the evaluation literature gives no scientific basis for their selection or validation. An even more significant omission in the literature is the validation of “business rules” associated with such indicators. It is presumed that managers instinctively know the “right way” to respond to indicator change. Research shows that managers typically misread delayed feedback effects and their decisions are correspondingly inappropriate. The balanced scorecard suffers from the above limitations, especially the feedback interrelationships between indicators.

Recent reports by the Commonwealth Auditor General raise concerns regarding agency performance management systems which echo concerns going back to the 1980’s. It is apparent that there remains a lack of understanding of performance indicators as well as evidence of loss of corporate knowledge.

In particular, a review of federal, state and local government guidelines, with the exception of NSW and Queensland, and a review of federal agencies, with some outstanding exceptions such as AusAID and various NSW agencies, have little conception of the critical importance of cause-and-effect, or program logic, in the development of their KPIs.

This paper discusses development of a “dynamic balanced scorecard” based on the system dynamics paradigm, which accounts for delayed feedback effects. The paper outlines the modelling of causal interactions within an organisation that impact on output quality.

Keywords: system dynamics; balanced scorecard; BSC; performance indicators ; KPI; performance management; systems thinking; cognitive mapping; effectiveness audit.

THE HEALTH OF PERFORMANCE MANAGEMENT IN THE PUBLIC SECTOR

Performance Management has improved & is still improving

First, there is no doubt that there have been dramatic improvements in public sector performance management over the past 2 decades or so since the Hawke-Keating Labor Government initiated a dramatic objectives-oriented shake up of the federal bureaucracy with its Financial Management Improvement Program (FMIP), and corresponding action by State Governments.

My comments and criticisms in this paper should be seen as a desire to lift the game even further rather than a condemnation of the ongoing efforts of so many good people.

Where have we come from?

Management attitudes and management systems have come a dramatic distance since 1980’s.

1984 FMIP survey of every senior executive in the federal bureaucracy revealed that 94% saw as their primary role to ensure that their budget was spent and that their staff numbers were at the authorised levels. Few saw as a high priority the achievement of program objectives.

1 Keith Linard is former Director of the University of New South Wales Centre for Business Dynamics & Knowledge

Management and a former Chief Finance Officer with the federal department of Finance.

Institute of Chartered Accountants – ACT Business Forum May 2008 KPI Reporting in the Public Sector

- 2 -

A 1985 FMIP survey of the Deputy Heads of 55 federal government departments and statutory authorities found that fewer than 70% included achievement of objectives amongst their top seven critical success factors!

the 1985 FMIP survey of management information systems across the federal public sector found that "no agencies have executive information systems which permit integration of financial and staffing data . . . with data on outputs."

Today, there would virtually be no public official, at any level, who has not been exposed to the rhetoric of performance achievement or who, even at a subconscious level, is not oblivious to organisation objectives. Thus, for example, in 2004, the federal Auditor General found2

All 63 APS agencies included in the audit included performance management as a specified goal in their planning documents and almost all included it in their certified agreements.

A focus on performance management and improvement has been integral to successive public service reforms and has been widely supported [by staff].

But still a way to go

A review of federal and state auditor general or program evaluation reports, however, continue to identify shortcomings in performance management:

Poorly defined or irrelevant KPIs

Poorly defined KPIs, leading to inconsistent interpretation throughout the business

KPIs that are not relevant to the business mission, objectives or stakeholder requirements

Multiple KPIs measuring very similar or highly-correlated things

KPIs that are outside of the control or influence of the business

No causal relationships in KPIs

Frameworks dominated by financial measures

A predominance of lagging measures and / or not enough leading measures

An inappropriate balance between measures of ‘process’ vs. ‘outcomes’

Inconsistent vertical application of KPIs:

Inappropriate level of detail: too much detail in high-level KPIs, or not enough in low-level KPIs

KPIs that are not cascaded below corporate level, or measures below corporate level not aligned with Corporate measures

No link between performance measurement and management:

KPIs used solely for reporting with no link to decision-making

No links between KPIs and individual Performance or Development Plans

Incentive based pay absent or not linked to KPIs

Incentive based pay that relies on a single measure, relies solely on financial measures or relies on measures outside the individual’s influence

For example, the 2004 Performance Audit of the federal Auditor-General ‘Performance Management in the Australian Public Service’ found:3

Many agencies do not have established systems that relate to, and support, their performance management systems.

2 The Auditor-General. Audit Report No.6 2004–05 Performance Audit - Performance Management in the Australian

Public Service. ANAO, Canberra. 2004. 3 The Auditor-General. Audit Report No.6 2004–05, op. cit..


- 3 -

Little attempt has been made by agencies to assess the organisational impact of their performance management system.

It is difficult for agencies to establish a strong link between the conduct of their performance management systems and improved performance of their organisations.

Up to one-third of responding agencies do not take staff adherence to the APS Values, as well as the agency’s own value systems, into account when assessing performance.

A significant percentage of salary advancement payments, and even bonuses, are unlikely to reflect ‘superior’ performance

KPIS – AREAS FOR MAJOR IMPROVEMENT & A PRACTICAL WAY FORWARD

In the remainder of this paper I outline my perceptions, based on 30 years intimate involvement in performance management, of the continuing deficiencies in the performance management paradigm, and I propose a practical way forward. I emphasise that I am not focusing on management attitudes or on performance reporting tools, but rather on the fundamental paradigm itself. What should KPIs be about, how should we select them and how can we validate them. The shortcomings I will focus on are:

Cause-and-effect’ chain;

Feedback and delay in process logic;

Basis for selecting performance indicators;

Business rules – ignored dimension of KPIs.

The tools I will propose as essential to address these shortcomings are:

Systems thinking;

System dynamics modelling; and

Dynamic Flight Simulator.

I will build this case around the concept of the Balanced Scorecard which, although not widely implemented in the Australian public sector, has all the elements of current ‘triple bottom line’ performance reporting. Accordingly, I ask you to suspend any negative thoughts about BSC and focus on the fundamental issues I address.

THE FUNDAMENTAL VALUE ADDED OF THE BALANCED SCORECARD

The Balanced Scorecard is an outcomes oriented performance management system that seeks to link the short and long term activities of an organisation to its vision, mission, and strategy through the establishment measurable, consensus-driven goals and the development of linked cause-and-effect performance indicators.

The Balanced Scorecard is more than performance measurement system. It is a management system that seeks to focus the efforts of staff throughout the organisation towards achieving strategic objectives. It seeks to give feedback on current performance so that managers will adapt their priorities towards value improvement and make decisions that give effect to this. It seeks to give feedback on the key drivers of longer term future performance, so that when the future arrives it has been long anticipated in managerial priorities and decision making.

The 1997 Management Advisory Board (MAB) report ‘Beyond Bean Counting’ recommended the Balanced Scorecard as

“ … a valuable tool for organisations in both the public and the private sectors that wish to drive a process of strategic change. It can provide a focus through a performance measurement system that balances the financial and the non-financial indicators of performance. It also harnesses the strategic vision of the organisation in a holistic view of the key drivers of success, and closely


- 4 -

integrates the strategic vision with the operational planning and budgeting activities of the organisation.”4

The MAB went on to say that the balanced scorecard had a number of potential benefits for APS departments and agencies:

it provides managers with complex information at a glance;

it ensures managers are managing all the important variables – not just the easy ones;

it balances time perspectives – current performance as well as the drivers of future performance; and

it prevents information overload by limiting the number of measures used.

A Good Balanced Scorecard

From the writings of Kaplan and Norton, and from the prolific marketing of Balanced Scorecard solutions by management consultants and software vendors, we know that a “good” BSC is not simply a limited list of measures gathered into four categories. Rather, a “good” Balanced Scorecard “… should tell the story of your strategy”5, communicating and promoting adherence to the strategy to all levels of the organisation.

Linked to strategy

Perhaps the single most important contribution of Kaplan and Norton has been to get the message across that the setting of the performance framework is a crucial corporate responsibility in which the direct and ongoing involvement of top executives is essential:

in setting the vision, and the strategy for achieving it

in implementing a performance measurement system focussed on strategy

in communicating down the line that executives are using the system.

“In essence, the strategies are a series of cause and effect relationships. If you want to achieve X, what needs to happen to cause that result? The performance measures would then track the trend of the organisation's progress toward the desired strategies outcome”. 6

A good Balanced Scorecard, says Kaplan, “… tells everyone in your organization, in a single page, the story of your entire strategy: Every measure is part of a chain of cause-and-effect linkages. All measures eventually link to organizational outcomes. A balance exists between outcome measures (financial and customer) and performance drivers (value proposition, internal processes, learning & growth)”.7

Based on cause-and-effect relationships

A good Balanced Scorecard will reflect the vertical cause-and-effect relationships for any given objective and summary measure, in the same way that the causal relationships are reflected horizontally across the business value chain by the four perspectives. In other words, having delineated the causal relationship between the Learning/Innovative perspective, the Internal Business Process Perspective, the Customer Perspective and the Financial Perspective we take each summary objective/measure and disaggregate them to determine causation.8

4 Commonwealth of Australia, Management Advisory Board, “Beyond Bean Counting - Effective Financial

Management in the APS - 1998 & Beyond”. PSMPC, Dec 1997. p.54. 5 Renaissance Worldwide Strategy Group. “The Balanced Scorecard -- An Overview”

http://www.rens.com/viewpoint/papers/scorecard.html 6 Schmid J. The Balanced Scorecard. FCN Reinvention News. 22/5/2000. 7 Kaplan RS, "The Balanced Scorecard", July 13, 1999. http://www.mastersforum.com/kaplan/kaplan.txt 8 Jamesford Consulting, “The Balanced Scorecard - A Strategic Management System”,

http://www.jamesford.co.uk/the.htm


- 5 -

“Every measure selected for a Balanced Scorecard should be part of a chain of cause-and-effect relationships that represent the strategy”.9

An integral aspect of the cause-and-effect chain, and one to which I will return in discussing adapting the Balanced Scorecard better to the public sectored environment, is the value chain.

“The value chain is a model that describes a series of value-adding activities connecting a company’s supply side (raw materials, inbound logistics, and production processes) with its demand side (outbound logistics, marketing, and sales). By analysing the stages of a value chain, managers have been able to redesign their internal and external processes to improve efficiency and effectiveness”. 10

Performance Drivers and Outcomes

A good Balanced Scorecard should have a mix of performance drivers (leading indicators) and outcome indicators (lagging indicators) of the organisation’s strategy - a linked set of measures that define the long term objectives and the mechanisms for achievement. Outcome measures without performance drivers do not communicate how the outcomes are to be achieved. They also do not provide an early indication about whether the strategy is being implemented successfully. Conversely, performance drivers without outcome measures may enable the business to achieve short-term operational improvements, but fail to reveal whether the operational improvements have been translated into outcomes achievement.11 The drivers of performance ("lead indicators") tend to be unique because they reflect what is different about the strategy.

Linked To Financials

A good Balanced Scorecard should (at least in the private sector) link strategy with the desired financial outcomes. With the proliferation of change programs underway in most organisations today, it is easy to become preoccupied with a goal such as quality, customer satisfaction or innovation. While these goals are frequently strategic, they also must translate into measures that are ultimately linked to financial indicators.12 The fundamental goals of a public sector organisation will be different (this is discussed later in this paper) but the principle remains valid, that strategy should link with the fundamental goals.

CAUSE & EFFECT CHAIN - THE CRUCIAL ELEMENT IN THE BSC

The criticality of cause-and-effect to a good Balanced Scorecard is a constant refrain of Kaplan and Norton. However, few practitioners (managers, consultants or software vendors) appear to have comprehended the full import of their message. In part this is due to some ambiguity on Kaplan and Norton’s part, as will be discussed shortly.

Is cause-and-effect really important?

Arie de Geus, as head of strategic planning of Royal Dutch Shell through the turbulent decades of the 1970’s and 1980’s, successfully steered the company through a systemic strategy driven approach. The research underpinning his approach demonstrated there is something dramatically wrong with corporate business survival rates13.

“A full one-third of the (Fortune '500' list of industrials) listed in 1970 had vanished by 1983 … The demographics of companies, their birth and death rates, seem to indicate that their average life expectancy is no more than 40-50 years. This finding seems to be valid in countries as wide apart as the USA, Europe and Japan.”

9 Ibid. 10 Rayport JF and Sviokla JJ. Exploiting the virtual value chain. Harvard Business Review. Nov-Dec 1995. 11 Renaissance Worldwide Strategy Group. “The Balanced Scorecard -- An Overview”

http://www.rens.com/viewpoint/papers/scorecard.html 12 Ibid. 13 De Geus, A. Strategy and Learning. Rotterdam School of Management. 1996


- 6 -

De Geus argues persuasively that attention to systemic issues (loosely, cause-and-effect relationships), and in particular to ‘lead indicators’ through use of business ‘flight simulators’ is critical to survival: “we will not perceive a signal from the outside world, unless it is relevant for an option for the future which we have already worked out”.

De Geus’ conclusions parallel the findings, from a different standpoint, of Kaplan and Norton whose research into company success and failure underpins their design of the Balanced Scorecard, and especially the cause-and-effect focus.

Cause-and-effect according to Kaplan and Norton

In article after article and example after example, Kaplan and Norton emphasise that the chain of cause-and-effect should pervade all four perspectives of the Balanced Scorecard and should include all key performance drivers that ultimately impact on strategy.

“Our experience is that the best Balanced Scorecards are much more than collections of critical indicators or key success factors organized into several different perspectives. The multiple measures on a properly constructed Balanced Scorecard should consist of a linked series of objectives and measures that are both consistent and mutually reinforcing. The metaphor should be a flight simulator, not a dashboard of instrument dials. Like a flight simulator, the scorecard should incorporate the complex set of cause-and-effect relationships among critical variables, including leads, lags, and feedback loops that describe the trajectory, the flight plan, of the strategy.” 14 (Emphasis added)

Kaplan and Norton argue that, by having an explicit set of linkages among Balanced Scorecard measures, managers can test the organisation’s hypothesised causal chain of performance drivers and outcomes and can learn how different business rules impact on organisation performance. Whilst acknowledging that many such hypothesised linkages will be subjective and qualitative, the end point would be “… the Balanced Scorecard … captured in a system dynamics model that provides a comprehensive, quantified model of a business’s value creation process.” 15 They go on to clarify what they mean by this, referring to the system dynamics paradigm developed by Jay Forrester at MIT, quoting various system dynamics references.

“In effect, the causal and dynamic relationships in a Balanced Scorecard can be modelled with a system dynamics approach.” 16

Unfortunately, few practitioners appear to have heard this, and fewer still appreciated its import.

Cause-and effect in the general Balanced Scorecard literature

A Google search of “balanced scorecard” will identify over 7 million ‘hits’. Digging deeper will find about 50,000 of these make some reference to cause-and-effect, which is staggering since the whole concept of the BSC is related to cause-and-effect. Further analysis will reveal that fewer than 5,000 documents, many of them from academia or software vendors, show any depth of understanding about cause-and-effect. A laborious review of several hundred US federal, state and municipal government reports on performance indicators and the BSC revealed almost total lack of substance regarding cause-and-effect indicators.

Review of federal and state public sector guides on performance management will typically include some reference to causality, but rarely offer guidance on how to address this. For example, the 2004 West Australian guidelines on “Outcomes Based Management” simply state: “Effectiveness indicators will generally be derived from some characteristic of the outcome and they should be

14 Kaplan, RS and Norton DP, “Linking the Balanced Scorecard to Strategy”, California Management Review, Vol. 39,

No. 1, Fall 1996. p.64. 15 Ibid., p. 67. 16 Ibid., p. 79.


- 7 -

designed to identify as clearly as possible the causal relationship between the service(s) and the outcome.”17 AusAID is shining exception.

Cause and effect in Australian public sector guidelines

Program logic analysis may be defined as the systematic study of the presumed relationships between political & regulatory environment, program resource inputs, ongoing program processes / activities, short term outputs, longer term results, and program objectives.

The 1984, Financial Management Improvement Program (FMIP) Handbook, Evaluating Government Programs18, proposed a structured logic analysis framework for planning and implementing program evaluations. The program logic framework was further developed by various State Governments, but especially NSW, over the intervening years.

A program logic framework remains significant in various federal departmental evaluation manuals and in the reports and guidelines of the Australian National Audit Office19 and the Federal Department of Finance and Administration20.

Figure 1 depicts the 2000 Department of Finance and Administration presentation of such a program logic framework.

Figure 1: Causal Logic Model for Government Program (DOFA 2000)

The logic model “…provides for the use of output groups that can strengthen the strategic and causal connections between each level ... The process of developing and analysing the underlying logic of programs … is a powerful mechanism for identifying the key areas and issues within a program, particularly in relation to outcomes, and hence enables the development of useful performance information”. 21

17 Department of Treasury and Finance, Government of Western Australia: Outcome Based Management - Guidelines

for use in the Western Australian Public Sector. 2004 18 Department of Finance. Evaluating Government Programs - A Handbook. AGPS, Canberra, 1987. pp.13-15. 19 ANAO. Better Practice Guide Performance Information Principles. 01/11/1996 20 Department of Finance & Administration. The Outcomes & Outputs Framework Guidance Document. Nov 2000.

NOTE: The Department changed its name to Department of Finance and Deregulation in Dec 2007. 21 ANAO (1996). Op. cit.


- 8 -

The NSW public sector has emphasised a program logic framework for over 20 years, with the ‘outcomes hierarchy’ a key tool. The outcomes hierarchy consists of an overarching goal, underneath which are a set of desired results which contribute causally to the goal. Underneath each desired result is a set of intermediate outcomes, the achievement of which is presumed to bead to the result. At the bottom of the chart are the specific services delivered to achieve each intermediate outcome. Arrows indicate which services contribute to which intermediate results and so forth. This is shown conceptually in Figure 2.22

Figure 2: Program Logic – Outcomes Hierarchy – From Services to Goals

Figure 3 shows this tool put into effect by a rural local government body in NSW.23 The analysis seeks to identify the presumed causal linkages between services provided by the Shire, the intermediate outputs from those services and their contribution to achieving the corporate goals.

Figure 3: Program Logic – Outcomes Hierarchy – From Services to Goals

22 NSW Council on the Cost and Quality of Government. Developing Whole of Government Performance Indicators.

2005. 23 Independent Inquiry into the Financial Sustainability of NSW Local Government. Developing whole of sector

performance indicators to improve the accountability, efficiency and effectiveness of local. Dec 2005


- 9 -

Why Logic Analysis ?

The purpose in undertaking a logic analysis is to describe clearly the intended or assumed processes by which a service or program is expected to accomplish its objectives. This presumes that the service or program can be considered, and represented, as a causal system, as illustrated in its simplest form in Figure 4.

Figure 4: Essence of Program Logic - Causal Chain Leading from Inputs to Outcomes

When presented in this blunt fashion it begs for the question to be asked: “How valid are these ‘presumptions’

Most government program activities are, of course, very complex: there are often many interactions with other programs and with the external social and political environment; the 'causal chain' from program activities to expected results is often unclear; and the mapping from outputs to outcomes or objectives is often problematic. Logic analysis inevitably involves simplification of the real world. Nevertheless, it imparts a salutary rigour to program design, implementation and evaluation by putting attention of those assumptions which may not hold up in the real world as illustrated in Figure 5.

Figure 5: Logic Analysis Helps Identify Breaks & Dispersion in the Causal Assumptions

This diagram highlights the risk of breaks or dispersion in the logic chain::

what are the key assumptions underlying the program and how will program success be affected if they are not valid;

what tests or measures might be appropriate to check whether these assumptions are sound;

what are the most sensitive or significant variables and what monitoring and control measures can we implement to rectify problems as they occur;


- 10 -

what changes to program design, program management or program operations would improve the likelihood of achieving the desired objectives;

how do various program element activities relate to each other, and how does performance in one affect the other activities;

what aspects of the program are likely to be affected significantly by other programs or by factors outside of the program manager's control; and

what are the significant unintended impacts of the program.

Flawed understanding of cause-and-effect

However, both the Kaplan & Norton BSC framework, and the logic framework and outcomes hierarchy used in the Australian public sector suffer from a flawed understanding of cause-and-effect.

‘cause-and-effect’ necessarily involves a time lag between the cause and the resulting effect, however the BSC and Logic Framework KPIs present the cause and effect indicators at the same time with no analytical basis to ascertain the implications of the time lag;

the almost universal presentation of cause-and-effect chain is a uni-directional causality which totally ignores feedback, and especially delayed feedback effects, both within and between the sectors of the Scorecard.

Time Dimensions of the Scorecard

The Balanced Scorecard presents a static snapshot of the performance drivers, ignoring the fact that the effects of the measures will occur at different points in time. Thus whilst streamlining the processing of claims may yield more satisfied clients within a matter of months and is readily measured, investment in research and development may take years to produce a result. The lack of apparent impact of the latter on the scorecard may lead to poorly advised decisions.

The significance of ignoring the time dimension, in relation to cause-and-effect, can readily be illustrated from the project management sphere where, regardless of the nature of a project, managers always need to know where the project stands in time and cost as measured by the original project schedule, and the approved project budget. Together these constitute the project baseline.24

Figure 6: Implications of Differences in Reported & Actual Schedule and Cost Variance

When cost variance (CV) and schedule variance (SV) are plotted on the same graph, a ‘bulls eye’ diagram is created. Referring to Figure 6, a result plotted at the centre (the intersection of the CV and

24 This example is based on research at the UNSW Centre for Business Dynamics and Knowledge Management by Dr

Alan McLucas.


- 11 -

SV axes) is on schedule and on cost; a result in the top-right quadrant would be ahead of schedule and under cost; and so on. This simple report shows not only project status, but how that status is trending. On the surface its appears to address timing issues.

But, like the Balanced Scorecard, this reporting system aggregates snapshot data from different time periods. It is readily demonstrated that when delays in data collection and reporting occur, and current information is aggregated with information one or two months old, CV and SV as reported bear little resemblance to actual progress. This is depicted in Figure 6 (a Defence project) where the reported CV and SV (suggesting the project is on-time but slightly behind on cost yet trending in the right direction) are compared to the actual CVact and SVact (which show a totally different and somewhat alarming picture).

More generally, the dangers associated with ignoring the time dimension relating to cause-and-effect chains are illustrated by the periodic boom-and-bust in the property market. The market is robust; returns are going up; it is a good time to invest in new development. Of course, the development process takes time. In the meantime returns continue to improve, encouraging even more developers to enter the market. The early developments come on stream and sop up the demand. When later developments enter the market, demand has already crashed. Investors and developers alike are hurt and development comes to a stand-still. Eventually the surplus stock is taken up, there are shortages in supply and prices start to climb. The market is robust; returns are going up … and we are back on the roller-coaster.

We are living with the results of a similar dynamic at the moment as a result of the boom and bust in the sub-primes marketplace.

Interrelationship amongst performance drivers - uni-directional causality

In most of their writings, the relationship between measures on the Balanced Scorecard is ambiguously described by Kaplan and Norton. Every example presented and every diagram describing cause-and-effect present or imply a unidirectional impact, as illustrated in Figure 7. This is also the typical case with diagrammatic representations of the logic framework or outcomes hierarchy of government programs, as is evident from Figures 1 to 3..

Figure 7: Example from Kaplan & Norton - Unidirectional Cause & Effect Chain Supporting Strategy ( Kaplan and Norton, 1996)

Increase Customer

Confidence in our Financial

Advice

Increase Customer

Satisfaction through Superior

Execution

UnderstandCustomerSegments

Develop New

Products

Cross-Sell the Product

Line

Minimize Problems

Shift to Appropriate

Channel

Provide Rapid

Response

Align Personal

Goals

Develop Strategic

Skills

Improve Operating Efficiency

Broaden Revenue Mix

FINANCIALPERSPECTIVE

INTERNALPERSPECTIVE

LEARNINGPERSPECTIVE

CUSTOMERPERSPECTIVE

The Revenue Growth Strategy The Productivity Strategy"Reduce the volatility of earnings by broadening the sources of revenue from current customers"

"Improve operating efficiency by shifting customers to more cost-effective channels of distribution"

Improve Returns

Increase Employee Satisfaction

Access to Strategic

Information


- 12 -

Figure 7 assumes that ‘Employee Satisfaction’ will not be affected by changes in work processes (internal perspective), changes which might involve staff reductions, reskilling, higher intensity work, etc. It assumes that there is no feedback from success in the customer sector to ‘Employee Satisfaction’. Arguably, the (ignored) feedbacks from the ‘Internal’ and ‘Customer’ perspectives would have greater impact on employee satisfaction than the rather rarefied ‘causes’ suggested in the Learning perspective. Figure 7 also ignores the obvious feedback link from financial returns which provides the capacity to invest in learning and growth or reengineering of internal processes.

Misperceptions of delayed feedback dynamics- the critical issue

An implicit and fundamental assumption behind the Australian public sector approach to selecting performance indicators is that the feedback they give to the decision maker (either directly or via resultant pressure or direction from others) will ‘cause’ the decision maker to make appropriate adjustments to the inputs or processes.

However, there is abundant research in the field of system dynamics25, as well as in the fields of experimental economics and psychology which suggest that managers have great difficulty managing dynamically complex tasks. Sterman argues persuasively from his work at MIT’s Sloan School of Management, that there is “systematic misperception of feedback” especially when there are delays in the system. Mosekilde, Larsen and Sterman26 present the results of 48 simulations of the “Beer Game”27 (a simulation of a simple factory-warehouse-retail system) run with 192 graduate students from MIT and senior executives of major US firms. They show that decision making on the basis of straight forward performance indicators, but in the face of delays, consistently resulted in costs averaging more than 10 times the optimum!

Simulations run by the author over 14 years at UNSW involving undergraduate and graduate students as well as many senior state and federal public sector managers showed a similar pattern. In both the MIT simulations and those at UNSW, highly educated managers and students failed to comprehend the significance of feedback in the face of delay induced dynamics.

In other experiments at MIT, where graduate students had full information, training, incentives and opportunities for gaining experience, Diehl and Sterman still found poor managerial performance in the face of variations in feedback strength and delay.28 Often they found the subjects were outperformed by a simple ‘no-control’ rule. Diehl and Sterman argue that the mental constructs and heuristics that managers bring to bear on complex tasks are fundamentally dynamically deficient:

“Subjects were unable to account well for delays and feedback effects because (1) people’s mental representations of complex tasks are highly simplified, tending to exclude side effects, feedback processes, delays, and other elements of dynamic complexity; and (2) even when these elements are known, people’s ability to infer correctly the behaviour of even simple feedback systems is poor.”

The first deficiency can certainly be addressed through training. The second, however, “... is a fundamental bound on human rationality - our cognitive capabilities do not include the ability to solve systems of high-order non-linear differential equations intuitively.”

25 Sterman, J. Modelling Managerial Behaviour: Misperceptions of Feedback in a Dynamic Decision Making

Experiment. Management Science, 1989, 35(3), 321-339.

Smith, V, G Suchanek and A Williams. Bubbles, Crashes and Endogenous Expectations in Experimental Spot Asset Markets, Econometrica, 1988, 56(5), 1119-1152.

Funke, J, Solving Complex Problems: Exploration and Control of Complex Systems, in R Sternberg and P Frensch (eds.), Complex Problem Solving: Principles and Mechanisms. Erlbaum Assoc., New Jersey, 1991.

26 Mosekilde, E, E Larsen and J Sterman. Coping With Complexity: Deterministic Chaos in Human Decision making Behaviour, in J Casti and A Karlqvist (eds.), Beyond Belief: Randomness, Prediction and Exploration in Science. CRC Press, Boston, 1990.

27 The Beer Game is described in detail in Senge, P, The Fifth Discipline - The Art and Practice of the Learning Organization. Doubleday, New York, 1990

28 Diehl, E and J Sterman. Effects of Feedback Complexity on Dynamic Decision Making. MIT Sloan School of Management, Research Report D-4401-1. March 1994.


- 13 -

Simulating the Beer Game

The so-called ‘Beer Game’ (more precisely an inventory management game) is a classic management game developed at MIT to simulate decision making in the face of dynamic processes, testing the systems principle that "structure influences behaviour. The scenario for the MIT game is depicted in Figure 8. Each team comprises 4 sets of decision makers - an ordering executive and a supply executive for each of the retail, wholesale, distribution and factory sectors. The aim of each team is to minimise total inventory costs and stockout costs, in the face of delays, as indicated in the diagram, and of demand uncertainty. As noted earlier, even with such a simple system senior executives and MBA students are unable to account for the dynamic interactions.

Figure 8: System simulated in the 'Beer Game'

In this game, the customers initially purchase 4 items per day. At, say, week 10, the orders jump to 6 items per day and thereafter remain constant. If the management team applies a simplistic “Only order what you have sold” heuristic, the stock pattern will look similar to Figure 9. The decline in inventory occurs because there is a two week delay before the higher level of sales (which deplete inventory) are translated into a higher level of orders (which replenish inventory).

Figure 9: Beer Game – Stock Pattern Over Time without KPIs

The putative performance management panels are now challenged to implement performance indicators and related decision rules which will force inventory to tend towards target. Group brainstorming with senior managers and graduate student groups (reminiscent of public service performance indicator workshops) inevitably results in decision rules which, as we see in Figure 10, lead to disaster. The players almost invariably set a single performance indicator (Target_Inventory less Actual_Inventory) and set the decision rule: order what was sold plus the performance gap.

In the face of a simple perturbation (a once-off increase in sales) and a short delay, this intuitive indicator sets up chaotic fluctuations. The mix of performance indicators, and the related decision


- 14 -

algorithms required to force the system to stability at the target inventory is by no means intuitive, notwithstanding the problem simplicity.

Figure 10: Beer Game – Stock Pattern Over Time with KPIs

CHOOSING INDICATORS AND VALIDATING BUSINESS RULES

A fundamental purpose of performance indicators is to give decision makers feedback on program operations in order to guide future decisions. In order to be confident regarding decisions made consequent on such feedback, however, two key conditions must be satisfied:

there should be a rigorously basis, beyond mere consensus or availability, for selecting indicators); and

there should be a way of developing and validating the business rules appropriate to ‘off-trend’ movements in these decision criteria.

Unfortunately the Balanced Scorecard literature is singularly silent on these points.

Basis for choosing indicators

I’ve looked at hundreds of Australian public sector documents seeking guidance on the selection and specification of KPIs. Unfortunately all one gets are truisms or platitudes:

1. indicators exist to assist decision making: an important purpose of performance indicators is to provide feedback to the managers to guide decisions on continuous program improvement;

2. there are a variety of indicator types (input, process, output, outcome), but performance indicators should focus on a limited number of key outcome areas designated by program managers as critical to the continuing successful functioning of the agency;

3. a first step in selecting indicators is ensuring that the objectives are ‘appropriately’ stated;

4. there should be widespread managerial and staff involvement in the selection of the performance indicators so that ‘ownership’ will occur;

5. there are a variety of uses for indicators, including strategic and program planning, leadership and motivation, management control, accountability and evaluation;

6. there are a variety of audiences for performance indicators, including agency managers at different levels, executive Government, Parliament, lobby groups, press and public.

Beyond these platitudes, the public sector literature has little to say on rigorously identifying and validating performance indicators. The assumption is that indicators are susceptible to the democratic process. The wider literature on performance management is similarly unhelpful in moving beyond vague platitudes.

Business rules for acting on indicator feedback

Even if we have the right KPIs, the management literature is essentially silent on the issue of validating business rules for responding to off-trend indicators and of training managers in this regard.


- 15 -

The assumption is that when the ‘red light’ starts flashing, managers will know intuitively which levers to push and in which direction. This is manifestly fallacious.

Whilst the Chernobyl and 3 Mile Island nuclear disasters are possibly the most striking examples of the fallacy of this assumption, there have been a host of major corporate bankruptcies over the past decade which attest to the significance of this point. Let me illustrate the significance of the issue.

The original VW ‘Beetle’, with its engine in the rear, was very susceptible to going into a ‘rear-wheel skid’ when cornering. Picture the scene: Our ‘digital dashboard’ KPI system flashes “Rear-Wheel Skid”. The manager (driver) has 3 decision levers: the brake, the accelerator and the steering wheel. The instinctive managerial response is to steer out of the skid and / or hit the brake. The technical reality is that either of these ‘business rules’ will accentuate the skid; whilst application of both could cause the car to roll. The correct business rule is to turn into the skid whilst gently accelerating, and only after regaining control, turn out of the skid.

Knowing a problem exists and knowing the decision levers available does not guarantee that the right lever(s) will be pushed in the right direction.

The assumption that a “good” KPIs will necessarily result in managers making “good” decisions based on indicator feedback is debateable. There is considerable research to suggest that the assumption is invalid, especially in the face of delayed feedback.

INDICATOR HIERARCHY AND CAUSAL INTERRELATIONSHIPS

The following sections suggest tools and methods to address the shortcomings, discussed above, of current approaches to implementing the Balanced Scorecard:

identifying potential KPIs and strategies

identifying interrelationships between KPIs and strategies

using ‘flight simulators’ to train managers on how to respond to KPIs

Cognitive mapping technique to identify perceived +ve & -ve relationships

Cognitive mapping derives from the Psychology of Personal Constructs developed by cognitive psychologist, George Kelly29. It is a powerful and effective tool to capture and analyse stakeholder concepts about strategic issues. Experience in many environments, including the Australian Defence Organisation, demonstrate that it can result in high levels of shared understanding among stakeholders, analysts and decision makers.

A ‘cognitive map’, ‘concept map’ or ‘causal map’ consists of nodes, known as concepts linked by arrows. The arrows may carry a sign, though the absence of a sign is usually taken to indicate a positive link and a negative sign indicates a negative link. Causality feedback and dominant

29 Kelly, G. A. (1955). “The Psychology of Personal Constructs: A Theory of Personality.” Norton, New York


- 16 -

mechanisms may be derived from these maps. McLucas has developed a robust methodology, within an Australia context, for the application of this technique.30

Figure 11 illustrates the end point of a cognitive mapping process focusing on elements of the Defence Science and technology Organisation (DSTO) draft Balanced Scorecard. The draft BSC exhibited the traditional uni-directional cause-and-effect chain. The cognitive mapping process, combined with a cluster analysis, not only revealed profound inter-relationships, but highlighted several negative relationships (dotted lines) which had not previously been appreciated. (A negative relationship suggests that a particular strategy designed to promote achievement of one strategy, has a side effect of countering another strategy … a possibility ignored in the BSC literature.)

Figure 11: DSTO KPI Influence Diagram Developed from Cognitive Mapping Process

Clustering techniques for consolidating indicators

The Balanced Scorecard literature emphasises the importance of focusing on a small number of key performance indicators, but again gives not suggestions on how this selection might be done. Statistical cluster analysis provides a simple and robust technique. In essence, cluster analysis groups like factors with like, and separates unlike from unlike.

Cluster analysis can be used at various stages in developing a Balanced Scorecard. For example, in relating KPI’s to strategy, the management team (individually or collectively) assigns to each KPI a score (e.g., on a scale of -10 to +10) based on judgement of the degree to which a given KPI supports (1 to 10), is neutral (0) or detracts from (-1 to -10) a given strategy. Figure 12 illustrates part of the output from the DSTO study.

30 McLucas, A. Qualitative and quantitative techniques for addressing systemic complexity in the context of

organisational strategic decision making. PhD Thesis. UNSW. Aug 2001


- 17 -

Figure 12: Worksheet for DSTO Clustering Exercise The results of such an exercise, e.g. on an Excel Spreadsheet, can then be fed directly to a cluster analysis program (standard with most statistical software packages). A variety of output formats are possible, including the dendogram tree in Figure 13.

Figure 13: Dendogram Produced by Cluster Analysis

The dendogram identifies ‘clusters’ of (in this case) KPIs, providing a rational basis for the development of the performance indicator hierarchy in the Balanced Scorecard and for the analysis of inter-relationships.

SYSTEM DYNAMICS AND PERFORMANCE INDICATORS

As noted earlier, Kaplan and Norton argue that the Balanced Scorecard should be based on the metaphor of the flight simulator, not a dashboard of instrument dials. “Like a flight simulator, the


- 18 -

scorecard should incorporate the complex set of cause-and-effect relationships among critical variables, including leads, lags, and feedback loops that describe the trajectory, the flight plan, of the strategy.” 31

System dynamics software enables just this, the creation of ‘management flight simulators’ which integrate with the corporate Balanced Scorecard system. Managers can ‘learn’ how performance criteria affect not only their decisions but the performance of colleagues and of the organisation. They can also test the longer term implications of different decision rules.

The use of dynamic models, particularly in a gaming environment, adds critical dimensions to the Balanced Scorecard. The following section discusses a prototype ‘dynamic balanced scorecard’ flight simulator designed to enable managers to ‘test drive’ performance indicators and related business rules.

BUILDING A PUBLIC SECTOR DYNAMIC BALANCED SCORECARD

The essential purpose of the various state & federal government outcomes and outputs framework is to answer three questions:

(i) What does government want to change? (outcomes) (ii) How does it want to make that change? (outputs) (iii) How does it know if it is succeeding in making that change? (indicators)

The system works as a hierarchy. Government, through its ministers and with the assistance of relevant agencies, specifies the outcomes it is seeking to achieve in a given policy arena. These outcomes are specified in terms of the impact government is aiming to have on some aspect of society, the economy or the national interest. Parliament appropriates funds to allow the government to achieve these outcomes.

These funds allow government to ‘buy’ outputs or services from agencies. Agencies specify the outputs (e.g. policy advice) they undertake to provide to contribute to the achievement of these outcomes, including through third parties.

Specification of what will be delivered can help improve the understanding and knowledge of those outside the agency who have an interest in its performance, including Ministers, Parliament and external accountability bodies such as the Auditor General.

The framework encourages agencies to focus on the specific products they deliver (outputs) and how to deliver them in the most efficient and effective ways; and to identify business lines they should drop or outsource. The implementation of accrual accounting using generally accepted accounting principles provides the basis for benchmarking comparisons so that decisions about preferred providers are better informed.

Where services are provided direct to industry and the community, the development of client service charters is an integral part of the reforms and an important indicator of performance. The Government itself makes decisions on programs and (usually) on guidelines for service provision. Within these guidelines, however, governments expect service delivery to be of a high standard and public servants are accountable to their Minister and to government for their service to customers in this context.

Charters must specify standards for service delivery and set out complaint handling and feedback mechanisms, and must be developed through consultation with customers and staff. Agencies are required to report annually, through the Department of Finance and Administration, to the Parliament on their performance against the charter.

The ‘Customers’ of the Federal Public Sector

The process outlined above provide a more complex view of the customer perspective for the Australian national public sector then that which exists in the private sector. In the public sector

31 Kaplan,RS and Norton DP, “Linking the Balanced Scorecard to Strategy”, California Management Review, Vol. 39,

No. 1, Fall 1996. p.64.


- 19 -

context, the prime customer (i.e. the agent who pays for a reciprocal service) is the Minister on behalf of the Government.

Departments also have a legal requirements which set up ‘customer relationships’, especially to the Auditor General and to the Parliament in respect of ‘governance’, especially in relation to fraud prevention. Other management oriented legislation, such as Equal Employment Opportunity and Occupational Health & Safety legislation, apart from their moral imperative, have the potential to create significant embarrassment for agency management and the Minister if they are breached. These clearly create a distinct ‘customer’.

In those cases where Federal Departments do provide services direct to the public, they are required to prepare and implement a service charter, providing a clear ‘customer relationship’.

Thus it is suggested that are four ‘Customers’ who may need to be addressed in a BSC: for most departmental activities, the Minister, and through him/her, the Government; in respect of governance, the Auditor General and Parliament, as well as the Minister; in respect of organisational health (OH&S, EEO etc) Parliament, the Minister, staff for service delivery activities, the corporate or individual service recipients.

These and other considerations have led us to propose that a 7 Sector Balanced Scorecard, depicted in Figure 14, may be more appropriate for public sector agencies that the ‘classic’ 4 sector model.

Figure14: Seven Sector Balanced Scorecard for Government

Dynamic Balanced Scorecard - Integrating System Dynamics & BSC

The following material summarises research done in a number of government agencies designed to build this seven sector dynamic Balanced Scorecard

The BSC template that has been developed focuses specifically on the relationship between managerial responses to any ‘workload-resourcing’ gap (the ‘Capacity Gap’), and how this impacts on the first two ‘Customers’ above. The model outputs relate specifically to output quality index; probability of fraud / probity incidents; and probability of major OH&S / EEO incidents.

The following causal loop diagrams illustrate the mechanisms by which these are impacted by departmental operations. The interrelationships between the respective work area outputs and their impact on achievement of outcome indicators is subject of a separate study.

Resources Performance Management in the Public Sector

All federal Government CEOs consider financial results to be one of the top three indicators of their organisation’s success. However, financial management in these agencies differs dramatically from the private sector context in that the revenue side of the budget is a given, and the focus is simply on effective and efficient management of the expenditure.

The quantum of funds made available to a Minister to implement Government programs is the end result of a complex interplay of macro-economic deliberations, ministerial bargaining and political judgement. Issues regarding what can be delivered (in terms of quality and quantity) for the proposed funds are significant inputs. But such relationships are largely approximations. Lack of skilled

Resource ManagementClient OrientationGovernance & ProbityOrganisational HealthStructure & ProcessesOrganisational Learning Corporate Communications

GOALS


- 20 -

management (for example as a result of high turnover or overwork) can result in errors in estimating required workload.

When the Government has determined its policy, the bureaucracy has only minor leeway in changing the quantum or time schedule of the service. In addition, a variety of unplanned business pressures inevitably impact on planned business. Unforeseen events such as fraud within the Department, a by-election in a sensitive electorate, or a major controversy relating to the Minister’s policy responsibility inevitably generate workload which is expected to be ‘absorbed’. The resource management task is to deliver the planned outputs within budget.

Figure 15, overleaf, illustrates interrelationships in the resource sector. ‘Customer Demand’ represents the Government’s (through the Minister) expectations which also determines the resourcing level. It is almost axiomatic that available resources will be less than that required for quality implementation of all the planned workload, let alone the inevitable unplanned demands. Whether the management response to any ‘capacity gap’ is innovative or dysfunctional is a function of the organisational competencies, which in turn is the result of leadership and investment in capability, the latter balancing short term impacts on recurrent resources.

Figure 15: Key Resource Sector Interdependencies in Public Sector Agency

Figure 16 depicts the staffing module of the Resources Sector in the Dynamic BSC model. This model tracks skill levels of senior executives, executives, technical and administrative support staff, based on staff turnover rates and time at each level to reach full efficiency.

DepartmentResources

s

AnnualWorkloadPlanned

RecurrentResources

CustomerDemand

s

o

o

BusinessPlanning

Errors

s

ss

s

s

s

o s

s'New'

Business

Base LoadBusiness

Total EffectiveCapacity

CapabilityInvestment

CAPACITY GAP

o

ManagementEffectiveness

DysfunctionalResponses to

Gap

InnovativeResponses to

Gap

o

s

s

s

Learning OrganisationCompetencies

so

ss


- 21 -

Figure 16: Staff Resources Segment of Dynamic Balanced Scorecard Model

Key resource management performance criteria relate to corporate governance (probity, fraud control, etc), managing expenditure patterns (and especially employment related expenditure patterns) to budget and achieving efficiencies in an increasingly efficient environment.

The Characteristics of the Internal Processes in the Federal Public Sector

The core processes in a public sector agency are essentially the same as for the private sector. These are to:

Establish direction; Acquire resources; Provide capability; and Execute the mission.

Whilst the Government establishes the policy and program outcomes that are to be achieved in exchange for the financial resources, management translates the vision and allocates the capabilities to achieve the delivery of agreed outputs. The resource management framework provides a backdrop for an integrated planning process that links corporate plans, business plans and individual plans. This planning process focuses on achieving results through the delivery of outputs as the agency, the business unit and the individual’s performance is linked to the outputs which in turn is linked to the outcomes that the Government’s desires for the community.

As noted in relation to resource management, once the budget framework is set, Federal Departmental managers have limited scope for obtaining extra resources. Any increase in workload, or workload underestimate, typically will be addressed by working harder:

more intense and longer hours of work (unpaid overtime) reduction in time devoted to training and development reduction in strategic management activities (through redirection of 'management time' to 'task

time') deferring some work (which simply postpones the day of reckoning) and reduction in target quality of inputs (e.g. through cutting background research effort) or

outputs

Personnel

Resources Return ToModelSector

progress1 progress3 progress4

NovAttRate SkilledAttRate

departing_

CompAttRate

progress2

TraiAttRate

hiring_SupStaff

Progress_delay Progress_delay Progress_delay Progress_delay Attriton_Rate

NoviceAttRate TrainAttRate_2 CompAttRate_1 SkilledAttRate_1

hiring_ExecStaff

Novice_ Trained_ Competent__ Skilled_ Experts

Support_Staff_Need InitTrained InitCompetent InitSkilled InitExpertInitNovice

Executive_Staff_Need

! !

!

!


- 22 -

As illustrated in Figure, if such responses are prolonged, they tend to bring about dysfunctional feedback effects which eventually increase the capacity gap, through increased re-work, falling moral, increased staff turnover.

Figure 17: Key Interrelationships Impacting on Internal Processes

Where there is excellence in leadership and a culture characterised by the ‘learning organisation’ one could expect innovative responses to any significant or prolonged ‘capacity gap’. In essence such innovation changes the ‘rules of the game’ and achieves the end result much more efficiently. There are many local examples of such behaviour across the bureaucracy, but at this juncture the authors would be sparing in their application of ‘learning organisation’ to Departments as a whole.

Only dysfunctional responses, as illustrated in Figure18, have been incorporated in to the model at this stage because of lack of data to permit modelling of the ‘learning organisation’ response.

Figure 18 is a simplified extract of the Powersim dynamic BSC model relating to the Internal Process Sector, corresponding to the causal loop diagram above. The module currently draws its base data from spreadsheet, including:

annual work plans new policy and extra ministerial projects average monthly patterns of work

The model also includes allows for outsourcing activities ( not included in the causal loop diagram).

StaffTurnover

s

s

o

s

so

o

s

s

s

s

WorkDeferred

o

s

s

Time on Strategic

Management

WorkIntensity


Re-work /Errors

Average Workload

CAPACITY GAP

s

o

ManagementEffectiveness

Target OutputQuality

StaffMorale

o

o

s

Productivity


- 23 -

Figure 18: Internal Process Sector of the Dynamic BSC Model

In the model’s ‘flight simulator’ panel (Figure 19), how the capacity gap is addressed is a matter, each three months, for user decision, with the user being given the choice of, e.g.,:

specifying a % of the excess workload to defer to the next year switching a % of strategic management time onto the excess workload switching a % of staff development time on to the excess workload absorbing the excess workload through reduction in output quality

Figure 19: Dynamic BSC Simulator Control Panel


- 24 -

Innovation and Learning in the Public Sector

Recruitment standards to the federal Public Service are high. Almost 50% of all recruits have a Bachelor Degree or higher. Staff receive support to upgrade qualifications. Middle & senior management training has a high priority. At the same time, work pressure on all staff, and unpaid overtime, has increased significantly which acts as an impediment to self-development. Also, in the face of work pressures, informal and formal staff development tend to be among the first areas to be constrained. Figure 20 shows key causal relationships impacting on the learning and growth sector.

Figure 20: Key Interrelationships Impacting on Learning & Growth

A critical factor in learning and growth is staff turnover. What little research is available suggests that turnover rates in excess of 15% are dysfunctional. A number of key government agencies at both federal and state level have staff turnover at the executive and senior executive levels of almost 30% per annum, and over 20% per annum across all staff. The model reflects this in loss of productivity.

Corporate Governance & Corporate Health

Figure 21 depicts the Corporate Governance sector of the dynamic scorecard. We have been undecided whether corporate governance and corporate health should be regarded as sectors in their own right. On the one hand, Corporate Governance might be viewed as a key indicator in the Customer Sector, whilst Corporate Health might be an indicator for the Internal Process Sector. On the other hand, failures in these areas tend to have major political implications for the Minister and the

StaffTurnover

s o

s


Average Workload

CAPACITY GAP

StaffMorale

o

Productivity

os

so o

Skill Level

Time forStaff

Development

CapabilityInvestment

InnovativeResponses to

Gap

Learning OrganisationCompetencies

s

s

s

RecurrentResources

o

o

s

s

o

sRe-work /

Errors


- 25 -

Government as well as significant feedback interrelationships with the other sectors. This suggests that they might appropriately be considered as sectors in their own right.

The Australian National Audit Office (ANAO) defines corporate governance to encompass authority, accountability, stewardship, leadership, direction and control.32 The dynamics BSC model, at this stage, uses fraud events as a general surrogate for the effectiveness of governance across the broad range of departmental activities.

Figure 21: Corporate Governance Sector of Dynamic Balanced Scorecard

Model Output

At the time of writing this paper, only illustrative reporting screens have been developed. Figure 22 is an example. It is apparent from discussions with departmental staff that ultimate use of the simulator will depend as much as anything on the “look and feel” of the output screens. The current version of Powersim has limited flexibility in screen design and it is likely that custom screens will be developed in Visual Basic or C++.

32 ANAO, Applying the Principles and Practice of Corporate Governance in Budget Funded Agencies, Australian

Government Publishing Service, Canberra 1997.

Corporate Governance Sector

Fraudulent and Dubious Practices Public Awareness of Fraud

Return ToModelSector

Internal Factors Impacting on Fraud

Fraud_Accumulating

in_progress

Fraud_occuring

Random_Num

FRAUD_TOTALFRAUD_EVENTPROBABILITY_OF_FRAUD

Fraud_Events_MAvg

Public_Awareness_of_Fraud

Effect_of_Fraud_Total_on_Aware

Effect_of_Fraud_Total_on_Awareness

Management_Effectivness

ffect_of_Morale_on_Fraud

MORALE

Effect_of_StaffDevelopment_on_Fraud

StaffDevelopment_Index

Increase_in_Fraud_Prob! !!!


- 26 -

Figure 22: Illustrative Output Screen for the Dynamic Balanced Scorecard

FUTURE DEVELOPMENTS

This paper has highlighted some serious theoretical and implementation shortcomings of KPIs in general and the Balanced Scorecard in particular, especially in relation to their implementation in the public sector. Solutions have been outlined to these. In particular, it is suggested that system dynamics modelling can be used, to provide ‘management flight simulators’ which “incorporate the complex set of cause-and-effect relationships among critical variables, including leads, lags, and feedback loops that describe the trajectory, the flight plan, of the strategy”.

____________________________________________

2008-ica_kpi reporting in the public sector

Documents