Using Key Risk Indicators

Dr Simon Ashby

Plymouth Business School &

Institute of Operational Risk

1

Overview

• The basics: a multiplicity of terms! 

• Elements of good practice

• Looking to the future: the state of the art

• Activity and discussion

2

KEY TERMS AND CHARACTERISTICSThe Basics....

3

Definitions (from IOR Standard)

Risk IndicatorMetric that provides information on the level of exposure to a given operational risk which the organisation has at a particular point in time.

Control IndicatorMetric that provide information on the extent to which a given control is meeting its intended objectives.

Performance Indicator Metrics that measure performance or the achievement of targets.

Key Indicator

Indicators are measurable metrics used to monitor identified risk exposures over time. An indicator becomes ‘key’ when it tracks an especially important risk exposure (a key risk), or it does so especially well (a key indicator), or ideally both.

4

Definitional Discussion Points

5

• Must indicators be measurable?• Almost any metric may be considered an

indicator. There is no universal list.• Indicators can have multiple personalities –

they may indicate different things at different points in time.

• What is ‘key’ can also vary over time. ‘Key’ may depend on both the relevance of an indicator and the significance of a risk.

The Role of Indicators

6

Contents Discussion Points

Support risk assessmentsCan support but no substitute. Remember that indicators rarely provide the full picture and cannot replace human judgement.

Monitor exposure between assessments

Leading vrs lagging. Remember that many indicator reports are often 1 month or more out of date!

Risk appetite and governanceIndicators are an important part of this, but keeping track of changing risk exposures is only part of the process.

Performance management Don’t forget the link between risk and strategy.

RegulationLow focus. Primarily relevant for FS and highly varied even here.

FACTORS TO CONSIDERGood Practice…..

7

Selecting Indicators

8

• Desirable characteristics– Relevance – Ease of monitoring

– Measurable – Auditable

– Predictive – Comparable (benchmark)

• Top down versus bottom up

• How many are enough? No right answer!

Thresholds and Limits

9

• ‘RAG’ thresholds/limits are pretty much essential, but…. if set incorrectly they can be very destructive.

• Often better to wait and build up some trends before setting ‘hard’ thresholds/limits.

• Review thresholds/limits and change as necessary. Over time consider tightening them up.

• Remember they are useless if no action is taken!

• Finally ensure thresholds/limits are linked to your stated board risk appetite.

Managing and Reporting

10

• Ideally link selection of indicators with the risk assessment process.

• Ensure indicators are properly documented and that procedures are in place to manage selection and reporting processes (including changes).

• Frequency – monthly reporting may not always be enough. Or may be too much.

• Keep reports simple:– Prioritise, using an exception basis where possible.

– Tailor for different ‘information consumers’.

THE FUTURE FOR RISK INDICATORSThe State of the Art…..

11

The Risk Chain

Cause(s) Event Effect(s)

Resources

Reputation

People

Processes

Systems

External Events

Human

12

Risk Chain Questions

• Should we fight the tyranny of the risk register – and collect indicators on causes and effects rather than events?

• How many indicators do we actually need? Less may be more.

• How can we be more leading and less lagging in our use of indicators?

13

Measurement Vrs Management

14

“You can’t measure what you can’t manage” (??)

Remember the 7 deadly diseases of management (Deming):

1. Lack of constancy of purpose 2. Emphasis on short term profits 3. Evaluation of performance, merit rating or annual review4. Mobility of top management (too much turnover causes numerous

problems) 5. Running a company on visible figures alone 6. Excessive medical costs7. Excessive legal damage awards swelled by lawyers working on

contingency fees http://curiouscat.com/deming/managewhatyoucantmeasure.cfm

Socio‐Technical Systems

People ‘Machines’

15

Risk is a function of both these factors. Hence indicators must reflect both the

objective and the subjective.

Things Can Get Very Complex!

16

AI Systems: Coping With Complexity

17

Front Office Trading

Trading  Error Rule

Unauthorised Trading Rule

Time between audits

Unauthorised Trading

Staff availability

Outstanding orders

Open audit issues

Mid Office Trading

Internal Fraud Rule

Mis‐pricing Rule

Growth in profits

Mis‐pricing

Time between  audits

Reporting Lines

Open audit issues

Internal Fraud

Back Office Trading

Mis‐pricing Rule

False Accounting Rule

Time between audits

False Accounting

Skill Shortage

Data feed quality

Open audit issues

Mis‐pricing

Segregation of Duties

Deutschmark Interest Rate Swaps Currency Back Office 

Trading Error

Risk Dependency Networks

18

SOME QUESTIONS TO ASK YOURSELF

Activity….

19

Questions for Discussion

• Are your risk indicators reports too detailed?

• Who receives reports and how do they differ?

• How often do you change the indicators in your risk indicator reports?

• What % of your indicators are leading indicators?

• Have you found any correlations between your indicators and risk exposures?

• Are your indicators linked to cause, event or effect? 

• How well do you cope with dependencies between risks?

20

USEFUL DOCUMENTS AND WEB LINKS

Further Research

21

Some Useful/Interesting Links

• IOR Sound Practice Guidance: http://www.ior‐institute.org/

• COSO Guidance on KRIs: http://www.coso.org/guidance.htm

• Risk Business KRI Library: http://www.kriex.org/

• DRisk: http://drisk.eu/Home.aspx

22

CONCLUSIONSThe End....?

23

Conclusions

24

• As with almost all areas of operational risk management, there is no one approach to developing and using risk indicators.

• However common ‘sound’ practice is emerging and developing.

• Remember that operational risk management is both an art and a science. Hence indicators have an important role to play, but do not place too much reliance on them.

Thank You

25

Dr Simon AshbyDeputy Chairman of the IOR andHead of the Accounting and Finance Group, Plymouth Business SchoolDrake Circus,Plymouth, Devon,PL4 8AA

Telephone: +44 (0)1752 585720 Email: simon.ashby@plymouth.ac.uk