PA T I E N T S A F E T Y

Quality Observatories: using information to createa culture of measurement for improvement

Samantha Riley and Katherine Cheema; NHS South East CoastQuality Observatory

AbstractIn June 2008 High Quality Care for All was published, a blueprint for the delivery of NHS services in the

future with quality of care put firmly at the heart of the principles guiding the NHS. Measurement of

quality was identified as a crucial aspect of achieving delivery, enabling clinical teams to focus on

where they need to improve most, and monitor the effects of interventions and initiatives. To date,

the South East Coast Quality Observatory has developed over 80 benchmarking tools, products and

analyses which contribute to the goal of delivery high quality analysis in a way that can be understood

and utilized effectively by all levels of staff in a variety of clinical and managerial settings to evidence

and drive improvement and innovation across organizations and local health economies. Patient safety,

as a fundamental aspect of high quality care, makes up a significant part of this output, focusing on

key topic areas that can deliver the best, and safest, results for patients. The following sets out details

of a selection of the patient safety areas that the Quality Observatory has focused on and describes the

analysis undertaken and, crucially, the key outcomes linked to its publication and usage.

Introduction: what is a QualityObservatory?

In June 2008 High Quality Care for All1 was published, a

blueprint for the delivery of NHS services in the future.

This milestone document put quality of care at the heart of

the principles guiding the NHS and made clear that

measuring this quality was a crucial aspect of delivery,

enabling clinical teams to focus on where they need to

improve most and monitor the effects of interventions or

initiatives. It was explicitly acknowledged that high per-

formance in all aspects of quality is nearly always present in

organizations that proactively and effectively measure their

activity and use the information gathered to drive improve-

ments forward. One aspect of embedding this measurement

agenda in the NHS as a whole was the requirement for

each of the 10 strategic health authorities (SHAs) to estab-

lish a formal ‘Quality Observatory’ which built on existing

analytical arrangements. Quality Observatories were

expected to provide a range of functions, with the key

stakeholders being clinical teams operating on the front

line. These functions primarily included:

† To enable and support benchmarking across regions;

† To support the development of metrics to enable frontline

staff to effectively monitor and improve their services;

† To identify opportunities for clinical staff to innovate

and improve services, providing the quantitative evi-

dence to support change where required.

In NHS South East Coast, this function had been in place

for sometime providing a free service to NHS professionals

within the region. Since 2007 there had been a substantive

‘Knowledge Management team’ focused on helping organ-

izations and individuals get more from the wealth of data

collected by the NHS. Based at the SHA headquarters, the

team provided analysis and benchmarking to the NHS

across Kent, Surrey and Sussex on a broad range of subjects

including performance, efficiency and a variety of clinical

services.

In addition to the provision of analysis and benchmark-

ing, the Quality Observatory has provided the following

services:

† An education and training function both for analysts and

‘customers’ of information. An example of the training

provided is the ‘de-mystifying data for clinicians’ training

session;

† How to develop clinical indicators and combine them

to understand more about a service;

† Identifying best practice and evidencing the variation

between teams and organizations;

† Acting as a conduit for local issues/news to be raised

with relevant national bodies and vice versa;

† A bespoke service to individual clinicians and teams

requiring support with analysis/development of measures.

To date, the South East Coast Quality Observatory has

developed over 80 benchmarking tools, products andEmail: samantha.riley@southeastcoast.nhs.uk

Samantha Riley, Katherine Cheema, NHS South East Coast – The

Quality Observatory, York House, 18–20 Massetts Road, Horley, Surrey

RH6 7DE, UK

DOI: 10.1258/cr.2010.010002 Clinical Risk 2010; 16: 93–97

analyses which adhere to the goal of delivering high quality

analysis in a way that can be understood and utilized effec-

tively by all levels of staff to evidence and drive innovation

and improvement across organizations and local health

economies. Patient safety, as a fundamental aspect of quality

care, has made up a significant part of this output, focusing

on key topic areas that can deliver the best results for

patients. The remainder of this paper sets out details of

some of the patient safety areas that the Quality

Observatory has focused on and describes the analysis

undertaken and the key outcomes linked to its publication

and usage.

Healthcare-associated infections(HCAIs)

When creating a suite of tools and analysis for this area of

patient safety, a key issue was the varied audience that

would wish to access the information. Directors of infec-

tion prevention and control, their nurse-led infection

control teams, ward-based staff, performance managers and

commissioning managers are just a selection of the range of

staff groups and backgrounds that would need to be able to

access what can be complex information in a simple way.

The way in which information is presented is very

important – it needs to be visually appealing and at the

same time enable the user to quickly and clearly understand

what the data are saying – often this is sadly not the case.

Figure 1 shows a sample of how HCAI information was

historically presented. The analysis of trends over time were

difficult to gauge at a glance, and the variation in cases was

hard to compare with other factors, for example, the time

of year. The majority of stakeholders viewing these data

would look at a single figure for one month and perhaps

one either side of it, and draw conclusions and potentially

make judgements. The use of colour coding also invited

users to make an immediate judgement based on a limited

amount of data which were often inappropriate.

Figure 1 ‘Old-style’ reporting for Clostridium Difficile AssociatedDisease (CDAD) cases. Source: NHS South East Coast (available in

colour online)

Figure 2 Healthcare acquired and associated CDAD reporting dashboard for PCTs. Source: NHS South East Coast (available in colour

online)

94 Riley and Cheema

Clinical Risk 2010 Volume 16 Number 3

The Quality Observatory worked with clinicians to

develop a ‘dashboard’ which allowed multiple views of data

on one sheet. Figure 2 gives a simple example, where the user

can see month by month cases against defined limits, the

cumulative position for these cases against limits, and a rate-

based view which allows comparison between organizations.

The dashboard uses colour and graphs to make it easier to

understand what the data are saying, whether the position is

improving or declining, and whether thresholds are being

met. The dashboard also features controls that allow users to

select their specific organization, aggregate views across the

counties or region or other organizations they wish to

compare their own performance against. This makes the dash-

board simple and intuitive to use; users can access a wealth of

data, including commentary, at literally the touch of a button.

Aside from the regular reporting for MRSA and

Clostridium Difficile Associated Disease (CDAD), the

application of specific improvement techniques have

enabled stakeholders to gain an in-depth knowledge of the

variation in cases. Initially, this has been applied to CDAD

but could be used for a multitude of patient safety events,

such as catheter-related urinary tract infections (UTIs),

in-hospital patient falls or drug administration errors. The

concept of looking at a system, over time and assessing the

variation within it, rather than making judgement on single

number or against a set target, is one that is well established

in the manufacturing industry and can be applied to great

effect in healthcare.

By identifying unusual, or ‘special cause’ variation

within a system, factors that affect it can be identified and

acted upon. Figure 3 is an example of how the Quality

Observatory applied this technique to CDAD data with the

development of an ‘early warning system’. The graph is

annotated to show points of unusual variation, in this case

mostly linked to bank and school holidays. Again, the

graphical presentation and the non-judgemental approach is

far more conducive to engaging all stakeholders in the use

of information to evidence their actions and be pro-active

in managing the system.

Serious untoward incidents

Perhaps the most explicit example of where effective display

of information has been key to widening engagement with

stakeholders is the area of reporting serious untoward

incidents (SUIs).

SUIs are a key facet of monitoring and improving

patient safety. SUIs are reported via a central system and can

be viewed and analysed by the SHA. However, the infor-

mation is largely textual and can be hard to review within

and between organizations. The development of a ‘SUI

dashboard’ (currently in draft) enabled a more coherent

view of the incidents reported and includes a variety of

measures concerning timely reporting and closure – all of

which can be broken down by incident type. Similar to the

HCAI dashboard already discussed, users can filter the

information as they wish using drop-down menus, the

display updating automatically.

One of the most useful aspects of the analysis is the

inclusion of National Reporting and Learning Service

(NRLS) information. This adds additional details to the

overall picture but also gives a comparison between

numbers of incidents reported via each route. The use of

Figure 3 Annotated statistical process control chart illustrating weekly CDAD cases. Source: NHS South East Coast (available in colour

online)

Quality Observatories 95

Clinical Risk 2010 Volume 16 Number 3

Figure 4 SUI dashboard including NRLS data. Source: NHS South East Coast (available in colour online)

Figure 5 Safer, Smarter Nursing Metrics dashboard. Source: NHS South East Coast (available in colour online)

96 Riley and Cheema

Clinical Risk 2010 Volume 16 Number 3

multiple data sources around a single subject is a powerful

way to build a full picture of an organization or service and

prevent bias towards one route or methodology of report-

ing. The dashboard also removes the need to examine

individual reports to gain an over-arching view of a trust’s

approach to patient safety incident reporting, thus saving

time and engaging a wider audience in the process and

importance of reviewing of this kind of patient safety

information.

Figure 4 shows the SUI acute trust dashboard, with the

NRLS data included in the boxes with the blue back-

ground. The figure shows all SUI incident types and has

been anonymized.

The use of multiple indicators from national data

sources is well exemplified above, but as yet does not show

how these can be triangulated with the wealth of local data

available through acute and primary care providers. The

Quality Observatory will be exploring this next phase of

development with commissioners and providers.

Safer, Smarter Nursing Metrics

The issue of patient safety is nowhere more crucial than on

the ward, where staff at the ‘front line’ are directly

responsible for ensuring their patients are treated to the

highest quality standards. The ‘Safer, Smarter Nursing

Metrics programme’ aimed to build a product that included

measures that could be ‘owned’ by the staff actually

providing care and measure how their influence could

tangibly improve outcomes for patients, for example drug

administration errors, in-hospital falls and pressure ulcers.

The programme had clinical sponsorship from Directors

of Nursing and has subsequently been developed, in con-

junction with the Quality Observatory, by individual trusts

to add more indicators and produce analysis at the ward

level. Figure 5 shows the regional Safer, Smarter Nursing

Metrics dashboard which has six indicators of nursing

quality, all of which were agreed by clinical stakeholders,

two of which are sourced from national data-sets, the

remainder from trusts’ clinical risk management systems.

The simple presentation of the trust data on the blue

bars against the regional rate on the red line, allows busy

clinicians to quickly assess their position and look at their

trend over time. In general, across the region, indicators

have shown a downward trajectory since the establishment

of this programme. Whether or not this is improvement has

been directly influenced by having the data available, freely

accessible and easy to use is not proven, however as High

Quality Care for All states, ‘we know that a defining charac-

teristic of high performing teams is their willingness to

measure their performance and use the information to

make continuous improvements’.

A key learning point of the Safer, Smarter Nursing

Metrics programme is the importance of ensuring clinical

teams are actively involved in developing and agreeing indi-

cators. The success of this programme can be seen through

the fact that organizations continue to submit local data to

the SHA for analysis, something that would be unlikely to

happen if they did not see the benefit.

Conclusions

The work that has been described here provides just a sample

of what a Quality Observatory could produce to support

their organizations and clinical teams across a region. The

absolute requirement, particularly in the current financial

climate, to focus on quality improvement and productivity in

the NHS means that monitoring and evidencing good prac-

tice in patient safety effectively is of paramount importance.

It is very important to make explicit that any analysis

should provide a starting point for discussion rather than

prompt an immediate (and often inappropriate) judgement. A

key role for the Quality Observatory in South East Coast has

been to educate our customers to move away from a more

‘judgemental’ approach to a constructive approach which

involves the use of measurement for improvement techniques

and discussion between teams and organizations. This

approach should ensure that the wealth of data that the NHS

has at its disposal is turned into intelligence that will not only

make practitioners lives easier, but patient treatment safer.

Different Quality Observatory models are being estab-

lished by each of the regions – further information can be

obtained from the Quality Observatory website hosted by

NHS South East Coast (http://www.qualityobservatory.nhs.

uk). If you are interested in any of the dashboards detailed in

this article, or are interested to learn about the other models

and products which have been developed by the team, look

at the South East Coast tab and register as a user.

Reference

1 Department of Health. High Quality Care for All. London:

Department of Health, 2008

Quality Observatories 97

Clinical Risk 2010 Volume 16 Number 3