Symposium 8 How to Succeed in Audit

5 year Audit Data on ‘MUST’:

How Audit has Targeted Practice

Dr Sorrel Burden

Lead Dietitian/ Research Fellow

Introduction

Screening has been recommended by the following reports:-

• Food and Nutritional Care in Hospitals

• Nutrition Support in Adults

• Patient Environment Action Teams Assessment

• Quality Care Commission

• Essence of Care Benchmarking for Food and Drink

• Quality Care Commission

Aim of Audit

• To evaluate the implementation of MUST on 32

wards over 3 hospital sites

Objectives

• Determine the rate of nutrition screening within

24 hrs of admission

After 7 days

• Audit recording of weight, height & BMI

• Identify low, medium and high risk malnutrition.

Method

• ‘MUST’ was implemented in July 2006.

• 850 ward nurses by 14 dietitians & 2 NNS.

• Jan 2007-Mar 2011 screening rates were audited.

• 1 week every month.

• NLN collected data using standard form

• Submitted electronically for collation to Audit Department

Audited Standards

• Screened within 24 hrs of admission.

• Screened weekly thereafter.

• Weight, height & BMI should be

recorded.

• Action plan should be followed for all

identified at risk of malnutrition.

Wards submitting data 2007-2011

0

10

20

30

40

50

60

70

80

90

100

2007 2008 2009 2010 2011

no. wards

% wards

Number of patients audited

2007-2010

Number of patients audited 2007-2010

17, 836 patients over 5 years

0

1000

2000

3000

4000

5000

6000

7000

8000

No. patients

2007 2008 2009 2010

Year

Screening rates at 24hrs & 7 days

0 50 100

2007

2008

2009

2010

2011

year

% of patients

7 days

24 hrs

High & moderate risk of Malnutrition

0

5

10

15

20

25

30

35

2007 2008 2009 2010 2011

Year

% patients 24 Hours

7 days

Recorded weight, height & BMI

0

20

40

60

80

100

2007 2008 2009 2010 2011

year

% p

atie

nts

weight

height

BMI

Screening 2007-2010

'MUST' screening incidence January 2007 - October 2010

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Jan-07

Mar-0

7

May

-07

Jul-0

7

Sep

-07

Nov

-07

Jan-08

Mar-0

8

May

-08

Jul-0

8

Sep

-08

Nov

-08

Jan-09

Mar-0

9

May

-09

Jul-0

9

Sep

-09

Nov

-09

Jan-10

Mar-1

0

May

-10

Jul-1

0

Sep

-10

Nov

-10

Months

Perc

en

tag

e o

f p

ati

en

ts

Screened with 24

hours of admission

Rescreened within

the previous 7 days

Recording of components of ‘MUST’

Elements of 'MUST' recorded (those rescreened within the previous 7 days) 2010

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Janu

ary

Feb

ruar

y

Mar

chApr

ilM

ay

June Ju

ly

Aug

ust

Sep

tembe

r

Octob

er

Nov

embe

r

Dec

embe

r

Months

Pe

rce

nta

ge

of

pa

tie

nts

% of patients with recorded BMI.

% of patients with a % weight loss recorded.

% of patients with acute disease effect recorded.

MUST >2 (2008-2010)

Incidence of scoring 2 or more on 'MUST' screening tool

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

Jan-

08

Mar

-08

May

-08

Jul-0

8

Sep

-08

Nov-

08

Jan-

09

Mar

-09

May

-09

Jul-0

9

Sep

-09

Nov-

09

Jan-

10

Mar

-10

May

-10

Jul-1

0

Sep

-10

Nov-

10

Months

Perc

en

tag

e o

f p

ati

en

ts

24 hours of

admission

Within the last

7 days

Why ‘MUST’ was not completed?

• Emergence surgery

• Short stay assessment units/ETU

• Transfer of patients

• Application of the disease severity score if weight & heights not recorded

• Poor compliance with screening tool in some areas

• Amputees and BMI

• Confusion & dementia patients

• Rapid discharges within 24-48hrs

• Pre-selecting patients for screening by ward staff ‘eye ball assessment’

Incidence of malnutrition on admission (scores one or more) 2010

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

January February March April May June July August September October November December

Months

% o

f p

ati

en

ts Low risk (score zero)

Moderate risk (score 1)

High risk (score 2 or more)

Screening not completed

Admission score (2010)

Scores at >7 days (2010)

Incidence of malnutrition (scores one or more) at > 7 days - 2010

0%10%20%30%40%50%60%70%80%90%

100%

January

February

March

April

May

June

July

August

Sep

tember

October

November

December

Months

Percen

tag

e o

f p

atien

ts

Low risk (score

zero)

Moderate risk

(score 1)

High risk (score

2 or more)

Screening not

completed

Risk category 24 hours screen

(7,312 pts)

7 days screen

(5,235 pts)

moderate 438 471

high 950 1465

Moderate & high 1388 1936

Numbers of patients

(one audit week/12 months 2010)

Dietetics Dept -2,736 nutrition support referrals

Moderate risk

22 pts

High risk

33 pts

Care plan followed 12 (54) 17 (51)

Food charts completed 14 (63) 19 (58)

Weekly weights 16 (72) 19 (58)

Referred to dietitian - 19 (58)

Actions completed from 100

patient’s care plans audited

Comparison with national survey data

(2010)

BAPEN

2010

7 Days

CMFT

24 hrs

CMFT

Moderate 14 9 6

High 21 19 13

Moderate &

high

34 28 19

Conclusion

• Continual audit has allowed areas in the hospital to be

targeted for specific training.

• Affect of training on screening rates has been

monitored.

• Modern Matron round (Jan 2009) increased the use of

‘MUST’ at ward level improved rates of screening.

• Audit has assisted in identifying the reasons for

missing data.

• Developed net work of Nutrition link Nurses at ward

level

Conclusion cont:

• Helped change the ward culture

• Embedded screening into ward based practice

• Engaged senior nurses in screening programme

• Assisted introduction of ‘MUST’ into pre-operative clinics (ERAS protocols)

Work still to be done

• ‘MUST’ in outpatients patient self

assessment.

• ‘MUST’ care plan development and audit

• QCC report.

• Maintenance of screening programme is a

dynamic process.

Acknowledgments

• Mrs Ruth Brierley Nutrition Nurse

Specialist

• Department of Nutrition and Dietetics

• Nutrition Link Nurses on all the wards

References

1. Council of Europe. (2003) Food and Nutritional Care in Hospitals: How to Prevent Undernutrition. Nutrition Programmes in Hospitals Group for the Committee of Experts on Nutrition, Food & Safety and Consumer Health. Strasbourg: Council of Europe.

2. National Collaborating Centre for Acute Care. (2006) Nutrition Support in Adults: Oral Nutrition Support, Enteral Tube Feeding and Parenteral Nutrition. NICE Clinical Guideline 32, London: National Collaborating Centre for Acute Care.

3. National Patient Safety Agency (2010) Patient Environment Action Team Assessment. National Patient Safety Agency, London.

4. Department of Health (2010) Essence of Care Benchmarking for Food and Drink. Her Majesty’s Stationery Office, United Kingdom.

5. Westergren A, Wann-Hansson C, Bergh Börgdal E, Sjölander J, Strömblad R, Klevsgård R, Axelsson C, Lindholm C Ulander K,. (2009) Malnutrition prevalence and precision in nutritional care differed in relation to hospital volume – a cross-sectional survey Nutrition Journal 8: 1-8

6. Russell C & Elia M., (2008) Nutrition Screening Survey in the UK in 2008. British Association of Enteral and Parenteral Nutrition

Measuring quality of nutritional care through audit:

Developing a national clinical audit of Essence of Care

(Food and Nutrition)

Emma Parsons MNutr RD

Clinical Audit

“Clinical audit is a quality improvement process that seeks to improve patient care and outcomes through systematic review of care against explicit criteria.

Where indicated changes are implemented at an individual, team, or service level and further monitoring is used to confirm improvement in healthcare delivery”

The commission

• A one year development plan to consider how a national clinical audit relating to Essence of Care – Food and Nutrition could be undertaken

• In health and social care settings

• To reflect the requirements of the Department of Health’s Nutrition Summit Stakeholder Action Plan

Objectives • To define the types of organisations

• To identify:

– The process and outcome criteria – The data collection methods

• To carry out a pilot audit

– Testing the feasibility of the audit tools

• To consider: – How the data should be analysed – To consider how the results should be fed back to

participating units

Audit standards

• Standards were produced based on: – National guidance

– Input from the project advisory board

• Covered both health and social care

• Feedback on the standards was sought from: – Project team

– Project advisory board

Essence of Care - Food and Drink Agreed person-focussed outcome (2010):

People are enabled to consume food and drink (orally) which meets their needs and preferences

Factors Promoting health Environment

Information Screening and assessment

Availability

Planning, implementation, evaluation and revision of care

Provision Assistance

Presentation Monitoring

Supporting Guidance

The Nutrition Action Plan, (DH, 2007)

Nutrition Support for Adults, (NICE, 2006)

From Malnutrition to Wellnutrition: Policy to practice, (ENHA, 2006)

Essential Standards of Quality and Safety, (CQC, 2010)

Dignity Factors: Eating and Nutritional Care, (SCIE, 2010)

Managing food waste in the NHS, (DH, 2005)

Patient Environment Action Team Assessments, (NPSA, 2010)

Meeting quality standards in nutritional care, (BAPEN, 2010)

National Service Framework for Older People, (DH, 2001)

Still Hungry to Be Heard, (Age UK, 2010)

10 Key characteristics of good nutritional care, (NACC, 2010)

Project Advisory Board • Key national organisations

• High level practitioners, patient, resident & carer groups

Key nutritional themes

Process and outcomes Communication

Nutritional care Transfer of care

Food service and delivery Personalised approach

Workforce

Audit Tools

• Tools were produced based on the audit standards:

– Organisational

– Staff

– Patients’ / Residents’ experiences

– Patients’ / Residents’ records

Pre – audit feedback on audit tools • Feedback on the audit tools was sought from:

– Project team

– Project advisory board

– Lambeth and Southwark care home support team

– District general hospital’s nutrition team

Hospitals

• Organisational

• Ward

• Staff

• Patients’ experiences

• Patients’ records

Care homes

• Organisational

• Staff

• Residents’ experiences

• Residents’ records

Pilot Sites

Hospitals (n=3) Care Homes (n=4)

Teaching (1) Residential (1)

District Generals (2) Nursing (1)

Dual Registered (2)

Contact with Pilot Sites

June 2011 Contact made via telephone, email

June-July 2011 Initial meeting with managers

July 2011 Ethical approval granted (KCL ethics)

August 2011 Pilot audit started

November 2011 Pilot audit completed

Pilot Audit

• All sites provided with a ‘master pack’ – Instruction sheet

– Information sheets

– Audit tools

• All audit tools were available on the project website

• Audit tools were collected by researchers as sections of the audit were completed

Number of completed audit tools

Hospitals Care homes Total

Organisational 3 4 7

Ward 6 6

Staff 85 36 121

Patients’ / Residents’ experiences

78 74 152

Patients’ / Residents’ records

69 105 174

Total number of completed forms = 460

Co-ordination of the Pilot Audit • Communication with sites during the audit

• Use of GANTT charts to record progress

• Updates during audit:

– Newsletters

– Website (www.kcl.ac.uk/nutrition-audit)

Resource Implications

• Production of audit packs

• Who should complete audit tools?

– Nurses

– Dietitians

– Other?

• Computer Access

• Returning completed audit tools

Practical Constraints

• Time period for the audit

• Time to produce copies of audit materials

• Availability of staff to complete the questionnaires

• Selection of staff, patients and residents

• Consent

• Interruptions

Results – Organisational Level

Present at all sites All sites use ‘MUST’

Hospitals: weekly Care homes: monthly

Hospitals: Staff ask patients Care homes: All record preferences

Hospitals: Use red tray systems

Care homes: Encourage staff and

family to assist

Nutrition Policy

Screening Tool

Needs and Preferences

Assistance with meals

Staff Training: Hospitals

0102030405060708090

RGN Healthcare Assistant

Sufficient Training

Pe

rce

nta

ge

Malnutrition Screening Assistancen=84

Staff Training: Care Homes n=36

0

20

40

60

80

100

RGN Senior Carer Care Assistant

Pe

rce

nta

ge

Malnutrition Screening Assistance

Patients’ & Residents’ Characteristics Patients (n=69) Residents (n=105)

Age (y) 60.4 ± 21.0 79.9 ± 10.8

Gender 50% male, 50% female 42% male, 58% female

05

101520253035

Dementia Diabetes Stroke CVD Cancer

Pe

rce

nta

ge

Patients Residents

Prevalence of Malnutrition

0

5

10

15

20

25

30

35

Medium High At Risk

Pe

rce

nta

ge

‘MUST’ Category

Hospitals Care Homes

Use of Screening Tools & Care Plans

0102030405060708090

Screened Care Plan Specific Goals

Patients Residents

Patients’ & Residents’ Experiences

Patients Residents

Asked about weight loss 27% 48%

Asked about food & drink preferences

26% 80%

Meal service explained to them 33% 81%

24hr Access to drinking water 99% 100%

Able to store food 63% 63%

Summary

• Clinical audit is an important tool in improving care for patients and residents

• Audit design should take into consideration:

– Setting

– Time

– Types of audit tools

– Data collection methods

– Feedback to participating centres

Acknowledgements

Project Team at KCL

Project Advisory Board

All participating pilot sites

Translating Audit Into Improvement

Dr Emma Donaldson

Aims

• A structured approach to clinical improvement

• Using data for improvement

• The importance of variation

• Making change and designing for reliability

• Identifies current performance against agreed standards

• Identifies areas of underperformance that require improvement

• Provides assurance that standards are being achieved

• Identify reasons for underperformance

• Identify interventions which will result in improvement

• Correlate audit results and changes implemented

What audit does and does not do

MAKING IMPROVEMENTS IN CLINICAL SERVICES

If you always do what you have always done, you will always get

what you have always got.

The Model for

Improvement

Improvement from current audit position to attain / exceed gold standard care

AIMS

• S - specific

• M - measurable

• A – attainable (but challenging!)

• R - realistic

• T – time limited

95% patients, admitted to general medical wards, to receive a MUST assessment within 6 hours of admission by April 2011.

A Driver Diagram

• Reinforces the aim statement as the goal

• Clarifies the big picture

• Identifies primary system components

• Aids in development of measurement

Most importantly: Helps teams to articulate their contribution to the overall aim and avoid missing important system components

MUST completion

within 6 hours for 95% of

admissions to L2, L3, L8

Education & leadership

Process standardization

Patient/Carer involvement

Measurement

Ward nurse training Nutritional link nurse Dietitian review

Purple paper work=nutrition MDT meal ward rounds Catering involvement

Relative input at meal time Patient preferences Breakfast Club

MUST compliance (EPR) MUST Accuracy

Question 1 - summary

• Have a SMART aim

• Use a driver diagram to conceptualise change

• Avoid focussing on one driver

The Model for

Improvement Data!

Why are we measuring?

Research Judgement Improvement

Aim New knowledge Achieve a target Improve a service

Testing strategy One large test No tests Sequential tests

Sample Size “Just in case” data Obtain 100% of available, relevant data

“Just enough” data

Hypothesis Fixed hypothesis No hypothesis Flexible hypothesis

Variation (Bias) Design to eliminate unwanted variation

Adjust measures to reduce variation

Accept consistent variation

Is change an Improvement?

Statistical tests No change focus Run charts or Shewhart control charts

Source: Robert Lloyd IHI 2006

Limits of Descriptive Statistics

Average Before=8 hours delay Average After=3 hours delay

Data for Improvement

• Time series • Multiple small datasets • “Real time” • Annotated with changes

tested Ave Days Wait for Colonoscopy

80

90

100

110

120

130

140

150

160

Jan

06

M M J S N J-

07

M M

Av

e D

ay

s

Match demand/capacityConfirmation

Wk backlog

% Patients Waiting for Colonoscopy < 90 Days

30

40

50

60

70

80

Jan

06

M M J S N J-

07

M M

Percen

t

Match demand/capacityConfirmation

Wk backlog

• Demonstrates variation in the system

• Ownership

• Motivate front-line staff

Data for Improvement

• Outcome Measures

– The “big dot”

• Process Measures

– Shown to contribute to the outcome

– Will see changes before the big dot moves

• Balancing Measures

– Avoid unforeseen consequences

• Identify variation in the system

Understanding variation

• The outcome of every process is affected by lots of little things

• Each of these little things varies naturally

• All these little variances add up

• This makes the process vary over time

Time to first dose antibiotics in pneumonia

Wait to see doc Time to get

CXR Time to XR

review Time to

prescription Time to draw up

Time to admin

0

50

100

150

200

250

1 2 3 4 5 6 7 8

Antibiotic drawn up

Antibiotic prescribed

Xray seen

Xray

Porters

Order Xray

Clerk/exam

See doc

0

50

100

150

200

250

1 2 3 4 5 6 7 8

0

5

10

15

20

25

30

Common Cause Variation

• A system can also be affected by a big, unusual influence

• The size of the change produced is BIG in relation to the common cause variances

• It happens much less frequently than the common cause variances

0

50

100

150

200

250

300

350

400

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Antibiotic drawn up

Antibiotic prescribed

Xray seen

Xray

Porters

Order Xray

Clerk/exam

See doc

Major Incident

Special Cause Variation

Question 2 - Summary

• Ensures that the changes are actually improving the system

• Show that the improvements are being sustained

• Motivate staff to continue testing changes and make improvements

• Prove that improvements are occurring (even before the “big dot” moves!

The Model for

Improvement

EVERY SYSTEM IS PERFECTLY DESIGNED TO GET THE RESULTS THAT IT GETS

Paul Batalden

Obstacles to making change

1. More of the same

“This would not be a problem if we had more….” people, money, resource x, etc.

2. Utopia syndrome

Trying to identify the perfect change before doing anything at all – paralysis by analysis

Two Types of Change

• Reactive (first order) – Keep the system running – Solve or react to problems – Return system to prior condition – Trade off among measures – Short term

• Fundamental (second order) – Create a new system – Design or redesign – Necessary if you are to improve beyond problems – Impacts several measures – Long term

The PDSA Cycle Plan • What are you going to test? • What do you predict will happen? • Develop the test (Who? What? When? Where? Data?)

Do • Try out the test on a small scale • Observe & document results

Study • Analyse data • Study the results • Compare results & predictions

Act What will you do next? • Adapt • Adopt • Abandon

Why test change before implementing it?

• It involves less time, money and risk

• The process is a powerful tool for learning; from both ideas that work and those that don't

• It is safer and less disruptive for patients and staff

• Because people have been involved in testing and developing the ideas, there is often less resistance

Multiple PDSA Cycle Ramps

Primary Drivers

• “Human beings are fallible” • “They make mistakes even when they’re trying to do

the right thing and even when they think they’re doing the right thing.”

• He estimates that, in health care delivery, a defect

(error / omission / other failure to accomplish an intended action) occurs, on average, 10 to 20 percent of the time

• This compared to .0001 percent of the time for airlines and nuclear power plants

Roger Resar, MD, Senior Fellow at the Institute for Healthcare Improvement (IHI)

The Critical Question

“Why, with all the good intentions and talent available in medicine, are clinical processes backed by solid medical evidence carried out at such low

levels of reliability?” • Few people come to work with the intention of

performing poorly

• Although the system may be defective, this answer may not be particularly helpful in detailing how to improve the clinical processes

IHI’s Learning:

• Current improvement methods in health care are excessively dependent on vigilance and hard work

• The current practice of benchmarking to mediocre

outcomes in health care tends to give clinicians and leaders a false sense of process reliability

• A permissive attitude toward clinical autonomy

creates and allows for wide, and unjustifiable, performance variation

• Processes are rarely designed to meet specific,

articulated reliability goals

Improvement Concepts to get you to 80–90% reliability

• Common equipment, standard order sheets, multiple

choice protocols, and written policies/procedures

• Personal check lists

• Feedback of information on compliance

• Suggestions of working harder next time

• Awareness and training

(Primarily can be described as intent, vigilance, and hard work – basic failure prevention, identification and mitigation)

Improvement Concepts to get you to 95% reliability

• Decision aids and reminders built into the system

• Desired action the default (based on scientific evidence)

• Redundancy utilised

• Habits and patterns known and taken advantage of in the

design

• Standardisation of process based on clear specification and

articulation is the norm

(Uses human factors and reliability science to design sophisticated failure prevention, failure identification, and mitigation)

Question 3 - summary

• Not all change will result in improvement so changes must be tested first

• Multiple tests can take place simultaneously

• Hard work and vigilance, although commendable, are not good design principles

• If 95% reliability change concepts do not make up at least 25% of the improvement effort on a given project the team should rethink the design

• Education & awareness alone are not enough!

Key points

• Good audit vital for assurance

• Identifies areas for improvement efforts

• Need to undertake further investigation to explain why a system is underperforming

• Making change requires a similarly robust framework to be successful

• Using data is key to improvement efforts

• Test changes before implementing

• Design for reliability

Thank you