the effectiveness of healthcare regulation: lessons from research

The effectiveness of healthcare regulation: lessons from research7th EPSO conference – Cork, 15 May 2009

Professor Kieran WalsheManchester Business [email protected]

mailto:[email protected]

Overview

• Defining regulation, and the growth of regulation in healthcare

• Thinking about regulatory agencies and processes: theories and frameworks

• The impact of regulation: some ideas and a case study

• Responsive regulation and regulation for quality improvement

• Conclusions

Defining regulation

“Sustained and focused control exercised by a public

agency over activities that are valued by a community”

(Selznick 1985)

• Public interest purpose

• Regulatory agency

• Formal regulatory powers and processes

• Economic and/or social regulation

People Technologies

Organisations

General Medical Council

General Dental Council

General Optical Council

Health Professions Council

Nursing and Midwifery Council

Council for Healthcare Regulatory Excellence

Healthcare Commission

Audit Commission

Mental Health Act Commission

National Institute for Clinical Excellence

European Medicines Evaluations Agency

Medicines and Healthcare Products Regulatory Agency

Equality and Human Rights CommissionHealth and

Safety Executive

EnvironmentAgency

The international regulatory landscape

http://www.jointcommission.org/

http://www.healthcarecommission.org.uk/

http://www.has-sante.fr/portail/index.jsp

http://www.ncqa.org/

http://www.audit-commission.gov.uk/

What is regulated?Healthcare 582 NHS organisations, 1,426 independent

healthcare providersSocial care 150 local authorities, 26,000 care providers

School education

23,000 maintained schools; 1,120 independent schools (also 358 FE colleges; 100,000 childcare providers)

Housing 2,000 registered social landlords (housing associations)

Criminal justice 43 police forces; 42 criminal justice boards; 139 prisons

Environment Major pollution risks to air, water and land

Health and safety

Workplace health and safety in mines, factories, farms, hospitals, schools, railways,…

What does it cost?Healthcare Healthcare Commission has 600 staff and costs

£63m paSocial care CSCI has 2,500 staff and costs £159m pa

School education

OFSTED schools inspections cost £73m pa

Housing Housing Corp has 500 staff and costs £42m pa

Criminal justice HMIC £8.2m; HMIP £3.2m; HMIP £2.6m – total of c250 staff

Environment EA has 11,296 staff and costs £470m pa

Health and safety

HSE has 4,000 staff and costs £284m pa; 1,100 enforcement staff in LAs

The growth of regulation: likely causes

• Societal expectations and changing attitudes – to risk, quality and safety in healthcare

• Diminished trust in health professions and organisations – the rise of the “audit society”

• Evidence of poor performance - regulation as a political response to scandals and failures in care

• Changing ownership and structure of healthcare industry – from state to private/corporate, from central planning and direction to market/competition

Questions, questions…• Does regulation work?

• Why and how does it change organisations’ behaviour and performance?

• What are the principles of effective regulation?

• What can we learn from how regulation is done in other countries or other sectors?

• Why regulate in the first place?

• How should we design the “regulatory regime”?

More questions...

• Should we set standards or rules for people to follow? About what? How detailed? How strict?

• How do we decide who can do the business? Do we licence or certify people to operate? How?

• What shall we inspect? How often do we need to inspect or check up on people? How do we do it?

• What do we do when we find people are not performing or are breaking our rules? When should we use sanctions or punishments?

• How do we show that what we are doing is achieving something (and what is it we are trying to achieve?)

Some theoretical perspectives

• Public choice theory

• Public interest or interest group theory

• Institutional theory

• Principal agent theory

• Game theory

Considerations for regulatory regimesPublic choice theory •Burden

•Capture!

•Life cycleInterest group theory •Stakeholder governance

Institutional theory •Organisations and their responses

•Environmental pressures

•Smart/responsivePrincipal agent theory •Regulator/regulated imbalance

Game theory •“Win-Win”

Framework for analysing regulation

• Policy purpose or objectives

• Regulatory body or agency

• Range and scope of regulated activities

• Regulatory regime

Policy purpose or objectives

• Reasons to regulate: market failure, social goals and organisational characteristics/design– Improving performance – efficiency, effectiveness at an

organisational or a system level

– Making regulated organisations accountable – empowering consumers,

– Providing information (“sunshine regulation”) – which others use in making decisions

• But – objectives conflict; must be negotiated; are one among many influences

Regulatory bodies or agencies: autonomy and accountability

• Three basic types of agency:– Part of a government department (eg HMIC, HMIP)– An arms-length agency/NDPB (eg OFSTED, Healthcare

Commission)– An autonomous statutory body (eg GMC)

• What statutory powers and authority does it have?• How is it resourced – from grants/fees/levies?• How is it governed – board/management arrangements?• How and to whom is it accountable?

Regulated organisations and activities

• Scope of regulation – vertical and horizontal

• Number and size of regulated organisations

• Nature of regulated organisations and their environment

• Heterogeneity, and ideas of “amoral calculators” and “good hearted compliers”

Regulatory regime

• Direction – system level actions like publishing standards or directives, setting rates/tariffs, etc

• Measurement – organisational level actions like self-assessments, inspections, permits/authorisations, investigations

• Enforcement – publication of information, recommendations for action, fines/penalties/sanctions, incentives and awards, interventions in management, licensing/delicensing

The impact of regulation: perceptions

Providers’ views• Regulatory

“burden” and overregulation

• Multiple, duplicated oversight by regulators

• Restrictive, rule-driven regime

• Disconnected from improvement

Public views• Close and cosy

relationship between regulator and providers

• Undemanding oversight, feeble sanctions

• Window-dressing for the industry and professions

Regulator’s views• Necessary,

effective and focused oversight

• Key driver for improvement – would not happen otherwise

• Limited regulatory resources constrains what can be achieved

How regulation has an impact

• Direction – regulator issues rules/requirements and organisations comply of their own accord

• Measurement – prospect of being assessed or inspected stimulates change in organisation; results of measurement prompt reflection and change

• Enforcement – recommendations from regulator, sometimes backed by sanctions or incentives

• Indirect effects – other stakeholders pay attention to information from regulator in making their decisions

Positive and negative impacts

• Specific changes

• Organisational reflection

• Priority to important but non-urgent

• Leverage for internal groups

• Ratchet effect of raising standards

• Temporary improvement not sustained

• Pointless conformance• Defensive or minimal

compliance• Creative compliance• Inhibiting innovation• Distorting internal

priorities• Opportunity costs

Common regulatory problems

• Regulatee resistance

• Ritualistic compliance

• Regulatory collusion and capture

• Measurement ambiguity and problems

• Goal displacement

• Regulatory proliferation and growth

• Regulatory rigidity, juridification and inflexibility

Problems knowing what is effective regulation

• Methodological – difficulties of measurement, causation and interpretation

• Practical – regulatory agencies find it hard to experiment and innovate, are often risk averse

• Cultural – regulators and other stakeholders lack an evaluative culture and aren’t necessarily interested in finding out what works

• Political – purpose of regulation is both symbolic and functional, its not just about technical effectiveness

The impact of regulation: evidence from MARQUIS

• Survey of 69 expert informants in 24 of 25 countries of the EU – focused on national policies and strategies for quality improvement

• Survey of 389 hospitals across 8 countries and site visits to some hospitals, gathering data at organisation, department and clinical service level

• Key conclusions – external pressure from regulation, accreditation etc linked to more developed/mature QI systems at organisational level, linked to better quality care at clinical level

The impact of regulation: a case study of the Commission for Health Improvement

• Range, nature and appropriateness of recommendations in CHI reviews

• Appropriateness, resource implications and viability of Trust action plans

• Extent of implementation of action plans, barriers to progress, future expectations

• Impact on trust organisation and patient care

Benson LA, Boyd A, Walshe K. Learning from regulatory interventions in healthcare: The Commission for Health Improvement and its clinical governance review process. Clinical Governance: An International Journal, 2006; 11(3):213-224.

Research methods

• Selected sample of 30 NHS trusts which underwent clinical governance review in 2001 or 2002

• Existing data: review report, action plan, StHA progress review

• New data: questionnaire survey and case study interviews

• Finding, categorising, tracking and following up recommendations – “key areas for action” or KAAs

CHI reports varied greatly in structure, content and presentation

• Variation – from 20 to 80 pages, from 5 to 57 KAAs, differing structures and designs, different focus and emphasis

• Causes of variation

0

10

20

30

40

50

60

11 26 19 8 10 20 9 30 21 1 17 22 13 23 16 18 15 5 25 14 2 6 27 28 7 4 3 12 24 29

NHS trusts

Num

ber o

f KA

As

Unique Duplicate

Widespread acceptance of CHI’s recommendations

• Generally confirmatory – NHS trusts agreed wholly or partly with 92% of KAAs in survey; 77% of issues raised before in trust

• Recommendations “really came from us”. Trust had “no qualms”. Report was “fair and not surprising”. “Report … did not find anything that was previously not known”

Most of CHI’s recommendations focused on systems, processes and management

72%

11%

5%

2%

10%

Strategic/board level

Management/administrative

Patient care system/process

Environment/facilities

Patient experience/outcomes

Most of CHI’s recommendations focused on systems, processes and management

13%

9%

15%8%

8%

13%

16%

11%7%

Patient involvementRisk managementClinical auditStaffing/managementEducation/trainingClinical effectivenessUse of informationUser experienceStrategic capacity

The nature of CHI’s recommendations and how they were expressed varied greatly

• Type, level and measurability of change

• Clarity of purpose (what needs to be achieved) and prescription (what should be done)

46%

37%

3%14%

Clearly measurable Largely measurableMight be measurable Not measurable

NHS trust action plans also variable in structure, content and presentation

• Variation – from 4 to 89 pages, 25 to 247 action points• Some tackled recommendations embedded in report

text, others didn’t• Structure and ordering variable, hard to connect with

KAAs in report, easy to miss some KAAs• Some KAAs reworded or edited


No of action points in action plan

0

50

100

150

200

250

8 19 9 22 20 13 1 11 21 5 15 17 18 30 25 16 7 4 6 23 27 10 14 26 2 24 12 3 28 29

NHS trusts

Num

ber o

f act

ion

poin

ts


No of action points per KAA

0

5

10

15

20

4 7 21 13 1 18 15 22 8 25 14 20 6 24 27 19 23 16 12 17 5 30 9 2 29 28 3 10 11 26

NHS trusts

Mea

n no

of a

ctio

n po

ints

pe

r KA

A


Percent of KAAs not addressed

0

10

20

30

40

50

14 8 19 9 20 1 11 21 17 18 25 4 6 23 27 26 2 24 12 28 29 3 30 22 10 15 7 16 13 5

NHS trusts

% o

f KA

As

Action plans mostly addressed CHI’s recommendations

17%

32%

46%

5%

Not addressed

Partly addressed

Mostly addressed

Completely addressed


Action points per KAA

0

50

100

150

200

1 2 3 4 5 6 7 8 9 10 11+

Number of action points

Num

ber o

f key

are

as fo

r ac

tion


Timescale for completion of action plans

0

50

100

150

200

1 2 3 4 5 6 7 8 9+

Timescale for completion (in quarters)

Num

ber o

f key

are

as fo

r ac

tion

Most of CHI’s recommendations have been acted upon

Data from stakeholder survey14%

40%

41%

4% 1%

Little or no action

Some implementation

Mostly implemented

Fully implemented

Don't know


Data from StHA progress reviews

22%

30%15%

31%

2%

Little or no action

Some implementation

Mostly implemented

Fully implemented

Not reported on


Data from StHA progress reviews, excluding KAAs not reported on

32%

43%

22%3%

Little or no action

Some implementation

Mostly implemented

Fully implemented


Data from StHA progress reviews

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Patient involvement

Risk management

Clinical audit

Staffing/management

Education/training

Clinical effectiveness

Use of information

User experience

Strategic capacity

Not reported No action Some implementationMostly implemented Fully implemented


% KAAs not reported on

01020304050607080

8 9 11 18 21 22 25 26 28 3 23 7 17 27 10 16 15 30 20 6 2 29 12 14 24 4 19 13 1

NHS trusts

% o

f KA

As

not r

epor

ted

on


0102030405060708090

10 19 1 29 12 21 16 24 28 4 2 15 6 5 30 18 14 20 9 13 8 23 17 27 3 7 22 26 11 25

NHS trusts

% o

f KA

As

fully

impl

emen

ted

There are some important factors involved in securing implementation

Correlated:• Measurability of KAA• Whether action plan addresses

KAA• Timescale for action plan• Clinical governance review area• Trust• NHS trust star rating

Not correlated:• Whether KAA is a priority

• Depth, breadth, type of change

• No of action points

• Clarity of timescale


Stakeholder views of what caused change

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

NHS trusts

PCTs

StHAs

All

CHI Don't know Other factors


• “But where they scored (meaning CHI) was things like they said we needed to do a staffing review, in fact, we had done it before we had the CHI review, but they brought it up in the report which enabled us to use it in our SAFF negotiations to get further investment in staffing. So we used it again as a lever for change but also a lever within the SAFF process…”

• “What made a big difference in this organisation was seeing it in black and white in terms of lack of leadership and strategic direction and again once the execs. and board had been on that (development) programme they had a better understanding of what was expected of them and in terms of what the organisation had to do as well. I think that has made a big difference in the review of arrangements for clinical governance in the organisation…”


• “Some new things for the Trust certainly,. The public involvement stuff… without the focus of the CGR they would not have made much progress with that. The concerns about Cardiology were raised by the Deanery before ……but the review did act as a catalyst to the patient and public stuff in particular.”

• “The review and report helped us to win over doubting clinicians, because it was not a “tick box” exercise. This was the biggest impact. Consultants who thought it was a passing fad now realise that CHI, CNST etc are part of the regulatory framework, so we/they need to be consistent about clinical governance – it is fundamental.”

Policy implications from the study

• Invest more regulatory attention and effort across the regulatory cycle

• Make regulatory interventions more consistent in approach

• Ensure all recommendations are constructed, expressed and communicated clearly

• Require detailed, explicit and comprehensive action plans, focus followup on the original review

• Shorten the regulatory cycle of attention, intervention and action

Responsive regulation• Regulatory regime recognises and responds to the

diversity of regulated organisations

• Regulatory interactions and interventions are highly contingent on how individual organisations behave

• “One size fits all” approaches are eschewed in favour of more flexible and graduated approaches

• Regulators and regulatory staff have considerable freedom to exercise judgement

(Ayres and Braithwaite 1992)

Responsive regulation Deterrence Compliance Regulator’s view of regulated

Amoral, out to get all they can, untrustworthy

Mostly good and well intentioned, not always competent

Regulated view of regulator

Policeman, enforcer, feared and disliked

Consultant, supporter, not a threat

Temporal perspective

Retrospective (find problems, punish and deter)

Prospective (prevent problems, promote improvement)

Regulatory methods Detailed written standards, formal inspection, routine use of penalties and sanctions

Standards supported by guidance, developmental approach to inspection, sanctions as a last resort

Relationship Distant, formal and adversarial

Close, friendly and co-operative

(Reiss 1984)

Responsive regulationStyle of regulation

Organisation

Deterrence Compliance

Amoral, out to get what they can

Tough stance pays, but organisations fight back too?

Organisations take regulator for a ride? Or rise to meet expectations?

Good hearted and well intentioned

Organisations bewildered and hurt by punitive stance? Become hard nosed?

Collaboration pays, and improvement results?

Responsive regulation

• Contingency in approach/methods• Hierarchy of regulatory intervention• Flexibility – scope for discretion• Tripartism – co-opting stakeholders• Parsimony – minimising intervention• Empowerment – supporting improvement

Walshe (2003)

A hierarchy of regulatory interventions

Closure or removal of licence; replacement of management

Detailed ongoing supervision or inspection; financial or other penalties;

limitations to activities or areas of work

Formal requirement to remedy problems; repeat or follow-up inspections; disclosure of findings to others;

referral to support or other agencies

Informal intervention to deal with minor problems; limited follow-up inspection; positive feedback on achievements and strengths

Granting of greater autonomy; relaxing of regulatory regime; financial incentives or rewards for good performance; public recognition of achievements; leading role in

transferring good practice to other organisations

Regulating for improvement

• Direction – setting detailed standards or high level principles with scope for interpretation

• Measurement - changing the intensity or nature of oversight (self-assessment, inspection holidays, shorter visits, etc)

• Enforcement - granting (or withholding) greater freedoms/powers, making remuneration partly dependent on regulatory status or performance

• Using successful, high performing organisations or other stakeholders in the regulatory process


• Information – need for high quality and timely data to make responsive regulatory decisions

• Regulatory agency staff – need for content knowledge currency and credibility, plus process skills in interaction and professional judgement

• Regulatory process design – need for discretion and flexibility allowed in its implementation, balanced with need for accountability and fairness


• The extent, motivations and nature of intrinsic regulatory compliance – and the place of extrinsic compliance?

• Longitudinality of relationship and knowledge crucial – its about feedback and interaction

• The setting of high moral expectations in the relationship – appealing to better nature

• The ability to be helpful, supportive, facilitative – and directive, interventionist and punitive

Conclusions

• Regulation works – it drives change in healthcare organisations, but it can have negative as well as positive impacts

• Research has a lot to offer in helping to develop and evaluate effective regulatory strategies – and has been underused to date – much regulation is not “evidence-based”

• Responsive regulatory strategies offer a real opportunity to make more effective use of regulation to support and reinforce organisational quality improvement efforts

the effectiveness of healthcare regulation: lessons from research

Documents