health outcomes monitoring toolkit: pilot results€¦ · health outcomes monitoring toolkit –...
Post on 29-Jul-2020
6 Views
Preview:
TRANSCRIPT
Health Outcomes Monitoring Toolkit: pilot results
March 2014
Executive Summary
Health Outcomes Monitoring Toolkit – Pilot Results March 2014 | 1
Contents
Executive Summary 2
Introduction 6
Two-stage survey approach 11
Single survey approach 21
Appendix: summary data tables 28
Executive Summary
Health Outcomes Monitoring Toolkit – Pilot Results March 2014 | 2
Executive Summary
Background
The Health Outcomes Monitoring toolkit
This document presents the results of a pilot of a health outcomes monitoring toolkit developed by Citizens Advice.
The purpose of this toolkit is to enable advice agencies to measure the impact of advice on health and well-being and the social determinants of health.
The toolkit includes guidance on the research process, questionnaires and an Excel tool for storing and analysing the data. The areas covered by the toolkit are:
• Client profile – including health and disability status and demographic characteristics such as ethnicity
• Mental well-being – measured using the shorter Warwick-Edinburgh Mental Well-being scale (WEMWBS) scale1
• Determinants of health – including debt, finances, housing and employment
Following consultation with bureaux, two versions of the toolkit were tested in the pilot:
• Single-stage approach: this involves conducting a one-off survey with clients between four to six weeks after advice
• Two-stage approach: this involves conducting an initial survey at the time of advice and a second survey four and six weeks after advice
Details of the pilot study
The toolkit was piloted by eight Citizens Advice Bureaux (CAB) from September to December 2013. The aim of the pilot was to test whether the toolkit is effective and practical, and to inform the development of a generic toolkit for the wider advice sector.
The breakdown of the survey approached used, and samples achieved, is set out below.
1 The Warwick-Edinburgh Mental Well-being Scale was funded by the Scottish Executive National
Programme for improving mental health and well-being, commissioned by NHS Health Scotland, developed
by the University of Warwick and the University of Edinburgh, and is jointly owned by NHS Health Scotland,
the University of Warwick and the University of Edinburgh.
Executive Summary
Health Outcomes Monitoring Toolkit – Pilot Results March 2014 | 3
• Single-stage approach: piloted by 5 CAB and achieving a sample of 214 clients
• Two-stage approach: piloted by 3 CAB and achieving a sample of 143 clients
Key messages
Overview: CAB services promote mental well-being and help to tackle some of the causes of health inequality
• Before advice, clients had lower than average mental well-being – but after receiving help from CAB, levels of mental well-being improved significantly
• After receiving advice from CAB, many clients reported positive improvements in their debt situation
• Some clients also reported improvements in their financial circumstances and
housing situation
CAB clients experience health problems and poverty
• Many CAB clients are either disabled people, people with learning difficulties, or people with a long-term condition or mental health problems
• Most also had a monthly household income of less than £800
• 2 in every 3 clients experienced fuel poverty before advice
Clients’ mental well-being improved after receiving advice from CAB
• The results from the two-staged approach offer the best evidence for the impact of advice on mental well-being
• On a scale ranging from 7 to 35, clients’ mean WEMWBS score before receiving advice was 19.15 – a full six points below the UK population average of 25.18
• Four to six weeks after receiving help from CAB, clients’ mean WEMWBS score
increased significantly to 24.67 – just short of the UK population mean of 25.18
Executive Summary
Health Outcomes Monitoring Toolkit – Pilot Results March 2014 | 4
Clients’ debt situations improved after receiving advice from CAB
• Almost 1 in 2 clients reported an improvement in their debt situation after advice
• More than 1 in 4 clients achieved a tangible debt outcome, such as debt rescheduled or debt written off
• More than 1 in 4 clients felt their understanding of managing their finances had improved after advice
Some clients improved their financial circumstances and housing situation after receiving advice from CAB
• Some clients reported improvements in their income and expenditure after receiving advice from CAB
• A small group of vulnerable clients who were either homeless or at risk of becoming homeless also reported that their housing situation had improved
• There was little evidence of a general effect on unemployment or risk of unemployment; it should be noted however that supporting transitions to employment is not a direct objective of the service
• The relationship between advice interventions and some of these broader social determinants of health requires further research
Conclusions and recommendations
The toolkit is an effective resource
• The results presented in this report suggest that the toolkit is effective in providing evidence of the impact of advice on health and poverty
• It is worth noting, however, that achieving suitable sample sizes is a considerable challenge for advice agencies with stretched resources
The two-staged approach is recommended
• The two-staged approach provides the most compelling evidence because it enables comparison of clients’ mental well-being before and after advice
Executive Summary
Health Outcomes Monitoring Toolkit – Pilot Results March 2014 | 5
• Although this requires more resources to be invested upfront, it is likely to provide a
more useful set of results for the agency
Evidence can help build a theory of change
• These results show a much clearer impact on mental well-being and debt problems than broader determinants of health such as employment circumstances
• This may have been influenced by the measures used – for example, the focus of the toolkit on measuring changes in employment status may have meant that more nuanced changes in employability were not captured
• These results may also reflect genuine differences in the impact that an advice
agency has on different social determinants of health – for example, an advice agency has scope to improve clients’ financial capability but little leverage over the energy costs that are a major factor in fuel poverty
• In addition to providing evidence of the impact of the advice, the toolkit can therefore
help agencies to reflect on the degree of change they can expect to bring about across different aspects of clients’ lives
Research can help us to understand change
• Although the two-stage approach helps to measure change before and after advice, further research is required to assess the causes and process of change
• The first priority is to develop the evidence base through conducting further research among a larger sample of clients – this would enable more robust conclusions to be drawn, and a firmer basis for comparative analysis across client groups and impacts
• A second priority is to use qualitative methods to gain some insight into the process
of change from clients’ perspectives, and to identify the reasons why change may occur in some instances and not others
• Funders could play a role in providing resources and support for advice agencies to
conduct further research in both of these areas
Introduction
Health Outcomes Monitoring Toolkit – Pilot Results March 2014 | 6
Introduction
About this document
This document presents the results of a pilot of a health outcomes monitoring toolkit developed by Citizens Advice. The purpose of this toolkit is to enable advice agencies to measure the impact of advice on clients’ health and well-being and the social determinants of health. This in turns provides evidence of the role for advice agencies in tackling health inequalities.
As this report focuses on the results recorded, it demonstrates the type of evidence that the toolkit can be used to collect. For a technical evaluation of the toolkit, please visit: www.citizensadvice.org.uk.
The development of this resource was part of a broader project led by Citizens Advice to enable Health and Well-being Boards and new Clinical Commissioning Groups to make use of evidence from voluntary sector advice agencies. As part of this project, Citizens Advice developed a Health and Poverty report that enables each Citizens Advice Bureau to profile the social determinants of health in their local area.
The health outcomes monitoring toolkit can provide evidence of the impact of a bureau’s service on most of these determinants of health, although some areas (such as domestic violence) are not covered. For further information about this project, or to access a generic version of the Health and Poverty report, please visit: www.citizensadvice.org.uk.
The health outcomes monitoring toolkit
The health outcomes monitoring toolkit provides a set of resources to help advice agencies collect this evidence through a basic client survey. This includes guidance on the research process, questionnaires and an Excel tool for storing and analysing the data. The areas covered by the toolkit are discussed below.
Client profile The toolkit covers a range of client profile criteria relevant to health and poverty. This includes disability and health status and demographic characteristics such as ethnicity. Mental well-being:
Subjective well-being is now widely considered an important component of health policy.2 Improved well-being can have a positive impact on health, for example by increasing longevity, improving recovery from illness and supporting healthy behaviours. As discussed further below, the health outcomes monitoring toolkit uses the Warwick-Edinburgh Mental Well-being scale (WEMWBS) to measure clients’ mental well-being.
2https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/277566/Narrative__January
_2014_.pdf
Introduction
Health Outcomes Monitoring Toolkit – Pilot Results March 2014 | 7
Determinants of health: before and after advice The toolkit covers a range of factors that can affect clients’ health and well-being before they receive advice, including:
• Finances: income profile (poverty) including detail on energy costs (fuel poverty) and number of dependent children (child poverty)
• Debt: whether clients are struggling to repay debt • Housing: whether clients are homeless or at risk of homelessness • Employment: whether clients are unemployed or at risk of becoming unemployed
The toolkit includes a corresponding set of questions to assess any changes in these factors after clients have received advice, for example:
Finances – whether clients have improved their understanding of managing their money, increased their income or decreased their expenditure
Debt – whether clients have managed to achieve a debt outcome, such as having debt written off or repayments rescheduled
Housing and employment – whether clients who were in a disadvantaged position before advice have managed to improve their situation
For full details on the topics covered, please refer to the questionnaires in the generic version of the toolkit at: www.citizensadvice.org.uk/
The WEMWBS scale of mental well-being
The toolkit uses the Warwick-Edinburgh Mental Well-being scale (WEMWBS) to measure clients’ mental well-being.3 WEMWBS is a validated scale for assessing mental well-being in the adult population used by health researchers across the UK.
The health outcomes monitoring toolkit uses the 7-item shorter WEMWBS scale rather than the full 14-item version.4 This consists of a set of 7 positive statements about mental well-being (e.g. “I’ve been feeling useful”, “I’ve been feeling relaxed”) that research participants respond to using an option on a graduated Likert scale (e.g. “Rarely”, “Often”). Each option on the scale is assigned a value.
As shown in the example below, the responses given by research participants to each statement can be used to derive total value. This can range from a minimum of 7 to a maximum of 35.
3 For further information, see: http://www.healthscotland.com/scotlands-health/population/Measuring-
positive-mental-health.aspx 4 For further information, see: http://www.healthscotland.com/documents/5238.aspx
Introduction
Health Outcomes Monitoring Toolkit – Pilot Results March 2014 | 8
The first wave of Understanding Society, the UK household longitudinal study, provides a population average for the shorter WEMWBS scale. Based on a sample of over 38,000 observations, this study recorded a mean value of 25.18.5 This provides a useful comparison for the mean scores recorded using the health outcomes monitoring toolkit.
The relationship between mean WEMWBS scores and mental well-being can be illustrated using hypothetical responses. The graph below shows the mean score that would be achieved if clients gave an identical response to each of the 7 items on the shorter WEMWBS scale.
For example:
Minimum: a score of 7 for a client responding ‘None of the time’ to each item
Maximum: a score of 35 for a client responding ‘All the time’ to each item
On this basis, a client with an average level of mental well-being would give a response between ‘Some of the time’ and ‘Often’ for each item. In practice, clients can give different responses for different items, but this method can be used to illustrate mean values.
5 https://www.understandingsociety.ac.uk/documentation/mainstage/dataset-
documentation/wave/1/datafile/a_indresp/variable/a_swemwbs_dv
WEMWBS statements Response Value
I’ve been feeling optimistic about the future None of the time 1
I’ve been feeling useful Rarely 2
I’ve been feeling relaxed Some of the time 3
I’ve been dealing with problems well Often 4
I’ve been thinking clearly All of the time 5
I’ve been feeling close to people None of the time 1
I’ve been able to make my mind up about things Rarely 2
Total value 18
Introduction
Health Outcomes Monitoring Toolkit – Pilot Results March 2014 | 9
Single and two-stage approaches
Following consultation with Citizens Advice Bureaux participating in the pilot, two versions of the health outcomes toolkit were developed.
The single-stage approach involves conducting a one-off survey with clients between four to six weeks after advice. To allow agencies to make a comparison between before and after advice, clients are asked to give both retrospective and current information about determinants of health.
The two-stage approach involves conducting two related surveys on separate occasions: one at the time of advice and a second during a later follow-up interview between four and six weeks after advice. This allows comparison of two sets of information about determinants of health from before and after advice.
It was acknowledged at the outset that there are strengths and weaknesses with both approaches. The single-stage approach is less resource-intensive for advice agencies to implement, but is limited because it relies on client recall of their situation before advice and includes a single measure of mental well-being. By contrast, the two-stage approach includes two data points for each client, enabling firmer comparison of the change from before to after advice, but is more resource-intensive to implement.
7 12 17 22 27 32
None of the time
Rarely
Some of the time
Population mean
Often
All of the time
Mean WEMWBS score
Mean WEMWBS scores and mental well-being
Health Outcomes Monitoring Toolkit – Pilot Results March 2014 | 10
Pilot details
The aim of the pilot project was to:
• test whether the toolkit offers an effective and practical way to assess the impact of advice on health and poverty
• identify the strengths and weaknesses of the toolkit in order to inform the development of a generic toolkit for the wider advice sector6
The toolkit was piloted by eight Citizens Advice Bureaux from September to December 2013. The breakdown of the survey approached used, and samples achieved, is set out below.
Approach Number of bureaux Total sample achieved
Single stage approach 5 bureaux 214 clients surveyed
Two stage approach 3 bureaux 143 clients surveyed
The majority of bureaux adopted the single stage approach, which suggests that limitations on resource for client research are an important factor.
In addition, the modest response rates reflect feedback from bureaux that conducting follow-up research with clients can be challenging and resource intensive.
6 To access this toolkit, see: www.citizensadvice.org.uk/
Two-stage survey approach
Health Outcomes Monitoring Toolkit – Pilot Results March 2014 | 11
Two-stage survey approach
Sample and response rate
Three of the eight Citizens Advice Bureaux piloting the toolkit opted for the two-stage approach. Across these three bureaux, 216 clients were surveyed at stage one, with 143 continuing on to stage 2. This gives a sample of 143 clients for analysis.
The two-stage survey covered 34 core questions. The average response rate across all of these questions was 89%.
The results below are aggregated from the 3 bureaux piloting the two stage approach.
Respondent profile
Disability and health status was recorded for 143 clients. 72 clients in the sample (50%) were either disabled people, people with learning difficulties, or people with a long-term condition or mental health problems.
This is significant proportion considering that these groups make up around 38% of Citizens Advice clients across the service, and around 21% of the population of England and Wales.
Black, Asian and Minority Ethnic (BAME) clients made up around 6% of the sample. This is slightly lower than the proportion of BAME clients across the Citizens Advice service as a whole.
7%
1%
25%
17%
50%
Disability and health status - would you describe yourself as...?
Having a disability
Having a learning disability
Having a long-term health condition
Having a mental health condition
Not disabled/no long-term healthconditions
Two-stage survey approach
Health Outcomes Monitoring Toolkit – Pilot Results March 2014 | 12
Mental well-being
Key message: clients’ mental well-being improves significantly after they have
received advice from CAB
The shortened version of WEMWBS was used to score individuals across the seven items related to mental well-being. For the two-stage approach, clients were scored twice: before advice and during a follow-up survey four to six weeks after advice.
On a scale ranging from a minimum 7 to a maximum of 35, WEMWBS scores were collected for 138 clients at both stages of the survey.
Before CAB advice: low than average levels of mental well-being
Before advice, clients recorded a mean score of 19.15 – a full six points below the UK population average of 25.18.
Matched WEMWBS scores for before advice (n=138)
Mean 19.15
Median 18.00
Mode 21.00
Std. Deviation 6.87
Confidence Level (95%) 1.16
7 12 17 22 27 32
Before CAB advice
Population mean
Mean WEMWBS score
Mean WEMBS scores before advice
Two-stage survey approach
Health Outcomes Monitoring Toolkit – Pilot Results March 2014 | 13
After CAB advice: mental well-being improves after advice
After advice, clients recorded a mean score of 24.67. This is an improvement of over 5 points from the mean value recorded before advice, and brings the average level of mental well-being close to the UK population mean of 25.18.
Matched WEMWBS scores for after advice (n=138)
Mean 24.67
Median 25.00
Mode 21.00
Std. Deviation 7.10
Confidence Level (95%) 1.19
A paired samples t-test was conducted to compare the WEMWBS scores of these 138 clients at each stage. The test showed that there was a significant difference in scores recorded at stage one and those recorded at stage two.
The specific causes of this improvement in mental well-being were not explored in this research. However, in response to a question on subjective attribution, 91% of clients said that they did not think that things would have got better if they had not sought advice from CAB.
7 12 17 22 27 32
Before CAB advice
After CAB advice
Population mean
Mean WEMWBS score
Mean WEMWBS scores: before and after CAB advice
Two-stage survey approach
Health Outcomes Monitoring Toolkit – Pilot Results March 2014 | 14
Improved WEMWBS scores – what difference does it make?
As discussed above, we can illustrate the relationship between mean WEMWBS scores and mental well-being using hypothetical responses.
The graph below shows the mean score that would be achieved if clients gave identical responses to each of the 7 items on the shorter WEMWBS scale. In this case, we have added the mean values recorded before and after advice in the two-stage results.
In this scenario:
• Before advice: a hypothetical client would be most likely to say ‘Rarely’ or ‘Some of the time’ in response to positive statements about mental well-being
• After advice: a hypothetical client would have be most likely to say ‘Some of the time’ or ‘Often’ in response to positive statements about mental well-being
In practice, clients gave different responses for different items, but this method can be used illustrate the change in mean WEMWBS scores before and after CAB advice.
7 12 17 22 27 32
None of the time
Rarely
Before CAB advice
Some of the time
After CAB advice
Often
All of the time
Mean WEMWBS score
Mean WEMWBS scores: before and after advice
Two-stage survey approach
Health Outcomes Monitoring Toolkit – Pilot Results March 2014 | 15
Case study: improved mental well-being
Mandy, divorced and with a five year old daughter,
had recently been diagnosed with bipolar disorder,
which had been affecting her ability to work
resulting in her unemployment. She was living on
means tested benefits and had multiple debts,
including a £37,000 mortgage shortfall and council
tax arrears.
Mandy’s mental health declined and, in the face of
her unmanageable debt, she found it impossible to
focus on getting better.
When Mandy contacted CAB, they helped her to
apply for bankruptcy, raising money from a trust
fund to cover the cost of the application. They also
advised Mandy to approach her community mental
health team to help her with her condition, and
provided a direct link to the bureau’s dedicated
mental health programme.
She is now able to manage her finances so her
outgoings don’t exceed her income. She has been
shown how to access further support, including
specialist advisers from the CAB mental health
team. Mandy says that she is now sleeping better.
Two-stage survey approach
Health Outcomes Monitoring Toolkit – Pilot Results March 2014 | 16
Determinants of health: before and after advice
For the two-stage approach, information on determinants of health was collected at two different points. An initial survey was conducted before advice, followed by a second survey conducted four to six weeks after advice.
Debt
Key message: the proportion of clients struggling with debt reduced after they
received help from CAB
Debt status was recorded for 141 clients before advice. Of these, 86 clients (61%) were struggling with debt.
Debt status after advice was recorded for 103 clients. Of the 86 clients struggling with debt before advice:
• Debt problems resolved: 37 clients (43%) said they were no longer struggling with debt after advice
• Debt outcomes achieved: 46 clients (53%) said they had had a debt rescheduled These positive outcomes are reflected in a shift in overall responses:
• Reduced prevalence of debt problems: the overall proportion of clients struggling with debt reduced from 61% before advice to 41% after advice
0
10
20
30
40
50
60
70
80
90
100
Before advice After advice
% o
f cl
ien
ts s
tru
gglin
g w
ith
de
bt
Proportion of clients struggling with debt
Two-stage survey approach
Health Outcomes Monitoring Toolkit – Pilot Results March 2014 | 17
Household income and finances
Key message: many clients improved their financial capability after advice, but
changing income and expenditure was more difficult to achieve
Monthly household income was recorded for 137 clients before advice. A majority of 77 clients (56%) belonged to the low income category of under £800. Of these, 48 clients (26%) had one or more dependent children.
Of the 143 clients in the sample, the following changes were reported after advice.
• Financial capability: 69 clients (48%) reported that their understanding of management of finances had improved
• Income and expenditure: 16 clients (11%) said they had managed to reduce their expenditure and 7 clients (5%) had increased their monthly income
0
10
20
30
40
50
60
70
80
90
100
Income increased Expenditure reduced Understanding ofmanagement of finances
improved
% o
f cl
ien
ts r
ep
ort
ing
eac
h o
utc
om
e
Financial capability - comparison of outcomes
Two-stage survey approach
Health Outcomes Monitoring Toolkit – Pilot Results March 2014 | 18
Case study: debt and finances
Kelly
Kelly contacted CAB about a council tax bill that was
being enforced by bailiffs. The debt started because
Kelly hadn’t been budgeting well, and hadn’t
responded to requests for contact from her local
authority. When she received notice of bailiff
enforcement, Kelly was 6 months pregnant.
Kelly felt very anxious and overwhelmed by the threat
of bailiff action and nervous about them entering her
property. She felt unable to manage the demands of
work and trying to maintain her home, whilst feeling
the impact of pregnancy on her physical and mental
well-being.
CAB identified that Kelly was not in receipt of single
occupancy discount for council tax, which she was
eligible for, and helped her to apply for a backdated
payment. CAB also arranged for a six week hold on
bailiff action while they advised Kelly on her options
for repayment. CAB then arranged for her to pay off
the debt in monthly instalments.
When asked how she felt about her health and
finances following CAB intervention, Kelly said she
felt “like a huge weight has been lifted off my
shoulders”.
Two-stage survey approach
Health Outcomes Monitoring Toolkit – Pilot Results March 2014 | 19
Energy costs and fuel poverty
Key message: fuel poverty was common among CAB clients, but few managed
to address this area of expenditure after advice
The two-stage approach captured details on energy costs in addition to the fuel poverty measure used in the single-stage approach. The cost of energy bills was recorded for 128 clients. They were asked to consider gas, electricity, coal and oil bills.
• A significant proportion of 47 clients (37%) said they spent between £100-199 per month
• The most commonly reported monthly cost was between £50-£99 per month, among 44 clients (34%)
Fuel poverty status before advice was recorded for 131 clients. The definition of fuel poverty used was whether clients were spending more than 10% of their household income on fuel bills.7
The results show that a majority of 86 clients (66%) were experiencing fuel poverty before advice. Of these, the majority did not report an increase in income or decrease in expenditure after advice.
7 Please note that the definition of fuel poverty has been revised since the toolkit was first designed.
22%
34% 18%
19%
4%
1% 1%
Total monthly cost of your energy bills
£0 - £49
£50 - £99
£100 - £149
£150 - £199
£200 - £249
£250 +
Not known
Two-stage survey approach
Health Outcomes Monitoring Toolkit – Pilot Results March 2014 | 20
Housing and homelessness
Key message: whilst some clients improve their housing situation after advice,
there was no evidence of a general improvement in this sample
Housing status was recorded for 143 clients before advice. Of these 4 considered themselves homeless, and a further 14 felt that they were at risking of becoming homeless.
Of the clients who described themselves as either ‘homeless’ or ‘at risk of becoming homeless’ before advice, 3 stated that their housing situation had improved after advice, 11 that it had stayed the same, and 4 gave no response.
Employment
Key message: in this sample, there was no evidence of a general improvement
in employment in status after advice
Employment status was recorded for 143 clients before advice. Of these, 59 clients stated that they were unemployed, and seven that they were concerned about keeping their current employment.
Although improving clients’ employment situation is not a direct objective of Citizens Advice services, the toolkit was used to record any changes in employment that did occur.
Of the clients who described themselves as either ‘unemployed’ or ‘concerned about keeping their current employment’ before advice, 41 said that their employment situation had stayed the same, 2 that it had got worse and 23 gave no response.
Single survey approach
Health Outcomes Monitoring Toolkit – Pilot Results March 2014 | 21
Single survey approach
Sample and response rate
Of the eight Citizens Advice Bureaux piloting the toolkit, five opted for the single-stage approach. Across these five bureaux, 214 clients were surveyed.
The single survey covered 19 core questions. The average response rate to core questions was 91%. This ranged from 76% for the question on fuel poverty before advice to 98% for questions on dependent children and disabilities.
The results below are the aggregated from the five bureaux piloting the single survey approach.
Respondent profile
Disability and health status was recorded for 209 clients. In total, 135 clients (65%) in the sample were disabled people, people with learning difficulties, or people with a long-term condition or mental health problems.
This is significant proportion considering that these groups make up around 38% of Citizens Advice clients across the service and around 21% of the general population of England and Wales.
The proportion of Black, Asian and Minority Ethnic (BAME) clients was around 16%, which is similar to the client profile across the Citizens Advice service and the population of England and Wales.
8%
2%
39%
16%
35%
Disability and health status
Having a disability
Having a learning disability
Having a long-term health condition
Having a mental health condition
Not disabled/no long-term healthconditions
Single survey approach
Health Outcomes Monitoring Toolkit – Pilot Results March 2014 | 22
Mental well-being
Key message: after advice, levels of mental well-being were two points from the
population mean, but it is challenging to identify change with this approach
The single-stage approach measures mental well-being scores after advice using the shortened version of the WEMWBS scale. Responses across the all 7 items in the WEMWBS scale were recorded for 162 clients.
On a scale ranging from a minimum 7 to a maximum of 35, the mean mental well-being score recorded four to six weeks after advice was 23.11. This is two points from the population mean of 25.18.
WEMWBS scores for after advice (n=162)
Mean 23.11
Median 24.00
Mode 28.00
Std. Deviation 6.43
Confidence Level (95%) 1.00
As noted, one limitation of the single-stage approach is that it does not include a baseline score for clients’ mental well-being before advice. This makes it difficult to assess any change in clients’ mental well-being after receiving advice.
7 12 17 22 27 32
After CAB advice
Population mean
Mean WEMWBS score
Mean WEMWBS score after advice
Single survey approach
Health Outcomes Monitoring Toolkit – Pilot Results March 2014 | 23
This limitation is significant given that results from the two-stage approach show that, before receiving advice, clients in that sample were significantly below the UK population mean of 25.18. These comparisons are summarised in the table below.
Approach Mean score before advice
Mean score after advice
Single-stage N/A 23.11
Two-stage 19.15 24.67
Determinants of health: before and after advice
For the single survey approach, all information on determinants of health was collected at the same point. Client surveys were conducted during one-off follow-up survey four to six weeks after advice provision. At this point, clients were asked about their circumstances before advice, and then whether this had changed after advice.
Debt
Key message: over half of clients who had been struggling with debt before
advice reported an improvement to their debt situation after advice.
Debt status before advice was recorded for 191 clients. Of these, 85 clients (45%) said they were struggling to repay a debt. Of those struggling with a debt before advice:
• Debt problems resolved: 52 clients (61%) said that their situation had improved after advice
• Debt outcomes achieved: 27 clients (32%) had debt rescheduled and 11 clients (13%) had debt written off
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Debt written off Debt rescheduled Debt situationimproved
% o
f clien
ts r
ep
ort
ing
each
ou
tco
me
Debt - comparison of outcomes
Single survey approach
Health Outcomes Monitoring Toolkit – Pilot Results March 2014 | 24
Case study: debt and finances
Jack
Jack, 56, is a single client in receipt of Jobseeker’s
Allowance, Housing Benefits and Council Tax
Benefits. Jack was a victim of identity fraud,
leading to debts of £2,000. He reported the matter
to the police, but was still awaiting follow-up from
them when he contacted CAB. He had been
physically threatened by three men from the
companies to whom he supposedly owed money.
CAB advised Jack to report the identity theft to
Action Fraud. Action Fraud began investigating all
involved parties. CAB helped Jack with a letter to
the creditors explaining the situation and including
his Action Fraud reference number.
As a result of the letter, Jack’s debts were written
off, which was a great relief to him. He commented:
“CAB resolved my problem with a high degree of
professionalism and great success. I am very
happy with the service I received.”
Single survey approach
Health Outcomes Monitoring Toolkit – Pilot Results March 2014 | 25
Household income and finances
Key message: more than 1 in 4 clients reported an increase in income and
improvement in financial capability after advice
Household income before advice was recorded for 190 clients. A majority of 134 clients (71%) of clients estimated that, after housing costs, belonged to the low income category of under £800 per month. Of these, 27% had one or more dependent children under 14 living with them.
Of the 214 clients in the sample, the following changes were reported after advice.
• Financial capability: 65 clients (30%) said that their understanding of how to manage their finances had improved
• Income and expenditure: 60 (28%) said that their monthly income had increased and 43 (20%) said that their monthly expenditure had decreased
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Expenditure reduced Income increased Understanding ofmanagement of finances
% o
f cl
ien
ts r
ep
ort
ing
eac
h o
utc
om
e
Financial capability - comparison of outcomes
Single survey approach
Health Outcomes Monitoring Toolkit – Pilot Results March 2014 | 26
Fuel poverty
Key message: some clients moved out of fuel poverty after advice, but there
was no evidence of a general improvement in this sample
Fuel poverty status before advice was recorded for 163 clients. The definition of fuel poverty used was whether clients were spending more than 10% of their household income on fuel bills.8
Of 163 clients who responded to this question, a majority of 119 clients (73%) estimated that their fuel bills had been costing them at least 10% of their household income.
Of those in fuel poverty before advice, the following results were returned after advice:
• 11 clients (9%) were no longer spending more than 10% of their income on energy costs
• the majority of 106 clients (89%) still remained within the fuel poverty bracket
Housing and homelessness
Key message: among a small group of vulnerable clients, over half reported an
improvement in their housing situation after advice.
Housing status before advice was recorded for 201 clients. Of these, 7 were homeless and a further 22 felt that they were at risking of becoming homeless.
Of these 29 clients who were either homeless or at risk of becoming homeless before advice, housing circumstances after advice were recorded in 26 cases: 17 said that their housing status had improved after advice, 7 that it had stayed the same, and 2 that it had got worse.
8 Please note: the definition of fuel poverty has now changed.
Health Outcomes Monitoring Toolkit – Pilot Results March 2014 | 27
Employment
Key message: among clients unemployed or at risk of unemployment before
advice, almost 1 in 5 improved their employment situation after advice
Employment status before advice was recorded for 200 clients. Of these, 62 (31%) were unemployed and 19 clients (10%) were concerned about keeping their current employment.
Although improving clients’ employment situation is not a direct objective of Citizens Advice services, the toolkit was used to record any changes in employment that did occur.
Of the 81 clients who were either unemployed or concerned about employment before advice, 14 clients (17%) said that their situation had improved after advice, 53 clients (65%) said that it had stayed the same, and 4 clients (5%) said that it had gotten worse.
0
2
4
6
8
10
12
14
16
18
Got worse Stayed the same Improved
Nu
mb
er
of
clien
ts
At risk clients - housing status after advice
Appendix: summary data tables
Health Outcomes Monitoring Toolkit – Pilot Results March 2014 | 28
Appendix: summary data tables
Two stage approach
Client profile – disability and health status
Response Number of clients (n=143)
Having a disability 10
Having a learning disability 1
Having a long-term health condition 36
Having a mental health condition 25
Not disabled / no long-term health conditions
71
Client profile – ethnicity
Response Number of clients (n=143)
Asian or Asian British 3
Black or Black British 3
Mixed 2
White 132
Other 3
Client profile – dependent children under 14
Response Number of clients (n=143)
Yes 94
No 48
No response 1
WEMWBS scores
Matched WEMWBS scores for before advice (n=138)
Mean 19.15
Median 18.00
Mode 21.00
Std. Deviation 6.87
Confidence Level (95%) 1.16
Appendix: summary data tables
Health Outcomes Monitoring Toolkit – Pilot Results March 2014 | 29
Matched WEMWBS scores for after advice (n=138)
Mean 24.67
Median 25.00
Mode 21.00
Std. Deviation 7.10
Confidence Level (95%) 1.19
Debt – client struggling with debt
Response Before advice (n=143) After advice (n=143)
Yes 86 42
No 55 61
No response 2 40
Analysis: changes after advice among clients struggling to repay debt before advice (n=86).
Response Struggling to repay debt after advice
Debt rescheduled
Yes 40 46
No 37 25
No response 9 15
Household income and finances
Response Number of clients (n=143)
Under £800 77
£800 to £1350 41
Over £1350 19
No response 6
Analysis: changes in financial behaviour after advice among all clients (n=143).
Response Understanding of management of finances improved
Expenditure reduced
Income increased
Yes 69 16 7
No 12 69 86
No response 62 58 50
Appendix: summary data tables
Health Outcomes Monitoring Toolkit – Pilot Results March 2014 | 30
Monthly energy costs
Response Number of clients (n=143)
£0-£49 28
£50-£99 44
£100-£149 23
£150-£199 24
£200-£249 5
£250+ 2
Not known 2
No response 15
Fuel Poverty
Response Number of clients (n=143)
More than 10% of income on fuel bills 86
Less than 10% of income on fuel bills 45
No response 12
Housing and homelessness
Response Number of clients (n=143)
Homeless (including B&B tenant) 4
At risk of becoming homeless 14
Neither of the above 125
Analysis: changes after advice among clients at homeless or at risk before advice (n=18).
Response Number of clients
Got worse 0
Stayed the same 11
Improved 3
No response 4
Employment
Response Number of clients (n=143)
Unemployed 59
Concerned about keeping your current employment
7
Neither of the above 77
Appendix: summary data tables
Health Outcomes Monitoring Toolkit – Pilot Results March 2014 | 31
Analysis: changes after advice among clients unemployed or at risk before advice (n=66).
Response Number of clients (n=66)
Got worse 2
Stayed the same 41
Improved 0
No response 23
Single-stage approach
Client profile – disability and health status
Response Number of clients (n=214)
Having a disability 16
Having a learning disability 3
Having a long-term health condition 82
Having a mental health condition 34
Not disabled / no long-term health conditions
74
No response 5
Client profile – ethnicity
Response Number of clients (n=214)
Asian or Asian British 21
Black or Black British 8
Mixed 4
White 166
Other / unknown 3
No response 12
Client profile – dependent children under 14
Response Number of clients (n=214)
Yes 56
No 153
No response 5
Appendix: summary data tables
Health Outcomes Monitoring Toolkit – Pilot Results March 2014 | 32
WEMWBS
WEMWBS scores for after advice (n=162)
Mean 23.11
Median 24.00
Mode 28.00
Std. Deviation 6.43
Confidence Level (95%) 1.00
Debt – client struggling with debt
Response Before advice (n= 214)
Yes 85
No 106
No response 23
Analysis: changes after advice among clients struggling to repay debt before advice (n=85).
Response Number of clients
Got worse 4
Stayed the same 29
Improved 52
Analysis: debt outcomes among those among client struggling to repay debt before advice (n=85).
Response Debt written off Debt rescheduled
Yes 11 27
No 37 25
No response 37 33
Appendix: summary data tables
Health Outcomes Monitoring Toolkit – Pilot Results March 2014 | 33
Household income and finances
Response Number of clients (n=214)
Under £800 134
£800 to £1350 36
Over £1350 20
No response 24
Analysis: changes in financial behaviour after advice among all clients (n=214).
Response Understanding of management of finances improved
Expenditure reduced
Income increased
Yes 65 43 60
No 22 67 83
No response 127 104 71
Fuel poverty
Response Before advice (n=214)
More than 10% of income on fuel bills 119
Less than 10% of income on fuel bills 44
No response 51
Analysis: changes after advice among clients in fuel poverty before advice (n=119).
Response After advice
More than 10% of income on fuel bills 106
Less than 10% of income on fuel bills 11
No response 2
Housing and homelessness
Response Number of clients (n=214)
Homeless (including B&B tenant) 7
At risk of becoming homeless 22
Neither of the above 172
No response 13
Appendix: summary data tables
Health Outcomes Monitoring Toolkit – Pilot Results March 2014 | 34
Analysis: changes after advice among clients at homeless or at risk before advice (n=29).
Response Number of clients
Got worse 2
Stayed the same 7
Improved 17
No response 3
Employment
Response Number of clients (n=214)
Unemployed 62
Concerned about keeping your current employment
19
Neither of the above 119
No response 14
Analysis: changes after advice among clients unemployed or at risk before advice (n=81).
Response Number of clients
Got worse 4
Stayed the same 53
Improved 14
No response 10
Aims and principles
The Citizens Advice service provides free, independent, confidential and impartial advice to everyone on their rights and responsibilities. It values diversity, promotes equality and challenges discrimination.
The service aims: • to provide the advice people need for the problems they face • to improve the policies and practices that affect people’s lives.
Citizens Advice 115-123 Pentonville Road London N1 9LZ
Telephone: 020 7833 2181
www.citizensadvice.org.uk www.adviceguide.org.uk
Citizens Advice is an operating name of The National Association of Citizens Advice Bureaux. Registered charity number 279057.
top related