xenzone is committed to developing an evidence base for online

Xenzone is committed to developing an evidence base for online counselling.

Since its flagship service KOOTH.COM was launched in 2004, Xenzone has been collecting and reporting data for commissioners and the wider world to show the impact of KOOTH.COM in young people’s lives.

The Development of Outcome Measures on KOOTH.COM In this time, Xenzone has developed a very strong working relationship with the University of Manchester to develop research projects that evaluate the work

we do on KOOTH.COM. Dr Terry Hanley, (Programme Director at University of Manchester for the Counselling Psychology Doctorate) was the first to

investigate. His findings (Hanley, 2012) indicated there was a strong therapeutic alliance (a key ingredient and indicator of effective therapy) experienced by

young people online.

At Xenzone, we currently use outcome measures as recommended for CYP IAPT services (see http://www.cypiapt.org), and by the BACP in their outcomes

toolkit for school based counselling (see http://www.bacp.co.uk). At present there are no direct recommendations for online counselling, and we have adapted

these to suit our needs.

The development of the CoGS (Counselling Goals System) outcome tool came from an internally commissioned piece of research to evaluate the use of

outcome measures online. It was published in the international peer-reviewed academic journal, Pastoral Care in Education (Sefi and Hanley, 2012 - visit

www.xenzone.com/resources.php for abstract). Its key finding was that the YP-CORE was insufficient to ‘tell the whole’ story’ of how a young person’s life

was changed through our intervention, and it recommended developing goal-based outcomes for a bigger picture of the ‘how, what and why’ online

counselling works.

We received a research award from the British Association of Counselling and Psychotherapy (BACP) for Xenzone Alliance, our research arm, to develop the

CoGS project as an innovative tool to evaluate online counselling and support, and to disseminate these findings through articles and peer-reviewed papers,

due to be published in 2015.

The key measures recommended are: A Clinical Outcomes Measure at Assessment - we use YP-CORE (for info, visit www.coreims.co.uk) End of Session Feedback to evaluate Therapeutic Alliance A Goal based Outcome Measure – we have developed this into a more interactive and youth-friendly measure called

CoGS (Counselling Goals System)

Aaron Sefi, Xenzone 2015 Report for CoGS Research on KOOTH.COM

2

Method We developed the CoGS as a goal-based outcome measure for use on KOOTH.COM. In a young person’s initial contact with the service, our practitioners can

identify, through collaborative assessment, what goals a young person has and wants to achieve. These can relate to anything in their lives, or specifically in

the work we do with them.

CoGS in action on KOOTH.COM

The young person can score where they are up to in achieving this goal at any time (in chat with a counsellor or simply when they are logged onto their home

page on KOOTH.COM). When they score 10, it indicates that the goal has been achieved, and this is illustrated by a gold star.


3

Training

As this is a new and innovative way of working, we trained all our practitioners across all our services in how to use the CoGS. The training included the key

elements of: theory (the CoGS are inspired by the Pluralistic Framework for counselling – see Cooper & McLeod, 2011); technical (how to use the interactive

tool on KOOTH.COM); and skills practice (how to use the CoGS effectively within their practice utilising knowledge of this work derived through experience and

academia, for example, Hanley et al, 2013). These training sessions were counted towards our counsellors Continuing Professional Development (CPD) and

have helped the service engagement with outcomes and research.

This training and on-going mentoring was backed up by a manual for using the CoGS and an Outcomes Toolkit for KOOTH Practitioners, which outlines

Xenzone’s ethos for KOOTH as an Outcome-Informed Service.

Data Collection

The CoGS were launched with our new build of KOOTH in November 2013. Before this, we had already been collecting CoGS data from our face-to-face

services as well (this data forms an important part of the comparison of what goals are achieved through different media for counselling, and will be published

later this year as part of a comparative study).

The CoGS have shown to be effective in 3 key ways (see Findings for further discussion):

1. They are a measuring tool to record how well we are helping young people to achieve their

goals in counselling – and in their lives.

2. They are a therapeutic tool – for our practitioners to use to focus the work and for young

people to take away. They are a useful way of describing their experience on KOOTH, and

can ‘demystify’ the process of counselling.

3. They help us demonstrate in great detail what the achievements of our service through user-

generated outcomes. The analysis of this data (shown below) tells us a lot about what

young people want from and achieve through our intervention.


4

The CoGS data is all recorded as a key element of our integrated case notes system:

Between November 2013 and August 2014, we collected goals data from the CoGS. This amounted to:

1137 goals articulated by 505 young people online through KOOTH.COM - these are the goals described in this

paper. NB. This does not reflect the total work being done on KOOTH.COM as it took an initial period of bedding in to get practitioners to

use the tool regularly (and since this end date, we have over 2000 goals at time of going to press)

Through KOOTH face to face services (in Powys, Knowsley and Cornwall) – 360 goals were articulated by 202

young people – analysis of face-to-face goals in comparison to online will follow in 2015.

In this time, we have conducted semi-structured interviews with six practitioners, to offer feedback about how

the CoGS work from a practitioner point of view. We have also collected feedback from young people – through

an online focus group conducted through live group discussion, and through comments posted in a specially

made forum thread. Some of this feedback is detailed below and will form basis for further research studies.


5

Data Analysis

The project has been run in conjunction with the University of Manchester. The data analysis is being conducted under their auspices and the supervision of

Dr Terry Hanley. It is being carried out by Aaron Sefi, Xenzone’s Lead for Research and Outcomes, alongside Zehra Ersahin, a Doctorate student from the

University of Manchester.

A key part of the analysis is to codify the goals into different categories and themes. This codifying has drawn from key literature in the field of goals research

– namely the ‘Berne Inventory of Goals’ (Grosse Holtforth and Grawe, 2002). However, as this is the first large scale research into therapy goals for children

and young people, it has been necessary to adapt categories and themes. These categories and themes also represent the wide range of therapeutic

approaches and models utilised by KOOTH.COM practitioners – from solution focussed techniques to Dialectic Behaviour Therapy (DBT) and Cognitive

Behaviour Therapy (CBT) to the Person-Centred Approach.

Three Key Overarching Categories were established:

1. Intrapersonal Goals – goals that are about the young person’s internal world, e.g. “Wanting to explore difficult feelings”

2. Goals on Self relating to others – goals about how a young person’s internal world relates to the external world, e.g. “Wanting to express myself

better with my peer group”

3. Interpersonal Goals – goals that involve the young person’s relationship to other people, e.g. “Wanting a better relationship with family”

From these overarching categories, sub categories were formed:


6

In keeping with the theme of CoGS, and how the different types and categories of goals can work together as a mechanism for change in young people’s

lives, they can also be illustrated thus:


7

Findings These sub-categories have been further broken down into themes relating to that category – e.g. the sub category of personal growth has six themes within it

– Exploring thoughts and feelings; Owing the problem; Improving confidence and self-esteem; Improving and developing skills for life and Accepting self.

Our data analysis was conducted using a research methodology called Grounded Theory (Glaser and Strauss, 1967). In this approach, the researcher

employs a conscious naivety when looking at the data, allowing themes and categories to emerge, as such it is a ‘bottom-up’ or grounded approach to data.

Through this method, Meaning Units (MUs which measure how often this goal theme is covered) are created and assigned to each theme as it emerges using

qualitative data analysis coding software Nvivo 9 (Richards, QSR International: 2010).

Overarching Category

Sub Categories Themes MUs Illustrative User-generated goals

Intra-personal Goals

Personal Growth exploring thoughts and feelings 99 “To explore why I feel people don’t like me” “Wear short sleeves in public” “Process, let go and move on for a better future” “Identify 5 things that I like about myself” “Acknowledge that it’s okay to feel sad”

owning the problem 109

improving confidence and self-esteem 48

improving and developing skills for life 26

accepting self 57

Emotional Wellbeing feeling happier 41 “Control emotions so not crying all the time” “To feel happy and more in control” “To manage my moods better” “I wanna be ok with my mind not let it control me” “Give permission myself for a shopping day with mum”

keeping calm (less anxious) & regulating emotions

49

working on anger, and grief (utilising alternative ways to cope)

25

enjoying self 9

Mental Wellbeing exploring (have a better factual understanding of the mental health problem- various)

24 “Read information on physical symptoms of anxiety” “To explore why I hurt myself for fun” “Write or draw feelings instead of self-harming” “Practise safe place visualisation, to control suicidal feelings” “Go for a walk often to help lift depression”.

work on self-harm, anxiety, depression, suicidal tendency, eating difficulties, obsessive compulsive disorder, and learning disabilities by utilising a coping strategy (mindfulness & yoga, relaxation exercises, narrative work)

92


8

Physical Wellbeing self-care (i.e. sleep hygiene, stopping smoking, heaving a healthy diet)

60 “Walk home from school for exercise” “Wear longer skirts/ trousers to school” “Start jogging at weekends” “Drop a size from 12 to 10 by eating healthy” “To shower and dress every day”

exercising to keep fit 18

keeping safe 9

School/Career aspirational (pursuing an interest at university, planning on a professional career, accomplish self to a fame, success, wealth)

48 “Practice online resources one hour a day to pass the RAF test” “To succeed in school and get a job to secure myself” “Become a good writer like John Green” “I want to be in a band and tour the world”. exams (getting organised & getting good

grades) 35

Inter-personal Goals

Improving relationships

improving relationships with family members

19 “Eat out with family” “Improve relationships with family and friends” “To have a better relation with girlfriend” “Get closer to brothers”

improving relationships with friends 55

improving relationships with significant others (boyfriend/girlfriend)

15

Goals on Self relating to Others

Getting help (it is okay to get help)

seeking health assessment (GP, nurse, psychiatrist)

48 “To ring brook for pregnancy test at clinic” “Speak to head of year about Camhs referral again” “Transfer from Kooth to face to face counselling” “Ask tutors for more support to improve my grade” “Get through difficult day by coming online”

connecting to support network (family, friends, teachers, tutors, pastoral care)

97

accessing f2f counselling services 10

Getting out of comfort zone (Challenging behaviours)

being resilient (against bullies/negative people)

7 “Come back to chat next Monday, 8pm” “Stay off Facebook for few days until more settled” “To learn to live with brother's Aspergers” “Stop saying sorry, change to respect” “Practice to bat back insulting remarks”

being assertive in relationships 6

setting boundaries 6

committing to therapy (and thus wellbeing) 32

Speaking up (communicating self better to others)

expressing self better and more open to significant others

84 “To talk to my girlfriend about hurt feelings” “Let someone know who I have strong feelings for” “To talk things through and get them off my chest” get things out of my chest 10

Fitting in (in external world

feeling comfortable in relationships 23 “Look after granma who has early dementia” “To be comfortable making and keeping relationships” “Succeed and be a good person to keep grandpa proud”

Working on being accepted 27

attending to needs of family members and friends

7


9

These findings can also be illustrated as charts:

61%

32%

7%

Overarching Categories

Intra-personal Goals Goals on Self Relating to others

Interpersonal Goals

46%

16%

15%

12%

11%

Intra-personal Goal sub categories

Personal growth Emotional wellbeing Mental wellbeing

Physical wellbeing School-Career

Improving relationships with family members

21%

Improving relationship with friends

62%

Improving intimate

relationships 17%

Inter-personal Goals

Asking for help (and

getting help) 44%

Speaking up! (Communicate

Self better) 26%

Fitting in! (In relation to significant

others) 16%

Challenging behaviour

14%

Goals on Self Relating to Others


10

As indicated on previous page,

intra-personal goals make up the

largest category (61% of all the

goals).

Within this category, the sub

category of personal growth is

the largest (46%)

Within this sub-category, the two

predominant themes are:

1. Owning the problem –

these are self-help style goals

focussing on what the young

person can do to support their

therapy

2. Exploring thoughts and

feelings – these are typical

exploratory goals for young people

– goals for therapy, that are

worked on through the sessions


11

This was the second most common category (32% of

all goals articulated)

The most common sub category in Self Relating to

Others was: Asking for help (and getting help) at

44% of the goals in this category.

The predominent theses in this sub-category are:

1. Connecting to Support Network – this

relates to young peoples’ goals around using family,

friends, teachers and significant others for support in

their issu – and to support the therapy

2. Seeking health assessment - this reflects

the level of integration and direct link we make on

KOOTH.COM to health professionals, encouraging

young people to link in with statutory service

This category accounts for 7% of total goals

articulated. The most common theme in this sub-

category is:

Improving Intimate relationships – these relate to

goals young people have around boyfriend/girlfriend

issues specifically.


12

0

2

4

6

8

10

0 - 1 1 - 2 2- 3 3 - 4 4 - 5 5 - 6 6 - 10 over 10

CHAT TIME in hours

Score (out of 10) by Chat Hrs

‘Distance travelled’

Finally, with the 1137 goals collated from the online service in the data collection period. As we chart the movement towards achieving a goal, we are able to

identify which goals are achieved and to what extent, thus giving us a picture of distance travelled in our service.

We collected this data in November 2014, and it indicates ‘work in progress’ as many of the goals set in the data collection period are still be worked towards,

but overall, it shows some very interesting results:

As mentioned, this outcome data from our CoGS research project will be published later this year once independently evaluated by the University of

Manchester, alongside some movement data from our face-to-face service, and in comparison with existing research data on similar tools used in CAMHs

(see Law, 2011) and other evaluation of youth counselling (see for example Rupani et al, 2013). On early reflections, this data is closely aligned, and

shows that young people achieve their goals well in an online environment.

We can also see a trend in young people’s achievement of goals and how long they spend on the service:

This table shows that the more time a young person spends

on KOOTH.COM, the closer they get to achieving the goals

they have set for themselves.

Goals that have been fully achieved: 28%

Goals that have moved by 8 points or more (i.e. nearly achieved): 40%

Goals that have moved by 5 points or more (i.e. at least half-way achieved): 69%

Overall, the median average movement for a goal is 7 points


13

Discussion

Taxonomy – or frequency of goal themes and categories

If we look firstly at the taxonomy of the goals (i.e. frequency of each category and sub-category), the first most striking aspect is how few inter-personal (i.e.

relationship) goals were articulated in comparison to intra-personal goals (i.e. around personal growth). Whilst this can be explained to some extent by the

addition of the ‘Self Relating to Others category (which was in itself devised to differentiate or ‘fill the gap’ between intra- and interpersonal), it can also be a

reflection of the priorities of an online client – that family relationships and friendships are not the primary focus of a young person seeking support online.

With so many intra-personal goals the clear primary focus is around personal growth, and specifically, around the two key themes of Owning the problem and

Exploring thoughts and feelings. Owning the problem can also be seen as a Life Goal – taking responsibility for decisions and actions in life that will help

achieve the goal set in therapy. Exploring thoughts and feelings is more of a Therapy Goal - setting a goal around using the counselling to explore thoughts

and feelings, primarily in order to understand oneself better (see Cooper and McLeod, 2011 for more on these distinctions).

The most common goal themes in the category of Goals on Self relating to others were: Connecting to support network; and Seeking health assessment.

Both of these reflect a similar drive to the life goal theme of Owning the problem, in that they require the young person to make efforts outside the counselling

to develop support for themselves. This reflects the skilful work of practitioners on KOOTH.COM to engage with young people around their global support

network. In this, crucially, we can help young people develop an autonomous and independent role in making change in their own lives. It also reflects the

anecdotal understanding of online therapeutic support – that it is often independently sought and autonomously undertaken. As it requires a high level of self-

motivation to access, online therapeutic support can be tailored to enhance and develop that self-motivation.

This is reflected in some of the feedback that young people and practitioners have given us around the use of the CoGS:

“Setting goals for myself means I can help myself and not rely on anyone else! I think it really good so everyone can help other people and they can set their

own goals!” Turtle

Please see Appendix 1 for additional feedback from young people and practitioners on the CoGS.

Participation

A lot of the feedback we have received from young people has shown how collecting outcomes data can and needs to be a positive part of a young person’s

experience of our service.


14

In this way, it can be a critical part of their active participation and involvement with developing the service. This can take place on a 1-1 basis between young

person and practitioner – we advocate the idea of our practitioners being ‘FIT’ that is to say, operating ‘Feedback Informed Therapy’ (inspired by the work of

Scott Miller in psychotherapy research)

The CoGS also enables participation on an organisational level, as feedback also shapes service delivery. As a service, we routinely collect a range of

feedback from satisfaction questionnaires, to end of session feedback forms, and these can directly inform how young people want their next session to be.

The CoGS adds to this data and allows us to look at how we can improve our service based on examining the extent of achievement of goals through our

interventions.

Outcomes

As outlined previously we can evidence positive outcomes for young people using KOOTH.COM through the CoGS. The median score of 7 points (out of

maximum possible 10 points) movement in goals that young people have worked on, with 69% of goals at least half-way achieved when we took a snapshot

of the work, indicates that most of the goals young people want to work on are being achieved. The next piece of work to achieve with this data is to

investigate which categories and themes of goals are more achieved than others. This work will be done as part of the research project in 2015 with the

University of Manchester.

Next Steps

We have used the taxonomy data presented here as the basis for a submission of a paper for the peer-reviewed academic journal: Counselling and

Psychotherapy Research (Sefi, Ersahin and Hanley, 2015: to be published). We will be using the data to produce two further papers in 2015, evaluating the

outcomes data, and the feedback from practitioners and young people. We will also be presenting the data at various research conferences in the coming

years.

As mentioned, all of this data will crucially continue to feed into service delivery and design, allowing for genuine user generation of the service model. We

continue to collect CoGS data, and since the collection process for this part of the project has ended, we have already collected nearly 1000 more goals at

time of going to press. We aspire to develop an application for young people to make it easier to use the CoGS out of session in face-to-face counselling, and

we are keen to share our learning and training on how to effectively use goal-based outcome measures as an outcome measure, therapeutic tool, and basis

for service design.

For more information, please contact:

Aaron Sefi Service Development Manager KOOTH.COM T: 0845 3307070 E: [email protected] W: www.xenzone.com


15

References for this report

Cooper, M. & McLeod, J. (2011). Pluralistic Counselling and Psychotherapy. Sage: London. Hanley, T., Humphrey, N. & Lennie, C. (Eds.) (2013). Adolescent counselling psychology: theory, research and practice. London: Routledge. Law, D. (2011). Goals and Goal-based Outcomes (GBOs) Some Useful Information Version 2.0 downloaded from http://www.corc.uk.net. Grosse Holtforth, M., & Grawe, K. (2002). Bern inventory or treatment goals: Part 1: Development and first application of a taxonomy of treatment goal themes. Psychotherapy Research, 12, 79–99. Hanley (2012). Understanding the online therapeutic alliance through the eyes of adolescent service users. Counselling and Psychotherapy Research 12:1,

35-43.

Rupani, P., Cooper, M., McArthur, K.,and Pybis, J., Cromarty, K., Hill, A., Levesley, R., Murdoch, J., and Turner, N. (2014) The goals of young people in

school-based counselling and their achievement of these goals. Counselling and Psychotherapy Research, 14 (4).

Sefi, Ersahin and Hanley (unpublished). Examining the goals of young people in online counselling and support

Sefi, A. & Hanley, T. "Examining the complexities of measuring effectiveness of online counselling for young people using routine evaluation data." Pastoral

Care in Education An International Journal of Personal, Social and Emotional Development 30, no. (1)(2012): 49-64.

Appendix 1 – Feedback from Young People and Practitioners about CoGS

Here are some examples of the feedback young people gave when asked on a live forum what they thought about CoGS (punctuation added!):

“Setting goals for myself means I can help myself and not rely on anyone else! I think it really good so everyone can help other people and they can set their

own goals!”

“I think it’s a good way of managing things – breaking things down into bitesize chunks”

“It’s something to work towards, to feel like you have achieved something.”

And from young people on how the CoGS works for them:


16

“The counsellors always support you when you try to get there, you get this really good feeling when you achieve them”

“The counsellors give you some ideas of the goals you could work on, and when you have a giant problem and turn it into little ones and work on them one by

one the big problem doesn’t seem so bad.”

“If you dont do it in time, the counsellors make you feel better about it. They just set it again with another deadline or they alter it a bit so it’s easier to do. They

always make sure you dont feel bad. They tell you it’s alright”

Here is some comments from practitioners collated as part of semi-structured interviews that will form basis of another academic paper from the

CoGS project:

“I think that the need to have goals (no matter how small they may seem to an observer) is a vital aspect of any therapy. The goal of 'see you again next week'

may not be officially seen as a goal but in my view, it’s probably THE most important goal that needs to be met”.

“For the client, [the CoGS] can provide focus and something to identify that they are working toward (making it achievable is also important in any goal-

setting). For the therapist it can show how the therapy is working and also allows he/she to reflect on their own practice and approach

“[The CoGS helped to] bring in more transparency of structure and objective when working with young people… it seemed to help them feel more secure and

focused in the work”

“There was one young person, who felt quite distant to me throughout our work together online. But then, when I checked in with her with the cogs, she moved

them along quite a lot... it was hard for her to put into words or express to me how things felt different - but through her cogs she could demonstrate to herself

and me that things were different, and that there had been quite a lot of movement for her (more than I would've guessed) plus it can be really orientating for

both of us - something really tangible and constant in a virtual online space!”

“I think it clarifies for them in a real and tangible way what they actual problems are and how they can work with that. If they are just stuck with this

overwhelming emotional experience that they don't know what to do with...and then you can help them to break it up into something that they can see and

actually do something with”.

xenzone is committed to developing an evidence base for online

Documents