Creative Evaluation Workshop

Victoria Quay, Edinburgh9th October 2014

Scottish Government Creativity Team and ImaginationLancaster, Lancaster University

Engaging with Design Morning Workshop

2

Cre

ativ

e Ev

alua

tion

| Vic

toria

Qua

y

The Creativity TeamCreativity is one of the four Scottish Government Business Strategy imperatives. The Creativity Team aims to foster the belief that being creative is an important part of our jobs; and to support creative practice. Together we can increase the Scottish Government’s creative capacity by exploring, testing out and implementing new ways of working to achieve better outcomes for the people of Scotland.

Design has become an increasingly prominent feature of our work. We collaborate closely with Glasgow School of Art Institute of Design Innovation (InDI) and their networks led us to Leon and his team at ImaginationLancaster (IL). I don’t think you should ever turn down the opportunity to work with people who develop tools for engagement using a strong business case and enormous beachballs .

Our third event workshop with Imagination brought together people who are interested in creative ways of evaluating quality engagement. This is essential to deliver the Scottish Approach tp policy making; incorporating asset based working and co-production. The information we gathered will have a direct impact on how the Creativity team continues to support teams across government, as well as how we work with our partners. We are very lucky to have been able to benefit from IL’s expertise and impeccable, creative preparation. It needn’t be expensive to do good evaluation just planning and imagination! We’re delighted that Lancaster will return in March 2014 to find out how people have been developing their ideas and to explore creative facilitation more widely

Anna Winters,

Head of Creativity Team, Organisational Development and Learning | People Directorate | Scottish Government

3

Cre

ativ

e Ev

alua

tion

| Vic

toria

Qua

y

AimsCreative Evaluation Workshop

This interactive workshop will focus on creative evaluation of engagement or consultation events. ImaginationLancaster, the design research center in Lancaster University will be presenting two new pilot tools. The first of these has its focus on the effectiveness of tools or techniques from the facilitators point of view. The second has its focus on participant evaluation and seeks to go beyond the ‘happy sheet’ to get real insights into participant experience.

Together we will be exploring, modifying and co-designing the next version of these tools. Within 2 weeks of the workshop these new redesigned versions will be available for you and your colleagues to use to evaluate your events.

4

Cre

ativ

e Ev

alua

tion

| Vic

toria

Qua

y

ParticipantsPeter Ashe: Peter.ashe@nhs.net, Catriona Hayes: Catriona.Hayes@scotland.gsi.gov.uk, Claire Tester: Claire.Tester@scotland.gsi.gov.uk, Louise Gallagher: Louise.Gallagher@ggc.scot.nhs.uk, Morag Williamson: Morag.Williamson@scotland.gsi.gov.uk, Claire Hider: Claire.Hider@scotland.gsi.gov.uk, Tammy Watchorn: tammy.watchorn@nhs.net, Imelda Giarchi: Imelda.Giarchi@scotland.gsi.gov.uk

FacilitatorsLeon Cruickshank: l.cruickshank@lancaster.ac.ukAnna.Winters: Anna.Winters@scotland.gsi.gov.uk

5

Cre

ativ

e Ev

alua

tion

| Vic

toria

Qua

y Tell us something amazing...The start of this workshop saw the participants asked to address one of two different questions, tell us something amazing or describe your job to a Martian. The aim of both these questions was to get participants to briefly introduce themselves and share something with the group that would help give a flavor of their approach to communication and engagement. On the reverse of these pro-formas was the first component of the ‘Koala Cutch’ tool described in detail in the following pages. The first part of this tool was to establish the expectations and skills of the participants as they started the workshop.

6

Cre

ativ

e Ev

alua

tion

| Vic

toria

Qua

y

7

Cre

ativ

e Ev

alua

tion

| Vic

toria

Qua

y

8

Cre

ativ

e Ev

alua

tion

| Vic

toria

Qua

y

9

Cre

ativ

e Ev

alua

tion

| Vic

toria

Qua

y

10

Cre

ativ

e Ev

alua

tion

| Vic

toria

Qua

y

Evaluation Tool for Facilitators...The first component of the workshop introduced a tool for documenting and evaluation the effectiveness of facilitation tools from the perspective of the facilitator. This involves establishing some of the basic metrics of the event, including the size and participants of the event. Following on from this the form asks about the skills that were employed to exploit the tool and finally invites suggestions for improvements to the tool.

This facilitation tool was motivated but the need to track the use of facilitation tools produced or shared by facilitation brokers such as the creativity team and Imagination Lancaster. We also wanted to produce an evaluation form that was atractive enough and where possible fun enough to get people to want to complete it, rather than it being a chore.

11

Cre

ativ

e Ev

alua

tion

| Vic

toria

Qua

y

12

Cre

ativ

e Ev

alua

tion

| Vic

toria

Qua

y

13

Cre

ativ

e Ev

alua

tion

| Vic

toria

Qua

y

14

Cre

ativ

e Ev

alua

tion

| Vic

toria

Qua

y

Peter presented a strong case for the WITTY (whats important to you) tool.

‘WITTY can help you to understand the positive assets and factors which you have and can better use in your day-to-day life by creating a visual listing of things you have done in the past, or things that you could do in the future.’

Developed by the IRISS project (www.iriss.org.uk/resources/witty-whats-important-you) this can be used through an App that can be downloaded for free or simply using pens and paper.

Tammy presented a strong case for an online innovation environment called Qube (www.qube.cc) this is

- An enterprise (social) media software platform

- A set of carefully-designed learning and working virtual environments with pre-loaded tools and techniques

- A series of facilitated or self-driven workshops, conferences, meetings and project activities leading to outcomes

15

Cre

ativ

e Ev

alua

tion

| Vic

toria

Qua

y

16

Cre

ativ

e Ev

alua

tion

| Vic

toria

Qua

y

17

Cre

ativ

e Ev

alua

tion

| Vic

toria

Qua

y

18

Cre

ativ

e Ev

alua

tion

| Vic

toria

Qua

y

19

Cre

ativ

e Ev

alua

tion

| Vic

toria

Qua

y

20

Cre

ativ

e Ev

alua

tion

| Vic

toria

Qua

y

21

Cre

ativ

e Ev

alua

tion

| Vic

toria

Qua

y

22

Cre

ativ

e Ev

alua

tion

| Vic

toria

Qua

y

Overview Facilitator Evaluation Tool Participants for the most part liked the design of the evaluation in terms of clarity and flow and those who ‘dotted around’ did not see this as a problem. There were a number of suggestions or discussion points to reflect on for the next version of the tool.

- Much of the factual information in the first third of the form would have been represented in the sesscion plan, is there a way to encorperate or exploit this? The ‘obviousness’ of a session plan speaks to the professionalism of the group, sometimes there is no formal plan for workshops.

- In a related issue the group felt an indication of expectations (of for example participants, or outcomes) and how reality deviated from this could be useful. This would also relate to both the outcomes and how the tool could be improved.

- There was a question about reflecting on the skills used in facilitating the tool and how useful that was.

- The distinction between participants (as individuals) and key players (organizations) was not well described in the evaluation tool and confused the group.

- The improvements section was largely found to be useful but an oportunity to tell a story would have been a useful addition.

- There was a recognition that the group were ‘super users’ and may not be representative of less experienced people using tools to support their running of a workshop but this needs further investigation.

23

Cre

ativ

e Ev

alua

tion

| Vic

toria

Qua

y

‘Koala Cutch’ Participant Evaluation ToolThis tool is based on the principle that the person`s starting point is an important component in understanding how they respond to an event. In response to this we trialled a new tool that is in two parts. The first is completed at the start of an event, it explores what the participants expectations and aspirations are and the skills and experience they feel they bring to this process.

This initial starting point then wraps around (or cutches) a summative evaluation sheet that uses these initial responses as a foil or stimulus for the participants to reflect on the workshop. We used the Koala metaphor to introduce an element of playfulness and fun into an aspect of workshops that can be rather dull. We also experimented with a tearing mechanism to get an initial response from participants before they wrote more considered evaluation comments.

Koala Evaluation Tool

1. Fold sheet A in half2. Stick sheet A to sheet B

Writing, writing, writing

Writing, writing

Writing

Writ

4. Write your answers5. Return to the facilitator

3. Rip to respond (We can then quickly visualise and compare all of the responses)

25

Cre

ativ

e Ev

alua

tion

| Vic

toria

Qua

y

26

Cre

ativ

e Ev

alua

tion

| Vic

toria

Qua

y

27

Cre

ativ

e Ev

alua

tion

| Vic

toria

Qua

y

28

Cre

ativ

e Ev

alua

tion

| Vic

toria

Qua

y

Participant Evaluation Tool OverviewThe 2 part nature of the tool was warmly received with a baseline taken at the start of the session and then building on this at the end of the session. This good feeling extended to the overall structure of the tool with suggestions to different uses this structure could be used for. Generally there a lack of consensus in the feedback from the group, the notes below show the spectrum of responses and discussion.

- One point of agreement was that the language could be simplified and clarified and that there could be a stronger relationship between the first and second parts.

- Some members of the group wanted to write then rip so they had time to reflect before making the decision to rip while others saw this as a way of drawing out intuitive responses before writing. One person could not see the point of ripping at all.

- There were some ambiguity with the scales of responses, where exactly did the scales start?

- In general the Koala name and branding was seen as a good thing but there are questions over the appropriate degree of playfulness. Some groups (especially children) will be very happy with more ‘Koala’ while others, e.g. Senior management may not be happy with current levels of playful use of Koala.

29

Cre

ativ

e Ev

alua

tion

| Vic

toria

Qua

y

Conclusion and Next Steps Over the coming months the Creativity Team will develop our creative facilitation skills to support colleagues across the Scottish Government and beyond. We will support several change projects ranging from how we communicate to developing policy and make good use of the brilliant tools that ImaginationLancaster has shared with us. We are committed to sharing our messes and successes with colleagues across the public sector and contributing to culture change – creativity is nothing to be scared of! More to follow in March…and if you’re interested in this work please get in touch.

Anna Winters,

Head of Creativity Team, Organisational Development and Learning | People Directorate | Scottish Government

30

Cre

ativ

e Ev

alua

tion

| Vic

toria

Qua

y

Delivered by...This workshop was hosted and conceived of by Anna Winters and the Creativity Team within Organisation and Development in the Scottish Government. The content and structure was developed by Dr Leon Cruickshank, in ImaginationLancaster, the cross-disciplinary design research center in Lancaster University.

For more information on ImaginationLancaster, please visit the following webpages or email Leon Cruickshank.

l.cruickshank@lancaster.ac.uk

http://imagination.lancs.ac.uk/people/Leon_Cruickshank