evaluation of the swansea rural development business plan 1: 2008 to 2011

61
wavehill ymchwil gwerthuso arolygon research evaluation surveys Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011 Final report A report for the Swansea Rural Development Partnership March 2011 This report and the projects discussed within it were part funded through the Rural Development Plan for Wales 2007 - 2013 which is financed by the European Union and the Welsh Assembly Government.

Upload: wavehill

Post on 20-Mar-2016

214 views

Category:

Documents


0 download

DESCRIPTION

The final report of an evaluation of activities funded in Swansea as part of the Rural Development Plan for Wales between 2008 and 2011.

TRANSCRIPT

Page 1: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

wavehill

ymchwil gwerthuso arolygon

research evaluation surveys

Evaluation of the Swansea Rural Development

Business Plan 1: 2008 to 2011

Final report

A report for the Swansea Rural Development Partnership

March 2011

This report and the projects discussed within it were part funded through the Rural Development Plan for

Wales 2007 - 2013 which is financed by the European Union and the Welsh Assembly Government.

Page 2: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

Acknowledgements We would like to offer our thanks to all those interviewed as part of the evaluation who gave their

time so willingly and answered our questions in such detail. This report would not have been

possible without all these contributions.

Report prepared by:

Endaf Griffiths ([email protected])

Louise Petrie

Client contact:

Paul Relf, European Team - City and County of Swansea ([email protected])

Report version: FINAL

Cover photograph: Oxwich Castle, the Gower Peninsula.

Page 3: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

Contents

Executive Summary .......................................................................................................... 1

1. Introduction .............................................................................................................. 6

1.1. Background ............................................................................................................................. 6

1.2. The evaluation......................................................................................................................... 8

1.3. Structure of the report ............................................................................................................ 9

2. Inputs, outputs, results and impact .......................................................................... 10

2.1. Input (expenditure) ............................................................................................................... 11

2.2. Outputs and results ............................................................................................................... 12

2.3. Impact ................................................................................................................................... 14

2.4. Achieving the aims and objectives of the LDS ...................................................................... 18

3. Management and delivery ....................................................................................... 20

3.1. Development of the LDS, the business plan and the projects .............................................. 21

3.2. Management structures ....................................................................................................... 23

3.3. Delivering the projects .......................................................................................................... 26

3.4. Looking forward – Business Plan 2 ........................................................................................ 30

4. Conclusions and recommendations .......................................................................... 31

4.1. Conclusions ........................................................................................................................... 31

4.2. Recommendations ................................................................................................................ 31

Appendix 1: Project level inputs, outputs, results and impact .......................................... 33

Rural Swansea Walking and Cycling Marketing Campaign ............................................................... 34

Improvement of Walking Facilities in Mawr Community ................................................................. 36

The Rural Youth Project .................................................................................................................... 38

Local Food for Local Markets ............................................................................................................ 41

Local Arts, Crafts and Natural Products for Local Markets ............................................................... 48

Appendix 2: Local Development Strategy themes and objectives .................................... 55

Page 4: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 1

Executive Summary

Introduction

The Rural Development Plan 2007-2013 (the RDP) is the mechanism by which the Welsh Assembly

Government is delivering European funded activities which support the countryside and rural

communities. In broad terms, the types of activities the RDP supports include working with the

countryside and rural communities, encouraging the sustainable management of agriculture and the

environment. The RDP Is split into four parts, known as ‘Axis’:

• Axis 1: The competitiveness of agriculture and forestry

• Axis 2: Our environment and countryside

• Axis 3: The quality of life in rural areas

• Axis 4: The Leader approach: locally based approaches to rural development

The Welsh Assembly Government is managing and implementing Axis 1 and Axis 2 activities

nationally. Axis 3 and Axis 4 are however being implemented on the basis of Local Authority areas.

Two local governing entities are required:

• A Partnership to devise and oversee the delivery of a Local Development Strategy (the LDS)

including steering the activities that are being run under Axis 3; and

• A Local Action Group (LAG) to steer activities that are being run under Axis 4.

In Swansea the role of the Partnership is undertaken by the Swansea Rural Development Partnership

and the role of the LAG by Rural Swansea Action. Both were set up specifically in order to deliver the

RDP in the rural parts of Swansea. The role of the Lead Body is undertaken by the Local Authority.

This includes administrating and monitoring the delivery of projects on behalf of the Partnership as

well as ensuring that all of the requirements of the funding agreement with the Welsh Assembly

Government are met.

The projects that will deliver the LDS are the basis for business plans that cover two periods.

Business Plan 1 – which is the subject of this evaluation - covered the period June 2008 to the end of

February 2011. Business Plan 2 will be implemented from the March 2011 to the end of February

2013.

In Swansea, Business Plan 1 included five projects, three funded from Axis 3 and two from Axis 4:

• Rural Swansea Walking and Cycling Marketing Campaign (Axis 3) - This project implemented a

marketing campaign to highlight the walking and cycling product offered within Rural Swansea.

• Improvements to Walking Facilities in Mawr Community (Axis 3) – The creation and promotion

of a circular walk in Cwm Cerdinen and a linear route along Cwm Clydach, both in Mawr

Community.

• Rural Youth Project (Axis 3) – This project developed and provided a mobile youth project to

work within the rural communities and wards to the West and North of Swansea City and County

Council.

Page 5: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 2

• Rural Swansea Action: Local Food for Local Markets (Axis 4) - This project aimed to undertake

work to analyse the capacity and needs of local agriculture and food sectors and engage the

wider community in economic activity using the ‘Leader Approach’.

• Rural Swansea Action: Local Arts, Crafts and Natural Products for Local Markets (Axis 4) - This

project aimed to undertake feasibility work to analyse the local Arts, Crafts and Natural Products

sector and local market opportunities, and to engage the wider community in economic activity

using the ‘Leader Approach’.

The evaluation

The evaluation was undertaken in three phases between December 2009 and March 2011 by

evaluation specialists Wavehill. Its aim was to review the implementation of Business Plan 1 with its

underpinning projects against:

a) Implementation of the LDS, and

b) How effectively the Partnership, Rural Swansea Action and Council worked under Business Plan

1, with an eye to preparations for Business Plan 2.

The research undertaken was split into three overlapping parts:

• Strategic level interviews with members of the Partnership, LAG and the Lead Body;

• Reflective interviews with those delivering the projects; and

• Interviews with a small sample of beneficiaries (businesses, communities and individuals

supported by the projects) together with a review and analysis of the monitoring information

being collected.

Importantly, the methodology was designed to collect views on the projects being delivered from a

range of different perspectives – those involved in the projects at a strategic level, those involved in

their day to day delivery and also from the beneficiary’s perspective. Over the course of the

evaluation, a total of 75 interviews have been undertaken.

Inputs, outputs, results and impacts

Following the ‘impact model’ illustrated below, the evaluation considered the input, outputs, results

and impacts that have been generated by the Business Plan 1 projects.

Page 6: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 3

The key findings were:

• It is forecast that each of the projects will achieve the targets set for expenditure (input),

outputs and results.

• The interim evaluation (March 2010) found that very little progress had been made by any of the

projects against the majority of the targets that had been set. The fact that projects have been

able to ‘catch-up’ is therefore to be welcomed although it suggest that much of the activity of

projects has been squeezed into the final year of the projects.

• There was a general lack of data available from projects that would allow the impact of activity

to be measured. The limited data that was gathered does however suggest that projects have

had (or will have) a positive impact. The evidence that is available also suggests that progress

has been made in terms of achieving the aims and objectives of the LDS via Business Plan 1.

• The introduction of a rural programme in Swansea and the structures that it entails has also had

an impact. Support is available specifically for the rural parts of the County for the first time and

stakeholders were of the view that the approach being used to manage and deliver the RDP

projects is contributing towards improving governance in the rural areas – one of the objectives

of the RDP programme.

Management and delivery

The management and delivery of the RDP programme in Swansea was reviewed with a particular

emphasis on identifying the ‘lessons learnt’ that can be applied during the delivery of Business Plan

2. The key findings were:

• When considering the management and delivery of the RDP programme in Swansea, it is

important to be aware of the fact that this is the first time a ‘rural programme’ has been

delivered within the County. All the structures put in place including the Partnership and the LAG

are therefore ‘new’. Accordingly, a conservative approach was taken for Business Plan 1 and, the

evidence suggests that the approach was the correct one.

• The feedback from stakeholders on the effectiveness of the management structures put in place

(the Partnership, the LAG and the Lead Body) was largely positive.

• Attendance records for the Partnership’s show that, on average, members from the ‘community

sector’ have attended a far higher percentage of meetings than their counterparts from the

other sectors.

• The Partnership has been reliant on a number of key individuals; only 7 members of the

Partnership have been to 75% or more of meetings. This is something that needs to be taken

into account as the Partnership (and the LAG) develops especially given the increase in

responsibility and workload that is inevitable with the implementation of the far larger Business

Plan 2.

• The role of the Lead Body has been the critical factor in the delivery of the RDP in Swansea to-

date. The programme would not have been ongoing in the area without the commitment to the

programme shown by the Local Authority.

Page 7: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 4

• Feedback from stakeholders on the delivery of the Business Plan 1 projects was generally

favourable. When asked what they would change given the opportunity to start again, tweaks

were suggested rather than fundamental changes.

• Bureaucracy was a common theme during discussions with stakeholders regarding the delivery

of projects and it is an issue that has clearly slowed progress in terms of delivering the projects.

• The short time in which Business Plan 1 projects needed to be delivered was an underlying issue

in all discussions with stakeholders at all phases of the evaluation.

• The ‘inclusive’ or ‘bottom-up’ approach being used to deliver the projects was highlighted as one

of the main strengths of Business Plan 1.

• The evaluation has found that monitoring was not embedded within all of Business Plan 1

projects and there was a general lack of basic data for the evaluators amongst a number of the

projects.

Conclusions

This evaluation has been positive in many respects. Our main conclusion is that Business Plan can be

considered a success in that it has ‘established’ the Rural Development Plan in the County for the

first time. A Rural Partnership and a LAG now exist and their capacity has clearly developed during

the Business Plan 1 period. However, it is also clear that Partnership and LAG members have been on

a ‘steep learning curve’ and that it is yet to become the ‘finished article’. The focus on developing

the Partnership and the LAG therefore needs to be maintained’

The Partnership faces a number of challenges as it gears up for the delivery of the significantly larger

Business Plan 2. In particular, the Partnership is reliant on a few members who attend meetings on a

regular basis and the pressure on those few members is likely to increase significantly as the

workload increases with the introduction of the much larger Business Plan 2.

Each of the projects delivered can also be considered a success in that they have been established –

especially the Axis 4 projects which are providing their particular type of support in the County for

the first time – and have (if the forecasts are accurate) achieved the output and result targets that

were set. Further, there are no obvious problems with the way in which they have been delivered

with stakeholders highlighting the ‘inclusive’ and ‘bottom-up’ approach being used as a particular

strength of the approach. The limited data that was available and which was gathered by the

evaluation also suggests that projects have had or will have a positive impact. However, the general

lack of data on the impact of the projects needs to be addressed, especially as the value of the

activities being delivered in the County under the RDP increases from just over £340,000 to in excess

of £5m.

We have also concluded that the conservative approach employed for Business Plan 1 was the

correct one as it allowed the Partnership, the LAG and the Lead Body the breathing space to develop

and become accustomed to the RDP and all it entails. The role of the Lead Body has also been critical

during this period; the programme would not have been ongoing in the area without the

commitment to the programme shown by the Local Authority.

Page 8: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 5

Recommendations

1. Options for increasing the membership of the Partnership should be explored. More

importantly, options for a more even distribution of roles and responsibilities amongst

Partnership members should be explored.

2. The potential to further develop the ‘capacity’ of Partnership / LAG members via appropriate

training should be explored. This should include training to increase members’ knowledge and

understanding of the RDP programme and of their role as Partnership / LAG members.

3. The Partnership and the LAG should explore options for increasing the focus on monitoring and

evaluating the impact of projects delivered as part of Business Plan 2. Process should also be put

in place to monitor the ongoing impact of Business Plan 1 projects over the coming months and

years.

4. Whilst recognising that the delivery of any European funded project comes with a certain

amount of bureaucracy, the Lead Body should work with the Welsh Assembly Government to

ensure that the burden upon the schemes being supported is as little as possible.

5. Whilst it may not be possible to take action relating to the findings on the impact of the short

timescales for projects and the consequence of splitting the RDP programme into two periods on

a local level, they are important lessons learnt. We would therefore recommend that they are

fed back to the Assembly Government with a view to informing the development of any future

programme(s).

Page 9: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 6

1. Introduction In October 2009, the City and County of Swansea – on behalf of the Swansea Rural Development

Partnership - commissioned evaluation specialists Wavehill to undertake an ongoing evaluation of

the first business plan put together to deliver projects funded by the Rural Development Plan for

Wales in the rural parts of Swansea. This is the final report of that evaluation which was undertaken

alongside the implementation of the projects within the business plan which ran until the end of

February 2011. It follows a baseline report produced in December 2009 and an interim report in

March 2010.

This first section provides the context for the remainder of the report by introducing the project,

how it was funded and how the evaluation was undertaken.

1.1. Background

1.1.1. The Rural Development Plan for Wales

The Rural Development Plan 2007-2013 (the RDP) is the mechanism by which the Welsh Assembly

Government is delivering European funded activities which support the countryside and rural

communities1. The RDP for Wales secures EU funding from the European Agricultural Fund for Rural

Development with Wales managing a budget of £795 million over the lifetime of the RDP with £195

million coming from Europe. There are 4 key areas that the plan will improve:

• Axis 1: The competitiveness of agriculture and forestry

• Axis 2: Our environment and countryside

• Axis 3: The quality of life in rural areas

• Axis 4: The Leader approach: locally based approaches to rural development

In broad terms, the types of activities the RDP supports include working with the countryside and

rural communities, encouraging the sustainable management of agriculture and the environment.

The Welsh Assembly Government is managing and implementing Axis 1 and Axis 2 activities

nationally. Axis 3 and Axis 4 are being implemented on the basis of Local Authority areas, by

accountable and representative local partnerships capable of reflecting the needs and aspirations of

indigenous rural communities.

The requirements of the Welsh Assembly Government for the delivery of the Rural Development

Plan for Wales note that two local governing entities are required: a Partnership to devise and

oversee the delivery of a Local Development Strategy including steering the activities that are being

run under Axis 3; and a Local Action Group (LAG) to steer activities that are being run under Axis 4.

One of the roles of the Partnership is to appoint a Lead Body that administers the RDP projects on

the Partnership’s behalf. This includes monitoring the delivery of projects and ensuring that all of the

requirements of the funding agreement with the Welsh Assembly Government are met.

1 Further information about the Rural Development Plan for Wales can be found here:

http://wales.gov.uk/topics/environmentcountryside/farmingandcountryside/ruraldevelopment/?lang=en

Page 10: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 7

In Swansea the role of the Partnership is undertaken by the Swansea Rural Development Partnership

and the role of the LAG by Rural Swansea Action. Both were set up specifically in order to deliver the

RDP in the rural parts of Swansea. The role of the Lead Body is undertaken by the Local Authority.

1.1.2. The Rural Swansea Local Development Strategy and Business Plan 1

The Rural Local Development Strategy (the LDS) 2007-2013 for the

City and County of Swansea area was prepared by the Swansea

Rural Development Partnership to enable delivery of Axis 3 and 4

of the RDP in the rural parts of Swansea. The overarching aim

agreed by the Swansea Rural Development Partnership (hereafter

‘the Partnership’) in the LDS is:

Improving the quality of life in City and County of Swansea’s rural

areas, encouraging diversification of the rural economy, and

maximizing advantages offered by the natural environment to

support the development of sustainable agriculture and tourism

and the potential for small scale rural based businesses such as

those linked to information and communications technology (ICT).

The content of the Strategy reflects the activities possible under the RDP viewed through the

distinctive needs of the rural parts of the City and County of Swansea. It is the first major funding

opportunity for the rural hinterland of Swansea, which has not previously benefited from rural

development programmes, despite the fact that over 60% of the geographical coverage of the

county is predominantly rural in nature.

The projects that will deliver the LDS are the basis for business plans that cover two periods.

Business Plan 1 – which is the subject of this evaluation - covers the period June 2008 to the end of

February 2011. Business Plan 2 will be implemented from the March 2011 to the end of February

2013.

Business Plan 1 included five projects, three funded from Axis 3 and two from Axis 4:

• Rural Swansea Walking and Cycling Marketing Campaign (Axis 3) - This project implemented a

marketing campaign to highlight the walking and cycling product offered within Rural Swansea.

• Improvements to Walking Facilities in Mawr Community (Axis 3) – The creation and promotion

of a circular walk in Cwm Cerdinen and a linear route along Cwm Clydach, both in Mawr

Community.

• Rural Youth Project (Axis 3) – This project developed and provided a mobile youth project to

work within the rural communities and wards to the West and North of Swansea City and County

Council.

• Rural Swansea Action: Local Food for Local Markets (Axis 4) - This project aimed to undertake

work to analyse the capacity and needs of local agriculture and food sectors and engage the

wider community in economic activity using the ‘Leader Approach’.

• Rural Swansea Action: Local Arts, Crafts and Natural Products for Local Markets (Axis 4) - This

project aimed to undertake feasibility work to analyse the local Arts, Crafts and Natural Products

sector and local market opportunities, and to engage the wider community in economic activity

using the ‘Leader Approach’.

Page 11: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 8

The delivery of all three Axis 3 projects was procured on behalf of the Partnership by the Lead Body

(the City and County of Swansea); in other words, organisations and businesses were invited to

tender for the contract to deliver those projects. The contracts to deliver all three projects were

ultimately awarded to the City and County of Swansea. Rural Swansea Action oversees and

implements Axis 4 activities.

1.2. The evaluation

The aim of the evaluation was to review the implementation of Business Plan 1 with its underpinning

projects against:

a) Implementation of the LDS, and

b) How effectively the Partnership, Rural Swansea Action and Council worked under Business

Plan 1, with an eye to preparations for Business Plan 2.

As noted previously, the evaluation was undertaken in three phases. The baseline report was

produced in December 2009. That report focused on the way in which the LDS and the projects

within Business Plan 1 had been developed. It also set out the framework that would be used in the

remaining phases of the evaluation. The interim evaluation report reviewed the progress that had

been made by March 2010 and was followed by this final report in March 2011.

Over the three phases, the methodology being used to undertake the evaluation split the fieldwork

into three overlapping parts:

• Strategic level interviews with members of the Partnership, LAG and the Lead Body;

• Reflective interviews with those delivering the projects; and

• Interviews with a small sample of beneficiaries (businesses, communities and individuals

supported by the projects) together with a review and analysis of the monitoring information

being collected.

Importantly, the methodology was designed to collect views on the projects being delivered from a

range of different perspectives – those involved in the projects at a strategic level, those involved in

their day to day delivery and also from the beneficiary’s perspective. Over the course of the

evaluation, a total of 75 interviews have been undertaken.

1.2.1. Limitations of the evaluation

Every evaluation has its limitations which need to be taken into account when findings are being

discussed. The main limitation in this case was resource; the limited number of days which the

Wavehill team had available to undertake the evaluation - 25.5 days over the three phases.

However, it is important to stress that, despite the limitations, we would consider the findings of this

evaluation to be robust.

Page 12: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 9

1.3. Structure of the report

The remainder of this report is structured as follows:

• Section 2 will review the inputs, outputs, results and impact of the projects;

• Section 3 will review the management and delivery of the projects and the programme as a

whole; and

• Finally, Section 4 will set out the conclusions and recommendations of the evaluation.

Page 13: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 10

2. Inputs, outputs, results and impact Following the ‘impact model’ illustrated by the graphic below, this section will consider the input,

outputs, results and impacts that have been generated by the Business Plan 1 projects.

Key findings

• It is forecast that each of the projects will achieve the targets set for expenditure, outputs and

results.

• The interim evaluation found that very little progress had been made by any of the projects

against the majority of the targets that had been set. The fact that projects have been able to

‘catch-up’ is therefore to be welcomed although it suggest that much of the activity of projects

has been squeezed into the final year of the projects.

• There was a general lack of data available from projects that would allow the impact of activity

to be measured. The limited data that was gathered does however suggest that projects have

had or will have a positive impact.

• The evidence that is available also suggests that progress has been made in terms of achieving

the aims and objectives of the LDS via Business Plan 1.

• The introduction of a rural programme in Swansea and the structures that it entails has also had

an impact. Support is available specifically for the rural parts of the County for the first time and

stakeholders were of the view that the approach being used to manage and deliver the RDP

projects is contributing towards improving governance in the rural areas – one of the objectives

of the RDP programme.

Page 14: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 11

2.1. Input (expenditure)

The total value of the RDP programme in Swansea was just under £339,000. As shown in table 1,

69% of the funding for the RDP projects in Swansea was provided from the RDP programme with the

‘match-funding’ being sourced from a variety of sources, most notably the City and County of

Swansea.

Table 1: Summary of funding sources

Funding source Amount Percentage

RDP programme (European Union/Welsh Assembly Government) £234,635 69%

City & County of Swansea £49,004 14%

Gower AONB Sustainable Development Fund £7,813 2%

Countryside Council for Wales £9,000 3%

Welsh Church Act Fund £1,500 0%

Volunteer research in-kind £13,892 4%

TOTAL £338,859 100%

Source: project monitoring data provided by the Lead Body

Table 2: Breakdown of funding per Axis 3 and 4

Category Amount Percentage

Axis 3 (total) £175,534 52%

Axis 4 (total) £130,643 39%

Lead body costs Axis 3 £16,748 5%

Lead body costs Axis 4 £15,934 4%

Total £338,859 100%

Source: project monitoring data provided by the Lead Body

Table 3: Breakdown of expenditure per project

Project Amount Percentage

Walking and Cycling Mktg Campn. (Axis 3) £28,000 9%

Improv. Walking Mawr Community (Axis 3) £28,009 9%

Rural Youth Project (Axis 3) £123,283 40%

Local Food for Local Markets (Axis 4) £82,485 27%

Local Arts Crafts and Natural Products for Local Markets (Axis 4) £48,158 16%

Total £309,935 100%

Source: project monitoring data provided by the Lead Body

Table 2 breaks down the expenditure per Axis and per project. It shows that 91% of expenditure was

on projects and 9% of administration in the form of lead body costs. In expenditure terms, the

largest project, by some distance was the Rural Youth project taking 40% of the total budget for

projects (Table 3).

Page 15: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 12

2.2. Outputs and results

Tables 4 and 5 show the performance of Business Plan 1 against ‘headline’ output performance

indicators set for the Axis 3 and 4 projects respectively.

Table 4: Axis 3 headline output indicators

Project Performance indicators Target

2008-

2011

Actual

achieve.

to date

%

achieved

Forecast

to

28 Feb

2011

Walking and

Cycling

Marketing

Campaign

Number of projects for the development and/or

marketing of rural tourism services financially

supported

1 1 100% 1

Improvement

of Walking

Facilities

Number of small scale tourism infrastructure projects

financially supported 2 2 100% 2

Rural Youth

Number of children (0-16) participating in the project 500 489 97.8% 500

Number of young people (16-20) participating in the

project 250 203 81.2% 300

Source: project monitoring data provided by the Lead Body

Table 5: Axis 4 headline output indicators

Project Performance indicators Target

2008-

2011

Actual

achieve.

to date

%

achieved

Forecast

to

28 Feb

2011

Local Food

Number of consultation events undertaken in the

development of a project 28 20 71% 28

Number of consultation exercises undertaken 29 31 106% 31

Number of feasibility studies undertaken 5 3 60% 5

Number of individuals from under-employed groups

participating in the project 10 6 60% 10

Number of cooperation initiatives advised or assisted

in the agri-food sector 29 31 106% 31

Local Arts,

Crafts and

Natural

Products

Number of consultation events undertaken in the

development of a project 20 18 90% 20

Number of consultation exercises undertaken 30 35 116% 30

Number of marketing & promotional activities

undertaken 80 119 148% 119

Number of individuals from 'under-employed groups'

participating in the project 30 33 110% 33

Source: project monitoring data provided by the Lead Body

In some instances, due to the fact that the evidence required to demonstrate that the target has

been achieved was not yet available, it was not possible to confirm that all targets had been

achieved at the time of this report. However, it is anticipated that the targets set for each of the

outputs will be achieved by the end of the project. In some instances, most notably the number of

marketing and promotional activities undertaken as part of the Local Arts, Crafts and Natural

Products for Local Markets projects, an overachievement is being reported.

Page 16: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 13

Tables 6 and 7 show the performance of Business Plan 1 against ‘headline’ result performance

indicators set for the Axis 3 and 4 projects respectively.

Table 6: Axis 3 headline result indicators

Project Performance indicators Target

2008-

2011

Actual

achieve.

to date

%

achieved

Forecast

to

28 Feb

2011

Walking and

Cycling

Marketing

Campaign

Distribute 200k walking and cycling guides 200,000 136,500 68% 200,000

Improvement

of Walking

Facilities

Gross number of additional visitors to a tourism

project that has been supported 1,000 0 0 1,000

Rural Youth Increase in recorded outcomes with 11-19 year olds 450 148 33% 450

Increase in accredited outcomes with 11-19 year olds 90 64 71% 90

Source: project monitoring data provided by the Lead Body

Table 7: Axis 4 headline result indicators

Project Performance indicators Target

2008-

2011

Actual

achieve.

to date

%

achieved

Forecast

to

28 Feb

2011

Local Food

Number of enterprises created 12 0 0% 12

Gross number of jobs created 18 0 0% 18

Gross number of jobs safeguarded 22 0 0% 22

Number of cooperatives potentially created 3 2 66% 3

Number of enterprise groups, producer groups,

cluster groups or formal networks created 3 2 66% 3

Local Arts,

Crafts and

Natural

Products

Number of micro enterprises potentially created 2 2 100% 2

Number of jobs potentially created 8 tbc tbc 8

Number of jobs potentially safeguarded 6 tbc tbc 6

Number of communities engaged 8 8 100% 8

Number of groups engaged 9 7 77% 9

Number of individuals engaged 100 218 218% 100

Source: project monitoring data provided by the Lead Body

Again, the Lead Body is forecasting that all targets will be achieved by the end of the project.

However, in some instances, the gap between what could be reported as being achieved and what

was forecast was far more significant; 100% in a number of instances.

The interim evaluation report in March 2010 found that, at that time, very little progress had been

made by any of the projects against the majority of the targets that had been set. The fact that

projects have been able to ‘catch-up’ is therefore to be welcomed, although it suggests that much of

the activity of projects has been squeezed into the final year, an issue discussed further in the

following section of the report. The ‘late running’ of the projects would also explain the lack of data

regarding results to date; results cannot be achieved until outputs have been achieved.

Page 17: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 14

2.2.1. The indicators being used

As evaluators, we are would question the validity of some of the indicators being used, in particular

those that measure the potential to create or safeguard jobs and enterprises. Whilst we understand

the argument for including such an indicator within a programme designed to pilot new and

innovative ideas (i.e. Axis 4), the ambiguity of the term ‘potential’ makes the indicator difficult to

define and, more importantly, to measure. How do we know when a potential job has been achieved

or a potential micro-business established? These are examples, we would argue, of performance

indicators being used to describe activity and priorities rather than measure achievement.

The ‘potential’ indicators are amongst the additional indicators introduced by the Assembly

Government from 2010 to provide additional evidence of the achievements of Axis 4 projects

(shown in the second part of the table on the following page). Again, we understand (and in fact

support) the argument for introducing indicators that allow Axis 4 achievements to be monitored

and that reflect the ‘innovative’ nature of the activities being undertaken under Axis 4. The

introduction of a revised set of indicators at the mid-point of projects has however clearly caused

confusion and uncertainty and added to the bureaucratic burden upon the projects. Again, this is an

issue that Section 3 of the report will discuss further.

2.3. Impact

2.3.1. The availability of data

A key part of the final phase of the evaluation was to review the impact of the projects and the RDP

programme in Rural Swansea as a whole. As such, the research team spent a significant amount of

time discussing the perceived impact of projects with stakeholders. However, part of the

methodology was also to review the impact monitoring information collected by projects and to

contact a small sample of project beneficiaries directly in order to explore the impact of the support

provided with them. There was however a general lack of relevant data available to do this.

For the Axis 4 projects the project manager provided the evaluation team with contacts for

beneficiaries of the project a small sample of whom we subsequently contacted and interviewed.

However, other projects could not provide even basic information, such as contact details for

beneficiaries within the timescales available to undertake the fieldwork. This limited the evaluation’s

ability to assess the impact of the support provided. One of the evaluation’s recommendations is

therefore that the Partnership reviews what information can and should be collected by projects in

order to review the impact of the projects at some point in the future. We would also recommend

that the monitoring of Business Plan 1 projects continue into the Business Plan 2 period where it is

necessary to do so in order to measure the impact of the support provided.

2.3.2. Impact of projects

A review of the impact of each of the individual projects can be found in Appendix 1 of this report.

The evidence suggests that each of the projects had or will have a positive impact. In some instances

that impact is however only perceived or anticipated for some point in the future.

Page 18: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 15

In many instances stakeholders considered it to be “too early” to ascertain what the impact of the

projects would be. For example, it is anticipated that the Rural Swansea Walking and Cycling

Marketing Campaign will lead to an increase in the number of visitors to the area / users of the local

walking and cycling resources. However, there was no evidence of such an impact to date primarily

because it was, as the project manager put it “too early to tell” but also because there was no

process within the project for measuring the impact of the campaign. Nor was it possible to

ascertain to where the leaflets had been distributed within the timescale available to undertake the

evaluation.

For the Walking Facilities in Mawr project, the focus had been very clearly on completing the

project: completing the works at the site. This is understandable and there is the potential to

measure the impact of the project later as counters had been installed at the site which would

monitor the number of users. The small number of local businesses interviewed were however

sceptical of the impact the project would have on their business. It was also clear that they had not

been engaged in the project which is another indication of the focus on outputs (completing the

works) rather that outcomes – maximising the impact of the project on the local community and

economy. One way to do that would be to engage with local businesses and encourage them to

promote the facility.

As evaluators we would have anticipated that impact would have been more evident in the other

three projects and in particular the Rural Youth projects which had engaged with 750 children and

young people. Whilst the project staff identified a wide range of positive impacts, the project could

provide no evidence in support of the perceived impacts. Neither was it possible to undertake

interviews with a significant number of participants in the project; the project only provided contact

details for 12 young people and that was at a very late stage in the evaluation process which limited

our ability to contact them. Two interviews were ultimately undertaken both of which were very

positive, but the sample was much smaller than we would have hoped.

• [I have benefited] massively. There's so much more to the Youth Service that I didn't know about

and I know a lot more about it now. It's brought a whole new area to me and I've realised how

big it is, I've met new people and established relationships with other people I've got to know.

The help that you get from the other workers has really increased my knowledge. It's increased

my confidence not only as a youth worker, it's something different to do, something different all

the time. I go up to Penclawdd where there's always community projects going on we've helped

with flower beds where you're working with people of all ages. When the bus comes round, it

parks and we use the environment as well as the bus for the sessions. I definitely feel I have a

voice as I said I've got to know more of the Youth Service and they come back to you asking what

you want to do which makes you feel valued.

To supplement the interviews with participants, we also interviewed 6 stakeholders who had

participated in the project in different ways: two police officers, a community development officer,

two volunteers and a representative of a local centre. They also identified a positive impact on the

children and young people being supported. There is therefore evidence of a positive impact but it

could be, and should be, stronger.

For the Axis 4 projects, adequate contact details for businesses participating in the project could be

made available. On this occasion however, it was the limited resources available within the

evaluation that constrained the amount of evidence that could be gathered. Nine interviews were

undertaken with businesses that had participated in the Local Food project and eight with

businesses that participated in the Local Arts, Crafts and Natural Products project.

Page 19: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 16

As set out in Appendix 1, there is evidence that both projects had a positive impact on the

businesses participating. Some of the businesses participating in the Local Food project seem to have

benefited significantly. Two of the businesses interviewed were more confident in their future due

to the support provided:

• As a result of meetings with [the project manager] it has opened up new avenues for our

business, we were new to this and we have now been able to access other like businesses for

information.

• It has increased the possibility of increasing our business and our profile, made other people

more aware of our product and our services.

Four of the businesses said that they had been encouraged to work more with local businesses.

• We supply [company name] next door.

• This also comes with working the markets too... you form allegiances with like minded people

and share information.

• With local businesses and further afield.

• I do have a few more customers.

Other benefits identified by the businesses were:

• Yes it has given an awareness of our business and it has opened new doors.

• Yes, it's about raising awareness and from an encouragement point of view it has been great.

They have taken information of our business to stands at events. I know I can ring [the project

officer] any time and I see her at events which is reassuring. The project is very responsive and

pro active. She sent us information on what we can do with our fleeces and although we have not

taken any action on that yet we may do so in the future.

• They have raised awareness and taken our products and logos to events and promote us.

• Yes, it has given us the useful opportunity, a window to portray our services and products at

events.

The impact of the project was however not universal with 4 of the 9 interviewed stating that their

expectations had not been met:

• I think the project is more suited to the crafts and small producers, we are more wholesale.

• There is no funding to invest in the business at the moment, from what I gather is that the last 2

years have been to get everything in place for this project and maybe funding will come through.

It would be useful to get funding for publicity.

• They are limited in what they can do. They can advertise and publicise and spread the word

around but when we asked them if we could use them to go through to the council to put up

banners and advertise they could not. A lot more could be done, Tourism in Swansea Bay when

they have things on they have lots of outside catering in when there should be more local

produce. Why aren't local producers at the Air Show and the Boat show, why not have an event

in Singleton Park

• It has not increased our business.

Page 20: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 17

A positive impact was more common amongst the eight participants in the Local Arts, Crafts and

Natural Products projects that we interviewed. When asked whether their confidence had changed

due to the support they had received, 6 of the 8 said that it had:

• I feel the project has given me confidence, they have been so encouraging, they make me feel

like I am more than just someone who sits in the house and sews all day. I don't have to go to

them, they come to me.

• My business will go somewhere but I do have a lack of confidence in myself because I can't

keep up now, the business has taken off and I have employed 2 knitters and looking for a

spinner. I have had to cancel 2 markets because I cannot knit fast enough. Not sure if I want all

this, I want to spin and knit and do it myself.

• They have given me more confidence and I am more driven to pursue. There are no ways of

showing your stuff, if there was a Craft guild which could suggest new events, something

regular to show your wares.

• I was confident and positive about my business and the future but my husband has had a

terrible accident and I am now looking after him, the odd order trickles in but home life is now

centred around my husband I don't have any spare time.

• I have more focus and more confident and my business has gone from being just local to

national, it really has evolved.

• There are quite a few strands of the business, we are breeding alpacas and using their fleeces,

there are now quite a few extra bits I can add to it. We have open days and walking with

Alpacas is something we might develop. The problem is the recession.

Whilst just 1 of the businesses had introduced new products or services as a result of the support

provided 7 of the 8 said that they had been encouraged to work more with local businesses.

• We are always looking to work with local crafters and producers.

• Through the producers network I now go to another craft fair and some shops stock my product.

• I was introduced to XXXXX who runs Gorseinon market that is where the business started going

out of control and I now have to employ knitters.

• I have been chatting to other crafters, picking up new ideas and sharing information.

• I was busy networking before my husband had the accident, now I don't do any work.

• I have talked and shared ideas with people who have Alpacas.

• I have talked to other people involved in wood and glass and spinning and the possibilities of

working together.

It is also worth noting that 13 of the 17 businesses that we interviewed had remained engaged with

the project in question after their initial contact with them by participating again in another activity.

It is unlikely that they would do so if they did not believe they were benefiting in some way by

engaging with the project.

It is however again important to stress that the samples were very small and we would recommend

that the Partnership considers commissioning a larger (more representative) sample of businesses

that have benefited from the support provided as part of the second phase of the projects in order

to provide better data on the impact the support is having.

Page 21: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 18

2.3.3. Impact of the process

Whilst the primary impact of any programme is that it has on its beneficiaries, it is also important to

consider the impact of ‘the process’. As discussed further in the following section of the report, this

is the first time a specific rural programme of support has been delivered in Swansea. As such, the

structures that have been put in place to manage the programme are new - there is now a structure

in place in Swansea to provide support that is designed to specifically support the rural parts of the

county.

One of the objectives of the European Commission for its Rural Programme (which funds the RDP in

Wales) is that its activities - particularly those of Axis 4 - should contribute to improving the

governance in the rural areas. The general consensus amongst stakeholders when discussing this

matter was that the approach being used to manage and deliver the RDP projects is contributing

towards improving governance in the rural areas. Their rational for this was that strong links have

been created through the RDP between Local Government at both County level and Community

Council level, which in turn has lead to better inclusiveness of the broader community through

consultation events, which have enabled the voices of citizens to be heard by decision and policy

makers.

It was also highlighted that local Members of the National Assembly for Wales and local Members of

Parliament had also been involved in RDP events in Swansea and/or have been briefed on the RDP

programme in the area. It was felt that this contributed further towards building links which will aid

and assist the improvement of governance and relationships between governments at local, regional

and national levels.

2.4. Achieving the aims and objectives of the LDS

One of the objectives of the evaluation was to consider how the Business Plan 1 projects had

contributed to the implementation of the LDS. As discussed in the following section, the LDS is very

broad in terms of the issues it discusses and the challenges it sets out to tackle; it includes 18

objectives within its 5 themes (please refer to Box 1 and Appendix 2). As such, Business Plan 1 was

only designed to contribute to achieving some of those objectives as explained in the extracts from

Business Plan 1 also included in Appendix 2. Has it done that?

There are clearly links between the themes of the LDS and outputs, results and impacts of the

Business Plan 1. Some of those are however more obvious than others. Local businesses have clearly

been supported and have benefited from the work of the Axis 4 projects. They will also, potentially,

benefit from the Walking and Cycling Marketing Campaign and the development of the walking

facilities in Mawr. The Rural Youth project has improved basic services in the rural areas.

However, the evidence of Business Plan 1’s progress against some of the other objectives is less

obvious. Has social exclusion been reduced? Have skills gaps been addressed? Has the provision of

public and community transport been improved? For example, the answer to these questions could

be yes. The Rural Youth project could have reduced social exclusion. There are links between the

walking facilities developed and public transport that could increase the use of public transport.

There is however, for reasons outlined previously, a lack of evidence at this time to confirm whether

these objectives has been met although we can say that progress has been made.

Page 22: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 19

Box 1: The LDS themes and objectives that Business Plan 1 project’s were designed to address

Theme 1: Supporting Rural Businesses

• Objective 1: To provide support for farming households to exploit new opportunities

(agricultural and non-agricultural, e.g. niche markets)

• Objective 2: To promote local food, drink, and crafts and link to branding

• Objective 3: To encourage sustainable tourism, and to strengthen the value and length of the

tourism season

Theme 2: Building Sustainable Rural Communities

• Objective 6: To improve basic services (including facilities for children, broadband, community

facilities)

• Objective 8: To support initiatives which encourage the use of the Welsh language and the

retention of Welsh culture

Theme 3: Tackling economic inactivity

• Objective 14: To reduce social exclusion by supporting local provision of services and education

facilities and better paid employment opportunities and by providing links between urban and

rural areas

Theme 4: Skills Development

• Objective 15: To address skills gaps - provide non-mainstream training to develop personal and

community confidence and to encourage people to obtain more qualifications

• Objective 16: To expand community led initiatives including capacity building, networking, and

essential skills for group management

• Objective 17: To support the development of outdoor education and youth activities in

conjunction with the City and County of Swansea, other local authorities, the YHA, and the

Swansea Environmental Education Forum

Theme 5: Sustainable Transport

• Objective 18: To improve public and community transport provision.

Page 23: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 20

3. Management and delivery As the title suggests, this section of the report focuses on management and delivery issues with a

particular emphasis on identifying the ‘lessons learnt’ that can be applied during the delivery of

Business Plan 2. The discussion draws upon the interviews with stakeholders during the baseline,

interim and final phases of the evaluation.

Key findings

• When considering the management and delivery of the RDP programme, it is important to be

aware of the fact that this is the first time a ‘rural programme’ has been delivered within the

County. All the structures put in place including the Partnership and the LAG are therefore ‘new’.

Accordingly, a conservative approach was taken for Business Plan 1 and, the evidence suggests

that the approach was the correct one.

• The feedback from stakeholders on the effectiveness of the management structures put in place

(the Partnership, the LAG and the Lead Body) was largely positive.

• Attendance records for the Partnership’s show that, on average, members from the ‘community

sector’ have attended a far higher percentage of meetings than their counterparts from the

other sectors.

• The Partnership has been reliant on a number of key individuals; only 7 members of the

Partnership have been to 75% or more of meetings. This is something that needs to be taken

into account as the Partnership (and the LAG) develops especially given the increase in

responsibility and workload that is inevitable with the implementation of Business Plan 2.

• The role of the Lead Body has been the critical factor in the delivery of the RDP in Swansea to-

date. The programme would not have been ongoing in the area without the commitment to the

programme shown by the Local Authority.

• Feedback from stakeholders on the delivery of the Business Plan 1 projects was generally

favourable. When asked what they would change given the opportunity to start again, tweaks

were suggested rather than fundamental changes.

• Bureaucracy was a common theme during discussions with stakeholders regarding the delivery

of projects and it is an issue that has clearly slowed progress in terms of delivering the projects.

• The short time in which Business Plan 1 projects needed to be delivered was an underlying issue

in all discussions with stakeholders at all phases of the evaluation.

• The ‘inclusive’ or ‘bottom-up’ approach being used to deliver the projects was highlighted as one

of the main strengths of Business Plan 1.

• The evaluation has found that monitoring was not embedded within all of Business Plan 1

project and there was a general lack of basic data for the evaluators amongst a number of the

projects.

Page 24: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 21

3.1. Development of the LDS, the business plan and the projects

3.1.1. Starting from scratch

Coverage of Swansea’s rural areas by a single comprehensive Local Development Strategy is a

new experience. This is the first time the many strands of rural development work in Swansea

have an opportunity to work together under a common banner. As such, this is a steep

learning curve for all involved, given the fairly limited levels of specific funding Swansea’s

rural areas have benefited from in the past. This is however, an opportunity that the new

Swansea Rural Development Partnership and Local Action Group are enthusiastic about

progressing.

Extract from Business Plan 1

The above extract from Business Plan 1 stresses the fact that this is the first time that Swansea has

benefited from support targeted at the rural areas in Wales; as such, the strategy – and its delivery

mechanism (the business plan and its projects) – have been developed or set up from scratch.

• “There is no history of cooperation between the rural parts of the County” – Partnership Member

(baseline phase)

This is a key point emphasised by stakeholders interviewed during each phase of the evaluation.

Unlike other parts of Wales who have been able to access funding to support their rural areas in the

past, everything that has been put in place to deliver the RDP in Swansea is, to a large extent, new.

• “We were a bit handicapped at the start because everything had to be done in such a rush

because of the WAG deadline - it left us with about 2 or 3 months to set everything up.” –

Partnership Member

As discussed later, it is also very clear from the discussions with interviewees that the timescales

that were in place for the development of the LDS and Business Plan 1 were “very challenging”. This

obviously caused problems, made all the more apparent by the fact that the LDS was being

developed “from scratch”; there was no track-record for delivering similar programmes or projects

in the area in the past. Hence, all the structures that were being put in place were new.

One of the main consequences of the above is that the LDS is very broad in terms of the issues it

discusses and the challenges it sets out to tackle; it includes 18 objectives within its 5 themes (please

refer to Appendix 2). This was intentional on the part of the Partnership who felt it was necessary to

be broad in order to ensure that the LDS was a document that all partners could support. Those

involved in its development also highlight the fact that, when the LDS was being developed, it was

unclear to them what type of project could be supported by RDP funding. Hence, it made sense to

limit the amount of detail within the LDS.

Those interviewed also highlighted the fact that the timescales involved meant that it was not

possible to undertake any kind of significant consultation exercise with local communities of

interested parties during the development of the LDS or the Business Plan; there simply wasn’t the

time to do so. As such, the focus was upon utilising structures that already existed (for example, the

Mawr Development Trust) and drawing interested parties into the Partnership.

Page 25: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 22

• “The strategy was written by those who were willing to get involved.” – Partnership Member

(baseline phase)

As evaluators, we support the approach taken. As one interviewee put it, “you’ve got to start

somewhere”. However, it is important to be aware that such an approach can lead to a situation

where some stakeholders may feel excluded from the process. This is something which the

Partnership was aware of and, as discussed below, sought to address during the delivery of Business

Plan 1 and the development of projects for inclusion in Business Plan 2.

3.1.2. A conservative approach

The SRLDS was intentionally drafted as an evidence base for development of activities as part

of the two main Business Plan phases. Due to the limited resources available, very new status

of the Partnership and embryonic Local Action Group, it was felt inappropriate to be too

prescriptive about potential activities in the Strategy itself. This would provide more scope for

elaborating solutions to the issues identified in the SWOT under the Business Plans. Due to

similar constraints during development of Business Plan 1, particular emphasis will be placed

on developing a more wide-ranging Business Plan 2 using the resources secured for Business

Plan 1 as the ‘launch-pad’.

Extract from the 2008 Annual Report

The above extract from the 2008 Annual Report for the RDP programme in Swansea provides an

excellent summary of Business Plan 1 which was intentionally conservative in its approach and

designed to act as a “launch pad” for a more ambitious Business Plan 2. Assessing the effectiveness

of this approach has been an important part of the evaluation.

• “The projects can be seen as lacking ambition and with little impact to the bigger picture” –

Partnership / LAG member

• “Maybe [we should have] started bigger... I am an optimist by nature and we may have struggled

with it being bigger but I think that is what I would have done.” – Partnership member

As the above quotes suggests, the approach taken with Business Plan 1 could be interpreted as a

lack of ambition. And, in hindsight, some Partnership members believe that more could have been

achieved during Business Plan 1. However, other stakeholders focused on the benefits of the

conservative approach:

• “The fact that it was small scale allowed us to build it up slowly.” – Partnership Member

• “We were very modest in our approach... we considered how we were going to deliver and we

made sure we could deliver and these were the underlining principles we all agreed to work to

from the off.” – Partnership Member

Page 26: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 23

3.2. Management structures

As introduced in Section 1, two local governing entities are required for the delivery of the RDP: a

Partnership and a Local Action Group (LAG). The Local Authority was also required to take on the role

of Lead Body which effectively means that they are ultimately responsible for the administration of

the programme and are the main link with the Welsh Assembly Government. Much of the discussion

with stakeholders at each phase of the evaluation focused on how effectively these three roles had

been undertaken during Business Plan 1.

3.2.1. The Partnership - the Swansea Rural Development Partnership

The new Partnership was set up in Autumn 2006 as a response to opportunities coming

forward as part of the new Rural Development Plan 2007-2013. For the first time, eight rural

wards within the City and County of Swansea qualify as rural areas for assistance to support

their economic regeneration activity.

Extract from Business Plan 1

It was clear from discussions during the baseline and interim phases of the evaluation that the

Partnership took some time, as one interviewee put it, “finding its feet” and that, as a body, it has

found the process of developing the LDS, the business plan and its projects challenging. Again, due

to the fact that the Partnership is ‘new’, this may not be unexpected. Nevertheless, it is an important

issue and ‘lesson learnt’.

• “It was difficult for some members of the Partnership to grasp what we were trying to achieve” –

Partnership member

• “Very few people on the Partnership understood the process” – Partnership member discussing

the development of the LDS and Business Plan 1

• “It [the Partnership] has worked well, was worried in the early days but always difficult when a

new organisation starts but it is getting better and better and the last 12months it has worked

particularly well and we get good feedback from the projects.” – Partnership Member

• “As Business Plan 1 progresses it [the Partnership] matured well, there was a gap in terms of

community involvement but this has improved and community attendance has been good.

Everyone is enthusiastic and people attended even if they knew they were not getting funding for

themselves - showed good commitment.” – Partnership Member

• “It took a long time to learn the processes, to learn the structure and learn what to expect and

what not to.” – Partnership Members

Comments made by stakeholders during discussions with the evaluation team suggest that the

Partnership have in many ways been reliant on a few key members and on the support that is

provided by the City and County of Swansea as the Lead Body and, in the case of Rural Swansea

Action, the staff who deliver the projects (see below). Such a situation is not unique to Swansea; a

number of far more mature partnerships are still reliant on a number of key individuals. However, it

is something that needs to be taken into account as the Partnership, hopefully, develops.

Monitoring the membership of the Partnership and attendance at meetings was identified in the

baseline report as one of the indicators that would be used by the evaluation. The Partnership is split

into four ‘quarters’ and includes representatives from the public, private, voluntary and community

sectors.

Page 27: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 24

Table 8: Attendance levels within the four quarters at the interim and final evaluation stages.

Interim evaluation (2010) Final evaluation (2011)

Public sector – 47%

Private sector – 41%

Voluntary sector – 45%

Community sector – 76%

Public sector – 42%

Private sector – 39%

Voluntary sector – 42%

Community sector – 75%

Source: project monitoring data provided by the Lead Body

Attendance records have been kept since the Partnership’s first meeting in November 2006 and

show that, on average, members from the community sector have attended a far higher percentage

of meetings than their counterparts from the other sectors (Table 8). This shows the commitment

from those who represent the community sector to the RDP programme in Swansea and supports

the comments made by stakeholders about the effectiveness with which the programme had been

able to engage with that sector.

The attendance figures underline the fact that the Partnership has been reliant on a number of key

individuals; only 7 members of the Partnership have been to 75% or more of meetings since the

Partnership’s establishment. The average number of attendees at Partnership meetings has been 8

out of the 20 members. This is something that needs to be taken into account as the Partnership

(and the LAG) develops over the coming months and years especially given the increase in

responsibility and workload that is inevitable with the implementation of Business Plan 2 (please

refer to Section 3.4).

It is also interesting to note that, whilst the change is not substantial, as a percentage the

attendance levels within each of the ‘quarters’ is lower at the final evaluation stage (March 2011)

than it was at the interim phase (March 2010) suggesting a downward trend in attendance. This is

something that the Partnership should monitor as any such trend will inevitably increase the

pressure on those who do attend on a regular basis.

• “I think for some people there was a lack of commitment - some people joined the partnership for

personal reasons or for a specific project interest and when they realised it was not happening

necessarily in their favour then they left. We struggled to engage some local counsellors and at

times it felt like there was a general lack of engagement from them and from the local mayor.” -

Partnership Member

The above quote is interesting as it highlights the changing role of the Partnership as the RDP

programme progresses and how this can potentially change how members perceive their role. The

role of a Partnership such as the Swansea Rural Development Partnership changes as a programme

develops. In the first instance, the Partnership is developing a strategy and/or projects to deliver

that strategy (in this case, the LDS and Business Plan 1). Its role will however subsequently change

into one of monitoring the delivery of projects. As the role of the Partnership changes, members’

views on what contribution they can make – and therefore their level of engagement/attendance -

may change. As the quote above suggest, those who are members of the Partnership due to a

perception that they can influence the projects supported, may become disengaged and stop

attending meetings

Page 28: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 25

The situation during Business Plan 1 of the RDP has been slightly less clear cut as Business Plan 2 was

being developed alongside the delivery of Business Plan 2. As such, the Partnership had a duel role:

monitoring and development. However, the change in role is likely to be more obvious during

Business Plan 2 and the Partnership should take it into account when reviewing its membership.

3.2.2. The Local Action Group – Rural Swansea Action

Again, Rural Swansea Action was a new body set up, from scratch, to deliver the RDP in the County.

As such, its members have little if any prior experience of undertaking a similar role in the past and

the feedback from its members has been that it took time to establish itself.

• “[The LAG is] very good... we are now getting more private sector involvement and that has given

it a boost.” – LAG member

• “Rural Swansea Action is more active [then the Partnership] a lots of meetings and good

discussions and a lot of feedback.” – LAG member

• “Not quite so smooth [as the Partnership]... We lost [the Chair] and he had a lot of knowledge

and then another chair was appointed and we have also lost him now due to ill health. The LAG

still meet regularly but commitment does not seem as strong as it was. Having said that we had

the first AGM last July and that received a good attendance and there was also some new

members from the private sector which is encouraging.” – LAG member

As the above quotes demonstrate, the feedback from those LAG members interviewed was mixed

although the general consensus was that the LAG was making positive progress as an entity,

especially in terms of its engagement with the private sector.

At the baseline phase, interviewees reported that it had proven to be very difficult to fill the key

roles of the LAG; those who had volunteered to be part of the LAG were not keen to take an active

role in its management. As with the Partnership, this has led to a situation where much of the

‘burden’ of administrating the LAG has fallen to the Chair, the Chair of the Partnership (who is also a

member of the LAG) and the office leading upon the delivery of the function of the Lead Body on

behalf of the Local Authority (both employees of the City and County of Swansea). It is also apparent

from the interviews that the lack of continuity in the post of chair of the LAG (due to circumstances

beyond the control of the LAG) has been a cause for concern. This is an indication of the importance

of that role.

3.2.3. The Lead Body – the City and County of Swansea

The Lead Body has a critical role in the delivery of the RDP including the fact that it is the Lead Body

that is ultimately responsible to the Welsh Assembly Government for the delivery and monitoring of

RDP funded activities.

• “We would have struggled without them [the Local Authority in their capacity as Lead Body]. We

have a chair and secretary provided by them and when we need 'professional' commitment their

help as enabled us to concentrate on what we are doing instead of worrying about paperwork

and the financial stuff as they deal with this and they are better at it - they are brilliant.” –

Partnership member

• “The European Officer has been superb to deal with, diligent and enthusiastic.” – Partnership

member

Page 29: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 26

• “Those leading the business plan for the Council have done a fantastic job, linking with different

sectors. It was a complex operation to get the funding, it's been a complex process. The lead

officers have been very supportive.” – Partnership member

As the quotes above suggest, those Partnership members interviewed were very positive when

discussing the way in which the City and Council of Swansea were undertaking their role as Lead

Body and the key role they have played in terms of developing the LDS and securing RDP funding for

the area.

The commitment of the Local Authority to the delivery of the RDP in the area is shown by its

willingness to allow one of its employees to take on the role of Chair of the Partnership (including a

number of active roles in terms of the delivery of the projects: e.g. the joint line management of

Rural Action Swansea staff) and the amount of time members of its staff has been able to commit to

undertaking the role of the Lead Body. Given the relatively inexperienced nature of the Partnership

and the LAG – especially at the start of Business Plan 1 - as evaluators, we are of the view that the

role of the Lead Body has been the critical factor in the delivery of the RDP in Swansea to-date. The

programme would not have been ongoing in the area without the commitment to the programme

shown by the Local Authority.

3.3. Delivering the projects

Feedback from stakeholders on the delivery of the Business Plan 1 projects was generally favourable.

When asked what they would change given the opportunity to start again, tweaks were suggested

rather than fundamental changes. The following were the main issues discussed when stakeholders

were asked to consider the strengths and weaknesses of the projects delivered as part of Business

Plan 1.

3.3.1. The procurement process

Organisations / contractors to deliver the Axis 3 funded projects within Business Plan 1 were

procured via Sell2Wales as per advice given to the Lead Body by the Welsh Assembly Government.

Those members of the Partnership who had been involved with that processe emphasised the

difficulties it had created; one went as far as to describe it as being “very painful”.

Interviewees reported that there had been a significant amount of interest in the contracts from

local businesses and organisations when the invitations to tender were initially posted on

Sell2Wales. However, that interest did not always translate to tenders being submitted and, in some

cases, the tenders received were either for only part of the contract, or too far from the requested

specification2.

Reflecting upon the situation, Partnership members identified a number of issues that restricted the

ability of organisations other than the Local Authority to tender. The contracts were for the delivery

of the projects as a whole. As such, the delivery organisation needed to have a very broad range of

skills and experience. Critically, the delivery organisation was also required to provide match funding

for the project; something which the private and voluntary sector may find difficult to do.

2 The Rural Youth Project is being delivered by the Swansea Youth Service, the Improvements to Walking facilities in the

Mawr Community project is being undertaken by the City and County of Swansea Countryside Access team and the Rural

Swansea Walking and Cycling Marketing Campaign project is being undertaken by the City and County of Swansea’s

Tourism team.

Page 30: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 27

The procurement process led to a situation where some key partners, who had been closely involved

in the development of projects, then tendered for the contract to deliver those projects. It is

however unclear how such a situation could have been avoided given the need to include key local

partners on the Partnership.

Has the Local Authority been too involved? One interviewee thought that they possibly had done so.

However, the question then is who else could deliver the project? Was there (at the time the

projects were being procured) capacity outside the Local Authority (e.g. within the voluntary sector)

to deliver the projects?

Some of the Partnership members interviewed would clearly have preferred to have a broader range

of local organisations involved in the delivery of Business Plan 1 projects. However, it is unclear what

steps could have been taken to facilitate this. One potential alternative discussed during the

interviews was to break the projects into smaller ‘lots’ which a broader range of organisations could

then tender to deliver. However, such an approach would clearly significantly increase the

administrative burden associated with delivering the Axis 3 funded projects; something which the

Partnership has been keen to avoid during Business Plan 1 for reasons previously discussed. It is also

important to consider the advantages of delivering projects via the Local Authority, especially their

experience as a deliverer of projects, at a time when one of the main objectives was to demonstrate

the potential to deliver ‘rural’ projects within the County for the first time.

3.3.2. Bureaucracy

• “Some of the projects were slow starting. Getting bogged down in paperwork and the

organisation side of things, it can be a bit clumsy.” – LAG member

Bureaucracy was a common theme during discussions with stakeholders regarding the delivery of

projects and it is an issue that has clearly slowed progress in terms of delivering the projects. A lot of

time of the officer leading the Lead Body functions within the Local Authority has for example,

clearly been spent resolving issues caused by the bureaucracy within the system. Our experience of

evaluating European projects suggests that, to some extent, bureaucracy is inevitable within an

European funded project. However, the feedback from stakeholders who also have extensive

involvement with other European programmes and projects was that, in the case of the RDP, it was

“far worse”. The following in particular was highlighted:

• A slow response from the Welsh Assembly Government to queries regarding eligibility - “[I was]

concerned about the length of time it took to get things going... there was no spend in the early

stages... we were waiting to get started, waiting for decisions at beyond Council level.”

• A lack of flexibility within projects and specifically the need to gain Welsh Assembly Government

approval for even what were considered to be minor changes to projects – “You cannot foresee

things that go wrong but equally you cannot foresee things that go right...”

• Changes to the performance indicators that were being made to monitor the outputs and results

of projects introduced at the mid-point of Business Plan 1

Page 31: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 28

One project manager also highlighted delays caused by problems with the bureaucracy when

recruiting staff although this is not necessarily related to the RDP programme.

• “Recruitment of staff was a nightmare because it took so long to get everything signed off within

the LA and get advert placed etc and this resulted in us not getting enough staff to be able to do

everything. The actual grass roots stuff that we did on the project was great but this recruitment

did prove to be a nightmare.”

Whilst it is easy to recommend that bureaucracy should be kept to a minimum - which it obviously

always should be – the important lesson learnt is to be aware of the delays that can be caused and

to work and to plan for such eventualities.

3.3.3. Engaging local community groups, partners and individuals

Much of the discussion with interviewees at both the interim and final stages of the evaluation

focused on the benefits generated by the ‘inclusive’ or ‘bottom-up’ approach being used to deliver

the projects – this was highlighted as one of the main strengths of the projects delivered as part of

Business Plan 1.

• “The fact that so many people from across the community are starting to get involved (is a

strength of the way in which projects have been delivered)” – Project Manager

At each stage of the evaluation, interviewees have reported that the RDP projects had worked well

and extensively with community groups and other initiatives in the rural Swansea area. Consultation

meetings had been arranged across the rural parts of the County. As a result, interviewees were of

the view that groups that had in the past been “hard to reach” in communities such as Mawr,

Pontarddulais and Penclawdd had engaged with the RDP programme. A new “network” of local

organisations was beginning which stakeholders believed represented a significant step forward for

the rural parts of the County.

3.3.4. A multi-sector approach

It was widely regarded that Business Plan 1 projects has made a valuable contribution to the

development of a ‘multi-sectoral approach’ in rural parts of Swansea; the relationship between the

public, private and voluntary sector has improved greatly through working together on the

formation and delivery of the RDP programme. Again, this was considered to be a significant step

forward.

• “People and organisations are now working together, talking to each other and looking at what

else they can do together in the future” – Partnership / LAG member

For example, according to interviewees, the input of partners from across the sectors had been

critical to the success of the consultation events held as part of the Axis 4 projects. Ensuring the

ongoing support of local people and organisations was also seen as critical to the continued

development of the RDP programme in the coming years; an issue that needs to be taken into

account during the implementation of Business Plan 2 projects.

• “There is less money about for everybody to do things, working together seems like the only

logical and effective thing to do” – Project Manager

Page 32: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 29

3.3.5. Project timescales and a slow start

The short time in which Business Plan 1 projects needed to be delivered was an underlying issue in

all discussions with stakeholders at all phases of the evaluation. As noted in the introduction, the

RDP programme in Wales has been split into two periods; Business Plan 1 (which covers the period

June 2008 to the end of February 2011) and Business Plan 2 will be implemented from the 1st

March

2011 to the end of February 2013.

• “(One of the weaknesses of the project to date is that) people (are) playing catch up all the time.”

– Project Manager

Due to delays in the start of the programme and subsequent delays caused by a complex

procurement process, the period for the delivery of Business Plan 1 was squeezed into little more

than two years. As discussed in the previous section, it is also apparent that much of project

activities’ took place during the final year.

This is clearly not a significant time in which to deliver a project. However, it is especially short in the

case of Swansea’s activities given that this was the first time that a coordinated programme of

support has been available in the rural parts of the County (an issue discussed at length in both the

LDS and Business Plan 1); much of the structure that has been put in place to deliver the RDP

programme is new. At the interim phase of the evaluation, progress had clearly been made in terms

of the development of new relationships and the structures that were needed to deliver a

coordinated programme of support in rural areas. However, progress on the delivery of the projects

themselves (and achieving the targets for outputs and results) had clearly been slower than

anticipated.

In Wavehill’s experience, a slow start is not uncommon amongst programmes or projects, especially

those which are European funded. Projects also need time to recruit staff, set-up management

systems and so on. The slow start of Business Plan 1 projects therefore is not surprising. However, a

difference in this case is that the timescale for the delivery of the projects was already short; there

was therefore little time to catch up. The review of project outputs and results in the previous

section and in Appendix 1 shows that, in almost all cases, projects have achieved their targets; they

did therefore ‘catch-up’ as the project managers were confident that they would do when

interviewed for the interim phase of the evaluation. The slow-start made by projects is however

another important ‘lesson learnt’ which should be taken into account by projects as they move into

the delivery of what is, as discussed later, a far more ambitious Business Plan 2.

Further to the discussion in the previous section of the report about the impact of the projects – in

particular the Improvements to the Walking Facilities in Mawr project – we would however again

recommend that even though projects may be considered completed in that they have achieved

their ‘output’ targets, they should not be considered completed by the Partnership. There should

continue to be an emphasis on maximising the impact of projects by, for example, helping local

businesses maximise the impact of improvements to the walking facilities in Mawr on their trade.

Page 33: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 30

3.3.6. A lack of focus on monitoring

As discussed in Section 2, the evaluation has found that monitoring was not embedded within all of

Business Plan 1 project. The data available was strongest within the Axis 4 projects that were able to

provide the evaluation team with a number of contact details for participants and beneficiaries as

well as a range of other supporting information. Other projects were however unable to provide any

significant monitoring information (please refer to Section 2 and Appendix 1).

One potential reason for this offered during discussions on the matter was that, due to the short

timescales involved, staff were focused in delivering their project rather than on monitoring

activities. Whilst we understand this argument, we would argue that effective monitoring should be

the cornerstone of any project. And, on this basis, we would recommend that the Partnership

ensure that there is a further emphasis on monitoring amongst those projects implemented as part

of Business Plan 2. As previously noted, we would also recommend that the monitoring of Business

Plan 1 projects continue into the Business Plan 2 period where it is necessary to do so in order to

measure the impact of the support provided.

3.4. Looking forward – Business Plan 2

A different approach was taken for the development of projects for inclusion in Business Plan 2.

Rather than developing projects within the Partnership and then producing their delivery (as was the

case for Business Plan 1) the Partnership invited interested parties to submit their ideas for projects

for inclusion in Business Plan 2 via an ‘open call’ via the City and County of Swansea website.

Between March and November 2009, 69 ideas were received and checked by the Partnership for fit

with the Rural Strategy. The majority of ideas were endorsed and included in a series of ‘umbrella’

‘expressions of interest’ to Welsh Assembly Government. Four of these were invited to full

application stage in July 2010, and are entering the final stages of the appraisal process. Formal offer

letters were received from the Assembly Government in March 2011, with a further project ‘on

appeal’ due for submission in April 2011.

Feedback from Partnership members on the process of developing the new set of projects was

largely positive. In particular, they welcomed the fact that the process had generated a wide range

of ideas from a wide range of organisations; 33 different organisations submitted project ideas.

However, there was some criticism of the bureaucracy involved and the “long-winded” nature of the

process; the closing date for initial ideas was November 2009 and formal approval was still being

awaited – 15 months later. Such criticism however needs to be considered in light of the fact that

the timetable for approving projects was advertised as such; everybody was aware that the projects

would not start until 2011.

Included within the plan are projects to continue/follow each of the projects delivered as part of

Business Plan 1 which provides the opportunity for further work to measure the impact of activities

to date. It also maintains the continuity from Business Plan 1. The most noticeable thing about the

projects to be delivered during the Business Plan 2 period is however their value (assuming formal

approval is received); in excess of £5million in total. This is a significant increase from the total value

of the projects delivered as part of Business Plan 1 (£534,261) and shows a departure from the

previous conservative approach. This represents a significant challenge for the Partnership. Can they

maintain and build upon the momentum that has developed during Business Plan 1? Careful

consideration will also need to be given to addressing some of the issues identified by this

evaluation in advance of and during the delivery of the much larger Business Plan 2.

Page 34: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 31

4. Conclusions and recommendations This final section will set out the conclusions of the evaluation and present a number of

recommendations based on the findings of the evaluation.

4.1. Conclusions

This evaluation has been positive in many respects. Our main conclusion is that Business Plan 1 can

be considered a success in that it has ‘established’ the Rural Development Plan in the County for the

first time. A Rural Partnership and a LAG now exist and their capacity has clearly developed during

the Business Plan 1 period. However, it is also clear that Partnership and LAG members have been on

a ‘steep learning curve’ and that it is yet to become the ‘finished article’. The focus on developing

the Partnership and the LAG therefore needs to be maintained.

The Partnership faces a number of challenges as it gears up for the delivery of the significantly larger

Business Plan 2. In particular, the Partnership is reliant on a few members who attend meetings on a

regular basis and the pressure on those few members is likely to increase significantly as the

workload increases with the introduction of the much larger Business Plan 2.

Each of the projects delivered can also be considered a success in that they have been established –

especially the Axis 4 projects which are providing their particular type of support in the County for

the first time – and have (if the forecasts are accurate) achieved the output and result targets that

were set. Further, there are no obvious problems with the way in which they have been delivered

with stakeholders highlighting the ‘inclusive’ and ‘bottom-up’ approach being used as a particular

strength of the approach. The limited data that was available and which was gathered by the

evaluation also suggests that projects have had or will have a positive impact. However, the general

lack of data on the impact of the projects needs to be addressed, especially as the value of the

activities being delivered in the County under the RDP increases from £340,000 to in excess of £5m.

We have also concluded that the conservative approach employed for Business Plan 1 was the

correct one as it allowed the Partnership, the LAG and the Lead Body the breathing space to develop

and become accustomed to the RDP and all it entails. The role of the Lead Body has also been the

critical during this period; the programme would not have been ongoing in the area without the

commitment to the programme shown by the Local Authority.

4.2. Recommendations

1. Options for increasing the membership of the Partnership should be explored. More

importantly, options for a more even distribution of roles and responsibilities amongst

Partnership members should be explored.

2. The potential to further develop the ‘capacity’ of Partnership / LAG members via appropriate

training should be explored. This should include training to increase members’ knowledge and

understanding of the RDP programme and of their role as Partnership / LAG members.

Page 35: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 32

3. The Partnership and the LAG should explore options for increasing the focus on monitoring and

evaluating the impact of projects delivered as part of Business Plan 2. Process should also be put

in place to monitor the ongoing impact of Business Plan 1 projects over the coming months and

years.

4. Whilst recognising that the delivery of any European funded project comes with a certain

amount of bureaucracy, the Lead Body should work with the Welsh Assembly Government to

ensure that the burden upon the schemes being supported is as little as possible.

5. Whilst it may not be possible to take action relating to the findings on the impact of the short

timescales for projects and the consequence of splitting the RDP programme into two periods on

a local level, they are important lessons learnt. We would therefore recommend that they are

fed back to the Assembly Government with a view to informing the development of any future

programme(s).

Page 36: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 33

Appendix 1: Project level inputs,

outputs, results and impact

Following the ‘impact model’ illustrated by the graphic below, this section will consider the input,

outputs, results and impacts that have been generated by each Business Plan 1 project.

Page 37: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 34

Rural Swansea Walking and Cycling Marketing Campaign

Inputs

A marketing campaign to highlight the walking and cycling product offer within Rural Swansea. There

is currently no provision of information, which highlights the rural and coastal walking and cycling

offer within Swansea Bay. This campaign will help to promote a very important niche activity as

walking and increasingly cycling, are popular reasons why people choose and then visit the

destination.

Total project expenditure: £43,749

Outputs

Performance indicators

Target

2008-

2011

Actual

achieve.

to date

%

achieved

Forecast

to

28 Feb

2011

Number of projects for the development and/or

marketing of rural tourism services financially

supported

1 1 100% 1

Number of actions that actively encourages or

promotes action to address environmental

sustainability

1 1 100% 1

Results

Performance indicators

Target

2008-

2011

Actual

achieve.

to date

%

achieved

Forecast

to

28 Feb

2011

Number of new or improved products, processes or

services launched 1 1 100% 1

Design one new walking and cycling guide to Rural

Swansea 1 1 100% 1

Print 200k walking and cycling guides 200,000 200,000 100% 200,000

Distribute 200k walking and cycling guides 200,000 136,500 68% 200,000

Commission Rural Swansea photography 1 1 100% 1

Cost per leaflet: £0.22 (assuming the distribution of 200,000 leaflets)

Impact

The main impact that the officer interviewed hoped the project would have was to increase the

number of visitors to the area / users of the local walking and cycling resources. However, the

evaluation found no actual evidence of the impact of the project to date. We would recommend that

the Partnership reviews the distribution of the leaflets and then seeks to contact those who have

received them (businesses, tourist information points, etc.) for their feedback on the impact of the

leaflet.

Page 38: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 35

Evaluator’s comments

The Partnership should also explore options that will allow the impact of any future editions of the

leaflet to be tracked; for example, by including a coupon or entry into a prize draw as part of the

leaflet. Better information on the distribution of the leaflets should also be maintained. How many

have gone where and why?

Page 39: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 36

Improvement of Walking Facilities in Mawr Community

Input

To complement the Walking and Cycling Marketing campaign, this project involves the creation and

promotion of a circular walk in Cwm Cerdinen and a linear route along Cwm Clydach, both in the

Mawr Community. The route at Cwm Cerdinen would be circular into and around the valley and

across common land, with links to Pontarddulais and the Upper Lliw Reservoir. The walk would rely

on the reopening of some footpaths. At Cwm Clydach, there is potential for a linear walk up the

valley following existing public footpaths through the RSPB reserve, from Clydach to the common

land at the Neath Port Talbot border/Penlle’r Castell. This path is mostly already in place on the

ground, but would need enhancing and possibly two legal diversions. The routes will be promoted

via leaflets devised in conjunction with the community and distributed to residents and outlets

visited by tourists.

Total project expenditure: £28,009

Outputs

Performance indicators

Target

2008-

2011

Actual

achieve.

to date

%

achieved

Forecast

to

28 Feb

2011

Number of small scale tourism infrastructure projects

financially supported 2 2 100% 2

Number of actions that actively encourages or

promotes action to address environmental

sustainability (Level 2)

1 1 100% 1

Total volume of investment (division according to)

type of action 25,700 28,009 108% 28,009

Results

Performance indicators

Target

2008-

2011

Actual

achieve.

to date

%

achieved

Forecast

to

28 Feb

2011

Gross number of additional visitors to a tourism

project that has been supported 1,000 0 0 1,000

Additional number of tourist visits (division according

to) 0 0 0 0

-- number of overnight stays 0 0 0 0

-- number of days visitors 1,000 0 0 1,000

Cost per additional day visitor: £28.00

Page 40: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 37

Impact

The impact that the project officer anticipated were focused on the walks potential to attract

additional visitors (locals and tourists) to the area and the benefit that would bring to local

businesses. As the work was only just being completed, there was however little evidence of any

impact to date. Counters have however been placed on the site to record the number of users and

used as a basis for counting the number of day visitors.

Interviews were undertaken with 9 stakeholders in the project: 5 members of the walking club and 4

local businesses. Four of the 5 members of the walking club anticipated that the project would

benefit the local area by increasing the number of people visiting the area to walk but also by

maintaining the area. Four of the 5 felt that the development would also lead to more people from

the local area walking. None of the businesses interviewed anticipated that they would benefit

significantly from the development; only 2 felt there would be any benefit to them. However they

believed that there would be some benefit to the area generally if more walkers were attracted but

the development needed to be effectively promoted to the public.

Evaluator’s comments

We would recommend that interviews are undertaken with local businesses during the coming

months and years in order to establish the impact of the project on their trade. The potential to

involve local businesses in the project at an earlier stage in order to get their buy-in should also be

explored. The potential to undertake research to establish the economic impact of visitors to the

walking facilities should also be explored.

Page 41: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 38

The Rural Youth Project

Input

To develop and provide a mobile youth project to work within the rural communities and wards to

the West and North of Swansea City and County Council, prioritising initially Pontardulais, and then

the Greater Gower area and wards. The project will work to reduce the social exclusion of young

people in rural communities by providing trained youth work staff to support young people

individually and within small groups, and by researching with young people in order to:

• provide opportunities for young people to improve their personal, social and educational skills;

• develop opportunities for them to participate in decision making groups such as youth forums,

and to become ‘active citizens’ including volunteering;

• create a range of positive opportunities, with and for young people, in sport, arts, music etc, as

well as more employment related skills, and dealing with issues that impact on young people in

Swansea such as health, fitness and substance misuse.

Total project expenditure: £176,500

Outputs

Performance indicators

Target

2008-

2011

Actual

achieve.

to date

%

achieved

Forecast

to

28 Feb

2011

Number of projects relating to basic services for the

rural economy and the rural population financially

supported

1 1 100% 1

Number of children (0-16) participating in the project 500 489 97.8% 500

Number of young people (16-20) participating in the

project 250 203 81.2% 300

Number of actions that actively encourages or

promotes action to address environmental

sustainability (Level 2)

1 2 200% 1

Cost per beneficiary: £235.33

Results

Performance indicators

Target

2008-

2011

Actual

achieve.

to date

%

achieved

Forecast

to

28 Feb

2011

Population in rural areas benefiting from improved

services provided by supported projects* 750 692 92% 300

New or improved provision for young people 2 2 100% 2

Increase in recorded outcomes with 11-19 year olds 450 148 33% 450

Increase in accredited outcomes with 11-19 year olds 90 64 71% 90

Youth forums/youth action groups established 9 7 77% 9

*Note that the total young people target of 750 has been split between 0-16 and 16-20 year old categories

following clarification of the definitions in the RDP Performance Framework.

Page 42: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 39

Impact

The project manager identified a wide range of impact on the children and young people supported

by the project, including:

• Created a service for young people not previously there

o Swansea Rural Youth Enterprise Truck

o New groups / sessions

o Supported access to provision

• Raise profile of Welsh language

o Encourage young people to speak welsh

o Talk about Welsh Culture

• Bringing communities together

o New friendships

o Help young people value themselves and others

o Encourage respect for the community and its people

• Building the capacity of young people

o Volunteering opportunities

o OCN (open college network) opportunities

o “In the long term we are giving people skills, knowledge and ability to develop

themselves.”

The project was however not able to provide the evaluation with any evidence of the impact on

participants other than the perception of the project manager.

The evaluation team sought to undertake interviews with the young people supported but only two

contacts were provided by the project within the window available to undertake the interviews. The

participants who were interviewed made the following comments when asked how, of at all, they

had benefited:

• Since being involved I've done my Level 3 in Youth Work, I've started a degree in Youth and

Community Work in Gorseinon and have been volunteering at the project for my placement. I

wouldn't have been able to get the £10,000 grant without their support and I've got tea and

coffee with the mayor of Swansea next week because of my involvement and been in the Evening

Post a few times. I've developed as a person ten-fold, I wouldn't be the person I am now without

the project. Some communities we go to speak Welsh so I speak more Welsh when we're there. I

started Swansea's Young People's Forum in 2009, I'm not involved now though as I haven't the

time. It's helped relationships with older people massively as we've had loads of help from

various people and done intergenerational work and I can adapt more to the environment now

as I've learnt a lot about politics, attended meetings and I'm meeting the mayor next week so I

definitely feel I have a voice. Before the project came into my area there were no facilities at all

for young people now the project is getting mentioned in other places and hopefully it can

happen in these areas too.

Page 43: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 40

• Massively. There's so much more to the Youth Service that I didn't know about and I know a lot

more about it now. It's brought a whole new area to me and I've realised how big it is, I've met

new people and established relationships with other people I've got to know. The help that you

get from the other workers has really increased my knowledge. It's increased my confidence not

only as a youth worker, it's something different to do, something different all the time. I go up to

Penclawdd where there's always community projects going on we've helped with flower beds

where you're working with people of all ages. When the bus comes round, it parks and we use

the environment as well as the bus for the sessions. I definitely feel I have a voice as I said I've got

to know more of the Youth Service and they come back to you asking what you want to do which

makes you feel valued.

Interviews were also undertaken with 6 stakeholders who had participated in the project in different

ways: two police officers, a community development officer, two volunteers and a representative of

a local centre. They all identified a positive impact on the children and young people being

supported. In particular, stakeholders identified the following benefits:

• Help for young people to develop their personal, social and educational skills particularly by

increasing their confidence;

• Encourage more young people to become ‘active citizens’;

• Helped young people to deal with issues that impact on young people in Swansea such as health,

fitness and substance misuse in particular via ‘informal learning’ (e.g. chats with youth workers);

and

• Helped young people appreciate their local environment more by reducing anti-social behaviour,

developing a sense of place, etc.

Evaluator’s comments

Whilst the comments discussed above are very positive, the evidence that is available to

demonstrate the impact of the Rural Youth project is primarily anecdotal. The Partnership should

therefore explore options for collecting additional evidence from the young people being supported.

We are aware that this is something currently being considered by the team within the Local

Authority responsible for the project. If possible, a system should be in place in advance of the

implementation of the Rural Youth 2 project to be delivered during the Business Plan 2 phase. As a

minimum, the project should be required to be able to provide the evaluators with the basic

information needed to undertake a survey with participants in the project.

Page 44: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 41

Local Food for Local Markets

Input

This project aims to undertake work to analyse the capacity and needs of local agriculture and food

sectors and engage the wider community in economic activity using the LEADER Approach. Local

Action Group members and staff, working closely with local volunteers who gather information,

relating to the capacity and needs of the sectors, to support the diversification of the rural economy,

enhance prosperity at a local level and develop the area’s identity as a high quality food producing

area able to supply local markets.

Total project expenditure: £102,671

Outputs

Original targets

Performance indicators

Target

2008-

2011

Actual

achieve.

to date

%

Achieved

Forecast

to

28 Feb

2011

Number of consultation events undertaken in the

development of a project 28 20 71% 28

Number of participants in consultation actions

undertaken in the development of a project 310 69 22% 310

Number of information dissemination actions

undertaken 40 2 5% 40

Number of new and innovative approaches to supply

chain integration / cooperation initiatives developed 10 3 30% 10

Number of new products, processes and technologies

developed (agri-food) 16 3 18% 16

Number of cooperation initiatives advised or assisted

in the agri-food sector 10 6 60% 10

Number of children participating in the project 80 27 33% 80

Number of young people participating in the project 80 5 6% 80

Number of women participating in the project 400 30 7% 400

Number of individuals from black and minority ethnic

groups participating in the project 40 0 0% 40

Number of Welsh speakers participating in the project 150 8 5% 150

Number of individuals from ‘under-employed groups’

participating in the project 50 3 5% 50

Number of actions undertaken to create awareness of

the environmental consequences of the project (Level

1)

110 10 9% 110

Number of actions that actively encourages or

promotes action to address environmental

sustainability (Level 2)

160 0 0% 160

Number of enterprises introducing changes that

enhances the environment or improves sustainability 50 0 0% 50

Page 45: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 42

Revised targets (1 April 2010 – 28 February 2011)

Performance indicators

Target

2010-

2011

Actual

achieve.

to date

%

Achieved

Forecast

to

28 Feb

2011

Number of enterprises advised or assisted of the

development of a cooperation initiative 5 3 60% 5

Number of individuals advised or assisted of the

development of a cooperation initiative 10 5 50% 10

Number of groups advised or assisted of the

development of a cooperation initiative 2 2 100% 2

Number of marketing & promotional activities

undertaken 100 109 109% 100

Number of consultation exercises undertaken 29 31 106% 29

Number of feasibility studies undertaken 5 3 60% 5

Number of children participating in project 80 43 53% 80

Number of young people participating in the project. 30 26 86% 30

Number of women participating in the project 100 104 104% 100

Number of individuals from black and minority ethnic

groups participating in the project 10 0 0% 10

Number of Welsh Speakers participating in the project. 50 26 52% 50

Number of elderly people participating in the project 80 94 117% 80

Number of individuals from under-employed groups

participating in the project 29 31 106% 29

Number of activities undertaken to create awareness

of the environmental consequences of the project 120 154 128% 120

Page 46: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 43

Results

Original targets

Performance indicators

Target

2008-

2011

Actual

achieve.

to date

%

Achieved

Forecast

to

28 Feb

2011

Increase in gross value added in supported

agricultural holdings £ 42,000 0 0% 42,000

Increase in gross value added in supported enterprises

(excludes agricultural and forestry holdings) £ 60,000 0 0% 60,000

Number of enterprises created 12 0 0% 12

Gross number of jobs created 18 0 0% 18

Gross number of jobs safeguarded 22 0 0% 22

Number of enterprise groups, cooperative and

producer groups or formal networks created 8 0 0% 8

Number of agricultural holdings introducing new

products and/or new techniques 12 1 8% 12

Number of enterprises introducing new products

and/or techniques (excludes agricultural and forestry

holdings)

10 0 0% 10

Number of new markets being supplied 30 7 23% 30

Number of agricultural holdings introducing new

products and/or new techniques 10 0 0% 10

Number of actions developed but now being

implemented outside the LDS (i.e. not funded by LDS

activity).

18 0 0% 18

Number of actions developed and being implemented

by other projects within the LDS (i.e. funded by LDS

activity).

2 1 0% 2

Revised targets

Performance indicators

Target

2010-

2011

Actual

achieve.

to date

%

Achieved

Forecast

to

28 Feb

2011

Number of communities engaged 8 8 100% 8

Number of groups engaged 3 5 166% 3

Number of individuals engaged 250 357 142% 250

Number of individuals/agencies consulted for benefit

of local producers 200 300 150% 200

Number of new or existing micro-enterprises advised

or assisted 20 13 65% 20

Number of new and innovative approaches to supply

chain integration/cooperation initiatives developed 4 3 75% 4

Number of cooperatives potentially created 3 2 66% 3

Number of enterprise groups, producer groups, cluster

groups or formal networks created 3 2 66% 3

Page 47: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 44

Impact

The project manager identified the following as the main impacts generated by the project:

• Promotion of businesses through events, website and social networks

• Promotion of local food to local business

o Taking part in food festivals

o Raise awareness of the value of local produce

• Supported local produce markets

• Helped businesses to diversify to attract tourism

o Providing information on grants and funding

Many of the above are in fact outputs of and results of the support provided rather than impacts.

What had been the consequence of the support provided to local businesses? In order to explore

this question, interviews were undertaken with 9 of the businesses engaged by the project. The

following is a brief analysis of those interviews.

Meeting expectations

Five of the businesses interviewed said that their expectations had been met by the support

provided, 1 of those saying that their expectations had in fact been exceeded. The expectations of

those respondents and how they were met is set out in the table below. The comments made give a

good indication of the impact that the support has had.

Expectations Why they were met

• We were starting up. We needed all the help

we could get

• In terms of being given advice to help market

the produce better. It appeared an

opportunity to making local food more

known to local people not just the tourist.

• Because I no longer had the shop I had to go

to events or find places to stock my

brownies. I was hoping to get more clients

and promote my produce.

• Networking with other producers, sharing

solutions to common problems, the

possibility of business to business sales and

the raising awareness of local producers.

• I thought I would be a little bit more

ambitious and find more outlets for my

produce

• It opened doors to access like minded people

outside this area, we have been put in touch

with people in Monmouth and Withybush

who are doing what we do.

• It is always difficult to quantify but we have

been put in touch with people at a couple of

events.

• Two businesses have ordered form me off the

back of some of the events, but still need

more clients.

• Because of the networking.

• It would have more than met my

expectations if my health had been better.

Page 48: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 45

Those whose expectations had not been met made the following comments:

Expectations Why they were not met

• Not sure, some ideas for the future possibly.

• Promotion of my products, anything.

• I had a good idea of what to do, I was hoping

to get official backing and arrange events. I

wanted them to help us get out and do this

because Swansea Council is not really

forthcoming.

• Letting local farmers know we are here.

• I think the project is more suited to the crafts

and small producers, we are more wholesale.

• There is no funding to invest in the business

at the moment, from what I gather is that

the last 2 years have been to get everything

in place for this project and maybe funding

will come through. It would be useful to get

funding for publicity.

• They are limited in what they can do. They

can advertise and publicise and spread the

word around but when we asked them if we

could use them to go through to the council

to put up banners and advertise they could

not. A lot more could be done, Tourism in

Swansea Bay when they have things on they

have lots of outside catering in when there

should be more local produce. Why aren't

local producers at the Air Show r the Boat

show, why not have an event in Singleton

Park

• It has not increased our business.

Business confidence

Eight of the 9 businesses interviewed were confident in the future of their business, the other being

unsure. When asked whether their confidence had changed due to the support they had received,

two said that it had:

• As a result of meetings with [the project manager] it has opened up new avenues for our

business, we were new to this and we have now been able to access other like businesses for

information.

• It has increased the possibility of increasing our business and our profile, made other people

more aware of our product and our services.

This is an indication of the potential for the project to benefit some businesses more significantly

than others.

Benefits to the businesses

Two of the 9 businesses had introduced new products or services as a result of the support provide.

• We have a stone barn here and [the project manager] discussed the potential opportunities what

we could do with it.

• We have more business to business sales and we have and the chance to show our products at

events.

Page 49: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 46

Four of the businesses said that they had been encouraged to work more with local businesses.

• We supply [company name] next door.

• This also comes with working the markets too... you form allegiances with like minded people

and share information.

• With local businesses and further afield.

• I do have a few more customers.

Other benefits identified by the businesses were:

• Yes it has given an awareness of our business and it has opened new doors.

• Yes, it's about raising awareness and from an encouragement point of view it has been great.

They have taken information of our business to stands at events. I know I can ring [the project

officer] any time and I see her at events, which is reassuring. The project is very responsive and

pro active. She sent us information on what we can do with our fleeces and although we have not

taken any action on that yet we may do so in the future.

• They have raised awareness and taken our products and logos to events and promote us.

• Yes, it has given us the useful opportunity, a window to portray our services and products at

events.

Attitude towards the area

Eight of the 9 businesses interviewed had positive views on their area as a location to run a business:

• It depends on what type of business [and] what you produce. We also have a holiday cottage so

we need to promote that as something different.

• There are areas of Swansea who are interested in niche produce.

• Just as good a place for business as anywhere else. Locals seem to be the worst for buying local

lamb.

• It is not the best place but there is nothing wrong with it.

• This is a fantastic location.

• Started up a co-operative, there is no local shop there and it is a very good outlet for us.

• More people are taking notice of free range and organic produce now.

• In the right month business is good; it is quite a seasonal business.

• We are very close to the M4 corridor and rural enough to source business.

Three businesses said that their views had changed (for the better) due to their involvement with

the project.

• That there is help and support for businesses out there.

• It feels as if there is support out there for businesses and it is trying to promote them.

• Previously I would not have though it was the best place to have a business because there is not

a massive amount of money here.

Page 50: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 47

Evaluator’s comments

In terms of outputs and results, the management data suggests that the project will achieve or

achieve against all of the performance indicators. There is however significant differences between

what has been achieved and what is anticipated will have been achieved by the end of the project,

most notably in terms of the result indicators. In particular, there are gaps in what the management

data says has already been achieved and what it is anticipated will be achieved for the following

indicators: increase in the gross value added in supported agricultural holdings, increase in gross

value added in supported enterprises (excludes agricultural and forestry holdings), the number of

enterprises created and the gross number of jobs created and safeguarded.

The interviews with a very small sample of businesses who have participated in the project suggest

that some businesses have benefited from the support provided. Indeed, some businesses would

seem to have benefited significantly. However, the interviews also show that all businesses do not

perceive that they have benefited.

We would recommend that the Partnership considers commissioning a larger (more representative)

sample of businesses that have benefited from the support provided as part of the second phase of

the project in order to provide better data on the impact the support is having.

Page 51: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 48

Local Arts, Crafts and Natural Products for Local Markets

Input

This project aims to undertake feasibility work to analyse the local Arts, Crafts and Natural Products

sector and local market opportunities, and to engage the wider community in economic activity

using the LEADER Approach. Local Action Group members and staff, working closely with local

volunteers will gather information to support the diversification of the rural economy, enhance

prosperity at a local level and develop the area’s identity for producing high quality local Arts, Crafts

and Natural Products able to supply local markets.

Outputs

Original targets

Performance indicators

Target

2008-

2011

Actual

achieve.

to date

%

achieved

Forecast

to

28 Feb

2011

Number of consultation events undertaken in the

development of a project 20 18 90% 20

Number of participants in consultation actions

undertaken in the development of a project 200 75 37% 200

Number of information dissemination actions

undertaken 14 2 14% 14

Number of participants in information dissemination

actions 140 0 0% 140

Number of new and existing micro enterprises

advised or assisted 40 1 2.5% 40

Individuals or groups advised or assisted to set up or

develop ideas for new micro enterprises (pre-start) 8 0 0% 8

Number of children participating in the project 40 27 67% 40

Number of young people participating in the project 40 3 7% 40

Number of women participating in the project 40 33 82% 40

Number of individuals from black and minority ethnic

groups participating in the project 34 0 0% 34

Number of Welsh speakers participating in the project 120 5 4.1% 120

Number of elderly people participating in the project 80 42 53% 80

Number of individuals from ‘under-employed groups’

participating in the project 40 3 8% 40

Number of actions undertaken to create awareness of

the environmental consequences of the project (Level

1)

60 10 17% 60

Number of actions that actively encourages or

promotes action to address environmental

sustainability (Level 2)

200 0 0% 200

Number of enterprises introducing changes that

enhances the environment or improves sustainability

(Level 3)

20 0 0% 20

Page 52: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 49

Revised targets

Performance indicators (Official)

Target

2008-

2011

Actual

achieve.

to date

%

Achieved

Forecast

to

28 Feb

2011

Number of existing micro enterprises advised or

assisted 13 11 84% 13

Number of individuals advised or assisted to support

the creation of a new micro enterprise (pre-start) 2 2 100% 2

Number of consultation exercises undertaken 30 35 116% 30

Number of marketing & promotional activities

undertaken 80 119 148% 80

Number of children participating in the project 60 43 71% 60

Number of young people participating in the project 20 28 140% 20

Number of women participating in the project 70 113 161% 70

Number of individuals from black and minority ethnic

groups participating in the project 7 0 0% 7

Number of Welsh speakers participating in the project 40 31 77.5% 40

Number of elderly people participating in the project 40 96 240% 40

Number of individuals from 'under-employed groups'

participating in the project 30 33 110% 30

Number of activities undertaken to create awareness

of the environmental consequences of the project 80 181 181% 80

Results

Original targets

Performance indicators

Target

2008-

2011

Actual

achieve.

to date

%

achieved

Forecast

to

28 Feb

2011

Number of actions developed and being implemented

by other projects within the LDS (i.e. funded by LDS

activity).

8 0 0% 8

Number of actions developed but now being

implemented outside the LDS (i.e. not funded by LDS

activity).

70 0 0% 70

Number of enterprises created 8 0 0% 8

Gross number of jobs created 8 0 0% 8

Gross number of jobs safeguarded 12 0 0% 12

Number of enterprise groups, cooperative and

producer groups or formal networks created 4 0 0% 4

Number of enterprises introducing new products

and/or techniques 12 0 0% 12

Number of new or improved products, processes or

services launched 11 1 9% 11

Number of new markets being supplied by supported

enterprises 40 1 2.5% 40

Number of individuals returning to work as a result of

support provided 9 0 0% 9

Page 53: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 50

Revised targets

Performance indicators

Target

2008-

2011

Actual

achieve.

to date

%

Achieved

Forecast

to

28 Feb

2011

Number of micro enterprises potentially created 2 2 100% 2

Number of jobs potentially created 8 tbc tbc 8

Number of jobs potentially safeguarded 6 tbc tbc 6

Number of communities engaged 8 8 100% 8

Number of groups engaged 9 7 77% 9

Number of individuals engaged 100 218 218% 100

Impacts

The project manager and staff identified the following as the main impacts generated by the project:

• Promotion of local businesses through events, website and social networks

• Promotion of local arts, crafts, etc. to local businesses

• Supporting local craft markets to aid local economy

o Go from hobbyist to full time occupation

• Establishment of “Groups of Interest”

o Wool and Sheep group – “It is hoped that the Wool and Sheep interest group will develop

and take advantage of the renewed interest in wool and the campaign that the Wool

sector has undertaken for the next 5 years.”

• Identified skills gaps and signposted to further develop

• “Help to keep rural communities alive.”

Again, a number of any of the above are outputs and results of the support provided rather than

impacts. Interviews were again undertaken with businesses to further explore the impact of the

support on them. Eight interviews were undertaken and the following is a brief analysis of the

findings.

Page 54: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 51

Meeting expectations

Four of the businesses interviewed said that their expectations had been met by the support

provided, 3 of those saying that their expectations had in fact been exceeded. The expectations of

those respondents and how they were met is set out in the table below. The comments made give a

good indication of the impact that the support has had.

Expectations Why they were met

• Hoping for networking, marketing and

finding events and venues

• I wasn't really sure, I didn't see how they

could influence the public.

• To network with people who are interested

in networking as businesses and help each

other in the community.

• To network with other people, more work.

• I have done some participation training

which was really useful, given me confidence

to speak out. I am now informed of all events

that would be suitable for my business and

have made many contacts.

• They have put me in touch with like minded

people and introduced me to a wool group. A

man from the wool marketing board came

and met me gave me a lot of information on

the background to wool industry and where I

might fit in. [The project officer] took cards

and leaflets to Withybush airfield and

introduced new customers.

• By networking and finding out about markets

and getting support from the project. I went

to 2 meetings, they were brilliant, fantastic.

• Work has increased; I have met many people

and learnt a lot.

The 4 who said that their expectations had not been met made the following comments:

Expectations Why they were not met

• Just to see if there was anywhere else to sell.

• A few more sales.

• It is not a business... it is my hobby that I am

passionate about. I was glad to meet other

people with a shared interest and go to craft

fairs with them. I thought if there was a guild

or a syndicate there was more chance of

spreading the word and a chance to show my

work to more people.

• I was hoping for two things really, to see

what was going on elsewhere and how they

were using their skills and giving business.

The second was general encouragement

because other people are in the same boat,

solutions shared and learning from other

people's successes.

• I think the niche is more for food producers.

• She has done all she can to help us network

and she informs us of events, but my sales

have not gone up.

• I have only been to one meeting and they

were a nice bunch of people and it triggers

you to say yes let’s get on with this. But it is

too soon to say if it will be a benefit to my

business.

• Progress is slow.

Business confidence

Page 55: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 52

Seven of the 8 businesses interviewed were confident in the future of their business (3 of those

being very confident), the other being unsure. When asked whether their confidence had changed

due to the support they had received, 6 of the 8 said that it had:

• I feel the project has given me confidence, they have been so encouraging, they make me feel like I

am more than just someone who sits in the house and sews all day. I don't have to go to them,

they come to me.

• My business will go somewhere but I do have a lack of confidence in myself because I can't keep up

now, the business has taken off and I have employed 2 knitters and looking for a spinner. I have

had to cancel 2 markets because I cannot knit fast enough. Not sure if I want all this, I want to spin

and knit and do it myself.

• They have given me more confidence and I am more driven to pursue. There are no ways of

showing your stuff, if there was a Craft guild which could suggest new events, something regular

to show your wares.

• I was confident and positive about my business and the future but my husband has had a terrible

accident and I am now looking after him, the odd order trickles in but home life is now centred

around my husband I don't have any spare time.

• I have more focus and more confident and my business has gone from being just local to national,

it really has evolved.

• There are quite a few strands of the business, we are breeding alpacas and using their fleeces,

there are now quite a few extra bits I can add to it. We have open days and walking with Alpacas is

something we might develop. The problem is the recession.

Benefits to the businesses

Just 1 of the 8 businesses had introduced new products or services as a result of the support

provide.

• I am always introducing new products. I started making cushions out of old clothes and now I

have evolved into driftwood.

However, 7 of the 8 said that they had been encouraged to work more with local businesses.

• We are always looking to work with local crafters and producers.

• Through the producers network I now go to another craft fair and some shops stock my product.

• I was introduced to XXXXX who runs Gorseinon market that is where the business started going

out of control and I now have to employ knitters.

• I have been chatting to other crafters, picking up new ideas and sharing information.

• I was busy networking before my husband had the accident, now I don't do any work.

• I have talked and shared ideas with people who have Alpacas.

• I have talked to other people involved in wood and glass and spinning and the possibilities of

working together.

Page 56: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 53

Other benefits identified by the businesses were:

• It has given me confidence... given us crafters a voice and made people aware of us.

• Yes, it has been nice to meet other like people, it is very social and you can benefit from other

people's experiences.

• [The project officer] has been very helpful, she sends emails of meetings, it is nice to know that

there are people there that you can ask if you have any questions. [The project officer] has

introduced me to some useful contacts.

• Confidence... it is a great thing to show new stuff at the meetings and give and get honest

advice.

• Networking and encouragement and sharing of ideas.

Attitude towards the area

Five of the 8 businesses interviewed had positive views on their area as a location to run a business,

2 were unsure and 1 had a negative view.

Positive Neutral or negative

• It is as good as any unless you are in the

Vale of Cowbridge.

• It's ok but I think it would be better to live in

Llandeilo or Cowbridge.

• I live on a farm 2 miles from the M4 and we

can get anywhere from this area, for this it

is ok. I don't know if people in this area

appreciate niche though, they are not

interested in quality, they like it cheap in

Swansea.

• It is quite good for a base, the centre of

Swansea is cheap, labour is cheap. I leant

that in Swansea there is more temporary

work that permanent.

• Swansea is no particularly worse than

anywhere else.

• It is the City by the sea, a poor sister to Cardiff.

It grew up as an ugly town but there are facets

that are beautiful, anything that can promote

the area would be good. I am a Swansea girl

but it needs to be more upmarket. I think

Swansea is trying, improving in a slow way.

Every place should be right to show good

quality stuff.

• I cannot compare it to anywhere else really.

There are lots of good people helping small

businesses; we have a lot of support around

here.

• The economy here is not a big spending

power. My product sells for £45 a shot if it was

in England it would be more like £80.

Four businesses said that their views had changed (for the better) due to their involvement with the

project.

• [The project officers] have improved things but it could be improved more. They have given us a

voice and made people aware of us.

• Definitely because of the support I have seen, the ideas, the networking it is amazing.

• It has widened my selling area.

• Seeing what other people are doing, the varied things, it is encouraging to see what is going on.

It feels like a lot of things are happening and I would not have known about it if I was not part of

the network.

Page 57: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 54

Evaluator’s comments

In terms of outputs and results, the management data suggests that the project will achieve or

achieve against all of the performance indicators. There is however again significant differences

between what has been achieved and what is anticipated will have been achieved by the end of the

project, most notably in terms of the result indicators.

As evaluators, we would question the validity of the indicators ‘the number of jobs potentially

created’ and the same for ‘safeguarded’. Whilst we understand the argument for including such an

indicator within a programme designed to pilot new and innovative ideas, the ambiguity of the

terms ‘potential’ makes the indicator difficult to define and, more importantly, to measure. How do

we know when a potential job has been achieved?

The interviews with a very small sample of businesses suggest that the support provided has had a

positive impact on the businesses who have been engaged. Most notably, businesses reported an

increase in their confidence, in their networking and in their trade with other local businesses. Our

sample is however again small. We would therefore recommend that the Partnership considers

commissioning a larger (more representative) sample of businesses that have benefited from the

support provided as part of the second phase of the project in order to provide better data on the

impact the support is having.

Page 58: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 55

Appendix 2: Local Development

Strategy themes and objectives Those objectives highlighted in red are those noted as being delivered by the projects within

Business Plan 1.

Theme 1: Supporting Rural Businesses

• Objective 1: To provide support for farming households to exploit new opportunities

(agricultural and non-agricultural, e.g. niche markets)

• Objective 2: To promote local food, drink, and crafts and link to branding

• Objective 3: To encourage sustainable tourism, and to strengthen the value and length of the

tourism season

• Objective 4: To develop products and activities using natural resources, including new energy

production methods, responsible use of local ecology, habitat and biodiversity as tourism

attractants – e.g. through eco-tourism

• Objective 5: To support rural businesses by improving the advice network and encourage

exchange of good practice and training events

Theme 2: Building Sustainable Rural Communities

(a) Capacity building and services

• Objective 6: To improve basic services (including facilities for children, broadband, community

facilities)

• Objective 7: To implement schemes to provide affordable housing

• Objective 8: To support initiatives which encourage the use of the Welsh language and the

retention of Welsh culture

(b) Buildings and Environment

• Objective 9: To support village enhancement schemes

• Objective 10: To encourage the conservation based industry in conjunction with stakeholders

such as the National Trust, the Countryside Council for Wales, Coed Cadw, the Wildlife of South

and West Wales Trust, and the City and County of Swansea

• Objective 11: To raise awareness of the potential impact of climate change through publicity and

events

• Objective 12: To promote energy conservation, micro-generation and the production of bio-fuels

Theme 3: Tackling economic inactivity

• Objective 13: To improve economic activity rates by tackling the underlying reasons such as

retraining and health improvement initiatives

• Objective 14: To reduce social exclusion by supporting local provision of services and education

facilities and better paid employment opportunities and by providing links between urban and

rural areas

Page 59: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 56

Theme 4: Skills Development

• Objective 15: To address skills gaps - provide non-mainstream training to develop personal and

community confidence and to encourage people to obtain more qualifications

• Objective 16: To expand community led initiatives including capacity building, networking, and

essential skills for group management

• Objective 17: To support the development of outdoor education and youth activities in

conjunction with the City and County of Swansea, other local authorities, the YHA, and the

Swansea Environmental Education Forum

Theme 5: Sustainable Transport

• Objective 18: To improve public and community transport provision

The following is an extract from Business Plan 1 which sets out how its projects will begin to deliver

against some of the key themes of the LDS:

Theme 1: Supporting rural businesses

The Walking and Cycling marketing campaign and Improvement of walking facilities in Mawr

Community projects will both undertake promotional activity to raise the profile of the existing

walking and cycling offer, as well as a small amount of infrastructure work. The promotional activity

undertaken through these projects will have a direct impact on numbers of visitors to the less well-

promoted areas, therefore generating income into those local communities, particularly in the more

northern rural parts of Swansea, and businesses in those areas both directly and indirectly

associated with tourism.

Capacity building work into linking local food to local businesses, and investigating the potential of

non-food and craft type activity, with a view to encouraging future development of these activities,

should scope and demand be identified to link to Axis 1 developments, and potentially Axis 3 work

under Business Plan 2.

Together these projects will begin to pull together various strands of initiatives and potential avenues

for the development of small businesses in Swansea’s rural areas.

Theme 2: Building sustainable communities (capacity building and services)

The Rural Youth Project will begin work against this theme by investigating the current level of

provision, and an initial start at addressing the gap identified in the Local Development Strategy

SWOT analysis of activities for young people, and addressing the associated risk of de-population of

the rural areas.

Theme 3: Tackling economic inactivity

Again, the Rural Youth Project will begin to make a contribution to development of this theme by

starting to address the lack of provision for young people identified in the Local Development

Strategy SWOT analysis.

Page 60: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

Evaluation of the Swansea Rural Development Business Plan 1 – 2008 to 2011

Final Evaluation Report

©Copyright Wavehill Ltd. 2011 Page | 57

Theme 4: Skills development

Skills and capacity development is essential for the implementation of the RDP in Swansea, and will

begin under Business Plan 1 through training for the Partnership and Local Action Group, raising

awareness and capacity to participate, and activities integrated into the consultation work as part of

the Axis 4 projects. The Rural Youth project will also, should initial scoping work deem it appropriate,

enable development of outdoor education and activities for young people.

As Swansea has not previously benefited from the LEADER programme, implementation of this

approach is new to Swansea’s rural areas, but the new challenge is already being met with

enthusiasm from the new Local Action Group, Rural Swansea Action. The activities proposed under

Axis 4 are fundamental to building capacity, both within the Local Action Group, as well as

underpinning the baseline information needed to make informed decisions about further activities

under Business Plan 2 for Axis 3 and 4, along with potential under Axis 1 and 2, focusing particularly

on research into demand and supply of local food and crafts businesses, and whether or how their

potential can be enhanced.

Theme 5: Sustainable Transport

To a lesser extent, there is a contribution to Theme 5 Sustainable Transport. The walking and cycling

promotional campaign will exploit the existing public transport infrastructure by encouraging the

use of public transport, especially by bus, to participate in these recreational activities. It will also

complement the work being undertaken by the BayTrans Partnership in the Swansea and Neath Port

Talbot rural areas to promote the use of public transport.

Page 61: Evaluation of the Swansea Rural Development Business Plan 1: 2008 to 2011

wavehill ymchwil gwerthuso arolygon

research evaluation surveys

t: 01545 571 711

e: [email protected]

w: www.wavehill.com

Wavehill Ltd, 8 Water Street | 8 Heol y Dŵr, Aberaeron, Ceredigion, SA46 0DG