increasing skills thematic evaluation report

Thematic Evaluations Contract Number: C-32/2011/12

Increasing SkillsFinal Report

30 Queen Square Bristol BS1 4ND T. 0117 927 3401 F. 0117 929 4189 E. [email protected]

W. www.ers.org.uk

Commercial in Confidence

Contents

EXECUTIVE SUMMARY ............................................................................................ 2

1. INTRODUCTION............................................................................................. 5

2. DELIVERY CONTEXT .................................................................................... 9

3. COHERENCE OF PROJECTS WITH POLICY GOALS............................... 17

4. PERFORMANCE TO DATE AND ANTICIPATED OUTTURN..................... 22

5. THEMATIC AREA SPECIFIC ANALYSIS.................................................... 38

6. CONCLUSIONS AND RECOMMENDATIONS ............................................ 57

APPENDIX 1: CONSULTEE LIST............................................................................ 62

Thematic Evaluations Increasing Skills Final Report

EXECUTIVE SUMMARY

In September 2011 the Welsh European Funding Office (WEFO)

commissioned thematic evaluations of the 2007-13 EU Structural Funds

Programmes to assess alignment with Operational Programme (OP)

objectives and progress of funded activity. The following report addresses the

Increasing Skills thematic area.

The thematic area comprises two Priorities: Convergence ESF Priority 3 and

Competitiveness ESF Priority 2. An allocation of £301.1 million ESF funding is

made for Convergence ESF Priority 3 across three Themes: raising the level

of skills in the workforce, supporting the provision of skills for the knowledge

economy and promoting gender equality in employment. Competitiveness

ESF Priority 2 is much smaller with £28.1 million allocated ESF funding. The

OP is complemented by four strategic frameworks.

Coherence of Projects with Policy Goals To date 35 projects have been approved, 27 under Convergence and eight

under Competitiveness. In absolute terms, there is good fit between the scope

of projects approved and the policy goals within the OP for Convergence ESF

Priority 3. However, some gaps have been identified in respect of specific

activities targeted to disabled participants, migrant workers and some issues

of gender balance in the workforce. There are more substantive gaps in

respect of Competitiveness ESF Priority 2 including a lack of research into

skills needs and systems for matching learning to employer demand as well

as activity focused at a number of target groups. For both Priorities there is

very little evidence of any duplication. There is some evidence of imbalance

between approved project activities and OP aims when considering the

financial commitments made to date within Convergence ESF Priority 3, as

this includes over commitment on Theme 2 addressing higher level skills.

There is limited evidence on which to assess the balance of approved

activities in terms of profiled indicators against targets for participant mix and

skill levels. However, there is some suggestion that the mix of approved

activity is lacking the focus on basic skills that is detailed within the OP.

Performance to Date and Anticipated Outturn

Thematic Evaluations Increasing Skills Final Report 2

Across the two Priorities 91% of EU funds have been committed and 32% of

funds have been spent (70% of forecast expenditure to date). Confidence

was expressed at Programme and project level that funds will be fully spent

by the end of the Programming period.

Delivery performance against output and result indicators is variable. There is

very strong performance in terms of the number of participants and

participants gaining qualifications with sound expectation that OP targets will

be exceeded. However, when looking beneath the headline level, there are

some concerns in terms of the ability to achieve the target profile of participant

type and activity by skill level as set out within the OP.

There is uncertainty in respect of indicators relating to employers assisted or

financially supported under Convergence ESF Priority 3, despite over-

commitment of outputs across approved projects. The equivalent indicator for

Competitiveness ESF Priority 2 is expected to be achieved. The target for

employers adopting or improving equality and diversity strategies and

monitoring systems is unlikely to be achieved on the basis of current

evidence. The target for projects delivering specialist training in sustainable

development should be met.

The specific reasons for variance in the delivery of indicators compared to

forecast profiles vary across projects. Common issues include delays in

commencement of project delivery following receipt of project approval.

These delays included over optimistic lead in times within project business

plans, delays caused by the procurement process and in some cases

procedural issues delaying the final sign off of project approval. There is also

some evidence of over optimistic profiling of indicators at the business

planning stage, this in part highlights the risks inherent with forecasting as

well as the change in economic climate.

Thematic Area Specific Analysis The level of collaboration engendered by the Structural Funds Programmes

was cited as a positive outcome by a broad range of consultees. This was

particularly driven by the commitment to fewer, larger projects. This has


brought together

Institutions, Further Education Institutions, Local Authorities and social

partners in collaborations that would have been unlikely without the

Programmes. A particular success is that of Sector and Provider Advocates,

which have strengthened communication between employers and their

representatives and the education and training sector.

Whilst there is very limited transnational activity or use of cross fund flexibility

tools, there is evidence of innovation within approved projects. There is also

evidence that projects are adding value through enabling the delivery of much

larger projects than would otherwise be the case, by broadening the scope of

activity which can take place and through testing innovative projects which in

some cases have already been adopted within the mainstream activities of

the skills sector. The evidence shows projects have responded well to

delivering demand led provision, particularly through Sector Skills Councils

(SSCs). However, changes to SSC arrangements pose a potential risk to the

remainder of the programme period.

Recommendations On the basis of the evidence recommendations to be considered by WEFO

within the current Programming round include:

Welsh Government departments, Higher Education

� Managing the balance of activity between Themes when committing

remaining funds;

� Careful monitoring of indicators over the next 12 months to ensure the

back-loading of indicators continues to remain feasible;

� Improved monitoring and management of indicators below the headline

participant level. In particular this needs to focus upon target groups and

activity and achievement by level of qualification;

� Increasing awareness of the role mainstream projects can play in

proactively engaging target groups; and

� Raising the profile of the availability and purpose of cross fund flexibility

tools.


Formatted: Indent: Left: 0 cm, Hanging: 0.63 cm, Bulleted + Level: 1 + Aligned at: 0 cm + Tab after: 0 cm + Indent at: 0.63 cm, Tabs: Not at 1.27 cm

1. INTRODUCTION

Formatted: Indent: Left: -0.01 cm, Hanging: 0.76 cm, Outline numbered + Level: 2 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at: 0.63 cm + Tab after: 0 cm + Indent at: 1.4 cm

1.1. Background

In September 2011 the Welsh European Funding Office (WEFO)

commissioned evaluations of the 2007-13 EU Structural Funds Programmes

in Wales: Convergence ERDF, Convergence ESF, Competitiveness ERDF

and Competitiveness ESF. The four Programmes were subdivided into eight

thematic areas. One report has been prepared for each thematic area, with a

further overarching report. This is one of the eight thematic reports focusing

on the Increasing Skills thematic area comprising Convergence ESF Priority 3

and Competitiveness ESF Priority 2. Formatted: Indent: Left: -0.01 cm, Hanging: 0.76 cm, Outline numbered + Level: 2 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at: 0.63 cm + Tab after: 0 cm + Indent at: 1.4 cm

1.2. Purpose of Report

The purpose of this report is to assess a) whether the right types of activities

are being delivered through Convergence and Competitiveness ESF

Programmes to meet the Increasing Skills objectives and b) whether enough

progress is being made in the delivery of these activities.

In more detail the objectives are to:

� Establish the financial and physical progress the Programmes are making

towards achieving their objectives as set out in the Operational

Programme(s) (OP);

� Establish the coherence of projects approved with the policy goals as set

out in the OP(s);

� Measure performance to date at Priority level and project level (where

appropriate) in terms of projects approved, financial commitment, spend,

and output and result indicators compared to project and programme

targets and to assess the likelihood of achieving targets by the end of the

Programmes;

� Uncover reasons for any apparent slippage or underperformance or over

performance to date and the potential to rectify this as projects continue;


Formatted: Indent: Left: 0 cm, Hanging: 0.63 cm, Bulleted + Level: 1 + Aligned at: 0 cm + Tab after: 0 cm + Indent at: 0.63 cm

� Assess the extent to which:

o projects are working together with other projects within the Priorities;

o Priorities are suited to undertaking transnational activities and to

maximising links to other EU Community Initiatives;

o the cross fund flexibility tool designed to encourage cross financing

(ESF and ERDF) is being utilised (and, if so, whether this is

successful);

o projects within the Priorities are conducting innovative activities as

stipulated in the Strategy section of the OP;

o investments made within the Priorities are adding value to national

policies; and

� Report conclusions and recommendations in terms of the likely outturn on

current trends for the Priorities compared with targets and the level of

resourcing for the Priorities.

In addition, for the Increasing Skills thematic area there are the following

specific questions:

� Has the Priority been sufficiently focused on improving the levels of basic

literacy and numeracy alongside development of other generic and

vocational skills? (Competitiveness only)

� Have the projects within the Priority responded well to delivering demand

led provision of qualifications at level 3 and beyond? (Convergence only)

� Has the Priority succeeded in achieving the right balance between lower,

intermediate and higher level skills?

� Has the Priority been sufficiently focused on challenging the traditionally

gender dominated cultures, environments and attitudes within the

workplace e.g. men in teaching and nursing, women in construction and

scientific professions?

� Is the Priority contributing to the European Employment Strategy (EES)?

Are projects approved consistent with the priorities set out in the EES?

� To what extent are social partners (employers and trade unions) engaged

in the Priority? What is the nature of the engagement?





1.3. Methodology

The method adopted for the work underpinning this report was: to review the

relevant background documentation (e.g. the Operational Programmes,

Strategic Frameworks, summary project business plans and other documents;

to analyse monitoring data relating to expenditure commitments and spend

and associated outputs and results; consultations with WEFO staff, members

of the Programme Monitoring Committee, Strategic Framework Coordinators;

and to carry out case studies interviews of up to ten projects1 within the

thematic area. The case studies were projects which had been operating for a

minimum period of six months and included two or three projects with the

largest financial allocations within the thematic area. In addition they were

selected to reflect:

� The range of thematic area activities;

� The balance of expenditure between the Convergence and

Competitiveness Programmes;

� A variety of types of project sponsors;

� A variety of localities; and

� The balance of performance to date.

Case studies included reviews of background documentation including project

business plans and evaluations where available, analysis of monitoring data

and consultations with project managers. Please note that not all projects

within the theme were interviewed.

The documentary review and fieldwork for the evaluation took place in late

2011. The analysis of monitoring data was initially undertaken in late 2011

using data for mid September 2011. The analysis was later updated using

data for end September 2012.

Within the Increasing Skills thematic area 16 approved projects were consulted. In all

instances of pan-Wales projects interviewees indicated that the issues discussed were

identical across both Convergence and Competitiveness projects.



1

1.4. Programme Overview

The Convergence Programmes for West Wales and the Valleys comprise

funding from the European Regional Development Fund (ERDF) of around £1


billion and the European Social Fund (ESF) of around £690 million. The

ERDF Programme aims to transform Wales into a sustainable and

competitive economy by investing in the knowledge economy and helping

new and existing businesses to grow. It also seeks to regenerate Wales’

most deprived communities, tackle climate change and improve transport.

The ESF Programme aims to tackle economic inactivity, increase skills and

boost employment. The Convergence Programmes cover 15 local authority

areas in West Wales and the Valleys and, together with match funding, are

expected to be worth around £3.5 billion.

The Regional Competitiveness and Employment (Competitiveness)

Programmes cover East Wales and also comprise funding from ERDF

(around £60 million) and ESF (around £50 million). The ERDF Programme

seeks to help new and existing businesses to grow and move up the value

chain, and increase the ‘value added’ per job, as well as regenerate Wales’

most deprived communities and tackle climate change. The ESF Programme

aims to tackle economic inactivity, increase skills and boost employment. The

Competitiveness Programmes cover seven local authority areas in East

Wales and, together with match funding, are expected to be worth around

£280 million.


2. DELIVERY CONTEXT

2.1. Background

The Increasing Skills thematic area comprises two Priorities:

Convergence ESF Priority 3 - Improving Skills Levels and Adaptability of the

Workforce – to support productivity and progression in employment by raising

skills levels at the levels of basic skills, intermediate and higher level skills, to

tackle skills gaps and shortages, and to promote gender equality in

employment. Within this Priority there are three Themes as outlined below:

Convergence ESF Priority 3 Theme 1

Theme 1 aims to improve levels of basic literacy and numeracy skills, also

ICT skills, other generic skills and occupational skills within the workforce and

support progression to Level 2, with demand-led progression up to, and

including Level 3.

This will be achieved by:

� Raising levels of basic literacy and numeracy in the workforce;

� Supporting the acquisition of ICT, generic and occupational skills in the

workforce;

� Supporting progression in employment through flexible training and

learning provision; and

� Helping low skilled workers to gain the additional skills and

qualifications needed to sustain their employment, improve productivity

and increase earnings via career progression.


Theme 2 aims to support the provision of higher level skills, including

graduate and post graduate, to support the knowledge economy, to improve

systems to identify skills needs, to help employers identify their skill needs,

anticipate potential gaps or shortages, and to help people adapt to changing

economic pressures impacting on businesses and organisations.

Formatted: Indent: Left: -0.01 cm, Outline numbered + Level: 2 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at: 0.63 cm + Tab after: 0 cm + Indent at: 1.4 cm

Formatted: Indent: Left: 0 cm, First line: 0 cm, Bulleted + Level: 1 + Aligned at: 0 cm + Tab after: 0 cm + Indent at: 0.63 cm



� Supporting the knowledge economy, including the development of

higher level and graduate and post graduate skills;

� Improving research into skills needs and systems for matching learning

supply to demand from employers and individuals;

� Support employers in identifying and addressing the skills needs of

their sectors at all levels, including management, technical and craft

skills;

� Matching and improving learning provision so that it better meets

labour market needs, thereby reducing skills gaps and shortages; and

� Helping workers and enterprises adapt to new forms of work

organisation and new technologies.


Theme 3 aims to promote gender equality in employment and career

advancement, in particular tackling the causes of the gender pay-gap.


� Raising awareness amongst employers and elsewhere of gender

disadvantage in the workforce; and

� Raising participation levels in occupations and sectors where men and

women are currently under-represented.

Competitiveness ESF Priority 2 - Improving Skills Levels and Adaptability of

the Workforce – to support productivity, adaptability and progression in

employment by tackling low skills, raising skill levels amongst managers and

workers in small enterprises, and improving systems for anticipating and

responding to skill needs. The aims and objectives as set out within the OP

are as follows:



Competitiveness ESF Priority 2

To improve the basic skill levels within the workforce, increase the leadership


and management skills of managers and workers in small enterprises, and

support systems for anticipating and responding to skill needs.


� Improving basic skills provision in numeracy and literacy, problem

solving and customer handling through targeted interventions working

with employees and employers;

� Support for leadership and management skills acquisition (at all levels)

for managers and employees in small enterprises; and

� Improving research into skills needs and systems for matching learning

to employer demand.

2.1.1. Strategic Frameworks

There are four Strategic Frameworks relating to each of the Priorities/Themes

outlined above. They reflect key economic and skills policy, the principles laid

down in the OP and also take into account spatial dimensions. The aims and

objectives of the Strategic Frameworks mirror those outlined in the

Operational Programme set out above. In addition they detail the types of

interventions through which the Strategic Objectives will be delivered. All four

strategic frameworks stress the need for engagement with social partners and

the need to build innovation and environmental sustainability into projects.

The four frameworks are:

� Convergence ESF Priority 3 Theme 1: Raising the Skills Base of the

Workforce

� Convergence ESF Priority 3 Theme 2: Skills for the Knowledge Economy:

Workforce Development and Learning

� Convergence ESF Priority 3 Theme 3: Promoting Gender Equality in

Employment

� Competitiveness ESF Priority 2: Improving Skill Levels and the

Adaptability of the Workforce



Formatted: Indent: Left: 0 cm, Hanging: 1 cm, Outline numbered + Level: 3 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at: 1.27 cm + Tab after: 0 cm + Indent at: 2.16 cm



2.1.2. Financial Allocations and Targets

An allocation of £301.1 million ESF is made for Convergence ESF Priority 3,

this is split across three Themes, with Theme 1 identified to take the majority

of the funding. Competitiveness ESF Priority 2 is much smaller with £28.1

million allocated ESF.

Table 1 details the output and result indicators and targets for the Priorities.

The output indicators include two Programme Level Indicators: Total

Participants (Employed) and Female Participants. In addition a range of

targets detailing the intended mix and balance of indicators across specific

groups and skill levels are included. All of the results indicators with the

exception of ‘employers adopting or improving equality and diversity

strategies and monitoring systems’ are also identified as Programme Level

Indicators.

The OP sets out a number of impact indicators, however, no targets were

detailed and these indicators not subject to assessment within this evaluation.


Table 1: Indicators and Targets

Convergence Priority 3

Competitiveness Priority 2

Total

Outputs Total participants (Employed) 90,000 12,600 102,600 Female participants 66% 55% n/a Key intervention groups:

Older participants 30% 30% n/a BME participants - - n/a Participants with work-limiting health condition or disability 13% 15% n/a Participants accessing basic skills qualifications 45% 55% n/a Participants accessing Level 2 training 30% 25% n/a Participants accessing Level 3 training 15% 10% n/a Participants accessing Level 4 and above training 10% 10% n/a Female participants who work part time 29% 25% n/a

Employers assisted (or financially supported)2 15,000 2,100 17,100 Research studies - - n/a Learning and development strategies - - n/a Projects delivering specialist training in sustainable development 10% 10% n/a Results Participants gaining qualifications (Employed) 37,530 5,450 42,980

Basic Skills 54% 63% n/a At Level 2 29% 24% n/a At Level 3 11% 7% n/a Level 4 and above 6% 6% n/a

Participants entering further learning - - n/a

Slightly different indicators between the Convergence and Competitiveness Programmes. In the Convergence Programme the indicator is employers assisted or financially supported. In the Competitiveness Programme it is employers assisted.

Thematic Evaluations Increasing Skills Theme Draft Report 13

2

Convergence Priority 3

Competitiveness Priority 2

Total

Progression schemes for women (Convergence only) - - n/a Employers adopting or improving equality and diversity strategies and monitoring 50% 50% n/a systems



2.2. Policy and Operational Context

The Convergence and Competitiveness Programmes contain a number of

high level ‘tracking indicators’ which provide the overall context for assessing

economic progress. The targets for the Programmes, and the basis on which

projects were approved, were predicated on a continuation of favourable

national and international economic conditions. Project and Programme

design was based on the aim of achieving structural change in order to close

the gap between Welsh and national/EU averages in respect of a selection of

key indicators. In the event, the Convergence and Competitiveness

Programmes have had to pursue this aim in the face of a deep recession in

the UK followed by a very slow recovery.

Whilst there has been a considerable change in the economic situation it is

not possible to apply either a generic discount or series of specific discounts

to the targets to which the Programmes are working as these have been

agreed with the European Commission. Indeed, the achievement of some

targets may have been aided by recession, such as the numbers of people

assisted with learning and/or employment opportunities. In some instances it

also led to the formulation of specific projects. For example, within the

Increasing Skills thematic area the onset of recession led to the creation of

the ProAct project, which emerged directly from a series of Economic

Summits in the autumn of 2008 that brought together government, businesses

and other stakeholders from across the Welsh economy. Nevertheless, more

widely, the recession created challenges for many projects looking to train

employees. Consultations have indicated that many employers have been

operating in survival mode, struggling to retain staff let alone invest in their

training and development. As a result, projects which have been looking to

trial increased employer contributions for training have indicated difficulties in

recruiting participants in some instances.

Since the Strategic Frameworks were drafted there have been no substantive

policy shifts which have been reported by consultees to have impacted upon

the design and delivery of projects within this thematic area to date, although

Economic Renewal: A New Direction has had an indirect impact, particularly


through its shift to a sectoral approach. The change in government within the

National Assembly for Wales and the publication of the Programme for

Government 2011-16 will influence the policy environment in which the

Structural Funds Programmes are delivered over the remainder of the

Programme period. This includes alignment of skills activity to the low carbon

agenda and sector priorities as well as building on a number of project

elements within the Increasing Skills thematic area including improved labour

market intelligence and apprenticeships.


3. COHERENCE OF PROJECTS WITH POLICY GOALS

3.1. Gauging Coherence of Projects with Policy Goals

As at 30 September 2012 35 projects have been approved within the

Increasing Skills thematic area, 27 within Convergence ESF Priority 3 and

eight within Competitiveness ESF Priority 2.

In assessing the coherence of approved projects with OP policy goals three

lines of enquiry have been explored:

� Mapping approved projects to strategic objectives, indicative activities and

target groups to identify any gaps and/or duplication in provision;

� Assessing the alignment of committed funds to the planned profile of

expenditure across activities within the OPs; and

� Assessing the alignment of forecast outputs from approved projects with

the balance of targets set out within the OPs.

3.2. Mapping Projects to Strategic Objectives

The mapping exercise has been undertaken based upon the review of

business plan executive summaries for each approved project, project level

interviews with a sample of projects and interviews with a range of wider

stakeholders, including Strategic Framework Coordinators, WEFO staff and

PMC members.

Current approved activity covers the vast majority of OP aims and objectives

within Convergence ESF Priority 3. Major activity areas are covered through

large scale Welsh Government sponsored projects, with smaller scale

projects targeted at more specialised areas. However, the evidence suggests

there are some target groups that are potentially under-served in the

committed activity.

Areas where the potential for gaps have been identified include:

� P3T1 – lack of focus on disabled participants. Whilst this target group is

noted within the business plans of a number of projects, there is no

specifically targeted project to address the needs of this group. Data






presented in Chapter 4 of this report shows levels of activity with disabled

participants is behind target.

� P3T1 – validating skills of migrant workers. Whilst a number of projects

have supported migrant workers this has not included any activity to

validate existing skills. One project, Minorities are Wales Resources,

refers to this activity within its business plan. However, consultations did

not provide any evidence of validation activity taking place or plans in

place to move this activity forward.

� P3T2 – lack of focused activity supporting women to secure higher level

skills in non-traditional areas. Whilst many projects are looking for women

to make up a substantial proportion of participants there is no clear

evidence from summary business plans or consultations of projects

explicitly supporting women to secure higher level skills in non-traditional

areas.

� P3T3 – lack of focus on challenging traditionally female dominated

occupations. Whilst there are two projects supporting women, the OP

makes explicit provision for tackling female “dominated cultures,

environments and attitudes e.g. men in teaching and nursing”. Activity to

address this has not been identified.

The mapping exercise identified very little evidence of potential for duplication

within committed activity. Two areas where possible duplication was

identified were:

� P3T1 – basic skills provision – with a large Welsh Government sponsored

project (Essential Skills in the Workplace) and regional FE sponsored

projects (e.g. Regional Essential Skills) which include some basic skills

elements. Consultees reported inter-project dialogue in order to clarify

roles and areas of focus to mitigate duplication.

� P3T2 – leadership and management – with a large Welsh Government

sponsored project (ELMS) and a regional HE sponsored project (LEAD).

Both projects target owner managers in SMEs to deliver leadership and

management training, albeit through slightly different approaches. Both

projects are currently running behind profile in terms of participant



numbers. The reasons for this are likely to be complex, but over capacity

as a result of two similar approved projects cannot be dismissed.

One consultee queried whether it was possible to avoid duplication with the

scale of projects in the current Programmes, particularly where there are

substantial amounts of detailed planning and implementation post-approval:

“I am not convinced WEFO processes avoid duplication. When a large

project is approved it doesn’t know who is going to deliver where” (Project

Sponsor)

When considering Competitiveness ESF Priority 2 a slightly different picture

emerges. Two of the three strategic objectives within the OP are well served.

However, the mapping exercise suggests a number of gaps in activity when

compared with the list of indicative activities set out within the OP. In

particular these gaps relate to:

� Improving research into skills needs and systems for matching learning to

employer demand. Whilst this activity is taking place within projects under

the Convergence Programme no evidence has been found of this type of

activity taking place under Competitiveness ESF Priority 2.


� Focused activities on target groups. Whilst some projects are intending to

deliver interventions to participants within the target groups there is little

evidence of bespoke activity to meet the particular needs of target groups

either within broader projects or through specific niche projects.

Specifically:

o Activity to train, mentor and support men and women in occupations

or sectors where their gender is under represented. Whilst there is

approved activity to deliver generic leadership and management

development there is no evidence of targeted activity to meet this

ambition.

o Activity to enable migrant workers to validate the qualifications

obtained in their own country. There is no approved project setting

out to deliver against this activity.



o Activity to develop new ways of supporting engagement of older

workers in learning. No project has been approved to address this

area.

The mapping exercise identified very little evidence of potential for duplication

within committed activity. One area where possible duplication was identified

was:

� Leadership and management – with a large Welsh Government sponsored

project (ELMS) and a regional HE sponsored project (20 Twenty

Leadership) offering almost identical support activities including leadership

and management workshops as well as coaching and mentoring.

However, project level interviews have not found evidence of this impacting at

an operational level. It was reported that “duplication is not cropping up as an

issue with clients” (Project Sponsor).

3.3. 3.3 Profiling Committed Funds to the Operational Programme

The Convergence ESF Operational Programme sets out an indicative split of

funds across the three Themes within Priority 3. Table 2 assesses the profile

of committed funds to date against the OP ambition. This shows a marked

variation between actual and target activity across Themes 1 and 2. Theme 2

was anticipated to account for around 35% of grant. To date this Theme

accounts for 44% of grant, well ahead of the expected share. There is no

thematic split of Competitiveness ESF Priority 2.

Table 2: Balance of Committed Funds


Deleted: Table 2


OP Indicative

Share Current

Share

Commitment Against

Total OP Value

Remaining Funds

(£m)

Theme 1 60% 52% 77% 41.0

Theme 2 35% 44% 112% -12.5

Theme 3 5% 5% 77% 3.5

In order to account for the fact that not all funds have yet been committed the

third and fourth columns in Table 2 profile committed against the balance of

total grant within the Priority. This shows Theme 2 is running at 112% of

expected value in comparison to Theme 1 and Theme 3, both at 77%.

Deleted: Table 2

3.4. Profiling Forecast Indicators to the Operational Programme

Within the OP a clear target mix of participants and activity is set out. This is

set out within Table 1 in this report. This includes reference to female, older,

disabled and BME participants, as well as the mix of training accessed and

qualifications achieved by level (basic, Level 2, Level 3 and Level 4 and

above). The target profiles by level of qualification are particularly pertinent

as they reflect the wider aims of the OP as set out in the supporting text. For

example, “The main focus of the priority will be to raise skill levels of those

with skill levels below level 2, especially employees without basic numeracy,

literacy and ICT skills, to bring them to level 2 and above”.

However, whilst targets for this mix of activity are set out within the OP, no

data on the forecast3 indicator profile of committed projects against this mix is

available for review. As a result, it is not possible to assess whether the mix of

committed activity is coherent with the goals of the OP in this regard. WEFO

should consider collating data from projects in order to assess the coherence

of committed activity and to enable a detailed assessment of likelihood of

achieving OP targets by the end of the Programme period.

Some data are available on delivered activities and considered in later sections of this

report.



3

4. PERFORMANCE TO DATE AND ANTICIPATED OUTTURN

4.1. Financial Progress

Funds are committed to projects at approval sign-off. The data relating to

commitment of funds to projects are set out within section 4.1.1. Each project

then profiles how funds are expected to be drawn down throughout the life of

the project. Section 4.1.2 presents data and analysis in respect of actual

project expenditure against profile to date.


4.1.1. Commitment of Funds WEFO records indicate that 91% of allocated EU funds have been committed

to projects. There is a difference between the Priorities with Competitiveness

ESF Priority 2 having committed 104% of funds, compared to 89% for

Convergence ESF Priority 3.

All WEFO consultees with relevant knowledge of this area of the Programme

indicated a clear confidence that all funds would be committed via new

projects and the active management of budgets for existing approved

projects.

Table 3: Committed Funds £ Millions

Programme and Priority Allocated Committed Not committed

Percentage Committed

Convergence ESF P3 301.1 269.1 32.0 89%

Competitiveness ESF P2 28.1 29.1 -1.0 104%

Total 329.1 298.1 31.0 91%


Source: WEFO, 30/09/2012

1 Note that the exchange rate is a planning rate of £1:€1.25


4.1.2. Expenditure Error! Reference source not found. illustrates Increasing Skills expenditure

as a percentage of profiled spend to 30th September 2012. Both Priorities

have failed to meet their profiled target, largely as a result of project delays as


Deleted: Figure 1

discussed in more detail at section 4.4. Overall, the Increasing Skills thematic

area has spent 70% of profile. Looking at individual Priorities,

Competitiveness ESF Priority 2 is slightly further ahead at 74% with

Convergence ESF Priority 3 at 69%.

Figure 1: Spend Against Profile

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Convergence ESF P3 Competitiveness ESF P2 Total

Programme / Priority

Just under two-thirds, 64%, of total committed funds remain to be spent within

the Increasing Skills thematic area (63% for Convergence ESF Priority 3 and

63% for Competitiveness ESF Priority 2). When including monies yet to be

committed the remaining expenditure rises to 68% of total allocated funds.

Therefore 32% (£106.5 million) of allocated funds have been spent in the first

57 months of the Programme period at a rate of £1.9 million per month.

Looking forward 68% (£222.6 million) needs to be spent in the remaining 39

months at a rate in excess of £5.7 million per month.

However, no substantive concern has been raised through project level

interviews about the ability of projects to spend approved funds within the

Programme period. There was at least one example of a project front-loading

expenditure with match funding, resulting in substantial back-loading of ESF

monies. There are also numerous projects which will claim expenditure upon


qualification achievement, which leads to back-loading of expenditure profiles.

Consultations with WEFO staff demonstrate confidence in the ability to

commit and spend remaining funds in this thematic area. No evidence to

dispute this has been found.

4.2. Indicator Progress

Set out in the following pages is a review of progress against profile and

target for a range of programme monitoring indicators (output and result). This

is coupled with qualitative evidence captured from consultations with projects

and stakeholders to provide an assessment of likely outturn at the end of the

Programme period. The summary position is presented in Tables 4 and 5

with further detail including output and profile charts and related data tables

set out in Appendix 4.


4.2.1. Participants Data on participants show over-commitment against OP targets. Current

output profiles show that Competitiveness ESF Priority 2 is anticipated to

deliver 343% of target, and for Convergence ESF Priority 3 over-commitment

is profiled at 184%. Performance against expected profile to-date is very

good for Competitiveness ESF Priority 2 at 76% and good for Convergence

ESF Priority 3 at 73%. As a result of the over-commitment, the

Competitiveness ESF Priority 2 target has already been met and

Convergence ESF Priority 3 is running at 80%.

Whilst two projects consulted as part of the evaluation that have substantial

participant forecasts cast some doubt on their ability to achieve current profile,

when considering the output data alongside the full range of views of

consultees from the sampled projects there is every reason to expect final

output levels to be well in excess of OP target for both priorities, particularly

with a number of pipeline projects expected to be committed which will

increase output commitment even further.


4.2.2. Participants Gaining Qualifications In common with the headline participant output indicator, the gaining

qualifications result indicator is similarly over-committed (351% in



Competitiveness Priority 2 and 248% in Convergence Priority 3).

Performance against expected profile is good at 78% in Convergence ESF

Priority 3 and 96% in Competitiveness ESF Priority 2. As a result of the over-

commitment, when compared with OP target, Competitiveness ESF Priority 2

is running at 169% and Convergence ESF Priority 3 at 102% at roughly the

mid-point of the Programme period4.

Consultees raised no substantive concerns relating to achieving profiled

levels that would suggest a failure to achieve OP target. In fact, considering

the result and profile data alongside the views of consultees from the sampled

projects there is every reason to expect final levels to be well in excess of OP

target for both Priorities. This is particularly the case given indications

through consultations that a number of pipeline projects and project

extensions are expected to be committed, which will increase result

commitment.

4.2.3 Participants Entering Further Learning There is no formal OP target for this indicator presented for the relevant

Priorities despite this indicator being listed as a Programme Level Indicator in

both Convergence ESF and Competitiveness ESF OPs. Recorded output

data are only available for Convergence ESF Priority 3, with no data for

Competitiveness ESF Priority 2.

This indicator has been subject to substantial performance revision and re-

profiling. Notwithstanding, performance to date is well behind expected

profile.

The total forecast profile of 13,350 equates to 8% of profiled participants.

However, evidence from the 2010 ESF Leavers Survey indicates higher levels

of participation in further learning, at 28% of respondents. This conflict is in

keeping with views reported by three project managers which suggested

some challenges with collecting accurate data for this indicator, which are

4 As project delivery can take place up to end 2015.


likely

challenges relate to the follow-up of participants. In some instances this

follow-up is lacking (i.e. consultees indicated that there is no established

mechanism in place to capture this data) and in other instances it occurs a

relatively short time after intervention completion (generally within a period to

a maximum of three months). For employed persons (the target group of this

thematic area), progression onto further learning may not be immediate,

particularly after lengthy training courses. One consultee cited an example

where a person may have undertaken learning to achieve an Apprenticeship

in the construction industry over a period of three years. Following completion

there may not be a need to undertake further learning immediately (i.e. within

three months), albeit this could be required after a year or more in order to

develop further skills. As a result the indicator, and the processes to capture

data against it lack the subtlety required.

On the basis of these reporting issues and the results achieved to date it will

be challenging to achieve reported results in line with the current end of

Programme forecast. Particularly as the Modern Apprenticeship World Class

Skills For A Knowledge Economy project is now completed and has achieved

only 40% of forecast.

to lead to under-measurement and reporting. In particular these

4.2.3. Employers Assisted (Competitiveness Priority 2) The employers assisted output indicator is substantially over-committed at

346%. Whilst performance against expected profile is poor at 46%, largely as

a result of a number of projects being slow to start for various reasons, more

than two thirds of total OP target has already been achieved.

Project level interviews suggest there will be some slippage against this

indicator. For example, there is heavy reliance upon the ELMS project which

is behind profile. However, as a result of the current over-profiling this should

not be to an extent that will threaten achievement of OP target. If

performance against expected profile continued at 46% the OP target would

be comfortably exceeded, and there is evidence from consultations to suggest

a degree of catch up against forecast profile will take place for a number of

projects.



The heavy reliance upon two projects5 to achieve target profile suggests a

need for careful monitoring of these projects.


4.2.4. Employers Assisted or Financially Supported (Convergence Priority 3)

This indicator is effectively the equivalent to Employers Assisted within the

Competitiveness ESF Priority 2 discussed above and is subject to broadly

similar issues. However, the level of over-profiling of outputs is lower at

140%. Performance against expected profile is just above 50%. Whilst three

projects are ahead of profile, the majority are behind and four have yet to

record any outputs against this indicator. Project interviews indicated a

number of projects, particularly those with substantial profiles against this

indicator (including Essential Skills in the Workplace), being slow to start for a

range of reasons (see section 4.4). When one consultee was questioned

about potentially missing current profiled targets they indicated that:

“WEFO want quality rather than quantity” (Project Sponsor)

This indicates an interpretation by at least one project that achieving profile is

not mission critical.

Project level interviews suggest there will be some slippage against this

indicator and some re-profiling of projects may be required. For example, the

ELMS project in Convergence Priority 3 has indicated it is unlikely to achieve

current forecast against this indicator. The full extent of any re-profiling is

uncertain, but the evidence suggests that the current degree of comfort as a

result of over-commitment does not give any cause for complacency.

4.2.5. Employers Adopting or Improving Equality and Diversity Strategies and Monitoring Systems

The OP target for this indicator will be measured as a percentage (50%) of

employers assisted. The ultimate target is therefore reliant on delivery of this

5 Enhancing Leadership and Management Skills and Essential Skills in the Workplace


Formatted: Indent: Left: 0.11 cm, Hanging: 1.16 cm, Outline numbered + Level: 3 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at: 1.27 cm + Tab after: 0 cm + Indent at: 2.16 cm

indicator and the final output level of another indicator (both numerator and

denominator within the equation). In addition, given uncertainties relating to

the employers assisted (or financially supported) indicators, discussed above,

there is a need for care when interpreting the following analysis.

Based on current profiles for all the relevant indicators, there is evidence of

under-profiling for this indicator. Within Competitiveness ESF Priority 2 total

Programme forecasts for employers adopting or improving equality and

diversity strategies run at 13% of profiled employer assists. In Convergence

ESF Priority 3 the figure increases to 20%. Both are well behind the 50%

target.

Current achievement as a percentage of employer assists is very low (2%

Competitiveness, 7% Convergence).

Project level interviews raised serious doubts about achieving current profile:

Considering the available data in conjunction with the evidence collected from

project level interviews there is substantial risk associated with achieving OP

targets for this indicator.

4.2.6. Projects Delivering Specialist Training in Sustainable Development

The OP target for this indicator has a moving target, based on the percentage

of approved projects. Based on current levels of project approvals there are

high levels of over-commitment in both Convergence Priority 3 and

Competitiveness Priority 2 and actual achievement already exceeding OP

target levels. Even if additional projects are approved it is unlikely that activity

levels will drop below OP targets.

Formatted: Indent: Left: 0.11 cm, Hanging: 1.16 cm, Outline numbered + Level: 3 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at: 1.27 cm + Tab after: 0 cm + Indent at: 2.16 cm

4.2.7. Learning and Development Strategies There is no target stated for this indicator within the OP. The expected profile

has changed substantially and to date the indicator is ahead of profile at

175% (138% Competitiveness and 197% Convergence).


Formatted: Indent: Left: 0.11 cm, Outline numbered + Level: 3 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at: 1.27 cm + Tab after: 0 cm + Indent at: 2.16 cm

Project level interviews suggested there is on-going discussion between at

least one project and WEFO over interpretation of this indicator which will

substantially influence final achievement. In the instance cited each business

supported by the project has a detailed training plan put in place. However,

there is a need for clarity about whether these can be classified as ‘learning

and development strategies’.

Formatted: Indent: Left: 0.11 cm, Outline numbered + Level: 3 + Numbering Style: 1, 2, 3, … + Start at: 1 + Alignment: Left + Aligned at: 1.27 cm + Tab after: 0 cm + Indent at: 2.16 cm

4.2.8. Research Studies Whilst not a quantified OP target, there are output data relating to the number

of research studies completed within the thematic area. To date, around 50%

of the profile has been achieved,I, 100% for Competitiveness, where the

current final forecast has been met, and 47% for Convergence, where 31% of

the current final forecast has been met. Some slippage is due to delayed

project commencement. Nevertheless, on the basis of project interviews

there is evidence to suggest this indicator will achieve, or potentially exceed

the forecast. Those projects reporting against this indicator were confident of

achieving forecasts, with progress already in hand. In addition, the Sector

Priorities Fund Pilots project indicated it will exceed the number of research

studies currently profiled.

4.2.9. Progression Schemes for Women (Convergence only)

There is no OP target for this Convergence only indicator, for which only one

project, Women Adding Value to the Economy (WAVE), is forecasting

achievement. In line with profile, there has been no progress to date against

this indicator but ten progression schemes for women are focussed to be

delivered by the end of the programming period.


4.2.10 Participant Mix

Targets for the participation by target groups6 are expressed as percentages

of the total participants indicator. Current completed activity shows a shortfall

in achievement against each target group relative to the OP target. Current

levels of activity with female participants, older participants, participants with a

work limiting health condition or disability, and female participants who work

part time are below target. As noted at section 3.2 there is evidence of a lack

of focus on disabled participants within Convergence ESF Priority 3 and a

lack of focus on older participants and other target groups in Competitiveness

ESF Priority 2 relative to the OP. This is reflected in the output data.

No forecast profiles for these indicators are available on which to assess likely

outturn. On the basis of activity to date there are substantial risks in respect of

achieving the OP targets for these indicators.

4.2.11 Participation and Qualifications by Level The available data show that activity to date has been dominated by Level 2

training and qualifications which are running well ahead of the target

proportion. Level 3 is also well ahead of target. The strong progress of the

Apprenticeships projects in both the Convergence and Competitiveness areas

are the primary driver of this. These projects were funded to deliver high

activity and output volumes in the period August 2007 to August 2011 and are

now complete. As these projects were delivering additional capacity to

existing structures there were no project implementation delays. The

indicators for these projects therefore dominate the landscape of participation

and achievement at this stage in the Programme period, particularly as other

projects are scheduled to run for many more years. Due to the way these

data are calculated (as a percentage of total participation/qualifications

achieved) the over-representation of one or two categories leads directly to

other categories showing an under-representation.

6 Female participants, older participants, participants with a work-limiting health condition or

disability, and female participants who work part time.


Basic Skills is well behind the target mix. The delay in implementing the

Essential Skills in the Workplace project (see case study at section 4.4) is the

primary reason for the low levels of activity and achievement. Level 4 and

above outputs are running a little behind the target mix in Convergence and a

little above in Competitiveness. Project interviews indicated that the longer

nature of some higher level courses, delays in commencement of some

projects and challenges recruiting to higher level learning were all partly

responsible for this mixed performance. However, the statistical anomaly as a

result of the very high rates of activity and achievement at Level 2 and 3 is

also an important factor.

Assessing outturn at the end of the Programme period is very challenging.

No forecast profile data for qualification mix exists. Due to the way these data

are calculated an increase in any one category will result in a decline in

others. It is not therefore possible to assume that current performance ahead

of target will continue. Indeed, with the Apprenticeship based projects

complete the balance between skills levels will undoubtedly adjust through the

remainder of the Programme period.

4.3. Indicator Summary

In summary, for the Increasing Skills thematic area,

� Three Convergence indicators and four Competitiveness indicators are

likely to meet or exceed the OP targets; and

� Two Convergence indicators (Employers assisted or financially supported

and Employers adopting or improving equality and diversity strategies and

monitoring systems) and one Competitiveness indicator (Employers

adopting or improving equality and diversity strategies and monitoring

systems) are likely to fail to meet the OP targets.

In terms of the targets for the particular target groups and levels of

qualification achieved, the Programmes are unlikely to achieve the participant

and qualification mix originally set out in the OP.



In terms of progress against those indicators without OP targets, progress is

strong against Learning and Development Strategies in both Programmes and

acceptable against Research Studies in the Competitiveness Programme.

Participants entering further learning in both Programmes and Research

Studies in the Convergence Programme are behind profile to date.

4.4. Reasons for Under- or Over-Performance

The financial and output/result performance data presented in the preceding

sections show variation in progress across different indicators. However,

whilst there are differences in the scale of variation, for the majority of

indicators, performance is below expected profile to-date, suggesting an

element of slippage. When coupled with the evidence collected from

stakeholder and project level consultations a number of potential reasons for

this emerge.

Delays in commencement of delivery post project approval have affected

projects to varying degrees. For the majority7 of projects there has been at

least some delay that was not fully accounted for within the original business

planning and indicator profiling process. Whilst this has been accounted for

through re-profiling for some projects, others are still in the process of, or are

about to embark on the process of re-profiling. In the main these re-profiling

exercises relate to the timeline for achieving indicators and expenditure,

rather than overall scale. Nevertheless, there are examples of projects (six out

of the sixteen interviewed) that are likely to re-profile in terms of scale of

output achievement. WEFO should continue to monitor achievement against

profile closely to assess whether re-profiled outputs and results continue to be

achievable or whether there will be further slippage leading to under-delivery

of output and result indicators.

Of the 16 projects covered by consultations (including those with mirror projects in both

Convergence and Competitiveness Programmes), only four (Apprenticeship projects and

ProAct, both operating in Convergence and Competitiveness) did not experience delays in

commencement of activity.



7

The reasons for delays vary on a project-by-project basis. The most

commonly cited reason was procurement (six of sixteen projects

interviewed), a finding common with the Evaluation of the Effectiveness of

Programme Implementation8 (WEFO, 2011). This led to particularly long

delays for project sponsors that had little or no previous experience of

procuring services. In the main these are projects with sponsors outside the

Welsh Government. There is also some evidence that the lack of clear

guidance on procurement has led to projects taking quite different approaches

and applying different levels of rigour that may have created further

unnecessary delays for some projects. For example, one project is using the

procurement route for every variation in activity “you cannot do anything

without going out to quote”, whereas a second is adopting a far more flexible

approach to the use of contracts post procurement “we had to make a

decision on how to procure and hope it will be ok at audit”.

The other reason cited by three of sixteen projects related to procedural issues in completing the project approval sign off process with WEFO.

One project noted a substantial delay after approval created as a result of

waiting for “financial sign-off” and stated “there is no over-sight during the

period from approval to project functioning.” The interviewee indicated a

perception that following approval there was a lack of joined-up action within

WEFO to ensure the project was fully able to proceed.

These issues do not give reason for specific rectifying intervention, as for

approved projects the moments of difficulty have passed, although the

impacts through delays to delivery are still being felt. However, there are

lessons to be learned for future programmes and any pipeline projects which

may be approved. In particular these include ensuring appropriate guidance

to ensure project set-up post approval is effective and efficient and to ensure

continued close and active over-sight of projects where approval is still

subject to detailed sign-off.

8 Found at

http://wefo.wales.gov.uk/publications/monitoringevaluation/researchreports/programmeimple

mentation/?lang=en


http://wefo.wales.gov.uk/publications/monitoringevaluation/researchreports/programmeimple

One major example of slippage in delivery against profile identified within the

project level interviews was within the Essential Skills in the Workplace project. This is the flagship Welsh Government sponsored project to tackle

basic skills and is critical to the achievement of OP targets. The project had

an official start date of October 2010 and the first participants were due to be

recorded by the end of 2010. Versions of the project operate in both the

Convergence ESF and Competitiveness ESF Programme areas.

However, there have been delays to project start for a number of reasons.

Firstly, there has been a loss of staff with expertise in the management of

basic skills projects as a result of internal reorganisation and restructuring

within Welsh Government. Secondly, the project has undergone a total

redesign since approval, which has included integrating the project with Rural

IT Wales and a shift to a totally different delivery model for basic skills

training. Whilst this redesign is intended to deliver much improved value-for-

money, it has taken time to put new delivery arrangements in place and

undertake soft market testing. The project manager highlighted that it was

vital that a working operational model was in place before a re-launch of the

project.

Furthermore, the initial profile of indicators was incorrect as a result of a

desire to enter an activity profile into the WEFO system.

A slightly wider issue than the detailed sign-off for projects, is the lengthy process from project idea to project approval, which has been

experienced by four of sixteen interviewed projects. One particular impact of

this lengthy process cited by three of these four projects was the loss of staff

with experience and expertise in the period between the closure of one

funding round and the commencement of a new round. In a number of cases

this led to slower start-up of activity whilst new staff were recruited. The full

extent of the impact upon the quality of delivery as a result of this loss of

expertise cannot be quantified within this evaluation. One project also cited


an experience where they perceived a lack of over-sight and joined-up activity

in WEFO to resolve an issue causing a delay. This was a very similar

perception to that indicated above which occurred post approval.

The over-optimistic profiling at business planning stage was cited by

seven of sixteen projects. The influence of the optimism impacted upon both

the timing of indicators, with little or no allowance for project set-up by several

projects, anticipating delivery from day one post approval, and upon the

overall scale of indicators to be achieved. Six of the seven projects citing this

situation could not explain in detail why such over-optimism had occurred as

the project manager had changed during the lifetime of the project. However,

various reasons were put forward by consultees for the optimism, with the

recession the most common, with forecast profiles prepared in a very different

economic environment. In particular, the recession has impacted in terms of

reduced funds to pay for training and through reduced staffing levels which

limit the ability of businesses to release staff for training. In support of the

finding related to available finance, one project, which does not incorporate a

charge to the business for training, indicated this was now a very positive

factor in recruiting to the project in the wake of the recession.

A further related issue is where there is no directly comparable historic

experience on which to base profile estimates. In some cases, whilst project

sponsors had previously delivered a similar project, the parameters within

their current project had changed e.g. transition from offering training which is

free at the point of delivery to introducing a charge. As a result, forecasting

was estimated. There is no direct rectifying action which can easily be taken

in this instance, but again, there may be cause for more detailed interrogation

of proposed indicator profiles within the project approval process.

A further issue in slippage against expected profile relates to indicator definition and monitoring. Project interviews identified indicator related

issues in respect of Employers assisted, Learning and Development

Strategies and Participants gaining qualifications. In some instances project

sponsors did not understand why particular indicators were present within the

profile for their particular project and as such there were challenges in


delivering profiled outputs. One particular example related to the Employers

assisted indicator, where the project was specifically focused on the individual

participant and not the employer. A further issue with this indicator occurs

where other organisations are to be supported or receive benefit, although not

in relation to their role as employers (for example, other learning received by

the organisation through participation in the project). Due to a lack of an

appropriate indicator to capture this activity it was noted that WEFO requested

the ‘employers assisted’ target be increased as a proxy. This may create

further issues for downstream targets which are based on a percentage of the

employers assisted indicator.

In other instances indicator definitions appear to have been misinterpreted at

business plan stage and now in the delivery phase it is evident that the

profiled indicators are deemed unrealistic by current project managers or are

subject to further clarification between the project sponsor and WEFO. The

outcome of these discussions will impact upon final indicator levels for a

number of projects. The Participants gaining qualifications indicator has an

associated OP target and is of particular importance in this regard. There is

evidence to suggest that one of the interviewed projects included ‘units of

learning’ or ‘modules completed’ as a proxy for qualifications. There are also

examples of projects that had believed certain qualification types to be eligible

to be counted as qualifications, but which have subsequently been deemed

outside the official definition (for example Prince 2). This may lead to a

downward revision against indicators at final outturn. There is no direct

rectifying action which can easily be taken in this instance, but again, there

may be cause for more detailed interrogation of indicator selection and

proposed indicator profiles within the project approval process.

Under-reporting of indicators is also a potential issue for two interviewed

projects. One interviewed project had not completed the relevant claims

procedure at the time of fieldwork and a second was undertaking activity for

which full compliance work was still outstanding and therefore had not been

registered on WEFO systems. As a result, there are some indicators which

may not have been claimed by the cut-off date for analysis within this

evaluation, but have been delivered and will be claimed in due course.


There is also evidence of a lack of ownership and commitment to Cross

Cutting Theme related targets. Project level interviews suggest monitoring of

participant profiles is undertaken, but this is passive i.e. recording the details

of those supported, rather than a proactive pursuit of achieving the participant

profiles indicated within the OP and many project business plans. There is

already evidence of this within the available data on the proportion of

participants from target groups. The project level interviews made clear that

the focus of projects is upon headline output and result indicators.

The purpose of the Participants entering further learning indicator is to

capture progression beyond the initial intervention and track whether

participants are engaging in on-going learning. This relates to strategic

aspirations to institute a culture of lifelong learning within the economy and

among the labour force.

Whilst the nature of this indicator is entirely valid, there are practical issues

relating to the accurate capture of data from a number of projects interviewed

as part of this evaluation. In particular, the concerns relate to the fact that in

many instances participants may have just completed a period of learning

which could have extended to three or four years in length. Whilst they may

well continue in further learning at an appropriate juncture, the participant may

choose to take a break from learning. Typically any monitoring for this

indicator is undertaken at a point three months post completion of the

intervention received. If the participant has not yet entered further learning

they are not counted and are unlikely to be picked up at a later date if they do

enter further learning.

As a result of challenges with accurate data collection five projects intimated

that they are not monitoring this indicator and others believe reported levels of

achievement to potentially be below actual levels of progression.


5. THEMATIC AREA SPECIFIC ANALYSIS

This chapter sets out answers to a range of further evaluation questions

posed within the study brief.

5.1. Project Integration and Collaboration

Stakeholder and project level interviews undertaken as part of this evaluation

have generally identified very positive experiences in terms of collaboration

and joint working within the Increasing Skills thematic area under this round of

Structural Funds Programmes.

The primary feature of collaboration has been stakeholders working together

within projects, particularly through the project formulation and design

process. There is evidence that the move to fewer, larger projects within this

round of Structural Funds Programmes in Wales has, as a minimum, speeded

up the process of collaboration within the skills sector. Examples were cited

of collaboration within Welsh Government departments (DFES), across Welsh

Government departments (DFES and BETS), between Higher Education

Institutions (HEIs), Further Education Institutions (FEIs) and between Higher

and Further Education Institutions as well as with Local Authorities, Sector

Skills Councils, private sector training providers, Job Centre Plus, employers,

third sector bodies and others. The consultations suggest that this has added

substantial value in terms of the design of projects, the value for money that

can be achieved and the contribution to wider policy and strategy agendas.

For example, within the Work Based Learning project eight HEIs are operating

together under a single brand. A further example was cited in reference to the

Modern Apprenticeship projects where NIACE is a core sponsor. It was

indicated that the process of collaboration has not just improved quality for the

ESF elements of the project but in the wider pan-Wales activities as well.

Positive examples of the Specialist European Teams (SETs) were cited by

project sponsors in bringing together projects to ensure duplication was

avoided and synergies maximised. This included examples of regional based

projects being encouraged to share knowledge and best practice at an early



stage. This experience is in line with the evaluation9 of SETs which found

positive experiences in the Increasing Skills10 thematic area.

Some evidence was found of working across Priorities, with linkages between

ESF Convergence Priority 2 and ESF Convergence Priority 3 as well as within

the Priority. These were cases where a single project sponsor was

responsible for projects delivering similar activities11 within the two Priorities

such as a single team within the Department for Education and Skills (DfES)

within the Welsh Government. Combined, these show potential linkages of

projects supporting participants before they enter the workforce to projects for

participants once they enter the workforce and then through the different

levels of learning.

The most prominent example of inter-project collaboration cited by numerous

interviewees is that of the Sector and Provider Advocates. Further details of

this are set out below.

A further example of post-approval collaboration is the merging of two

projects, Essential Skills in the Workplace and IT for Rural Enterprises. Whilst

this has brought about delivery challenges in the short-term, the flexibility to

see where synergies exist and respond is anticipated to ultimately enhance

the quality of project delivery.

One element of the Sector Priorities Fund Pilots (SPFP) project was to fund

sector and provider advocates to improve communication between Sector

Skills Councils (representing employers) and training providers. In particular,

this activity was targeted at enabling SSCs and providers to clearly articulate,

understand and develop effective responses to employer demand for training.

9 Spatial European Teams Evaluation, WEFO, November 2010. Found at

http://wefo.wales.gov.uk/publications/monitoringevaluation/researchreports/setsevaluation/?la

ng=en (accessed 5th December 2011). The SETs were renamed Specialist European Teams

following this evaluation.10 As covered by references to ESF and DfES focused SET activity.11 Two interviewees felt it was sometimes a hindrance to have the activity split by participant

type across ESF Convergence Priorities 1-3, rather than by the nature of activity.


http://wefo.wales.gov.uk/publications/monitoringevaluation/researchreports/setsevaluation/?la

Seven sector advocates were supported, aligned around established SSC

clustering, and four provider advocates were supported, aligned to the four

geographic regions in Wales.

The primary focus of the advocates as outlined within the project business

plan was to support communication in respect of other piloted activities within

the SPFP project. However, the project level interviews found very positive

evidence of the work of these advocates outside the SPFP project with other

activities within Convergence ESF Priority 3, with the advocates mentioned by

numerous project level and stakeholder consultees in a range of organisations

(including Welsh Government, third sector organisations and Higher

Education Institutions). This highlights the added value generated through this

innovation which was possible as a result of ESF funding, including a fit with

the wider agendas for collaboration and transformation.

The advocates have enabled the SSC network and provider network to come

much closer together, enabling better developed projects focused on demand-

led skills development. This included the transfer of training expertise in the

energy sector in Pembrokeshire into the Anglesey Energy Island activity. In

another example:

“Asset Skills identified demand outstripping supply for facilities management

skills. This information has been fed through the advocates into college led

projects, therefore encouraging training providers to get close to the SSC”

Examples were also cited of how the independence of the advocates enabled

them to undertake activities which may be treated with deep suspicion if

driven by Welsh Government. As a result there has been progress made in

identifying duplication of activity and in delivering innovative curriculum

development.



5.2. Transnational Activities

This evaluation has identified limited transnational activity within the

Increasing Skills thematic area. Of the 16 projects covered by consultations

only two expressed clear intention to undertake transnational activity to

exchange knowledge. Neither project had yet commenced work in this area

and indicated some challenges in progressing this including the lack of ring-

fenced funding and process issues which can lead to difficulties identifying

and securing international partners. One project sponsor said:

“You cannot do too much until you receive WEFO approval. WEFO want a

business case with identified partners, but you cannot identify partners if you

don’t know what you are proposing, as you have not yet received approval. It

is difficult to do transnational work at the best of times.”

“Under EQUAL, transnational activity worked well, although it was difficult.

You knew you had to and there was a ring-fenced budget.”

The two projects keen to undertake transnational activity are:

� Minorities are Wales’ Resources; and

� Agile Nation

Both projects are heavily focused in the Equality and Diversity arena, and

both have sponsors operating in the third sector. This may indicate a

difference in suitability to particular topic areas within the Priority, or

potentially a difference in attitude in some organisations.

One other project did indicate that transnational activity had been considered,

but was decided against. This was also a project with a sponsor outside

Welsh Government. They indicated that there “…was no resource available

within WEFO to support transnational activity and no common framework

across countries to make it possible…….it is not the role of Convergence to

do transnational, there is enough to do to get the core Programme working.”

There was no mention of any consideration of transnational activity from the

Welsh Government sponsored projects.


Formatted: Indent: Left: 0 cm, Hanging: 0.75 cm, Bulleted + Level: 1 + Aligned at: 0.63 cm + Tab after: 0 cm + Indent at: 1.27 cm

5.3. Use of Cross Fund Flexibility Tools

Within the Increasing Skills thematic area project data indicate one project


that has expressed an intention to use the cross fund flexibility to create or

refurbish premises. However, this project was not part of the sample

interviewed.

Across the projects consulted as part of this evaluation none indicated any

intention to use the cross fund flexibility tool. In fact, some sponsors were

uncertain as to its role or function. For example, one interviewee referred to

the mix of match and ESF funding within the project.

Overall the evidence suggests the cross fund flexibility tool is not being widely

utilised within this thematic area. The limited level of understanding across

consulted projects would suggest a higher profile needs to be given to the

availability of the cross fund flexibility tools.

5.4. Innovation

Project sponsors are readily able to signpost elements of all projects which

demonstrate innovation, particularly in terms of changes from previous modes

of delivery across the thematic area. As would be expected, in some

instances the level of innovation is substantial, in others incremental.

The level of collaboration taking place within the thematic area has already

been discussed, and is one example of innovation and added value that has

been kick started by this round of Structural Funds Programmes in Wales.

The OP and Strategic Frameworks identify a number of ‘possible areas’ for

specific innovative actions, set out below. The research has identified projects

addressing three of the four areas identified as a focus for innovation, shown

in parenthesis. No specific project activity supporting the retention of older

workers in employment has been identified.

1. Promoting the integration of migrant workers within the workforce (e.g.

Minorities are Wales Resources);

2. Supporting the retention of older workers in employment (no project

identified);



3. Innovative learning delivery approaches and methodologies for identifying

and addressing skills needs (e.g. Sector Priorities Fund Pilots and Work

Based Learning Strategies); and

4. New ways of helping employers and employees adapt to job and

organisation change (e.g. Shaping the Future and ProAct).

The project examples which follow set out further examples of projects which

are delivering innovation.


The Shaping the Future project in North West Wales aims to create

economic opportunity by mitigating the decommissioning of nuclear power

stations at Wylfa and Trawsfynydd. The project supports workers whose

nuclear site employment is at risk as a consequence of decommissioning,

through proactive engagement and providing skills training to supplement

transferable skills, particularly focused at growth sectors. The intention is to

retain the highly skilled workforce in the region, thereby attracting new

employers and avoiding any further substantial loss of economic activity from

the region.

The proactive approach to engagement is totally new in respect of nuclear

decommissioning. Whilst there have been challenges in delivery, the Nuclear

Decommissioning Authority has taken valuable learning from the project and

has worked with partners to establish a similar project for the Dounreay area,

with support from the Shaping the Future project manager.

The Sector Priorities Fund Pilots (SPFP) project has innovation in delivery

at its heart. At the time of conception it was intended that a Sector Priorities

Fund would be established in line with Skills that Work for Wales. This fund

would ensure more responsive delivery and better align post 16 skills

provision to the needs of employers in Wales.

The SPFP project includes four primary activities:

� Testing innovative training solutions

Formatted: Indent: Left: 0 cm, Hanging: 0.63 cm, Bulleted + Level: 1 + Aligned at: 0 cm + Tab after: 0 cm + Indent at: 0.63 cm, Pattern: Clear (Gray-15%)

� Facilitating partnerships between SSCs and providers

� Raising the skill levels of participants in the pilot projects; and

� Supporting the implementation of the Cross Cutting Themes

Within the project 14 pilots are being tested. As with any innovative project,

there is an element of risk. Within SPFP all pilots are either testing brand new

delivery innovations or new innovations for Wales. The range of pilot

activities is varied and includes the establishment of sector and provider

advocates as referenced in an earlier case study.

Through the course of the SPFP project the decision was taken that the

Sector Priorities Fund would no longer be established, but the innovation in

delivery has already led to mainstreaming of new activity in a number of

areas. In particular, two apprenticeship pilot projects are now to be adopted

within mainstream work based learning provision in Wales. For many of the

other pilots it is too early to assess whether activity will be mainstreamed,

however, early indications are that 12 of the 14 pilots have been a success to

some degree.

The ProAct project was established as part of the Welsh Government’s

response to the recession. The project concept emerged from a series of

‘Economic Summits’ held in various locations around Wales in the autumn of

2008. The project specifically enabled employers facing difficulties retaining

staff as a result of the recession to both part-finance their retention and fund

training to enable the business to prepare for the upturn.

The coupling of a wage subsidy and 100% training funding within ProAct was

viewed as highly innovative, with no other similar scheme elsewhere in

England and Wales. The scheme was also designed with substantial

flexibility to be fully responsive to employer needs, although eligibility criteria

were strict in respect of evidencing clear risk of redundancy or short time

working for staff.

The scheme benefited from a swift approval process enabling it to respond to

the challenges presented to employers by the recession at the appropriate


time. A lag in approval would have led to a failure to deliver the intended

benefits (Impact evaluation of ProAct, SRD, 2011).

“It is testament to the skills and hard work of the core and delivery teams that

the rapid introduction of the programme has never been cited as a reason for

delay and the delivery of the programme has itself demonstrated real

innovation in its approach that deserve wider consideration” (SRD, 2011).

It was reported that the project has also delivered added value through the

close working of DCELLS (now DFES) and DE&T (now BETS) as well as

collaboration with other delivery organisations. However, there were

concerns some of this added value has been lost due to the reorganisation of

BETS and the changing roles of individuals.

The positive response of businesses to the training element of the scheme

has led to the creation of a follow on scheme Skills Growth Wales which does

not include the wage subsidy, but allows firms with clearly identifiable growth

prospects to train in priority areas.

5.5. Added Value

The project level interviews confirmed that projects were directly and indirectly

adding value to national policies. As a minimum, supported activities should

be ‘additional’ (i.e. the activity would not take place in the absence of ESF

funding). Whilst some degree of deadweight and displacement may be

expected (and the assessment of this is outside the scope of this evaluation),

at a conceptual level there should be a clear rationale for supporting each

project.

Table 4 classifies each sampled project into one of four types of added value

that have been used as part of this evaluation. This shows a spread across

all four categories.

Within the volume category, these projects are delivering substantial levels of

activity over and above that which would be funded without ESF. They are

not seeking to deliver marginal uplifts in the scale of activity. For example the


Deleted: Table 6


Modern Apprenticeship World Class Skills for a Knowledge Economy is

supporting an additional 18,000 participants, a 54% increase on the 15,000

participants that could have been supported over the same period without

ESF support.

Within the scope category there is a range of projects delivering in different

ways or to groups that would not otherwise be supported. As featured in the

case studies within section 5.4 on innovation, there are projects delivering

new forms of activity that widens participation. In addition, organisations such

as Chwarae Teg have experience of supporting women in a range of settings,

but would not have the resources to undertake any of the activities within the

Agile Nation project without ESF support.

Within the role category there are a number of projects that are delivering

particular innovations that either have or will become mainstreamed as a

result of pilot testing within the ESF project. As indicated within the project

example at section 5.4, some elements of the SPFP project have already

been mainstreamed. Furthermore, consultations indicated four elements

within the Modern Apprenticeship project that have been mainstreamed: a

shared apprenticeship model (allowing one apprentice to be supported by

more than one employer), flexible learning (ensuring a much more tailored

approach to employer needs), an apprenticeship matching service (in

partnership with Careers Wales to match potential apprentices with

employers) and an apprenticeship marketing pilot.

Table 4: Added Value Categorisation Deleted: 6

Type Definition Projects

Volume Project ‘adds’ to existing

action, either by supporting

national action in general

(‘mirroring’) or specific areas of

national policy (‘boosting’).

Modern Apprenticeships, Basic

Skills in the Workplace/IT for

Rural Enterprises, ELMS.


Type Definition Projects

Scope Project ‘broadens’ existing

action by supporting groups or

policy areas that would not

otherwise receive support.

Shaping the Future, SPFP, Work

Based Learning, ELMS, ProAct,

Agile Nation, MAWR, Business

ICT for SME Principals.

Role Project supports local/regional

innovations that are taken up

at national level or national

innovative actions that are

then ‘mainstreamed’.

SPFP, Modern Apprenticeships,

Work Based Learning, ELMS.

Process Project influences Member

States administrations and

organisations involved in the

Programmes.

ProAct, SPFP.

In addition to the core classification of projects there are other examples of

added value which have been achieved as a result of this round of ESF

funding. In particular:


� Procurement – whilst there are mixed views among project sponsors as to

the costs and benefits of procurement, there are examples cited where the

use of procurement within the project has delivered improved outcomes

that would not have occurred without the spur to institute a procurement

based approach. This includes improved value-for-money as well as

better quality inputs, which in turn are anticipated to deliver better quality

outputs. For example, within the Modern Apprenticeship project the

procurement exercise was cited as a primary driver in improving the

quality of provision, with a number of providers that had previously been

contracted failing to secure contracts under the new process. However,

there are other examples where the procurement process may have led to

less favourable outcomes, particularly where established working

relationships were in place. For example, one project, which had existing

relationships with suppliers, tendered for the supply of the same services


leading to higher prices. The interviewee stated that “cheaper rates could

have been negotiated if we had not procured.”

� Collaboration – this round of funding has led to a substantial level of

collaboration that would not otherwise have taken place. This is dealt with

in detail under a separate heading.

5.6. Basic Skills Focus in Competitiveness ESF Priority 2

Has the Priority been sufficiently focused on improving the levels of

basic literacy and numeracy alongside development of other generic

and vocational skills? (Competitiveness Only)

There is limited available evidence on which to base an answer to this

question. Within the OP the focus upon basic literacy and numeracy is

evident in the targets for Participants accessing basic skills training, 55% of

participants, and Participants gaining qualifications - basic skills, 63% of

participants gaining qualifications. These targets reflect the emphasis upon

basic skills within the supporting OP text.

In response, a large scale Welsh Government sponsored project Essential

Skills in the Workplace has been approved. Whilst some other projects are

delivering some basic skills activity, Basic Skills in the Workplace is critical to

achieving the OP aims. As no data are available on forecast profiles by skill

level the Essential Skills in the Workplace project is used a proxy for all basic

skills activity in the following analysis.

Based on OP target levels of participants (12,600) and participants gaining

qualifications (5,450) current approved forecasts for Essential Skills in the

Workplace exceed the notional targets when applying the percentage based

targets for activity and achievement at the basic skills level.

However, this analysis tells only part of the story. As the targets are

percentage based, a more sound analysis would consider the mix of activity

compared with total profiled participants, rather than measuring against the

participant levels within the OP. Essential Skills in the Workplace is profiled to

deliver 26% of all forecast participants and 28% of forecast participants



gaining qualifications. This compares with OP targets of 55% and 63%

respectively for basic skills. On this basis, the mix of approved activity is

lacking the focus on basic skills that is detailed within the OP.

In addition, there have been substantial delivery delays and project

restructuring within the Essential Skills in the Workplace project across both

Convergence and Competitiveness areas with the project being behind profile

to date.

When considering expenditure, around 9% of the Competitiveness ESF

Priority 2 budget has been committed to the Essential Skills in the Workplace

project. This compares to 25% of funding allocated to the two leadership and

management projects, 41% to Skills Growth Wales and 17% to World Class

Apprenticeships in East Wales. Whilst no indicative financial breakdown is

provided in the OP, and training at different levels will require different levels

of financial support, the degree of imbalance in the financial commitment is

broadly in keeping with the under commitment of indicators for basic skills

relative to the OP.

5.7. Delivering Demand Led Qualifications

Have the projects within the Priority responded well to delivering

demand led provision of qualifications at level 3 and beyond?

(Convergence Only)

Six of the sampled projects are relevant to this question, and in addition

stakeholders with a wider remit were asked to comment upon this issue. In

general the evidence suggests that projects have responded well to delivering

demand led provision of qualifications at level 3 and beyond.

There is evidence of substantial consideration of priority areas for demand,

primarily through the use of labour market intelligence (LMI) and engagement

with Sector Skills Councils (SSCs). Indeed, there are projects, including

SPFP, Skills for the Digital Economy and Agile Nation, which are actively

looking to improve the quality of LMI specific to Wales, through research

activities, in order to support the achievement of these aims.



The Sector Advocates which are highlighted in a case study, at section 5.1 of

this report, highlight the importance of linkages between the SSCs/employers

and providers. Whilst the modus operandi of these advocates varies from

region to region, there is anecdotal evidence of positive impact to date in a

number of situations which is improving the quality and nature of provision.

A risk to the remainder of the Programme period is the potential restructuring

of SSCs. Whilst details are not yet confirmed, there is anticipation of

substantial structural changes to the make-up of the SSC arena which could

have an important bearing on provider/employer interaction in the future.

In addition to interaction between providers and SSCs there is further

evidence of much more detailed demand led activity, tailored very specifically

to the business. This is apparent in the design of Modern Apprenticeship

learning frameworks as well as through projects such as Enhancing

Leadership and Management Skills and ProAct, where highly tailored

solutions are provided to individual businesses.

5.8. Balance between Skill Levels

Has the Priority succeeded in achieving the right balance between lower, intermediate and higher level skills?

The clearest measure of the intended balance between skill levels are the OP

targets for participants accessing training and participants achieving

qualifications by the four levels (basic, Level 2, Level 3 and Level 4 and

above). However, the available data does not allow comment on the balance

of committed activity against target.

Partial data on activity to date is presented at section 4.2 of this report. This

indicated the current balance of activity is strongly weighted towards Level 2

and Level 3, with a substantive under performance at the basic skills level

relative to the OP target. This is largely driven by the strong delivery of the

Modern Apprenticeship projects and the substantial implementation delays for

the Essential Skills in the Workplace projects. The issues around basic skills

delivery are also discussed in more detail at the case study at section 4.4 and

section 5.6, which although focused on Competitiveness ESF Priority 2,



identifies underlying messages that consultees indicated are equally true in

Convergence ESF Priority 3.

As many projects are delivering indicators across the differing skill levels it is

not possible to make even a crude analysis on the basis of allocating projects

to a level. However, rather than looking across the four level categories in the

OP, or the three levels within the evaluation question, a simpler analysis can

look at the balance between Convergence ESF Priority 3 Theme 1 and

Theme 2. Theme 1 is largely focused at basic and lower skills and Theme 2

largely at intermediate and higher skills. This demarcation is not absolute, but

may be helpful to consider balance. When looking at committed funds across

these two themes, at section 3.3, it was commented that Theme 2 had

received a greater funding commitment than was identified within the OP,

largely at the expense of Theme 1.

Although the data available are not perfect, the analysis presented above

gives some indication of an imbalance of activity across skills levels to that

intended at the time of drafting the OP, with an under-representation of

activity at the basic/lower level to date.

5.9. Challenging Traditional Gender Dominated Cultures

Has the Priority been sufficiently focused on challenging the

traditionally gender dominated cultures, environments and attitudes

within the workplace e.g. men in teaching and nursing, women in

construction and scientific professions?

Within Convergence ESF Priority 3 there are two themes that mention

activities related to tackling traditional gender roles. Within Theme 2 one of

the listed indicative activities in the OP states “positive action measures which

support women to secure higher level skills in non-traditional areas such as

science and engineering, construction and ICT”. Theme 3 is specifically

targeted at promoting gender equality in employment and includes two related

indicative activities: “support employers, learning providers and others to

promote diversity and challenge traditionally gender dominated cultures,

environments and attitudes e.g. men in teaching and nursing, women in



construction and scientific professions” and “develop actions with Sector Skills

Councils and partners to tackle gender stereotyping in work-based learning,

vocational training and apprenticeship programmes”.

In respect of Theme 2 the review of project documentation and project level

interviews has found no clear evidence of projects taking a proactive role in

tackling gender dominated cultures. A frequent response from interviewees

was that Convergence ESF Priority 3 was dealing with employed persons and

therefore there was little that could be done through the limited training activity

the projects were focused on. This response highlights three issues:

� Projects were focused upon the core activity within their business plans.

As has been shown previously in this report, there is some evidence of a

lack of proactivity towards pursuing targets and activities related to Cross

Cutting Themes and target groups, with the exception of projects

specifically targeted at one of the identified groups.

� Many projects are targeting activity at the individual and felt that they had

limited influence over the business culture. This is also a factor in the risks

around achievement of the employers assisted indicators.

� There is a limited understanding of the issues around gender dominated

cultures. The issue is not restricted to ‘access’ e.g. whether men or

women are able to enter a sector or occupation, but also includes

‘progression’ e.g. movement along a career path towards more senior

roles within a sector or occupation. Within Theme 2 it is the ‘progression’

issue in respect of women that is particularly highlighted and no evidence

was found of any specific activity to tackle this issue. However, due to the

sampling of projects it is not possible to definitively say that no activity of

this nature is taking place.

In respect of Theme 3, a single project, Agile Nation, has been commissioned

which is tackling the full remit of activities identified within the OP.12 As

Since the fieldwork for this evaluation took place, a further project has been approved:

Women Adding Vale to the Economy (WAVE)


12

identified at section 3.2 there is the potential for gaps around specific work

addressing the participation of men in female gender dominated cultures and

environments.

Whilst the Agile Nation project has been approved and activity is underway, it

is still too early to assess the impact that it is having in terms of challenging

traditionally gender dominated cultures, environments and attitudes. This will

be an issue for testing within later stage project and programme level

evaluation.

5.10. Contribution to the European Employment Strategy

Is the Priority contributing to the European Employment Strategy (EES);

Are projects approved consistent with the priorities set out in the EES?

The EES is a mechanism designed to coordinate the employment policies of

the EU member states. It therefore sets a broad framework, within which

national governments are responsible for shaping and implementing detailed

policy. The scope of the EES is much wider than the Priorities within the

Increasing Skills thematic area as it addresses wider labour market issues

such as inclusion and participation which are considered within different

thematic areas of the 2007-13 Structural Funds Programmes in Wales as well

as areas outside the scope of the Structural Funds Programmes, such as

employment law. However, the Priorities within the Increasing Skills thematic

area sit entirely within the context of the EES.

The EES is clearly referenced within the OPs in respect of the Priorities within

the Increasing Skills thematic area. Since the OPs were drafted there has

been some further development of the EES through the adoption of the

Employment Policy Guidelines (2008-2010)13. Of particular relevance to the

Increasing Skills thematic area, the revision to the Employment Guidelines

has strengthened the commitment to policies in the fields of active ageing,

gender equality and lifelong learning.

13 Council Decision 2008/618/EC of 15 July 2008 on guidelines for the employment policies of

the Member States



In addition, there have been various documents setting out country specific

recommendations in respect of implementation of policies. The

recommendations of 14 October 2004 were explicitly referenced within the

OPs, notably in respect of the prevalence of low skills amongst the workforce,

including basic skills and the presence of one of the widest gender pay gaps

within the EU. The recommendations indicated that, amongst other things,

the UK should give immediate priority to improving literacy and numeracy of

the workforce. Further recommendations in 200714 and 200815 continually

identified a need for the UK to focus on improving performance in the field of

skills development, with basic and intermediate skills specifically referenced.

With specific reference to the relevant elements of Guidelines 18-24, the

committed activity within the Increasing Skills thematic area is clearly

contributing to the aims of the EES. This includes activities ranging from Agile

Nation, focusing on female participation and progression in the workforce and

influencing employer attitudes to gender roles, to labour market intelligence

within projects such as the Sector Priorities Fund Pilots, which looks to better

anticipate skills needs and labour market shortages to Shaping the Future

which is actively managing change as a result of economic restructuring, and

Access to Masters and other projects which are improving access to training

at all levels through innovation in delivery and proactive engagement.

From a delivery perspective there are two potential challenges in the context

of the EES, already noted elsewhere in this report.

� The focus on active ageing and engagement of older workers is

substantially behind target in terms of the current participant profile .

There is no specific project targeting this cohort of workers. For other

target groups there are vertical projects with a specific focus e.g. Agile

Nation (women) and Minorities are Wales Resources (BME and migrant

workers).

14 Council of the European Union document number 7456/07. 15 Council of the European Union document number 8276/08



� The progress towards basic skills targets is very slow at this stage in the

Programme as a result of challenges with the Essential Skills in the

Workplace project.

� Lifelong learning is a recurrent theme of the EES. There are identified

challenges with the primary indicator within the OPs for tracking this. On

the basis of the data that are available there is evidence of an under

commitment and poor performance in terms of reported progress. A better

measure may be through the ESF Leavers Survey, which suggests a

higher level of further participation. Looking more widely, the work to

make training more accessible with more flexible delivery methods has the

potential to impact lifelong learning well beyond the Programme period

and the cohort of participants directly supported by ESF activity.

5.11. Engagement of Social Partners

To what extent are social partners (employers and trade unions)

engaged in the Priority? What is the nature of the engagement?

There is some degree of social partner engagement in all of the sampled

projects. Across the Increasing Skills thematic area there are a range of

types of engagement of social partners. The most common are:

� Direct interface with employers in receipt of intervention via projects.

All sampled projects have some degree of direct interface with employers.

� Indirect interface with employers through Sector Skills Councils.

In part the discussion at section 5.7 related to delivery of demand led

provision highlights the involvement of SSCs and employers.

� The presence of employer and/or Trade Union representation within

project steering groups or other similar boards or fora.

Specific examples were identified within the ProAct and Agile Nation

projects. One project sponsor stated:

“The Wales TUC representative was brilliant and very supportive,

attending all the panel meetings. They were able to provide useful views

on businesses and the relationship worked well.”






As highlighted previously, the potential changes to the SSC structure could

create a challenge for the remainder of the programme period given the

number of projects that either are, or are planning to use SSCs as a route to

information or wider employer engagement.


6. CONCLUSIONS AND RECOMMENDATIONS

6.1. Conclusions

The Increasing Skills thematic area comprises Convergence ESF Priority 3

and Competitiveness ESF Priority 2, both titled Improving Skills Levels and

Adaptability of the Workforce. Convergence ESF Priority 3 has an allocation

of £301.1 million split across three Themes. Competitiveness ESF Priority 2

is much smaller with a £28.1 million funding allocation.

To date 35 projects have been approved, 27 under Convergence and eight

under Competitiveness. In absolute terms, there is a good fit between the

scope of projects approved and the policy goals within the Operational

Programme (OP) for Convergence ESF Priority 3. Potential gaps that were

identified relate to disabled participants, supporting women to secure higher

level skills in non-traditional areas, and challenging traditionally female

dominated cultures.

There are more substantive gaps in respect of Competitiveness ESF Priority

2. In particular these include OP ambitions to support focused activities at a

range of target groups including women, older workers and migrant workers.

For both Priorities there is very little duplication of effort. Where these were

identified, specifically relating to activities supporting leadership and

management development, there is evidence that these are effectively

managed at the operational level.

There is some evidence of imbalance between approved project activities and

OP aims when considering the financial commitments made to date, with an

over commitment to Convergence ESF Priority 3, Theme 2 and an under

commitment to Theme 1 focusing on basic and lower skills. Whilst there are

limited data on which to assess the balance of approved activities in terms of

participant mix and skill levels, there is some indication of under commitment

of activity in respect of basic skills which aligns to the financial analysis.

Strengthening of data and monitoring in this regard will be important to track

performance towards OP target for the remainder of the programme period.



Across the two Priorities 91% of EU funds have been committed.

Competitiveness ESF Priority 2 is over-committed at 104%. Across both

Programmes, 32% of total funds have been spent, this equates to 70% of

profiled expenditure to date, indicating some requirement for catch up.

However, at both Programme and project level there is confidence that

remaining funds will be committed and all existing committed funds will be

spent by the end of the Programming period.

Performance against output and result indicators is variable. There is very

strong performance in terms of the number of participants and participants

gaining qualifications, and, coupled with substantial over commitment of

outputs this provides a sound expectation that OP targets will be exceeded.

However, whilst data are limited, there are some concerns in terms of

achieving the target profile of participant type and activity by skill/qualification

level. Although there are no Priority level targets for participants entering

further learning, there is also a likely shortfall in achievement of project level

forecasts for this indicator.

There is uncertainty in respect of indicators relating to employers assisted or

financially supported, despite substantial over-commitment of outputs across

approved projects. It is likely that there will be de-commitments against some

of these outputs. The target for employers adopting or improving equality and

diversity strategies and monitoring systems is unlikely to be achieved on the

basis of current evidence. However, the target for projects delivering

specialist training in sustainable development is likely to be met.

Strong performance on headline participant indicators is largely driven by over

commitment of outputs across approved projects. More mixed performance

against other indicators has a number of causes including delays to project

commencement post approval, over ambitious delivery profiles, issues

relating to indicator definition and possible under reporting in some instances.

Beneath the headline level data are more limited. However, when considering

the participant mix there is some evidence that mainstream projects, without a

focus on a particular target group, are fairly passive in their approach to

engaging with target groups. This is reflected in the very low proportion of


disabled and older participants to date, groups for which no dedicated projects

have been commissioned. When considering activity across skill levels there

is some evidence of imbalance relative to the OP targets. In part this reflects

the success of the Modern Apprenticeship projects and substantial delays

related to the implementation of the Essential Skills in the Workplace project,

which is profiled to make a substantial contribution to OP targets for basic

skills. Notwithstanding, there is evidence of under-commitment of basic skills

activity relative to OP aims which is only exacerbated by delivery challenges.

There is positive evidence in respect of collaboration within and between

projects, in large part driven by the desire to approve fewer, larger projects in

this Programming period. This has been cited as a key positive impact of the

Programme, driving forward collaboration which, whilst desirable, would not

have happened so quickly. There is also evidence of social partner

engagement with projects which closely relates to the collaboration which has

taken place.

There is evidence of innovation within projects. In some cases this is

manifest in the form of incremental change, in others there are whole new

approaches to delivery being tested and in some cases mainstreamed. This

innovation has also supported the aim of supporting demand led provision.

However, there is little transnational activity, or use of cross fund flexibility

tools within the thematic area.

The evidence points to projects delivering added value in a number of ways.

In some cases through adding scale to existing activity, others broadening

scope and others through the creation of innovative new approaches.

Overall approved activity is contributing to and consistent with the European

Employment Strategy, but there are some areas where this contribution could

be strengthened. In particular these relate to supporting older workers,

ensuring the delivery of basic skills activities and strengthening the activity

around female participation and progression in projects outside Convergence

ESF Priority 3 Theme 3.


6.2. Recommendations

On the basis of the evidence found in this evaluation the following

recommendations should be considered by WEFO:


Committing Outstanding Funds


� When committing the remaining funds consideration should be made of

the potential for imbalance within current approved activity in order that

Programme outturn is consistent with OP aims.

Project and Programme Monitoring

� Careful monitoring and management of project outputs over next 12

months is required, particularly as a result of planned reprofiling and

backloading of outputs.

� Careful monitoring of the Essential Skills in the Workplace project to

manage risks related to overall OP ambitions regarding basic skills.

� Improved monitoring of indicators below the headline level, specifically

relating to participant mix and level of activity.

Project Liaison


� Increase awareness of the role mainstream projects can play in proactively

engaging target groups and tackling wider issues such as female

progression in employment through skills development.

� Raising the profile of the availability and purpose of cross fund flexibility

tools to ensure projects are fully aware of the opportunities available to

them.

Future Programmes


� Consider in more detail the potential future need for vertical projects on

specific issues to avoid important groups and Programme aims being lost.

This can include Cross Cutting Themes and Transnational Activity.

� Seek to avoid big delays between projects and slow approval processes in

next round of Programmes to avoid loss of knowledge and expertise from



sponsor organisations and the potential loss of strategic added value of

current round of projects.

� Investigate whether initial indicator profiles are tested hard enough for

optimism bias in terms of both total scale and timing, through the project

approval process.

� Ensure improved support following project approval, particularly for non

Welsh Government project sponsors and on issues that are new to the

funding round (such as procurement in the current Programmes)


APPENDIX 1: CONSULTEE LIST

Project Sponsors:

Naz Malik and Tegwen Malik AWEMA

David Waller and Deborah Lewis WG DFES

Judy Craske Menter Mon

Chris Perrin WG-DFES

Sion Meredith WG-DFES

Andrew Martin WG-DFES

Kate Reed Computeraid

Leanne Jones, Clare Naylor and University of Glamorgan

Martin Jeffries

Nicola Hunter WG-DFES

Hayley Dunne Chwarae Teg

Strategic Framework Co-ordinators:

Phil Tristram WG-DFES

Joanne Snaith WG-DFES


increasing skills thematic evaluation report

Documents