mid-term evaluation of smartcymru final 030314 · diversity profile of participants is skewed...
Post on 24-Aug-2020
0 Views
Preview:
TRANSCRIPT
Sdf
Ymchwil gymdeithasol
Social research
Rhif/Number:
Mid-term Evaluation of the
SMARTCymru RD&I Financial
Support for Business Programme
Mid-term Evaluation of the SMARTCymru RD&I Finance for Business Programme
CM International
N.B. English text in curly brackets for English version only Welsh text in curly brackets for Welsh version only (swap positions with English text)
(Views expressed in this report are those of the researcher and not
necessarily those of the Welsh Government) (Eiddo’r ymchwilydd yw’r farn a
fynegir yn yr adroddiad hwn ac nid o reidrwydd farn Llywodraeth Cymru).
For further information please contact / I gael gwybodaeth bellach cysyllter â;
Joanne Coates
Social Research and Information Division
Welsh Government / Llywodraeth Cymru
Cathays Park
Cardiff
CF10 3NQ
Tel/Ffôn: 02920 825540
Email/Ebost: Joanne.Coates@wales.gsi.gov.uk
Welsh Government Social Research, 2014 / Ymchwil Gymdeithasol
Llywodraeth Cymru
ISBN
© Crown Copyright / Hawlfraint y Goron
Table of contents
Glossary of acronyms 2
Acknowledgements 3
List of tables 4
List of figures 5
Summary 6
1 Introduction 11
2 Programme logic model 15
3 The programme context 22
4 The programme profile 30
5 Outputs and impacts 41
6 Value for money 59
7 Conclusions and recommendations 67
Annex 78
1
Glossary of acronyms
A4B Academic Expertise for Business
BETS Business, Enterprise, Technology and Science - the former Welsh
Government Department, currently known as Economy, Science and
Transport
BIP Business Innovation Programme
ED Experimental Development
ERDF European Regional Development Fund
ERP Economic Renewal Plan
EST The Welsh Government department for Economy, Science and
Transport
EU European Union
FP7 Seventh Framework Programme
FTE Full Time Equivalent
GDP Gross Domestic Product
GVA Gross Value Added
HEI Higher Education Institution
ICT Information and Communication Technologies
IP Intellectual Property
IR Industrial Research
R&D Research and Development
RD&I Research, Development and Innovation
SME Small and medium sized enterprise
TCF Technical and Commercial Feasibility
TSB Technology Strategy Board
WEFO Welsh European Programme Office
WG Welsh Government
Acknowledgements
This report has been written by Dr Dylan Henderson (project manager and
lead researcher), Meirion Thomas (project director) and Ioan Teifi
(researcher) at CM International.
Important contributions have also been made by the Welsh Government’s
Innovation team.
List of tables
Table 1. SMARTCymru intervention rates ...................................................18
Table 2. SMARTCymru target indicators .....................................................19
Table 3. SMARTCymru programme funding ...............................................20
Table 4. Workplace based GVA at current basic prices ..............................23
Table 5. Business expenditure on R&D.......................................................24
Table 6. Mid-term programme cost – Convergence ....................................37
Table 7. Mid-term programme cost – Competitiveness ...............................39
Table 8. Convergence output indicators ......................................................42
Table 9. Competitiveness output indicators.................................................42
Table 10. How important is the outcome of the project for the company?..44
Table 11. To what extent were the project objectives met?........................45
Table 12. Prospect of commercialisation....................................................45
Table 13. Protection of IP...........................................................................46
Table 14. Satisfaction with Welsh Government's support in developing
projects.......................................................................................47
Table 15. Action taken in projects had SMARTCymru funding not been
available .....................................................................................48
Table 16. Other sources of funding sought had SMARTCymru not been
available .....................................................................................49
Table 17. New products/process/services as a result of SMARTCymru ....50
Table 18. Have you invested more in R&D as a result of the project? .......51
Table 19. Extent to which SMARTCymru encouraged/improved capability to
apply for competitive R&D funds in companies ..........................51
Table 20. Maintenance of linkages and collaborative relationships post-
SMARTCymru ............................................................................52
Table 21. What impact has the SMARTCymru programme had on your
company's attitude to collaboration ............................................53
Table 22. Comparative deadweight assessments......................................55
List of figures
Figure 1. Mid-term evaluation work stages................................................13
Figure 2. SMARTCymru programme logic model......................................21
Figure 3. SMARTCymru programme applications .....................................25
Figure 4. Geographic location of projects – Convergence area.................30
Figure 5. Geographic location of projects – Competitiveness area ...........31
Figure 6. Participant company turnover – Convergence area ...................32
Figure 7. Participant company turnover – Competitiveness area ..............33
Figure 8. Employees – Convergence area ................................................33
Figure 9. Employees – Competitiveness area ...........................................34
Figure 10. Sectors – Convergence area......................................................34
Figure 11. Sectors – Competitiveness area.................................................35
Figure 12. RD&I Phase – Convergence area ..............................................36
Figure 13. RD&I Phase – Competitiveness area .........................................36
Summary
1. This report sets out findings from the Mid-term evaluation of the
SMARTCymru Research Development and Innovation (RD&I) financial
support for business programme, and provides an analysis of
achievements to date, and recommendations for its remaining period of
finding.
2. The evaluation research was undertaken in the period June to October
2013 by CM International, and is based on a management and delivery
workshop and interviews, analysis of monitoring data, and interviews with
a sample of businesses.
Programme model
3. The SMARTCymru programme offers all-Wales support for businesses at
different stages of the RD&I process, including Technical and Commercial
Feasibility (TCF), Industrial Research (IR), Experimental Development
(ED), and Exploitation. It represents the Welsh Government’s core support
for business RD&I, and operates alongside other Welsh Government
programmes such as the Business Innovation programme and the
Academic Expertise for Business (A4B) programme.
4. The results of this evaluation suggest that the programme has been
established with a sound programme logic model, based on an identified
need, clear objectives, grant activities focused on the different RD&I
phases, and anticipated targets both informed by early programme
experience, and aligned to the objectives. Clear links are also embedded
in the model, to maximise the potential for follow-on support, access to
academic expertise and so on.
5. The programme is intended to contribute towards the cross-cutting
themes, through the referral to sources of expertise in the all Wales
Regional Centre Services (now Business Wales Centres).
Programme context
6. The findings from the mid-term evaluation suggest the SMARTCymru
programme has faced significant challenges resulting from the economic,
policy and organisational context. The economic downturn, and associated
decline in business R&D expenditure contributed to lower than expected
demand for the programme. This was compounded by the changes
brought about by the implementation of the Economic Renewal
Programme (ERP), notably the decision to move towards repayable
finance in place of grants and the reorganisation of the then BETS
department into sector teams. Together these factors hindered the
Innovation team from effectively promoting the programme, and resulted in
confusion, both internally and externally, about the availability and offer of
the programme.
7. While the confusion surrounding the programme’s availability and offer has
subsided with the decision to revert back to grant funding, (re)adopting the
SMART branding, and relocating the Innovation Specialists (back) to
within the Innovation team, the challenge for the programme continues to
be one of stimulating demand in the remaining part of the programme. The
enhancement of the links to potential referral programmes and
organisations indicate one of the ways in which this challenge has been
addressed.
Programme profile
8. At the mid-term stage SMARTCymru has supported 116 projects (84 in the
Convergence area, and 32 in the Competitiveness area). These projects
are typically with SMEs, with a number of key priority sectors represented,
including sectors such as the Advanced Manufacturing and Materials
sector, the ICT Sector, Life Sciences, and Financial Services. All are
consistent with the SMARTCymru quality criteria – defined by a robust
technical and financial due diligence processes.
9. With regard to one key element of the cross cutting themes, anecdotal
evidence provided by the Innovation team suggests that the equality and
diversity profile of participants is skewed towards ‘white, middle aged,
male’ participants.
10.Programme costs are below the forecasts at the mid-term. This can largely
be explained by the difficulties faced in adapting to the policy and
organisational changes associated with the ERP process, on one hand,
and the economic downturn/recovery on the other.
11.Programme costs are also below what was expected at the start of the
project, however, the results of the evaluation highlight the important
challenge of encouraging more companies to progress to the later RD&I
phases.
Programme outputs and impacts
12.The outputs and results collected for the SMARTCyrmu programme at the
mid-term state suggests that the programme is some way behind the
original forecasted profile targets. Results for Convergence area exceed
those of the Competitiveness area, with 85 per cent of the profiled targets
for gross jobs created and 61 per cent of the investment induced profile
target (to date) achieved. The number of (Convergence) enterprises
financially supported is, however, some way below the profiled targets,
with only 27 per cent of the delivery profile target achieved to date. It
should be noted, however, that the profiled targets reflect assumptions
made at the start of the process, and do not take account of the decline in
applications brought about by the economic downturn and the ERP
process.
13.Programme impacts at the mid-term stage are both economic- and
innovation-related. Importantly, based on findings from the business
interviews (10), these impacts have been achieved with a low level of
deadweight (30 per cent). They also suggest that Welsh Government is
funding projects that are strategically important to the companies.
14. Innovation impacts, at the mid-term, are limited by the early stage nature
of the RD&I projects. The business interviews do, however, point to the
majority of companies both meeting the technical objectives, and
anticipating good commercial potential for the new/improved, products,
processes and services. Companies are also investing more in R&D,
improving their capability to secure additional funding from public support
programmes. While the potential for innovation spillovers is also evident,
these are likely to take time to fully develop.
15.The economic impact results suggest that the programme results to date
will contribute 29 net additional jobs and a cumulative net total of £4.5m
GVA based on the estimated persistence of benefits achieved at the mid-
term (£1.7 actual)1. It should be noted, however, that the main aim of the
See Annex for more details on these net additional economic impact calculations. 1
programme has not been to create jobs but to increase R&D activity, and
develop new products, processes and services.
16.This performance equates to estimated return on investment, based on
cumulative net additional impacts, of:
£1 SMARTCymru total programme spending produces £0.44
GVA to the Welsh economy
£1 SMARTCymru Welsh Government spending produces £5.62
GVA to the Welsh Economy
SMARTCymru programme cost per project of £87,800
17.The Return on investment (based on total programme spending) is
consistent with comparator programmes such as the Invest NI Start
scheme, although the results of an earlier evaluation of SMARTCymru
suggest that this impact is likely to be greater at the final evaluation point.
Recommendations
18.There are a number of areas where the management and delivery of the
programme could be strengthened in the remaining phase of the project:
a. Recommendation 1: Welsh Government should continue to review
longer term innovation impacts through its Innovation Impacts
programme, complementing the regular SMARTCymru programme
evaluation.
b. Recommendation 2: Welsh Government should prioritise demand
stimulation measures, including supporting companies to move on
to the later RD&I phases in a timely manner.
c. Recommendation 3: Welsh Government should seek to ensure
businesses and stakeholders are aware that SMARTCymru is
available and open to applications.
d. Recommendation 4: Welsh Government should consider the
options available for SMARTCymru to revise the final and profiled
performance targets, with a focus on prioritising projects in the ED
and Exploitation phases.
e. Recommendation 5: Welsh Government should seek to ensure
delivery efficiency is maximised through appropriate integration
across the SMARTCymru management and delivery processes.
f. Recommendation 6: Welsh Government should review its support
for innovation projects where ICT plays an important enabling role.
g. Recommendation 7: Welsh Government should consider
refocusing funding for the Exploitation phase.
h. Recommendation 8: Welsh Government should explore the
potential to strengthen synergies and knowledge exchange with key
stakeholders such as Finance Wales.
i. Recommendation 9: Welsh Government should establish robust
procedures to support companies in the area of the cross-cutting
themes and the ongoing monitoring of results.
1 Introduction
1.1 The SMARTCymru Research, Development and Innovation (RD&I)
Financial Support for Business Programme (the programme) was
launched2 in the Convergence region of Wales in 2010, and
subsequently rolled out in the Competitiveness region in 2011. It offers
all-Wales support for businesses at different stages of the RD&I process,
including Technical and Commercial Feasibility (TCF), Industrial
Research (IR), Experimental Development (ED), and Exploitation. It
represents the Welsh Government’s core support for RD&I, and
operates alongside other Welsh Government programmes such as the
Business Innovation programme and the Academic Expertise for
Business (A4B) programme.
1.2 The programme was initially managed as part of the Single Investment
Fund, but is now managed by the Economy, Science and Technology
(EST) Innovation team, with the support of Welsh Government
Innovation Specialists providing all Wales coverage.
1.3 This report sets out evaluation findings for both the Convergence and
Competitiveness programmes at the mid-term stage (June, 2013).
Aims of the evaluation
1.4 The focus of this evaluation is to understand the mid-term performance
of the SMARTCymru programme against both its objectives and
specified targets. It is intended that the results of the evaluation will feed
into the future direction of the programme in the remaining phase.
1.5 The objectives of the evaluation, taken from the Specification, are to:
To provide a mid-term evaluation report for the SMARTCymru, RD&I
programme that assesses the programme in relation to its progress
towards the criteria outlined by the Welsh European Funding Office
(WEFO) by considering:
o The progress towards the project’s long-term aim, objectives
and targets.
2 It was originally known as the RD&I Financial Support for Business Programme.
o How the project is contributing to the cross-cutting themes
(equal opportunities and environmental sustainability).
o The need and demand for the grants provided through the
project.
o The quality of businesses supported and created by the project,
including growth potential, ability to create jobs and
sustainability.
o The integration with other provision for businesses, including the
New Business Start-Up Support and JEREMIE projects;
o Any evidence of ‘behavioural additionality’ as defined in the
business plan.
o The appropriateness of the project indicators and related
targets, including an assessment of potential achievements after
the Project End Date.
o The impact of grants provided through the project.
o The extent to which the project is contributing to structural
change and sustained impact beyond the funding period
including the move from a grant to an investment culture.
o Progress towards an exit strategy.
Source: adapted from Welsh Government, n.d.
1.6 The evaluation objectives also include a number of ‘considerations’ set
out within the programme’s Grant Award Letter from WEFO.
What has been done by the programme and how has this been
understood by those involved?
How has the programme been delivered and what outcomes have
been achieved? This also includes an assessment of the monitoring
data collection processes and monitoring processes for softer
outcomes.
What barriers and constraints has the programme faced? This includes
consideration of the internal and external influences on the programme
during its initial stages (e.g. policy, organisational and staffing changes
within the Department, recessional effects) and analyse their effect on
achieving targets and outputs.
To consider what are participants gaining from the programme and
how is this different from what they would have done anyway? This
includes consideration as to how businesses are exploiting and
commercialising the technologies financially supported through this
programme.
To identify examples of best practice and lessons learned, with
evidence based recommendations for taking the programme forward.
Source: adapted from Welsh Government, n.d.
Research methodology
The mid-term evaluation is based around five work stages and builds largely
on monitoring data collected by the Innovation team at the Welsh
Government, alongside a small number of interviews with businesses and a
workshop with management and delivery staff. A summary of the approach
adopted can be found in figure 1 below detailing the focus and scale of the
fieldwork undertaken for the evaluation.
Further details of the approach adopted can be found in Annex I, including
details of the approach and impact methodology adopted.
Figure 1. Mid-term evaluation work stages
1.7 The mid-term evaluation research was undertaken in the period June to
October 2013.
Structure of the report
1.8 The findings in this report are set out as follows: section 2 provides an
overview of the SMARTCyrmu programme logic model and its
underpinning assumptions (2), followed by a profile of activity at the mid-
term based on activity and characteristics of the beneficiaries (3). The
report then considers the policy and operational context of the
programme (4). The main focus of the report, however, is found in
sections 5 and 6, where mid-term impacts of the SMARTCymru
programme and its value for money are considered. The report
concludes by examining the key mid-term evaluation findings, including
key issues and recommendations for future delivery (7).
2 Programme logic model
2.1 The following sections outline key features of the programme logic
model including its aims, rationale, activities, management and targets
(output, outcome and impact).
Programme aims
2.2 The SMARTCymru aims for both the Convergence and Competitiveness
programmes are shared. They include:
Create high quality R&D related jobs.
Increase business expenditure on R&D through the provision of
financial support to undertake innovative research and technological
development with commercial potential, leading to new products,
processes and technologies.
Encourage and support industry collaborations with other partners with
research-based organisations in carrying out industrial research and
experimental development activities.
Measure the return on investment of supported R&D projects and
provide continuous improvement of the project management of R&D in
companies supported.
Enable business links to other business support to optimise
commercialisation.
Sources: (Welsh European Funding Office, 2012a; Welsh European Funding Office,
2012b)
Programme rationale
2.3 The economic rationale for the SMARTCymru programme is set out in
the related Business Plans prepared for the Convergence (Welsh
European Funding Office, 2012a) and Competitiveness (Welsh
European Funding Office, 2012b) programmes. These documents
highlight the role of business R&D in producing new product, processes
and services, and as a driver of economic development. Wales,
however, has long underperformed in business expenditure on R&D
statistics, accounting (at the time of the Convergence business plan
analysis 2010) for some two per cent of total UK business R&D
expenditure. This figure, it is noted, is some way below what might be
expected given Wales’ population share in the UK (5 %). Providing
financial support for business R&D, it was argued, would further help to
address UK targets for business R&D to reach a total of three per cent of
Gross Domestic Product (GDP).
2.4 The Community Innovation Survey (Eurostat) provides data on the
innovation performance of European member states. This points to key
barriers to innovation in the UK, including the lack of finance, new ideas
and time to innovate, many of which are likely to be shared by Wales.
2.5 The strategic rationale for SMARTCymru is also highlighted in the
SMARTCymru Business Plans (Welsh European Funding Office; 2012a,
Welsh European Funding Office, 2012b). This points to SMARTCymru’s
role in supporting the ‘Innovation and R&D Strategic Framework’ (Welsh
Assembly Government, 2008) objective to ‘increase investment in
commercially driven R&D, focussed on the commercialisation of
knowledge and Intellectual Property’. It will also seek to contribute
towards the strategic objective to ‘support knowledge based/technology
companies, within Wales’, and ‘encourag(e) more to move to Wales’.
The encouragement provided by the programme for collaborative
engagement with higher education researchers further suggests there is
potential to contribute towards the second objective of the Innovation
and R&D framework to ‘maximise the economic impact of academia and
business through technology transfer and the creation of a stronger
science, engineering and technology base with clear commercial
potential’.
2.6 The programme also contributes to Wales’ (then) primary economic
development strategy - the Economic Renewal Plan (ERP) (Welsh
Government, 2010). This document stated that:
‘… Wales must move towards a more R&D intensive and
knowledge-based economy where the right conditions exist for
innovation to flourish.’
2.7 The lack of a large public or private R&D base is cited by the ERP as an
important barrier to improving Wales’ performance in the UK business
R&D ‘league table’. This highlights the importance of encouraging SMEs
to both recognise the importance of R&D, and helping to build their
capability to conduct R&D. These are challenges that the SMARTCymru
programme seeks to address directly.
Programme activities
2.8 The SMARTCymru programme provides match funding to innovative
businesses to ‘develop new, technologically innovative products and
processes with commercial potential’ (Welsh European Funding Office,
2012a). Financial support available according to the stage that the
company has reached in its innovation process. This includes:
TCF (Technical and Commercial Feasibility) study – funding for
SMEs to investigate and assess technological and commercial viability
of new, innovative ideas for products, processes or technologies.
IR (Industrial Research) – funding to support research to acquire new
knowledge, which will facilitate the development of new products,
processes or technologies, or significant improvement of existing ones.
ED (Experimental Development) - funding to implement the results of
industrial research for the development of new products, processes or
technologies, including the creation of results such as a pre-production
prototype, conceptual formulation, product design, and initial
demonstration of pilot projects.
Exploitation – funding to assist with the exploitation costs of a new
product or process (developed in an earlier SMARTCymru funded
phase), including marketing, publicity, advertising, publication of sales
literature, trade fairs and product certification.
Source: (Welsh Government, 2013)
2.9 The published intervention rates for both small, medium and large
enterprises are summarised in the table below:
Table 1. SMARTCymru intervention rates
Small
enterprise
Medium
enterprise
Large
enterprise
Funding
limits
Technical &
Commercial Feasibility
Up to 75% Up to 75% Up to 65% £15,000
Industrial Research Up to 70% Up to 60% Up to 50% £100,000
Experimental
Development
Up to 45% Up to 35% Up to 25% £200,000
Exploitation Up to 50% Up to 50% Up to 50% £20,000
Source: (Welsh Government, 2013)
Programme outputs and impact targets
2.10 The SMARTCymru Business Plans identify the following intended
outputs for SMARTCymru (Welsh European Funding Office, 2012a;
(Welsh European Funding Office, 2012b):
job creation
increase in Gross Value Added (GVA) and turnover
increased business expenditure on R&D
introduction of new products, processes and technologies
industry collaborations with other industrial partners and with
research based organisations in carrying out the industrial research
and experimental development activities
links to other business support to optimise commercialisation of new
products and processes
2.11 Alongside these core output areas, the Business Plans further recognise
the potential for longer-term outcome benefits to emerge, including
encouraging businesses to go on to conduct and attract public and
private sources of funding for innovation, and positive behavioural
changes to R&D (so-called ‘behavioural additionality’), and commercial
value.
2.12 Anticipated impacts from the programme are expressed primarily in
economic terms, and include:
(increase) in company level GVA
change (Increase) in turnover of enterprises supported
number of jobs created
sales of products, processes or service supported by this Fund
jobs safeguarded
exploitation of IP (Intellectual Property)
2.13 Finally, the programme, in line with the ERDF Operational Programme
(Welsh European Funding Office, 2010), is also intended to contribute
towards the overall targets for cross-cutting themes - environmental
sustainability (20% of companies to adopt or implement environmental
action plans), and equal opportunities (50% of employers adopting or
improving Equality & Diversity strategies and monitoring systems). In
achieving these objectives the SMARTCymru business plans indicate
that the programme will support companies by referring them to obtain
advice from the All-Wales Regional Support Centre (now Business
Wales centres). The programme’s outputs were also anticipated to
include environmental impacts through the introduction of new products
and processes.
2.14 Monitoring of the SMARTCymru programme is undertaken through a
suite of indicator targets. This includes both final targets, and profiled
forecasts covering the full period of the programme. The final
Convergence and Competitiveness programme targets are set out in
table 2 below.
Table 2. SMARTCymru target indicators
Indicator Convergence target Competitiveness target
Enterprises financially supported (number)
200 125
Gross jobs created (FTE) 120 75
Investment induced (GBP) £800,000 £500,000
Collaborative R&D (number) 24 15
New or improved products, processes or services launched
80 45
Products, processes or services registered
80 45
Source: WEFO projects database (Welsh European Funding Office, n.d. a; Welsh European
Funding Office, n.d. b)
2.15 The Innovation team are understood to be in discussion with WEFO to
reprofile the Convergence area targets identified in table 2, to take
account of the challenges it has faced in implementing the
SMARTCyrmu programme (see section 3 for more details of these
challenges). The proposal is for the Convergence target for enterprises
financially supported to be reduced from 200 to 120.
Programme funding
2.16 As noted in the introduction, the SMARTCymru programme is funded
through the EU Convergence and Competitiveness ERDF programmes,
with additional public funding provided by the Welsh Government, and
match funding from the private sector beneficiaries. The financial profile
intended for the programme is outlined in table 3 below:
Table 3. SMARTCymru programme funding
Convergence Competitiveness
ERDF £8,000,000 £4,980,000
Welsh Government £671,000 £303,558
Private sector match funding £9,478,157 £6,195,237
Gross spend £18,149,954 £11,478,795
Sources: (Welsh European Funding Office, 2012a; Welsh European Funding Office, 2012b)
Programme management
2.17 The SMARTCymru programmes is managed by a dedicated team within
Welsh Government’s Innovation team. It is managed by a senior director
based in the Welsh Government Penllergaer office, with overall
responsibility for the programme. The SMARTCymru team includes
technical appraisal and finance and monitoring staff.
2.18 In addition to the dedicated SMARTCymru team members the Welsh
Government’s Innovation Specialists (funded by the Business Innovation
programme) play an important role in the SMARTCymru process. They
work closely with the company to understand their innovation needs, and
support the application process. This may be for an application to
SMARTCymru, or to other support mechanisms. The role of the
Innovation Specialist is particularly important for SMEs, where there is
limited experience or capacity to plan, develop, assess and fund R&D
projects. Here, the Innovation Specialist is able to help businesses in
developing the scope of a project and determining the most appropriate
support route. They are then able to provide application support and
ongoing assistance.
2.19 Once companies submit an application to the programme a technical
appraisal is conducted by the SMARTCymru team, supported by
monitoring and due diligence staff, to ensure financial due diligence.
Final approval is made by the senior programme manager, and a formal
offer letter is sent.
2.20 When a project is underway the Monitoring team will maintain contact
with the SME in order to monitor and quantify project benefits. The
Innovation Specialists will also seek to maintain contact with the
business, with a particular focus on assisting with any future innovation
support requirements (for example, a subsequent phase of
SMARTCymru R&D).
Summary
2.21 The preceding section sets out the key elements of the SMARTCymru
programme, and its underpinning rationale. This is summarised in an
integrated (Convergence and Competitiveness) project logic model
diagram shown in figure 2 below:
Figure 2. SMARTCymru programme logic model
Inputs Activities Outputs Outcomes Impacts
ExperimentalDevelopment (ED)
Technical & Commercial Feasibility Study (TCF)
Exploitation
Convergence (gross)- £18.1 million
Competitiveness (gross)- £11.5 million
Industrial Research (IR)
New Product, processes or services
registered
Innovation impact
Economic impact
Social & environmental impact
New collaborative R&D projects
Gross jobs created
New or improvedproducts, processes or
services launched
R&D / Investmentinduced
R&D behavioural additionality
Context & Rationale
New knowledge toenable
development/improvements of products,
processes or technologies
Investigation of technical and commercial viabilityof new products, processes ortechnologies
New prototypes, conceptformulation, design ofproducts, processes,
services, initial demonstration
Marketing, publicity, advertising, sales
literature, trade fairs and product certification
Need Policy alignment
• Contributing to the Economic Renewal Plan, the Science Strategy, and Innovation Wales,
• Contributing to the objectives of the Innovation, R&D and Technology Strategic Framework
• RD&I recognised as a key driver of business growth • Comparatively low levels of business expenditure on R&D (BERD) in
Wales • Identified barriers to RD&I activity in businesses - lack of finance,
lack of new ideas, lack of time, and difficulties in appraising risk • Key barriers recognised to be strongest in SME sector
2.22 This model demonstrates a logical flow from an identified need, a set of
delivery activities and intended outputs, through to the achievement of
outcomes and impacts in both the Convergence and Competitiveness
areas. The main potential weakness in the logic model relates to the
support for social and environmental cross-cutting themes/benefits,
where the model is reliant on building strong synergies between
specialist support measures for the cross-cutting themes.
3 The programme context
3.1 The implementation of the SMARTCymru model has faced a number of
contextual challenges since its introduction in 2010/11. This has
included challenges associated with the economic recession, and
significant management issues linked to the introduction of the ERP by
the Welsh Government in 2010 (Welsh Government, 2010).
The economic and business innovation context
3.2 In relation to the economy, SMARTCymru was implemented during a
period of economic recession, followed by slow recovery. Regional GVA
is the UK’s preferred measure of economic value at the regional level.
On a ‘per head’ basis estimates produced by National Statistics (see
table 4 below) suggest that GVA in Wales, in 2012, was 72.3 percent of
the UK average, and the lowest amongst the devolved countries and
English regions. Like the other parts of the UK, Wales also suffered from
the global economic slowdown in the early part of the programme’s
introduction. This saw Wales’ GVA per head fall between 2008 and
2009, recovering in 2010 and 2011.
Table 4. Workplace based GVA at current basic prices
GVA per head (£)2
2007 2008 2009 2010 2011 2012
United Kingdom 20,643 21,026 20,472 21,023 21,368 21,295
North East 15,569 15,694 15,294 15,723 15,842 16,091
North West 17,206 17,408 16,973 17,532 17,754 18,438
Yorkshire and The Humber 16,922 16,999 16,458 16,862 17,037 17,556
East Midlands 17,820 17,887 17,355 17,832 18,083 17,488
West Midlands 17,125 17,190 16,691 17,218 17,486 17,429
East of England 19,390 19,338 18,579 19,025 19,355 19,658
London 33,744 35,046 34,910 35,422 35,638 37,232
South East 21,636 21,877 21,266 21,898 22,369 23,221
South West 18,400 18,632 18,244 18,798 19,093 19,023
England 20,681 20,992 20,531 21,054 21,349 21,937
Wales 15,097 15,179 14,794 15,407 15,696 15,401
Scotland 19,653 20,124 19,789 20,314 20,571 20,013
Northern Ireland 16,093 16,117 15,635 16,203 16,531 16,127
Source: (Office for National Statistics, 2013)
3.3 Business R&D expenditure data provides further evidence on the
context facing the SMARTCymru programme. Here the latest National
Statistics data shows that business expenditure on R&D in Wales is now
one of the lowest amongst in the UK regions and devolved
administrations. Table 5 below, for example, indicates that business
R&D expenditure in Wales was flat in the 2008-2009 period, before
declining in 2010. Despite recovery in 2011 and 2012, Wales’ business
R&D expenditure continues to lag behind other regions and devolved
administrations as a proportion of regional GVA -1.17 per cent, against
the UK average of 2.04 per cent.
Table 5. Business expenditure on R&D
Business Expenditure
on R&D (£ million)
2008 2009 2010 2011 2012
United Kingdom 15814 15532 16053 17408 17107
North East 318 315 308 259 282
North West 2130 1926 2074 2260 1784
Yorkshire and the
Humber
433 454 488 543 603
East Midlands 976 992 1137 1149 1203
West Midlands 886 847 886 1237 1461
East 4182 3812 3851 3638 3449
London 1109 907 877 1142 1477
South East 3466 3758 3798 4528 4086
South West 1345 1349 1454 1358 1364
England 14847 14360 14873 16113 15708
Wales 243 243 234 255 272
Scotland 554 631 622 689 707
Northern Ireland 171 297 324 352 420
Source: (Office for National Statistics, 2013)
The policy and organisational context
3.4 The policy and organisational context for SMARTCymru has undergone
a significant period of change in the period since its launch. Much of this
turbulence was associated with the publication and implementation of
the ERP in 2010. This document set out the Welsh Government’s
strategy for economic development, and identified its priorities for future
years. Within the ERP, ‘Encouraging Innovation’ was identified as an
important priority, with the objective of promoting the importance of
innovation (R&D, product and service innovation) for business.
3.5 While SMARTCymru is consistent with the objectives of the ERP, other
changes linked to the implementation of the new strategy had significant
implications for the programme, most notably the requirement for all
business finance (including grants) to be repayable. The implication of
this policy change was that the SMARTCymru programme was no
longer able to offer support in the anticipated format (grant). This change
resulted in a period of internal review to determine whether a repayable
grant was possible under the terms of the ERDF funding scheme, which
continued until the 2011-2012 period.
3.6 The ERP also established a number of priority sectors (six, subsequently
increased to nine). To maximise the focus on the needs of these sectors
the Welsh Government department for Business, Enterprise, Science
and Technology (BETS) was reorganised into sector teams. While the
Innovation team was positioned outside the sector teams, the Innovation
Specialists were reassigned to the sector teams.
3.7 The implications of these changes were significant. The confusion and
difficulties associated with the transition to repayable finance, for both
businesses and Welsh Government staff, resulted in the short term
growth of applications under the old grant regulations (as companies
sought to begin their projects before repayable finance was introduced),
before a substantial decline after the change was implemented in Q3
2010-11 (see figure 3). Alongside this, the reorganisation of the
Innovation Specialists meant that direct line management responsibility
moved from the Welsh Government Innovation team, to the Sector
teams. In practice, this led to the focus of the Innovation Specialists
moving from the regional to a sector focus (irrespective of geography).
The Innovation Specialist interviews conducted for this evaluation
revealed that regional stakeholder contacts were damaged in the
process.
Figure 3. SMARTCymru programme applications
Convergence project start date
Competitiveness project start date
0
5
10
15
20
25
30
35
40
45
50
Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1
2009-10 2010-11 2011-12 2012-13 2013-14
RD&I Applications
Source: (Welsh Government, n.d.)
3.8 A further implication of the move towards sector teams was the transfer
of promotional responsibility to the sector teams. This hindered the
ability of the Innovation team in promoting the programme directly,
without the sector team, and is likely to have contributed to the decline in
applications in the subsequent period.
3.9 Programme branding was reviewed at the same time as the introduction
of ERP. In this respect the programme had been launched as part of the
Single Investment Fund. This, however, replaced the well-established
brand of SMART, which had been used for a number of years, and was
shared by the UK scheme operated by TSB. Interviews with the
SMARTCymru staff indicate that the lack of a recognisable brand further
contributed to confusion surrounding availability of the programme, and
the subsequent decline of applications.
3.10 The election of a new Welsh Government in 2011 marked a turning point
for the ERP inspired changes, with the subsequent announcement by
the new Minister that grants would no longer need to be ‘repayable’.
While this allowed the Innovation team to ‘re-open’ the programme to
applications, ongoing confusion within the business sector, and lack of
regional Innovation Specialist capacity meant that take-up was not as
swift as might be expected.
3.11 To address these weaknesses the programme was re-branded as
SMARTCymru (RD&I Financial Support for Business) in order to raise
awareness of its availability. The line management responsibility for the
Innovation Specialists, was also switched back to the Innovation team.
3.12 While these changes have helped to encourage greater take up,
interviews with a number of Innovation Specialists suggests that the
removal of regional focus to their work during the ERP period damaged
regional linkages and network relationships. The implication of these
factors is that levels of demand for SMARTCymru continue to be lower
than they were before the ERP process. Indeed, the Innovation team
point to ongoing limitations in awareness of SMARTCymru’s availability
in Wales (companies and potential referral bodies such as universities).
The presence of other sources of grant funding such as the Welsh
Government’s own Economic Growth Fund, and other ERDF–funded
projects such as ASTUTE (ASTUTE Wales), may have also taken some
demand away from SMARTCymru, although there is no firm evidence of
this in practice.
3.13 To counter the ongoing weakness in demand the programme team
have, in recent months, sought to raise awareness in the universities,
with the objective of encouraging referrals and greater engagement in
SMARTCymru projects.
Synergies with other programmes
3.14 The operational context facing the SMARTCymru programme includes a
range of innovation and technology programmes focusing on R&D
subsidy and support. This includes a range of Welsh Government
programmes.
3.15 The Business Innovation Programme (BIP) (Welsh Government,
2012) has direct links to SMARTCymru, via its funding of the Innovation
Specialists. These Specialists, as noted earlier, provide advice and
support to the application process, and ongoing support for the
company, as part of their wider role within the Business Innovation
Programme. The BIP also provides access to Innovation Vouchers for
university projects.
3.16 In addition to the BIP, SMARTCymru is also aligned to the A4B
programme (Welsh Government, 2012). While academics are not able
to initiate a SMARTCymru project, or be a formal partner, there is
potential for them to participate in company-led projects. In this respect
there have been attempts to strengthen the referral potential between
the two programmes.
3.17 Outside of the Welsh Government’s Innovation team, Finance Wales
(Finance Wales) is an important stakeholder for SMARTCymru. Finance
Wales have made investments in a number of companies with earlier
SMARTCymru funding (11 in total). This illustrates the synergy between
SMARTCymru’s project focus, and Finance Wales’ equity funding
(typically, a £150,000 to £500,000 first time investment) to support the
development of high growth potential businesses. Finance Wales
indicate that a SMARTCymru project can help to ‘give confidence’ with
respect to their investments, particularly where the company’s
turnover/profit growth predictions are not rapid enough to be attractive to
mainstream funders. A number of potential areas for strengthening the
link between SMARTCymru and Finance Wales’ funding were also
noted.
3.18 Other programmes providing support for R&D in Wales include the
university-led ASTUTE project. This project supports technological
development, sustainability and growth in businesses in the aerospace,
automotive and high technology sectors (in West Wales and the
Valleys). Comprising a partnership of Wales’ universities in the
Convergence area, ASTUTE provides support to manufacturing
companies to create sustainable, higher value goods and services and
bring them to a global market. This support is provided on a 50 per cent
contribution basis, although under the European Commission ‘di-
minimis’ (Welsh Government) arrangements this does not necessarily
require a direct cash contribution for small levels of assistance (under
€200,000 over a three year period). Given the potential to offer financial
support for relatively small projects in the Convergence area, there is
some potential for ASTUTE to overlap with SMARTCymru. This,
however, is likely to be greater with respect to other programmes such
as the Innovation Voucher scheme, or A4B, which have a more
research-industry focus.
Summary
3.19 The findings from this section suggest that the SMARTCymru
programme has faced significant challenges resulting from the
economic, policy and organisational context. The economic downturn,
and associated decline in business R&D expenditure contributed to
lower than expected demand for the programme. This was compounded
by the changes brought about by the implementation of the ERP, notably
the decision to move towards repayable finance in place of grants and
the reorganisation of the then BETS department into sector teams.
Together these factors prevented the Innovation team from effectively
promoting the programme, and resulted in confusion, both internally and
externally, about the availability and offer of the programme.
3.20 While the confusion surrounding the programme’s availability and offer
has subsided, with the decision to revert back to grant funding, (re)using
the SMART branding, and relocating the Innovation Specialists (back) to
within the Innovation team, the challenge for the programme continues
to be one of stimulating demand in the remaining part of the programme.
The links to potential referral programmes and organisations indicate
one of the ways in which this challenge has been addressed.
4 The programme profile
4.1 The Welsh Government’s monitoring data provides an overview of
programme participation, and highlights the nature of demand at the
mid-term stage. The following section sets out the mid-term profile for
both the Convergence and Competitiveness elements, to the end of
June, 2013, based on SMARTCymru monitoring data (Welsh
Government, n.d.).
Profile of company participation
4.2 At the mid-term point there were 84 (53 complete and 31 in progress)
projects in the Convergence area, and 32 (15 complete and 17 in
progress) in the Competitiveness area. This equates to an overall total of
116 projects at the mid-term point.
Geographic location
4.3 Figures 4 and 5, below, illustrate that the 116 projects are spread across
Wales. The most prominent location for SMARTCymru projects in the
Convergence area (figure 4), by some distance, is the Swansea area
with more than double (23) the number of any other Convergence area,
and accounting for over 27 per cent of the total. A number of other
Convergence locations had between five and 11 projects, including
Bridgend (11), Denbighshire (10), Torfaen (eight), Rhondda Cynon Taf
(seven) and Gwynedd (six).
Figure 4. Geographic location of projects – Convergence area
4.4 In the competitiveness area, as might be expected, SMARTCymru
projects were primarily located in the Cardiff area (see figure 5). Indeed,
at the mid-term point half (16 of 32) of the Competitiveness projects
were based in Cardiff. This was followed by Monmouthshire and Powys
(four each). Three or less projects were located in the remaining areas.
Figure 5. Geographic location of projects – Competitiveness area
Equality and diversity
4.5 Equality and diversity data for the SMARTCymru programme is held by
the Business Innovation Programme (which manages the Innovation
Specialists), and was not available at the time of the mid-term
evaluation. Discussions with the manager of the Business Innovation
Programme, however, suggest that the vast majority of applicants to the
Business Innovation programme fall within a narrow equality and
diversity profile characterised as ‘white, middle aged, males’.
Company size
4.6 In the Convergence area figure 6 below indicates that the majority of
company participants, at the mid-term stage, are relatively small,
reporting turnover of less than £1 million (67 per cent of projects). Within
this category the largest single group of projects are companies with a
turnover of less than £100,000, accounting for 23 per cent of projects.
Figure 6. Participant company turnover – Convergence area
4.7 Figure 7, in contrast, reports on turnover in Competitiveness area
participants. This points to the comparatively larger size of companies in
Competitiveness area projects, with a quarter reporting a turnover
greater than £10 million. This figure is consistent with the relative
prosperity of the Competitiveness area. The turnover data does,
however, also point to strong participation in smaller companies, with the
majority of projects (nearly 67 per cent of projects or 16 of 24) reporting
less than £1 million turnover.
Figure 7. Participant company turnover – Competitiveness area
4.8 The employment profile of the SMARTCyrmu projects provides a further
indication of company size. Here, figure 8 suggests that some 67 per
cent of Convergence company projects, employ between 1 and 10
employees.
Figure 8. Employees – Convergence area
4.9 Similarly the largest proportion of Competitiveness area participants
employ 1-10 employees (68 per cent of projects). However, three
Competitiveness area projects involved companies with more than 250
employees (figure 9).
Figure 9. Employees – Competitiveness area
4.10 Taken together the employment and turnover data confirms that the
majority of companies participating in SMARTCymru can be classified as
SMEs3, with only three projects falling into the large sized firm category.
Industry sectors
4.11 The profile of participation is further illustrated by the industry sectors of
SMARTCymru companies. Figure 10, for example, suggests that the
Advanced Manufacturing and Materials sector accounts for the majority
of projects in the Convergence area. This is closely followed by ICT
sector projects (28 per cent). Financial and Professional Services and
Life Sciences are also represented (29 per cent), alongside two
companies from Energy and the Environment.
Figure 10. Sectors – Convergence area
3 According to European Commission definitions (European Commission)
4.12 The importance of the Advanced Manufacturing and Materials and the
ICT sector is also evident in Competitiveness area projects (66 per
cent). Life sciences projects are also strongly represented in the
Competitiveness area projects (34 per cent). In overall terms, however,
a smaller range of sectors are represented in the Competitiveness area
projects.
Figure 11. Sectors – Competitiveness area
RD&I Phase
4.13 The SMARTCymru programme funds projects according to RD&I
phases, with companies expected to make their way through the phases
in a sequential manner. At the mid-term stage, for example, it could be
anticipated that the balance of participation is towards the earlier RD&I
phases such as TCF and IR. Figure 12 confirms that this is the case,
with 70 per cent of projects in the TCF and IR phase. A smaller
proportion (30 per cent) are currently in the Experimental Development
Phase (25 per cent) and Exploitation phase (5 per cent). This profile, will
continue to evolve throughout the remaining period of the programme.
Figure 12. RD&I Phase – Convergence area
4.14 Figure 13 provides phase data for the Competitiveness area. Here a
larger proportion of projects fall within the early phases (TCF and IR
account for 31 per cent and 35 per cent respectively). This, however,
can be explained by the later start date for the SMARTCymru
Competitiveness programme. Only 6 per cent of projects have
reached/completed the Exploitation phase.
Figure 13. RD&I Phase – Competitiveness area
Programme spend
4.15 The Convergence programme costs at the mid-term stage suggest that
31 per cent of the budget has been used, with income received at a
similar level. As table 6 shows, both expenditure and income figures are
lower than their profiled/forecasted targets - 69 per cent and 68 per cent
respectively. Each expenditure category for the Convergence
programme is lower than forecasted with administration and legal and
professional fees at 17 per cent and 24 per cent of their forecasts
respectively. Only 68 per cent of the forecasted grants has been spent
while the only expenditure close to its forecast is staff costs at 97 per
cent.
4.16 In terms of income, private match funding is well below its forecast at 64
per cent, while 92 per cent of the Welsh Government funding forecast
has been received. This reflects the higher than expected number of
projects in the early SMARTCymru phases (TCF and IR projects), where
there are higher intervention rates. This pattern of projects has resulted
in the Welsh Government funding being higher (and vice versa for
private match funding) than would have been the case had more
projects made their way to the later SMARTCyrmu phases.
Table 6. Mid-term programme cost – Convergence area
Convergence
80504 EXPENDITURE TO JUNE 2013
Expenditure Type
Cumulative
Expenditure To Date
Eligible (GBP)
Delivery Profile
Cumulative Expenditure
To Date (Eligible GBP)
% of Delivery Profile
Cumulative Expenditure
To Date spent
Total Forecast
Project Cost
Eligible (GBP)
Forecasted budget
remaining
% of budget
used
Administration 105£ 624£ 16.83% £ 1,308 £ 1,203 8.03%
Grants 5,160,819£ 7,617,361£ 67.75% £ 17,385,528 £ 12,224,709 29.68%
Legal & Professional 16,637£ 68,122£ 24.42% £ 131,252 £ 114,615 12.68%
Staff 376,366£ 389,838£ 96.54% £ 539,359 £ 162,993 69.78%
Travel & Transport 8,017£ 11,593£ 69.15% £ 18,718 £ 10,701 42.83%
Total 5,561,945£ 8,087,538£ 68.77% £ 18,076,165 £ 12,514,220 30.77%
80504 INCOME TO JUNE 2013
Organisation
Cumulative Income To
Date
Delivery Profile
Cumulative Income To
Date (Eligible GBP)
% of Delivery Profile
Cumulative Income To
Date received
Total Forecast
Project Income
Total Forecast
Project Income
remaining
% of Total
Forecast Project
Income received Various Private Match Funders 2,502,222£ £ 3,907,978 64.03% £ 9,422,915 6,920,693£ 26.55% WAG The Economic Development
Group within DEandT 604,034£ 660,029£ 91.52% £ 653,259 49,225£ 92.46%
Total 3,106,256£ 4,568,007£ 68.00% 10,076,174£ 6,969,918£ 30.83%
Source: (Welsh Government, n.d.)
4.17 For the Competitiveness programme, table 7 shows that activity and
expenditures have been far lower than forecasted, even when compared
to the Convergence programme. The total percentage of the forecasted
budget used and income received is close to only 8.5 per cent of the
profiled target. Only 22 per cent of the forecasted grants expenditure at
the mid-term stage has been used, and every other expenditure
category is between 9 per cent and 26 per cent of the forecast. In
contrast to this and similar to Convergence, staff expenses have
exceeded its forecast by 8 per cent; with £99,024 spent compared to the
forecast of £91,456.
4.18 Income received from Competitiveness projects to date is only 24 per
cent of its forecast to June 2013 with income received from private
match funders as low as 17 per cent of that profiled. Income received
from Welsh Government, however, is slightly more than had been
forecast with £195,070 received compared to the forecast of £194,537.
This, again, is a result of the higher number of projects than expected
that engaged in the early phases where there are higher intervention
rates.
4.19 Taken together these figures highlight the contextual challenges faced
by the programme in generating and maintaining demand, and are
consistent with numbers of companies supported.
Table 7. Mid-term programme cost – Competitiveness area
Competitiveness
80504 EXPENDITURE TO JUNE 2013
Expenditure Type
Cumulative
Expenditure To
Date (Eligible GBP)
Delivery Profile
Cumulative
Expenditure To Date
% of Delivery Profile
Cumulative Expenditure
To Date spent
Total Forecast
Project Cost
(Eligible GBP)
Forecasted
budget
remaining
% of budget used
Administration £ 37 £ 432 8.56% £ 888 £ 851 4.17%
Grants £ 855,718 £ 3,865,475 22.14% £ 11,115,267 £ 10,259,549 7.70%
Legal & Professional £ 12,360 £ 47,650 25.94% £ 101,584 £ 89,224 12.17%
Staff £ 99,024 £ 91,456 108.28% £ 191,140 £ 92,116 51.81%
Travel & Transport £ 1,240 11,593 £ 10.70% £ 9,228 £ 7,988 13.44%
Total 968,380£ 8,118,155 £ 11.93% £ 11,418,107 £ 10,449,727 8.48%
80504 INCOME TO JUNE 2013
Organisation
Cumulative
Income To Date
(Eligible GBP)
Delivery Profile
Cumulative Income
To Date
% of Delivery Profile
Cumulative Income To
Date received
Total Forecast
Project Income
(Eligible GBP)
Total Forecast
Project Income
remaining
% of Total
Forecast Project
Income received
Various Private Match Funders 353,185£ 2,075,475 £ 17.02% 6,176,612£ £5,823,427 5.72%
WAG The Economic Development
Group within DEandT 195,070£ 194,537 £ 100.27% 261,495£ £66,425 74.60%
Total 548,255£ 2,270,012 £ 24.15% 6,438,107£ £5,889,852 8.52%
Source: (Welsh Government, n.d.)
Summary
4.20 The results of this section suggest that SMARTCymru programme
(Convergence and Competitiveness areas) participants, at the mid-term,
have been largely micro and small companies, according to the
European Commission definitions. This is consistent with the target
group for the project (SMEs). The participants are primarily located in
Wales’ major urban areas on the M4 corridor, with Cardiff and Swansea
accounting for a large proportion of company participants (36 per cent).
Sector participation is consistent with the Welsh Government’s sector
priorities, with the Advanced Manufacturing and Materials and ICT
sectors accounting for the largest proportion of projects (74 per cent in
the Convergence and Competitiveness areas). Finally, the mid-term
profile suggests that the most projects are in the earlier RD&I phases of
their projects (68 per cent are in TCF and IR phases while only 5 per
cent reached/completed the Exploitation phase).
4.21 Programme costs are below the forecasted targets at the mid-term. This
can largely be explained by the difficulties faced in adapting to the policy
and organisational changes associated with the ERP process, on one
hand, and the economic downturn/recovery on the other.
4.22 Programme costs are below what was expected at the start of the
project, with 31 per cent of the Convergence budget used, against just
over 8 per cent of the Competitiveness budget. Within this, expenditure
on grants and private sector match funding have also been significantly
below target in both the Convergence and Competitiveness areas. Of
particular concern, however, is the comparatively high levels of Welsh
Government match funding. In the Convergence area, for example, the
level of Welsh Government match funding is consistent with the profiled
target (i.e. 100 per cent), in contrast to the private match contribution
which stands at 17 per cent. This imbalance between Welsh
Government and private sector match funding is linked to the higher
than expected number of projects in the early phases of the programme,
where intervention rates are higher (meaning that Welsh Government
match funding is higher). While the overall private and Welsh
Government match funding income is likely to balance out to some
degree as companies move through the programme (to the later phases
with lower intervention rates), there remains a risk that Welsh
Government match funding will be exhausted. This highlights the key
challenge for the programme in encouraging companies to progress to
Experimental Development RD&I phase.
5 Outputs and impacts
5.1 This section of the report summarises the key impacts arising from a
small sample of SMARTCymru projects; contrasting these with the
outputs indicators collected by the programme team. The 10 companies
interviewed for the evaluation accounted for 17 projects (equivalent to 15
per cent of the total number of projects based on those completing one
or more stage). Seven out of the 10 interviewees were based in the
Convergence Region.
SMARTCymru output indicators
5.2 Tables 8 and 9, below, illustrate the primary outputs and results
collected for the SMARTCyrmu programme against the mid-term
business plan targets. Based on the column ‘Cumulative claims to 2013
Q2’4 this data suggests that the programme is some way behind the
original forecasted profile targets. Results for Convergence exceed
those of the Competitiveness area, with 85 per cent of the profiled
targets for gross jobs created and 61 per cent of the investment induced
profile target (to date) achieved. The number of (Convergence)
enterprises financially supported is, however, some way below the
profiled targets, with only 27 per cent of the delivery profile target
achieved to date). It should be noted, however, that the profiled targets
reflect assumptions made at the start of the process, and do not take
account of the decline in applications at reflects the decline in
applications brought about by the ERP process.
5.3 As for other indicators, there are results reported for collaborative R&D
projects (10), products, processes or services registered (10), and new
or improved products, processes or services launched (39), but delivery
profile target to date figures weren’t provided for these.
5.4 As for the Competitiveness area, generally the achievement of profile
target to date of indicators is far lower. The only indicator with a
significant proportion of profile target achieved is collaborative R&D
4 SMARTCymru monitoring data is collected during the financial claims process. The
‘Cumulative Claims to 2013 Q2’ refers to all results and outputs data collected by the end of June 2013. This data is presented in its unverified format, and may change once the SMARTCymru monitoring process is complete.
projects, with two thirds (4 of 6) of its profile target to date achieved. No
other indicator has reached half the profile target to date, with new or
improved products, processes or services launched, next in line at 23
per cent. The achievements for Gross jobs created and enterprises
financially supported are relatively weak, with only 13 per cent and 14
per cent of profile target to date achieved respectively. It should be
noted, however, that the Competitiveness programme projects have had
less time to build up such outputs.
Table 8. Convergence output indicators
Indicator Indicator Type
Cumulative claims to 2013 Q2
Delivery Profile Target To Date
Achievement of profile target to date (%)
Project Final Target
Remaining to Final Target
% Current Final Target Achieved to date
Enterprises Financially Supported
Output 25 92 27% 200 -175 13%
Gross Jobs Created
Result 29 34 85% 120 -91 24%
Investment Induced Total
Result £229,404 £374,149 61% £800,000 £-570,596 29%
Collaborative R&D Projects
Output 10 n/a n/a 24 -14 42%
Products Processes or services registered
Result 10 n/a n/a 80 -70 13%
New or improved products, processes or services launched
Result 39 n/a n/a 80 -41 49%
Source: (Welsh Government, n.d.)
Table 9. Competitiveness output indicators
Indicator Indicator Type
Cumulative claims to 2013 Q2
Delivery Profile Target To Date
Achieveme nt of profile target to date (%)
Project Final Target
Remaining to Final Target
% Current Final Target Achieved to date
Enterprises Financially Supported
Output 7 49 14% 125 -118 6%
Gross Jobs Created
Result 2 15 13% 75 -73 3%
Investment Induced Total
Result £34,721 £160,000 22% £500,000 £-465,279 7%
Collaborative R&D Projects
Output 4 6 67% 15 -11 27%
Products Processes or services registered
Result 2 13 15% 45 -43 4%
New or improved products, processes or services launched
Result 3 13 23% 45 -42 7%
Source: (Welsh Government, n.d.)
5.5 Interpretation of the output indicators is complicated by the process that
has been adopted for their collection. This relies on the company
participants to provide such data alongside their financial claims. Given
that some companies have yet to complete a financial claim, it is likely
that the figures underestimate the actual achievements at the mid-term.
It should further be noted that the collection and processing relies on
Welsh Government’s own staff resources. Interpretation is also
complicated by the long-term nature of the innovation process, and the
likelihood that outputs can take up to four years to deliver and build,
according to estimates (PricewaterhouseCoopers, 2009).
5.6 Indicators for cross-cutting theme outputs are collected by the Innovation
Specialists, and recorded against the Business Innovation Programme.
Difficulties in sharing this information across the programmes do,
however, exist (linked to the anonymity of certain datasets), and suggest
an area where further investigation will be required for the final
evaluation.
Business interview results
Baseline R&D activity
5.7 Prior to engaging with SMARTCymru the majority of companies were
active in conducting R&D (six out of 10 interviewed). The remaining
companies reported no previous activity, but this can largely be
expected due to small size of the participants. A number of companies
were created explicitly as part of the SMARTCymru process (three out of
10).
5.8 Of the six companies that reported prior R&D activity, four out of six had
a dedicated R&D budget, and five out of six had dedicated R&D
employees. The average R&D budget for the four companies who
reported one was £195,000 and the average dedicated number of R&D
employees for the five companies who had one was four.
5.9 These results suggest that a high proportion of interviewed companies
were ‘R&D experienced’, with dedicated R&D budgets and employees.
However, the results also indicate that companies’ R&D activities were
undertaken internally with limited experience of other external R&D
support. Indeed, only one out of ten cited previous public sector R&D
support and only two out of 10 citing previous engagement in
collaborative R&D projects.
Project significance
5.10 The interview results further suggest that projects were strategically
significant to the company, with six of the 10 companies indicating that
they probably would not have survived had it not been for the
programme, with a further two pointing to the significance of the project
for future turnover (see table 10). As noted earlier, three companies
were created explicitly as part of the SMARTCymru process. Two more
reported that without SMARTCymru support they wouldn’t have
recorded any turnover or would have struggled for a period before going
out of business. All companies indicated that their project was significant
to its future prospects. These results underpin the importance of
SMARTCymru support in developing or in addressing important strategic
company needs.
Table 10. How important is the outcome of the project for the company?
Frequency % The outcome is crucially important to the company, without which we probably wouldn’t have survived 6 60 The project outcome is very important, it will account for a significant proportion of turnover in the near future 2 20 The project outcome is important to the firm but we have a number of other products or services in the pipeline 2 20 The outcome of this project is of modest importance to the company 0 0 The outcome of this project is not important to the company 0 0
Project achievements
5.11 The findings from the interviews suggest that project objectives have
been met by a moderate to very large extent (see table 11 below), with
the majority of these falling into the large / very large extent categories
(nine out of 10). One company reported that the project met and
exceeded its objectives to date; ‘we are further ahead than we thought’
while another reported that the project was a ‘great success with all
objectives met’. Indeed, this company had managed to introduce its
products into some of the largest financial services companies in the UK.
Table 11. To what extent were the project objectives met?
Frequency %
Very large extent 5 50
Large extent 4 40
Moderate extent 1 10
Small extent 0 0
No extent 0 0
N/A 0 0
5.12 This positive picture of the programme is further highlighted in table 12
below where all of the companies indicated strong prospects for
commercialisation. Indeed nine out of 10 companies noted that their
project had the potential to bring some / significant benefits. The
remaining company also expected to receive further benefit but it was
too early to say. One of the interviewees, for example, reported that his
company had already managed to commercialise the results of an
earlier SMARTCyrmu project, and were currently working on a second
project to add to their offer. The tendency for optimism bias (HM
Treasury, 2013) in such assessments should, however, be noted.
Table 12. Prospect of commercialisation
Frequency %
Significant benefits/potential 8 80
Some benefits/potential 1 10 Company only exploitation 0 0 Expected but too early 1 10
No possibility 0 0
N/A 0 0
Protection of results captured
5.13 Results of the SMARTCymru projects were, in many cases, protected
through formal mechanisms including patents, copy rights, trade marks
etc. Table 13, below, suggests that six of 10 companies had some form
of protection (ranging from limited to complete) for their project outputs.
A number of the companies indicated that formal protection mechanisms
were not necessary or available because the
products/processes/services developed were entirely based on know-
how and therefore not protectable (for example, patent searches
revealed they were free to operate in this area). In this respect one
company estimated that it would take up to one year to copy the
company's product. Another company indicated that their strategy for
protection was to keep innovating the service in order to make it difficult
for competitors to copy or to provide similar quality. This respondent
reported that the Welsh Government can be too ready to prioritise
patents, even in cases where they were not perceived to be relevant
(e.g. ICT service projects).
Table 13. Protection of IP
Frequency %
Completely protected 3 30 Limited protection 3 30 No protection 4 40
Satisfaction with SMARTCymru
5.14 The results of the interviews suggest a positive picture in relation to
satisfaction with the support provided by SMARTCymru (see table 14).
On a scale of one to five, where one equated highly unsatisfied and five
highly satisfied, the average response from the ten companies was four
point three. In fact, four out of 10 companies were very satisfied and half
(five of 10) were satisfied with Welsh Government's support in
developing their project, with the remaining company moderately
satisfied.
Table 14. Satisfaction with Welsh Government's support in developing projects
Frequency %
Very satisfied 4 40
Satisfied 5 50
Moderately satisfied 1 10
Unsatisfied 0 0
Very unsatisfied 0 0
5.15 The application process was described as being ‘long-winded’ by a
number respondents with the Welsh Government team felt to lack an
understanding of the ‘commercial urgency’ associated with innovation
projects. Companies would often have tight timetables for launch and
any delays were said to be potentially ‘catastrophic.’ One company
interviewed claimed it was close to being wound up as a consequence of
this process.
5.16 One of the larger, more established companies, suggested that the
process should become ‘more streamlined’. The company felt that
undertaking a TCF was unnecessary because the basic research had
been completed internally5. The general consensus was that the process
should become more flexible with the exploitation and commercialisation
phase introduced earlier if appropriate.
5.17 Another company was very critical of the exploitation phase and had
decided not to press ahead with it after having initial discussions with the
SMARTCymru team. The issue here was that the company viewed a
marketing approach focussed on ‘selling the company’ as the best route
ahead rather than focussing solely on marketing the product itself. The
company claimed to have had backing from three independent Welsh
PR companies on this strategy but was rejected by SMARTCymru for
being ‘too broad’.
Impacts analysis
It should be noted, however, that companies are not required to conduct at TCF, if they can already prove that they have undertaken such activity prior to their SMARTCymru application.
5
5.18 The following section seeks to outline the benefits of the SMARTCymru
projects to Wales at the mid-term stage. The analysis takes into account
both quantitative and qualitative analyses to provide an overall picture of
the impact of the SMARTCymru programme.
Project additionality
5.19 The interviewed companies were asked about the extent to which
projects would have progressed without SMARTCymru funding. Here
the results suggest that none of the projects would have gone ahead
unchanged in terms of timescales, scale or location, had it not been for
SMARTCymru funding. This represents significant additionality in
SMARTCymru funded projects, and demonstrates the relative
importance placed on the funding programme by the companies
interviewed.
5.20 In examining the responses to this question in more detail, three of 10
respondents argued that their project would not have taken place at all,
had it not been for SMARTCymru funding. This assessment was not
altogether hypothetical for one of the companies interviewed, with the
respondent reporting that they did abandon their project once the
funding basis changed to a repayable grant. The project did, however,
continue when the Welsh Government later contacted the company to
offer grant funding terms. This reflects the higher degree of uncertainty
and risk associated with industrial research projects (contrasted with
potential higher returns), in comparison with near market R&D.
5.21 For projects that would have gone ahead, the perception of half (five of
10) the respondents was that without the SMARTCymru funding the
projects would have taken longer to complete. Other respondents
predicted that their project would have gone ahead on a reduced scale
(one of 10) or at a later date (one of 10). The project outcomes therefore
would have been delayed and the benefits potentially reduced for the
vast majority (seven of 10) of projects (see table 15).
Table 15. Action taken in projects had SMARTCymru funding not been
available
Frequency % Gone ahead as now unchanged – that is the same scale, time 0 0
and location
Gone ahead at a later date (a delay in project) – on a same scale 1 10 Gone ahead but would have taken longer to complete 5 50 Gone ahead but on a reduced scale 1 10 Gone ahead but lower quality 0 0 Been abandoned 3 30
5.22 Finally, had SMARTCymru funding not been available companies
indicated that other funding sources may have been considered (see
table 16), including bank loan (23 per cent), equity finance (15 per cent),
own capital (15 per cent), other public sources (8 per cent), and bank
overdraft (15 per cent). Three companies, however, reported that they
would not have gone ahead with the project, had funding been
unavailable. For many of the companies who suggested that they would
have looked for other options, they didn’t consider any to to be
particularly viable in reality. One of the larger companies interviewed
stated that it would have been very difficult to make a case for funding
internally, and if SMARTCymru assistance hadn't been available, the
team would have had to consider whether it was possible to proceed; ‘it
would have been touch and go’. This was consistent with answers
reported to the earlier question.
Table 16. Other sources of funding sought had SMARTCymru not been
available
Frequency %
Bank loan 3 23 Bank overdraft 2 15
Equity finance 2 15
Other public sector organisation 1 8
Own capital 2 15 Joint venture 0 0 None (project would not have gone ahead) 3 23
Other 0 0
Innovation impacts
5.23 A key indicator of benefit to a region in supporting RD&I programmes is
the development and launch of new/improved products; processes;
and services by companies participating in the programme. This factor
represent key objectives of funding R&D.
5.24 Table 17 below shows, the interviewed companies produced 10 new
products, six of which represented entirely new products to the market.
One new service was developed to the company, market and Wales,
and there are also four improved products in Wales from companies in
CMI’s sample due to SMARTCymru support.
5.25 These results indicate that in funding innovative product and process
development, SMARTCymru is supporting projects that are consistent
with its objectives. They also highlight the challenges for the programme
in generating demand from companies with service and process
projects.
Table 17. New products/process/services as a result of SMARTCymru
Company Wales Market
New products 10 10 6
New processes 0 0 0
New services 1 1 1
Improved products 0 0 4
Improved processes 0 0 0
Improved services 0 0 0
5.26 Encouraging additional R&D expenditure is a complementary
objectives (to the development of products, processes and services),
and can help to embed R&D activity within a company. The intention,
here, is that companies will continue to undertake R&D over the longer
term, with potential spin-off benefits for Wales. The results from the
interviews suggest that capacity has been built in key areas, most
notably in the ability of companies to invest more in R&D as a result of
SMARTCymru. This is supported by the interview evidence (see table
18), where nine of 10 companies indicated that they now invest more in
R&D as a result of the support received. Of the eight companies that
provided an estimated figure for this, the average investment made as a
result of SMARTCymru was £282,750 per annum.
Table 18. Have you invested more in R&D as a result of the project?
Frequency %
Yes 9 90
No 1 10
5.27 Leverage of additional funding is an area where further R&D funds can
be raised. Here, the interviews pointed to examples where companies
had participated in subsequent R&D projects funded through sources
such as SMARTCymru, FP7 and TSB. The findings also pointed to
improved capability to apply for such funding, with half (five of 10)
reporting that SMARTCymru has encouraged/improved their capability
to apply for competitive R&D funds to some extent (see table 19). These
benefits illustrate SMARTCymru’s role in generating so-called R&D
behavioural additionality in supported companies.
Table 19. Extent to which SMARTCymru encouraged/improved capability to apply for competitive R&D funds in companies
Frequency %
To a great extent 3 30.0
To some extent 1 10.0
To a small extent 1 10.0
No impact 4 40.0
Don't know 1 10.0
5.28 New areas of knowledge and skills are important wider benefit to
participation in R&D programmes. Here the results of the interviews
suggest that SMARTCymru projects enabled company participants to
develop new technical skills (seven of 10) in areas related to the
research undertaken during the project. Related to these benefits
projects also provided the opportunity for companies to attract new skills
and improve the skills of staff (six of 10 and seven of 10 respectively). In
a small number of cases (two of 10) qualifications were also developed
through the projects. These skills and knowledge benefits were
important to the companies concerned, but also potentially to the sector
based within (and outside) Wales if, for example, staff were to leave to
take up opportunities elsewhere.
5.29 Collaborative innovation linkages are a further important area of
impact from the SMARTCymru projects. These linkages have the
potential to produce long term collaborative benefits in the areas of
research and innovation. The results from the interviews suggest that
four of 10 companies established collaborative linkages during their
SMARTCymru projects. This included relationships with the university
partners (three of 10), as well as with other companies; large companies
and SMEs (one of 10). These findings highlight that while collaboration
with other partners is not a formal requirement of SMARTCymru many of
the projects supported do include such linkages, many of which are
sustained post project.
Table 20. Maintenance of linkages and collaborative relationships post-
SMARTCymru
Frequency %
Large companies 3 25
SMEs 2 17
University/HEIs 7 58
5.30 Attitudes towards further collaboration are another area in which
SMARTCymru has helped build the capacity of companies and
universities to conduct (collaborative) industrial research. However,
results from CMI’s sample (as illustrated in table 21) suggest that this
has been a modest impact with only one company reporting a positive
impact in their attitude towards collaboration with large companies and
SMEs; and three reporting on a positive impact on their collaboration
with HEIs. The experience of one SMARTCymru project led to a
negative impact on the company’s attitude towards collaboration with
HEIs. The majority of companies, however, reported no impact – six in
terms of their attitude towards collaboration with HEIs and nine of 10
with large companies and SMEs. This, in the majority of cases was a
result of the projects containing no HEI involvement, or having pre-
existing relationships (which did not change attitudes etc.).
Table 21. What impact has the SMARTCymru programme had on your company's attitude to collaboration with:
Significant
+ve impact +ve impact No impact -ve impact
Significant
-ve impact
Large companies 1 0 9 0 0
SMEs 1 0 9 0 0
University/HEIs 1 2 6 1 0
5.31 Innovation spillover benefits were examined from a number of
perspectives in the interviews, with a focus on understanding the nature
and extent of innovation impacts beyond the immediate confines of the
companies concerned. At the mid-term stage the findings suggest that in
most cases it will be too early for spillovers to have occurred. A number
of instances of spillovers were identified however, with one company
that that they had acquired new technical knowledge through a
SMARTCymru project transferring this to a sister company, which had
subsequently gone on to develop a new area of business based on
digital interoperability services. In the context of this research it was not
possible to produce a definitive benefit statement. The collaborative
nature of many of the projects established, however, implies that this is
an area where future benefits are possible. The challenge, however, will
be to allow sufficient resources to ‘trace’ these spillovers beyond the
immediate boundary of the SMARTCymru participants.
5.32 These results suggest that collaborative linkages and positive attitudes
towards their importance are a key area of R&D behavioural
additionality. These relate to network benefits, and illustrate how the
provision of SMARTCymru funding can help to produce a longer lasting
benefit, beyond the immediate project.
5.33 Cross-cutting theme benefits were also explored in the company
interviews, with interviewed companies asked about their use of Welsh
Government support for equality and diversity and environmental
sustainability. No company, however, was able to identify either support
provided or benefits in these areas.
Economic impacts
5.34 A key aim of SMART Cymru is to produce economic impacts for Wales,
as expressed in uplifts of additional employment (jobs) and GVA
(growth) achieved as a result of the programme.
5.35 This analysis draws on the impacts reported in the WEFO monitoring
returns for June 2013. These impacts have then been adjusted for a
series of factors to allow a calculation to be made of net additional
impacts achieved by the SMARTCymru programme in the period.
5.36 The following diagram outlines, in brief, the process for calculating net
additional impact from gross economic impact results:
Assumptions
5.37 The above formula uses various elements to inform the discounting of
gross direct effects. These are outlined below.
5.38 Deadweight is a measure of how different the company perceived their
employment would have been without the SMARTCymru programme.
The results of the interviews suggested that employment would have
been 70 per cent lower had it not been for the programme. On this basis
a deadweight assumption of 30 per cent is used. This, as table 22 below
illustrates, suggests a low level of deadweight when compared to other
similar programmes in the UK. It is likely to reflect the small size of
participant companies, and the high strategic importance attached to the
SMARTCymru projects. This is a positive factor and confirms that the
Welsh Government is funding projects that are strongly additional. It also
points to likely underinvestment in such R&D projects if Welsh
Government chooses to exit from the provision of such support at the
end of the current programme.
Table 22. Comparative deadweight assessments
Name Programme type Estimated deadweight
Period
SMARTCymru RD&I 30% 2010/11 to 2013 Enterprise Ireland RTI RD&I 35% 2002-2006 IDA Ireland R&D Grant RD&I 63% 2002-2006 Invest NI START Industrial research 92% 1996-2007 Invest NI COMPETE Experimental
development 79% 2002-2006
TSB Collaborative R&D Collaborative R&D 35% 2004-2011 Sources: (Frontline Consultants, n.d.; Frontline Consultants, n.d.; CM International, 2009a;
CM International, 2009b; CM International, 2011; PACEC, 2011)
5.39 Displacement (Dp) is the proportion of benefits that are displaced
outside of the company/Wales through support from SMARTCymru. This
may occur where a new product, for example, could make that of a
Wales-based competitor obsolete. No evidence of this was found from
the company interviews, and this was consistent with most projects
being in their early stages of development. A zero percent figure has
therefore been applied to this factor, in line with current additionality
guidance (English Partnerships, 2008), and eliminated from the result
below.
5.40 Leakage (L) is the proportion of benefits achieved that have leaked
outside of Wales. The results of the interviews suggested that this was
low, principally because of the low level of job creation and the location
of companies. A 10 per cent figure has been applied to this factor in line
with additionality guidance (English Partnerships, 2008).
5.41 There is less scope to benchmark substitution (S). No evidence of
substitution was found in any of the beneficiary companies and it has
been assumed to be zero percent and eliminated from the result below.
5.42 The fact that our assessment of leakage and substitution are low may be
regarded as a positive finding that suggest benefits are not being lost to
Wales and that companies are not taking public sector resources to do
activity they would have anyway done themselves.
5.43 Finally, multipliers (M) take into account the level of usage by each
company of suppliers in Wales. CMI uses the additionality guidance
prepared by English Partnerships (English Partnerships, 2008) to
calculate the level of multiplier and has applied a composite multiplier of
1.5 to the additionality calculation representing a medium level of local
benefits anticipated to the wider supply chain.
Conversion of outputs from gross to net
5.44 Gross employment impacts of 31 jobs were created across the
Convergence and Competitiveness programme by the SMARTCymru
programme. Applying the conversion factors produces an estimate of a
net additional employment impact of 29 jobs, with an average of 0.25 net
additional jobs per project.
5.45 Gross Value Added impact for the SMARTCymru programme, at the
mid-term point, is £1.7m based on jobs created since the start of the
programme (i.e. the Actual6 GVA benefit).
5.46 Taking account of discount factors and estimated persistence of
impacts, the Cumulative GVA benefit to Wales of mid-term impacts
produces an estimated net additional GVA of £4.5m.
5.47 Future Potential GVA benefit from SMARTCymru programme, taking
account the economic impact of the programme is likely to produce an
annual £3.4m in net additional GVA, based on the estimated gross job
outputs in 2015.
6 Further details on the calculation of Actual, Cumulative and Future Potential GVA can be found in the annex.
5.48 While economic impact measured according to conventional method
such as above is low, it is important to view these results in context. A
programme such as SMARTCymru was always likely to yield limited
short-term economic benefits for a number of reasons:
5.49 Firstly, the main aim of the programme has not been to create jobs but
to increase R&D activity, and develop new products, processes and
services. These are factors where the interviews suggest that
companies are investing in further R&D and produce new product,
processes and services.
5.50 Secondly, to accrue benefits via the technology development process in
small companies takes a significant time. In this respect it should be
noted that many of the benefits (for example, jobs) associated with
SMARTCymru projects are likely to be created on the basis of
successful commercialisation of a new product, process or service.
5.51 Thirdly, the challenges associated with the economic downturn and the
Welsh Government’s reorganisation and shift away from grants
(associated with the ERP process) has impacted on demand for the
programme. These factors are likely to have contributed to the relatively
low economic impact figures at this stage.
Summary
5.52 The findings from section 6 suggest the following outputs and impacts
from SMARTCymru at the mid-term:
Development of new and improved products and process, with a high
proportion of these (six out of 10) representing new to the market
innovations
Nine of the 10 companies have invested more in R&D as a result of the
project (an average of £282,750 per business), indicating strong levels
of R&D behavioural additionality
Improved capability to access additional R&D funds (five of the 10
companies), with success in accessing funding from other public
funding sources, including FP7 and TSB (five of the 10 interviewees)
Development of new technology and market knowledge and skills of
staff (seven out of 10 companies)
Maintenance of collaborative innovation linkages (four out of 10
companies), including universities and other companies
5.53 Innovation spillovers represent a potentially important area of innovation
impact, particularly in those projects where there is a collaborative
element. A number of such impacts were identified, with the potential for
these to produce future spillover benefits at the final evaluation stage.
5.54 The economic impact results suggest that the programme results to date
will contribute 29 net additional jobs and a cumulative GVA of £4.5m.
This result has been achieved with comparatively low levels of
deadweight, again suggesting that project results are strongly additional.
It should be noted, however, that the main aim of the programme has
not been to create jobs but to increase R&D activity, and develop new
products, processes and services.
6 Value for money
6.1 This section of the report considers the value for money of the
SMARTCymru programme using the so-called ‘3 Es’ approach:
Economy - How much has the intervention cost; how was this decided
on; have the funds been used for the stated activities and, what
additional funds have been levered in?
Efficiency - Have activities been delivered in line with expectations
including: were the funded activities delivered in line with the plan; what
additional activities were delivered; was the cost of delivery as
expected?
Effectiveness - Have the funded activities achieved the expected
results or outcomes; what additional outcomes have been achieved, if
any; and, how has effectiveness been maximised?
Economy
6.2 The SMARTCymru programme was delivered to a total budget at the
mid-term evaluation point of £10.2m (based on eligible expenditure -
Convergence - £8.7m, and Competitiveness - £1.5m), equivalent to
£87,800 per project supported. The Welsh Government’s contribution to
the overall programme costs at the mid-term was £800,000, equivalent
to £6,900 per project supported. Public sector contributions (Welsh
Government and ERDF) represent 72 per cent of the overall programme
cost at the mid-term stage. This figure is consistent with the profile of
projects supported (with the balance of projects in the TCF and IR
phases).
6.3 The budget for the programme was determined by the SMARTCymru
team based on their experience of previous programmes, and a new
assessment of need and demand7. The Business Plans for the
Convergence and Competitiveness programmes suggest that
reasonable steps have been taken by managers to ensure the accurate
planning of the programme and its ‘economy’. This has included the
following:
7 See, for example the Business Plans for both Convergence and Competitiveness programmes.
Funding is fixed in terms of public grant aid application through
WEFO.
Positive synergies have been achieved with the Business Innovation
programme, for example through the use of the Innovation
Specialists.
Establishing intervention rates that strike an appropriate balance
between the risks of different RD&I activities, and encouraging
company investment in R&D.
6.4 As described in section 3, the implementation of the project has seen a
number of policy and organisational changes. The refocusing of the
Innovation Specialists (to sectors rather than regions), and the impasse
brought about by the proposed move to repayable finance, have not
enabled the most efficient use of staffing and resources. Although it is
difficult to quantify this at the mid-term, these factors have the potential
to impact negatively on the economy of the programme.
6.5 Despite the challenges faced in the first stage of the programme, the
recent decision to adjust the budget and targets to respond to the lack of
take-up and capacity represent sensible steps to improve economy.
Efficiency
6.6 The SMARTCymru management and delivery process involves a
relatively complex series of stages, including application support,
technical and financial appraisal, ongoing monitoring and financial
management. Despite positive feedback on the role of the Innovation
Specialist in providing advice and guidance, the main area of feedback
from companies concerned the management and delivery process. In
this respect the majority of companies felt that the application process is
overly ‘long winded’. While, for some, this was merely an inconvenience,
others highlighted the potential implications for both project and
company success and failure. Such problems were felt to be potentially
detrimental for projects in fast moving, innovative markets.
6.7 While the issues raised by the companies point towards an important
area for development, it is important to recognise the necessary ‘due
diligence’ that has to be undertaken to minimise the risk to public
funding, and the high level of skills that are needed to conduct financial
and technical appraisal of projects. These factors highlight the
importance of striking a balance between the company needs and the
need for good practice in public spending.
6.8 The challenge for the remaining period of the programme will therefore
be to minimise the impact of this important process to the companies,
through activities such as simplifying the application form and better
integrating the application and appraisal stages. This points to the need
to ensure that the application process involves, where possible, closer
integration between the appraisal (technical and financial) and
monitoring stages. It also highlights the important role of the Innovation
Specialist and the need to ensure that, where possible, relationships
with companies are maintained throughout the SMARTCymru process.
6.9 The success of the programme in generating sufficient demand is a
further area where the programme delivery efficiency could be improved.
The results of the evaluation suggest that the number of companies
entering the programme will need to be increased substantially. In this
respect many of the steps required to address this challenge are now in
place, with the move towards direct (Innovation team) promotion and
‘brand’ awareness raising, and the refocusing of the Innovation
Specialists on regional links. The evaluation results, and the scale of the
challenge, suggest that still more effective referral linkages to other
innovation and technology programmes, and organisations such as
HEIs, local authorities and so on. The Welsh Government’s sector teams
should also be an important focus, including those where sector teams
have yet to provide significant number of applications (notwithstanding
the difficulties of supporting ICT projects). For example opportunities to
generate demand from the Professional and Financial Services, Creative
and Digital should be further explored.
6.10 Post-project monitoring is a further area where management efficiency
improvements should be considered. Here, company feedback suggests
that they tend to find this element to be cumbersome. Indeed, the
Innovation team recognise that there is potential to improve efficiency by
reducing fragmentation, and to better integrate stages of the
SMARTCymru process in relation to appraisal (technical and financial),
and monitoring. To this end a dedicated monitoring unit for
SMARTCymru projects should be further considered.
6.11 A key indicator of overall programme efficiency is its return on
investment. Here the evaluation results suggest that based on total
programme expenditure8 at the mid-term point (£10.2m), the cost-benefit
ratio based on the GVA is: 1:0.44. This means that for every £1 invested
in the programme £0.44 is likely to be returned to the Welsh economy.
6.12 The GVA performance of SMARTCymru is comparable with other early
stage R&D programmes, for example, the Invest NI START (industrial
research) programme (1:0.55). GVA performance is, however, below
that identified in a final evaluation of the earlier SMARTCymru
programme (1:5.24) (Welsh Government, 2010a). This final impact data
from an earlier programme suggests that the potential GVA return will
increase further in the final stages of the current programme. It is
important to recognise, however, that the primary aim of the
SMARTCymru programme is not necessarily to create jobs but to
increase R&D activity and produce new/improved products and
processes.
Effectiveness
6.13 Against the original objectives of the SMARTCymru programme the
results of the mid-term evaluation suggest that it be considered to have
made reasonable progress, in light of the economic, policy and
organisational challenges faced, and the early stage nature of most of
the RD&I projects supported:
Creation of high quality R&D related jobs
6.14 The programme has created 31 new jobs (29 Convergence and 2
Competitiveness) by the mid-term stage. These jobs have been created
in R&D intensive companies, and are likely to be high quality in nature.
Increase businesses expenditure on R&D through the provision of financial
support to undertake innovative research and technological development
8 ERDF grant plus Welsh Government and private sector match funding.
with commercial potential, leading to new products, processes and
technologies
6.15 The programme has produced induced investment of £264,125. The
interviewed companies invested a further £282,750 on R&D on average.
If this average figure was consistent across all of the companies
supported financially, a total additional investment in R&D at the midterm
would be £27,709,500.
Encourage and support industry collaborations with other partners with
research-based organisations in carrying out industrial research and
experimental development activities
6.16 The programme has produced 14 R&D collaborations between
companies and research based organisations at the mid-term. This
relatively low number of such collaborations is largely attributed to the
presence of a range of other support mechanisms for research—industry
collaborations in Wales, for example the A4B programme, the Innovation
Voucher scheme, and the Knowledge Transfer Partnership programme.
It should also be noted that the Innovation and Technology Team has
also taken steps to address this weakness.
Measure the return on investment of supported R&D projects and provide
continuous improvement of the project management of R&D in companies
supported
6.17 Given the long term nature of the innovation process, capturing return on
investment for RD&I projects is typically undertaken some years after
the initial intervention. For this reason no ongoing measurement of
return on investment is carried out by the Welsh Government. The
results of the mid-term evaluation, however, suggest that the total
programme will contribute £1.7m in actual GVA from the mid-term
projects and £4.5m in cumulative GVA. For every £1 of Welsh
Government investment produces £5.62 net GVA.
Enable business links to other business support to optimise
commercialisation
6.18 This is an area where the synergies developed with the Business
Innovation programme and (potentially) the A4B programme provide
substantial potential to optimise commercialisation. The early role of the
Innovation Specialists, for example, in identifying companies and helping
the development of application forms is clearly important in shaping
projects. Similarly, the ongoing support of the innovation Specialists can
provide ongoing advice, and support to access further RD&I support
(Welsh Government and beyond) and funding after a SMARTCymru
project has been completed.
6.19 The results of the mid-term evaluation do, however, suggest that there is
an opportunity to strengthen synergies with the A4B project, in particular.
This will be important in addressing the programme’s aim to strengthen
collaborative links to researchers, and support collaborative R&D
activities.
Launch new or improved products, processes and services
6.20 The results of the evaluation suggest that 42 new products, processes or
services have been launched as a consequence of SMARTCymru, at
the mid-term stage. Given that studies recommend evaluating innovation
and technology projects approximately six years after the final
intervention, these figures should be treated with care. Indeed, it is
highly likely that they will grow by the end of the programme, and
beyond.
6.21 The cross-cutting themes of equality and diversity, and environmental
sustainability form an underpinning set of objectives for all European-
funded projects. In the Business Plans for Convergence and
Competitiveness targets are highlighted with respect to companies
adopting equality and diversity strategy and systems (50%), and
enterprises adopting or implementing environmental action plans (20%).
The results of the mid-term evaluation suggest that a robust process of
data collection and referral to support has been established for
SMARTCymru companies. This begins with the initial application
process, with the Innovation Specialists, and can include referral to
assistance from a range of external sources of expertise. There is
anecdotal evidence, however, that the programme demand is from a
relatively narrow equality and diversity profile. In this respect there is an
opportunity to work with the Business Innovation Programme (who fund
the Innovation Specialists) to address this weakness.
6.22 In summary, the results of the evaluation suggest that the SMARTCymru
business beneficiaries are beginning to achieve the anticipated outputs.
The early stage of many projects, however, suggests that it will be some
time before a full assessment of impact can be made. Moreover, had it
not been for the contextual challenges noted in earlier sections, outputs
would likely have been higher.
6.23 The SMARTCymru businesses generally state that they would not have
not gone ahead with the project in the same form, if it had not been for
the Welsh Government’s support. This suggests that SMARTCymru
funding is largely additional, and a good use of public funding. Demand
for SMARTCymru programme is therefore evident, however, greater
awareness and promotion is needed for final targets to be achieved.
6.24 The mid-term evaluation results further suggest that both Welsh
Government and the delivery partners have taken steps to support
delivery effectiveness. This has included a flexible approach to adapt
activities in light of changing circumstances. The cross-cutting themes
of equality and diversity, and environmental sustainability have not been
addressed in any detail.
6.25 The mid-term evaluation results point to a number of key lessons which
are important to the future success of the programme. These can be
summarised as follows:
Effective appraisal of R&D projects requires an appropriate blend of
technical and financial skills within the SMARTCymru team.
Innovation Specialists, with strong regional linkages and local
knowledge, are key to supporting companies throughout the RD&I
process, and signposting to related support.
Efficient operation of the programme relies on the ability of
managers to maintain line management control of key staff and
SMARTCymru activities, such as promotion of the programme, and
monitoring of projects and claims.
Brand awareness and effective and consistent promotion are
necessary to raise overall demand for the programme.
Strategic partners such as the universities and Finance Wales offer
important benefits with respect to raising awareness and generating
demand for the programme.
While a phased approach to SMARTCymru R&D funding
encourages companies to follow a robust and planned approach to
innovation projects, there is a need to ensure that blockages in the
pipeline of projects do not occur to the detriment of achievement of
impacts or profile of private sector funding.
Demand for the programme is likely to come from the main science
and technologically R&D intensive sectors such as advance
materials and manufacturing, and while demand is also likely to
exist in other sectors, this will be more difficult to stimulate.
In the current and recent economic climate, without Welsh
Government business R&D funding companies are likely to
underinvest in R&D.
6.26 The remaining section of report draws on these, and earlier findings, in
setting out conclusions and recommendations.
7 Conclusions and recommendations
7.1 This report provides a summary of mid-term evaluation conclusions and
recommendations for further development of the programme. The
results of this analysis suggest that the programme was established with
a sound programme logic model, based on an identified need, clear
objectives and grant activities focused on the different RD&I phases, and
anticipated targets both informed by early programme experience, and
aligned to the objectives. Clear links are also embedded in the model, to
maximise the potential for follow-on support, access to academic
expertise and so on.
7.2 At the mid-term stage the programme is some way below its anticipated
targets. In this respect it has faced a number of significant challenges
that have contributed to demand for the programme being lower than
expected. These include:
The economic downturn precipitated a R&D activity fall in Wales
(and elsewhere), indicating a lower propensity towards risk in the
business community for such activities.
The introduction of the ERP, leading to substantial organisation
reorganisation within the Welsh Government, and the loss (for a
period) of key resources such as the Innovation Specialists.
Difficulties in raising awareness and promoting the RD&I Financial
Support for Business brand, necessitating the rebranding of the
programme as ‘SMARTCymru’.
The proposed move towards repayable finance, and the difficulties
in developing the new SMARTCymru programme ‘repayable grant’
offer.
7.3 In spite of these challenges the programme has, at the mid-term stage
supported 116 projects (84 in the Convergence area, and 32 in the
Competitiveness area). These projects are typically with SMEs, with a
number of key priority sectors represented, including sectors such as the
Advanced Manufacturing and Materials sector, the ICT Sector, Life
Sciences, and Financial Services. All are consistent with the
SMARTCymru quality criteria – defined by a robust technical and
financial due diligence processes.
7.4 The performance indicators established for SMARTCymru largely reflect
those available for ERDF projects, including:
Enterprises financially supported (number)
Gross jobs created (FTE)
Investment induced (GBP)
Collaborative R&D (number)
New or improved products, processes or services launched
Products, processes or services registered
7.5 These are activity driven indicators (enterprises financially supported)
and output indicators. In this respect the indicators are focused on
capturing economic benefits (gross jobs, investment induced),
innovation benefits (investment induced, collaborative R&D), and longer
term innovative results (new products, processes or services). While
these indicators are consistent with the programme logic model set out
in section 2, they are, by their nature, quantitative. This presents
challenges for all programmes, particularly those with a focus on
behaviour change (e.g. R&D behavioural additionality), and suggests the
importance of additional evaluation research to understand these
impacts, and the potential for spillovers.
7.6 Implicit in these indicators is the long term nature of the RD&I process.
By capturing and reviewing indicators both during and at the end of a
programme, there is a danger that the indicators will underplay the true
(potential) extent of impact. This gives support to the Welsh
Government’s decision to establish an Innovation impacts team to
explore such issues through case studies.
7.7 While there is a robust process of collecting cross-cutting themes
indicator data, there is a need to address the programme’s narrow
equality and diversity demand profile. This is challenge that is shared by
its partner programme – the Business Innovation Programme, and
suggests the need for joint action if a full contribution to the cross-cutting
themes are to be met.
7.8 A related challenge is one of attributing cross-cutting theme results
specifically to SMARTCymru. This is problematic, at present, given that
the data is collected by the Business Innovation Programme. This,
again, is an area where joint discussion and action is needed.
7.9 Programme impacts at the mid-term stage are both economic- and
innovation-related. Based on findings from the business interviews (10)
these impacts have been achieved with a lower level of deadweight (30
per cent). They also suggest that Welsh Government is funding projects
that are strategically important to the companies.
7.10 Innovation impacts, at the mid-term, are limited by the early stage nature
of the RD&I projects. The business interviews do, however, point to the
majority of companies both meeting the technical objectives, and
anticipating good commercial potential for the new/improved, products,
processes and services. Companies are also investing more in R&D,
improving their capability to secure additional sources of funding from
public support programmes. The potential for innovation spillovers is
also evident, with the potential for these to produce an additional £132
thousand based on the mid-term projects.
7.11 The economic impact results suggest that the programme results to date
will contribute a net cumulative additional GVA of £4.5, and 29 net
additional jobs. This result has been achieved with comparatively low
levels of deadweight, again suggesting that project results are strongly
additional. It should be noted, however, that the main aim of the
programme has not been to create jobs but to increase R&D activity,
and develop new products, processes and services.
7.12 This performance equates to estimated return on investment, based
on cumulative net additional impacts, of:
£1 SMARTCymru total programme spending produces £0.44 GVA to the Welsh
economy
£1 SMARTCymru Welsh Government spending produces £5.62 GVA to the Welsh
Economy
SMARTCymru programme cost per project of £87,800
7.13 Return on investment (based on total programme spending) is at the
lower end of the comparators examined, however, this can largely be
explained by the comparatively short period over which the programme
activities has been evaluated.
7.14 These economic impact results have been achieved with a relatively low
level of deadweight, with the companies estimating that employment
would have been 70 per cent lower had it not been for the
SMARTCymru programme. This suggests that the projects funded by
Welsh Government are strongly additional. Of those companies that
would have gone ahead in some for the main source of alternative
finance would likely to have been bank related (loan/overdraft).
7.15 The recommendations from the mid-term evaluation are summarised
below.
7.16 Recommendation 1. Welsh Government should continue to review
longer term innovation impacts through its Innovation Impacts
programme, complementing regular programme evaluation.
7.17 Programme spend to date is consistent with the lower than expected
demand for the programme. The high proportion of projects in the early
RD&I phases is also contributing towards higher than anticipated match
funding demands for Welsh Government contributions.
7.18 Recommendation 2: Welsh Government should prioritise demand
stimulation measures, including supporting companies to move on
to the later RD&I phases in a timely manner.
7.19 ‘Brand awareness’ continues to be relatively low, and efforts are needed
to strengthen awareness of companies and other stakeholders such as
universities. Activity in this area has, understandably, been limited by the
ERP process (sector teams and so on). It is clear, however, that greater
awareness that SMARTCymru is ‘open for business’ is needed, as will
better links with potential referral bodies such as HE/FE, Business
Wales and so on.
7.20 Recommendation 3: Welsh Government should seek to ensure
businesses and stakeholders are aware that SMARTCymru is
available and open to applications.
7.21 Given the period of time lost with little or no promotional activities and
uncertainty regarding grant availability during the ERP roll out phase the
assumptions that originally underpinned the profiled and final targets are
no longer accurate. As a consequence action will be needed to review
these targets.
7.22 Discussions with the SMARTCymru team suggest that analysis has
been carried out on the pipeline of projects to determine the potential
worst case/best case/median case outcomes on project spend. The
reprofiled targets that have been developed for the programme as a
result should form the basis for a discussion with WEFO.
7.23 It will important that any discussions with WEFO take account of the
phasing of the programme, and the balance that will need to be struck
between encouraging new enterprise assists and achieving greater
innovation impacts. This balance should take account of the long term
nature of the innovation process, and the potential that projects funded
in the latter part of the programme will not achieve significant impacts in
this period.
7.24 Recommendation 4: Recommendation 4: Welsh Government
should discuss options available for SMARTCymru to revise the
final and profiled performance targets, with a focus on prioritising
projects in the ED and Exploitation phases.
7.25 There are, however, a number of areas where the management and
delivery of the programme could be strengthened to maximise the
impacts discussed above. These relate to speeding up the application
process, while maintaining appropriate due diligence procedures, for
example:
Simplifying the language used in application process
Strengthening integration across the different elements of the
SMARTCyrmu process – financial and technical appraisal, monitoring,
Innovation Specialists
7.26 Recommendation 5: Welsh Government should seek to ensure
delivery efficiency is maximised through appropriate integration
across the SMARTCymru management and delivery processes.
7.27 In relation to the RD&I phases, these are generally well aligned to the
needs of companies. Software companies appear to be one exception,
where it is difficult to meet the application criteria. Partly as a
consequence there are a number of related sectors (such as creative
industries, media and so on) where participation rates are low. Given the
importance of ICT as an enabling factor for many new innovations, there
is a need to explore whether SMARTCymru or another Welsh
Government support programme could better support such projects and
sectors.
7.28 Recommendation 6: Welsh Government should review is support
for innovation projects where ICT plays an important enabling role.
7.29 The Exploitation phase is one that very few companies have reached at
the mid-term. Of those companies that have done so, the amount of
funding available has been noted as inadequate for purposes, and a
barrier to applications. It will be important, in this respect that the Welsh
Government monitor use of this phase, to determine whether funding
can best be allocated to other programme activities / phases. Funding
for this phase, for example, could be used to develop better links to
universities.
7.30 Recommendation 7: Welsh Government should consider
refocusing funding for the Exploitation phase.
7.31 SMARTCymru helps to reduce the risks associated with downstream
public investments made by organisations such as Finance Wales.
While linkages exist between the programmes the mid-term evaluation
research suggests that there may be greater potential for exchange of
information in areas such as appraisal findings. This could help to
improve the efficiency of programme delivery for both partners.
7.32 Recommendation 8: Welsh Government should explore the
potential to strengthen synergies and knowledge exchange with
key stakeholders such as Finance Wales.
7.33 The cross-cutting themes of equality and diversity and environmental
sustainability form part of the objectives for all European-funded
projects. While robust procedures have been established for the
Innovation Specialists to collect cross-cutting themes data (at the
application stage), and to signpost to external sources of expertise, there
is a need for SMARTCymru to work with the Business Innovation
Programme more closely to both access the relevant results, and to
address shared challenges such as the potential to broaden out the
demand from underrepresented groups.
7.34 Recommendation 9: The SMARTCYmru team should work closely
with the Business Innovation Programme to share cross-cutting
themes results, and to address opportunities to improve equality
and diversity.
References
ASTUTE Wales. (n.d.). ASTUTE Wales. Retrieved October 21, 2013, from http://www.astutewales.com/en/
BIS. (2009). RDA Evaluation: Practical Guidance to Implementing the Impact Evaluation Framework.
CM International. (2009). Evaluation of the Enterprise Ireland RTI programme. CM International. (2009). Post-project evaluation of the COMPETE programme,
InvestNI. CM International. (2011). Post-project evaluation of the InvestNI Start programme. EMDA. (2010). Toolkit for the Evaluation of EMDA Strategic Programmes 2007/08-
2008/09. English Partnerships. (2008). Additionality Guide: A Standard Approach to Assessing
the Additionality of Interventions, 3rd Edition. English Partnerships. Retrieved October 21, 2013, from https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/ 191511/Additionality_Guide_0.pdf
European Commission. (n.d.). What is an SME? Retrieved October 22, 2013, from European Commission: http://ec.europa.eu/enterprise/policies/sme/facts-figures-analysis/sme-definition/
Eurostat. (n.d.). Community Innovation Survey (CIS). Retrieved October 21, 2013, from European Commission, Eurostat: http://epp.eurostat.ec.europa.eu/portal/page/portal/microdata/cis
Finance Wales. (n.d.). Finance Wales. Retrieved October 21, 2013, from http://www.financewales.co.uk/
Frontline Consultants. (n.d.). Evaluation of the IDA Ireland R&D Grant scheme. Frontline Consultants. (n.d.). R&D Grant Final Evaluation for Scottish Enterprise. HM Treasury. (2013). Supplementary Green Book Guidance: Optimism Bias.
Retrieved November 18, 2013, from https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/ 191507/Optimism_bias.pdf
Office for National Statistics. (2013, November). Business Enterprise Research and Development, Research and Development in UK Businesses, 2012. Retrieved October 21, 2013, from http://www.ons.gov.uk/ons/dcp171778_337993.pdf
Office for National Statistics. (2013, December). Statistical bulletin: Regional Gross Value Added (Income Approach), Regional GVA NUTS1 dataset. Retrieved October 21, 2013, from Office for National Statistics: http://www.ons.gov.uk/ons/dcp171778_345191.pdf
PACEC. (2011). Evaluation of the TSB Collaborative R&D programmes. Retrieved October 22, 2013, from https://www.innovateuk.org/documents/1524978/1814792/Evaluation+of+the +Collaborative+Research+and+Development+Programmes+Final+Report/e09 dbba0-1cfb-4607-973f-ae9caa73f66d
PricewaterhouseCoopers. (2009). Impact of RDA Spending: National Report Volume 1. Department for Business, Enterprise and Regulatory Reform. Retrieved October 21, 2013, from http://webarchive.nationalarchives.gov.uk/+/http://www.berr.gov.uk/whatwed o/regional/regional-dev-agencies/Regional%20Development%20Agency%20Impact%20Evaluation/pa ge50725.html
Statistics Wales. (2014, February). Statistics Wales. Retrieved October 22, 2013, from https://statswales.wales.gov.uk/Catalogue/Business-Economy-and-Labour-Market/People-and-Work/Employment/Jobs/Whole-Workforce/WorkplaceEmployment-by-Area-Year
Welsh Assembly Government. (2008). European Structural Funds 2007-2013 Strategic Framework: Innovation, R&D and Technology, ERDF Convergence: P1T1 ERDF Competitiveness. Retrieved October 21, 2013
Welsh European Funding Office. (2010). West Wales and the Valleys Convergence Programme European Regional Development Fund 2007-2013. Retrieved October 21, 2013, from http://wales.gov.uk/docs/wefo/publications/convergence/erdfoperational/1106 01erdfconvopen.pdf
Welsh European Funding Office. (2012a). Revised Business Plan Outline: RD&I Financial Support for Business, Convergence.
Welsh European Funding Office. (2012b). Revised Business Plan, RD&I Financial Support for Business, Competitiveness.
Welsh European Funding Office. (n.d. b). Progress of 2007-2013 programmes: RD and I Finance for Business Competitiveness. Retrieved October 21, 2013, from Welsh European Funding Office: http://wefo.wales.gov.uk/programmes/progress/searchprojects/80628?lang=en
Welsh European Funding Office. (No date a). Progress of 2007-2013 Programmes -RD and I Finance for Business, Convergence. Retrieved October 21, 2013, from Welsh European Funding Office: http://wefo.wales.gov.uk/programmes/progress/searchprojects/80504?lang=en
Welsh Government. (2010). Economic Renewal: A New Direction. Retrieved October 21, 2013, from Welsh Government: http://wales.gov.uk/topics/businessandeconomy/publications/economicrenewa l/?lang=en
Welsh Government. (2010a). Impact Evaluation Report. Internal Report. Welsh Government. (2012, September). Academic Expertise for Business Project
Directory: Directory of projects supported through the Welsh Government's European funded Academic Expertise for Business (A4B) programme, 2008-2012. Retrieved October 21, 2013, from Expertise Wales: https://www.expertisewales.com/resource/A4B%20Directory%20(English).pd f
Welsh Government. (2012). Business Innovation. Retrieved October 21, 2013, from Business Wales, Welsh Government Support for Innovation: https://business.wales.gov.uk/sites/default/files/documents/Growing%20a%20 business/Business%20Innovation%20Brochure_Eng_Nov%202012_growing. pdf
Welsh Government. (2013). SMARTCymru: Research, Development and Innovation Funding. Retrieved October 21, 2013, from https://business.wales.gov.uk/sites/default/files/documents/Growing%20a%20 business/SMARTCymru%20Brochure%20Eng_growing.pdf
Welsh Government. (n.d.). SMARTCymru Monitoring data. Welsh Government. (n.d.). Specification for Mid-term Evaluation of SMARTCymru
RD&I Fianancial Support for Business Programme. Research and Evaluation Framework Agreement No. C255/2008/09.
Welsh Government. (n.d.). SMARTCymru monitoring data, Convergence & Competitiveness. Retrieved September 2013
Welsh Government. (n.d.). The Di Minimus Regulation. Retrieved October 21, 2013, from http://wales.gov.uk/topics/businessandeconomy/stateaid/sarules/deminimis/?la ng=en
Annex
A. Detailed methodology
7.35 The mid-term evaluation of SMARTCymru incorporated a range of
methodological stages. This included a number of work stages designed
to review the management and delivery process, impacts achieved at
the mid-term, and their value for money. The main focus was to make
use of the available documentary and monitoring information collected
by the programme, and to minimise the need for new fieldwork. This
approach formed part of the Welsh Government’s requirement for the
mid-term evaluation.
7.36 The mid-term evaluation began with a series of stakeholder interviews
including Welsh Government SMARTCyrmu programme managers and
other relevant staff (see Annex II for details of interviewees). These
interviews were designed to identify the key features of the programme
logic model, and to understand expectations for the mid-term evaluation.
A subsequent half-day workshop was facilitated with the main delivery
staff for the programme (the Innovation Specialists, finance and
monitoring staff).
7.37 The stakeholder interviews and management and delivery workshop
were complemented by a documentary review. This comprised analysis
of programme documentation, including the SMARTCymru Business
Plans (Convergence and Competitiveness), appraisal and monitoring
forms and so on. Alongside this analysis comparator impact data was
collected to contextualise the economic and innovation impact results.
7.38 The focus of the company beneficiary analysis was to analyse the
monitoring data collected by the Welsh Government as reported to
WEFO. This included both economic and innovation performance
indicators, as well as programme spend/income data, including (Welsh
Government, n.d.):
Enterprises financially supported (number)
Gross jobs created (FTE)
Investment induced (GBP)
Collaborative R&D (number)
New or improved products, processes or services launched
Products, processes or services registered
7.39 This monitoring data was used in developing an understanding of the
mid-term profile of participation in SMARTCymru, as well as
achievements against the forecasted targets.
7.40 A series of ten telephone interviews were conducted with companies
that had completed one or more of the SMARTCymru RD&I phases.
These interviews were not intended to provide a fully representative
sample but to highlight the diversity of participation and experiences in
SMARTCymru, and, to ‘signpost’ attention towards some of the impact
areas emerging and key issues for the remaining part of the programme.
The interviews were also used to inform the assumptions used in the
economic impact analysis (see below for more detail).
Economic impact calculations
7.41 The economic impact assessment and calculations draws on the
methodology employed in the ‘Impact of RDA Spending’
(PricewaterhouseCoopers, 2009), BIS Practical Guide (BIS, 2009) and
the English Partnerships Additionality Guide (English Partnerships,
2008). This methodology calculates economic impact from the (WEFO)
reported jobs created indicator, and uses a conversion rate for
determining GVA impact. This produces three levels of net GVA impact
Actual benefit – defined as the net additional GVA produced by the
mid-term point
Cumulative benefit – defined as the actual net additional GVA plus an
estimate of the persistence of these impacts.
Future benefit – the estimated future annual net additional GVA if the
final targets are achieved
7.42 The data that is produced by this method should, given the mid-term
nature of the results, be considered as an estimate, and treated with
appropriate caution.
7.43 The following steps were followed in calculating the Actual achieved
GVA impact from SMARTCymru at the mid-term:
Calculate gross jobs created – the number of FTE gross jobs created
is taken from the SMARTCymru (Convergence and Competitiveness)
indicator data (to June 2013). To estimate the build-up of these jobs it
is assumed that 7.75 FTE jobs were created in 2011, growing to 15.5
FTE jobs in 2012, and 31 FTE in 2013.
Convert gross jobs to net jobs for all three years using the
additionality factors identified in the business interviews (described in
section 5).
Summary results of the net additional jobs calculations are set out in
table 23 below.
Table 23. Net additional jobs (Actual)
Estimated Gross Jobs in Year 1 (2011)
Estimated Gross Jobs in Year 2 (2012)
Gross Jobs in Year 3 (2013)
Gross Impacts 7.75 15.50 31.00
less deadweight (30%) 21.70
less displacement (0%) 21.70
less substitution (0%) 21.70
less leakage (10%) 19.53
Total Net Additional Impacts
plus multiplier (150%) 29.30
7.44 The following steps were followed in calculating the Cumulative
achieved GVA impact from SMARTCymru at the mid-term:
Calculate GVA per job for Wales – Using the Wales Workplace
based GVA at current basis prices for 2012 (Office for National
Statistics), divide by employee numbers in Wales for 2012 from the
Statistics Wales data (Statistics Wales, 2014) to calculate the GVA per
job in Wales. This figure works out as £35,886 per job in 2012.
Similarly, GVA per job for 2011 is calculated by taking Workplace GVA
in 2011 and diving by the employee numbers in Wales for 2011
(£35,112).
The GVA for Wales in 2013 is currently unavailable. Therefore, to
calculate the GVA per job in 2013, it is assumed that the 2012 figure
for Workplace GVA for Wales remains constant, but is discounted by
3.5% to take account of inflation. This figure is then divided by the 2013
employee numbers in Wales to get GVA per job in 2013.
Calculate the gross GVA achieved by the programme – multiply the
number of gross jobs reported each year by the GVA per job in year for
Wales.
Convert Gross GVA to net GVA – Using additionality factors derived
from the business interviews calculate the net additional GVA.
Summary results of the Actual net additional GVA calculations are set
out in table 24 below.
Table 24. Net additional GVA – Actual
Wales GVA and Workforce Jobs Data (000s)
GVA per job (£000s)
2011 GVA generated from jobs (£000s)
2012 GVA generate d from jobs (£000s)
2013 GVA generat ed from jobs (£000s)
Total Actual GVA generatedfrom jobs (£000s)
Wales Workplacebased GVA at current basic prices 2011 (£,000)
46,450,000.00 35.11 272.12
Wales Workplace based GVA at current basic prices 2012 (£,000)
47,344,000.00 35.89 556.23
Wales Workplace based GVA (Present Value) 2013 (£,000)
45,742,995.17 32.98 1,022.37 1,850.72
Gross GVA Impacts
272.12 556.23 1,022.37 1850.72
less deadweight (30%)
190.48 389.36 715.66 1295.51
less 190.48 389.36 715.66 1295.51
displacement (0%) less substitution (0%)
190.48 389.36 715.66 1295.51
less leakage (10%)
171.44 350.42 644.10 1165.96
plus multiplier (150%)
Net Additional
257.15 525.64 966.14 1748.93
7.45 In order to understand the cumulative and future potential benefits at the
mid-term the following assumptions are made for the persistence of the
benefits: how many years the stream of benefits is expected to persist.
Cumulative impacts- is calculated based on an estimated persistence
of the mid-term (actual) impacts – three years. This is consistent with
the BIS (BIS, 2009) guidance on innovation persistence. To calculate
persistence the 2013 net additional GVA is multiplied by three, giving a
total persistence of £2.9m. This is then added to the Actual net
additional GVA to give a total project net additional GVA of £4.6m.
Present Value of GVA impacts – the net additional GVA impact
(£4.6m) is then adjusted to produce the net cumulative additional GVA
of £4.5m with the base year 2013 (i.e. net present value) (EMDA,
2010). Summary results for the net cumulative additional GVA
calculations are set out in table 25 below.
Table 25. Net additional GVA – Cumulative
Net Additional GVA
Actual Impacts Persistence of Impacts
Discount Rate Net Present Value of Persistence
Year 1 2011 £ 257.15 1.07 £ 275.47
Year 2 2012 £ 525.64 1.04 £ 544.03
Year 3 2013 £ 966.14 1.00 £ 966.14
2014 £
966.14 0.97 £ 933.47
2015 £
966.14 0.93 £ 901.91
2016 £
966.14 0.90 £ 871.41
Total Persistence of Impacts
£ 2898.43
£ 2,706.78
Total Net Cumulative Additional GVA (£000s)
£ 4,647.36
£ 4,492.43
7.46 The following steps were followed in calculating the Future potential
GVA impact from SMARTCymru:
Calculate future potential gross GVA – by taking the number of FTE
gross jobs projected by the end of the project and multiplying by the
GVA per job in 20139.
Calculate the Net additional future potential GVA- Using the
additionality weighted factors explained above calculate potential net
additional GVA. The potential net additional GVA is then adjusted to
produce the future estimate of net additional annual GVA of £3.4m with
the base year 2013 (i.e. net present value). Summary results of future
annual potential net GVA is set out in table 26 below.
Table 26. Future potential impact per annum
Future Potential Future Estimate (£000s)
Wales GVA 2013 £ 45,742,995
Wales GVA Per Job £ 32.98
Future Estimate Gross GVA Impactfrom SMARTCymru
£ 3,858.64
Future Estimate Net GVA Impact from SMARTCymru in 2015
£ 3,646.41 Present Value of Future Net GVA Impact from SMARTCymru (in 2013 prices) £ 3,403.97
9 For the purpose of this calculation the final target for jobs created has been reduced to 117 (72 Convergence jobs, and 45 Competitiveness jobs) from 195 to
account for revised targets developed by the programme team. It should be noted, however, that these revised targets had not been agreed with WEFO at the
time of the mid-term evaluation, and have therefore not been included in section 5.
B. Mid-term business interviewees
7.47 Ten business were interviewed as part of the mid-term evaluation. This
included:
NextGen Venturi
Assay Innovation
Clyne Energy
MC Diagnostics
Cyden
The R&D Factory
Eysys
Norland Technology
Structured Productivity Solutions
Dulas
top related