Lesson 5: Relevance

Macerata, 6th December

Alessandro Valenza , Director, t33 srl

Agenda

What does relevance mean?

Assessing the needs

Context Indicator

The SWOT and PEST analysis

Utility / Relevance: Definition

• The term RELEVANCE, in the context of an evaluation, refers to the appropriateness of the explicit objectives of the program in relation to the socio-economic problems it is supposed to address.

• In ex ante evaluation, questions of relevance are the most important because the focus is on choosing the best strategy or justifying the one proposed. (European Commission – DG REGIO EVALSED GUIDE)

• UTILITY is used in ex post evaluation connecting the real impact with the social needs.

Utility and Relevance

In relation to the projectRelated Evaluation Questions

•  To what extent are the programme objectives justified in relation to the needs?

• Can their raison d'être still be proved? Do they correspond to local, national, international priorities?

• Are the expected or unexpected effects globally satisfactory from the point of view of direct or indirect beneficiaries?

(Source Evalsed)

The Utility / Relevance Evaluation Process

Identify the theory change

Collecting context information

Setting the correct

Indicators

SWOT analysisAssessing the

needs and objectives

Collecting Information

Qualitative• Interviews to project actors

(Donors, Policy Owner, Control body, Implementing Body)

• Workshop / Focus group with the stakeholders

• Interview with Sector experts (Accademics, researchers, etc.)

Quantitative• National / regional statistics

• Data from the project

• Data and Information from previous studies (literature review)

Context Indicator

“Simple”Nature Economic Indicator

DYNAMIC

Growth in Export (%) 2007-2008

Growth level of GDP over 2006-2007 (%)

STATIC

Income level for GDP per capita ( STATIC)

“Comparable”• Sector - Area (eg. Regional/

National)

• Average / Median - Maps

Context information - Context

Map 1: Life expectancy at birth, 2009

Thematic - Research and development

 

Map 5: Total intramural R&D expenditure, 2009

Research and development expenditures are key factors both for gaining competitiveness on a global scale and for overcoming the current economic recession. R&D expenditure of 3% of the GDP is one of the five headline targets in the Europe 2020 Strategy (Map 5).

Total intramural R&D expenditures in Umbria are up to 1% of the GDP. Such value is far from the European target of 3%.

The proposed regulations for the next funding period suggest “strengthening research, technological development and innovation” as one possible thematic objective.

Analytical tools: PEST

• PEST analysis is a tool from business science. It is a useful tool for understanding the general feature of the Context.

• PEST is an acronym for Political, Economic, Social and Technological factors which provide a framework for reviewing a situation, or a strategy.

• It is very simple, and is a good subject for work group session or brainstorming meeting.

• PEST factors are essentially external, completing a PEST analysis is helpful prior to completing a SWOT analysis

PEST MATRIX CRITERIApolitical•ecological/environmental issues•current legislation home market•future legislation•international legislation•regulatory bodies and processes•government policies•government term and change•trading policies•funding, grants and initiatives•home market lobbying/pressure groups•international pressure groups•wars and conflicts

economic•home economy situation•home economy trends•overseas economies and trends•general taxation issues•taxation specific to product/services•seasonality/weather issues•market and trade cycles•specific industry factors•market routes and distribution trends•customer/end-user drivers•interest and exchange rates•international trade/monetary issues

social•lifestyle trends•demographics•consumer attitudes and opinions•media views•law changes affecting social factors•brand, company, technology image•consumer buying patterns•fashion and role models•major events and influences•buying access and trends•ethnic/religious factors•advertising and publicity•ethical issues

technological•competing technology development•research funding•associated/dependent technologies•replacement technology/solutions•maturity of technology•manufacturing maturity and capacity•information and communications•consumer buying mechanisms/technology•technology legislation•innovation potential•technology access, licencing, patents•intellectual property issues•global communications

Analytical tools: SWOT

• SWOT is an acronym for Strengths, Weaknesses, Opportunities, Threats.

• A SWOT analysis is a subjective assessment of data which is organized by the SWOT format into a logical order that helps understanding, presentation, discussion and decision-making. The four dimensions are a useful extension of a basic two heading list of pro's and con‘

• Strengths and Weaknesses are internal factors , whereas 

• Opportunities and Threats  might be: Obstacles or Advantages for strengths and weaknesses coming from

external factors The evolution in the near future

SWOT Analysis

NOW Strengths Weaknesses

IN THE FUTURE

Opportunities (external driving

forces empowering the strengths )

Threats (external obstacles)

PEST AND SWOT COMBINATION

• PEST is useful before SWOT - not generally vice-versa - PEST definitely helps to identify SWOT factors.

• PEST assesses more generally the global situation.

• SWOT is an assessment of the more specific sector / territory or policy/ project / programme.

Practical example (1) Assessment the relevance of the issues: PESTPEST

Analysis factors

Importance / Impact/ Time frame

PoliticalEconomicalSocialTechnological

Relative Importance:- Critical- Important- Un-important- Unknown

Impact• Very positive• Positive• Medium• Negative• Very negative

Time frame (short / long term)

SWOTP EST

StrengthsHere you have to locate the items that in the PEST Are short term important / criticalPositive/ very

positive

WeaknessesHere you have

to locate the

items that in the PEST Are short term important /

criticalNegative / Very

negative

Opportunities

Here you have to locate the items that in the PEST Are long term important /

criticalPositive/ very

positive

ThreatsHere you have to locate the items that in the PEST Are Long term important / criticalNegative / Very

negative

POLITICAL

ECONOMIC

SOCIAL

TECHNOLOGICAL

FROM SWOT TO NEEDS

S W

T O

Vantaggi comparati

Debolezze strutturali

COMPARATIVE ADVANTAGE

STRUCTURAL GAPS

Compiti del valutatore:Verificare la “consistency” del’analisi bisogni

S T

T O

Potenziali di soluzione

Ostacoli – colli di bottiglia

• Obstacles (needs)

• Potential for solutions

See you

www.t33.ita.valenza@t33.it