health promotion for disadvantaged youthcognitivimarca.altervista.org/download/h25/05...
TRANSCRIPT
The Project HEALTH25 has received funding from the European Union in the framework of the Health Programme.
Health promotion for disadvantaged youthWP 3 | Evaluation
Quality Management Concept
PresentationPresentation
Max SaurugAlfred Strassegger
Ancona, 28.09.2011
The Project HEALTH25 has received funding from the European Union in the framework of the Health Programme.
Health 25 | Introducing IFA Steiermark
Our OrganisationOur OrganisationIFA Steiermark, seated in Graz, Austria (5 employees) was founded in 1989.
IFA is a research institution working for the common good of society. Our organisation is aimed at
assisting politically responsible people, decision makers in public corporations and private companies,
persons involved in projects, special interest groups and so on.
Research = Accompaniment + Optimisation
The Project HEALTH25 has received funding from the European Union in the framework of the Health Programme.
Health 25 | Introducing IFA Steiermark
Our ServicesOur Services
Evaluations
Short term analyses
Consultation and development
Our work includes questions concerning labour market and social policy, regional development,
vocational education for companies and personnel and the integration of people with special needs into
the labour market.
The Project HEALTH25 has received funding from the European Union in the framework of the Health Programme.
Health 25 | Introducing IFA Steiermark
Our ExperienceOur ExperienceOver many years, IFA has acquired a vast wealth of experience in
the planning, conception and
undertaking of practical scientific investigations and in accompanying consultations about programs and
measures in the social field.
Through numerous all‐embracing national and transnational research and development projects
conducted throughout Europe (more than 100), IFA is able to fall
back on a wealth of both national and
international experience.
To learn more: www.ifa‐steiermark.at
The Project HEALTH25 has received funding from the European Union in the framework of the Health Programme.
Health 25 | WP 3: Evaluation | 1. Quality Management Concept
Quality Management Quality Management ConceptConceptThis quality management concept will give you detailed description of the quality management and an
overview of the evaluation strategies in regard to the quality management of
the project process,
the project product (outputs and outcomes) and
the project impact.
In respect of the assessment of the quality of the Health 25‐project the quality indicator identification
process and its outcomes should be started as one of the next steps.
from http://ec.europa.eu/eahc/management/Fact_sheet_2010_05.html
The Project HEALTH25 has received funding from the European Union in the framework of the Health Programme.
Health 25 | WP 3: Evaluation | Quality Management Concept
The objectivesThe objectives# 1 Literature review and survey
# 2
Development of the health training and intervention programme for
the target group
# 3
Development of pedagogical‐psychological concept for the target
group
# 4 Synthesis of the holistic health training and intervention programme
for 16‐20 year old people without a job and not in school education
# 5 Pilot implementation
The Project HEALTH25 has received funding from the European Union in the framework of the Health Programme.
Health 25 | WP 3: Evaluation | Quality Management Concept | Overview
Criteria Based EvaluationCriteria Based EvaluationFor a successful venture the quality management approach has to rely on specific quality criteria. This
concept relies on the following characteristics:
Scientifically an methodologically correct
Transparent
Flexible
Realistic
Possible to control
Targeted
The Project HEALTH25 has received funding from the European Union in the framework of the Health Programme.
Health 25 | WP 3: Evaluation | Quality Management Concept | Overview
Description of the modelDescription of the modelThe quality management concept is based on a triangulate evaluation approach. In the frame of this
approach there are three different dimensions to focus on during
the evaluation:
Project process
Project products
Project impact
Only by taking into account these three dimensions the project quality can be estimated in a holistic
approach. The model contains so called Critical Success Factors (CSF’s) for all three dimensions, which are
partly given in advance, but mostly tailored to the project.
The Project HEALTH25 has received funding from the European Union in the framework of the Health Programme.
Health 25 | WP 3: Evaluation | Quality Management Concept | Overview
DescriptionDescription of of thethe modelmodel
This triangulate Quality Management Approach implies that all three quality dimensions
interdependently interfere, which allows to ensure quality through a holistic approach. Furthermore, this
method is absolutely transparent and the evaluation data is accessible anytime to all the project partners.
The quality management method estimates the actual status quo by covering the targets set; it
recognizes deviances and suggests possible steering measures, which allows to keep the project on track
from the start and to reach the project quality proposed.
The Project HEALTH25 has received funding from the European Union in the framework of the Health Programme.
Health 25 | WP 3: Evaluation | Quality Management Concept | Overview
QM of the project processQM of the project process
One of the components of this quality management strategy involves the monitoring of the project in
terms of its processes. In this context the following processes will be observed:
Communication among partners
Communication between the contractor and the partners
The level of information and awareness of the partners towards the contents of the project (also goals,
schedule and budget)
The degree of involvement of the different partners in the project activities
The Project HEALTH25 has received funding from the European Union in the framework of the Health Programme.
Health 25 | WP 3: Evaluation | Quality Management Concept | Overview
QM of the project processQM of the project process
The reports serves as monitoring tool for the coordinator as its part of the quality Management and
Evaluation Strategy. The questionnaire developed focuses on the following areas:
Processes during the meeting
Organisation of the meeting
Suggestions, recommendations and personal comments
Project progress reports
Implementation of the peer evaluation system
The Project HEALTH25 has received funding from the European Union in the framework of the Health Programme.
Health 25 | WP 3: Evaluation | Quality Management Concept | Overview
QM of the project processQM of the project process
Criteria collection for the product evaluation
Methods and instruments for the evaluation
The Project HEALTH25 has received funding from the European Union in the framework of the Health Programme.
Health 25 | WP 3: Evaluation | Quality Management Concept | QM of the project products
Criteria collection (CSFCriteria collection (CSF’’s)s)
In the process of ensuring the quality of the Health 25‐project all participating partners will be assigned
the identification of applicable quality indicators and subsequently asked to rate these items. The partner
responsible for the WP3 – Evaluation of the project –
will collect and analyse the data provided by all
organisations involved in the Health 25‐project.
Stage 1 | Initial Phase –
Definition of the CSF’s
Stage 2 | Rating the elaborated items
Stage 3 | Processing the contributions
The Project HEALTH25 has received funding from the European Union in the framework of the Health Programme.
Health 25 | WP 3: Evaluation | Quality Management Concept | QM of the project products
Evaluation methods and instrumentsEvaluation methods and instruments
The proposed instruments used for the evaluation of the Health 25‐project are based on the experience
of the evaluator in the area of quality management. In order to assess the individual CSF’s of the Health
25‐project it is recommended to use
(mainly) questionnaires and
guided Interviews (if reasonable and necessary)
as methods most suitable for the evaluation purposes.
The use of questionnaires along guided interviews within a partner meeting is intended to produce a
combination of qualitative and quantitative information which should support the evaluation of the
Health 25‐project in
respect of the work process and the product itself.
The Project HEALTH25 has received funding from the European Union in the framework of the Health Programme.
WP 3: Evaluation of „Health 25“ | The Quality Management Concept | Overview
QM of the project impactQM of the project impact
Similar to the evaluation of the product quality, a method of impact evaluation is recommended; where in
some case partly standardised CSF’s may be useable. These CSF’s will be developed similar to those in the
product quality involving all partners and stakeholders.
It is suggested to evaluate the impact of the product on representatives of the target groups in the
partner countries at the end of the project; the Health 25‐project can also be assessed in respect to its
short term impact. There could/should be mentioned measurement criteria like physical measurements
(baseline and at the end of the training programme) in relation to the Health Training and Intervention
programme as well as (set and fulfilled) expectations, knowledge, skills, attitudes and behaviour.
The Project HEALTH25 has received funding from the European Union in the framework of the Health Programme.
Thank you for your attention
The Project HEALTH25 has received funding from the European Union in the framework of the Health Programme.
Health 25 | WP 3: Evaluation
Evaluation Evaluation –– Basic approachBasic approach
Objective of the planned research project is the evaluation of the cooperation “Health 25”
on a
transnational level. The central tasks of the research assignment are to analyse the implementation
process and the attainment of the common set objectives.
The Project HEALTH25 has received funding from the European Union in the framework of the Health Programme.
Health 25 | WP 3: Evaluation
Levels of analysis Levels of analysis and key questionsand key questions
Level 1:
This concerns outputs of work packages and the ”Health 25”
project as a whole: Specifying tasks
and duties ‐
What are the central common aims of ”Health 25”
? What is each member is in charge of?
How do the partners describe their roles within the transnational partnership? What are the
expectations? What are the duties?
Level 2:
Specifying the implementation steps: What working steps, methods and tools are to be used to
attain the common objectives?
Level 3:
Analysing the implementation process: What are the general conditions and factors that can
promote and impede the cooperation process?
The Project HEALTH25 has received funding from the European Union in the framework of the Health Programme.
Health 25 | WP 3: Evaluation
Levels of analysis and key questionsLevels of analysis and key questions
Level 4:
Analysing goal‐attainment: To what extent were the set objectives (outputs/outcomes) attained?
Could the expected impact on the target groups be reached?
Level 5: SWOT analysis and analysis of ”lessons learned”: What are the strengths and the weaknesses of
this transnational cooperation? What is the potential for improvement? Are there any recommendations
for future partnerships?
In addition the project partners of the participating countries are to be assisted in evaluating their project
activities (within their work packages) themselves by putting at
their disposal research instruments
developed for this evaluation assignment. The results from this self‐assessment are to be made available
to the evaluator and incorporated into the evaluation as a whole.
Special task: A Quality Management Handbook has already been worked out (and also distributed to the
partners)
The Project HEALTH25 has received funding from the European Union in the framework of the Health Programme.
Health 25 | WP 3: Evaluation
Duration, methods and working stepsDuration, methods and working steps
36 months (February 2011‐
January 2014)
Both qualitative and quantitative instruments are to be
used for this accompanying evaluation. Internet‐
based questionnaires for quantitative and partly standardised surveys are to be used.
The Project HEALTH25 has received funding from the European Union in the framework of the Health Programme.
Health 25 | WP 3: Evaluation
Supporting the lead partner and reportingSupporting the lead partner and reporting
Periodic written reports about the progress of Health 25 are to be presented at every transnational
meeting. The evaluator is a member of the steering committee and
is to support the project lead partner
BBRZ Austria.
The evaluator is obliged to attend all transnational meetings of Health 25.
Representatives of the transnational project partnership and the lead partner are to be kept regularly up‐
to‐date orally and by periodic short reports after the meetings.
These reports ensure that the evaluation results can be used as a basis for project management.
Written
reports are to be submitted every six months as well as the
final results of the evaluation are to be drawn
up in a final research report.
The Project HEALTH25 has received funding from the European Union in the framework of the Health Programme.
Health 25 | WP 3: Evaluation | Working Steps
Presentation of the evaluation concept at the
transnational meeting in Ancona/It
(September 2011)
Presentation of the evaluation concept at the
transnational meeting in Ancona/It
(September 2011)
Development of questionnaires for self‐assessment of the
work packages of the project partners
(October/November 2011)
Development of questionnaires for self‐assessment of the
work packages of the project partners
(October/November 2011)
Development of an internet‐based questionnaire in order
to evaluate the progress of target attainment and the
implementation process of the project partners and the
project “Health 25”
as a whole (October/November 2011)
Development of an internet‐based questionnaire in order
to evaluate the progress of target attainment and the
implementation process of the project partners and the
project “Health 25”
as a whole (October/November 2011)
Participating in the 1st working group meetings of
WP 5, WP 6, WP 8
Participating in the 1st working group meetings of
WP 5, WP 6, WP 8
Drawing up the Final Report | January 2014
Presentation of the final results | January 2014
Drawing up the Final Report | January 2014
Presentation of the final results | January 2014
Participating observation at all transnational meetings; 6
qualitative interviews with leading representatives of the
partner organisations (Ongoing)
Participating observation at all transnational meetings; 6
qualitative interviews with leading representatives of the
partner organisations (Ongoing)
Evaluating and reporting the self‐assessment of the project
partners in the final evaluation report
Evaluating and reporting the self‐assessment of the project
partners in the final evaluation report
Periodic evaluation of the progress of the project including
short evaluation reports after every transnational meeting
(including first SWOT analyses)
Periodic evaluation of the progress of the project including
short evaluation reports after every transnational meeting
(including first SWOT analyses)
Elaboration the Quality Management Handbook (June
2011)
Elaboration the Quality Management Handbook (June
2011)