evaluation planning & tools workshop€¦ · 2. a theory of change for your project 3. an...
TRANSCRIPT
Evaluation Planning & Tools Workshop
Text and content copyright © 2014 Project Oracle. All rights reserved. 2
Evaluation Planning
Text and content copyright © 2014 Project Oracle. All rights reserved.
The correct design will depend
on what questions you
want to answer through your
evaluation.
3
Evaluation design
Text and content copyright © 2014 Project Oracle. All rights reserved. 4
Potential Questions…
Questions Evaluation Type
How well was the programme done? How could we improve it?
Process Evaluation
What difference did the programme make to those taking part?
Outcome/Impact Evaluation
How did the change happen?Why did the change happen?
Realistic Evaluation
Would this have happened anyway?Did our project cause these changes?
Comparisons in Evaluation
Is the programme worth the money? Economic Evaluation
Text and content copyright © 2014 Project Oracle. All rights reserved.
Design your evaluation
5
Standards of Evidence & Evaluation Design
Standard Types of Evaluation
Standard 2 Outcome evaluation
Standard 3 Outcome evaluation with comparison
Standard 4 Outcome evaluation focussing on if the project works in other settingsHow and why changes happenProcess evaluationEconomic evaluation
Standard 5 As Standard 4
Text and content copyright © 2014 Project Oracle. All rights reserved. 6
Outcome evaluation plan
When planning consider is it: appropriate? feasible? necessary? ethical?
Be realistic and integrate existing practice!
WHAT? HOW? WHEN? WHO? HOW MANY?
Outcome Indicator Data
source &
how you
will collect
this
When will it
be
collected?
Who will
collect this?
What proportion of
people will you
gather this from, and
how will they be
selected?
Text and content copyright © 2014 Project Oracle. All rights reserved. 7
Methods for collecting data
Text and content copyright © 2014 Project Oracle. All rights reserved. 8
Robustness
Your methods should be
Valid: measure what it intends to measure
Reliable: the same result is obtained from using the method on repeated occasions
Both of these lead to confidence in the robustness of the evidence
Text and content copyright © 2014 Project Oracle. All rights reserved. 9
Types of data collection
• based on numbers • Closed questions: How many? How long?
• descriptive & in-depth• Open questions: Why? How?
Quantitative
Qualitative
Text and content copyright © 2014 Project Oracle. All rights reserved. 10
Mixed Methods
Mixed methods are exactly that – using different techniques to answer the same question
For example for a school based project:
• Test scores are compared with a comparison group and the start and end of the project
• Interviews with children, parents and teachers to gain a more in-depth view as to how and why changes happened
Text and content copyright © 2014 Project Oracle. All rights reserved.
Validated tools are designed to measure specified changes. They have been tested and are widely used, meaning that they are reliable.
Examples:• NPC Well-being Measure • Strengths and difficulties • Rosenberg Self Esteem Scale• Outcomes Star
11
Validated tools
Text and content copyright © 2014 Project Oracle. All rights reserved.
1. Basic information on your project
2. A Theory of Change for your project
3. An evaluation plan for your project
Register and validate your project at: www.project-oracle.com
Deadlines: 1st March, 1st June, 1st
September and 1st December
12
Standard 1 Validation: What’s required
Text and content copyright © 2014 Project Oracle. All rights reserved.
1. Further Training on Evaluation Planning & Tools…
2. Cohort Participation…
3. Events
Contact us: Haile Warner
Provider Lead
020 7148 6726
Follow @Project_Oracle
13
How we can support you further…