evaluation basics principles of evaluation keeping in mind the basic principles for program and...
TRANSCRIPT
Evaluation BasicsPrinciples of Evaluation
Keeping in mind the basic principles for program and evaluation success, leaders of youth programs can begin learning and using evaluation practices…
Evaluation BasicsPractices of Evaluation
Practices of evaluation include tools and procedures that help leaders…
implement evaluation processesaccomplish evaluation purposesrecognize evaluation principles
Practices are the “nuts and bolts” of doing evaluation
Evaluation BasicsPractices of Evaluation
Sample practices for each process step include…
Planning: SMART ObjectivesFocusing: Evaluability/Standards ChecklistDesign: TimetableMethods: Survey Template, Observation rubricData Collection: Excel Spreadsheet 1Analysis and Interpretation: Excel Spreadsheet
2Communicating Results: Reporting Template
Evaluation BasicsPractices of Evaluation
Planning: The SMART acronym describes steps to turn a general goal into a detailed roadmap for change. SMART objectives are…
Specific about the type, amount, timing of change
Measurable in setting criteria for observing change
Achievable, recognizing program limits, possibilities
Relevant to larger goals and purposesTime-bound in the program scope and sequence
Evaluation BasicsPractices of Evaluation
Focusing: An Evaluability Checklist helps leaders decide whether and how to evaluate their program. Two key evaluation processes are…
Formative, assessing the operation of the program
Summative, determining the results of the program
Evaluation BasicsPractices of Evaluation
Focusing: Formative Evaluability Indicators describe elements to check to see if the program is on track. Key elements of program operation include…
Participants, assessing the operation of the program
Staff determining the results of the programResources, materials and equipment
identified, if not procuredEvaluation, plan and procedures in place
Evaluation BasicsPractices of Evaluation
Focusing: Evaluation Standards help program leaders determine evaluability, or when the program is ready to evaluate. Evaluation Standards include…
Utility, usefulness of the evaluation to stakeholders
Feasibility, potential for effectiveness and efficiency
Propriety, potential for fair and ethical assessment
Accuracy, dependability of findings for a setting
Evaluation BasicsPractices of Evaluation
Designing: A Timeline enables program leaders to plan and manage each stage of evaluation. Key elements of program operation include…
Time 1 Time 2 Time 3 Time 4
Task 1
Task 2
Task 3
Evaluation BasicsPractices of Evaluation
Selecting Methods: Adaptable data collection tools that , enable program leaders to gather fixed-choice and open-ended data from program participants and/or observers include…
Survey, which asks questions about program objectives and experiences; and
Observation Rubric, which provides a framework for judging conditions or performance
Evaluation BasicsPractices of Evaluation
Data Collection: Simple spreadsheets provide excellent ways to organize and store data as follows:
Quantitative (numerical) data typically documents participant data by row, each question category by column; and
Qualitative (verbal) comments can be entered one-per-row, usually without respondent characteristics
Evaluation BasicsPractices of Evaluation
Analysis and Interpretation: Most spreadsheets can be used to make simple counts or comparisons, or to organize responses by category:
Quantitative (numerical) data can be added or averaged by column, with items totaled by frequency (count) or percentage (ratio); and
Qualitative (verbal) comments can be grouped by theme, set either before or after data collection
Evaluation BasicsPractices of Evaluation
Communicating Results: Although different stakeholders or persons interested in a program, appreciate different descriptions, all persons benefit from understanding:
Summary Information: program title, purpose, audience, timing and setting; evaluation findings simply stated, followed by conclusions, implications, and credits for ideas and funding.
Evaluation BasicsPerformance in Evaluation Glossary
Evaluation as PYDEvaluationProgram
Development Taking Stock Implementing Evaluating
(Valued) Outcomes(Valued) ProcessesOrganizational
MissionAssessed Needs
YD Outcomes: Caring, Competence, Confidence, Contributing, Character
Eval. Process Steps Planning Focusing Designing Selecting Methods Data Collecting Analyzing & Interp. Communic. Results
Evaluation BasicsPerformance in Evaluation Glossary
Evaluation Purposes Show accountability Grow capacity Know how to improve
programs Bestow relationships
Evaluation BasicsPerformance in Evaluation Glossary
Evaluation Principles
Develop programs with strength and focus
Intensity Frequency Duration Evidence-based Research-based Program fidelity Readiness for change Stages of change
Develop programs for long-term impact
Programs activities
Develop & evaluate in stages (maturation)
Start-up/pilot Functioning/
Established Mature Formative evaluation Summative
evaluation
Evaluation BasicsPerformance in Evaluation Glossary
Evaluation PracticesPlanning/SMART objFocusing/EvaluabilityDesign/TimetableMethods/Survey &
Observation Rubric Fixed choice Open-ended
Data Collection/Spreadsheet
Quantitative Qualitative
Analysis & Interpretation/Spreadsheet
Communicating Results/Template:
program title, purpose, audience, timing and setting; evaluation findings simply stated, followed by conclusions, implications, and credits for ideas and funding.