quick feedback on preliminary findings · 2017. 12. 8. · 12 september 2017 dhet colloquium on nqf...
TRANSCRIPT
![Page 1: Quick Feedback on Preliminary Findings · 2017. 12. 8. · 12 September 2017 DHET Colloquium on NQF Antonio Hercules . Some key steps in NES evaluation: NQF 1 NQFA TWG 2 TOR & Procure](https://reader036.vdocument.in/reader036/viewer/2022071002/5fbe79d73e2f266e4c42141e/html5/thumbnails/1.jpg)
Quick Feedback on Preliminary Findings
12 September 2017 DHET Colloquium on NQF
Antonio Hercules
![Page 2: Quick Feedback on Preliminary Findings · 2017. 12. 8. · 12 September 2017 DHET Colloquium on NQF Antonio Hercules . Some key steps in NES evaluation: NQF 1 NQFA TWG 2 TOR & Procure](https://reader036.vdocument.in/reader036/viewer/2022071002/5fbe79d73e2f266e4c42141e/html5/thumbnails/2.jpg)
Some key steps in NES evaluation: NQF
1 NQFA TWG
2 TOR &
Procure
3 Start
& Confirm
4 Collect Data
5 Stake-holders Validate
2
9 Cabinet
6 Draft
Report
7 Final
Report
8 Cluster
9 Cabinet
![Page 3: Quick Feedback on Preliminary Findings · 2017. 12. 8. · 12 September 2017 DHET Colloquium on NQF Antonio Hercules . Some key steps in NES evaluation: NQF 1 NQFA TWG 2 TOR & Procure](https://reader036.vdocument.in/reader036/viewer/2022071002/5fbe79d73e2f266e4c42141e/html5/thumbnails/3.jpg)
Characteristics of Implementation Evaluations
• Type of evaluation: Assessment of programme delivery, strategies, procedures and processes. In other words… an assessment of policy/programme (or part of) performance to-date
• Implementation evaluation can happen at any time after or during policy/programme implementation
• Essential part of effective programme management
3
![Page 4: Quick Feedback on Preliminary Findings · 2017. 12. 8. · 12 September 2017 DHET Colloquium on NQF Antonio Hercules . Some key steps in NES evaluation: NQF 1 NQFA TWG 2 TOR & Procure](https://reader036.vdocument.in/reader036/viewer/2022071002/5fbe79d73e2f266e4c42141e/html5/thumbnails/4.jpg)
Characteristics of Implementation Evaluations
4
Main results reported here
![Page 5: Quick Feedback on Preliminary Findings · 2017. 12. 8. · 12 September 2017 DHET Colloquium on NQF Antonio Hercules . Some key steps in NES evaluation: NQF 1 NQFA TWG 2 TOR & Procure](https://reader036.vdocument.in/reader036/viewer/2022071002/5fbe79d73e2f266e4c42141e/html5/thumbnails/5.jpg)
Characteristics of Implementation Evaluations
Key elements of implementation evaluations:
1. Relevance and appropriateness: (is policy/prog relevant and aligned with related policies and frameworks? Is it appropriate for the context and especially for intended beneficiaries and stakeholders?),
2. Effectiveness: (is the policy/prog working? Is it achieving the initial goals/objectives? Is the design appropriate? (logic, coherence, ToC, “results framework”), Current level of delivery)
3. Efficiency: (Is the organizational design appropriate to deliver on intended goals/objectives? Adequate systems and resources? is there reasonable value-for-money, and comparability with similar policies/progs at least in the government sector? Is management and accountability mechanisms effective?)
5
![Page 6: Quick Feedback on Preliminary Findings · 2017. 12. 8. · 12 September 2017 DHET Colloquium on NQF Antonio Hercules . Some key steps in NES evaluation: NQF 1 NQFA TWG 2 TOR & Procure](https://reader036.vdocument.in/reader036/viewer/2022071002/5fbe79d73e2f266e4c42141e/html5/thumbnails/6.jpg)
Characteristics of Implementation Evaluations
Key elements of implementation evaluations:
4. Sustainability: (is the policy/prog being institutionalised in a given ministry or sector? Is there sufficient funding in the foreseeable future?)
5. Impact: (what do we need to do now, in order to prepare for an impact evaluation in the future?)
“In-depth and comprehensive information about the quality of service delivery”.
Emerging insights… practice improvements, and policy options.
6
![Page 7: Quick Feedback on Preliminary Findings · 2017. 12. 8. · 12 September 2017 DHET Colloquium on NQF Antonio Hercules . Some key steps in NES evaluation: NQF 1 NQFA TWG 2 TOR & Procure](https://reader036.vdocument.in/reader036/viewer/2022071002/5fbe79d73e2f266e4c42141e/html5/thumbnails/7.jpg)
Implementation Evaluations: sense of emphasis, importance
Sust
ain
abili
ty
Effi
cie
ncy
Imp
act
Effe
ctiv
enes
s
Rel
evan
ce
1. Design (ToC, results framework)
2. Outputs
3. “Results” (immediate outcomes) to date
1. Org. Design
2. Manage-ment & Account-ability
3. Value-for-money? Comparison
4. Systems, Procedures
1. Institut-ionalization
2. Financial
3. Content
1. Current Data
2. Emerging Insights, Lessons
1. Relevant & aligned?
2. Appro-priate for Context and Stake-holders
Sources of Evidence: Policy documents, prog/dept. monitoring records, literature data
Sources of Evidence: prog/dept. monitoring records, other records eg minutes, primary evaluation data
Sources of Evidence: Policy documents, prog/dept. monitoring records, primary evaluation data
Sources of Evidence: quality prog/dept. monitoring records, primary evaluation data
Sources of Evidence: Policy documents, programme/dept monitoring records
![Page 8: Quick Feedback on Preliminary Findings · 2017. 12. 8. · 12 September 2017 DHET Colloquium on NQF Antonio Hercules . Some key steps in NES evaluation: NQF 1 NQFA TWG 2 TOR & Procure](https://reader036.vdocument.in/reader036/viewer/2022071002/5fbe79d73e2f266e4c42141e/html5/thumbnails/8.jpg)
Quick! Feedback on NQF Act Evaluation
1. Evaluation Design
2. Stakeholder engagement & consultation (data)
3. Quality of conclusions and recommendations
4. Now… formal written feedback
5. Evaluation usefulness…
8
![Page 9: Quick Feedback on Preliminary Findings · 2017. 12. 8. · 12 September 2017 DHET Colloquium on NQF Antonio Hercules . Some key steps in NES evaluation: NQF 1 NQFA TWG 2 TOR & Procure](https://reader036.vdocument.in/reader036/viewer/2022071002/5fbe79d73e2f266e4c42141e/html5/thumbnails/9.jpg)
9