technical oversight of annual reporting portfolio committee of public works 11 march 2008
TRANSCRIPT
![Page 1: TECHNICAL OVERSIGHT OF ANNUAL REPORTING Portfolio Committee of Public Works 11 March 2008](https://reader030.vdocument.in/reader030/viewer/2022032313/56649e665503460f94b61728/html5/thumbnails/1.jpg)
TECHNICAL OVERSIGHT OF ANNUAL REPORTING
Portfolio Committee of Public Works
11 March 2008
![Page 2: TECHNICAL OVERSIGHT OF ANNUAL REPORTING Portfolio Committee of Public Works 11 March 2008](https://reader030.vdocument.in/reader030/viewer/2022032313/56649e665503460f94b61728/html5/thumbnails/2.jpg)
2
Session Outcomes Explaining PEM cycle Evaluating Measurable Objectives Critiquing Performance Measures &
Targets Legislative Oversight of Annual
Reporting Alignment between SPP and AR
![Page 3: TECHNICAL OVERSIGHT OF ANNUAL REPORTING Portfolio Committee of Public Works 11 March 2008](https://reader030.vdocument.in/reader030/viewer/2022032313/56649e665503460f94b61728/html5/thumbnails/3.jpg)
3Public Expenditure Management Cycle
Planning
Budgeting
Expenditure monitoringand management
Reporting
![Page 4: TECHNICAL OVERSIGHT OF ANNUAL REPORTING Portfolio Committee of Public Works 11 March 2008](https://reader030.vdocument.in/reader030/viewer/2022032313/56649e665503460f94b61728/html5/thumbnails/4.jpg)
4
![Page 5: TECHNICAL OVERSIGHT OF ANNUAL REPORTING Portfolio Committee of Public Works 11 March 2008](https://reader030.vdocument.in/reader030/viewer/2022032313/56649e665503460f94b61728/html5/thumbnails/5.jpg)
Assessing Performance Information
![Page 6: TECHNICAL OVERSIGHT OF ANNUAL REPORTING Portfolio Committee of Public Works 11 March 2008](https://reader030.vdocument.in/reader030/viewer/2022032313/56649e665503460f94b61728/html5/thumbnails/6.jpg)
6
Analysis of Service Delivery EnvironmentOrganisational and Institutional Environment
Overview: Components of SPP
Whole of Department
Subprogramme
Programme
Situation Analysis
Strategic Goals
Overall StrategicObjectives
Situation Analysis
Part A
Part B
Part CBackground Information
Situation Analysis(if required)
Five-year SPP
Resource Information
Measurable Objectives
![Page 7: TECHNICAL OVERSIGHT OF ANNUAL REPORTING Portfolio Committee of Public Works 11 March 2008](https://reader030.vdocument.in/reader030/viewer/2022032313/56649e665503460f94b61728/html5/thumbnails/7.jpg)
7
Why Evaluate Performance?
• Achieving objectives?
• Outputs delivered?
• Institutional comparisons
• Productivity
• Output – outcome?
![Page 8: TECHNICAL OVERSIGHT OF ANNUAL REPORTING Portfolio Committee of Public Works 11 March 2008](https://reader030.vdocument.in/reader030/viewer/2022032313/56649e665503460f94b61728/html5/thumbnails/8.jpg)
8
Defining objectives“Measurable objectives” need to: Reflect organisational priorities. Be related to activities and resources. Adhere to S.M.A.R.T. principle:
Specific
Measurable
Appropriate
Realistic
Time-bound
E, and R?
![Page 9: TECHNICAL OVERSIGHT OF ANNUAL REPORTING Portfolio Committee of Public Works 11 March 2008](https://reader030.vdocument.in/reader030/viewer/2022032313/56649e665503460f94b61728/html5/thumbnails/9.jpg)
9
Three components of a good objective
1. Primary output that the programme will achieve.
2. Intended impact that the programme’s output will have on the public or client.
3. Level of performance.• The desired level of service delivery
![Page 10: TECHNICAL OVERSIGHT OF ANNUAL REPORTING Portfolio Committee of Public Works 11 March 2008](https://reader030.vdocument.in/reader030/viewer/2022032313/56649e665503460f94b61728/html5/thumbnails/10.jpg)
10
Objectives checklist
Primary output
Is it observable?
Intended impactCan you see what impact it will have?
Level of performanceIs it measurable?
![Page 11: TECHNICAL OVERSIGHT OF ANNUAL REPORTING Portfolio Committee of Public Works 11 March 2008](https://reader030.vdocument.in/reader030/viewer/2022032313/56649e665503460f94b61728/html5/thumbnails/11.jpg)
11
What’s wrong with this MO?
The objective of Further Education and Training (FET) is to provide services in terms of the FET Act.
![Page 12: TECHNICAL OVERSIGHT OF ANNUAL REPORTING Portfolio Committee of Public Works 11 March 2008](https://reader030.vdocument.in/reader030/viewer/2022032313/56649e665503460f94b61728/html5/thumbnails/12.jpg)
12
It only refers to activities. It does not refer to any level of performance – and is therefore not measurable!
And, even more importantly, it does not state the impact of this objective on society.
![Page 13: TECHNICAL OVERSIGHT OF ANNUAL REPORTING Portfolio Committee of Public Works 11 March 2008](https://reader030.vdocument.in/reader030/viewer/2022032313/56649e665503460f94b61728/html5/thumbnails/13.jpg)
13
Revised Objective
“To provide FET colleges with the required resources necessary to roll
out and implement the requirements of the FET Act in
KZN.”
Further improvements?
![Page 14: TECHNICAL OVERSIGHT OF ANNUAL REPORTING Portfolio Committee of Public Works 11 March 2008](https://reader030.vdocument.in/reader030/viewer/2022032313/56649e665503460f94b61728/html5/thumbnails/14.jpg)
14
Change in focus Measurable objectives change the focus
from activities to outputs and outcomes. For example:From
Administer polio vaccines to children under 6 years old in certain hospitals.
To To eradicate polio among children under 6 years in certain areas.
![Page 15: TECHNICAL OVERSIGHT OF ANNUAL REPORTING Portfolio Committee of Public Works 11 March 2008](https://reader030.vdocument.in/reader030/viewer/2022032313/56649e665503460f94b61728/html5/thumbnails/15.jpg)
15
MO from Public Works
Programme 3: National Public Works P.
MO/output: Formalized mentorship programme as a regulated profession
Is there an output? Is there an outcome?
![Page 16: TECHNICAL OVERSIGHT OF ANNUAL REPORTING Portfolio Committee of Public Works 11 March 2008](https://reader030.vdocument.in/reader030/viewer/2022032313/56649e665503460f94b61728/html5/thumbnails/16.jpg)
16
Exercise Choose a programme from DoPW. Examine MOs for that programme.1.) Assess whether it contains all the
necessary components of an MO. 2.) Are all MOs in the control of the
department? 3.) Are they aligned to the strategic
obj?
![Page 17: TECHNICAL OVERSIGHT OF ANNUAL REPORTING Portfolio Committee of Public Works 11 March 2008](https://reader030.vdocument.in/reader030/viewer/2022032313/56649e665503460f94b61728/html5/thumbnails/17.jpg)
17
Performance Measures & Indicators
Performance measures and indicators are statements that describe the dimension of performance that is to be monitored.
The dimension of performance to be monitored must be the most appropriate and under the control of the component.
![Page 18: TECHNICAL OVERSIGHT OF ANNUAL REPORTING Portfolio Committee of Public Works 11 March 2008](https://reader030.vdocument.in/reader030/viewer/2022032313/56649e665503460f94b61728/html5/thumbnails/18.jpg)
18
Criteria for Performance Measures and Indicators:
NT’s “Framework for Managing Programme Performance Information” - 2007
Reliable: accurate for intended use, respond to changes in level of performance
Well-defined: clear, unambiguous definition, consistency
Verifiable: validate the processes and systems that produce indicator
Cost-effective: usefulness of indicator to justify cost of collecting data
Appropriate: indicator must avoid unintended consequences
Relevant: relate logically and directly to aspect of mandate
![Page 19: TECHNICAL OVERSIGHT OF ANNUAL REPORTING Portfolio Committee of Public Works 11 March 2008](https://reader030.vdocument.in/reader030/viewer/2022032313/56649e665503460f94b61728/html5/thumbnails/19.jpg)
19
Dimensions of Performance Measures
Quantity
Cost
Quality
Timeliness
![Page 20: TECHNICAL OVERSIGHT OF ANNUAL REPORTING Portfolio Committee of Public Works 11 March 2008](https://reader030.vdocument.in/reader030/viewer/2022032313/56649e665503460f94b61728/html5/thumbnails/20.jpg)
20
Quantity
Describe outputs in terms of how much or how many.
Require a unit of measurement (e.g. kg, litres, km).
Examples:• number of students passing per year per
grade;• number of schools build;• number of earmarked FET colleges
invested in;• number of finance management
personnel on SCoA, BAS training.
![Page 21: TECHNICAL OVERSIGHT OF ANNUAL REPORTING Portfolio Committee of Public Works 11 March 2008](https://reader030.vdocument.in/reader030/viewer/2022032313/56649e665503460f94b61728/html5/thumbnails/21.jpg)
21
Cost
Should reflect full cost of producing an output
Should include unit cost for each deliverable described under quantity targets
Examples• Cost per unit of materials used; • Average annual operating cost per
learner per year; • Cost per ABET targeted individual
served;• Total operating expenditure.
![Page 22: TECHNICAL OVERSIGHT OF ANNUAL REPORTING Portfolio Committee of Public Works 11 March 2008](https://reader030.vdocument.in/reader030/viewer/2022032313/56649e665503460f94b61728/html5/thumbnails/22.jpg)
22
Quality Reflect service standards based on customer
needs and contribute to government outcomes. Product or service should fit intended purpose. Balance efficiency with effectiveness so that price
is not predominant factor.
May address:• Parent relations, quality of schooling.
Examples:• Number of parent complaints filed; • Minimum standards of electronic connectivity at
schools;• Minimum set of qualifications attained by teachers.
![Page 23: TECHNICAL OVERSIGHT OF ANNUAL REPORTING Portfolio Committee of Public Works 11 March 2008](https://reader030.vdocument.in/reader030/viewer/2022032313/56649e665503460f94b61728/html5/thumbnails/23.jpg)
23
Timeliness Provide parameters for how often, or within what
time-frame, outputs will be delivered.
Measured by turnaround times, waiting or response times (deliver service yearly/quarterly).
Examples:• Whether the brief and instructions to the Minister have
been completed within the deadline;
• Proportion of case reviews conducted by due date;
• Percentage of responses answered within a given timeline;
• Children enrolled yearly.
![Page 24: TECHNICAL OVERSIGHT OF ANNUAL REPORTING Portfolio Committee of Public Works 11 March 2008](https://reader030.vdocument.in/reader030/viewer/2022032313/56649e665503460f94b61728/html5/thumbnails/24.jpg)
24
PMI from DoPW
SP 3.1: Constr. Industry Development Policy & Monitoring
PMI – Implement and Monitor HIV/AIDS Policy
Target – Ongoing
Is the PMI measurable? Is it one of four performance
dimensions?
![Page 25: TECHNICAL OVERSIGHT OF ANNUAL REPORTING Portfolio Committee of Public Works 11 March 2008](https://reader030.vdocument.in/reader030/viewer/2022032313/56649e665503460f94b61728/html5/thumbnails/25.jpg)
25
Constraints & MOs/PMIs
SP constraints to be addressed by MOs and PMIs
SP 3.1: Constr. Industry Development Programme [P49 of SPP]
Lack of co-ordination of infrastructure depts
Unco-ordinated BEE programmes Obstacles such as access to credit for
emerging enterprises Lack of appropriate skills in the industries
![Page 26: TECHNICAL OVERSIGHT OF ANNUAL REPORTING Portfolio Committee of Public Works 11 March 2008](https://reader030.vdocument.in/reader030/viewer/2022032313/56649e665503460f94b61728/html5/thumbnails/26.jpg)
26
Exercise – Measures & Constraints
Exercise: Choose a SP. Evaluate whether SP address the constraints by specifying MOs and PMIs for them.
![Page 27: TECHNICAL OVERSIGHT OF ANNUAL REPORTING Portfolio Committee of Public Works 11 March 2008](https://reader030.vdocument.in/reader030/viewer/2022032313/56649e665503460f94b61728/html5/thumbnails/27.jpg)
27
Developing Performance Measures & Indicators
E.g. if a performance measure is the number of schools built, property developers might use cheap labour and cheap building materials in order to increase productivity levels, but in fact effectiveness and efficiency are compromised.
E.g.: “number of policies, guidelines, and legislation formulated…”
Be aware of perverse incentives!
![Page 28: TECHNICAL OVERSIGHT OF ANNUAL REPORTING Portfolio Committee of Public Works 11 March 2008](https://reader030.vdocument.in/reader030/viewer/2022032313/56649e665503460f94b61728/html5/thumbnails/28.jpg)
28
Perverse Incentives
Activity: For a measurable objective in the Department of Public Works, give an example of how focussing on one dimension of performance (eg. cost, quantity) can undermine other performance dimensions (eg quality, equity etc).
Type: GroupTime: 20 min.Require: Flipchart sheet and markers
![Page 29: TECHNICAL OVERSIGHT OF ANNUAL REPORTING Portfolio Committee of Public Works 11 March 2008](https://reader030.vdocument.in/reader030/viewer/2022032313/56649e665503460f94b61728/html5/thumbnails/29.jpg)
29Developing Performance Targets
Key Characteristics of Targets
Defined in precise terms relating to delivery of outputs
Relate to a single performance measure of a particular output
Specify a time frame or milestone
Are measurable i.e. actual numbers and percentages (not terms like increase, decrease or optimal unless quantified)
Linked to baseline achievements
![Page 30: TECHNICAL OVERSIGHT OF ANNUAL REPORTING Portfolio Committee of Public Works 11 March 2008](https://reader030.vdocument.in/reader030/viewer/2022032313/56649e665503460f94b61728/html5/thumbnails/30.jpg)
30
Performance Targets
Set the quantity, quality, cost and timeliness levels for output delivery
Governments will use targets to:
• set delivery levels; and
• assess departmental performance.
To ensure targets are achieved, need to involve all stakeholders in the process
SMART applies
![Page 31: TECHNICAL OVERSIGHT OF ANNUAL REPORTING Portfolio Committee of Public Works 11 March 2008](https://reader030.vdocument.in/reader030/viewer/2022032313/56649e665503460f94b61728/html5/thumbnails/31.jpg)
31
Examples:Performance targets
Performance target (1)• Less than 10% permanent staff
turnover rate Performance target (2)
• 1:34 teacher/learner ratio for KZN.
![Page 32: TECHNICAL OVERSIGHT OF ANNUAL REPORTING Portfolio Committee of Public Works 11 March 2008](https://reader030.vdocument.in/reader030/viewer/2022032313/56649e665503460f94b61728/html5/thumbnails/32.jpg)
32
Legislative Oversight of AR (1)
S55 (2) of Constitution outlines oversight powers of NA
AR allows parliament to evaluate performance of dept after financial yr
PFMA requires AO to table performance targets for their dept – ENE & SP
Challenge for PC is to get depts to provide good quality perf. info. with tight perf. targets & then get depts to report against these in AR
![Page 33: TECHNICAL OVERSIGHT OF ANNUAL REPORTING Portfolio Committee of Public Works 11 March 2008](https://reader030.vdocument.in/reader030/viewer/2022032313/56649e665503460f94b61728/html5/thumbnails/33.jpg)
33
Oversight Process
![Page 34: TECHNICAL OVERSIGHT OF ANNUAL REPORTING Portfolio Committee of Public Works 11 March 2008](https://reader030.vdocument.in/reader030/viewer/2022032313/56649e665503460f94b61728/html5/thumbnails/34.jpg)
34
Oversight Process (2)
Key question: How did the executive perform in using its budget effectively to deliver services?
Biggest weakness is the poor quality of the non – financial performance information
![Page 35: TECHNICAL OVERSIGHT OF ANNUAL REPORTING Portfolio Committee of Public Works 11 March 2008](https://reader030.vdocument.in/reader030/viewer/2022032313/56649e665503460f94b61728/html5/thumbnails/35.jpg)
35
Consideration of AR1. What is the technical quality of AR?2. Does the dept report on each and every
performance target specified in ENE & budget?
3. What is the quality of perf. info. as highlighted by performance audit by A-G?
4. Is the dept. achieving economy, efficiency & effectiveness?
5. Equity in service delivery?6. Evaluating mgt’s explanations of why dept’s
peformance did not attain targets set in SP and budget
7. Investigating circumstances of under & over-expenditure & its impact on service delivery
![Page 36: TECHNICAL OVERSIGHT OF ANNUAL REPORTING Portfolio Committee of Public Works 11 March 2008](https://reader030.vdocument.in/reader030/viewer/2022032313/56649e665503460f94b61728/html5/thumbnails/36.jpg)
36
Alignment of AR and SPP PMIs in SPP should be reported on in
AR PMIs and planned targets should be
consistent Poor specification of PMIs in SPP
non-reporting in AR Require internal processes to
capture non – financial information
![Page 37: TECHNICAL OVERSIGHT OF ANNUAL REPORTING Portfolio Committee of Public Works 11 March 2008](https://reader030.vdocument.in/reader030/viewer/2022032313/56649e665503460f94b61728/html5/thumbnails/37.jpg)
37
Example - DoPW P3 (NPWP);SP1: Construction Industry
Development Programme Output 1 (ENE) – Construction Industry T.
Charter PMI 1 – Charter Gazetted Target 1– Dec 2006 Output 2 – Contractors exit from incubator progr. PMI 2 – No. of contractors graduated Target 2 – At least 50 contractors by Dec 2008 SPP – Pge 53 AR – Pge 35
![Page 38: TECHNICAL OVERSIGHT OF ANNUAL REPORTING Portfolio Committee of Public Works 11 March 2008](https://reader030.vdocument.in/reader030/viewer/2022032313/56649e665503460f94b61728/html5/thumbnails/38.jpg)
38
Exercise - DoPW
Examine PMIs for a Programme in the DoPW budget statement. Cross – check SPP to see if PMIs are recorded there.
Turn to AR and assess whether it reports on these PMIs.
![Page 39: TECHNICAL OVERSIGHT OF ANNUAL REPORTING Portfolio Committee of Public Works 11 March 2008](https://reader030.vdocument.in/reader030/viewer/2022032313/56649e665503460f94b61728/html5/thumbnails/39.jpg)
ENDQuestions?