AUDITING OF PREDETERMINED OBJECTIVES

27 November 2015

Reputation promise/mission

The Auditor-General of South Africa has a constitutional mandate and, as the Supreme Audit Institution (SAI) of South Africa, it exists to strengthen our country’s democracy by enabling oversight, accountability and governance in the public sector through auditing, thereby building public confidence.

Topics to discuss

1. Introduction

2. Legislative requirements and framework for performance management and reporting

3. Audit strategy and approach

4. Findings for local government 2013-14

3

5. Recommendations and good practices

6. Key controls for local government 2013-14

Introduction

• Supreme audit institutions (SAIs) do not only conduct financial audits. They also provides assurance on the reporting of predetermined objectives produced by municipalities, government departments and entities

• Why is this necessary? Public sector reforms Improving public reporting Providing better information on what government is accomplishing regarding service delivery

Need for reporting on predetermined objectives

Audits performed by AGSA

Annual mandatory audits

• Financial statements

• Report on predetermined objectives

• Compliance with laws and regulations

Discretionary audits

• Performance audits e.g. infrastructure

Difference between auditing of predetermined objectives (AOPO) and performance auditing

AoPO Audit of Predetermined Objectives

• Individual audits

• Focused on a specific government policy or management process

• Audit criteria are the 3E’s:

Economy

Efficiency

Effectiveness

• Done by subject matter experts

Annual audit of reported actual performance against predetermined objectives, indicators and targets as contained in the annual performance report.

Integral part of the annual regularity audit process, confirming:

compliance with related laws and regulations

usefulness of performance information

reliability of performance reporting

PA

Performance Auditing

The planning, budgeting and reporting cycle

Legislative requirements and framework for performance management and reporting

An audit report must reflect an opinion or conclusion on the performance of the auditee against predetermined

objectives

Sections 20(2)(c) and 28(1)(c) of the Public Audit Act (PAA) require that:

Applicable to all spheres of government

Auditing requirements

10

Legislative requirements and framework

Municipal Finance Management Act (MFMA) Municipal Systems Act (MSA) MFMA circulars LG: Municipal planning and performance management regulations, 2001 –

GNR.796 of 24 August 2001

NT Framework for managing programme performance information – issued by the National Treasury in May 2007

LG: Municipal performance regulations for municipal managers and managers

directly reporting to municipal managers, 2006 – GNR.805 of 1 August 2006

This represents the performance management and reporting framework against which the performance information should be managed and reported.

Principles and requirements from framework are used as a basis for the audit.

Audit strategy and approach

AGSA strategy

2009-10 to 2010-11 2011-12 to 2014-15

2004-05 to 2008-09

Phased-in approach Factual audit findings

reported in both management and audit reports

No audit opinion in audit reports

Interaction with stakeholders (NT, Presidency, DPSA) to determine and test audit approach

Inputs to drafting of NT frameworks (FMPPI)

Completed phased audit approach

Audit to the extent necessary to express an audit conclusion

Audit conclusion in the management report for all high-capacity municipalities and their entities

Audit reports contain audit findings – not audit opinions

Completed phased audit approach

Audit to the extent necessary to express an audit conclusion

Audit conclusion in the management report for all municipalities and their entities

Audit reports contain audit findings – not audit opinions

This will continue until the environment shows a state of readiness to provide an audit conclusion in the audit report

Audit Approach

1

2

3

Obtaining an understanding of the entity's information system and related control

activities.

Assessing risks of material misstatement to form a basis for designing and

performing further audit procedures.

Obtaining sufficient appropriate evidence on which to base the conclusion on the validity, accuracy, and completeness of reported performance information.

• The audit approach for the audit of predetermined objectives is similar to that of the audit of the financial statements in the following ways:

• Auditees need to instil the relevant controls to ensure that the reported performance for each indicator is valid, accurate and complete in the same way controls are implemented to ensure line items in the financial statements are valid, accurate and complete

• Proper record-keeping measures should be in place to ensure that information is accessible and available to support performance reporting

Main criteria Sub-criteria

Presentation

Measurability

Relevance

Consistency

Validity

Accuracy

Completeness

Usefulness

Reliability

Audit criteria (derived from performance management and reporting framework)

Definition

Performance reporting using the relevant NT reporting principles

Indicators are well defined and verifiable Targets are specific, measurable and time bound

Indicators relate logically and directly to the entity mandate and realisation of strategic goals and objectives

Objectives, indicators and targets are consistent between planning and reporting documents.

Reported performance has occurred, have been recorded and reported correctly and completely.

Measurability and relevance criteria – further details Measurability

of indicators

Measurability of targets

Relevance of indicators

• Test well defined

• Test verifiability

• Specific

• Measurable

• Time-bound

• Direct and logical link of indicators and targets

• To mandate, strategic goals and objectives

What do auditors test when auditing measurability?

Measurability of indicators

• Test whether indicators are :

• well defined (Does the indicator have a clear definition and is there technical indicator descriptions for each indicator? )

• verifiable (Is it possible to verify the processes and systems that produce the indicator?)

Measurability of targets

• Test whether targets are:

• Specific (Is the nature and required level of performance clearly identified?)

• Measurable (Can the required performance be measured?)

• Time-bound (Is the time period/deadline for delivery specified?)

What is appropriate audit evidence?

Relevant

Reliable evidence

Appropriate evidence

Examples of relevant and reliable evidence

• Evidence is relevant to planned target and reported achievement e.g. Target is to build a dam but evidence provided is a report on a infrastructure plan to be developed

• The primary sources or origin of the actual performance is available to the auditor

Relevant

• Evidence is more reliable when

• In documentary form – e.g. paper, register, electronic or other media

• Generated through management and information systems with proper controls

• All source information exist and is complete – e.g. a list of houses built that can be verified against completion certificates

• Information is valid – e.g. every entry in register can be traced to a invoice/payment certificate or appropriate record

Reliable

Audit reporting – management report

An audit conclusion will be expressed for ALL municipalities and municipal entities in the management report, on –

USEFULNESS of reported performance information for the selected development priorities/ objectives

RELIABILITY of the reported performance for selected development priorities/objectives

Audit reporting – audit report

REPORT ON OTHER LEGAL AND REGULATORY REQUIREMENTS

Predetermined objectives

Usefulness of information Material audit findings focusing on consistency, relevance and measurability of reported performance information for selected development priorities and objectives

Reliability of information Material audit findings focusing on reliability of reported performance information for the selected development priorities and objectives

Compliance with laws and regulations

Non-compliance findings relevant to the performance management systems and processes

Findings for local government 2013-14 (278 municipalities and 57 municipal entities)

Three-year trend – findings on annual performance report (Improvements noted)

3% (10)

59% (199) 69% (230) 76% (251)

38% (126) 31% (103)

24% (79)

2013-14 2012-13 2011-12

There has been an increase in auditees with no material findings on the quality of their annual performance reports when compared to the previous year.

With no findings With findings Outstanding audits

* Statistics at 20 January 2015

13

120

147

6

38

20

13

36

51

No report

Reliability

Usefulness

Findings on the lack of performance reports as well as the usefulness and reliability of annual performance reports

Finding addressed New finding Repeat finding

Findings on the lack of performance reports as well as the usefulness and reliability of annual performance reports

• The annual performance reports of 55% (167) of the auditees was not useful • The most common findings on usefulness were that auditees reported on

indicators that were not well defined (105) or verifiable (87) with targets that were not measurable (101) or not specific enough (89) to ensure that the required performance could be measured and reported in a useful manner. A total of 99 auditees reported information that was not consistent with the objectives, measures and/or targets in their plans

• Findings on reliability were identified at 52% (158) of the auditees • The most common challenges on reliability were as follows:

No supporting documents or insufficient supporting documents was provided to support the reported targets (Validity)

The amounts, numbers and other data relating to the actual targets reported did not agree to the supporting documentation provided (Accuracy)

The report has missing information, not everything relating to the actual results and events was recorded (Completeness)

Root causes of defective service delivery reports

Inadequate formal planning

for PDO reporting

PDO reporting not included in performance agreements

Structures not capacitated

Organisational structures not

aligned

Inadequate internal audit

testing systems and data

Ineffective audit

committee oversight

Lack of monitoring by

leadership

Poorly defined roles and

responsibilities

Lack of accountability and consequences for actions / inaction

Do not meet regulatory requirements Not useful Not reliable

Recommendations and good practices

Recommendations and good practices

• Policies and procedures to report on performance information should be developed and implemented

• A set of performance indicators that are well defined and verifiable should be developed

• Performance targets that comply with the SMART criteria should be developed • Data definitions for all performance indicators should be defined • CoGTA should support local government by developing customised indicators

and targets • The methodologies and systems used for compiling budgets should be improved

to strengthen the link between budgets and performance targets • The correlation between planned and actual performance (performance

implementation) in relation to the budgeted versus actual expenditure (financial implementation) should be monitored and evaluated

• Predetermined objectives, financial reporting and compliance reporting should be viewed in an integrated fashion as these are inseparable processes

• Auditees need to ensure that they have adequate skills and resources to perform proper strategic planning, performance monitoring and reporting in line with applicable requirements

Recommendations and good practices

• Formal processes and systems for the collection, collation, verification and storing of actual performance information should be developed, documented and approved by municipal manager.

• Auditees should ensure that there is an adequately resourced and functioning internal audit unit, audit committee and MPAC that provides assurance over the quality of the quarterly and annual performance reports

• Management should maintain portfolios of evidence to support reported targets, which are reviewed monthly and audited by the internal auditors.

• There needs to be sufficient oversight and monitoring of performance during the reporting cycle to ensure that performance targets are reported as planned

• Larger auditees should consider a dedicated strategic planning and/or monitoring and evaluation unit

• Auditees should prepare an audit action plan based on the audit findings of the previous financial year, including definite actions to address these audit findings

• Leadership must insist on credible quarterly performance reports that link to key projects undertaken and progress to date

Key controls for local government 2013-14 (278 municipalities and 57 municipal entities)

31

Drivers of internal control

32

Basic controls

Dashboard report indicators

Significant deficiency is not applicable

Significant progress had been made to address it

Urgent attention to the matter is required

Components of the dashboard report

3 Audit dimensions

Fundamentals

of internal control

Financial Performance

objectives Complian

ce with

laws and

regulatio

ns

Assessment

Leadership • Provide effective leadership based on a culture of honesty,

ethical business practices and good governance, protecting

and enhancing the best interests of the entity

• Exercise oversight responsibility regarding financial and

performance reporting and compliance and related internal

controls

• Implement effective HR management to ensure that adequate

and sufficiently skilled resources are in place and that

performance is monitored

• Establish and communicate policies and procedures to enable

and support understanding and execution of internal control

objectives, processes, and responsibilities

• Develop and monitor the implementation of action plans to

address internal control deficiencies

• Establish an IT governance framework that supports and

enables the business, delivers value and improves

performance

Components of the dashboard report

3 Audit dimensions

Fundamentals

of internal control

Financial Performance

objectives Compliance

with laws and

regulations

Assessment

Financial and performance management • Implement proper record keeping in a timely manner to

ensure that complete, relevant and accurate information is

accessible and available to support financial and performance

reporting

• Implement controls over daily and monthly processing and

reconciling of transactions

• Prepare regular, accurate and complete financial and

performance reports that are supported and evidenced by

reliable information

• Review and monitor compliance with applicable laws and

regulations

• Design and implement formal controls over IT systems to

ensure the reliability of the systems and the availability,

accuracy and protection of information

Components of the dashboard report

3 Audit dimensions

Fundamentals

of internal control

Financial Performance

objectives Compliance

with laws and

regulations

Assessment

Governance • Implement appropriate risk management activities to

ensure that regular risk assessments, including

consideration of IT risks and fraud prevention, are

conducted and that a risk strategy to address the risks is

developed and monitored

• Ensure that there is an adequately resourced and

functioning internal audit unit that identifies internal control

deficiencies and recommends corrective action effectively

• Ensure that the audit committee promotes accountability

and service delivery through evaluating and monitoring

responses to risks and providing oversight over the

effectiveness of the internal control environment including

financial and performance reporting and compliance with

laws and regulations

THANK YOU

Questions?

37