PRESENTED BY: RAVEEN SARJU

(SENIOR AUDIT MANAGER : TRANSVERSAL AUDITS )

ETHEKWINI MUNICIPALITY AUDIT AND RISK ASSURANCE SERVICES (EMARAS)

INTERNAL AUDIT

THE AUDITING OF PERFORMANCE INFORMATION (PREDETERMINED OBJECTIVES) PRESENTATION

01 NOVEMBER 2018

AGENDA

NO DESCRIPTION

A. DEFINITION AND PURPOSE OF INTERNAL AUDIT

B. THE DIFFERENT ROLES OF KPI OWNERS , PME UNIT AND INTERNAL AUDIT

C. THE ROLE OF INTERNAL AUDIT

D. AUDIT PROCESS AND QUALITIES OF EVIDENCE

E. THE ROLE OF OVERSIGHT COMMITTEE– AUDIT COMMITTEE

F. AUDIT CHALLENGES

G. TYPES OF AUDIT FINDINGS

H. PERFORMANCE FUNCTIONALITY AUDIT

I. PREPARATION FOR AGSA AUDIT REVIEW

DEFINITION AND PURPOSE OF INTERNAL AUDIT

The MFMA and King report strongly emphasize the need for an Audit Committee

and an Internal Audit function:

OUR PRIMARY CONSIDERATION

Institute of Internal Auditors (IIA)

Internal Audit is an independent, objective, assurance and consulting activity that

adds value to and improves an organization's operations. It helps an organisation

accomplish its objectives by bringing a systematic, disciplined approach to evaluate

and improve the effectiveness of governance, risk management, and control processes.

What is Internal Auditing?

AttributeStandards

PerformanceStandards

Institute of Internal Auditors (IIA)Standards

Implementation Standards

Our values…… our office ambiance…….

Integrity

Accountability

Honesty

Equality

Fairness

Objectivity

Confidentiality

Competency

Values

THE ROLE OF MANAGEMENT / KPI OWNERS

Purpose of Performance Management

Performance Management defined as “a strategic approach to management, which equips leaders,

managers, employees and stakeholders at different levels with a set of tools and techniques to regularly

plan, continuously monitor, periodically measure and review performance of the organization in terms

of indicators and targets for efficiency, effectiveness and impact”

ROLE OF MANAGEMENT & KPI

OWNERS

Developing & Defining KPI,

Project Plans & Setting of KPI

targets,

Capturing, reporting of KPI scores (monthly

& quarterly) on PMS system, Uploading of POE, Review of KPI

scores captured.

Compiling of POE and making available for

audit purposes including source

documents

Attend timeously to audit queries and findings , provide

management comment and action plans

Regular Liaisons with PME Unit regarding KPI performance ,

SMART, definitions, mid terms

amendments, POE issues,

Monthly Business review sessions to

continuously evaluate KPI

performance, relevance of target,

POE etc.,

Role of Management / Plan & KPI Owners

1st Line of Defence

THE ROLE OF PERFORMANCE MANAGEMENT & EVALUATION (PME)

Role of PME in Performance Management

2nd Line of Defence

Develop and Implement Performance Management framework and systems to ensure the reporting of performance information for the Municipality

Ensuring and Advising

KPI owners in :

Aligning of National Strategic Focus areas (SFA’s) to IDP 8 point plan

Aligning of KPI’s on the scorecard to SDBIP and Budget

Developing ‘SMART’ KPI’s

KPI definitions

Developing Project plans and milestones for KPI’S ,

Compiling Portfolio of Evidence

Responsible for the Performance Management systems to facilitate monthly and quarterly reporting –Organisational scorecard

Quality Review of reported information by management

Report monthly and quarterly to City Manager , Council and oversight structures

(Audit Committee , MPAC )

THE ROLE OF INTERNAL AUDIT

Legislative - Audit Mandate

Municipal Systems Act, Act no.32 of 2000, Section 45 states that the results of performance measurements in terms of section 41 (1) (c) must be audited:

As part of the Municipality’s Internal Auditing processes; and

Annually by the Auditor General.

Section 165(2)(b) of the Municipal Finance Management Act, 2003 (Act 56 of 2003) prescribes that the Internal Audit Unit of the Municipality or Municipal Entity

must advise the Accounting Officer and report to the Audit Committee on the implementation of the Internal Audit plan and matters relating to:

Internal Audit;

Internal controls;

Accounting procedures and practices;

Risk and risk management;

Performance management;

Loss control; and

Compliance with the MFMA, the annual Division of Revenue Act and any

other applicable legislation.

Legislative - Audit Mandate

Municipal Planning And Performance Management Regulations, 2001

Section 14. (1) (a) A municipality must develop and implement mechanisms,

systems and processes for auditing the results of performance measurements as

part of its internal auditing processes.

(b) Any auditing in terms of paragraph (a) must include assessments of the

following:

(i) The functionality of the municipality’s performance management system;

(ii) whether the municipality’s performance management system complies with the

Act; and

(iii) the extent to which the municipality’s Performance measurements are reliable

in measuring

performance of municipalities on indicators referred to in regulation 9 and 10

(c) A municipality’s internal auditors must -

(i) on a continuous basis audit the performance measurements of the municipality;

and

(ii) submit quarterly reports on their audits to the municipal manager and the

performance audit

committee

Assess the functionality of the Performance Management &

Evaluation Unit.

Evaluate whether the system complies with applicable Acts.

Determine whether the performance measurements are

reliable in measuring performance.

Continuously audit the performance measurements of the

municipality.

Submit quarterly reports on the audits of performance

information to the Municipal Manager and the performance

Audit Committee.

Internal Audit role in Performance Management System (PMS)

3rd Line of Defence

Risk Based Approach

Quantitative consideration – relate to the Total budget both Capital and Operating budget size) of thedevelopment priority and strategic objectives as reflected in the Integrated Development Plan (IDP).

Qualitative consideration - take into account the information needs of the stakeholders and users of theperformance information, the qualitative significance of the development priorities and strategic objectivesagainst qualitative criteria (basic service delivery, financial sustainability).

Legislative reporting requirements of KPIs – KPIs that require monthly reporting to National Treasury andother external parties

KPIs financed by external or grant funding - KPIs that require compliance with conditions of finance or grantagreement.

Service Delivery Backlogs – KPIs directly relate to the level of service in addressing backlogs, performance ofbusiness against objectives

Frequency of the KPIs in the relevant Performance Management reporting cycle. (quarterly, biannually andannually per plan)

AOPI Audit Objectives

Review performance information as reported in the scorecard and the evidence in support of the performance scores

Determine the accuracy, completeness and validity of performance information reported

Evaluate the targets in terms of the “SMART” criteria

Evaluate the risks and control measures of the systems used to manage and report performance information

THE AUDIT PROCESS AND QUALITIES OF EVIDENCE

AOPI Audit Process

Evaluate portfolio of evidence in terms of Accuracy, Validity and Completeness for those KPI’s reported as achieved.

Confirm the achievement or non- achievement of the

performance target

Conclude on the KPI’s in terms of the ‘’SMART” criteria

Report audit findings to respective plan owners, KPI

owners for response and action plans to address poor

Performance

Determine trend analysis of KPI’s that are continuously

not achieved throughout the performance reporting cycles

Quarterly reporting to the Audit Committee on the results and outcome of

Performance information

Qualities of Audit Evidence

Evaluation of Portfolio of Evidence

• Information is the best attainable information through the use of appropriate methods. Reliable information is competent, valid which accurately represents the status of the KPI being reported.

• Information supports engagement observations and recommendations and is consistent with the objectives for the engagement.

• Information is factual, adequate, and convincing so that the prudent person would reach the same conclusion. The information should have a degree of persuasiveness .

• Information helps the Municipality to meet its goals

Useful Sufficient

ReliableRelevant

Qualities of Audit Evidence

Accuracy

Completeness

Accuracy is free from errors and distortions and are faithful to the underlying facts. The re-performance of a calculation or formulae with the same information should yield the same result as the actual score

presented by the KPI owner.

Completeness include all significant and relevant information that the auditor will be able to verify and validate from source documentation to final result of the KPI status reported and vice versa (i.e. from final

result back to source documentation)

Validity

The information supports the actual score reported based on source documents and actual work completed to fairly represent that status of

the KPI achievement for the reporting period in question.

Specific

Measurable

Achievable

Relevant

Time - Bound

SMART Principles

“SMART” Acronomy

A strategic outcomes oriented goal should ideally be written as a statement of intent that is specific,measurable, achievable and time bound. A useful set of criteria for selecting performance targets is the"SMART" criteria:

• The nature and the required level of performance can be clearly identifiedSpecific

• The required performance can be measured.Measurable

• The target is realistic given existing capacity.Achievable

• The required performance is linked to the achievement of a goalRelevant

• The time period or deadline for delivery is specifiedTime-bound

What Internal Audit Don’t Do?

Do not own the process for reporting or quarterly performance management

Do not develop , design KPIs , project plans , KPI definitions for plan owners

Do not manage the electronic systems used in the reporting of performance information.

Do not produce , maintain a portfolio of evidence for KPIs or sub projects.

Do not make decisions when disputes between KPI owners and PME arise affecting KPI achievement or portfolio of evidence, project plans etc.

THE ROLE OF OVERSIGHT COMMITTEES

Council

Mayor / EXCO

Portfolio/ EXCO

Committee’s

MPAC

Audit and Risk

Committee

Oversight Role

Review and comment on compliance with statutory requirements and performance management best practices and standards.

Review and comment on the alignment of the Integrated Development Plan, the Budget, Service Delivery and Budget Implementation Plan and performance agreements.

Review and comment on relevance of indicators to ensure they are measureable and relate to services performed by the municipality and its entities.

Reviews compliance with in-year reporting requirements.

Review the quarterly performance reports submitted by internal audit.

Review and comment on the municipality’s and entities annual reports within the stipulated timeframes. Review and comment on the municipality's performance management system and make recommendations for its improvement.

Meet Quarterly during the financial year. Review the municipality’s Performance Monitoring and Evaluation system and make recommendations to EXCO & Council.

Role of the Audit Committee in

Performance Management

The Audit Committee will be responsible for

monitoring the assessment by Internal Audit of

the performance management system.

Includes reviewing the strategy and plan to

monitor

Evaluation of the implementation and

effectiveness of performance management

processes and framework; the results of the audit

of performance management and the quarterly

internal audit reports on performance

management and performance information.

Submit audit report to EXCO & Council at least

twice during the financial year.

Audit Committee role in implementation

of Performance Management System

Review Annual report in terms of matters raised by AG

including Performance management and seek accountability

and responsibility in developing their report to Council.

Role of Municipal Public Accounts

Committee (MPAC)

AUDIT CHALLENGES

Audit Plan vs. Reality

Tight reporting timelines to Audit Committee on performance information.

Non Availability and lack of timeous response from KPI owners to audit queries and issues

Weak management responses and action plans to address audit findings.

Poor attendance by Plan owners at Audit committee meetings

Oversight of changes/deletions of KPIs during Midterm.

Challenges

Increased number of KPIs and sub-projects to audit

Extent of audit testing and audit sample size

Decentralisation of POE and source documentation

Culture of Performance management not embedded and integrated into business processes

Client relationships and expectations of KPI owners impacting on future audits

Challenges

34

Resource Planning

TYPES OF AUDIT FINDINGS

Types of Audit Findings Raised

• No Portfolio of evidence submitted for Audit purposes.

• Non Achievement due to Portfolio of evidence with insufficient, irrelevant, or inaccurate information.

• KPIs not meeting the ‘SMART’ principle.

• Pure non-achievement as reflected on the scorecard.

• Lack of Project plans / milestones / KPI definitions.

• KPI Scores not reported on scorecard.

Types of Audit Findings

Raised

• Incorrect weighting of sub projects failure of one sub project leads to total non achievement of the KPI.

• Evidence not uploaded on the Performance Management system.

• Consistent Non achievement of KPI’s in the past 3 years.

• No Targets defined for KPI’s to enable quarterly reporting.

• Absence of KPI definition with No defined linkage to sub-projects and milestones negatively contributing to overall achievement of KPI’s.

• Oversight of changes/deletions of KPIs during Midterm.

• Impact on non achievement on individual performance plans and consequence management.

INTERVENTIONS REQUIRED FROM KPI OWNERS

Identify KPIs performing poorly and

hold relevant Plan & KPI Owners to account

Zoom into areas of pure non-achievement with specific emphasis on

consistently non –achieved KPIs and

recommend actions plans to address

Engage with relevant standing committees

on their areas of oversight that are performing poorly

Performance Bonus cant be paid out when performance Targets

are not Achieved

39

Areas of Improvement & Learnings

Engagement of plan owners and relevant KPI owners on audit outcomes

Monthly action tracking as part of business unit meetings; and quarterly Self Assessment outcome prior to audit

Quarterly business Performance presentations by business to A BUSINESS PERFORMANCE meeting and reporting to Accounting Officer

Engagement with Performance Management Evaluation Unit throughout audit process

AG liaisons on AOPI issues, scope as well as reliance work

40

Impact on Risk Profile

Non-achievement of predetermined objectives indicates a risk to organizational strategic objectives

The reasons associated with not meeting the targets, may also be indicative of other emerging risks

Risk treatment plans to be put in place to expedite delivery

This must inform the quarterly risk reviews

Consideration must be given in the development of the following year scorecard and organizational risk profile

PERFORMANCE AUDIT FUNCTIONALITY

42

Regulatory Audit

This is a mandatory AGSA Regulatory audit for all municipalities in terms of Municipal Planning And Performance Management Regulations, 2001

Section 14. (1)(a) A Municipality must develop and implement mechanisms, systems and processes for auditing the results of performance measurements as part of its internal auditing processes.

(b) Any auditing in terms of paragraph (a) must include assessments of the following:

(i) The functionality of the municipality’s performance management system

(ii) whether the municipality’s performance management system complies with the Act; and

(iii) the extent to which the Municipality’s Performance measurements are reliable in measuring performance of

Municipalities on indicators.

43

Regulatory Audit

A compliance audit of the performance regulations relating to the: • IDP• Budget • Performance Management

Audited using a comprehensive compliance checklist of the relevant regulations and supporting evidence of the process/events/activities that occurred in the previous, current and future financial reporting periods of the Municipality.

44

Regulatory Audit

Audit focus areas:

1. Existing of policies and procedures • Performance Management Framework• Performance Management System • Strategic Policies

2. Compliance with laws and regulations• Municipal Finance Management Act No. 56 ,2003• Municipal Systems Act 32 Of 2000.• Municipal Planning and Performance Management Regulations • Municipal Performance Regulations for Municipal Managers• Municipal Budget and Reporting Regulations

45

Regulatory Audit

3. Monitoring and Reporting process • IDP• Budget • Organizational Scorecard and SDBIP

2. Employees Performance management (Executive Management)• Performance Agreements• Performance Contracts• Performance evaluation/assessments

46

Regulatory Audit Guidelines And Approach

Objective Procedures

Obtain understanding of internal control Environment Is there an official responsible for function? Are roles and responsibilities formalized and implemented? Is there monitoring – senior management and performance/audit committee? Are guidelines/ framework/ templates/formats implemented? Is there adequate segregation of duties?

Information Has policy for development of performance indicators and the recording and reporting of information been

formalized and implemented? Are staff fully aware of policies and been trained?

Monitoring Does management prepare quarterly reports? Does management monitor controls in place to ensure performance information is accurate and complete.

Obtain copy of previous reports to note concerns and follow-up with management to ascertain if any action taken to rectify.

Understanding of system Document system implemented to Establish measurable objectives, performance measures, indicators and targets during strategic and annual

planning process. Collect and record data (frequency/electronic or manual) on quarterly basis and annual reporting Monitor and report on performance information on quarterly and annual basis.

By recording process, designation of person(s) responsible, inputs, outputs, systems, documents, policies and procedures and laws and regulations.

Approved policy and procedures Establish the existence of approved policies and procedures for performance measurement.

Procedures and polices are aligned to laws and regulations

Review legislation, policy and procedures and process implemented to evaluate any deficiency in the policy and or non-compliance with legislation.

Process implemented in accordance with polices and procedures

Perform a walk through on system documented and note any deviations from documented system and procedures and polices.

47

Regulatory Audit Guidelines and Approach

Objective Procedures

Adequate monitoring of system

Review the process documented and results of above to determine if is being adequately monitored by management (review, comparisons, independent checks, exceptions reports, follow-up of corrective action etc)

Consistency and alignment

Compare objectives and KPI’s from IDP to budget to objectives reported in the annual report to ensure consistency

Compare current year measurable objectives to previous year and from quarter to quarter in the current year

Compare a sample of objectives/SFA and KPI’s from organisational scorecard to the CM and individual performance plans of Direct Reportees

Review IDP and scorecard to ensure that SFA/objectives has KPI’s and targets. Inspect measurements to determine if clearly linked to mandate of municipality and Is aligned to priorities and strategies

Quality and presentation

Inspect reported information and determine whether it is easy to follow from objective to KPI and target (straightforward and meaningful and not fragmented)

Inspect to ensure that both financial and non-financial indicators Inspect indicators to ensure meet the SMART principle. Inspect to ensure that actual performance is reported. Inspect annual report to ensure that reasons for deviation from planned is included.

Compliance How does the municipality ensure compliance to laws and regulations. Test for compliance - Refer to listing of legislation.

Other – Has policies, procedures and systems implemented taken into account guidelines as per framework.

PREPARATION FOR AGSA AUDIT

49

Guidelines to Prepare for AGSA Audit Review

Attending and resolving previously reported AG findings and queries and incorporating learnings and lessons to improve efficiencies in the AOPI processes.

Increasing AG reliance on the work of Internal Audit through:• Comprehensive and detailed audit methodology on AOPI • Adequate audit sample size, working papers, evidence etc.• Competent and skilled staff resources

Preferably an automated performance management systems that is reliable and accurate performance information

Regular communication and engagement with AGSA prior and post audit on:• Understanding AGSA approach • Technical matters and updated legislation

50

Guidelines to Prepare for AGSA Audit Review

Clearly defined roles and responsibilities in the Performance management policy & framework between KPI owners,PME and Internal audit including understanding of the AOPI process.

AGSA predominately focusses on service delivery and financial plans / KPIs requires management systems , processand information to be adequate and effective.