PERFORMANCE AUDIT REPORT UTILIZATION BY THE EXECUTIVE AND THE LEGISLATURE

COMPARATIVE REVIEW PAPERFOR

SIXTH ASOSAI WORKSHOPHELD IN TOKYO

DURING 2-13 JULY 1997

Written byMuhammad Akram Khan on behalf of SAI of Pakistan

makram1000@gmail.com

_______________________________________________________________

1. INTRODUCTIONThe present paper is a bare summary of the country papers received on sub-theme 2 for the ASOSAI Assembly held in Indonesia in October 1997. The subject of the sub-theme is:

The SAIs Performance Audit Report and Its Utilization by the Legislature and the Executive

Islamic Republic of Pakistan prepared the principal paper on the sub-theme. The principal paper divided the subject into three parts as follows:

Performance audit report Utilization of performance audit report by the executive Utilization of the performance audit report by the legislature

It discussed these subjects briefly and identified issues relating to each subject. In all, the principal paper raised thirteen issues.

The member countries of the ASOSAI received the principal paper and attempted the position of their respective countries on each issue. The format of the principal paper expected the member countries to develop their position in the same sequence as the issues raised in it. In all fifteen countries responded. The list is at the end of this paper.

While attempting this paper, we have tried to group countries which had similar responses on an issue. This by itself is a formidable task. We have tried to give a brief gist of the country papers. In no way can we claim precision as every country has its own peculiar circumstances and trying to club their position under similar responses can never depict a precisely true picture. At best, it can be a rough summary, trying to identify, in the form of groups, countries which have nearly similar position on a particular issue.

2. GENERAL OBSERVATIONS

2.1 Not all SAIs addressed all the issues raised in the principal paper. It was because in some countries their peculiar circumstances or their legal framework made some issues redundant. While making a summary we have indicated the names of the countries which addressed a particular issue. 2. 2 Israel, Jordan, Indonesia and Saudi Arabia did not follow the format suggested by the

principal paper on the subject. They gave a general description of their constitutional and legal framework, clarifying the relationship of the legislature, the executive and the SAI. These papers did not exactly discuss the issues raised in the principal paper. But in general their papers contained information on the subject. In issue-wise summary we have incorporated their position under respective heads.

3. The country paper from Iraq dealt with a specific question: performance evaluation of higher education in Iraq. It did not take into account the principal paper. The present review paper, therefore, does not summarize the Iraq paper.

3. ISSUE-WISE SUMMARY3.1 Performance Audit Approach

The principal paper identified three approaches to performance auditing: Systemic Approach: It focuses on the systems and controls and lays down audit criteria in the

light of generally accepted management practices. The audit examination compares the criteria with the actual practice.

Effectiveness Attributes Approach: Developed by the CCAF, it focuses on 12 effectiveness attributes on which the management prepares representations and the auditors give an opinion on the position of the management on each effectiveness attribute.

Performance Measurement by Auditors: In this approach, the auditors carry out performance measurement on a pre-agreed measurement scale.

On the basis of this discussion the principal paper raised the following issue:

Issue No.1 Which particular audit approach do you adopt in most of the cases? Do you follow more than one approach? Do you adopt an audit approach which is not covered in the three approaches discussed above. If so, what are its main features and how it affects your audit report?A summary of the responses was as follows:

Australia, Bangladesh, Japan, Israel, Jordan, Korea, Nepal, Pakistan: Combination of systemic and performance measurement approaches. Australia combines all the

three approach according to the situation.

Brunei Darrusalam:

Performance measurement approach is applied.

Cyprus, India, New Zealand, Saudi Arabia:

The systemic approach is applied. Cyprus also uses Effectiveness Attributes Approach occasionally.

3.2 Performance Audit Scope

The principal paper raised the following issue after discussing briefly the importance of performance audit scope and its various dimensions:

Issue No. 2What are the main factors which guide you in determining the scope of audit in your SAI? Does the performance audit report in your SAI describe the scope of audit? If so is it obligatory to do so? Who has made it obligatory? Are there any instances of performance audit reports in your SAI where the scope of audit was limited by some agency or authority?

2

Australia:a) Factors for determining scope of audit: Reasons for conducting an audit helps determine the scope Cost of audit Time availableb) The scope of audit is mentioned in the audit report. It is mandatory.

Bangladesh, Nepal, Pakistan:a) Factors influencing scope of audit:

Financial resources Qualified personnel Audit objectives

b) Scope of audit described in the PSR and sometimes in the audit report as well.c) No restriction on the scope of audit.

Cyprus:a) Factors affecting the scope of audit: Magnitude of activity, project or program Money value Impact of such activity on other programs Availability of staff Expected findings Impact of final report on auditees, executive and legislatureb) The scope of audit is mentioned in the audit report.

India:a) Factors affecting selection of project: Money value Complexity of the scheme Topical interest of the legislative assemblyb) Scope of the audit is not subject to any limitation nor obligatory to mention in the audit report. But in practice it is usually mentioned.

Israel:a) Scope of audit is determined by the following factors: Degree to which other units of the audited body influence the main task unit Degree to which other audited bodies are involved in the issued Availability of time Availability of personnel

b) No limits on the cope of audit by any external body or lawc) Scope of audit is not advised to the audited body in advance to maintain independence of audit

as well to cover uncertain developments during the course of audit.d) The scope of audit is mentioned in the beginning of the report.

Japan:a) Factors affecting selection of project for audit: Availability of data Human and physical resources Time needed for auditb) Scope of audit is mentioned in the audit report.

Jordan:a) Factors influencing selection of project for performance audit:

Visibility and sensitivity of the program under audit Newness of the program or changes in its conditions Expected improvement in accountability and decision-making

3

b) The report discusses the subjects covered.

Korea:a) Factors affecting selection of an audit: Money value Newness Multiyear funding Improper implementation Economy and efficiency Interest of large numbers of people Institution-building Internal controls

b) No limitation on scope of audit. But performance audit of legislature, judiciary and constitutional court not allowed.

New Zealand:a) Scope of audit is determined after carrying out the risk analysis and the benefits from the

proposed audit.b) Each report describes the scope of audit and the reasons for concentrating on particular aspects

of the audit.c) It is obligatory to describe the scope of audit but is usually done to make the report more

understandable.d) The only statutory limitation is on the audit of policies. Other limitations are self-imposed in

each case.

Saudi Arabia:The report discusses the subjects covered.

3.3 Intended Degree of Audit Assurance

The principal paper emphasized the need for specifying the intended degree of audit assurance which a reader of the performance audit report should get. It highlighted the fact that there was as yet no standard format for communicating the degree of assurance to the reader of the audit report. In this perspective it raised the following issue:

Issue No. 3:

In your SAI, how do the performance audit reports communicate the degree of assurance the reader should obtain from them? [You can quote the language used for the purpose of illustration.]

Australia: The audit report mentions that the audit has been carried out according to Australian

Auditing Standards. The audit opinion uses suitable qualifiers to indicate the level of assurance which a user

can obtain from the report.

Bangladesh, Pakistan :There is no mechanism to communicate the degree of assurance the reader should obtain from the audit report.

Brunei, Cyprus, India, Nepal, New Zealand:

The degree of assurance is provided by mentioning the scope and method of audit. Sometimes it is done by comparing similar auditees.

Japan:

4

Quantifies the audit results as far as possible. While presenting the results, classifies the achievement of objectives into different levels, e.g. those projects which achieved 50% of the objects, or those which achieved 30% of the objectives, etc.

Israel:

a) The significance of audit findings depends on the context of the findings, and norms of the user. However, the SAI uses bold type in indicating importance of a finding, places more important findings in the beginning, highlights suspected dishonesty, classifies irregularities into “grave” or “very grave”, etc., sends cases of serious irregularities to Attorney General.

Korea:

The report usually indicates the frequency of the defect noted or quotes some percentage. It avoids reporting in such a manner as may mislead the reader.

3.4 Significance Criteria

The auditors generally agree that the auditors should report only significant matters. It is also agreed that the auditors should try to keep their objectivity and impartiality in reporting their results. For these reasons, it is important that the auditors specify their significance criteria in their reports. On the basis of this general assertion the principal paper raised the following issue:

Issue No. 4:

In your SAI, is there a practice to specify the significance criteria in the audit report? If so, quote some examples of the language in which this was specified.

Australia: The factors in the principal paper influencing auditors about what to report are

concurred. However, in the final analysis it is a judgment of the auditors.

Bangladesh, Brunei: The projects for audit are selected on grounds of significance. The reasons for selecting are

mentioned in the audit reports.

India, Pakistan:

No practice of specifying the significance criteria. However, in Pakistan PSR mentions the reasons for selecting an organization or project for audit.

Cyprus, Jordan, Nepal, New Zealand: The audit report mentions the audit criteria.

Japan:

Does not mention significance criteria, but quantifies the audit findings in terms of money value of the waste, or the possible savings, loss incurred, or expected benefits, etc.

Korea:

No significance criteria for reporting has been laid down. However, matters involving large sums or large number of people or ethical conduct are selected.

3.5 Management Perception

The principal paper argues that the full utilization of the performance audit report by the executive is possible only if the executive aware of its objectives and framework. For this purpose some sort of “marketing” effort is necessary. In the light of this main point, the principal paper raised the following issue:

5

Issue No. 5:In your SAI, what formal and informal efforts have been made to make performance auditing a going concern for the executive departments? How far these efforts have borne fruit?

Australia: Two-way communication Discussion of the findings before formal reporting Communication of annual audit strategy papers to auditees

Bangladesh, Nepal, Pakistan: Seminars, regional conferences, workshops. Has gained acceptability with the executive agencies.Cyprus: Meetings are held with the auditees where the purpose and scope of audit is explained. Positive

character of P.A. is highlighted.Japan: Close communication with the auditees throughout audit to ensure that audit objectives are

understood by them. Understanding of the audited organizations by the auditors and the type of information they

generate. Different levels of scrutiny of audit findings to ensure fairness and objectivity. Korea: Specialists participate in performance audit. Auditors consult researchers and professors for best advice. Discussions with the project managers.

3.6 Auditors’ Competence

One of the conditions for fruitful performance auditing is that the auditors should be fully conversant with its procedures. They should have the imagination, creativity and competence to handle different types of assignments in performance auditing. The issue raised by the principal paper was:

Issue No. 6:In your SAI, what efforts have been made to reinforce the competence of the auditors enabling them to audit technical areas? How far these efforts been successful?

Bangladesh, Brunei, India:Theoretical training, supply of reading material and formation of audit teams with senior members as leaders.

Japan, Korea:

Employment of specialist staff. 19 CPAs, 5 lawyers, 7 tax accountants, 2 real estate evaluators, 84 business management counselors, 2 architects, 11 technical experts, 10 auditors with Ph. D.

Japan also trains its recruits who are fresh from the university. In-service training is also arranged for other levels.

Pakistan:

Training is carried out by specialists of technical fields. Consultants are helping the auditors in the field during audit.

Australia, Israel, New Zealand: Employment of qualified auditors in the first instance. Engaging experts as advisers.

6

Cyprus: Besides accountants, civil and mechanical engineers are also employed by the SAI to help the

auditors.3.7 Recommendations

The performance audit report can make general or specific, short or detailed recommendations. The practice differs in each country. The principal paper raised this issue to synthesize the position.

Issue No. 7:In your SAI, what are the standards for making recommendations? Do you make recommendations in detail or merely identify a problem and indicate direction for action? Is there any system of follow-up on the compliance of the audit recommendations in your SAI?

Bangladesh: No standards for practice. No report has been placed before the legislature so far.

Australia, Brunei, Israel, Japan, Korea: Makes a specific recommendation if the improvement required is obvious. Suggests alternatives. Decision to be made by the auditee. Points out only problems; leaves the remedial action upon the auditee. Japan follows up its previous recommendation until action recommended is taken and case is

closed.

Pakistan: Make general recommendations. System of follow-up exists.

Cyprus, Jordan, Nepal: Indicate areas for improvement. Recommendations are in detail.

New Zealand: No standards. Make general recommendations. The style of recommendation depends on nature of the audit conclusion. Follow-up audits are being considered these days.

3.8 Auditee Response to Recommendations

The executive respond to the performance audit report recommendations in a variety of manner. It could be positive, negative, lukewarm, enthusiastic or indifferent. The principal paper raised the following issue:

Issue No. 8: What has been the reaction, in general, of the executive toward recommendations made in your performance audit reports? Will you like to cite any specific example where the management accepted your recommendations with positive outcomes in the future?

Bangladesh, Brunei: The executive are not fully aware of the context and scope of performance audit. Some are

apprehensive others favor it.

Australia, India, Japan, Pakistan:

7

Generally favorable.

In India, through free and frank discussion, the mutually agreed set of recommendations emerges. The executive takes remedial action, generally, quite promptly. The paper cites examples of 25 cases of a wide variety where the recommendations of the auditors were accepted by the executive.

In Japan, the executive implements all the recommendations, during or after the audit. In case the implementation requires a long term action, the executive does so over years.

The paper from Pakistan cited examples of six recommendation made for improving performance of the Hajj office. The government accepted those recommendations.

Nepal:

Generally positive. The auditors discuss the findings and recommendations with the auditee before including them in the report. Gives example of four recommendations in case of an organization engaged in agricultural research.

Korea, New Zealand:

The recommendations are agreed with the auditees before hand. They react positively and often implement

Jordan, Saudi Arabia:

Royal Orders are issued and cabinet decisions are taken on the basis of recommendations made by the SAI.

Cyprus:

A committee of auditee and SAI staff discusses the recommendation and strategy to implement them. The general reaction of the executive is positive.

Cites example of an audit of Driving Schools where the auditee accepted the recommendations within a week.

3.9 Lessons for the Future

One of the objectives of performance auditing is to learn lessons for the future. But this is possible if we have an institutional arrangement to do so. The principal paper raised this issues as follows:

Issue No. 9:

Is there any institutional arrangement in your country to ensure that the executive departments make use of the recommendations of the audit while planning future projects?

Australia, Bangladesh, Cyprus, Nepal, Pakistan: No institutional arrangement. But in Australia, considerable efforts are made to get the lessons

learned by the executive through dissemination of audit findings, lectures by the audit staff and follow-up audits.

In Cyprus, the Council of Ministers issued a circular in March 96 Prescribing a procedure for dealing with the SAI report. The Ministry of Finance issued a circular to get a compliance report on the decision of the Council of Ministers. A Ministerial Committee along with the Auditor general and the Attorney General will examine the replies of the Ministries to be submitted to the Council of Ministers.

India, Korea:

In India, the executive departments have to report back to the PAC in the form of Action Taken Notes (ATN) on the audit reports. In Korea, the law makes it compulsory for the auditees to take action on the recommendations of the SAI and inform it accordingly. Repeated problems indicate serious and deeper institutional and systemic problems.

Brunei, Japan:

8

The SAIs hold regular meetings with the Ministry of Finance and other executive agencies, explaining to them the audit recommendations. It also takes several measures to educate its clients about the audit findings. The objective of all this is to ensure that the executive departments remain aware of the audit requirements for the future.

New Zealand: The Parliamentary select committees, during the annual review of performance of the

departments ask if the audit recommendations are being followed and if not why. The SAI encourages the select committees to do so.

3.10 Special Focus

The accountability of public servants and that of the politicians involves different sets of problems. The SAIs may consider giving a special focus to an audit assignment, depending upon the nature, scope and background of the auditee. In this regard, the issue was:

Issue No. 10:Are there guidelines in your SAI for the auditors to focus the report on accountability of any particular group, such as ministers, senior civil servants, operational staff?

Australia, Bangladesh, Brunei, Cyprus, India, Japan, Korea, New Zealand, Pakistan, : No guidelines.

In Australia and Brunei, focus is on improvement rather than on personal blame. The report takes into account environmental factors of a program while framing the criticism.

Israel: The SAI considers that it is not possible to delineate responsibility precisely in each case.

Therefore, it addresses the organization in most of the cases and expects the organization to take remedial action. Only in very rare cases, where, it is obvious, the responsibility of individual is fixed by the SAI

Nepal: General guidelines exist to see that the accountability relationships are served properly.

3.11 Systemic Inertia

Despite the best stuff in the audit report and despite the agreement by the executive to implement it , sometimes, the overall system of governance does not allow its implementation. What is the position in individual countries?

Issue No. 11

Are their any examples in your SAI where the audit report could identify the persons responsible for some irregularity yet the system did not permit any action against them? Was it due to unrealistic rules, high turnover of persons, political interference or all of these factors?

Cyprus:Yes. There are examples where action could not be taken against the persons responsible due to: lack of adequate evidence influence of the persons faulty rules lengthy investigation procedure

9

Pakistan: There is no systemic obstacle to take action against the person responsible for some irregularity.

However, Government Servants (E & D ) Rules sometime stand in the way.

Australia, Japan, New Zealand:

Performance audit are not focused on any person in the first instance. Action against a person, if necessitated by the audit, will be taken after due process of law has been observed. There is no systemic obstruction in it.

3.12 Legislative Requests

The legislatures in certain countries request the SAI to carry out investigation into a certain question and to report back in a non-traditional manner. The issue in this regard was:

Issue No. 12:

Do you receive in your SAI any requests from individual legislators for auditing any particular organization and with a specific angle?

Cyprus, Nepal, Pakistan: No references from individual legislators. No formal arrangement exists for liaison with the

Parliament.

Israel:

The SAI receives requests from the State Audit Affairs Committee of the Knesset and is obliged to comply. She also receives requests from individual MPs but is not required to respond or investigate into those requests. It is autonomous in this respect.

Japan:

The Board of Audit remains conscious of the debate in the Diet and selects topical issues. But the BOJ is totally independent in the selection of topic.

Korea:

Only recently, the SAI has started receiving requests from the national Assembly or local councils for investigation into some specific issue or area and the SAI considers these requests.

Australia, New Zealand:

Individual MPs are free to suggest any particular organization or its particular aspect to be audited. The SAI has the final say in deciding about it.

3.13 Partisan Interests

Performance audit report, because of its independence, can also be a good stick in the hands of the interested parties. It is possible, that the report of the auditors is used by the legislature or the executive to whip a certain category of persons. The main issue in this regard was:

Issue No. 13:

Do you have any experience of legislators using your performance report for their partisan interests?

Cyprus, India, Nepal, Pakistan: No such experience.

10

Australia, Israel, Japan, Korea, New Zealand: Audit reports are public documents. MPs are free to use them. Opposition members do use them

for asking questions from the Ministers or in Japan and Korea, for expediting remedial action.

11

LOG OF THE DATES ON WHICH COUNTRY PAPERS WERE RECEIVED BY THE REVIEWER

DATE COUNTRY23.10.96 Pakistan1.12.96 Saudi Arabia11.12.96 Bangladesh21.12.96 New Zealand21.12.96 Nepal29.12.96 Jordan9.1.97 Cyprus

30.1.97 Republic of Korea24.2.97 India28.2.97 Japan19.5.97 Indonesia16.6.97 Israel18.6.97 Australia18.6.97 Brunei18.6.97 Iraq

ASOSAI-2. DOC [19.6.97]

12