TCU Recent TCU Recent

Experience in Experience in Evaluating Evaluating

Governmental Governmental ProgramsPrograms

THE FEDERAL AUDIT COURT ( TCU )

Secretariat for Control and Evaluation of Governmental

Programs - SEPROGOctober 2001

Presentation ContentsPresentation Contents

Institutional Mark

The TCU- United Kingdom Project

Future Outlook

G e t t i n g t o k n o w t h e T C UG e t t i n g t o k n o w t h e T C U

Mission: to ensure the effective management of public resources in benefit of the whole society

Legality, legitimacy, economy, efficiency and efficacy in public administration.

Public activities that involve income, spending, originate or extinguish rights or duties.

FOCUS

WHAT

Public service, public or public related organizations and institutions, anyone collecting, keeping or managing public moneys, goods or values.

OVER

ATTRIBUTES OF TCU CONTROLATTRIBUTES OF TCU CONTROL

Geographic CoverageGeographic Coverage

5.561 municipalities

26 states plus the Federal District

172 foreign managing units

2.600 organizations / institutions

3.756 managing units in Brazil

ForeignRelationsMinistryFinanceOfficein New York

Brazilian Embassy

In Australia

Municípality of Serra Nova Dourada/MT (pop 562)

Performance DiversityPerformance Diversity

Audits 877

Congress requests 47

Inspected constructions 197

Denunciations and petitions 1.046

Tried accounts 3.008

Special account tasks 3.016

Admission and concession acts 17.430

Forwarded summons, audiences e notifications

9.412

Privatization processes 40

Evaluation of governmental programs 8

2000

Complexity of AttributionsComplexity of Attributions

• Institutional performance;• Legality of revenue and expenditure

acts;• Privatization program;• Effectiveness of governmental

programs;• Public works;• Financial system;• Governmental performance;• Admissions and concessions inquiries.

Growing DemandGrowing Demand

0

200

400

600

800

1.000

1.200

1998 1999 2000

Denunciations,consultations,representations

CongressRequests

InspectedWorks

Processesforwarded toFederal Justice

Available ResourcesAvailable Resources

Analysts 1.096

Technical and Support Staff

1.024

Total 2.120

Active Personnel

193.742

Other expenses 55.045

Capital 15.550

Total 264.337

Personnel Budget / 2001 R$ millions

Secretarias de Controle

Externo-Sede(6)

Secretarias de ControleExterno-Estados

(26)

Secretaria deFiscalização

deObras e

Patrimônioda União

Secretaria de MacroavaliaçãoGovernamental

Secretaria deRecursos

Secretaria de Fiscalização de

Pessoal

Secretaria deFiscalização de Desestatização

Secretaria de Fiscalização e Avaliação de

Programas de Governo

Secretaria Adjunta de Contas

Secretaria Adjunta

de Fiscalização

Secretaria-Geral de Controle Externo

Assessoria

THE FEDERAL AUDIT COURT

General Secretariatfor Sessions

General Secretariatfor External Control

General Secretariatfor Management

Plenary Sitting

1st Chamber 2nd Chamber

Ministers(9)

Cabinets Cabinets

Auditors(Deputy Min.)

(3)

Vice-Presidency(Corrective Magistrate)Cabinets

Public Ministryattached to TCU

CabinetsCabinet

Presidency

Presidency Secretariat

General Coordination CommitteePlannning and Management Secretariat

Information Technology SecretariatSerzedello Corrêa Institute

Internal Control Secretariat

Legal Consultation Office

Parliamentary Advisory Body

Foreign Relations Advisory Body

Public Relations Office

Ceremonial and Institutional Relations Advisory Body

Structural InnovationsStructural Innovations

Office for External ControlExternal-Headquarters(6)

Office forState Works and Property Control

Office for Governmental Macro-evaluation

Advisory body

Assistant Office for Accounts

Assistant Office for Control

Office for PersonnelControl

Office forPrivatization Control

Office for External Control - States (26)

GENERAL SECRETARIAT FOR EXTERNAL CONTROL

Office for Control and Evaluation of Governmental Programs

Office for Resources

The TCU - United Kingdom The TCU - United Kingdom

ProjectProject

• Training in the 80´s: Office of the Auditor General of Canada

• Training in the 90´s :General Accouting Office (EUA) e

National Audit Office (UK)

Fundação Getúlio Vargas / National Academy of Public Administration (NAPA) / Virginia Polytechnic Institute - 1996

HistoryHistory

Technical Cooperation Technical Cooperation ProjectProject

TCU- UKTCU- UK •Duration: April 1998 to December 2001•Organizations involved:

Tribunal de Contas da União - TCU Foreign Relations Ministry - MRE (Brazilian Agency for Cooperation - ABC) Department for International Development -DFID KPMG

ProjectProject ObjectivesObjectivesContribution for a better performance of governmental organizations and entities, through the implementation of the resulting recommendations of the audits of operational nature accomplished by TCU .

To develop and to disseminate, in the sphere of TCU action, methodologies and techniques of operational nature audit.

DefinitionDefinition

A revision and evaluation modality of the government activity with focus on the results of public management

Audit of Operational Nature Audit of Operational Nature

Program evaluation

Operational audit

Modalities

Audit of Operational Nature Audit of Operational Nature

Mission / Purpose

General objectives

Specific objectives

Functions

Activities

Operational audit economy efficiency efficacy

Program evaluation effectiveness

Impact

ProcessesInputs

Intermediate Results

Needs

Final Results

Socioeconomic problems

ProductsObjectives

Organization / Program / Activity / Project / Policy

Economy

Efficiency

Effectiveness

Sustainability

RE

LE

VA

NC

E

RE

LE

VA

NC

E

Operational Operational AuditAudit vs. vs. Program Program Evaluation Evaluation

Management analysis

Planning Matrix

Common methodologies / techniques: case studies, researches, performance indicators, SWOT analysis, stakeholder

Focus on economy, efficiency and effectiveness

Includes improvement of processes

Specific methodologies / techniques: product map, process map, analysis RECI

Examines management aspects of different programs

Focus on results/effects

Includes cause-and-effectand economical analysis

Specific methodologies / techniques: almost-experimental models, cost-benefit analysis, multiple regression

Uses techniques of operational audit to explain differences in results

Environment Health Education Social welfare Agriculture

Thematic areas (1998-2001) Thematic areas (1998-2001)

Key Points Key Points

Better focus

risk matrix in the selection of the works

planning matrix

Greater methodological rigour

Planning Matrix

Logicalmark

Bench-marking

Process Map

SWOT Analysis

PerformanceIndicat

ors

RECIanalysis

Product Map

Stakeholder Analysis

Key PointsKey Points

Meeting between

training and practice

Key PointsKey Points

Participation of the audited and of internal control in all of the phases of the work

selection - feasibility study planning - planning matrixexecution report - manager's comments monitoring

Key PointsKey Points

Work in partnership

Applied Research Institute - IPEA

Federal Secretariat for Internal Control

Ministry for Planning, Budget and Management

Key PointsKey Points

Identification and dissemination of good practices

Quantification of expected benefits

Evaluation of the impact and of the audit´s cost/benefit relationship

Key PointsKey Points

Building of performance indicators

Monitoring and evaluation of the audit impact

Approach of equity dimension in the audits

Main Results Main Results

training,in the field of the Project, of 80 analysts in new methodologies and techniques of operational nature audit

dissemination of operational nature audit techniques to other civil servants of TCU, of TCE and of the Executive

24 pilot audits done in 1998, 1999, 2000 and 2001

Accomplished audits - Accomplished audits - 1998 1998 Emphasis on managing aspects

Health: acquisition and distribution of medicines against tuberculosis and allocation of resources in the Program of Prevention of the Breakbone Fever (Dengue)

Education: rendering of accounts and transfer of financial resources of FNDE

Environment: IBAMA´s debt collection and control system

Accomplished audits - 1999Accomplished audits - 1999

Health: Increase of the covering of the National Program of Immunization and REFORSUS (reorganization of SUS)

Education: The School Lunch Program (PNAE) and the Textbook Program

Environment: Prevention and Combat of Forest Fires Program (IBAMA) and Control of Radioactive Residues Program (CNEN)

Accomplished audits - Accomplished audits - 20002000Focus on poverty and social inequality reduction

Health: Prevention of the Maternal Mortality and Incentive to the Combat of Nutritional Deficiency (ICCN)

Education: National Program of Computer Science in Education (PROINFO) and School TV Program

Environment: Proágua/Semi-arid and establishment of Desalinators in the Semi-arid Northeast

Audits - 2001Audits - 2001Focus on poverty and social inequality reduction

Health: Health of the Family (PSF)

Education: National School Library and National Student Health Programs

Social welfare: Valuation of Senior Health and Uprooting of the Children Work (PETI ) Programs

Agriculture: Irrigation and Drainage

Internal impacts Internal impacts

growing recognition on the part of the Magistrates and the technical body on the importance of Operational Audits

audit techniques used in the Project incorporated to other audit processes

creation of the Secretariat of Control and Evaluation of Governmental programs

External impacts External impacts

change in the nature of the relationship with the audited organizations

identification of significant opportunities of performance improvement in the public management

recognition by the audited and Congress of the new role carried out by TCU

“... the initiative of TCU in orienting audits towards the evaluation of the quality of federal programs is praiseworthy..."

"Works of that nature must be stimulated in the sphere of the Court, extending the social gains arosen from the external control system performance."

“It is clearly noticeable in this work the intention of the Court to establish partnerships with the involved organizations..."

“Therefore, we understand that the implementation of TCU recommendations will enable clear improvements to the PNCT planning system"

Federal SenateSenate Commission on Social Commission on Social Matters National Program of Control Matters National Program of Control

of Tuberculosis - PNCT of Tuberculosis - PNCT

Challenges Challenges

Implementation of the audit quality control

Monitoring and evaluation of the audits impact

Publication of the results in a friendly way

Enlargement of operational nature audits in TCU

ChallengesChallenges

Incorporation of aspects related with social issues to the operational nature audits

ChallengesChallenges

MissionMission

“To ensure the effective management of public

resources in benefit of the whole society”