approach of the office of the c&ag to examining value for money seamus mccarthy deputy director...
Post on 21-Dec-2015
213 views
TRANSCRIPT
Approach of the Office of the C&AG to Examining Value for Money
Seamus McCarthyDeputy Director of Audit
Office of the Comptroller and Auditor General
Irish Evaluation NetworkFriday 14 December 2007
Objectives
Place VFM examination work by OCAG in the context of accountability and evaluation structures in Ireland
Outline how evaluation challenges are dealt with in VFM examination work
Briefly survey some possible future developments for OCAG assessment of VFM
C&AG’s VFM reporting mandate
C&AG may examine and report on
– the economy and efficiency achieved by audited bodies in the acquisition, use and disposal of resources
– the adequacy of the systems, procedures and practices used by audited bodies to evaluate the effectiveness of their operations
C&AG may make such comparisons as he considers approporate (including comparisons of systems, procedures and practices)
Evolution of the mandate
C&AG role is part of the Westminster model of government
Original focus on financial audit
– Appropriation accounts system
– State agencies’ accounts
Progressively more reports on cases of loss, waste or extravagance e.g.
– Project cost overruns
– Fishery harbours
– Prison planning
Project audits: 1990-1993
Comptroller and Auditor General (Amendment) Act, 1993
The accountability cycle
Comptroller and Auditor General
Comptroller and Auditor General
Committee of Public Accounts
Committee of Public Accounts
Departments/ agencies
Departments/ agencies
Report
Dáil ÉireannDáil Éireann
Report
Information
Public hearing
Minister for FinanceMinister for Finance
InformationMinute
Public auditing principles
Independence
Objectivity/freedom from bias
Wide scope of mandate
Freedom to report publicly
Authority
Certification audit vs Reporting
Certification auditStatutory/obligatory
Output is a certificate/pro forma
Accounts submitted for audit
Numbers based
Standards set
Set audit procedures
Answers are clear
Ongoing relationship with auditees’ staff
ReportingDiscretionary
Output is a report
Nothing formal to ‘audit’
Narrative/analysis based
Standards must be developed
No fixed procedures
No ‘right’ answers
New relationships with auditees’ staff
Organisation of OCAG
C&AGC&AG
Director of AuditDirector of Audit Director of AuditDirector of Audit Secretary and Director of Audit
Secretary and Director of Audit
Vote audits
Vote audits
Semi State audits
Semi State audits
VFM exams
VFM exams
Corporate Services
Corporate Services
Certification audit
Certification audit
Financial management
reporting
Financial management
reporting
Value for money reporting
Value for money reporting
Strategic reporting issues
Reporting governed by ethical/professional standards
Quality assurance
– Expert input
– Internal review/audit of working papers
– Challenge/clearance process
– ‘Cold’ review
Resources
– Internal staffing
• Developing capacity
– Consultants
• Ensuring independence/objectivity
• Understanding of requirements for VFM reporting
Stages in the examination process
Selection of examinations
Selection of examinations
PlanningPlanning
InvestigationInvestigation
ReportingReporting
Follow-upFollow-up
Selection of examinations
C&AG exercises discretion
Strategic themes/priorities e.g.
– Public sector management reforms
– PPPs
– Health sector
Matters arising (from audits, controversies, etc)
Consultation with PAC
Review of potential reporting issues based on
– Risks to value for money in the body under review (performance risks)
– Potential added value of a C&AG report
– Feasibility
Some recent reports
Schools PPP
Waste management in hospitals
Rent supplement
The Financial Regulator
Follow up review of examinations carried out 1997-2000
eGovernment
Drug addiction treatment and rehabilitation
Examination planning
Scope setting - bound by statutory mandate
– Issues to be examined – may include financial management issues
– Time frame
– Financial scope
Analytic framework development
Liaison with auditee(s)
Tests of data availability
Time and budget planning
Formal review of plan by C&AG
Socio/economic statistics
Business targets
Output measures
Activity management
Activity measures
Capacity management
Measures of resource use
Budgeting Financial results
Public sector business model
Financial outturnFinancial outturn
Inputs (resources)Inputs (resources)
Activities/processesActivities/processes
OutputsOutputs
Identify social needsIdentify social needs
Set objectivesSet objectives
BudgetBudget
Planned inputsPlanned inputs
Planned activitiesPlanned activities
Planned outputsPlanned outputs
Planned impactsPlanned impacts
Planned resultsPlanned results
ImpactsImpacts
ResultsResults
Analytic framework
Core VFM concepts (economy, efficiency, effectiveness) are very high level
Risk-based prioritisation of key performance areas
For each performance area, identify
– Performance standard in terms of 3 Es (+ equity)
– Performance measure(s)
– Target(s)
Focus on performance of relevant bodies – things they control or influence, rather than context indicators
Framework set at outset, but may evolve in light of information
Engagement with auditees
Objectives
– Brief management about plans, and get their views
– Identify data sources/data holders
– Agree working procedures
– Resolve difficulties
– Inform management about progress
– Share findings (no surprises principle)
– Get the management perspective
Investigation
Statutory access rights
Data sources
– Official documents
– Literature search
– Administrative data e.g. databases
– Semi-structured interviews (with managers, staff, clients)
– Questionnaires/surveys
– Sampling of cases (e.g. for process review)
Data handling procedures
Analysis
Data checking/validation incl. system review
Forms of analysis used
– Exploratory data analysis
– Descriptive statistics
– Trend analysis
– Regression
– DCF-based financial models
– Activity-based costing
Review/audit of results
Analysis of professional judgements results
Reporting
Target audiences: members of the Dáil; the public; public service managers
Style: non-specialist; concise but comprehensive; clear
Conclude on examination questions
Include
– recommendations for improvement
– examples of transferrable good practice
Formal clearance of drafts
– two (or more) rounds of clearance
Follow-up
Publication of report
Briefing/update for C&AG for appearance at PAC meeting
PAC hearing
Follow-up review or re-audit
Potential impacts (during or post examination)
– Expenditure savings
– Greater efficiency
– Improved service
– Better systems for evaluating effectiveness
Possible future developments
Review of C&AG’s legislative mandate
Reporting of transferable good practice
Ad hoc reference groups for specific evaluations
Development of performance reporting (e.g. Output Statements, VFM reports)
Changing structure of public sector – new agencies, inspectors, regulators, PPPs, community/voluntary sector, etc.
– Revised performance models/accountability regimes
– New issues e.g. gaps in service provision; boundary disconnection; contract/service level management