performance audit adding value
DESCRIPTION
TRANSCRIPT
www.theiia.org/Training
Performance AuditAdding Value
ICGFM Conference May 19, 2011
Lily Bi, CIA, CGEIT, CISADirector, Standards and Guidance
Institute of Internal Auditors
www.theiia.org/Training[2]
Program Objectives
· Understand the Landscape – · Internal Audit· Concept and Benefits of Performance Audit
· Increase your ability to work with management in a positive and constructive partnership
• The International Standards for Professional Practice of Internal Auditing
• Analyze risks and develop a risk-based performance audit
• Learn a value-for-money approach for performance audit
• Final Thoughts – Trend of Internal Audit Profession
www.theiia.org/Training[3]
Program Topics
Unit 1 - Understand the LandscapeUnit 2 - Management Functions and Performance
MeasuresUnit 3 - International Standards For Performance
AuditUnit 4 - Risk-Based Approach (Case Study)Unit 5 - Value-for-Money Approach (Case Study)Unit 6 – Final Thoughts
www.theiia.org/Training[4]
Working Agreement
P = ParticipationO = OpennessS = Sense of funE = Enthusiasm
www.theiia.org/Training
Unit 1Understand the Landscape
• The road map of internal audit profession• The definition of internal Auditing• The definition of performance audit• Benefit of performance audit
www.theiia.org/Training[6]
Road Map of Internal Audit Profession
www.theiia.org/Training[7]
Road Map of Internal Audit
1941 - Internal Audit, a separate and distinctive discipline.
Modern Internal Audit
Single ServiceSingle Client• Review accounting and
financial reports• Serve the management
Multiple ServicesSingle Client• Review accounting,
financial and other operations
• Serve the management
Complex ServicesClients – the organization • Review all critical
functions in an organization
• Play roles in governance, risk management
• Server the organization: Audit Committee and Management
• Increase reliance from external stakeholders
www.theiia.org/Training[8]
About the IIA• Established in 1941, global
headquarters in Altamonte Springs, Florida, USA
• Nonprofit professional association• 170,000 members worldwide• 103 national institutes worldwide• Key focus:
– Standards-setting body for internal auditors
– Professional certifications– Global research center– Principal educator – Global voice for the profession
www.theiia.org/Training[9]
Definition of Internal Auditing
www.theiia.org/Training[10]
Images of Internal Auditors
Which metaphor do you like?• Magnifying glass• Telescope• Compass• Hunting dogs• Watch dogs• Policemen• Consultants• Eyes and ears of the Audit Committee
www.theiia.org/Training[11]
Definition of Internal Auditing
Internal auditing is an independent, objective assurance and consulting activity designed to add value and improve an organization’s operations. It helps an organization accomplish objectives by bringing a systematic, disciplined approach to evaluate and improve the effectiveness of risk management, control, and governance processes.
Source: International Professional Practices Framework (IPPF) The Institute of Internal Auditors
www.theiia.org/Training[12]
Internal Auditing Is
Add Value
Improve Operations
Independent
Objective
Assurance Activity
Consulting Activity
designed to
www.theiia.org/Training[13]
Internal Auditing Helps
Organization accomplish it’s
Objectives
Evaluate
Improve
Risk Management Process
To
Control Process
Governance Process
The Effectiveness of To Help
www.theiia.org/Training[14]
Performance Audit
www.theiia.org/Training[15]
Definitions of PA
• INTOSAI: Performance auditing is an independent examination of the efficiency and effectiveness of government undertakings, programs, or organizations, with due regard to economy, and the aim of leading to improvements.
• US Government Auditing Standards: Performance audits are defined as engagements that provide assurance or conclusions based on an evaluation of sufficient, appropriate evidence against stated criteria, such as specific requirements, measures, or defined business practices. Performance audits provide objective analysis so that management and those charged with governance and oversight can use the information to improve program performance and operations, reduce costs, facilitate decision making by parties with responsibility to oversee or initiate corrective action, and contribute to public accountability.
www.theiia.org/Training[16]
Working Definition of PA
Performance Audit is an independent and objective examination of a program, function, operation or the management systems of a governmental entity to:– assure the entity’s objectives are carried out
in an economic, efficient and effective way, and
– identify opportunity for improvement
www.theiia.org/Training[17]
Financial vs. Compliance vs. Performance Auditing
Financial Compliance Performance
Objective Attest to the fairness of financial statements
Determine the adherence to policies, procedures, laws, and regulations
Evaluate and improve the effectiveness, efficiency, and economy of operations
Information primarily for
LegislatorsStakeholders
Regulators ManagementAudit Committee
Direction of Audit
Looking Back Looking back Looking at the present and to the future
Audits based on
Financial reporting standards such as IFRS
Specific laws and regulations; Government standards of business conduct; internal policies;
Mission, vision, and objectives of the organization and it’s management
Examples Annual audits performed by public accountants - may be supported by specific internal audits
Contract audits; business conduct reviews; audits by banking or other regulators
All other audits such as those of departments, processes, information systems and other functions
www.theiia.org/Training[18]
What Makes this Performance Audit?
An Example:“…to determine whether laws, contracts, policies
and procedures have been properly observed and whether all business transactions were conducted in accordance with established policies and with success. In this connection, the auditors are to make suggestions for the improvement of existing facilities and procedures, criticisms of contracts with suggestions for improvement, etc.”
www.theiia.org/Training[19]
Benefit of Performance Audit
www.theiia.org/Training[20]
Benefit of PA – Adding Value
• Relevant– Focus on the key initiatives
• Flexible – Define the scope of the audit based on
risk• Improving organizational performance• Strengthen the governance• Fraud prevention and detection• Gaining public trust
www.theiia.org/Training[21]
Internal Audit Value
Assurance = Governance, Risk Management, Control
Insight = Catalyst, Analyses, Assessments
Objectivity = Integrity, Accountability, Independence
www.theiia.org/Training[22]
Exercise - Connect the Dots
o o o
o o o
o o o
Connect all nine dots using just 4 lines without taking the pencil off the paper
www.theiia.org/Training[23]
Think Outside the Box
o o o
o o o
o o o
www.theiia.org/Training[24]
Unit 2Management Functions and
Performance Measures
• Understanding the management functions• Seeing the organization through the eyes of
management• Understanding performance measures
www.theiia.org/Training[25]
Management Functions
www.theiia.org/Training[26]
Management Issues and Concerns
• Cost Containment• Human Resources • Values and Vision
Initiatives • Empowered
Environments vs. Traditional Structures
• Technological Changes and Innovations
• Communication• Customer
Satisfaction• Public Perception
www.theiia.org/Training[27]
Management’s Roles
Plan
Organize
Direct
Control Get the Job Done
www.theiia.org/Training[28]
Management’s Roles
www.theiia.org/Training[29]
Performance Auditor’s Roles
• Evaluate the management processes and identify the heart of the problem
• Alert to actual and potential changes• Identify the opportunity for improvement
All units, programs, systems and activities are subject to internal auditor’s evaluations
www.theiia.org/Training[30]
See though the Eyes of Management
Almost every deviation or deficiency results from the
violation of some principle of management or good
administration.
See the organization and its activities through the eyes of management
www.theiia.org/Training[31]
Three Simple Questions to Ask Management
• What can go wrong?• How do you it won’t go wrong?• So what?
www.theiia.org/Training[32]
Performance Measures
www.theiia.org/Training[33]
Types of Management Performance Measures
• INPUTS - Measures of service efforts, e.g., number of hours, amount of materials.
• OUTPUTS - Measures of service level, e.g., number of residences served, amount of service provided.
• OUTCOMES - Measures of service accomplishments, e.g., measures related to program goals, including effectiveness of quality.
• EFFICIENCY - Measures that relate service efforts to service accomplishments, e.g., output/unit of input, productivity indexes.
www.theiia.org/Training[34]
Principles
• Measure only what are important to the organization
• Use of output-oriented measures• Identify the total costs of service delivery• Focus on continuous process improvement• Performance measures should interconnect
throughout the organization
www.theiia.org/Training[35]
One Example – Five Performance Categories:
• Effectiveness – the degree to which process output conforms to requirements
• Efficiency – the degree to which the process produces the output at a minimum cost of resources
• Quality – the degree to which the product or service meets customer expectations
• Timeliness – the degree to which a unit of work was done correctly and on time
• Safety – the measure of health and the working environment of the organization
www.theiia.org/Training
Unit 3International Standards For Performance Audit
International Professional Practices Framework - IPPF from the IIA
www.theiia.org/Training[37]
Why the Standards Matter
The Standards
Advancement of the Profession
Lead Represent
www.theiia.org/Training[38]
Road Map of Internal Audit- Changes to the IIA Standards
Single ServiceSingle Client• 1947 Statement of
Responsibilities of the Internal Auditor
Multiple Services Single Client• 1957, 1971 and 1976
Statement of Responsibilities of the Internal Auditor
Complex ServicesClients - the Organization • 1978 The Standards for
the Professional Practice of Internal Auditing
• 1999 New Definition of Internal Auditing
• 1999 Professional Practice Framework (PPF)
• 2009 International Professional practices Framework (IPPF)
www.theiia.org/Training[39]
International Professional Practices Framework
The IIA’s IPPF
www.theiia.org/Training[40]
AUTHORITATIVE Guidance
Mandatory
Strongly recommended
Authoritative =
www.theiia.org/Training[41]
Code of Ethics• Integrity
– The integrity of internal auditors establishes trust and thus provides the basis for reliance on their judgment.
• Objectivity– Internal auditors exhibit the highest level of professional objectivity
in gathering, evaluating, and communicating information about the activity or process being examined. Internal auditors make a balanced assessment of all the relevant circumstances and are not unduly influenced by their own interests or by others in forming judgments.
• Confidentiality– Internal auditors respect the value and ownership of information
they receive and do not disclose information without appropriate authority unless there is a legal or professional obligation to do so.
• Competency– Internal auditors apply the knowledge, skills, and experience
needed in the performance of internal auditing services.
www.theiia.org/Training[42]
International Standards for Professional Practice of
Internal Auditing
www.theiia.org/Training[43]
• They define the profession.• They set the bar that every auditor
should comply with.• They give you a reference guide
for how to conduct yourself.• They lay the ground work, but are
not the ultimate goal.• They give our customers peace of
mind and confidence they’re getting a quality product.
Importance of the Standards
www.theiia.org/Training[44]
The International Standards
• Mandatory requirements consisting of:– Statements of basic requirements for
professional practice of internal auditing
– Interpretations which clarify terms or concepts within the Statements.
– Glossary26 changes
effective Jan
2011
www.theiia.org/Training[45]
Overview of the IIA Standards
Attribute Standards: Purpose, Authority and Responsibility……………………1000 Independence and Objectivity………………………………..1100 Proficiency and Due Professional Care……………….….1200 Quality Assurance and Improvement Program……..…1300
Attribute Standards: Purpose, Authority and Responsibility……………………1000 Independence and Objectivity………………………………..1100 Proficiency and Due Professional Care……………….….1200 Quality Assurance and Improvement Program……..…1300
Performance Standards: Managing the Internal Auditing Activity……………………2000 Nature of Work.……………………………………………….…………2100 Engagement Planning…………………………………….……..…2200 Performing the Engagement…………………………..……… 2300 Communicating Results………………………………..….………2400 Monitoring Progress………………………………………….……. 2500 Resolution of Management’s Acceptance of Risks……..2600
Performance Standards: Managing the Internal Auditing Activity……………………2000 Nature of Work.……………………………………………….…………2100 Engagement Planning…………………………………….……..…2200 Performing the Engagement…………………………..……… 2300 Communicating Results………………………………..….………2400 Monitoring Progress………………………………………….……. 2500 Resolution of Management’s Acceptance of Risks……..2600
www.theiia.org/Training[46]
IIA CBOK 2006 - Figure 2-1
Important Knowledge for Satisfactory Performance Of Internal Auditing
2010 IIA Global Internal Audit Study
www.theiia.org/Training[47]
Who Uses the Standards
• Mandatory requirements for 170,000 IIA members and 100,000 Certified Internal Auditors
Translated into 21 languages
• Recognized or referenced by International Standards Setting Bodies, such as:
INTOSAI (IIA Standards are recognized globally for public sector audit professions)
Basel Committee on Banking Supervision OECD Internal Audit Function
• Referenced on the mandated legislation or regulation in countries or territories, such as
Belgium, Bosnia & Herzegovina, Canada, Chinese Taiwan, Estonia, Poland, Romania, South Africa, Sweden, Thailand, Tunisia, Unites States, United Kingdom, Zimbabwe, and …
www.theiia.org/Training[48]
IPPF Strongly Recommended Guidance
• Practice Advisories (56)Address approach, methodology and considerations, but NOT detailed processes and procedures. Concise and timely guidance to assist internal auditors in applying Code of Ethics and Standards and promoting good practices.
• Position Papers (2)IIA statement to assist a wide range of interested parties, including those not in internal auditing profession, in understanding significant governance, risk or control issues and delineating related roles and responsibilities of internal auditing.
• Practice Guides (26)Detailed guidance for conducting internal audit activities. Includes detailed processes and procedures, such as tools and techniques, programs, and step-by-step approaches, including examples of deliverables.
www.theiia.org/guidance
www.theiia.org/Training
Unit 4Risk-Based Performance Audit
• Performance audit process
• The importance of clearly defined business objectives and associated performance measures (goals) to a performance audit
• Risk assessment using a Risk/Control Matrix methodology
• Case Study
www.theiia.org/Training[50]
Performance Audit Process
• Planning • Examining and Evaluating Information• Communicating Results• Following Up
www.theiia.org/Training[51]
IIA Standards Related to Performance Audit Process
www.theiia.org/Training[52]
Plan Performance Audit
• The most important part of an audit is the planning phase.
• Standard 2010 – Planning: The chief audit executive must establish risk-based plans to determine the priorities of the internal audit activity, consistent with the organization’s goals.
www.theiia.org/Training[53]
• Standard 2201 – Planning Considerations: In planning the engagement, internal auditors must consider:– The objectives of the activity being reviewed and the means by
which the activity controls its performance;– The significant risks to the activity, its objectives, resources, and
operations and the means by which the potential impact of risk is kept to an acceptable level;
– The adequacy and effectiveness of the activity’s risk management and control processes compared to a relevant control framework or model; and
– The opportunities for making significant improvements to the activity’s risk management and control processes.
Plan Performance Audit
www.theiia.org/Training[54]
Risk-based Performance Audit
• Start with an organization’s objectives and associated performance measures.
• Focus on an evaluation of performance risks and controls related to those objectives.
• Help the organization achieve the desirable goals and protect it from bad or undesirable things happening.
• Help reduce the chance of missed opportunities.
• Provide suggestions for improvement in controls designed to mitigate the risks associated with meeting performance objectives.
www.theiia.org/Training[55]
Risk Assessment Formula
Objective Risks Controls
www.theiia.org/Training[56]
Identification of Objectives
Objectives are the things an organization wants to
accomplish.
Objectives should be S.M.A.R.T.
www.theiia.org/Training[57]
Objectives Cascade
Mission
Vision
Objective 3Objective 2Objective 1
Sub-Objective
Sub-Objective
Sub-Objective
Sub-Objective
Sub-Objective
Sub-Objective
Sub-Objective
Sub-Objective
Sub-Objective
www.theiia.org/Training[58]
What is Risk
• Risks are things that could prevent an organization from meeting its objectives.
• IIA definition - Risk is the possibility of an event occurring that will have an impact on the achievement of objectives. Risk is measured in terms of impact and likelihood.
www.theiia.org/Training[59]
Business Risk Examples
1. Erroneous records and/or information2. Business interruption (Government shutdown)3. Public criticism or legal action4. High costs5. Loss or destruction of assets6. Customer dissatisfaction due to ineffective
program/service design7. Fraud or conflict of interest8. Inappropriate mgmt. policy and/or decision making
process
www.theiia.org/Training[60]
Strategic & Business 60% Operational 20%
Financial 15% Compliance 5%
Focusing on the “Real Risks”
www.theiia.org/Training[61]
Risk Assessment
Total Audit Universe
High
Low
Likelihood
Ris
k I
mp
ac
t
L
H
H
www.theiia.org/Training[62]
Risk Responses
Examples of risk response options:• Acceptance• Avoidance• Transfer• Mitigation
www.theiia.org/Training[63]
Risk Response Strategy
• Management identifies available risk response options
• Considers their effect on event likelihood and impact, in relation to risk appetite and cost versus benefit
• Effective enterprise risk management does not dictate which response management should chose, but that the chosen response brings the expected likelihood and impact within the desired risk tolerances
www.theiia.org/Training[64]
Risk Assessment - Two perspectives
• Inherent (Gross) - BEFORE RISK RESPONSE• Residual (Net) - AFTER RISK REPONSE
Inherent Risk
Responses Residual Risk
www.theiia.org/Training[65]
Exercise: Rain and UmbrellaWhen it rains, where are Inherent and
Residual Risk (IR and RR)?
www.theiia.org/Training[66]
When it rains, where are IR and RR?
IR IR IR IR
IRIR
IR
RR
RR
RR
RR
RR
IR = All the raindropsRR = The raindrops outside the umbrellaCR = Control Risk, possibility the umbrella leaksRisk Appetite = How big the umbrella is
CR
www.theiia.org/Training[67]
What is Control
• Controls are things that help meet an organization's objectives.
• IIA Definition Control - any action taken by management, the board, and other parties to manage risk and increase the likelihood that established objectives and goals will be achieved. Management plans, organizes, and directs the performance of sufficient actions to provide reasonable assurance that objectives and goals will be achieved.
www.theiia.org/Training[68]
Control to Mitigate These Risks
1. Erroneous records and/or information2. Business interruption3. Public criticism or legal action4. High costs5. Loss or destruction of assets6. Customer dissatisfaction due to ineffective
program/service design7. Fraud or conflict of interest8. Inappropriate mgmt. policy and/or decision making
process
www.theiia.org/Training[69]
Risk Management and Control
• Two sides of the same coin: – Risk is managed by having in place the right controls
to safeguard against its occurrence;– Internal control exists only in relation to what they do
to mitigate risk.
• Risk management and internal control are integrated parts of an entity’s overall governance and management system.
www.theiia.org/Training[70]
Control - Who Is Responsible
• Management is responsible to design, implement and monitor controls
• Internal auditors is responsible to assess the adequacy and effectiveness of controls
www.theiia.org/Training[71]
Risk Control Matrix
Use RCM to • Plan an audit• Document an audit
Objectives Risk Control
Name Likelihood Significance Ranking Name Evaluate Adequacy
Test Effectiveness
www.theiia.org/Training[72]
Benefits of Risk Control Matrix
• Open-ended• Disciplined• Risk-based• Inclusive
Most organizations modify, delete, and add columns on the Risk/Control Matrix to fit their own environment.
www.theiia.org/Training[73]
Validate the Audit Plan
Total Audit Universe
High
Low
Mandated
Likelihood
AUDIT RESOURCES
Ris
k I
mp
ac
t
L
H
H
*
Special Request
www.theiia.org/Training[74]
Case Study
State Department of Fruit and Vegetable
www.theiia.org/Training
Unit 5Value for Money Approach
• Why Value-for-Money approach?• Three E’s Performance Measures• Difference between Risk-Based and Value-for-Money
approaches• Twelve Attributes for Evaluating Effectiveness• Case Study
www.theiia.org/Training[76]
Needs for Performance Audit
To evaluate a unit or program and answer questions like:• Do we get value for money?• Is it possible to spend the money better or
more wisely?• Are the right things been done?• If so, are things been done in the right way?• If not, what are the causes?
www.theiia.org/Training[77]
Value-for-Money
• Definition: VFM is utility derived from every purchase or every sum of money spent. VFM is based not only on the minimum purchase price (economy) but also on the maximum efficiency and effectiveness of the purchase.
• Looks at how well an organization provides value for money.
• Focuses on economy, efficiency, and effectiveness• Based on the Twelve Attributes for Evaluating
Effectiveness
www.theiia.org/Training[78]
Audit Performance Measures – 3E’s
• The principle of ECONOMY is keeping costs low. It requires that the resources used by the audited entity for its activities shall be made available in due time, in appropriate quantity and quality and at the best price.
• The principle of EFFICIENCY is getting the most from available resources. It is concerned with the best relationship between resources employed, conditions given and results achieved.
• The principle of EFFECTIVENESS is meeting the objectives set. It is concerned with attaining the specific aims or objectives set and/or achieving the intended results.
www.theiia.org/Training[79]
12 Attributes For Evaluating Effectiveness
1. Management Direction2. Relevance3. Appropriateness4. Achievement of
Intended Results5. Acceptance6. Secondary Impacts
7. Costs and Productivity8. Responsiveness 9. Financial Results10. Working Environment11. Protection of Assets12. Monitoring and
Reporting
www.theiia.org/Training[80]
Conducting Performance Audit- Planning
• Gather background information on the audit area.• Understand the organization’s business, objectives,
mission, etc.• Interview management and staff.• Use the twelve attributes to scope the audit by looking at
each attribute to choose which are most applicable.• For the selected attributes, form questions to be
answered during the next phase.
www.theiia.org/Training[81]
Conducting Performance Audit- Examining and Evaluating
• The questions are answered through:- Interviews with management, employees and
others- Industry research- Performance measures (criteria)- Benchmarking (criteria)- Other management and audit reports.
- Site visits.
www.theiia.org/Training[82]
Conducting Performance Audit- Reporting and Following Up
Communicating Results Phase• Issues should be communicated to client throughout the
audit.• The report is written and presented to the client.
Following Up• Management implements action items from the report.
Audit assists as required.
www.theiia.org/Training[83]
Case Study
State Department of Fruit and Vegetable
www.theiia.org/Training
Unit 6Final Thoughts
• Summary of What We Discussed• Internal Audit - Today and Tomorrow
www.theiia.org/Training[85]
Summary
• Understanding of internal audit and performance audit
• Performance measures• IIA’s International Professional Practices
Framework (IPPF)• Management functions• Risk-based performance audit• Value-for-money performance audit
www.theiia.org/Training[86]
Modern Internal Auditing• Client-focused, value-added service to management and
oversight bodies• Guided by international standards and enhanced emphasis
on quality• Adoption of risk-based methodologies• Consulting service + assurance service• More independence and enhanced stature• Add value to the organization and stronger alignment• More strategic approach to staffing: out-sourcing and co-
sourcing• Integration of IT and non-IT audit resources• Enhanced use of technology tools/services• Started to be part of governance structure
www.theiia.org/Training[87]
Top 5 Internal Audit Activities Today
• Operational auditing (89% of respondents).
• Audits of compliance with regulatory code (including privacy) requirements (75% of respondents).
• Auditing of financial risks (72% of respondents).
• Investigations of fraud and irregularities (71% of respondents).
• Evaluating the effectiveness of control frameworks (i.e., using COSO and COBIT) (69 percent of respondents).
2010 IIA Global Internal Audit Study
www.theiia.org/Training[88]
What Is Next? Top Five Imperatives
• Assess and align with key stakeholder expectations
• “Step up to the plate” in risk management
• Enhance internal audit knowledge of the business
• Streamline internal audit processes and operations
• Coordinate and align with other risk, control and
compliance functions
www.theiia.org/Training[89]
Performance Audit Adds Value By
• Reducing risk exposure
• Improving opportunities to achieve goals
• Identifying operational improvement