risk rating the audit universe

Risk Rating the Audit Universe A critical look at traditional audit universe risk-rating factors

Prepared by: Bruce McCuaig Chief Risk Officer and Principal Consultant

A Paisley White Paper

2

Risk Rating the Audit Universe


INTRODUCTION

One outcome of the Sarbanes-Oxley Act, and the related Public Company

Accounting Oversight Board AS2, and more recently AS5, is more information

in the public domain about the performance (or failure) of internal controls

over financial reporting. The information comes from the hundreds of internal

control deficiencies reported by accelerated filers. Analyzing this data to deter-

mine what kinds of companies reported deficiencies, how deficiencies were

detected, what business processes the deficiencies related to, and what ac-

counts and assertions they impacted provides great insight into how controls

work in modern public companies. This information also provides insight into

the role and performance of internal auditors. Knowledge gained from these

deficiency disclosures may challenge internal auditors’ assumptions about

where risk lies and how to better prioritize an audit universe. Specifically,

can we learn more about how to risk rate an audit universe to better focus

resources on where the deficiencies lie? Big risks can lurk under small rocks,

and the indicators of big risks are often ignored in audit planning. Internal

audit has played an important role in finding and reporting SOX deficiencies,

however, external audit has played a far bigger role. This paper will identify

some areas for improvement.

INTERNAL VS EXTERNAL AUDITOR PERFORMANCE

Internal audit professionals are guided to establish a risk-based audit universe

by the Institute of Internal Auditors International Standards for the Profes-

sional Practice of Internal Auditing and related practice advisories. Currently

under revision, the proposed International Professional Practices Framework

(IPPF) Performance Standard 2010, Planning, states,

“The chief audit executive must establish risk-based plans to

determine the priorities of the internal audit activity, consistent

with the organization’s goals.”

The proposed standard is more explicit than its predecessor, making it

mandatory for the chief auditor to develop a risk-based plan.

There is room for improvement in the execution of a risk-based audit

approach. A recent study published by the Financial Executives Research

Foundation, Control Deficiency Reporting: Review and Analysis of Filings

Table of Contents

Introduction 2

Internal vs. External Auditor Performance 2

How Should Internal Auditors Prioritize Audit 4

Guidance For Improvement 9

About Paisley 10

3


During 2004, analyzes the control deficiency disclosures made by 329 companies

in various SEC filings from November 1, 2003, to October 31, 2004. It analyzes

over 950 disclosures to identify trends to help users of financial statements better

understand the nature of control deficiency reporting made by SEC registrants.

Management and internal auditors appear to have performed poorly in detecting

and reporting deficiencies. Evidence suggests that only about 28 percent of com-

panies were proactively bringing reportable deficiencies to the attention of their

audit committees or external auditors. This strongly suggests that internal audi-

tors either used risk prioritization models that routinely scoped out high-risk areas

for internal control deficiencies or did not detect or report deficiencies that were

found.

More recent statistics confirm this trend. A February 2007 trend alert from Glass

Lewis & Co, a leading investor analyst firm, reported: 2,931 U.S. companies,

about 23 pecent, filed at least one restatement during the last four years; 683

companies restated two or more times.

There is little to suggest that either internal or external auditors are improving

their track record of looking in the right places or finding problems if they exist.

The February 27, 2007, Yellow Card Trend Alert produced by Glass Lewis & Co

titled, The Errors of Their Ways, concluded:

“Companies take note: If you restated, you must have had material weaknesses.

We still have a hard time figuring out how so many companies that restated also

could have reasonably concluded that their internal controls are effective and that

they have no material weaknesses – or that no material weaknesses even existed

at the time of the errors.”

The trend in reported deficiencies is alarming. While individual companies

and their internal auditors may fail to detect or report some internal control defi-

ciencies in audits they conduct, the trend in the total number of restatements and

the number of companies reporting deficiencies, and their late and sudden disclo-

sure suggest a systemic problem. Material weaknesses and significant deficiencies

are simply not being found and reported by management. Restatements continue

at a high level.

Unless internal auditors are applying completely different risk-based standards to

planning audits of internal control over financial reporting, it is reasonable to sug-

gest that the method of prioritizing internal audit activity may be a problem. Is

the error rate experienced in audits of ICFR the same as the error rate in audits of

other areas?

Internal auditors

either used risk

prioritization

models that

routinely scoped

out high-risk

areas for internal

control deficien-

cies or did not

detect or

report deficiencies

that were found.

4

HOW SHOULD INTERNAL AUDITORS PRIORITIZE AUDITS?

The IIA provides practice advisories to assist in the interpretation and imple-

mentation of the Professional Standards. Practice Advisory 2010-2, Linking

the Audit Plan to Risk and Exposures, suggests that the following risk

factors, among others, should be considered:

• Dollar materiality

• Asset liquidity

• Quality of internal controls

• Degree of change or stability

• Complexity

• Management competence

Individual internal audit departments are free to establish their own prioriti-

zation frameworks, however, based on the last several years of publicly

disclosed information; company management and their internal auditors

may have missed the boat on finding and reporting internal control deficien-

cies. The alarming increase in reported deficiencies begs an evaluation of

how the risk factors suggested by the IIA correlate to reported disclosures.

Dollar Materiality as a Risk Factor

Internal audit departments frequently take into account the dollar material-

ity of auditable entities or processes in determining audit risk. If dollar

materiality was a significant factor in internal control deficiencies, one

should expect to see larger companies with more deficiencies or at least

more material weaknesses.

According to the FERF study, the average large cap company (>$1B) in the

sample reported 3.71 deficiencies and the average small cap (<$250M)

reported 2.51 deficiencies; the reporting rate is far less than the size ratio

would suggest. The relationship between dollar materiality and risk is dis-

proportionate to size. As a risk factor, dollar materiality seems to have an

inverse relationship. Entities or processes with low dollar materiality bear a

disproportionate amount of disclosure risk. Billion-dollar companies do not

report four times as many deficiencies as are reported by companies one

quarter as large. Clearly dollar materiality should be a factor, but its weight

should be determined by other factors.


Material

weaknesses and

significant

deficiencies are

simply not being

found and

reported.

5


Asset Liquidity as a Risk Factor

Many internal audit departments are charged with ensuring the safeguarding

of assets and preventing fraud and theft. Liquid assets are perceived to be

particularly vulnerable to fraud and theft. If liquid assets were truly at risk,

one would expect to see a large number of deficiencies related to cash and

equivalents and certain inventories and one would expect to see a large

number of deficiencies related to cash and equivalents and certain invento-

ries and one would expect the existence assertion to be related to many

reported deficiencies. Neither has proven to be true.

According to the FERF study, the following accounts were most frequently

involved in internal control weaknesses: accounts receivable, sales, inven-

tory, cost of goods sold, accrued expenses/reserves, and selling, general

and administrative. Furthermore, according to an analysis of related asser-

tions, the

existence assertion was the one least likely to be attributed to a reported

deficiency in the sample. There is no doubt that liquid assets can be lost or

stolen. But on the whole they have not proven difficult to control and their

existence has not proven to be a significant risk factor for internal control

deficiencies. Internal audit departments may in fact be misdirecting re-

sources by focusing too much attention on liquid assets.

The relationship

between dollar

materiality and

risk is dispropor-

tionate to size. As

a risk factor,

dollar materiality

seems to have an

inverse relation-

ship to risk.

6

Quality of Internal Controls as a Risk Factor

Internal auditors tend to consider the quality of internal controls as a

significant risk factor. In doing so, internal auditors often use the COSO

internal control framework component of control activities as their benchmark

in assessing the existence and quality of internal controls.

One would then expect that a significant number of control deficiencies could

be classified as to control activities. In other words, broken or missing control

activities, if they are truly important, should be behind a significant number

of reported control deficiencies in the FERF study sample. This has not proven

to be true. Where sufficient information made it possible, the authors of the

FERF study classified each control deficiency into its related COSO framework

component. Many deficiencies were so poorly reported as to defy

classification, but of those that were classified, control activities were a

relatively minor category.


… on the whole

[liquid assets]

have not proven

difficult to control

and their exis-

tence has not

proven to be a

significant [SOX]

risk factor As can be seen in Exhibit 2, across the range of companies in the sample,

between 6 percent and 9 percent of reported deficiencies were attributable

to control activities. If the quality of internal control is an important risk

factor, one should expect missing or broken control activities to be associated

with a significant number of control deficiencies. If the lack of evidence of sig-

nificant absences of or breakdowns in control activities suggests they are,

in fact, present and working well in most companies, where are all the

deficiencies coming from? Just how important are control activities as a risk

factor? If internal auditors are using the existence or absence of control

7


… where are all

the deficiencies

coming from? Just

how important

are control

activities as a

risk factor?

activities as evidence of the quality of internal control in risk rating their

audit universe, they may be placing more confidence on these controls

than evidence warrants.

Other COSO framework components seem to be much better predictors of

risk. It seems logical to attribute extra risk to a turbulent, rapidly changing

business environment, but the rate of business change or stability is not

among the deciding factors in determining whether a control deficiency

exists or is reportable. Risk assessment is the COSO framework component

one would expect to see cited as a weakness if the degree of business

change was a factor. Change management is part of the risk assessment

component in COSO. Interestingly, risk assessment is the least cited attrib-

ute when attributing deficiencies to COSO framework components.

Change or Stability Risk Factors

It is not clear if change or stability are reasonable factors. What is clear is

that risk assessment is not being performed adequately. A better factor than

stability or degree of change to consider is whether the auditable entity has

a risk assessment process and, if so, what are its results. Supporting this

argument is a table (Exhibit 3) from the Glass, Lewis & Co. study that

breaks down material weaknesses by type.

8


Whatever the

complexity of the

industry, the vast

majority of

control

deficiencies are

concentrated in

only a few

business

processes.

According to the study, almost 60 percent of material weaknesses are attributable

to financial systems and procedures and personnel. Both categories are likely to

be impacted by rapid change in a business and both suggest a lack of change

management practices. Moreover, risk assessment, with one percent of reported

deficiencies, seems to contradict the notion that instability is a problem.

Business Complexity as a Risk Factor

Internal auditors often assess the complexity of their auditable locations. There is

no standard definition of complexity. Some industries have complex business mod-

els, some have complex technology, and others have complex, nonstandard trans-

actions. Size alone often infers complexity, particularly if it leads to complex corpo-

rate structures or multiple locations. But size has been assessed as a risk

factor and found to be a significant but not determining factor. In fact, one could

argue that disclosure risk decreases with size. Smaller companies tend to have

relatively more internal control deficiencies.

However, another picture emerges when one looks at the breakdown of control

deficiencies reported by business process in the FERF study, as partially excerpted

in Exhibit 4. Whatever the complexity of the industry, the vast majority of control

deficiencies are concentrated in only a few business processes. Period-end report-

ing and revenue cycles account for 58 percent of the deficiencies in the FERF

sample. Are these two processes significantly impacted by technological or operat-

ing complexity? Paradoxically, information systems, often assigned high complexity

scores, accounted for only 5 percent of deficiencies. There is little convincing

9


Management ethics

may be the single

best risk predictor:

• Does management

activity reflect

ethical behavior

(think backdated

stock options)?

• Are earnings

being managed?

• Does management

stress the need for

ethics?

• Does a corporate

code of conduct

exist?

evidence in either study that suggests a subjective assessment of business

complexity, in itself, is a reliable risk factor in prioritizing an audit universe.

Management Competence as a Risk Factor

The control environment component of the COSO framework is the one closest

related to directly dealing with management competence. This COSO control

environment component includes integrity, ethical values, competence and a

range of other factors likely to affect the organization as a whole. As the table in

Exhibit 3 indicates, about 50 percent of all reported control deficiencies can be

attributed to problems with the control environment, making it potentially the

single most significant risk factor in prioritizing the audit universe.

Clearly, of all the factors considered, an assessment of the control environment of

a company or any of its auditable entities should play a major role in prioritizing

an audit universe. Internal control deficiencies are directly and strongly correlated to control environment scores. Soft controls do count. Specifically, gaps in the

following elements of the control environment must be considered as specific risk

factors:

• Integrity and ethical values

• A commitment to competence

• The board of directors or the audit committee

• Management philosophy and operating style

• The organizational structure

• Assignment of authority and responsibility

GUIDANCE FOR IMPROVEMENT

The importance of accurately prioritizing the audit universe is obvious. Until now,

little empirical evidence has been available to test prioritization methodologies.

That is no longer true. Tested against the evidence of publicly reported internal

control deficiencies, many traditional risk factors look extremely questionable at

best. At worst they are causing valuable internal audit resources to be potentially

misdirected.

What is clear is that internal audit plays an integral part in an organization’s

governance, risk, and compliance initiatives and a critical role in providing assur-

ance to the integrality of the organizations governance framework. In an effort to

improve the effectiveness of internal audit processes, history would suggest that

changes need to be made. Recommended changes include:

10

For more information, call 320.286.5870, email [email protected] or visit www.paisley.com


© 2008 Paisley. All Rights Reserved

• A standards-based approach to internal audits will drive greater

consistency and integrity of audit data. Business process improvement is

achievable through a feedback loop of audit results. Financial process

performance should be monitored as a key factor in developing a risk

based plan. Below target performance suggests unidentified risks or

ineffective controls.

• Root cause analysis of internally reported deficiencies and insight into

how control deficiencies are detected and how they impact the entity

are essential if internal auditors want to refine their audit planning and

prioritization models. Root cause analysis is simply not required today

under AS5.

• Greater investment in internal audit processes and systems is a

pre-requisite to any effective governance, risk and compliance initiative.

ABOUT PAISLEY

Paisley is an industry leading software vendor that provides solutions for

governance, risk and compliance (GRC) including financial controls

management, internal audit, operational risk management, compliance, IT

governance, and enterprise risk management. For more than a decade,

Paisley has delivered superior software and services to both large enterprise

and mid-market organizations. Governance, risk and compliance software has

always been and continues to be the company's focus.

Leveraging industry best practices, standards based technology, and a choice

of software platforms and deployment options, Paisley customers are

empowered to improve the accuracy, consistency and efficiency associated

with internal audit, financial controls management, enterprise risk

management, operational risk management, IT governance, and compliance

initiatives. Developed for companies of every size and across multiple

industries, Paisley’s solutions enable organizations to streamline governance,

risk, and compliance processes, reduces costs of compliance, manage and

mitigate risks, and provide visibility, oversight and assurance.

For more information, call 320.286.5870, email [email protected] or

visit www.paisley.com.

risk rating the audit universe

Documents