Under the newSEC rulesimplementing

Section 404 of the Sar-banes-Oxley Act, man-agement is required toreport on the “effec-tiveness” of their com-pany’s internal control.But how can you tell what levelof “effectiveness” your internalcontrol operates at and, moreimportantly, whether this level isacceptable?

The Committee of Sponsor-ing Organizations of the Tread-way Commission (COSO) frame-work provides some guidance,describing the following fivecomponents of internal control:

1. Control environment.Senior management must setan appropriate “tone at thetop” that positively influ-ences the control conscious-ness of entity personnel. Thecontrol environment is thefoundation for all other com-ponents of internal controlsand provides discipline andstructure.

2. Risk assessment. The entitymust be aware of and dealwith the risks it faces. Itmust set objectives, integrat-

ed throughout all value-chain activities, so that theorganization is operating inconcert. Once these objec-tives are set, the entity mustthen identify the risks toachieving those objectivesand analyze and developways to manage them.

3. Control activities. Controlpolicies and procedures mustbe established and executedto help ensure the actionsidentified by management asnecessary to address risksare effectively carried out.

4. Information and communi-cations. Surrounding thecontrol activities are infor-mation and communicationsystems, including theaccounting system. Thesesystems enable the entity’speople to capture andexchange the informationneeded to conduct, manage,and control its operations.

5. Monitoring. Theentire control processmust be monitored andmodifications made asnecessary. In this way,the system can reactdynamically, changingas conditions warrant.

But even a detailed, thoroughreading of COSO may fail to pro-vide you with the practical guid-ance necessary to make anassessment about internal controleffectiveness.

In this article, I describe a“reliability model” for assessingcontrol effectiveness. Your exter-nal auditors may use a similarmodel to plan and perform theiraudit of internal control. Bybecoming familiar with themodel described here, you maybe better able to plan your inter-nal control testing and, ultimate-ly, evaluate the results of thosetests and reach a conclusionabout the overall effectiveness ofyour company’s internal control.

TWO DIMENSIONS OFEFFECTIVENESS

The overall objective of yourinternal control project is to pro-vide management with a basis

To implement Section 404 of the Sarbanes-OxleyAct (SOX), management must report on the “effec-tiveness” of the company’s internal control. But howcan you gauge the level of effectiveness? And howcan you tell if that level is acceptable? The authoroffers some solid answers. © 2004 Wiley Periodicals, Inc.

Michael Ramos

Just How Effective Is Your InternalControl?

featu

reartic

le

29© 2004 Wiley Periodicals, Inc.Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/jcaf.20053

for making an assertion aboutthe effectiveness of the entity’sinternal control. Consider thefollowing statement about effec-tiveness:

Sue is the most effectivepoint guard in basket-ball.

This statement seemsstraightforward, but upon furtherconsideration, you are likely toask two important questions:

1. Effective compared towhom? In this example, theperson making this state-ment would probably becomparing Sue to herpeers—others who play hersame position, at the samelevel of competition.

2. Effective measured how?Basketball, like most othersports, has a variety of sta-tistical and nonstatisticalmeasures of a player’s effec-tiveness. The person makinga claim about Sue could

point to measures such asthe average number ofassists or turnovers per gameor a variety of measuresrelated to scoring points as away to measure effective-ness.

Similarly, assertions aboutthe effectiveness of internal con-trol must be supported along twodimensions:

1. Effective compared to what?Typically, the entity’s inter-nal control will be comparedto COSO as one means ofassessing effectiveness.

2. Effective measured how?COSO provides an overallframework of the five inte-grated components of inter-nal control. But it provideslittle guidance on how tomeasure relative effective-ness. There are no common-ly accepted measurementtechniques for internal con-trol (as there are for basket-ball players). For example,

COSO identifies integrityand ethical values as beingan important piece of theentity’s control environment,and the report makes a com-pelling argument for whythis is so. But the reportdoes not describe how tomeasure or otherwise evalu-ate whether an ethical cli-mate is “effective.” This arti-cle introduces an “internalcontrol reliability model”that can be used as a tool formaking such an evaluation.

Exhibit 1 describes this two-dimensional process for evaluat-ing internal control effective-ness.

In Exhibit 1, across the hori-zontal axis are the entity’s signif-icant control objectives, whichmirror the COSO framework.The vertical axis depicts theinternal control reliability model.As indicated, this model has fivedifferent levels of effectiveness.

Exhibit 2 provides an exam-ple of a matrix that has been

30 The Journal of Corporate Accounting & Finance

© 2004 Wiley Periodicals, Inc.

Exhibit 1

A Two-Dimensional Process for Evaluating Internal Control Effectiveness

completed at the conclusion ofone test of effectiveness. Thecontrols for each significant con-trol objective have been evaluat-ed and “scored” for effectivenessbased on the reliability model.The result is a visual interpreta-tion of the effectiveness of theentity-level controls taken as awhole.

THE RELATIONSHIP BETWEENTESTING AND EVALUATION

The way in which you planto evaluate effectiveness ulti-mately drives the design of yourengagement tests. Using themodel summarized in Exhibit 1,you should plan your tests toinclude each of the significantcontrol objectives identifiedacross the horizontal axis. Addi-tionally, the nature and scope ofyour tests should be sufficient toallow you to evaluate the controlreliability level as indicated bythe vertical axis.

This article provides guid-ance on the testing and evalua-

tion of internal control. In prac-tice, you will test first and thenevaluate. However, because thedesign of tests is so dependenton how you will evaluate effec-tiveness, this article will presentguidance on evaluating testresults before providing guidanceon the design and performanceof the tests themselves.

Following is a discussion ofthe second dimension of assess-ing effectiveness: the internalcontrol reliability model.

INTERNAL CONTROLRELIABILITY MODEL1

Five Levels of Reliability

Over time, as businessesexpand and change, their inter-nal control evolves. What startsout as a relatively informalprocess can mature and becomemore well defined and reliable.Exhibit 3 summarizes this devel-opment process. It identifies fivedistinct levels of internal control

reliability, as indicated in theblue pathway. The model alsodescribes what entities must doin order for their systems toevolve from one level to thenext, as indicated in the lightlyshaded boxes in Exhibit 4.

Referring to Exhibit 3, wenote five levels of evolution ofinternal controls:

Reliability Level 1: Initial.Control objectives are not welldefined or consistently under-stood throughout the organiza-tion. Policies and procedures aread hoc and generally undocu-mented. As a result, control poli-cies and procedures generallyare not linked to objectives orare inconsistent with each other.The effectiveness of an initialsystem depends almost exclu-sively on the skills, competence,and ethical values of the individ-ual. Because of this dependenceon the individual rather than theorganization, the reliability ofthis system can vary greatly overshort periods of time or amongbusiness units.

September/October 2004 31

© 2004 Wiley Periodicals, Inc.

Exhibit 2

An Effectiveness Matrix

Reliability Level 2: Infor-mal. Common, intuitive controlpractices begin to emerge, butdocumentation is sporadic andinconsistent. Informal communi-cation of information aboutinternal control matters exists,but the lack of formal communi-cation methods, together with alack of training, prevents muchof this information from reach-ing below the manager andsupervisory levels. Managementis aware of the need for controlsbut still views internal control asseparate from, not integral to,the operation of the business.However, the emergence ofrepeatable processes andimproved communication anddissemination of informationimproves the reliability of thesystem and reduces risk.

Reliability Level 3: System-atic. Management understandsthat internal control is an inte-

gral part of the company’s busi-ness and that maintaining aneffective system is one of theirprimary responsibilities. Theybegin to devote substantialresources in a coordinated effortto develop and maintain morereliable internal controls. Indi-vidual control components com-bine into a cohesive whole. Doc-umentation of control policiesand procedures is comprehensiveand consistent; some trainingexists on control-related matters.With more formal, standardizedcontrols in place, the system ismore reliable, as its overalleffectiveness depends more onthe organization and less on thecapabilities of the individual.

Reliability Level 4: Integrat-ed. Management understands thefull requirements for maintainingan effective system of internalcontrol. Control implications andissues are routinely considered

as business decisions are evalu-ated and made. Controls arefully integrated into the strategicand operational aspects of thebusiness. Comprehensive train-ing exists. The company begins aformal process for the regularmonitoring of the effectivenessof internal control.

Reliability Level 5: Opti-mized. Management commits toa process of continuousimprovement of controls. Theentity uses automation andsophisticated tools and tech-niques to monitor controls on areal-time basis and makechanges as needed.

Exhibit 4 summarizes theInternal Control ReliabilityModel along five different char-acteristics used to gauge systemreliability.

WHAT THE MODEL CAN ANDCANNOT DO

The internal controls relia-bility model provides you with aframework for:

• designing tests of controleffectiveness,

• evaluating the effectivenessof controls, and

• presenting and discussingyour findings with yourclients.

Recognize that the bound-aries between the various levelsare hazy and that the levels ofreliability themselves may not bediscrete. In reality, componentsof an entity’s internal controlmay exhibit qualities of morethan one of the identified levels.The purpose of the model is notto determine the proper way tocategorize an entity’s controls,but rather to have a basis foranalyzing control effectivenessto determine whether controlsare capable of achieving their

32 The Journal of Corporate Accounting & Finance

© 2004 Wiley Periodicals, Inc.

Exhibit 3

How Internal Control Evolves

ultimate aim—to reduce to anacceptable level the risk thatmaterial misstatements to thefinancial statements will goundetected.

NOTE

1. This model has been adapted from the“capabilities maturity model” for soft-ware development, which was devel-

oped by the computer software com-munity with stewardship and resourcesprovided by the Software EngineeringInstitute. The model provides a basisfor judging the capabilities of a soft-ware development process, and it alsoidentifies key practices required to fur-ther the maturity of this process.

Since its introduction, the maturi-ty model approach has been adopted bythe Control Objectives for Informationand Related Technology (COBIT) as a

way to assess the effectiveness of IT-related controls. More recently, severalpublic accounting firms have recom-mended this type of approach to theirclients as a means to assess overallinternal control effectiveness. It isanticipated that a capabilities maturitymodel approach ultimately will gainwide acceptance within the independentauditor community as a means for eval-uating management’s assertion aboutthe effectiveness of its internal control.

September/October 2004 33

© 2004 Wiley Periodicals, Inc.

Summary of Internal Control Reliability Model

Reliability Awareness & ControlLevel Documentation Understanding Attitude Procedures Monitoring

Initial Very limited Basic awareness Unformed Ad hoc, unlinked

Informal Sporadic, Understanding not Controls are Intuitive,inconsistent communicated separate from repeatable

beyond business management operations

Systematic Comprehensive Formal communication Controls integral Formal,and consistent and some training to operations standardized

Integrated Comprehensive Comprehensive Control processes Formal, Periodic and consistent training on considered as standardized monitoring

control-related part of strategy beginsmatters

Optimized Comprehensive Comprehensive Commitment to Formal, Real-time and consistent training on continuous standardized monitoring

control-related improvementmatters

Exhibit 4

Michael Ramos, CPA, is a consultant and professional writer working primarily in auditing and accountingtechnical matters. He has written numerous successful products, including general practice aids, imple-mentation guides, and authoritative American Institute of Certified Public Accountants audit and accountingguides. How to Comply with Sarbanes-Oxley Section 404 (published by John Wiley & Sons, Inc., 2004) is hiseighth book. This article was adapted from that book. He currently is working on The SOX 404 Toolkit, acomprehensive collection of SOX 404 implementation practice aids for management and external auditors.