Automating SOX Compliance:Bringing Efficiencies & Effectiveness to Annual Compliance

Automating SOX Compliance:Bringing Efficiencies & Effectiveness to Annual Compliance

Sandra C. Keaveny

Principal, DFSG Corporation

2

AgendaAgenda

• History of Compliance• Case Study

• Challenges• Opportunities• Solution• Benefits

• Next Steps• Questions• Conclusion

3

Living in a Controlled Environment – Challenges and CostsLiving in a Controlled Environment – Challenges and Costs

• The Sarbanes-Oxley Act of 2002, commonly called Sarbanes-Oxley or SOX, is a United States federal law enacted on July 30, 2002, as a reaction to a number of major corporate and accounting scandals including:• those affecting Enron, Tyco International, Adelphia, Peregrine Systems and

WorldCom.

• The most contentious aspect of SOX is Section 404, which requires management and the external auditor to report on the adequacy of the company's internal control over financial reporting (ICFR). • This is the most costly aspect of the legislation for companies to implement, as

documenting and testing important financial manual and automated controls requires enormous effort.[25]

• The FEI 2007 study indicated that, for 168 companies with average revenues of $4.7 billion, the average compliance costs were $1.7 million (0.036% of revenue).[10] • The 2006 study indicated that, for 200 companies with average revenues of $6.8

billion, the average compliance costs were $2.9 million (0.043% of revenue), down 23% from 2005.

4

Living in a Controlled Environment – Challenges and CostsLiving in a Controlled Environment – Challenges and Costs

• SOX 404 compliance costs represent a tax on inefficiency, encouraging companies to centralize and automate their financial reporting systems.

• Cost for decentralized companies (i.e., those with multiple segments or divisions) were considerably more than centralized companies, those with centralized, more efficient systems.• For example, the 2007 FEI survey indicated average compliance costs for

decentralized companies were $1.9 million, while centralized company costs were $1.3 million.[28]

• The number of controls defined can range significantly based on size, complexity, whether centralized or decentralized: • the average number of SOX controls ranges from 75-225, with a significant

portion (over 60%) typically identified as manual controls.

• Costs of evaluating controls can be dramatically reduced through automation, based on using their ERP software to control their business, • they can leverage application controls – however, these also are normally tested

and documented by a business user, also annually

Living in a Controlled Environment – The Automation OpportunityLiving in a Controlled Environment – The Automation Opportunity

• The average number of hours required to manually test and document a single SOX control is approximately 6 - 8 hours and sometimes more, based on complexity.• This requires concentrated effort of defined business personnel,

involving critical subject matter experts across all functional areas, regions and division, which takes significant time (days or weeks) from their normal business efforts.

• In addition, the work paper documentation around testing of SOX controls requires detail and precision since they will be reviewed/audited by both internal and external auditors.• Accurate and complete work papers are critical.

Living in a Controlled Environment – The Automation OpportunityLiving in a Controlled Environment – The Automation Opportunity

• Each control must be tested with two considerations in mind: • Test of Design (TOD) –does the control as designed continue to

represent how the business is operating and• Test of Effectiveness (TOE) – does this control work effectively based

on its intent or design.

• Evidence of both TOD and TOE must be shown for each control to prove they are working effectively

• Because external auditors cannot rely on testing by the business owners, all critical controls must be re-tested by external auditors as well• This cost comes back to the company in terms of audit fees.• These tests need to be repeated on an annual basis, at minimum.

Compliance Automation Case Study: BackgroundCompliance Automation Case Study: Background

SAP® Compliance Case Study:• A $5 billion global manufacturer with operations in North America,

Latin America, Asia-Pacific and Europe.• Approximately 15,000 employees. • Running on a global instance of SAP® ECC 6.0, with a distributed

environment including SAP CRM, SAP APO, SAP NetWeaver®, BI, SAP SRM, two Portals…..

• 175 Sarbanes-Oxley controls, 80 of which are SAP application/ system controls – this is a higher ratio than most and considered a leading practice.

Compliance Automation Case Study: Business CaseCompliance Automation Case Study: Business Case

Case Study (continued)• Their external auditor has agreed that if they use a qualified

third party to execute some of the testing, they can rely on their work and not have to re-test. • They have chosen to have all the system controls (80) tested by the

third party to reduce both business effort and the cost of external audit retest.

• The cost of the third party testing is approximately $250,000 annually.

• Their external auditors have also agreed that if they can document that there has been no change to the transactions associated with the system controls, the company can reduce the number of tests that they need to execute each year• Allowing the full baseline of tests (not impacted by change) to be

executed over three years (one third each year).

Case Study: Automation GoalsCase Study: Automation Goals

1. Eliminate the cost of the third party by creating automated application controls that can be executed internally by one person – business review involvement only

2. Ensure that the automated control testing can be relied upon by the external auditor even though it is “executed” internally

3. Ensure work papers are automatically created and meet SOX test and auditor work paper standards

4. Provide an automated report that can easily and quickly indicate if no changes have occurred to SOX significant transactions over a defined period of time (one quarter to one year) or, if changes have occurred, what changed

5. Reduce business involvement while retaining high standards of compliance testing and documentation

6. Manage, monitor and communicate test execution and results with online dashboards and metrics.

10

Case Study: Prep for Automated BuildCase Study: Prep for Automated Build

Gather •Working with SOX management, obtain list of all controls – manual and system - and related test documentation

Analyze •Working with relevant business lead, assess each control to determine how it can be automated, including current manual controls•Engage subject matter experts (technical and business) to answer any questions regarding how control is used and how automation can be applied

Prioritize and Plan •Working with SOX management and external auditor, prioritize those controls that external audit relies on most heavily during their review •Working with audit schedules, build plan to automate controls in support of SOX testing time frames

11

Case Study: How to Build and Execute Automated SOX TestingCase Study: How to Build and Execute Automated SOX Testing

Build all the TODs First and Test

TODs Fail

Fix TODs and Retest

Build and Execute TOEs

for each TOD in this group

TODs Pass

TOEs Built and Executed as part of 3 year rotation

cycle

12

Case Study: Automation of an Application ControlCase Study: Automation of an Application Control

SOX Defined Test (Audit Test

Description)

• Validate that field status settings are configured to enforce key fields as required entry when entering a purchase order.

Associated Control (Control Activity)

• The SAP R/3® system is configured to require key fields (price, cost object, etc) when entering a purchase order in the system.

Case Study: To Automate, Start with the Manual Control TestCase Study: To Automate, Start with the Manual Control Test

A portion of the manual version of the TOD for that control:

Test of Design (TOD):

 The screen layout is formed by a combination of the transaction code, activity, document type, and item category/document category. Each of these has a corresponding entry in the Field Selection column.

Creating a standard NB purchase order for a standard item would have the following field selections:

AKTH - Activity

ME21N - Transaction

NBF - Document Type

PT0F - Item category/Document category

The standard SAP implements Fields configured as ‘required entry’ over Fields configured as ‘optional entry’ (AKTH) when entering a purchase order.

 

14

Case Study: Review Work Paper Documentation RequiredCase Study: Review Work Paper Documentation Required

Case Study: Timing of Manual Test of TODCase Study: Timing of Manual Test of TOD

• Manual time to execute the test:• Number of screens/fields to validate: 75• Run the test: 4 hours• Document the results: 8 hours

• If this test is executed by business, may still need to retested by external auditor

• If this is done by third party, incur the cost of testing and documentation – must be repeated each year.

16

Case Study: Automated Testing Executed the Same WayCase Study: Automated Testing Executed the Same Way

17

Case Study: Automated Test Captures All FailuresCase Study: Automated Test Captures All Failures

18

Case Study: Automated Test Shows Each Step and ScreenCase Study: Automated Test Shows Each Step and Screen

19

Case Study: Differences Between Manual and Automated TestingCase Study: Differences Between Manual and Automated Testing

• Automated time to execute the test:• Number of screens/fields to validate: 75• Run the test: 30 minutes• Document the results: 0 hours, since test automatically reports

results in work paper ready format

• No business intervention in getting tests done. One person can execute both tests and results, which can then be reviewed/confirmed with business and auditors

20

Case Study: Results Are Reported Like Work PapersCase Study: Results Are Reported Like Work Papers

Case Study: Screen Shot Detail with Results of Automated SOX TestsCase Study: Screen Shot Detail with Results of Automated SOX Tests

22

Case Study: When Failures Occur, Know What and WhenCase Study: When Failures Occur, Know What and When

23

Case Study: Failure Results DocumentedCase Study: Failure Results Documented

24

Case Study: Detecting and Managing FailuresCase Study: Detecting and Managing Failures

• With automated testing, you can configure the test to stop at any failure and document which step in the process caused the failure.

• This documentation can then be provided to the developers to fix and prepare for retest

• The automated test can then be pointed• First to the test environment to validate and• Second, to the production environment for further

validation if fix passes initial test• All this repeated testing can be executed by one person

with all documentation fully detailed

25

Case Study: Leveraging TOD Automation to Build the TOECase Study: Leveraging TOD Automation to Build the TOE

• By automating the test of design first, the additional build showing the effectiveness of the test or TOE, even using multiple sets of data, is quick and efficient. Let’s look at an example.

• Manual time – Total of 8 hours• Number of screens/fields to validate: 75• Run the test: 4 hours• Document the results: 4 hours

• Automated time to execute the test (TOE):• Number of screens/fields to validate: 75• Run the test: 10 minutes• Document the results: 0 hours, since test automatically reports

results in work paper ready format

26

Case Study: Using Dashboards to Manage Build/Execution of TestsCase Study: Using Dashboards to Manage Build/Execution of Tests

27

Case Study: Using Dashboards to Manage Testing Across ProjectsCase Study: Using Dashboards to Manage Testing Across Projects

Additional Tools to Further Optimize Management of SOX ControlsAdditional Tools to Further Optimize Management of SOX Controls

• Certify Live Compare™ Transaction Change Report• This report, run against your SAP production environment, will define

what transactions have changed over a period of time and provide clear documentation for your auditors of what needs to be tested from a TOD and TOE perspective (show example)

• Worksoft Certify® SOX-specific SAP role• By creating a specific SAP role for use with Worksoft Certify testing, you

can lock and unlock this role based on when testing is needed and further control your production access and environment

29

Live Compare: Identifying What’s ChangedLive Compare: Identifying What’s ChangedPrograms that are the same (unchanged) are not included. The icons report programs that are new (the little “1”) icon and different. New programs are those that have been created since the last comparison.

30

Live Compare: Identifying How It Has ChangedLive Compare: Identifying How It Has Changed

This shows side-by-side comparison of what changed within the transaction and targets where to look

Benefits of Using Automated Approach to SOX Controls TestingBenefits of Using Automated Approach to SOX Controls Testing

1. You can quickly assess your production environment and determine and validate what has remained the same and what has changed

2. You can use expedited testing, leveraging automated TODs for all application controls, to further define what needs extended testing/validation

3. All your documentation for both TODs and TOEs is automatically created as part of the Worksoft Certify® test runs; no need for additional documentation

4. The results of all your automated test efforts are presented in an easily readable format that both internal and external audit/SOX teams can use and review

5. One person can execute all automated testing, significantly reducing the time and effort normally required by both business and audit teams to obtain results

Benefits of Using Automated Approach to SOX Controls TestingBenefits of Using Automated Approach to SOX Controls Testing

5. Because these tests need to run against your production environment, by using the automated test build, you can specifically control what is executed and when it is stopped• Eliminating any potential opportunity of executing more than

what’s needed or planned

6. Per external audit agreement, the external audit team can rely on these results without third party intervention, thereby eliminating third party costs and significantly reducing external audit costs associated with executing these tests manually• Only need to review results versus re-executing critical

application control tests

Time for Q & ATime for Q & A

• Questions????

34

Next Steps in Achieving Automation EfficienciesNext Steps in Achieving Automation Efficiencies

• The tools being used here to achieve efficiencies in compliance are also being used successfully across development, testing and production environments to bring further efficiencies to the daily management of SAP systems.

• If you want to further explore these opportunities or have additional, specific questions that you’d like to address further, please contact:

35

Contact UsContact Us

Phone: 866.836.1773

Email: sales@worksoft.com

www.worksoft.com

KEAVES712@YAHOO.COM

Certified for SAP NetWeaver®