transforming your audit department with technology (lessons from
TRANSCRIPT
Transforming your Audit
Department with Technology
(lessons from the field)
Mike Gowell Vice President, Wolters Kluwer
Mike Gowell • 22 Years with PwC –
Managing Director, Technology Centre
Founder of TeamMate – Audit Management System
• 6 Years with Wolters Kluwer Responsible for TeamMate Global Operations
Two Baseline Questions Rate the importance of effectively using technology in your audit department 1. Very Important 2. Important 3. Somewhat Important 4. Not Very Important
Rate your level of performance in effectively using technology 1. Very Effective 2. Adequate 3. Somewhat Adequate 4. Inadequate
77.4% of CAEs – “Important” or
“Very Important”
85.7% of CAEs – “Inadequate,”
“Somewhat Adequate,” or “Adequate
48% rate
themselves
inadequate
There is a significant gap between the importance that CAEs ascribe
to utilizing technology effectively and their perceived ability to do so.
February, 2011 IIA GAIN survey
Imperatives for Change:
The IIA’s Global Internal Audit Survey in Action (Report V)
• Identified 10 forward-looking imperatives, including Technology
• Stepping up the use of audit technology cuts across multiple imperatives
The Effectiveness Gap
Imperative 10:
CBOK Survey Insights & Implications
• Need to do more with audit technology and automated tools – Technology has become a key enabler for a broad range of internal audit activities
• Develop a long-term technology strategy that addresses: – Automation of core internal audit processes – Support of data mining and analysis and continuous monitoring activities – Technology-related skill sets – Budget requirements – Anticipated benefits – Metrics to measure the effectiveness of technology investments, processes, and
activities
Step Up your Use of Audit Technology & Tools
Audit Technology Budgets
Over the next 2
years, we anticipate
that our technology
budget will:
Source - 2012 TeamMate User Survey
There is more to it than the technology
Mike Gowell Internal Auditor 2001
Closing the Effectiveness Gap:
Uniquely positioned to study the gap – We have a consulting group that has deployed audit technology to 2,200
audit departments
– We have a research and development group that spends 10,000 hours per month developing audit technology solutions
– Have access to 90,000 auditors using technology
– Host 4 regional audit technology conferences that are attended by over 1,000 auditors on a year basis
– Formally surveyed over 11,000 auditors on technology best practices
Open Innovation Contest
Available Research
Technology Survey 2011 & 2012
Survey focus: “How internal audit can more
effectively leverage technology”
• V.1 Published December, 2011
• V.2 Published December, 2012
• Nearly 1,400 responses from internal auditors
around the world
• Respondents described key factors that helped
them increase their effective use of technology
• Year over Year Comparisons
2011 - Identifies Five Key Success Factors
Survey followed up by series of interviews
with CAEs on topic of technology
effectiveness
2012 – Deep Dive into Data Analytics
Technology Survey 2011 & 2012
Survey focus: “How internal audit can more
effectively leverage technology”
Survey Follow-up
How to Increase Technology Value
Key Success Factors
a 5-STEP Strategic framework
Based on surveys, interviews, focus groups, 20 years
of experience with 2,000 audit departments
1. Convey Right “Tone at the Top”
“One of my first priorities when joining MGM Resorts in 2003 was to increase technology support for Internal Audit ”
Robert Rudloff, Vice President Internal Audit MGM Resorts
Single biggest factor for enhancing
technology effectiveness is strong
support from the CAE TeamMate Survey respondents cite
importance of CAE leadership and
support
• Top-rated factor in using
technology more effectively
• Sets the right tone for the internal
audit organization
Key Factor: Openness to Change
Strategic or Tactical Orientation
– CAE tone also determines whether IA treats technology as a tactical or strategic priority
Convey Right “Tone at the Top”
Is Technology a Strategic Priority?
22,6%
63,0%
6,7%
7,7%
Enhancing the use of technology within our department is best described as :
A major strategic priority for internalaudit
An ongoing objective as opposed toa top priority
Being dependent on significantfunding to foster further improvement
A goal that is being adverselyaffected by limited funding
TeamMate Observation:
If technology is not treated as a strategic priority and measured
accordingly, the organization is unlikely to be above average or
exceptional in its use of technology.
Given the continued growth in the use and importance of technology,
can it be viewed as anything but a strategic priority?
• Effective CAE leadership is essential for internal audit to have a good relationship with management
• IA needs management support to get the budget needed to implement a technology strategy
Inadequate budgets:
Second most frequently cited barrier to achieving greater success
with technology, according to TeamMate survey respondents
IA needs effective leadership to address this potential barrier
Convey Right “Tone at the Top”
• CAE leadership can provide clarity to the executive board about the benefits being sought from technology
• The CAE, using clear communications, can help members of the internal audit staff understand the anticipated benefits from technology, starting with more efficient audits
• Conducting more efficient audits: – Most sought-after benefit of technology, by far
– Our top benefit, say 64% of TeamMate survey respondents
#2 Improve issue tracking
#3 Improve the quality and
consistency of audits
#4 Improve risk
assessment process
Convey Right “Tone at the Top”
Make technology use a high priority or even compulsory “We strive to use
data analytics in every audit”
Tara Rexroth, Internal Audit
Director Western Union “We expect every member of our team to apply technology
to every audit. At the end of each audit, staff are required to demonstrate how they leveraged technology, and if they failed to do so, then they need to explain why.”
Michael Sekoni VP & General Auditor
Accident Fund Holdings
2. Prioritize Technology Use
It is imperative that the audit staff both understand and
use the technology tools on every audit
Technology use needs to be an expectation as opposed
to a “nice to use.”
3. Hire Technology Skills Selectively Inability to dedicate sufficient staff to technology activities and the lack of technology skills the most cited barrier to leveraging technology
To address these obstacles: a.) hire outside talent with strong technology skills to jump start
major technology initiatives
b.) establish a technology “champion” within your department
c.) partner with IT to create value
d.) tap outside resources to expand expertise
““My top goal is to find people with deeper, more intense data analysis experience that we can leverage,”
Kimberly Phegley, Internal Audit Director
Lifeway Christian Resources TeamMate Observation:
The ability to hire people with significant technology skills is
challenging in today’s operating environment:
Technology talent is likely to remain in short supply
Adequate budgets can be difficult to obtain
4. Train, train, train … Training does more than any other practice to help utilize technology more effectively.
In addition to training conducted within the corporation, I allocate about $5,000 per staff member for outside technology and business training – and that’s over and above the cost of travel and lodging. I also budgets $7,500 per manager and $10,000 per director for training.
Kevin McMahon, Senior Vice President
& Chief Compliance Officer Calpine
Which best describes your internal audit skill set?
Which best describes your technology training for new hires?
The evidence demonstrates a strong correlation
Train, train, train…and train some more
5. Measure what Matters Measure what really matters because what gets measured gets done
Performance Measures Take on New Importance
Some internal audit groups have implemented performance measures to assess the nature and extent of their technology utilization.
A solid program of performance measures may be the “missing link” to enhance technology effectiveness.
Measure what Matters
Metrics can include: • Length of Audits –
pre and post technology
• Days to issue report • Timeliness of issue
follow-up • Time to prepare
audit committee reports
• Timeliness of audit file review
• QAR results
TeamMate survey responses:
80% lack performance measures to know how
effectively they are leveraging technology
73% do not track or measure the extent to which they
utilize technology to perform audits
Key Q: Is there a direct relationship between
the lack of performance tracking measures
and overall performance levels?
One area of possible concern:
The number of respondents indicating that internal audit is failing to keep up or is significantly falling behind their organization‘s use of technology increased from 9% to 12.5%.
• While small, this may point to significant problems for some internal audit groups.
• It may also reflect the speed with which some companies are implementing newer technologies
Relative to the overall use of technology
by our organization, the audit group is:
Summary – Barriers to Leveraging Technology
19,4%
16,6%
11,6%
8,2%
10,1%
25,4%
8,6%
Our chief barrier to making more effective use of technology is :
An inadequate budget
Lack of technology skills/training
Insufficient focus on technology within thedepartment
Inability to make a compelling cost/benefitjustification
Lack of reasonably priced, effectivetechnology tools in the market
Inability to dedicate sufficient staff totechnology-related activities
How to Increase Technology Value
1. Convey the right “Tone at the Top”
2. Make technology use a high priority or even compulsory
3. Selectively hire technology-skilled people
4. Train, train, train … then train some more
5. Measure what really matters because what gets measured gets done
TeamMate Technology Survey
Key Success Factors
a 5-STEP Strategic framework
Audit Executive Center - Special Report
Developing an Effective IA Technology Strategy
- Technology Maturity Model
- Key Success Factors for improving the use of Technology
- The importance of a technology “strategy”
Technology Maturity Model
Audit scheduling tool
Automated work papers
Data retrieval used on most audits
Custom data mining / data
analytics
Initial ad hoc data mining
Risk assessment tools
Continuous controls testing
and monitoring
Formal technology strategy
Standalone automated testing
routines, e.g. fraud
Online training programs available on
demand
Issues availability, tracking updating by
management
Intranet for audit knowledge
sharing, training, and access to
tools
Automated sharing of audit
programs and files
Fully integrated audit management system
Files, etc., in electronic format
Highly skilled data team
Technology specialist(s)
Drill-down dashboards of
all key audit activity
Reusable programs and checklists
Initial use of CAATs Access to external risk and
control databases
Continuous risk assessment
Quality assessment tool
Use of technology a core
competency Expanded technical
training for staff
Expanded suite of data tools
Structure of the Technology Maturity Diagnostic Tool
Core Technology Process (CTP)
Initial Adequate Enhanced Optimized
1. Technology Strategy & Focus
2. Risk Assessment & Monitoring
3. Audit Planning & Scheduling
4. Knowledge Management
5. Data Analysis & Mining
6. Audit Reporting & Issue Tracking
7. Audit Execution & Documentation
8. Training
9. Human Resources
10. Quality Improvement
4 stages of maturity
10 dimensions to consider for analysis and
planning
Two key points:
1. Need for a strategic focus on technology • Movement up the technology maturity curve requires viewing
technology as more than a tactical tool
• Strategic orientation sets the stage to have a significant, positive impact on current internal audit processes
2. Need for a formal strategic framework for technology • Makes strategic focus operational
• Creates opportunity to communicate both the strategy and vision
Leadership and personal involvement from the CAE is
necessary to progress beyond the “Adequate” stage
CTP #1. Technology Strategy & Focus Does IA have a strategic focus on technology and a formal strategy to enhance its use?
1-Initial 2-Adequate 3-Enhanced 4-Optimized
Use of technology not
emphasized by CAE
Technology tools viewed as
tactical element of audit
process
Basic investment in
technology tools: EWP and
basic data tool
Technology still viewed as a
tactical element of the audit
process
Audit policies require use of
EWP and data tools
Expanded investment in
training of audit staff on tools
Technology viewed as a
strategic enabler
Clear direction from CAE on
need to embrace use of
technology including the re-
engineering of certain audit
processes
Technology strategy is both
developed and implemented
Effective use of technology is
recognized by audit staff as
core competency
Internal audit recognized by
business units as being highly
proficient in the use of
technology
Continued investment in
human resources, advanced
technology tools and
training
CTP #1. Technology Strategy & Focus Does IA have a strategic focus on technology and a formal strategy to enhance its use?
Technology Maturity Model
Key dynamics demonstrated by the model:
• Not just a linear progression – Moving up the maturity curve results in more simultaneous technology activities
going on
– Also demonstrates how increasing maturity broadens the use of technology across audit processes
• Lines between stages of maturity tend to blur – No clear-cut dividing line between stages; overlaps are common
• Not every dimension needs to end up at the Optimized level – You can be at different points on the maturity curve and still be very effective for your
particular circumstances
• Dynamics also reflect interdependencies across the curve
• Our intent is to help you think through, develop or revise your technology strategy
• Treat technology as an important, strategic imperative for internal audit
• Use the tools and research available to prompt your thinking and planning
• “Adapt,” don’t Adopt
Resources Available www.teammatesolutions.com
Mike Gowell V.P. & General Manager, TeamMate
Appendix – Technology Maturity Model
• Builds on the data & findings of surveys noted
• Neither tool nor technology specific
• Takes a holistic look at technology as a key enabler for internal audit success
– Technology is not an end unto itself
• Application of the maturity model is an art, not a science!
– Focus on directional aspects, not absolutes; things in actual practice are not so clear and clean-cut
Technology Maturity Model
Audit scheduling tool
Automated work papers
Data retrieval used on most audits
Custom data mining / data analytics
Initial ad hoc data mining
Risk assessment tools
Continuous controls testing
and monitoring
Formal technology strategy
Standalone automated testing
routines, e.g. fraud
Online training programs available on
demand
Issues availability, tracking updating by
management
Intranet for audit knowledge sharing,
training, and access to tools
Automated sharing of audit
programs and files
Fully integrated audit management system
Files, etc., in electronic format
Highly skilled data team
Technology specialist(s)
Drill-down dashboards of all
key audit activity
Reusable programs and checklists
Initial use of CAATs Access to external risk and
control databases
Continuous risk assessment
Quality assessment tool
Use of technology a core
competency Expanded technical
training for staff
Expanded suite of data tools
How to use the Technology Maturity Diagnostic Tool
The two-phased diagnostic tool includes three sections to facilitate a maturity analysis and a strategic planning framework for technology.
Key Point: Focus on Plan & Actions, not Analysis!
Section 1: Diagnostic for Core Technology Processes
• This section is the basic diagnostic tool.
• After reviewing listed attributes, users should identify which level of maturity most closely represents the current state of that process in their internal audit function.
• Next, identify the process level to which the function aspires.
• Do not assume that every internal audit group will want to achieve the Optimized level in every process: Doing so might not be cost effective or necessary.
• Use an iterative process; identify the next logical steps, not necessarily the final endpoint.
How to Use the Technology Maturity Diagnostic Tool
Section 2: Technology Process Gap Analysis
• The gap analysis provides a simple, visual tool to display the gap between the current state of a process and its desired state.
• This tool provides a means to assess and compare the relative size of process gaps and the relationship between gaps.
How to Use the Technology Maturity Diagnostic Tool
Section 3: Technology Strategy Summary
• Section 3 is an overall summary to facilitate the development of a strategic plan for technology to achieve the next desired maturity state.
• This summary provides a basis for a technology strategic plan and is a simple communications tool to convey the plan and related action plan information.
Structure of the Technology Maturity Diagnostic Tool
Core Technology Process (CTP)
Initial Adequate Enhanced Optimized
1. Technology Strategy & Focus
2. Risk Assessment & Monitoring
3. Audit Planning & Scheduling
4. Knowledge Management
5. Data Analysis & Mining
6. Audit Reporting & Issue Tracking
7. Audit Execution & Documentation
8. Training
9. Human Resources
10. Quality Improvement
4 stages of maturity
10 dimensions to consider for analysis and
planning
Two key points:
1. Need for a strategic focus on technology • Movement up the technology maturity curve requires viewing
technology as more than a tactical tool
• Strategic orientation sets the stage to have a significant, positive impact on current internal audit processes
2. Need for a formal strategic framework for technology • Makes strategic focus operational
• Creates opportunity to communicate both the strategy and vision
Leadership and personal involvement from the CAE is
necessary to progress beyond the “Adequate” stage
CTP #1. Technology Strategy & Focus Does IA have a strategic focus on technology and a formal strategy to enhance its use?
1-Initial 2-Adequate 3-Enhanced 4-Optimized
Use of technology not
emphasized by CAE
Technology tools viewed as
tactical element of audit
process
Basic investment in
technology tools: EWP and
basic data tool
Technology still viewed as a
tactical element of the audit
process
Audit policies require use of
EWP and data tools
Expanded investment in
training of audit staff on tools
Technology viewed as a
strategic enabler
Clear direction from CAE on
need to embrace use of
technology including the re-
engineering of certain audit
processes
Technology strategy is both
developed and implemented
Effective use of technology is
recognized by audit staff as
core competency
Internal audit recognized by
business units as being highly
proficient in the use of
technology
Continued investment in
human resources, advanced
technology tools and
training
CTP #1. Technology Strategy & Focus Does IA have a strategic focus on technology and a formal strategy to enhance its use?
Technology Maturity Model
Key dynamics demonstrated by the model:
• Not just a linear progression – Moving up the maturity curve results in more simultaneous technology activities
going on
– Also demonstrates how increasing maturity broadens the use of technology across audit processes
• Lines between stages of maturity tend to blur – No clear-cut dividing line between stages; overlaps are common
• Not every dimension needs to end up at the Optimized level – You can be at different points on the maturity curve and still be very effective for your
particular circumstances
• Dynamics also reflect interdependencies across the curve
1-Initial 2-Adequate 3-Enhanced 4-Optimized
Use of technology not
emphasized by CAE
Technology tools viewed as
tactical element of audit
process
Basic investment in
technology tools: EWP and
basic data tool
Technology still viewed as a
tactical element of the audit
process
Audit policies require use of
EWP and data tools
Expanded investment in
training of audit staff on tools
Technology viewed as a
strategic enabler
Clear direction from CAE on
need to embrace use of
technology including the re-
engineering of certain audit
processes
Technology strategy is both
developed and implemented
Effective use of technology is
recognized by audit staff as
core competency
Internal audit recognized by
business units as being highly
proficient in the use of
technology
Continued investment in
human resources, advanced
technology tools and
training
CTP #1. Technology Strategy & Focus Does IA have a strategic focus on technology and a formal strategy to enhance its use?
Improving Risk Assessments: Key 2010 CBOK Imperative
• Strengthening timing and other key aspects of risk assessments is a key imperative from the 2010 CBOK results
• As a strategic enabler, technology can: • Facilitate move to more periodic or continuous risk assessments
• Foster monitoring of key risk indicators or internal controls for SOX testing
• Create a natural flow of monitoring capabilities from IA to business units
• Provide the means to share assessment results with other risk & control units
• This CTP is strongly linked to CTP #5 (Data Analysis & Mining)
Risk Assessment & Monitoring activities offer significant potential to
enhance internal audit processes
CTP #2. Risk Assessment & Monitoring Does IA use technology to improve its risk assessment and monitoring processes?
1-Initial 2-Adequate 3-Enhanced 4-Optimized
Manual risk assessment
conducted annually based on
high-level risk factors and
interviews with key
executives
Manual updating of risk
profile and audit plan
Use of Word and Excel
Audit department uses risk
assessment tool to document
risk assessments at the
process/entity level and to
capture results of interviews
at the process/entity level
Manual updating of risk
profile and audit plan
Internal Audit has periodic or
continuous monitoring of key
risk indicators
IA shares results with
business units and
encourages lines of business
to adopt effective monitoring
activities
IA provides management
with access to risk
assessment tool that enables
management to conduct
periodic self assessments and
generate risk assessment
reports
Automated linkage to audit
plan
Continuous monitoring of key
risk indicators
Business units look to
Internal Audit for assistance
in developing monitoring
programs
Automated updates to the
risk assessment and audit
plan
Audit testing results are
automatically fed back into
the risk assessment model
Audit testing results are
shared with other risk-and-
control units on a real-time
basis
CTP #2. Risk Assessment & Monitoring Does IA use technology to improve its risk assessment and monitoring processes?
CTP #3. Audit Planning & Scheduling Does IA use technology to support its audit planning and scheduling processes?
Applying Technology to Enhance Internal Audit Processes
• Key Goals: – Increase efficiency
– Enhance operational effectiveness
• Explore Beneficial Linkages to … – Skill Inventories
– Work Paper Systems & Files
– Personal Scheduling Systems
1-Initial 2-Adequate 3-Enhanced 4-Optimized
Manual audit planning and
scheduling
Use of Excel
Centralized administrative
group uses scheduling tool
for planning and scheduling
Schedules posted on-line or
distributed to staff via e-mail
Automated scheduling tool
available to all staff for
planning and scheduling
Advanced functionality to
schedule by skill type and
availability
Automated scheduling tool
available to all staff for
planning and scheduling,
including the ability of staff
to download schedules to
their personal calendars (e.g..
Outlook)
Scheduling tool is linked to
audit work papers as part of
an integrated audit support
system
CTP #3. Audit Planning & Scheduling Does IA use technology to support its audit planning and scheduling processes?
CTP #4. Knowledge Management How does IA acquire, manage, and share its knowledge?
Streamline Acquisition, Retention & Sharing of Knowledge
• Pursue technology-enabled knowledge management to ascend the maturity curve
• Move from changing the formats of old hard-copy materials to saving documents in electronic and digital formats
• Enhance the knowledge process by moving beyond knowledge internal to the IA function to include:
– Knowledge from across the company
– External source material
• For many organizations, applying technology to enhance knowledge management is a deferred priority to be addressed in a later stage of the technology maturity curve
– Dependent on internal audit having a broad-based technology platform
1-Initial 2-Adequate 3-Enhanced 4-Optimized
Combination of manual and
automated permanent files
“Standard” audit programs
and checklists available in
Word
Permanent files, background
info available electronically
Audit programs are indexed,
searchable and available for
electronic import to the audit
file
IA accesses/uses third-party
risk and control databases
Additional indexed content is
available for import (Risk and
Control information, pro-
forma issues, work paper
templates)
Automated KM process in
place department-wide
supported by formal policy
Knowledge tool fully
integrated into internal audit
technology platform and
related processes (training,
audit execution, scheduling)
Advanced technology (e.g. ,
smart tags) identifies and
informs auditor where
actionable content is
available
Direct access to business-unit
data and information
Staff afforded access to
external data sources & tools
CTP #4. Knowledge Management How does IA acquire, manage, and share its knowledge?
CTP #5. Data Analysis & Mining Does IA utilize data mining and analytics to enhance its auditing and monitoring processes?
High Priority Spanning Much of Technology Maturity Curve
• Facilitates periodic risk assessment & monitoring (see CTP #2)
• Current priority for many internal audit groups
• Moving up the curve depends more on skills than data tools
• Pursuit of data analysis & mining potential should lead to a re-engineering of traditional internal audit routines
• Acquiring specialized capabilities is a key success factor in starting data mining and then moving up the technology maturity curve
• Significant issue: understanding data – How to access it
– What data are present
Another dimension of internal auditing offering significant
opportunities to enhance internal audit processes
1-Initial 2-Adequate 3-Enhanced 4-Optimized
Initial data access/retrieval
tool acquired
Initial use of CAATs
(computer-assisted audit
techniques) on an ad hoc
basis on some audits
Dedicated technology
specialist(s) used to develop
and run programs
Data access/retrieval tool
used on most audits
Support by specialized staff
with basic training for some
staff auditors
Initial ad hoc data mining
routines developed
Data access tools used in
almost all audits
Audit staff trained and using
software themselves
Data mining team supporting
audit staff but focusing on
building custom data mining
routines
Expanded suite of data tools
including using capabilities
built into ERP systems
Continuous audit routines in
operation replacing some
traditional audits
Highly skilled data mining
team developing new
routines, including
continuous controls and risk
monitoring
Internal audit routines
sought by business-unit
management
Data mining used for
business-issue analysis
beyond just auditing
CTP #5. Data Analysis & Mining Does IA utilize data mining and analytics to enhance its auditing and monitoring processes?
CTP #6. Audit Reporting & Issues Tracking Does IA leverage technology to report findings & track related issues?
Technology-Enabled Streamlining Offers Great Potential
• Cut down on the “black hole” of audit hours often associated with report writing
• Achieve direct link between audit work & finding/reporting
• Enhance issues tracking in place with responsibility for updating where it belongs, with management
Enhancing the audit reporting & issues-tracking process can
free up sizable hours for other priorities.
1-Initial 2-Adequate 3-Enhanced 4-Optimized
Audit reports produced and
distributed electronically
Manual reporting and
tracking of open audit
issues
Audit reports produced and
distributed electronically
Automated tool used for
issues tracking and
reporting
Audit findings circulated
directly from audit work
papers
Audit reports produced and
distributed electronically
Automated triggers identify
past-due items
Issue Owner can access
system directly to provide
updates
Audit findings and reports
produced and circulated
directly from audit work
papers
Automated tool used by
management to track and
report status of audit issues
Full management and audit
committee reporting
packages automatically
prepared by the system
CTP #6. Audit Reporting & Issues Tracking Does IA leverage technology to report findings & track related issues?
CTP #7. Audit Execution & Documentation How does IA use technology to support audit execution and documentation processes?
Typical Starting Point for Audit Technology Enhancements
• Basic electronic work-paper packages are the most widely used audit technology tools
• Larger, more advanced internal audit functions often have fully integrated suites that handle scheduling, risk assessment, audit execution, data access and reporting
• Future priorities: Include social and mobile media
Smaller internal audit groups may not find it beneficial to optimize
this dimension of the audit technology maturity curve.
1-Initial 2-Adequate 3-Enhanced 4-Optimized
Use basic EWP package
Additional audit
documentation maintained in
hard-copy file
Separate, unlinked packages
used for other audit
processes such as data
analytics or risk assessment
Basic EWP package used for
all audit documentation
Some linkage of other tools
such as risk assessment
Basic report generation and
linkage with issues-tracking
system
Full Electronic Audit
Management suite used to
support audit execution and
risk assessment
Linkage with other tools
including data analytics and
controls testing
Fully integrated system
includes scheduling, risk
assessment, audit execution,
data access and reporting
System also supports
knowledge management and
training
System also incorporates
social and mobile formats
Drill-down dashboards for
real-time monitoring of
internal audit activities
CTP #7. Audit Execution & Documentation How does IA use technology to support audit execution and documentation processes?
CTP #8. Training How does IA train its staff in the use of technology tools?
Survey Feedback: Training Is Critical Dimension • Systematic and comprehensive training for every staff member is key
to achieving organizational progression up technology maturity curve
• View training as a process, not an event
• Move to desk-top training and “just in time” availability is accelerating
• Training has major interdependencies: – Staff members must appreciate the critical need for training and its strategic
importance to their individual success
– Strong links to human resources, knowledge management, quality, execution, and other key factors
Training needs to be reinforced by an expectation or
requirement for usage. If not put into practice, training efforts
are wasted.
1-Initial 2-Adequate 3-Enhanced 4-Optimized
Staff trained in use of basic
electronic documentation
tools (Word and Excel) and
storage and retention
protocols
Staff trained in use of basic
automated work-paper
software
Selected staff trained in
CAATs (computer-assisted
audit techniques)
Selected training available
electronically
Formal staff training on
entire technology suite
including ability to develop
and run CAATs (computer-
assisted audit techniques)
Staff evaluated on use of
technology
Training available, as needed,
electronically
Formal in-house training
procedures established; all
staff trained and evaluated
on proficiency in use of
technology tools
Continuous updating of
training for new technology
tools and applications
CTP #8. Training How does IA train its staff in the use of technology tools?
CTP #9. Human Resources Does IA have the skills and human resources to effectively leverage its use of technology?
Skills: Major Limiting Factor in Moving Up Technology Curve
• Two interrelated factors facilitate maturity curve movement
– Acquiring specialized staff
– Training overall staff
• Requires IA to realistically assess its skills and needs
– As they relate to technology
– As they relate to the risks and business elements they cover
• A moving target: As an IA group becomes more sophisticated in its use of technology, it will likely require more specialized skills
1-Initial 2-Adequate 3-Enhanced 4-Optimized
Staff provided with basic
training and skills
development in EWP and
office software tools
Specialized technology staff
acquired or third party used
for initial data tools
Staff provided with basic
training and skills
development in EWP and
office software tools
Data technology supported
by combination of dedicated
specialists and selected audit
staff
All staff trained in use of
technology tools/suite
Certain specialized staff
acquired or third party used
to supplement skill sets
All staff trained and equipped
to conduct continuous
procedures
Highly skilled data analysis
skills present or available via
third-party support
Data analysis/audit teams
include staff with non-
accounting, auditing skills
CTP #9. Human Resources Does IA have the skills and human resources to effectively
leverage its use of technology?
CTP #10. Quality Improvement Does IA conduct a continuous improvement process aligned to its technology strategy?
Technology Impacts Two Key Aspects of Quality:
1. Measuring the amount and types of technology usage aligned to strategy
– Movement up the technology maturity curve shifts performance focus from the nature of technology usage to its “quality”
– Example: Moving from ad hoc to continuous routines
2. Using technology as part of IA’s overall quality improvement program
– Primary tool to build consistency and compliance with policies and internal standards
• Lock-out features to prevent closing out an audit without completing required steps or answering open review questions
– Technology tools also can assist with the assessment of quality and tracking results
• Review audits; track and report findings from an internal QA review process
Supports view that “what gets measured gets done!”
1-Initial 2-Adequate 3-Enhanced 4-Optimized
Manual quality improvement
and assessment process
Quality steps embedded into
automated work-paper
package
Work papers, report
automatically sent to QA
after completion
Performance measures used
to determine extent of
technology use on audits
Execution software aligned to
support and enforce audit
policies
QA assessment includes
assessing adequacy of use of
technology in executing audit
Technology used to support
the internal QA process and
reporting
Performance measures both
extent and types of usage as
part of effort to encourage
broader participation
Audit execution software
aligned to overall audit
polices and strategies
QA tool used to facilitate
both reviews and reporting
of findings and trends
Findings of QA process rate
use of technology
Assessment includes
assessing adequacy of
technology usage and
identifying best practices to
share
CTP #10. Quality Improvement Does IA conduct a continuous improvement process aligned to its technology strategy?
Use of the maturity curve
Keep in mind:
• It’s an iterative tool
• Take one step at a time
• Lines between stages are not clear cut
• Need to understand linkages and dependencies