closing the gap between auditor performance and regulators’ expectations …€¦ · ·...
TRANSCRIPT
Closing the Gap between Auditor Performance and Regulators’ Expectations when Auditing Fair Value Measurements: Evidence from Practicing Audit Partners
Steven M. Glover Brigham Young University
Mark H. Taylor Case Western Reserve University
Yi-Jing Wu Case Western Reserve University
April 2014
2014 Deloitte Foundation/University of Kansas Auditing Symposium
Acknowledgements: We thank the Center for Audit Quality for funding this project as well as the Center’s Research Advisory Board for its feedback to help refine our survey. We also appreciate those practicing partners who responded to our survey. This paper has benefited from helpful comments from the workshop participants at Case Western Reserve University. Steve Glover is grateful for the funding provided by the K. Fred Skousen Endowed Professorship. Mark Taylor appreciates the funding provided by the Andrew D. Braden Endowed Professorship. Finally, although funding for the research project described in this paper was provided by the Center for Audit Quality, the views expressed in this paper are those of the authors and not those of the Center for Audit Quality or the participating accounting firms.
Closing the Gap between Auditor Performance and Regulators’ Expectations when Auditing Fair Value Measurements: Evidence from Practicing Audit Partners
ABSTRACT: Auditing fair value measurements (FVMs) is a challenging area for auditors, as evidenced by the numerous PCAOB inspection findings in recent years. Ambiguity in auditing standards coupled with the complexity associated with auditing of FVMs results in disagreements between audit firms and the PCAOB regarding what constitutes sufficient appropriate audit evidence. Using a survey, we obtain audit partners’ views regarding the PCAOB inspection process related to FVMs. In doing so, this study aims to narrow the gap between auditor performance and the PCAOB’s expectations by identifying areas for potential improvement. This study provides insights regarding audit partners’ views on the auditability of and reporting on FVMs with significant estimation uncertainty. Our results also provide insights regarding significant challenges auditors face in auditing of FVMs with a focus on two key areas: evaluating the reasonableness of management’s assumptions and valuation methods, and use of valuation experts. We find that the lack of verifiable and corroborative evidence, and auditors’ reliance on valuation experts due to their limited knowledge and expertise regarding complex valuation inputs, analyses, and models contribute to the gap between auditor performance and regulators’ expectations. Our results suggest that current auditing standards may not provide sufficient guidance to auditors or inspectors on what constitutes sufficient, appropriate audit evidence around FVMs. Finally, regarding estimates with extreme estimation uncertainty, our results indicate that extreme estimation uncertainty exists in audited financial statements; and, perhaps no amount of additional audit procedures can limit the irreducible inherent estimation uncertainty to levels that regulators would deem to be reasonable.
1
Closing the Gap between Auditor Performance and Regulators’ Expectations when Auditing Fair Value Measurements: Evidence from Practicing Audit Partners
INTRODUCTION
Auditing fair value measurements (FVMs) is a challenging and complex area given the
subjectivity inherent in estimating future events and the potential for a high degree of
measurement uncertainty (Bratten, Gaynor, McDaniel, Montague, and Sierra 2013; Christensen,
Glover, and Wood 2012; PCAOB 2007). Auditing complex FVMs requires considerable
professional judgment. The inherent uncertainty of FVMs and the subjective nature of the audit
evidence provide regulators and financial statement users more than typical leeway to second-
guess the quality of the auditors’ work (Peecher, Solomon, and Trotman 2013).
PCAOB inspections have repeatedly identified audit deficiencies in auditing FVMs (e.g.,
PCAOB 2008a; PCAOB 2012a). PCAOB inspection reports from 2007-2011 reveal that the
most common deficiency is failure to evaluate the reasonableness of management’s underlying
assumptions and methods. The difficulties of auditing FVMs have increased auditors’ use of
pricing services and valuation specialists, which is an area that has induced the PCAOB to
criticize the firms for over-relying on the work of specialists and pricing services (PCAOB
2010a; PCAOB 2012a; Bratten et al. 2013). However, audit firms have frequently disagreed with
these PCAOB conclusions (Church and Shefchik 2012) highlighting an apparent expectation gap
that exists between auditors and regulators regarding the nature and extent of audit evidence
necessary to support audits of FVMS (hereafter, the “FVM expectation gap”).
While several concurrent studies provide insights regarding the process and challenges of
auditing complex estimates including FVM, and auditors’ use of in-house valuation specialists
(Cannon and Bedard 2013; Griffith, Hammersley, and Kadous 2014; Griffith 2013), research
regarding the audit of FVM remains sparse (Bratten et al. 2013). The current study extends the
literature on auditing of FVM by gathering survey data from audit partners to identify possible
2
causes of the FVM expectation gap as well as potential steps that could be taken to reduce the
gap. We then provide additional insights regarding key challenges auditors face in auditing
FVMs to identify areas for potential improvement in auditor performance, auditing standards,
and the PCAOB inspection process. Specifically, we focus on challenges auditors encounter in
the following two key FVMs areas most frequently mentioned in PCAOB inspection reports to
contain deficiencies: (1) evaluating management’s assumptions and valuation methods, and (2)
use of valuation experts. Finally, we analyze audit partners’ views regarding the auditability of
and reporting on estimates with extreme measurement uncertainty, especially in light of recent
commentary that questions the auditability of these complex estimates (Christensen et al. 2012).
To address our research objectives, we distributed the survey primarily to field practice
audit partners with significant experience in the planning, supervising, and executing audits of
FVMs.1 The survey uses both objective and open-ended questions and is divided into sections
aimed gathering insights regarding auditors’ views regarding PCAOB inspection process,
general challenges in auditing FVMs, specific challenges with use of valuation experts, and high
estimation uncertainty and presentation.
We find that a majority of participants (over 60 percent) believe that the PCAOB
inspectors expect more evidence to be gathered than auditing standards technically require. For
audit partners serving on engagements where inspectors identified a FVMs-related audit
deficiency, 100 percent disagreed with the inspector’s assessment. Primary reasons for
disagreement are (1) the PCAOB did not appropriately consider misstatement risk and totality of
1The survey results presented in this study are based on our primary survey of practicing audit partners. We also designed a separate survey for national office audit partners that is more focused on PCAOB inspections and high estimation uncertainty and presentation of FVMs. We received responses from 5 national office partners, one from each of the participant firms, and we use their responses to open-ended questions to provide further insights regarding auditing FVMs. Thus, in the presentation of results, when appropriate, we report some of the national audit partner responses to corroborate responses from practicing audit partners and to highlight unique insights on the topic experienced by partners at the national level.
3
the evidence gathered, (2) the PCAOB expects a level of precision in auditing FVMs that is
simply not achievable, and (3) PCAOB inspectors have unreasonable expectations regarding
management’s and the auditor’s use of and reliance on third-party specialists. The top two
suggestions audit partners provided for improving the PCAOB inspection process and audits of
FVMs are implementation of a within-engagement risk-based inspection approach and revising
auditing standards to provide better guidance.
With respect to significant challenges in auditing FVMs, a number of key findings
emerged that may be contributing to the FVMs expectation gap. The top three challenges our
participants cite are (1) lack of verifiable and corroborative evidence, (2) degree of difficulty in
assessing reasonableness of assumptions, and (3) high degree of subjectivity involved in
estimates. Moreover, management’s lack of knowledge regarding the valuation process creates
additional challenges for auditors. Over 50 percent of participants report concerns with
management’s knowledge regarding valuation processes. Further, over 60 percent believe that
the audit challenges differ between financial vs. nonfinancial assets/liabilities primarily with
respect to the availability of market data, as auditors must rely more on management’s
assumptions for the latter due to the lack of observable market information. In terms of issues
encountered when using third-party pricing services, partners frequently note that their inability
to obtain sufficient information about the pricing services’ proprietary models is problematic
when trying to understand how the pricing services derive the FV of investment securities. These
challenges may explain why PCAOB inspection reports repeatedly note deficiencies in the area
of evaluating the reasonableness of management’s assumptions and valuation models and use of
valuation experts.
Further adding to the complexity of auditing FVMs, as well as calling into question the
fundamental auditability of FVMs, is “extreme measurement uncertainty” that can exist in
4
financial statements in which the reasonable range of estimation uncertainty on the measurement
date can be many times greater than materiality (Christensen et al. 2012). We find that one of the
primary reasons that auditors disagree with PCAOB inspection findings relates to the estimation
uncertainty inherent in FVMs. A majority of our participants (60 percent) suggest that the current
requirement to provide a high level of assurance that a point estimate is fairly stated within
auditor materiality be reconsidered because providing such assurance on point estimates is
unrealistic and potentially misleading given the level of subjectivity, complexity and uncertainty.
Nearly all participants (93 percent) support the idea of revising auditing standards to provide
additional clarity and guidance around estimates characterized by extreme measurement
uncertainty. The top three areas in which auditors would welcome more guidance from standard-
setters include (1) clarification about an acceptable reasonable range of estimation uncertainty,
(2) clarification regarding audit requirements and procedures, and (3) the standard-setting
process should influence audit procedures rather the PCAOB inspection process.
Our results highlight a number of factors that may be contributing to the FVM
expectation gap. Findings suggest that existing auditing standards may not provide sufficient
guidance to inform auditors or inspectors on what constitutes sufficient, appropriate audit
evidence, and that PCAOB inspectors expect auditors to gather more evidence than auditing
standards technically require. As a result, in some cases auditors may be reacting to inspectors’
preferences rather than carefully applying existing auditing standards. A potential consequence
of an inspection-driven approach to interpreting/applying auditing standards is that audit efforts
may be linked to “inspection risk” rather than audit risk (Glover and Prawitt 2013), which may
lead auditors to devote less attention to areas that present greater audit risk. Our results highlight
two additional issues related to the FVM expectation gap. One concerns the availability of
reliable data to support key assumptions (either management’s or pricing services’). Another
5
relates to auditors’ overreliance on pricing services and valuation specialists, caused in part by
management’s lack of knowledge/expertise to fully evaluate complex valuation inputs, analyses,
and models (Bratten et al. 2013; Griffith et al. 2014; Martin, Rich, and Wilks 2006). These are
additional areas in which regulatory efforts may be able to help close the FVM expectation gap.
To our knowledge, this is the first study that addresses antecedents of and possible
solutions for narrowing the FVMs expectation gap. We extend the literature by providing
insights regarding ways to improve audit practice and the inspection process around FVMs.
Second, our study is also the first to provide important insights from practicing audit partners on
the auditability of and reporting on FVMs with high estimation uncertainty, an area in which
audit partners believe standard-setters can provide additional guidance and clarity. Finally, this
study contributes to the extant research regarding key challenges in auditing FVMs. In particular,
we extend the literature on the use of valuation experts (e.g., Griffith 2013; Griffith et al. 2014)
by providing evidence on the use of and challenges related to different types of experts,
including third-party pricing services and valuation specialists.
The remainder of this paper is organized into the following sections. The next section
discusses relevant background literature followed by descriptions of the survey design, data
collection and analysis. The fourth section presents our results. Finally, we discuss conclusions
and implications of this study.
BACKGROUND & RELATED LITERAURE
Previous research and our own analysis of PCAOB inspection reports identify numerous
deficiencies in the area of auditing FVMs (Church and Shefchik 2012; Griffith et al. 2014;
PCAOB 2010a). Valuation-related audit issues remain one of the top deficiencies noted and the
frequency of reported deficiencies has increased in recent years (Griffith et al. 2014; VRC 2013).
6
However, the firms’ have frequently disagreed with PCAOB conclusions (Church and Shefchik
2012) highlighting an apparent FVM expectation gap between auditors and regulators.
Previous research suggests that one source of the gap may be that auditors lack the
knowledge and expertise to fully understand and evaluate FVMs (Bratten et al. 2013; Griffith et
al. 2014; Martin et al. 2006). To address these limitations, auditors frequently engage valuation
specialists and pricing services to assist with valuation analyses (Bratten et al. 2013; Griffith et
al. 2014). However, extensive reliance on valuation specialists to assess the reasonableness of
assumptions and valuation methods has lead to additional audit deficiencies frequently cited in
PCAOB inspection reports, such as overreliance on the work of specialists (e.g., PCAOB 2010b;
PCAOB 2012a; Bratten et al. 2013).
Research regarding audits of FVMs is limited and in its early stages (Bratten et al. 2013;
Martin et al. 2006).2 To ensure that the direction of future research addressing auditing FVMs is
meaningful to practice and standard setters, Bratten et al. (2013) call for researchers to continue
to gain a better understanding of the process of auditing FVMs and the critical challenges
auditors are encountering in the field. To do so, several concurrent studies, using interview and
field-based survey methods, examine the process of auditing FVM, challenges auditors
encounter when auditing FVM, and auditors’ use of in-house valuation specialists (Bratten et al.
2013; Cannon and Bedard 2013; Griffith et al. 2014; Griffith 2013).
Griffith et al. (2013) interviewed audit partners and senior managers from six large audit
firms to better understand the process of auditing of complex estimates, including FVMs.
Consistent with the PCAOB inspection findings, Griffith et al. (2014) find that auditors tend to
over rely on management’s assertions, fail to notice and/or reconcile conflicting evidence
2We do note that recent experimental research investigates how various environmental and task factors affect auditors’ judgments and decisions in this complex audit area (e.g., Backof, Thayer, and Carpenter 2014; Earley, Hoffman, and Joe 2013; Griffin 2011; Griffith, Hammersley, Kadous, and Young 2013; Joe, Vandervelde, and Wu 2014; Maksymov, Nelson, and Kinney 2013).
7
between management’s assumptions and external evidence, and encounter coordination
difficulties when dealing with the firm’s in-house specialists. The paper identifies two potential
causes for the difficulties encountered by auditors when auditing of complex estimates. One is
that audit firm policies and auditing standards tend to over emphasize verification of
management’s model instead of “critically evaluating the reasonableness of the estimate”
(Griffith et al. 2014, 6). The other cause is that the division of knowledge and work between
auditors and the audit firm’s in-house valuation specialists results in auditors’ lack of knowledge
and expertise to evaluate assumptions and valuation models related to complex estimates.
Conducting interviews with audit partners and managers, Griffith (2013) provides more
detailed description of auditors’ use of the firm’s in-house specialist when auditing FVM and
problems encountered with the involvement of in-house valuation specialists. Griffith finds that
given the lack of clear auditing standards on the use of internal firm specialists, auditors adapt
existing guidance for use of external specialists to their use of an internal specialist. Her study
identifies three main areas that are challenging to auditors when using an in-house specialist.
They include (1) differences in views of auditors and valuation specialists, (2) coordination
difficulties between the two parties, and (3) ambiguity in the responsibilities of each party.
Another related study by Cannon and Bedard (2013), using a survey, provide additional
evidence regarding significant challenges audit managers and senior managers encounter when
auditing FVM. Consistent with the PCAOB inspection findings, they find that evaluating the
reasonableness of management’s assumptions and valuation methods is one of the most
frequently encountered challenge auditors face when auditing FVM. Cannon and Bedard (2013)
also find that a majority of participants report encountering extreme measurement uncertainty
such that the reasonable range is greater than materiality. Based on their results, they conclude
8
that auditors encounter difficulties with providing positive assurance on FVM and proposing
audit adjustments given that model assumptions and valuation methods are highly subjective.
Accessing practicing partners in the field, the current study extends prior literature in the
key following ways. Our study is the first to gather and report on audit partners’ views regarding
the PCOAB inspection process and primary causes for disagreement between auditors and
inspectors. Hence, we contribute to the literature by identifying ways in which the FVM
expectation gap can be closed. Second, while prior literature notes problems with current
auditing standards and guidance that are contributing to challenges auditors encounter with
auditing complex estimates (Bratten et al. 2013; Griffith 2013; Griffith 2014), this study extends
the extant literature by providing insight from audit partners regarding areas in which regulators
can provide clarification and additional guidance to help improve audit performance. Third,
concurrent studies focus primarily on auditors’ use and challenges related to use of the firm’s in-
house valuation specialists (Griffith 2013; Griffith 2014). The current study contributes to the
existing literature by exploring auditors’ use of third-party pricing services and valuation
specialists. Finally, this study contributes to prior literature by reporting on auditors’ views
regarding the auditability of and reporting on estimates with high estimation uncertainty along
with auditors’ views on areas in which they welcome improved clarity in the auditing standards
and guidance from standard-setters.
In light of PCAOB inspection reports and research in the area, we first present results
regarding audit partners’ views about the PCAOB inspection process and why in some cases they
disagree with the PCAOB inspectors. Partners provide their suggestions for how the PCAOB can
help improve auditing of FVMs as well as the inspection process. We next report results
regarding key challenges auditors encounter with respect to two key areas: (1) the
evaluating/testing management’s assumptions and valuation methods, and (2) the use of
9
valuation specialists and pricing specialists. Finally, we present results regarding the auditability
of and reporting on estimates with high estimation uncertainty. Results of this study, along with
insights from prior and other concurrent studies (e.g., Cannon and Bedard 2013; Griffith et al.
2014; Griffith 2013), will allow scholars, practitioners, and regulators to better understand how
to reduce the expectation gap noted above, and in turn improve auditor performance.
SURVEY DESIGN
We designed a survey that included both objective and open-ended questions.
Beforehand, we examined PCAOB inspection findings, prior studies, auditing standards, and we
also conducted interviews with two national-level partners. Two additional audit partners and
one retired audit partner who did not participate in the study provided initial feedback regarding
our survey. Further, partners from participating firms and the CAQ’s Research Advisory Board
provided additional feedback on the survey, which we incorporated to ensure our questions
captured the areas of primary importance and interest to auditors regarding auditing of FVMs.
The survey included several sections.3 The first section includes demographic questions
including position, level of involvement with FVMs, primary industry, specialization, etc. Other
sections were designed to gather information regarding auditors’ views in the following key
areas: key challenges when auditing FVMs and use of valuation experts, the PCAOB inspection
processes, and the auditability and presentation of estimates with high estimation uncertainty.
DATA COLLECTION AND ANALYSIS
Data Collection
We distributed our survey with assistance from the CAQ’s Director of Research, who
coordinated with five audit firms annually inspected by the PCAOB to identify audit partners
with experience in audits of FVMs. The majority of surveys were completed online; the 3Results of this study were collected using a survey that is also part of another study. We only present results of the survey that address the key objectives of the current study.
10
remaining surveys were completed via hard-copy. The survey was provided directly to potential
participants, and follow-ups were initiated to encourage participants to complete the survey.
Coding of Open-Ended Questions
Our survey included numerous open-ended questions to improve the potential richness of
the data gathered. Two of the researchers independently read responses associated with each
open-ended question to derive coding themes to categorize responses to each question and later
met to finalize the themes. Two doctoral students then independently categorized responses
based on the pre-established themes. Responses range in length from a few words to a brief
paragraph. Thus, individual responses could be coded to several themes or to as few as one. The
two Ph.D. students met to resolve coding differences. The Cohen’s Kappa measure is 0.61,
which suggests an acceptable inter-rater agreement level.4
Participants
Thirty-two audit partners responded to the survey. Table 1 presents participants’
demographic information. Mean response regarding the extent to which participants are involved
in the planning, supervising, or executing auditing significant estimates, including FVMs, is
10.28 on an 11-point scale anchored “very little involvement” (1) and “a great deal of
involvement” (11), indicating our participants have had significant involvement in auditing
FVMs. On average, participants have 22.3 years of audit experience. Participants practice
primarily in three industries: Financial Services/ Depository/Financial Institutions/Asset
Management (43.8 percent), Industrial Products and Manufacturing (25.0 percent), and
Consumer Products and Goods (15.6 percent). Participants rated their valuation expertise as 8.53
on an 11-point scale anchored with “very little expertise” (1) and “a great deal of expertise” (11).
4A rule of thumb is that Kappa measures between 0.40 and 0.70 are acceptable (Neuendorf 2002). Further, Landis and Koch (1977) provide benchmarks for interpreting Kappa. Kappa of 0.61 to 0.80 represents “substantial” inter-rater agreement strength.
11
Nearly 19 percent of participants indicate they specialized in a particular type of FVM. Top
specializations include investment securities, asset impairment, real estate and goodwill.
[INSERT TABLE 1 ABOUT HERE]
RESULTS
Auditors’ views regarding PCAOB inspection process
As noted, PCAOB inspection reports identify numerous deficiencies in the area of
auditing FVMs (Church and Shefchik 2012; Griffith et al. 2014; PCAOB 2010a). Consistent
with Griffith et al.’s (2014) analyses of inspection reports for 2008 and 2009, our analysis of
PCAOB inspection reports from 2007 - 2011 indicates that one of the most frequently noted
deficiencies PCAOB inspectors identify is failure to evaluate the reasonableness of
management’s underlying assumptions and valuation methods. Evaluating the reasonableness of
model assumptions is an especially challenging task given the degree of judgment and
measurement uncertainty involved (Bratten et al. 2013; Christensen et al. 2012; Griffith et al.
2014; PCAOB 2012a). To overcome these challenges, auditors have often turned to valuation
specialists and pricing services, which has resulted in PCAOB criticism over relying too much
on these specialists (e.g., PCAOB 2010b; PCAOB 2012a; Bratten et al. 2013).
Examples of both types of deficiencies include:
1. For hard-to-value financial instruments “the Firm failed to obtain an understanding of the specific methods and/or assumptions underlying certain fair value measurements that were obtained from pricing services or other third parties and used in the Firm’s testing of the fair value of the hard-to-value financial instruments…. for certain securities, the Firm tested the issuer’s process of obtaining prices from third parties and corroborating these prices through the use of models. The Firm, however, failed to evaluate the appropriateness of the models and the reasonableness of the underlying assumptions” (PCAOB 2011c, 17).
2. “The Firm failed to obtain an understanding of the specific methods and/or assumptions underlying fair value measurements that were obtained from pricing services or other third parties and used in the Firm's testing of the fair value measurements of certain financial instruments without readily determinable fair values (hard-to-value financial instruments)” (PCAOB 2012d, 16).
12
3. For goodwill impairment testing “the Firm selected the market approach to support its
audit conclusions, one of the three valuation methods used by the specialist…The Firm failed to sufficiently evaluate the reasonableness of the assumptions used by the specialist in that it failed to evaluate the comparability of the transactions used to develop the control premium, despite indications that the transactions included companies that differed significantly from the issuer” (PCAOB 2010b, 7).
The high degree of subjectivity inherent in FVMs along with ambiguity in auditing
standards require auditors to exercise significant professional judgment, which is susceptible to
second-guessing by the PCAOB in hindsight (Peecher et al. 2013). Consequently, the firms’
responses to the PCAOB inspection findings often indicate firm’s disagreement with the PCAOB
inspectors’ conclusions (Church and Shefchik 2012). For instance, firms often express in their
letters included in the PCAOB inspection reports that differences between auditor performance
and the PCAOB inspectors’ expectations should be expected given differences in professional
judgment (e.g., PCAOB 2011c; PCAOB 2008b).
Although a number of factors undoubtedly contribute to FVMs expectation gap,
including situations in which the audit actually was deficient, we first consider partners’ views of
the inspection process and address auditors’ reported challenges in subsequent sections. To learn
more, our survey addresses the following questions related to the PCAOB inspections.
1. Do the PCAOB inspectors expect the audit team to collect more evidence than required by existing auditing standards?
2. What are primary reasons audit firms disagree with the PCAOB’s assessment when inspectors identified an audit deficiency related to the auditing of FVMs?
3. How can the PCAOB help improve the auditing of FVMs as well as the inspection
process? Over 75 percent of our participants (N=23) served as the engagement partner on an
engagement that has been selected for PCAOB inspections. Participants whose engagement had
been selected for PCAOB inspection were instructed as follows: “In answering the following
13
questions in this section, if you have had more than one audit engagement inspected, please
consider an engagement that contained the most complex FVMs.”
To address our first key question, we asked engagement partners: “In your opinion, did
the PCAOB inspectors expect the audit team to collect more evidence than what the firm
believes represents the requirements of auditing standards (e.g., PCAOB AU 328) when auditing
FVMs?” As shown in Panel A of Table 2, approximately 60 percent of participants believe that
the PCAOB inspectors expected more than the auditing standard requires. Consistent with these
engagement partners’ views, one national partner provided an example:
“The engagement team documented risks associated with various investments and thus bi-furcated the population into various risk buckets. The engagement team took each bucket and designed testing strategies. [The] PCAOB did not have any issues with the higher risk buckets. However, in the low risk bucket (only had 49 securities - level 1), the engagement team tested 38 of the 49 but did not test an item over [tolerable misstatement]. Overall, 80 percent of fair values had been tested through various strategies involving risk ratings and we disagree that [auditing standards require that] the engagement team MUST test the item over [tolerable misstatement] in the low risk bucket. The PCAOB inspector disagreed. When they write observations, you must address them. Most of the time, it is less costly to agree and perform some minor testing and put the matter behind us, than to fight.”
[INSERT TABLE 2 ABOUT HERE]
This raises a potentially interesting question for PCAOB leadership to consider and
clarify for low risk securities that represent low risk of material misstatement: do the auditing
standards require substantive testing of each security over tolerable misstatement? Audit theory
and practicing experts would say no. However, as the respondent points out, theory and practice
are of little comfort when inspector’s criticisms must be addressed.
Another national partner also elaborated on the challenge of gathering sufficient evidence
in subjective and judgmental areas:
“By experience, no matter how much evidence you gather and you believe you have gathered sufficient appropriate audit evidence on judgmental assumptions, someone can always second guess you. What is reasonable? Reasonable is extremely difficult concept and one that causes massive issues in the audit industry. Your concept of reasonable and
14
my concept of reasonable will always likely be different. When you add pages of documentation to support why you believe something is reasonable, it provides additional information to be challenged as well. This is a never-ending cycle. Auditors must make decisions based on the evidence gathered and sometimes looking at something [with the benefit of hindsight] a year later - even the auditor may think - hmmm, maybe I should have done something different. But, really - at the time the judgment had to be made - it was reasonable based on [the circumstances and] all the evidenced gathered.” This dilemma provides another fertile area for the PCAOB to consider both the
profession’s frustration with the “never-ending cycle” and the practical realities of the
tremendous subjectivity inherent in the concept of “reasonable.” The PCAOB should also
recognize the fact that assessments of what is “reasonable” in hindsight versus foresight are
likely to differ. Perhaps auditing standards and inspection policies could be revised to
contemplate these issues and provide guidance in ways auditors can appropriately exercise
professional judgment to gather sufficient appropriate evidence without fear of PCAOB
inspector reprisals based on hindsight and differences of interpretation rather than legitimate
shortcomings in the auditor’s performance at the time of the judgment.
To identify reasons for auditor disagreements with PCAOB inspection results, we asked
audit partners that had their engagements inspected by the PCAOB whether PCAOB inspectors
identified any audit deficiency related to the auditing of FVMs. Roughly half (N=11) of our
participants indicated “yes.” For those that answered “yes” we asked whether they agreed with
the PCAOB’s assessment and to explain why or why not. Even with the benefit of hindsight and
knowing the concerns articulated by the PCAOB inspectors, 100 percent of the engagement
partners disagreed with the PCAOB’s conclusions. We asked why they disagreed. Table 3
reports the three themes that emerged from the open-ended responses. The most frequently
mentioned reason for disagreement is that the partners believed that the PCAOB did not give
appropriate consideration to the risk of misstatement, the nature of the security, and the totality
of the audit evidence gathered. For example, one partner noted:
15
“The PCAOB comments related to roll-forward procedures on a low risk securities portfolio (the roll-forward analytical was inside tolerable range but the overall balance was greater than materiality) and the PCAOB did not give appropriate consideration to the risk of misstatement, the nature of the securities, and the totality of audit procedures performed in the roll-forward period. The PCAOB is attempting to legislate auditing standards and improvements to management’s internal controls through their [inspection] process instead of issuing appropriate guidance or examples.”
Another partner indicated:
“We believe we had tested a significant portion of the investment portfolio and had covered the key risks. We also evaluated the key controls over valuation and placed reliance on those controls, yet the PCAOB gave us no “credit” for the controls testing.”
This is another area for the profession as well as PCAOB leadership and inspectors to consider—
What evidential “credit” or assurance is provided to the audit of internal control over financial
reporting? Auditing theory and practice suggests risk assessment and controls testing are
important considerations in evaluating the nature, timing, and extent of substantive testing.
Consistent with practice partners’ responses, one national office partner noted that differences
exist regarding what PCAOB inspectors and auditors believe constitutes an audit failure:
“Evaluation to the audit as a whole. [The] PCAOB believes if a [potential] difference is greater than performance materiality, an audit fails. No credit is given to other evidence. They measure to [a hypothetical standard of] what might happen, regardless of [the] remote [likelihood].”
[INSERT TABLE 3 ABOUT HERE]
This may be yet another area for the profession and the PCAOB to consider—What
constitutes an overall audit failure and how should auditors properly factor in the totality of audit
evidence and the risk of misstatement in the evaluation of the magnitude of a perceived audit
deficiency when considering whether it constitutes an audit failure.
The second most frequently mentioned reason for disagreement is that the PCAOB
expects a level of precision that is not be achievable, as one partner notes:
“In my opinion, the PCAOB standard of auditing regarding estimates, assumptions and inputs to valuation models are wholly unreasonable. By definition, the fair value models
16
are estimates [that] cannot be audited to the level of precision that the PCAOB is expecting.” Our findings combined with similar findings in other studies (e.g., Christensen et al.
2012, Cannon and Bedard 2013) suggests that the profession and the PCAOB may find it fruitful
to acknowledge the potential for inherent and irreducible uncertainty in complex Level 2 and
Level 3 FVMs and to consider the appropriate ways to report the uncertainty and what
constitutes sufficient appropriate audit evidence to support an FVM with a reasonable range of
estimation uncertainty that exceeds auditor materiality. 5 Related to this issue, we provide further
discussions later in the paper regarding the reporting of and auditability of estimates with
extreme measurement uncertainty.
The third most frequently mentioned reason for the disagreement with the PCAOB is
participants’ belief that the PCAOB’s expectation regarding management and auditors’ reliance
on third-party specialist is not reasonable. As one partner describes:
“The PCAOB’s expectation as to the level of management’s review of [third] party pricing is at a level that is likely not reasonable. The regulator seemed to assert that the issuer is required to actually calculate a specific fair value for a given security as its control, or to specifically validate each assumption in a specific security’s value in order to monitor its pricing vendor. However many company’s have knowledge of broad asset classes and movements within market indices that would detect significant differences in security values, without having to actually employ another specialist to validate their chosen specialist (pricing vendor).”
Later we discuss challenges auditors encounter when using third-party valuation and pricing
services and ways in which regulators can help improve auditor performance.
Obviously, reasonable disagreements between experienced audit partners and PCAOB
inspectors will occur. The fact that all of our partners disagreed with the PCAOB inspector’s
conclusions along with the reasons for disagreement noted above provide the PCAOB an
opportunity to consider ways to (1) improve auditing standards and guidance to clarify what 5We discuss further below that partners report that it is common for complex estimates to have significant and irreducible estimation uncertainty many times larger than performance materiality.
17
constitutes sufficient, appropriate evidence when auditing complex FVMs, (2) better train
inspectors on audit risk, evidence, accounting for FVMs, and what constitutes an audit deficiency
in the area of auditing FVMs, and (3) create an appeal or arbitration process whereby important
but reasonable differences of opinion between engagement teams and inspectors can be elevated
and resolved, perhaps by independent experts. Some form of appeal or arbitration process would
undoubtedly go a long way toward improving audit quality as it would help to clarify areas
where auditors need to improve. Also, such a process would identify areas where inspectors may
be imposing unnecessary costs and inefficiencies in terms of the firms’ responses to specific
inspection findings (i.e., additional audit procedures performed) as well as firms’ changes to
audit methodologies vis-à-vis areas of “inspection risk” (Glover and Prawitt 2013).
Finally, we asked partners to provide suggestions about how the PCAOB could help
improve auditing FVMs. Results presented in Table 3 show that the most frequently mentioned
suggestion is the PCAOB to use a risk-based approach consistent with auditing standards and to
consider the totality of audit evidence when auditing fair values. One partner stated:
“While risk-based audit approaches are routinely discussed in SEC/PCAOB literature, [inspection] teams have struggled to understand or be satisfied with various sampling approaches rather than achieving near or actual 100 percent coverage of the population. Fair value measurements routinely involve the use of multiple assumptions. PCAOB inspectors struggle to differentiate the audit work performed on certain, but not all of those assumptions, when certain of those assumptions have the greatest potential for impact or change in that value.”
Another partner also indicated:
“Focus on where the risk of misstatement is highest. This would be in the area of whether a security is Level 2 or Level 3. Then focus on how we look for management bias or incompetence in their estimation process. The PCAOB seems to preach about adequate risk assessments, then looks at every security like it should be treated the same way. And if they really want to improve audit quality, then they should treat all firms in the inspection process the same way. My understanding is that they do not look at the ICFR work of the firms that are not inspected annually (that was told to me by the PCAOB during this year’s inspection). That creates an uneven playing field in the market place that requires larger firms to do more audit work than smaller firms (that may lack
18
expertise and not even know it). If you want to know how that is working out, just ask Madoff’s investors.”
The second most frequently mentioned recommendation is for the PCAOB to provide
clearer guidance of audit requirements regarding, for example, evidence, audit procedures and
documentation. One partner explained:
“We have moved away from auditors being able to apply audit judgment in the area of fair value measurement to where it is now a rules-based approach. That is all fine and good as long as the rules are clear and effectively communicated. However, the problem is that the PCAOB continues to change their expectations and the primary mechanism for communicating these changes seems to be through inspection comments versus proactively educating the profession on their expectations. This “gotcha” approach is counterproductive to improving the quality of audits. Rather, more energy should be spent by the PCAOB on auditor outreach.”
A national office partner’s response further emphasizes the practicing partner’s opinion:
“The audit standards need to address both the accounting literature as well as the ability to operationalize the requirements. We have no responsibility to manufacture evidence, but what [audit evidence] is available to all market participants and evaluated as to consistency and method (such as used by pricing services) is not accepted as adequate. We evaluate the audit evidence available. If we conclude it is relevant and reliable it may be used. The PCAOB would assume we need to evaluate all other choices. Is the auditor required to hire a specialist for every security for which there is no market? Or is the auditor to assess the risk of error to the financial statements as a whole?”
The third most frequently mentioned recommendation is for the PCAOB to acknowledge
that evidence gathered may not be able to get to the level of precision desired (i.e., within
materiality, inspectors need more realistic expectations). One partner explains:
“Evaluate the procedures in light of a reasonable range around the fair value estimate rather than in light of materiality, which can often be too small a range to use for purposes of evaluating an estimate for which there is no correct answer.”
Another partner adds:
“[For] allowance for loan losses or impaired loans, the level of precision between cause and effect is not as real as they would like it to be. Therefore their demands on conclusions to be based on accurate estimates using judgment cannot be supported as precisely as they would like. So a lot of time is spent in drafting memos and other support for estimates that end up being [imprecise] estimates in the end.”
19
In summary, our results suggest that differences exist between auditors and the PCAOB
inspectors regarding interpretations/judgments about the amount of evidence required when
auditing FVMs. In addition, participants overwhelming report disagreeing with the PCAOB
inspectors’ identified audit deficiencies related to FVMs. One primary reason for the
disagreement is that participants felt that the PCAOB inspectors did not give appropriate
consideration to the level of audit risk, nature of the security, and the totality of the audit
evidence gathered. Consequently, the top suggestion participants provide regarding how the
PCAOB could improve the audit of FVMs is for inspectors to use a risk-based approach
consistent with auditing standards, and to consider the totality of audit evidence gathered.
Results also reveal that current auditing standards may not provide sufficient guidance to
inform auditors on how complex FVMs should be audited. Instead, changes in audit practice are
being driven by the inspection process rather than through changing existing auditing standards
(cf. Griffith et al. 2014). This approach is problematic in that audit effort is driven the likelihood
of PCAOB scrutiny or “inspection risk” rather than audit risk (Glover and Prawitt 2013). This
concern is further emphasized by one national office partner’s response:
“We are less likely to do what is reasonable and required by standards and more likely to appease the inspection team.”
Another practice partner also notes audit strategies that are being driven by the inspection
process rather than audit risk:
“The deficiency related to the precision of the review by management of the pricing assessments made by the third party engaged by management. The issuer has a non-complex securities portfolio, engaged a national broker/dealer to value securities, obtained a SAS 70 that included testing of securities pricing as a key control, but it wasn't precise enough for the PCAOB. The PCAOB seems to be focused in the right areas in their exams, but they don’t pay particular attention to real risk of misstatement. The focus should be solely on identifying and valuing hard to price securities, not every security. In this case, we engaged a third party every year to value 100 percent of the securities and we never had a security price fall outside of our acceptable range of prices, but this was a substantive test the PCAOB didn’t consider a dual test when assessing our ICFR work.”
20
Audit efforts driven by inspection risk rather than audit risk could potentially lead to audit
efficiency and effectiveness issues. For lower audit areas, auditors may be over-auditing which
potentially shifts auditors’ focus and resources away from higher-risk areas (e.g., level 3 FVMs).
Moreover, professional judgment must be exercised when deciding which of several
applicable auditing standards (e.g., AU 328, Auditing Fair Value Measurements and Disclosures
and AU 342, Auditing Accounting Estimates) and practice alerts to apply when auditing FVMs
(AICPA 1989; AICPA 2003). The presence of multiple standards, practice alerts, and additional
clarifications through informal channels further compounds auditors’ task difficulty when
auditing FVMs (Bratten et al. 2013). Our findings emphasize the need for improvement and
clarification to existing auditing standards with respect to auditing FVMs.
Challenges in auditing of fair value measurements
In the previous section we report results regarding auditors’ views of the PCAOB
inspection process and areas in which audit partners suggest that regulators can improve auditing
of FVMs. In this and the next section, we explore key challenges auditors encounter when
auditing FVMs. Specifically, we focus on two key areas that PCAOB inspection reports
frequently identify as containing audit deficiencies: (1) evaluating the reasonableness of
management’s assumptions and valuation methods, and (2) using valuation experts, including
pricing services, in-house and third-party valuation specialists. In doing so, we identify areas that
may be contributing to the expectation gap. This section focuses on general challenges related to
auditing of FVMs; the next section focuses on challenges related to use of valuation experts.
When we asked participants what are the most significant challenges encountered when
auditing inputs for significant level 2 or level 3 FVMs where small changes to the inputs have
potentially material changes to the reported value, the three top challenges reported in Table 4
are (1) lack of verifiable and corroborative evidence including lack of 3rd party evidence (42.9
21
percent), (2) degree of difficulty in assessing reasonableness of assumptions and benchmarks
(14.3 percent) and (3) high degree of subjectivity involved in estimates (14.3 percent). To
elaborate on the challenges around obtaining verifiable and corroborative evidence, one partner
points out the following:
“The most difficult challenge is determining whether the assumption represents "market". Usually you can look at historical experience to determine if assumption is reasonable for the specific asset but it is hard to tell is such is what the market might use. For example prepayment assumption, you can determine based one experience of the asset, but that does not mean the marketplace would use similar assumption as they would build in some expectation of what interest rates are going to do in the future.”
Another partner echoes the above point:
“[A key challenge is] obtaining enough evidence to support the overall assertion, which is generally going to be based on numerous judgments that can't be proven but have to be considered for reasonableness.”
Consistent with PCAOB inspection findings, audit partners note significant challenges with
assessing the reasonableness of assumptions. For example one practicing partner notes:
“Where there is a lack of liquidity, fair value measurements are incredibly challenging. With the benefit of hindsight, what appeared to be reasonable assumptions at the time they were developed can appear to be entirely unreasonable. Often times, I do not think management is doing enough documentation and memorialize the thinking and rational at the time assumptions are developed.”
Finally, in terms of challenges with high degree of subjectivity involved in estimates, one
practicing partner points out the following:
“[the] biggest challenge is that inputs within a reasonable range could result in indicated values that differ greater than materiality. This precludes the ability to "stress-test" management's analysis. Thus, supporting one number for an input over another is a matter of significant judgment.”
[INSERT TABLE 4 ABOUT HERE]
Results and audit partners’ comments provided above highlight key challenges auditors face
when auditing complex FVM. These very challenges are likely causes of why the PCAOB
inspection findings repeatedly report deficiencies in the area of evaluating the reasonableness of
22
management’s assumptions and valuation models. Further, these challenges reported by our
participants raise questions regarding the auditability of estimates with extreme measurement
uncertainty (discussed later).
We also asked participants whether they believe audit challenges they previously
described are different between financial vs. nonfinancial asset/liabilities. Approximately 63
percent report that they believe challenges differ between these two types of FVMs. As shown in
Table 4, the top two most frequently mentioned differences are (1) difference in availability of
market data (56.5 percent), and (2) degree of auditor familiarity (13 percent). In terms of
differences between financial and non-financial assets/liabilities, one partner notes,
“It has been my experience, that when dealing with nonfinancial assets/liabilities you are typically having to rely on management prepared data (e.g. forecasts), where for many financial assets you are able to source some other independent data.”
Similarly, another partner mentions,
“Financial instruments are more likely to have some observable market comparisons than non-financial instruments.”
Finally, another partner expresses the following:
“There seem to be more third party valuation experts involved in valuing financial assets/liabilities. The valuation for nonfinancial assets/liabilities relies more heavily on management’s assumptions.”
Our study extends prior research by providing evidence that the challenges encountered in
auditing of FVMs differ between financial and nonfinancial assets/liabilities. The above findings
and audit partner comments indicate that auditors encounter ever more difficulty when trying to
obtain verifiable and corroborative evidence from independent sources for non-financial
instruments, thus leading to more reliance on management’s assumptions.
Management’s lack of knowledge can create additional challenges for audit teams when
auditing FVMs. As presented in Panel B of Table 2, approximately 58 percent of participants
report frequently encountering situations in which management’s knowledge of valuation
23
processes and methodologies is relatively shallow. We asked participants to describe their team’s
audit responses when management’s knowledge of valuation processes and methodologies is
relatively shallow, and how such situations impact audit procedures around FVMs. Table 4
reports the top four audit responses are (1) auditor expects that management will place greater
reliance on outside specialists (20.7 percent), (2) consider impact on internal control
effectiveness and inherent risk (19 percent), (3) auditors place greater reliance on valuation
specialists (17.2 percent), and (4) perform additional audit procedures (17.2 percent). Results
suggest that when management’s knowledge regarding the valuation process and methods is
limited, auditors anticipate greater reliance on valuation specialists both by management and the
audit team.
Taken together our results suggest that auditors consider the impact of management’s
lack of knowledge of the valuation processes and methodologies on internal controls assessment
and inherent risk. However, the above results indicate that auditor’s react to potentially higher
control and inherent risks by placing greater reliance on valuation specialists. This finding
highlights auditors’ and management’s potential overreliance on specialists when auditing
complex FVMs, consistent with the PCAOB inspection findings.
Auditors’ use of valuation experts
Research regarding the use of pricing services and valuation specialists when auditing
FVMs is limited. Based on interviews with audit partners and managers, Griffith (2013) provides
insight regarding auditors’ use of in-house valuation specialists. The current study extends the
literature by distinguishing between auditors’ use of third-party pricing services and valuation
specialists (both in-house and 3rd party). In this section, we highlight challenges auditors
encounter when using valuation experts that may be contributing to the FVMs expectation gap.
24
PCAOB inspecting findings often note that auditors over rely on pricing services and
valuation specialists (both third-party and in-house) by failing to obtain sufficient understanding
of the specific assumptions and methods used by the pricing service to derive the FV estimate
(e.g., PCAOB 2008b; PCAOB 2011a; PCAOB 2011b; PCAOB 2012b; PCAOB 2012c).
Therefore, we ask questions directed at better understanding how auditors assess the
reasonableness of assumptions and methods used by pricing services and valuation specialists.
First, with use of pricing services, results in Table 5 indicate that for audit areas considered to be
“higher” risk, participants most frequently mention using the following approaches to assess the
reasonableness of specific assumptions, methods, and models used by third-party pricing
services: (1) involve the firm’s in-house specialists (26.7 percent), (2) gain an understanding of
the pricing methodology—e.g., method, model, and assumptions (24.4 percent), and (3)
comparison with auditor’s independent estimate, another pricing service, or similar instrument
(22.2 percent). Second, with use of valuation specialists, we find that the three most frequently
mentioned approaches, as presented in Table 5, are (1) obtain support/audit evidence for data
used in model from both internal and external sources (30.2 percent), (2) reliance on audit firm’s
in-house valuation specialist to evaluate the appropriateness of assumptions (24.5 percent), and
(3) auditor to gain understanding of key assumptions via the firm’s valuation specialists (15.1
percent). These results further emphasize that auditors may lack the training and expertise to
perform the most subjective aspects related to auditing FVMs. Consequently, they extensively
use in-house valuation specialists to assess reasonableness of assumptions underlying the FVMs.
[INSERT TABLE 5 ABOUT HERE]
The PCAOB has set up a Pricing Source Task Force to address challenges auditors
encounter when auditing FVMs of financial instruments. To inform regulators, we asked our
participants what challenges they encounter when using pricing services. One of the primary
25
challenges reported is obtaining sufficient evidence to support the service’s proprietary model
and assumptions used to derive the FV of investment securities. As presented in Panel B of Table
2, results show that one third of our participants report that the audit team is frequently unable to
obtain sufficient information on the third-parties’ proprietary models or assumptions. This issue
further exacerbate difficulties auditors are encountering when trying to understand and assess the
reasonableness of management’s assumptions and valuation methodologies. The national office
partners we surveyed indicated that some pricing services continue to be reluctant to share
proprietary information. A national office partner stated:
“Pricing services will not share with you their assumptions and models on how they priced investments. We have worked with one in which we can get that information and thus, much of our practice in this area relies on that pricing service. For Level 3 investments we tend not to have the pricing service provide a price since we believe it is more effective and efficient to test management’s assumptions used to price the instrument.”
Another national office partner notes:
“Inconsistency between pricing services and a lack of full transparency of the computation. I recognize that the latter is a function of safe guarding corporate processes and is unlikely to change. The former is extremely challenging - if the client uses one pricing service and the team uses a different pricing service - at times the approaches are often different. We see this with CMOs as an example.”
Further emphasizing the issue, a national office partner points out:
“Although substantial progress has been made in the last couple years, brokers and pricing services often treat their valuation methods and models as proprietary. So, "getting behind the model" is easier said than done. It is expensive and often a matter of triangulation rather than transparent insight.”
Finally, a national office partner expresses the need for clearer guidance in this area:
“We need or would like more immediate transparency but currently it is largely only available for actively traded securities when transparency is not needed. There are no guidelines for providing pricing, minimum requirements or standards to assure greater consistency.”
One national office partner questioned whether it would make sense for PCAOB standards to
cease requiring the auditor to obtain such information. Alternatively, perhaps the PCAOB can
26
work with the pricing industry to foster cooperation in sharing information with auditors, under
confidentiality agreements, to comply with PCAOB auditing standards.
In summary, our findings provide insights that should be helpful for regulators as they
consider how to improve performance in this area and narrow the FVM expectation gap. Our
results suggest that auditors frequently rely on both pricing services and valuation specialists to
assist with the most subjective and challenging aspects of FVMs, evaluating management’s
assumptions and/or valuation models. As previously discussed, overreliance on pricing services
and valuation specialists is a frequently noted deficiency in the PCAOB inspection reports
(PCAOB 2011a; PCAOB 2012b; PCAOB 2012c). Despite auditors’ reliance and inspection
findings, current PCAOB auditing standards do not provide clear guidance to auditors’ regarding
use of in-house specialists vs. third-party pricing services and valuation specialists (e.g., Griffith
2013). Therefore, providing additional guidance to auditors regarding use of pricing services and
valuation specialists is another area in which regulators can help close the FVM expectation gap.
High estimation uncertainty and presentation
The degree of measurement uncertainty and prevalence of FVMs in financial statements
have increased over time (e.g., Bratten et al. 2013; Christensen et al. 2012; PCAOB 2012a).
Consistent with this notion, as presented in Panel C of Table 2, nearly 97 percent of the audit
partners’ surveyed in our study agreed that over the last two decades the number of FVMs
included in financial statements has significantly increased and nearly 94 percent agreed that the
relative proportion of FVMs with high degrees of measurement uncertainty has increased.
Christensen et al. (2012) highlight “extreme measurement uncertainty” that can exist in
the financial statements in which the reasonable range of estimation uncertainty on the fair value
measurement date can be many times larger than materiality. Results of the current study
support Christensen et al. (2012) assertions. Approximately 33 percent of participants (Panel C
27
of Table 2) indicate that they frequently encounter situations in which the degree of measurement
uncertainty exceeds materiality. Cannon and Bedard (2013) report that over 70 percent of their
participants indicate encountering measurement uncertainty that is equal to or greater than
materiality when considering the most complex FVMs.
Given the increase in measurement uncertainty, Christensen et al. (2012) question
whether auditors can provide assurance on accounts that contain extreme measurement
uncertainty. They assert that “no amount of auditing can remove the extreme uncertainty inherent
in reported values derived from management’s valuation models based on unobservable inputs
subject to estimation uncertainty” (Christensen et al. 2012, p. 143). Bell and Griffin (2012) also
question the auditability of estimates with extreme measurement uncertainty, including FVMs.
Further, Christensen et al. (2013) report that auditors and investors believe that high estimation
uncertainty significantly affects auditors’ ability to provide the requisite level of assurance.
Highlighting estimation uncertainty could be the subject of proposed critical audit matter
paragraphs as described in the PCAOB proposal to revise audit reports (PCAOB 2013). The
proposal retains the existing pass/fail reporting model, but would require auditors to
communicate critical audit matters in a separate paragraph. Christensen, Glover, and Wolfe
(2014) find that investors who receive a critical audit matter paragraph highlighting uncertainty
in a FVM are more likely to change their investment decision than are investors who receive the
standard report or investors who receive the same information as in management’s footnotes.
The views expressed by Christensen et al. (2012) and Bell and Griffin (2012) appear
contrary to expectations of inspectors and regulators. Previously, we reported that PCAOB
inspectors and some in the PCAOB leadership may believe that more auditing can reduce the
apparently irreducible inherent uncertainty associated with complex FVMs. While the prior
literature calls into question the auditability of estimates with such extreme measurement
28
uncertainty, ours is the first study to gather and report on audit partners’ views regarding this
issue. Our findings provide important insights and recommendations for how practice can be
improved. Specifically, in this section we address the following key questions:
1. How can the level of measurement uncertainty in complex estimates be best communicated to users of financial statements?
2. Given the increase in estimates with extreme measurement uncertainty, do auditors believe the current requirement to provide a high level of positive assurance that a point estimate is fairly stated be reconsidered? Why?
3. How can existing auditing standards provide further clarification and guidance?
We asked audit partners’ views regarding the current communication of measurement
uncertainty to users of financial statements and how best to convey the level of uncertainty.
Responses in Panel C of Table 2 indicate that 50 percent of the participants disagreed that current
footnote disclosures around the level of uncertainty at the measurement date are consistent
among preparers and only 50 percent agree that current footnote disclosures about measurement
uncertainty are adequate. Our findings raise potential concerns regarding reporting consistency
and adequacy. We also asked: “For estimates with a reasonable range of estimation uncertainty
that is greater than overall materiality, which of the following methods do you believe would be
most useful to users in conveying the level of uncertainty.” Table 6 provides a list of suggested
methods and the frequency that audit partners selected each method. The top three methods
selected are: (1) narrative description of estimation methods and the source and level of
uncertainty (37 percent), (2) quantitative disclosure of the range of reasonable estimates at the
measurement date (30 percent), (3) tables in footnotes listing measurement assumptions and the
values used (18 percent).
[INSERT TABLE 6 ABOUT HERE]
Recent commentaries suggest that the current audit approach for reported amounts with
extremely high estimation uncertainty is more consistent with negative assurance rather than
29
positive assurance. PCAOB and SEC rules require that auditors provide reasonable positive
assurance (interpreted in the auditing standards to be a high level of assurance) that all account
balances are fairly stated in all material respects. To address the issue of auditability of estimates
with extreme measurement uncertainty (the second key question in this section), we asked
participants whether they believe the current requirement to provide a high level of positive
assurance that a point estimate is fairly stated be reconsidered when the estimate’s reasonable
range of estimation uncertainty is many times larger than overall materiality (e.g., consider
instead providing positive assurance on the estimation process). Approximately 60 percent of the
audit partners responded “yes” to this question, which is telling for a couple of reasons. First, it
raises the very real possibility that the current requirement for auditors to provide a high level of
positive assurance that all point estimates reported in the financial statements are materially
correct is not realistic and that current auditing standards and regulator expectations may be
demanding a service that even the best and brightest simply cannot provide. Second, our
conversations with some national office partners suggests that the profession takes enormous
pride in providing positive assurance on the most difficult and complex estimates, including
FVMs, and that no one can do a better job than the outstanding professionals at the largest
firms,6 and (3) auditors, like most people, get comfortable with the status quo and are hesitant to
consider or recommend what some might consider radical change.
We asked the 60 percent of participants that responded “yes” to explain why they think
auditing standards and the level of assurance associated with highly uncertain estimates should
be reconsidered. Table 7 indicates that the top two reasons are (1) it is unrealistic to provide
assurance on point estimate given the level of subjectivity, complexity and uncertainty of
6Christensen et al. (2013) report one partner in their survey commented, “in regard to impacting the provision of assurance, there’s always a way to get there. If there isn’t, you shouldn’t be issuing an opinion.” The authors note that the profession recognizes “the practical realities they face. Until and unless the auditing standards change, auditors are prepared to continue doing their best to provide the desired level of assurance.”
30
estimates (28.6 percent), and (2) no amount of evidence could achieve high level of assurance
given high estimation uncertainty (23.8 percent). For those who responded “no,” standards and
the level of assurance should not be reconsider, their top two reasons are (1) the problem is
inherent in the audit process, thus no change is warranted (31.6 percent), and (2) auditing and
financial reporting has always involved estimation and uncertainty (21.1 percent).
[INSERT TABLE 7 ABOUT HERE]
To address our third key question in this section, we asked participants whether their firm
would support additional clarity and guidance from standard setters and/or regulators regarding
auditing estimates including FVMs. Approximately 93 percent of participants indicated “yes”
they would support additional clarity and guidance. For participants that answered “yes,” we
asked them to provide specific recommendations to add to the existing literature. For the 57
percent of participants that did provide a recommendation, Table 7 indicates that the top three
recommendations are (1) more guidance regarding an acceptable reasonable range of estimation
uncertainty (29.4 percent), (2) more clarification in standards regarding audit requirements and
procedures (29.4 percent), (3) the standard-setting process should influence audit procedures and
requirements rather than using PCAOB inspection process to drive behavior (17.6 percent). A
specific recommendation mentioned by one practice partner is:
“I think [the issue] needs more clarity just so there is a common understanding of what is required. Clients don’t understand the extent of procedures that are required because they have been forced on the auditor by the PCAOB via the inspection process, but are not well spelled out in their published standards.”
Another practice partner recommends adding:
“…clarity around range of tolerable differences when there is high estimation uncertainty.” A national office partner also specifically noted an area for improvement:
31
“It needs to [come] from standard setters - frankly there needs to be an emphasis from the SEC on the need for the registrants to tighten their estimates within an appropriate range. This is an area of audit confusion - more guidance of the application will be very useful.”
We concluded the survey by asking participants, in the context of high estimation
uncertainty, whether they believe a potentially viable approach in the future (assuming standards
change accordingly) is to require the auditor to provide assurance on the rigor and quality of the
estimation process and negative assurance on the reported point estimate (i.e., the entity followed
an appropriate and rigorous process to form the estimate and nothing came to our attention that
would suggest the reported point estimate is not fairly stated). Approximately, 36 percent of
participants responded “yes.” We follow up this question by asking these participants to explain
their response. Table 7 shows that the top two explanations are (1) suggestion is consistent with
other private equity fund opinions, represents current environment, and is reasonable (40
percent), and (2) suggestion offsets the extremely high expectations of regulators regarding level
of assurance possible for estimates with high estimation uncertainty (20 percent).7
Collectively, we find that the majority of audit partners surveyed (60 percent) believe that
the current requirement to provide a high level of positive assurance on a point estimate should
be reconsidered. Results in this section further highlight the FVM expectation gap that exists
between auditors and regulators. As noted earlier, PCAOB inspectors and some in the PCAOB
leadership may believe that more auditing can reduce the irreducible inherent uncertainty as
evidenced by PCAOB inspection findings. Our findings provide insight for regulators and
standard-setters regarding areas in which they can help improve the auditing of FVMs, and
perhaps reduce the FVM expectation gap. These areas include the following: regulators and
standard setters consider the feasibility of the requirement to provide a high level of positive
7For those who responded “no” to positive assurance on the estimation process and negative assurance on the point estimate, the top explanation provided is that such a change is inconsistent with other standards as the auditor is responsible to provide positive assurance the financial statements taken as a whole, so that the positive opinion cannot exclude FVMs (27.8 percent).
32
assurance on estimates with extreme estimation uncertainty; standard-setters and regulators
should reconsider how estimation uncertainty is conveyed to financial statement users; and
provide more guidance and clarification about what is sufficient, appropriate audit evidence to
support FVMs with extreme estimation uncertainty.
CONCLUSIONS
The current study aims to begin to close the gap between auditor performance and
regulators’ expectations around audits of FVMs. Participants largely believe that the PCAOB
inspectors expect more evidence than auditing standards require, thus suggesting that existing
guidance requires further clarification. This discrepancy often leads to auditor disagreement with
the PCAOB inspector’s assessments of audit deficiencies.
We find that one frequently reported reason for auditor disagreement with the PCAOB is
that inspectors did not appropriately consider risk of misstatement and totality of the audit
evidence gathered. This is also the area audit partners surveyed most frequently mentioned as an
area in which the PCAOB could help auditors improve performance and improve the inspection
process. This issue if unaddressed could result in audit efforts that are not driven by audit risk as
prescribed by auditing standards, but instead incorrectly driven by “inspection risk” (Glover and
Prawitt 2013). Consequently, auditors may end up focusing more attention on managing
inspection risk while devoting less effort to areas with the greatest audit risk.
Another frequently reported reason for auditor disagreement with the PCAOB inspectors’
assessment is that inspectors expect a level of precision that may not be achievable given the
nature of FVM. This finding is consistent with questions raised in prior literature regarding the
auditability of estimates with extreme measurement uncertainty (Bell and Griffin 2012;
Christensen et al. 2012). Further, our results show that a majority (60 percent) of participants
believe that the current requirement to provide a high level of positive assurance that a point
33
estimate is fairly stated should be reconsidered particularly when the estimate’s reasonable range
of estimation uncertainty is many times larger than overall materiality. Participants frequently
expressed that providing assurance on point estimate given subjectivity, complexity and
uncertainty of estimates is unrealistic to and that no amount of evidence could achieve high level
of assurance given high estimation uncertainty as a key reasons why the current requirement
should be reconsidered. Moreover, an overwhelming majority of participants support additional
clarity and guidance from regulators regarding auditing of estimates with high estimation
uncertainty. The top two recommendations mentioned by participants to improve existing
auditing standards are (1) provide more guidance regarding an acceptable reasonable range of
estimation uncertainty, and (2) provide more clarification in standards regarding audit
requirements and procedures.
Finally, the third most frequently mentioned reason for auditor disagreement with the
PCAOB’s assessment is that inspectors have unreasonable expectations regarding management
and auditor’s reliance on third-party specialists. Consistent with this finding, our results
regarding auditors’ use of pricing services and valuation specialists suggest that auditors
frequently rely on pricing services and valuation specialists to evaluate assumptions and/or
valuation models. However, the PCAOB auditing standards currently do not provide clear
guidance for auditors’ use of in-house specialists vs. third-party pricing services and valuation
specialists (Griffith 2013). Results also indicate that auditors encounter problems with obtaining
sufficient information from third-party pricing services regarding their proprietary model and
assumptions used to determine the FV of investment securities.
In summary, results of our study identify areas in which auditor performance, regulation,
and auditing standards need improvement. In doing so, this study highlights areas for future
research regarding auditability of complex accounting estimates as well as suggestions for how to
34
reduce the apparent gap between current auditor performance and regulator expectations.
Importantly, our results provide important insights for regulators regarding unintended
consequences of the PCAOB inspection process. Results reveal that existing auditing standards
do not provide sufficient guidance to auditors regarding how to audit complex FVMs. Instead,
changes in audit performance are driven by the inspection process. This consequence could be
significant from both the audit effectiveness and efficiency standpoints. That is, from an
effectiveness standpoint, auditors may be shifting focus away from higher-risk areas (e.g., level 2
and level 3 FVM), which has a greater risk of misstatement, to attend to lower risk areas (e.g.,
level 1 FVM). From an efficiency standpoint, auditors may be over-auditing low-risk audit areas.
35
REFERENCES
American Institute of Certified Public Accountants (AICPA). 2000. Auditing Derivative Instruments, Hedging Activities, and Investments in Securities. AU Section 332. New York, NY: AICPA.
American Institute of Certified Public Accountants (AICPA). 2003. Auditing Fair Value
Measurements and Disclosures. AU Section 328. New York, NY: AICPA. Bell, T. B., and J. B. Griffin. 2012. Commentary on auditing high-uncertainty fair value
estimates. Auditing: A Journal of Practice & Theory 31: 147 – 155. Backof, A. G., J. Thayer, and T. Carpenter. 2014. Auditing Complex Estimates: The Presentation
Format of Management’s Supporting Evidence and Auditors’ Professional Skepticism. Working paper, University of Virginia and University of Georgia.
Bratten, B., L. M. Gaynor, L. McDaniel, N. R. Montague, and G. E. Sierra. 2013. The audit of
fair values and other estimates: The effects of underlying environmental, task, and auditor-specific factors. Auditing: A Journal of Practice & Theory 31: 127 – 146.
Cannon, N., and J. C. Bedard. 2013 Auditing challenging fair value measurements: Evidence
from the field. Working Paper, Bentley University. Christensen, B. E., S. M. Glover, and D. A. Wood. 2012. Extreme estimation uncertainty in fair
value estimates: Implications for audit assurance. Auditing: A Journal of Practice & Theory 31 (1): 127 – 146.
Christensen, B. E., S. M. Glover, T. C. Omer, and M. K. Shelly. 2013. Understanding audit quality: Insights from audit partners and investors, working paper, Texas A&M University.
Christensen, B.E., S. M. Glover, and C. J. Wolfe. 2014. Do critical audit matter paragraphs in the
audit report change nonprofessional investors’ decision to invest? Working paper, Texas A&M University.
Church, B. K., and L. B. Shefchik. 2012. PCAOB inspections and large auditing firms. Auditing:
A Journal of Practice & Theory 26 (1): 43 – 63. Earley, C. E., V. B. Hoffman, and J. R. Joe. 2013. Are auditors skeptical of management’s Level
2 versus level 3 fair value classification judgments? Working paper, Providence College, University of Pittsburgh, and University of Delaware.
Griffin, J. B. 2011. The effects of uncertainty and disclosure on auditors’ fair value materiality
decisions. Working paper, University of Notre Dame. Griffith, E. E. 2013. How Do Auditors Use Valuation Specialists when Auditing Fair Values?
Working Paper, The University of Georgia.
36
Griffith, E. E., J. S. Hammersley, K. Kadous, and D. Young. 2013. Auditor Mindsets and Audits of Complex Estimates. Working Paper, The University of Georgia, Emory University, and Georgia Tech University.
Griffith, E. E., J. S. Hammersley, and K. Kadous. 2014. Audits of Complex Estimates as
Verification of Management Numbers: How Institutional Pressures Shape Practice. Forthcoming Contemporary Accounting Research.
Glover, S. M., and D. F. Prawitt. 2013. Enhancing Auditor Professional Skepticism. Monograph
produced by the Standards Working Group of the Global Public Policy Committee, www.thecaq.org/docs/research/skepticismreport.pdf.
Joe, J. R., S. D. Vandervelde, and Y. Wu. 2014. Use of Third Party Specialists’ Reports When
Auditing Fair Value Measurement: Do Auditors Stay in their Comfort Zone? Working Paper, The University of Delaware, University of South Carolina, and Case Western Reserve University.
Landis, J.R., and G. G. Koch. 1977. The measurement of observer agreement for categorical
data. Biometrics 33: 159- 174. Maksymov, E., M. W. Nelson, and W. R. Kinney, Jr. 2012. Effects of procedure frame,
procedure verifiability, and audit efficiency pressure on planning audits of fair values. Working paper, Cornell University and The University of Texas at Austin.
Martin, R. D., J. S. Rich, and T. J. Wilks. 2006. Auditing fair value measurement: A synthesis of
relevant research. Accounting Horizons 20 (3): 287 – 303. Neuendorf, K. A. 2002. The content analysis guidebook. Sage publishing. Peecher, M. E., I. Solomon, and K. T. Trotman. 2013. An accountability framework for financial
statement auditors and related research questions. Accounting, Organizations and Society (Forthcoming).
Public Company Accounting Oversight Board (PCAOB). 2007a. Auditing Accounting Estimates
and Fair Value Measurements. Standing Advisory Group Meeting. June 21. Washington, DC: PCAOB.
Public Company Accounting Oversight Board (PCAOB). 2008a. Report on 2007 Inspection of
Deloitte & Touche LLP. May 19. Washington, DC: PCAOB. Public Company Accounting Oversight Board (PCAOB). 2008b. Report on 2007 Inspection of
KPMG LLP. August 12. Washington, DC: PCAOB. Public Company Accounting Oversight Board (PCAOB). 2010a. Report on Observations of
PCAOB Inspectors Related to the Audit Risk Areas Affected by the Economic Crisis. Release No. 2010-006. September 29. Washington, DC: PCAOB.
37
Public Company Accounting Oversight Board (PCAOB). 2010b. Report on 2009 Inspection of McGladrey & Pullen, LLP. June 24. Washington, DC: PCAOB.
Public Company Accounting Oversight Board (PCAOB). 2011a. Report on 2010 Inspection of
Ernst and Young, LLP. November 30. Washington, DC: PCAOB. Public Company Accounting Oversight Board (PCAOB). 2011b. Report on 2010 Inspection of
Deloitte & Touche LLP. December 7. Washington, DC: PCAOB. Public Company Accounting Oversight Board (PCAOB). 2011c. Report on 2010 Inspection of
PricewaterhouseCoopers LLP. November 8. Washington, DC: PCAOB. Public Company Accounting Oversight Board (PCAOB). 2012a. Auditing the Future. June 7.
Washington, DC: PCAOB. Public Company Accounting Oversight Board (PCAOB). 2012b. Report on 2011 Inspection of
KPMG LLP. August 15. Washington, DC: PCAOB. Public Company Accounting Oversight Board (PCAOB). 2012c. Report on 2012 Inspection of
KPMG LLP. July 30. Washington, DC: PCAOB. Public Company Accounting Oversight Board (PCAOB). 2012d. Report on 2011 Inspection of
Ernst and Young, LLP. December 6. Washington, DC: PCAOB. Public Company Accounting Oversight Board (PCAOB). 2013. Proposed Auditing Standards:
The Auditor’s Report on an Audit of Financial Statements when the Auditor Expresses an Unqualified Opinion; The Auditor’s Responsibilities Regarding Other Information in Certain Documents Containing Audited Financial Statements and The Related Auditor’s Report; and Related Amendments to PCAOB Standards. Release 2013-005. Washington, D.C.: PCAOB.
Valuation Research Corporation (VRC). 2013. Alert: PCAOB comments increase auditor
scrutiny of fair value measurements. January. Boston, MA: VRC.
38
* For certain questions participants are able to select more than one option. Thus, for those questions, the total percentage is greater than 100%. Percentage calculations are based on participants who finished the entire survey.
TABLE 1 Participant Demographics*
Extent of involvement in the planning, supervision, or execution of auditing significant estimates, including fair value measurements. Mean (1 - 'very little involvement' to 11 - 'a great deal of involvement') 10.28 Total Number of Participants 32 Percent of Participants who are Audit Partners 100% Mean Years of audit experience 22.3 Industry
Financial Services/ Depository/ Financial Institutions/ Asset management 43.8% Industrial Products and Manufacturing 25.0% Consumer Products and Goods 15.6% Energy and Public Utilities - Regulated and Non-Regulated 12.5% Healthcare and Managed Care 9.4% Technology 9.4% Commercial 6.3% Life Sciences 6.3% Other 6.3%
Valuation and audit expertise in fair value measurement Mean (1 - 'very little expertise’ to 11 - ' great deal of expertise') 8.53 Do you specialize in a particular type of estimate or fair value measurement? Yes 18.8% No 81.2% If “yes,” areas of specialization
Investment Securities 66.7% Other Financial Instruments 50.0% Liabilities at Fair Value (e.g., asset retirement and debt obligations 16.7% Asset Impairment 50.0% Real Estate Portfolio 50.0% Business Valuation 16.7% Goodwill 50.0% Other (Please describe below) 33.3%
39
TABLE 2 Scale - Rated Responses
Panel A: Questions Regarding PCAOB Inspections
Panel B: Challenges in Auditing of FVM and Use of Valuation Experts
Scale Rating Difference from Scale's
Midpoint?*
Question
1-5 Expect what the standard
requires
6 Expect more
than the standard requires
7-11 Expect
significantly more than
the standard requires Mean t p - value
In your opinion, did the PCAOB inspectors expect the audit team to collect more evidence than what the firm believes represents the requirements of auditing standards (e.g., PCAOB AU 328) when auditing fair value measurements? 30.43% 8.70% 60.87% 7.13 1.65 0.11
Scale Rating Difference from Scale's
Midpoint?*
Question 1-5
Infrequently
6 Neither
infrequently nor frequently
7-11 Frequently Mean t p - value
1. Based on your experience, how frequently do auditors encounter situations in which management’s knowledge of valuation processes and methodologies is relatively shallow? 29.03% 12.90% 58.06% 6.74 1.94 0.06
1-5
Infrequently
6 Neither
infrequently nor frequently
7-11 Frequently
2. Based on your experience, how often is the audit team unable to obtain sufficient proprietary information from the third party pricing service to support the model and assumptions management used to determine the fair value of these investments? 50.00% 16.67% 33.33% 5.57 -0.99 0.33
40
TABLE 2 (Cont.)
Panel C: Questions on High Estimation Uncertainty and Presentation
* We performed t-tests to determine if the mean rating for each question differs from the midpoint of the scale (p = two-tailed).
Scale Rating Difference from Scale's
Midpoint?*
Question
1-5 More strongly
disagree
6 Neither agree or
disagree
7-11 More strongly
agree Mean t p - value 1. For the following two questions, please rate how strongly you
agree or disagree with each of the following statements: Over the last two decades the number of fair value measurements included in financial statements has significantly increased (e.g., estimates that were not previously on the balance sheet that now are). 3.1% 0.0% 96.9% 9.53 11.60 <0.01 Over the last two decades, the relative proportion of fair value measurements with high degrees of measurement uncertainty (i.e., lack of measurement precision resulting in reasonable range of uncertainty around the estimate that is equal to or greater than overall materiality applied by your firm) has increased. 6.3% 0.0% 93.8% 8.88 8.82 <0.01
2. Based on your experience, for significant level 2 or level 3 fair value estimates WHERE INPUTS ARE NOT OBSERVABLE, how frequently does the degree of measurement uncertainty or reasonable range around the reported point estimate exceed overall materiality? 52.9% 14.7% 32.4% 5.69 -0.80 0.43
3. Current footnote disclosures around the level of uncertainty at the measurement date are consistent among preparers. 50.0% 18.8% 31.3% 5.16 -2.03 0.05
4. Current footnote disclosures about measurement uncertainty are adequate. 37.5% 12.5% 50.0% 6.28 0.65 0.52
41
TABLE 3 Questions Regarding PCAOB Inspections – Open-ended Responses
Question1: On your engagement, did the PCAOB inspectors identify any audit deficiency related to the auditing of fair value measurements? If yes, did you agree with the PCAOB’s assessment? Why or why not?
Responses by Themes Frequency (%) 1. PCAOB did not give appropriate consideration to the risk of misstatement, nature of the security, and the totality of the audit evidence gathered. 54.5% 2. PCAOB expects a level of precision that may not be achievable given nature of FVM 27.3% 3. PCAOB’s expectation regarding management and auditor’s reliance on 3rd party specialist is not reasonable 18.2% Question 2: Given your experience with PCAOB inspections specifically in the area of fair value measurements, please provide one or two suggestions for how the PCAOB could help improve the auditing of fair value measurements. 1. Inspectors could use a more risk-based approach which is consistent with auditing standards and consider the totality of audit evidence gathered by auditors 25.9% 2. PCAOB to provide clearer guidance of audit requirements (e.g., evidence, procedures, documentation) 18.5% 3. Acknowledgement that there will be a range of FV estimates and evidence gathered may not be able to get to the level of precision desired (i.e., within materiality, inspectors need more realistic expectations) 18.5% 4. Need for more knowledgeable, qualified inspectors 11.1% 5. Acknowledgement that audit of FVMs requires judgment on the part of the auditor 11.1% 6. More clearly define and understand the role of the auditor vs. third-party specialists 7.4% 7. Consistent inspection requirements for both annually and non-annually inspected firms 3.7% 8. Bigger picture…work with accounting standard setters to reduce the number of items accounted for using FV 3.7%
42
TABLE 4 General Questions Regarding the Auditing of FVMs – Open-ended responses
Question 1: In your opinion, what are the most significant challenges in auditing inputs for a significant level 2 or 3 fair value measurement that are not observable and where small changes to the inputs have potentially material changes to the reported value?
Responses by Themes Frequency (%) 1. Lack of verifiable/corroborative evidence (including lack of 3rd party evidence) 42.9% 2. High degree of subjectivity involved in estimates 14.3% 3. Degree of difficulty in assessing reasonableness of assumptions and benchmarks 14.3% 4. Developing a range and placing point estimate within the range 9.5% 5. Inability to audit to the level of precision desired 7.1% 6. Lack of disclosure about imprecision 4.8% 7. Removing management bias 4.8% 8.Ensuring the competency and expertise of the valuation specialists 2.4% Previous Question: Do you believe the audit challenges are different between financial vs. nonfinancial assets/liabilities? Question 2: If “yes,” please briefly describe the differing challenges.
Responses by Themes Frequency (%) 1. Difference in availability and relevance of market data 56.5% 2. Degree of auditor familiarity 13.0% 3. Difference in perceptions of precision by regulators and users 8.7% 4. Degree of consistency possible when applying processes, methods, models, and/or assumptions) 8.7% 5. Difference in dollar amounts (i.e., size) 4.3% 6. Degree of subjectivity 4.3% 7. Complexity of models 4.3% Question 3: Based on your experience, what is your audit response when management’s knowledge of valuation processes and methodologies is relatively shallow, and how does this impact your audit procedures when testing the issuer’s fair value measurements?
Responses by Themes Frequency (%) 1. Auditor expectation that management will place greater reliance on third-party specialists 20.7% 2. Consider impact on internal control effectiveness and inherent risk 19.0% 3. Auditors place greater reliance on valuation specialists 17.2% 4. Perform additional audit procedures 17.2% 5. Reconsider auditor’s risk assessment 10.3% 6. Auditor to develop independent estimate 8.6% 7. Increase skepticism and scrutiny 5.2% 8. Communication with board or audit committee 1.7%
43
TABLE 5 Questions Regarding Auditors’ Use of Valuation Experts – Open-ended responses
Question 1: When estimates are obtained from a third-party pricing service in an audit area considered to be “higher” risk, what does the audit team usually do to assess the reasonableness of specific assumptions, methods, and models used by the pricing service to value the specific financial instruments in order to establish a basis for reliance on the information provided by the pricing service?
Responses by Themes Frequency (%) 1. Involve firm’s in-house specialists 26.7% 2. Gain understanding of the pricing methodology (e.g., method, model, and assumptions) 24.4% 3. Comparison with auditor’s independent estimate, another pricing service, or similar instrument 22.2% 4. Assess qualifications and reliability of information supplied by pricing service 20.0% 5. Assess the pricing service’s policies and procedures 6.7% Question 2: If the valuation specialist’s work used to support the audit involves use of significant assumptions and methods, how does the audit team test the data and assess the reasonableness of those assumptions and methods?
Responses by Themes Frequency (%) 1. Obtain support/audit evidence for data used in model (internal and external sources) 30.2% 2. Reliance on audit firm’s valuation specialist to evaluate appropriateness of assumptions 24.5% 3. Auditor gains understanding of key assumptions (via the firm’s valuation specialists) 15.1% 4. Comparison to company’s historical information 13.2% 5. Test completeness and accuracy of data used in management’s valuation 9.4% 6. Follow firm and/or PCAOB recommended practices 5.7% 7. Search for contrary evidence 1.9%
44
TABLE 6 Questions on High Estimation Uncertainty and Presentation
Question: For estimates with a reasonable range of estimation uncertainty that is greater than overall materiality, which of the following methods do you believe would be most useful to users in conveying the level of uncertainty (please ignore the potential costs associated with the method). (check all that apply)
Responses by Themes Frequency (%) 1. Narrative description of estimation methods and the source and level of uncertainty 37.1% 2. Quantitative disclosure of the range of reasonable estimates at the measurement date 30.0% 3. Tables in footnotes listing measurement assumptions and the values used 18.6% 4. Statistical distributions providing the upper and lower bounds at say 90 or 95%
confidence 5.7% 5. Rounding reported estimates to a higher level than other reported values in the financial
statements to remove the illusion of exactitude 2.9% 6. Separate columns on the face of the financial statements for each year that add up to the
current columnar totals and presentations. The separate columns would include a column that presents reported vales with relatively low estimation uncertainty (a reasonable within materiality) and a column for reported values with a reasonable range of estimation uncertainty greater than materiality 1.4%
7. Other 4.3%
45
TABLE 7 Questions on High Estimation Uncertainty and Presentation – Open-ended Responses
Previous question: In your opinion, should the current requirement to provide a high level of positive assurance that a point estimate is fairly stated be reconsidered when the estimate’s reasonable range of estimation uncertainty is many times larger than overall materiality (e.g., consider instead providing positive assurance on the estimation process)? Question1: Please explain why or why not.
Responses by Themes (‘Yes’ response to the previous question) Frequency (%) 1. Unrealistic to provide assurance on point estimate given level of subjectivity, complexity and
uncertainty of estimates 28.6%
2. No amount of evidence could achieve high level of assurance given high estimation uncertainty 23.8%
3. Current presentation may mislead readers of the financial statements – investors may not understand the level of uncertainty in estimates 14.3%
4. However, not sure if negative assurance on the point estimate and positive assurance on the process is the way to go 14.3%
5. Disclose range and confidence level so investors can evaluate 14.3% 6. Auditors’ need to rely on specialist, thus makes more sense to focus on the estimation process 4.8%
Responses by Themes (‘No’ response to the previous question) Frequency (%) 1. Problem inherent in the audit process, thus no change is warranted. 31.6% 2. Auditing and financial reporting has always involved estimation and uncertainty 21.1% 3. Responsibility of auditor to provide opinion on the whole financial statement not just fair value measurements 15.8% 4. More or less auditing will not solve problem. More disclosure is necessary 15.8% 5. Excessive disclosures may cause confusion and information overload for users 10.5% 6. Suggestion inconsistent with other audit standard 5.3% Previous Question: Regarding the existing guidance on how to audit estimates with high estimation uncertainty, would your firm support additional clarity and guidance from standard setters and/or regulators regarding auditing estimates or do you believe the current guidance is sufficient? Question 2: Are there specific recommendations you have on what could be added to the literature?
Responses by Themes Frequency (%) 1. More guidance regarding an acceptable reasonable range of estimation uncertainty 29.4% 2. More clarification in standards regarding audit requirements and procedures 29.4% 3. The standard-setting process should influence audit procedures and requirements rather than using PCAOB inspection process to drive behavior 17.6% 4. Realize that in some cases PCAOB guidance and expectations assume greater accuracy and precision than is achievable in reality 11.8% 5. Provide more concrete examples 5.9% 6. More guidance regarding reliance on specialists for hard-to-value FVM 5.9%
46
TABLE 7 (Cont.) Questions on High Estimation Uncertainty and Presentation – Open-ended Responses
Previous question: For estimates with high estimation uncertainty, in your opinion, would a potentially viable approach in the future, assuming standards change accordingly, be to require the auditor to provide positive assurance on the rigor and quality of the estimation process and negative assurance on the reported point estimate (i.e., the entity followed an appropriate and rigorous process to form the estimate and nothing came to our attention that would suggest the reported point estimate is not fairly stated)? Question 3: Please explain why or why not?
Responses by Themes (‘Yes’ response to the previous question) Frequency (%) 1. Suggestion is consistent with other private equity fund opinions, represents current environment, and is reasonable 40.0%
2. Unrealistic to provide assurance on point estimate given level of subjectivity, complexity and uncertainty of estimates 20.0%
3. Different assurance levels may cause confusion 10.0% 4. Additional clarification regarding range of tolerable differences 10.0% 5. Suggestion offsets the extremely high expectations of regulators regarding level of assurance
possible for estimates with high estimation uncertainty 10.0%
6. Less judgment involved with auditing the process than outcome 10.0%
Responses by Themes (‘No’ response to the previous question) Frequency (%) 1. Suggestion is inconsistent with other standards as the auditor is responsible to provide positive assurance the financial statements taken as a whole, so can’t exclude fair value measurements 27.8% 2. Too difficult to separate process from outcome, both are needed 16.7% 3. Additional change can’t address the inherent imprecision in FVMs so suggestion may still not reduce risk of material misstatement or identify inappropriate estimates 16.7% 4. Suggestion is similar to the ICFR audit 16.7% 5. Results in potential differences due to differences in auditors’ judgments 11.1% 6. Believe current audit process and reporting is sufficient 5.6% 7. Create confusion for users and the disclosures necessary would be difficult to manage 5.6%