the informational value of emphasis of matter … informational value of emphasis of matter...

The Informational Value of Emphasis of Matter Paragraphs and Auditor

Commentaries: Evidence from an Eye-tracking Study

Louis-Philippe Sirois, HEC Montréal,

Palash Bera, Saint Louis University,

Jean Bédard, Université Laval

Anand Jha, Texas A&M International University

August 30, 2013

This project has been funded by the International Association for Accounting Education Research (IAAER) under the Informing the IAASB Standard Setting Process IAAER – KPMG Research Opportunities.

The Informational Value of Emphasis of Matter Paragraphs and Auditor

Commentaries: Evidence from an Eye-tracking Study

ABSTRACT

In order to respond to the demand for more information on matters important to users’

understanding of audited financial statements and the audit, the International Auditing

and Assurance Standards and the U.S. Public Company Accounting Oversight Board

have both proposed a new auditing standard requiring the communication of additional

information in the auditor report. In an experiment, using eye-tracking technology we

examine whether and how alternative audit report formats affect how users navigate

through and integrate the information presented in the related audited financial

statements. The main results show that the matters mentioned in the audit report affect

the participants’ information search, increase their attention to financial statements

disclosures mentioned in the audit report but reduce their level of attention to the rest of

the financial statements.

1

The Informational Value of Emphasis of Matter Paragraphs and Auditor Commentaries:

Evidence from an Eye-tracking Study

Introduction

This paper reports the results of an experiment that examine the effects of communicating

additional information in the audit report to financial statement users as proposed by the

International Auditing and Assurance Standards (IAASB) and the U.S. Public Company

Accounting Oversight Board (PCAOB) in their proposed auditing standard (IAASB 2013;

PCAOB 2013) issued during the Summer 2013.

These proposals are a response to users demands for auditors to be more transparent and to

provide entity and engagement specific information in their reports(e.g., CFA Institute 2011;

Footprint Consultants 2011). In recent years, regulators have examined various alternatives for

providing additional information in the audit report, including greater use of “Emphasis of

Matter” paragraphs (i.e., matters of significant importance related to the entity’s financial

statements) and “Other Matter” paragraphs (i.e., matters related to the audit that are relevant to

user’s understanding of the audit). Following a consultation process, the IAASB and the PCAOB

propose requiring the communication of “key/critical audit matters” in the audit report. These

additional communications are expected to enhance the communicative value of the auditor’s

report by among other, helping “investors and other financial statement users focus on aspects of

the company's financial statements that the auditor also found to be challenging” (PCAOB 2013,

p. 6) and providing a “(…) roadmap to help users better navigate complex financial reports and

focus them on matters likely to be important to their decision-making” (IAASB 2011, par. 36).

Yet, it is unclear whether these changes will mitigate the persistent “expectations” and

“information” gaps standard setters wish to address. Indeed, research suggests that despite modest

improvements to the content and format of the audit report over the years, it is still largely

perceived as providing little informative value to users (e.g., Church et al. 2008; IAASB 2011).

Experimental studies on the impact of additional disclosure in the auditor’s report pertaining to

the entity’s financial statements are inconclusive and most find that there is no significant impact

on users (Mock et al. 2013). Also, while financial statement users may ask for greater entity and

engagement specific disclosures in the auditor report, the benefits of such disclosure remain

uncertain and any such (unproven) benefit would have to be weighed against non-trivial

2

challenges and issues associated with the increased disclosure. Among such issues is the risk of

including too many other matters in the auditor report, which may diminish the overall

effectiveness of the auditor’s communication on such matters. Users may also inappropriately rely

on auditor commentary as a “substitute for reading the financial statements” (IAASB 2012 par.

63(d)). Ultimately, the auditor’s commentary may unintentionally bias users’ reading of an

entity’s financial statements, leading to discard otherwise useful information not referred to in the

auditor’s commentary while placing exaggerated emphasis on the information explicitly

commented on. In turn, the overreaching objective of standard setters to “improve users’ ability to

make informed decisions on the basis of the financial statements and the audit” (emphasis added,

IAASB 2012 par. 8) may be compromised.

This study provide evidence on the “costs and benefits” of auditor communication of

matters related to information included in the financial statement. Specifically, we examine

whether users read and integrate both actual information and audit matters proposed by regulators

in the auditor report, and whether the attention given to specific financial statements information

is affected by the associated audit matter section of the audit report. We evaluate this additional

disclosure under the existing mechanism provided by the current standards (i.e., Emphasis of

Matter paragraph) as well as the proposed, wider concept, of Auditor Commentary as outlined by

the IAASB (2012).

Based on cognitive psychology we develop expectations as to how highlighting

information in the auditor reports affects investors’ information acquisition. We test these

propositions in a laboratory experiment employing unobstructive eye-tracking technology. This

technology allows us to objectively and precisely measure users’ attention to specific information

contained in the auditor report and the related financial statements. This further allows us to

understand the actual process by which this information is acquired and how it is integrated in the

users’ decision process.

Ninety-eight post-graduate accounting students participated in an experiment where they

played the role of junior financial analysts and analyzed the financial statements of a

manufacturing company that presented one of four versions of the auditor report; current audit

report, audit report with one or three additional matters presented in the form of an emphasis of

matter paragraph, and three additional matters presented in the form of an auditor commentary.

This between-subjects design allows to evaluate whether users behavior differ between (1) the

3

current audit report and an audit report with additional matters, (2) the number of matters

presented in the audit report, and (3) the format of the presentation (emphasis of matter paragraph

or commentary).

Our results show that participants read (i.e., acquire) the supplementary information

presented in the auditor reports and that the matters mentioned in the audit report affect the

participants information acquisition. Thus, participants accessed the related financial statements

disclosure in fewer steps when these matters are communicated in the auditor report. In addition,

participants paid higher attention to disclosure when the matters are communicated in the auditor

report. However, increased attention to financial statements disclosures mentioned in the audit

report, reduce the level of attention to the rest of the financial statements. This result suggest that

the communication of additional information in the audit report could lead to less attention to

otherwise useful information not referred to in the report. The number of matters mentioned in the

auditor report (one versus 3) also has an effect on the information acquisition. Thus, users access

more rapidly to an item mentioned in the audit report and give a higher level of attention to it,

when there is only one matter in the audit report than when there are three mattes.

This study suggests that the communication of audit matters in the auditor report as

proposed by the IAASB and the PCAOB provides a roadmap that help users navigate complex

financial statements and focus on matters highlighted by the auditor. The findings also indicate

that such highlighting may bias the users’ information acquisition of other financial statement

information and that more matters in the audit report reduce the attention devoted to matters

highlighted by the auditor. Accordingly, auditors will have to be careful in the selection of the

matter and the number of matters to communicate in the audit report. This study extend previous

audit report research by examining the effect of the auditor report on the information search

behavior of users and introducing eye-tracking technology to auditing and financial statement

analysis research.

The remainder of this paper is organized as follows. The next section develops our

research propositions and questions from existing theories and previous studies. The third section

presents the research design used to test these propositions and questions, the measurement of

variables, and the sample selection procedures. Our experimental results are presented in the

fourth section and a conclusion follows.

4

Theory and propositions

Attention directing role of the audit report

We expect that compared to users using audit reports without supplementary information, users

using audit reports with emphasis paragraphs will have different navigation pattern of the

financial statements. This expectation is based on the idea that the supplementary information in

the auditor’s report will guide viewers to look at specific parts of the financial statements

The proposition is based on the theory of cognitive overload (Mayer, 2001) which

originates from multimedia learning where a common problem is that individuals are faced with a

learning task that demands more from their cognitive resources (such as working memory) than

they can sustain - a situation known as cognitive overload (Mayer and Moreno 2003). This theory

is relevant here as multimedia (such as text and picture) learning is similar to learning financial

statements (table and text). Viewers face cognitive overload in reading financial statements as

they are relatively long and interconnected. Without any guidance, viewers face problems in

navigating the statements and are unsure on where to start reading the statements. Moreover, the

volume of information presented in financial statements is overwhelming and without any

guidance viewers might skip relevant parts and focus on the irrelevant parts of the statements.

Mayer and Moreno (2003) suggests a method termed signalling effect to overcome this cognitive

load. It is a type of cognitive bias that modifies the display of data and it biases the way

information is perceived and processed (Arnott 2006). Cognitive overload can be reduced by

providing cues on how to use the information presented in the problem. Mayer (2001) mentions

that people learn better when cues are added that highlight the organization of the essential

material. This bias can be introduced in several ways such as highlighting or emphasizing of texts

or tables or by formatting the learning material (Mayer and Moreno 2003). Such highlighting can

help viewers in information acquisition as according to Elliott et al. (2007), information

acquisition occurs when financial statement users find and read specific pieces of financial

information sufficiently well to recall the exact locations of such information.

We suggest that introduction of auditor’s commentary/emphasis paragraph is a type of

signal as it highlights certain parts of the financial statements. As signaling helps in the process of

selecting and organizing relevant information (Mayer and Moreno, 2003), therefore we suggest

5

that emphasis paragraph will help viewers in guiding the viewers in reading the financial

statements.

Proposition 1- Supplementary information: Users will access the parts of the financial

statements that are referred to in the auditor’s report more rapidly and will

pay higher attention to these parts.

Number of matters mentioned in the supplementary information

As indicated by the IAASB and the PCAOB, the auditor might identify either one or several

key/critical matters. While the PCAOB is silent as to the impact of the number of matters that

could be communicated in the audit report, the IAASB (2012 par. A9) specifies that “[I]n general,

the greater the number of key audit matters, the less useful the auditor’s communication of key

audit matters may be.” Indeed, providing too many cues my not help users might create confusion

and not reduce the cognitive overload as much as when there are few items.

Proposition 2- Number of matters: The effect of the supplementary information in the

auditor report on the user information acquisition behavior will be lower

when there are a higher number of matters communicated in the audit

report.

Research method

This study uses eye-tracking techniques to analyze how financial statement users read audit

reports in relation to the audited financial statement. Eye tracking offers a window into how

individuals read and scan information that is displayed to them. Recording of eye movements can

provide a direct trace on where a person’s attention is being directed for a particular visual area

(Jacob and Karn, 2003). Involuntary and voluntary eye movement responses reflect the internal

processing of information by a viewer (Rayner 1998).

Jacob (1995) explains the physiology behind the eye movements. An area termed fovea is

located near the center of the retina and is densely covered with receptors. Fovea provides high

acuity (near) vision than the peripheral vision. Peripheral vision is generally inadequate to see an

object clearly and generally requires that the object to be viewed foveally. When an object is

viewed foveally by a reader, he/she makes attempt to understand the content. Therefore, for eye

tracking purposes, in order to see an object clearly, one must move the eyeball to make that object

6

appear directly on the fovea. At that moment, a person’s eye position can be directly measured by

an eye tracker.

Research in tracking eye movements is common in psychology and physiology. Much of

the early work on these areas focused on how human eyes operate and what it can reveal about

perceptual processes (Jacob and Karn 2003). Recently researchers have begun to focus on the

relation between eye movements and cognitive processes. Eye tracking has been used in

improving graphics design and website design (e.g., Chu et al. 2009; Canham and Hegarty 2010),

understanding consumers perceptions and attentions of advertisements (Lam et al. 2007; Lohse

1997; Lindberg and Näsänen 2003). Eye tracking has also been used in domains such as

healthcare and transportation. For example, to investigate the attention to health warnings on

cigarette packs (Munafò et al. 2011; Kessels and Ruiter 2012) and to understand the relation

between attention and police pursuits (Crundall et al. 2005).

Although eye-tracking technology has been used for over 30 years, the technology was

unreliable and data interpretation was time consuming (Collewijin, 1999). Over the years, the

technology has become more reliable, user friendly, and affordable (Jacob and Karn, 2003) and

thus suitable for analyzing mental processes of users. This technology now allows to objectively

and precisely measure users’ attention to specific information. This further allows us to

understand the actual process by which this information is acquired and how it is used to

understand a visual content.

Three common eye movement metrics are: eye fixations, eye saccades, and eye dilations

(Sharif and Maletic, 2010). Eye movements are made up of short bursts of stationary visual

display termed fixations and are filled up with rapid and continuous movements termed saccades

(Jacob, 1995). During fixations, eyes remain almost motionless, whereas saccades are movements

from one fixation to another. A typical fixation lasts for 200-300 milliseconds approximately and

is generally understood to indicate where viewer’s attention is directed (Rayner 1998). When eyes

fixate on a certain area, the brain starts to process the visual information received from the eyes

(Rayner, 1998). Users cannot interpret what they have seen until they pay attention and fixate

their eyes on it (Wedel and Pieters, 2007).

Fixation movements can reveal the amount of information that is processed. The amount

of time spent fixated on certain visual areas can be correlated with the degree of cognitive

7

processing (Rayner 1998). Research has demonstrated that information complexity, task

complexity, and familiarity of visual display will influence fixation duration (Duchowski, 2002).

Wooding (2002) introduced the concept of fixation maps which depict a contour analysis of the

intensity of eye gaze and higher peaks indicate regions of more fixations. Eye fixation is the most

relevant metrics for evaluating information processing as other metrics such as saccades occur too

quickly to absorb new information (Rayner 1998).

Pupil dilation is a measure that is generally used to indicate an individual’s arousal or

interest in the viewed content, with a larger diameter reflecting greater arousal (Duchowski, 2002;

Rayner, 1998). It has been found that mean pupil dilation is a useful measure of cognitive load on

research related to education and learning (Van Gerven et al. 2002). Thus if a subject has

increased eye pupil size by looking at a particular visual area, then it can be inferred that the

subject is paying high attention to that area.

In general, two types of analysis are done with eye movement data. First, on the overall

level for viewing the entire area (auditor’s report in our case) and second at a specific level,

researchers define “areas of interest” (AOI) over certain parts of a display and analyze the eye

movements that fall within such areas. In this way, the visibility, meaningfulness, and placement

of specific elements can be objectively evaluated (Goldberg and Kotval, 1998). Table 1

summarizes the definition and interpretation of the eye metrics. In order to more fully assess how

readers integrate the information and consult the financial statements, we complement our

analysis by also using standard web navigation metrics. They are described in Table 1 as well.

Insert table 1 about here

Experimental Design

To test these propositions, we conducted an experiment with 1 x 4 between-subjects design with

four versions of the auditor report. The groups were as following- control group (exposed to an

audit report without supplementary information), 1 matter emphasis group (exposed to an audit

report with one matter presented in the form of an emphasis of matter paragraph), 3 matters

emphasis group (exposed to an audit report with three matters matters presented in the form of an

emphasis of matter paragraphs), and 3 matters commentary group (exposed to an audit report with

3 commentaries). Ninety-eight subjects were randomly distributed into these four groups. Each

8

group received the same set of financial statements, except that the auditor’s reports were

different in each case as mentioned above (See appendix A for a copy of the four audit reports).

Insert figure 1 about here

As indicated in figure 1, one financial statement note (note 5) was referred to in the three

groups with supplementary information (B, C, D) while two other notes were referred to in two

groups (note 1k and note 14). (See appendix B for the content of the three notes) This between-

subjects design allows to evaluate whether users behavior differ between (1) the current audit

report and an audit report with additional matters, (2) the number of matters presented in the audit

report, and (3) the format of the presentation (emphasis of matter paragraph or commentary).

Procedures and materials

The Tobii X-60 Eye Tracker reader was used for this experiment. This reader has the main

advantage of being unobstructive; subjects are simply required to read the material on screen in a

natural setting without the need of special eyewear. The experiment was done in two stages with

the entire questionnaire, including instructions, context and the complete set of financial

statements available in an HTML format. Subjects were asked to assume the role of a loan analyst

working for a local bank. For their task, the subjects were asked to read on a computer screen the

latest audited financial statements of a potential client seeking to refinance an expiring loan for

the amount of 2 million dollars in order to perform a simple credit evaluation and conclude on the

potential client’s loan application (i.e., credit score, decision, amount and rate), similar to Miller

and Smith (2002) and Viger et al. (2008).

At the beginning of the experiment, subjects were briefed about the study and were told

that their eye movements will be recorded. The subjects would then sit at a desk in an individual

laboratory room equipped with a one-way mirror. Overall, very little guidance was provided as

subjects were told all the instructions and relevant material would be presented to them in web

format. Then their eyes were calibrated using the software before starting the recording of their

eye movements and navigation activity. In the first stage, subjects would advance sequentially,

without possibility to return, to an instruction page followed to the task description and context

page before beginning the credit evaluation task by accessing the complete set of audited financial

statements, including the auditor report. Once subjects clicked to proceed to the financial

statements they were not be able to return to the description page.

9

A major challenge of the study was having subjects read the auditor’s report without

explicitly asking them to do so in order to limit any bias. Indeed, the subjects were never told that

the experiment dealt with an auditing issue1. Hence, after leaving the task description page, a page

with a navigation menu consisting of hyperlinks to the sections of the financial statements2

appeared on the left-side, along with only a large, bold text message mentioning “Click on the

Auditor’s Report” appearing in the main section. None of the hyperlinks in the navigation menu

would be active, except the first one called “Auditor’s report”. Once they clicked on the link the

auditor’s report would appear. At this point, the links to the other sections of the financial

statements still remained inactive until subjects scrolled down at the bottom of the page and click

the “Continue” button.

Once they clicked on the “Continue” button, the subjects remain on the auditor’s report

page although the other links would then become active and change color. From then on, the

subjects could freely navigate through the financial statements by clicking on the links on the left-

side menu with would remain always available, much like any other web-enabled financial

statement available on the web3. They could also, at any time, click on a button at the bottom of

each page to advance to the “Evaluation” page where they would perform the first part of their

task4. The subjects could always return to the financial statements by clicking a “Return to F/S”

button or, alternatively, click a “Finalise evaluation” button to move on to the second part of the

task5. If the latter button was clicked, a popup message was shown asking the subject to confirm

that they have read the financial statements in order to continue. If the “Return to F/S” button was

clicked, the subjects would return to the auditor’s report and could then navigate again within the

financial statements. Once the two parts of the task were completed, the subjects moved on to

post-experiment questionnaire pages.

1 Post experimental discussions with the subjects confirm that the subjects were not biased in this regard. When asked on what they believe the experiment was, none of the subjects mentioned “auditing”. 2 One link per section of the statements, starting with: 1) Auditor’s report, 2) Balance sheet, 3) Cash flow statement, 4) Note 1 to Note 15. The titles of the notes were not provided in the links so that subjects were forced to read the content of the notes to identify their content. Each section of the statements had a unique URL address. 3 See for example: http://www400.abbext.com/2011/ar/financialreview/consolidatedfinancialstatements/reportofthestatutoryauditorontheconsolidatedfinancialstatements.html?cat=m 4 This part of the task consist of assigning a credit score on a 9 point scale, decide on the loan application (accept or not), along with a brief text explanation the subjects are asked to provide in a textbox. This explanation is merely intended to add realism to the task and stimulate subjects in their reflection and analysis. 5 For this part, subjects were asked to decide on the rate and amount of the loan if it had been approved by their supervisor. They could not return to the first part of the task, however they could return to the full set of financial statements as before if needed.

10

Prior to conducting this study, we conducted a pilot test to validate the experiment material

with 15 subjects. The experiment with the final material was also conducted with a few

experienced loan analysts who confirmed the realism of the task and the material. Financial

statements were based on a medium sized, manufacturing Canadian public company listed on the

Venture TSX. We modified the financial statements to limit note size and the number of notes to

15, and insure coherence between the statements and the audit report information. Moreover, to

further simplify the financial statements, they were adapted following Canadian Accounting

Standards for Private Enterprise. 6

The experiments were conducted over a two week period at a large Canadian university.

On average, complete experiment time was 35 minutes, excluding setup time and calibration, with

an average (median) of 27 minutes (26 minutes) spent on performing the two parts of the tasks

and reading the financial statements.

Subjects

A total of 98 students enrolled in post bachelor accounting degree at a large -Canadian university

were recruited as subjects; 91 were enrolled in the school’s graduate diploma in public

accounting, a one year intensive study program compulsory to write the Uniform Final

Examination (UFE) required to become a Charted Professional Accountant in Canada. The strict

admission requirements for this program insure that all students have a thorough training in

accounting and finance. Moreover, close to a third of the first year students and the majority of the

second year students will have done a minimum 3-month internship in an accounting firm. The

program curriculum is uniform for all students, further ensuring uniformity across the subjects.

The other 7 subjects were all registered in a Master of accounting program with similar

requirements, of which 5 have already completed the Graduate diploma and passed the UFE or

other professional accounting designation exams. Overall, the participants had sufficient

understanding of financial statements and audit reports to serve as appropriate proxies for

nonprofessional investors (Maines and McDaniel 2000). We assigned the participants randomly to

6 During the experiment subjects did not have access to a calculator, pen or paper. Having done so would have distracted the subjects away from the screen and it would have therefore been impossible to record their eye movements. However, such tools were not necessary as subjects did not have to explicitly perform calculations. Moreover, the subjects’ true mental processes can be more easily inferred from the eye data as they are forced to carefully look at the statements as they perform mental calculations in conducting their task.

11

the four groups; no participants were excluded from the analysis. Table 2 presents the distribution

of participants based on sex and their year of enrolment in the Graduate diploma.


Data analyses and results

Attention to audit report

The first part of our analysis provides direct evidence of how individuals actually read the audit

report in general and whether the additional information provided in the auditor

commentaries/emphases is read or not. For the proposed auditor report to have any informational

value, it is necessary that the additional disclosures in the report be read. Table 3 presents

evidence that subjects read all parts of the auditor report, with greater attention given to the

additional disclosures.


Table 3 presents total fixation duration and count for the first time the report is shown to

subjects (i.e., after they click on the “Auditor’s report” link before the other links are activated).

Because different sections of the report have different lengths, these measures are also scaled by

the number of words per section. Of the common section of the report, the introduction is read

more, arguably as it comes first at the top of the page. While the opinion section is often time

listed as the most informative section to readers, it is interesting to note a slight decrease in the

time spent reading and fixation count per word. Overall, no clear difference across groups

emerges.

Groups B, C and D were presented with revised versions of the auditor report. All subjects

in these groups spent some time reading these additional disclosures. Moreover, there is clear

evidence, as illustrated by the fixation duration and count scaled by the number of words, that the

subjects focused proportionally greater attention to the additional disclosures. The average total

fixation duration (count), scaled by the number of words in the commentaries/emphases part,

ranges from 127 (0.54) to 228 (0.97) for groups B to D, while the same metrics range from 101

(0.41) to 122 (0.55) for the common section of the report for those groups. Overall, there is

evidence that the additional auditor disclosures are read carefully. We analyses in the next section

12

whether this additional information is integrated by the subjects and how it affects their search

strategies within the financial statements.

Data analyses and results

Proposition 1 Attention to the notes referred to in the audit report

To test proposition 1, which relate to the effect of having supplementary information in the audit

report, we compare the results for group A (control group) with group B, C, and D for note 5 and

groups A and B with groups C and D for note 1k and note 14 (see figure 1). The results are

presented in Table 4. To test the effect of the supplementary information in the audit report on

users access the parts of the financial statements that are referred to in the auditor’s report, we use

three metrics: number of other pages visited before, time to first fixation (ms.), and proportion of

subject with 0 fixations. We test the expectations regarding attention to these financial statements

parts, we use three metrics: relative total fixation time, relative total number of fixations, and

relative pupil size (first fixation).


For note 5, as reported in Panel A of Table 5, when note 5 is referred to in the audit report,

the proportion of participants who did not look at the note (proportion of subject with 0 fixations)

is significantly lower (11.1 percent versus 26.9 percent, p. = .0588), the number of other pages

visited before accessing note 5 is significantly lower (10.80 pages versus 17.68 pages, p. = .0005),

and the time to first fixation is also lower (323.4 seconds versus 472.9 seconds, p. = .0048).

Overall these results suggest that when note 5 is referred to in the audit report, not only a greater

proportion of participants looked at note 5, but they also went more directly and faster to note 5

after reading the audit report.

Regarding participants’ attention to note 5, we find that the relative total fixation duration

on note 5 is significantly higher when the note is referred to in the audit report (4.29 percent of the

total duration versus 3.06 percent, p= .0191) and the relative total number of fixation is also

significantly higher (4.26 percent of the total number of fixations versus 3.03 percent, p. = .0156)

suggesting that participants paid more attention and engaged more in note 5, when note 5 is

referred to in the audit report. The pupil dilation is higher than 1 but similar in both groups (1.029

versus 1.023, p= .3416) suggesting that all participants paid more effort and attention to note 5.

13

Results for note 1k, are reported in Panel B of Table 5. As for note note 5, when the audit

report refers to note 1k, participants went more directly and faster to note 1k after reading the

audit report. Thus the number of other pages visited before accessing note 1k is significantly

lower (8.14 pages versus 10.43 pages, p. = .0375) and the time to first fixation is also lower

(28.90 seconds versus 39.71 seconds, p. = .0045). While lower, the proportion of participants who

did not look at the note is not statistically different (proportion of subject with 0 fixations = 12.0

percent versus 16.7 percent, p. = .3554). The three measures of participants’ attention to note 1k

are significantly higher when the audit report refers to the note (20.46 percent of the total fixation

duration in note 1 versus 12.66 percent, p= .0168; 20.01 percent of the total number of fixations in

note 1 versus 12.04 percent, p. = .0148; and a relative pupil dilation of 1.008 versus 0.996, p =

.0536). Overall, these results suggest that participants paid more effort, attention and engaged

more in note 1k, when the audit report refer to the note.

Finally, Panel C presents the results for note 14. Except for time to first fixation which is

lower for the groups where the audit report referred to note 14 (658 seconds versus 789 seconds,

p. = .0268), all the other metrics are not statistically different between the groups. Thus, while the

participants went faster to note 14 when the audit report referred to it, the attention accorded to

this note is similar in both groups. The nature of note 14 could have caused this result. Thus, note

14 is refers to a $ 10 million contingency and it thus a material matter. Indeed, the relative pupil

size is 1.02 for both situations, suggesting a greater than average effort and attention paid to this

note. However, for note 5 the relative pupil size is also large, but is greater when the audit report

refers to the note. Another explanation could be the serial position of the note in the financial

statements. Note 14 being the penultimate item of the financial statements which may have

increased participants attention to this note in all groups.

Proposition 2 Number of matters in supplementary information in the audit report

Table 6 reports the results for the effect of the number of matters communicated in the audit

report on users’ information acquisition behavior. We test this effect by comparing the results for

group B, where the report includes one matter (note 5) and group C and D where the report

includes three matters (note 5 and two other notes) (see figure 1 for the design).


14

Regarding the access to note 5, while the three metrics are in the expected direction, only

the number of other pages visited before accessing note 5 is significantly higher when there are

three matters instead of one in the audit report (11.86 versus 8.62, p = .0600). The other two

metrics are proportion of subject with 0 fixations and time to first fixation.7 Overall, these results

suggest that the number of matter mentioned in the audit report does not appears to significantly

affects the access to parts of the financial statements referred to in the audit report.

Contrary to access, participants’ attention to note 5 is significantly affected by the number

of matter. Thus the relative total fixation duration on note 5 is significantly lower when there is

three matters in the audit report (3.90 percent of the total duration versus 5.17 percent, p= .0321),

the relative total number of fixation is significantly lower (3.85 percent of the total number of

fixations versus 5.20 percent, p. = .0208), and the pupil dilation is also significantly lower (1.020

versus 1.045, p= .0322). Results from these three metrics suggest that participants paid less

attention, effort and engagement in note 5, when three matters are communicated in the audit

report than when the report includes only one. Thus, while the supplementary information still

play its attention directing role, when there is several matters, having more than one matter

reduces the attention devoted to individual parts of the financial statements referred to in the audit

report.

Effect of supplementary information on other disclosures

In this section, we examine the effect of supplementary information in the audit report on the

acquisition of the other information in the financial statements. While highlighting critical matters

may increase users’ attention to these matters, it may reduce the attention to other information that

is not specifically referred to in the audit report. Table 6 presents the effect of communicating

supplementary matters and Panel B the effect of reporting three matters instead of one. In each

panel we reports the number of URL visited as well as total fixation time and number for the other

notes in the financial statements.


A comparison of the control group (A) with the three groups with supplementary

information (B,C,D) indicates that communicating matters in the audit report significantly reduces

7 It should be noted that given that the matter referring to note 5 is the second, if participants follow a sequential

acquisition, the number of pages should be greater than when the note is mentioned first.

15

the attention paid to other information in the financial statements. Thus, as shown in Panel A of

Table 6, the three measures of attention are significantly lower for the other financial statements

notes. For the other notes, communicating matters in the audit report significantly reduce the total

fixation time from 304 seconds to 286 seconds, the total fixations from 1355 to 1266, and the

number of URL visited from 21.9 to 18.9. As reported in Panel B, the pattern is similar for the

number of matters. All the measures are significantly smaller when the audit report communicates

three matters instead of one, suggesting that a greater number matters reduces the attention paid to

other information in the financial statements. We also tested whether having one matter instead of

none influenced participants’ attention. The tests of differences for the three measures (not

tabulated) are not statistically significant, indicating that the reduction in attention exists only

when the audit report communicates several matters.

These results suggest that highlighting several critical matters reduce the attention to other

disclosures that are not specifically referred to in the audit report. If the information for which the

attention is reduced is less important, this would suggest that the audit report increased users’

information acquisition efficiency. However, it could also lead to less attention to otherwise

useful information not referred to in the report

Conclusion

The results of our experiment show that communicating additional matter in the audit report

significantly affects users’ financial statements information acquisition. Thus, participants

accessed the related financial statements disclosure in fewer steps when these matters were

communicated in the auditor report. In addition, participants paid higher attention to disclosure

when the matters are communicated in the auditor report. Our results suggests that the

communication of key/critical audit matters in the auditor report as proposed by the IAASB and

the PCAOB provides a roadmap that help users navigate through financial statements and focus

on matters highlighted by the auditor. In real life this effect might even be greater because the

financial statements are more complex than those used in our experiment.

The results also show that communicating several matters in the audit report reduces the

level of attention devoted to the rest of the financial statements disclosures, suggesting that the

audit report proposed by the IAASB and the PCAOB could possibly lead to less attention to

otherwise useful information not referred to in the report. In addition, the results show that

16

communicating multiple matters versus one, decreases the level of attention devoted to the part of

the financial statements mentioned in the audit report.

Our results are based on nonprofessional users. Research on financial analysts suggests

that they have fewer preconceived ideas of the importance of and relations among various

financial statement items than professional analysts and that they tend to read the financial

statements in the order presented (Bouwman 1982; Anderson 1988). Our results may not hold for

professional analysts who have valuation models and acquire information relevant for these

models using a directed-search strategy. Given that professional analysts are one of the groups

that asked for more communication by the auditor, future research could examine the effect of

critical/key audit matters on their information acquisition.

An additional avenue for future research is to examine more systematically the effect of

the number of matters in the audit report on the user information acquisition behavior. Rather than

examining one and three matters as in this study, these studies could use a complete factorial

design varying both the nature of the matter and the number of matters. This design will allow

separating the effect of the matter and the number of matter.

17

Appendix A Audit reports presented in the four groups

GroupA – base report (identical for all groups)

INDEPENDENT AUDITOR’S REPORT

To the Shareholders of Alpha Industries Inc.

We have audited the accompanying financial statements of Alpha Inc. which comprise the balance sheets as at January 31, 2013 and January 31, 2012 and the statements of earnings and cash flows for the years then ended, and the related notes, which comprise a summary of significant accounting policies and other explanatory information.

Management's responsibility for the financial statements

Management is responsible for the preparation and fair presentation of these financial statements in accordance with Canadian Accounting Standards for Private Enterprises, and for such internal control as management determines is necessary to enable the preparation of financial statements that are free from material misstatement, whether due to fraud or error.

Auditor's responsibility

Our responsibility is to express an opinion on these financial statements based on our audits. We conducted our audits in accordance with Canadian generally accepted auditing standards. Those standards require that we comply with ethical requirements and plan and perform the audit to obtain reasonable assurance about whether the financial statements are free from material misstatement.

An audit involves performing procedures to obtain audit evidence about the amounts and disclosures in the financial statements. The procedures selected depend on the auditor's judgment, including the assessment of the risks of material misstatement of the financial statements, whether due to fraud or error. In making those risk assessments, the auditor considers internal control relevant to the entity's preparation and fair presentation of the financial statements in order to design audit procedures that are appropriate in the circumstances, but not for the purpose of expressing an opinion on the effectiveness of the entity's internal control. An audit also includes evaluating the appropriateness of accounting policies used and the reasonableness of accounting estimates made by management, as well as evaluating the overall presentation of the financial statements.

We believe that the audit evidence we have obtained in our audits is sufficient and appropriate to provide a basis for our audit opinion.

Opinion

In our opinion, the financial statements present fairly, in all material respects, the financial position of Alpha Inc. as at January 31, 2013 and January 31, 2012 and the results of its operations and cash flows for the years then ended in accordance with Canadian Accounting Standards for Private Enterprises.

Signature

City, Date For groups B to D, the emphasis paragraph(s) or auditor commentaries are inserted after the opinion paragraph and before the signature.

Group B condition – 1 emphasis of matter

Auditor commentary

We draw attention to Note 5 to the financial statements, which describes the facts indicating that the company is facing a liquidity risk. Our opinion is not qualified in respect of this matter.

18

Group C condition – 3 emphases of matters

Auditor commentaries

We draw attention to the following notes:

Long-lived asset impairment

Note 1k to the financial statements, which describes accounting policies relating to the impairment of long-lived assets and the fact that the company has recorded impairment losses for fiscal years 2013 and 2012. Our opinion is not qualified in respect of this matter.

Liquidity risk

Note 5 to the financial statements, which describes the facts indicating that the company is facing a liquidity risk as it was in default regarding some financial ratios related to its credit facilities and certain debts. Our opinion is not qualified in respect of this matter.

Contingency

Note 14 to the financial statements which the existence of a material contingency related to litigation involving the company. No adjustment has been made to record this contingency. Our opinion is not qualified in respect of this matter.

Group D condition – 3 commentaries

Key audit matters

This section of our auditor’s report describes matters that, in our professional judgment, were of most significance in our audit of the financial statements.

Long-lived asset impairment

The company performs long-lived assets impairment tests when there is evidence of loss of value. The company has recorded impairment losses for fiscal years 2013 and 2012. In the course of our audits, we have reviewed the procedures and methodologies used by the company as well as the assumptions used by Management. Note 1k to the financial statements describes the information related to the impairment of long-lived assets.

Liquidity risk

The company is facing a liquidity risk as it was in default regarding some financial ratios related to its credit facilities and certain debts. In the course of our audits, we have, among other things, obtained confirmation from the creditor with respect to a waiver agreement. Note 5 to the financial statements describes the information related to the liquidity risk.

Contingency

The company is facing a material contingency related to litigation. No adjustment has been made to record this contingency. In the course of our audits we have evaluated the reasonableness of the elements on which is based this conclusion and examined that the note to the financial statements provides the appropriate information. Note 14 to the financial statements describes this contingency.

Further Information Relevant to Understanding Key Audit Matters

This information is intended to enhance users’ understanding of our audit of the financial statements. Our opinion is not modified with respect to any of these matters, and our audit procedures relating to these matters were designed in the context of our audit of the financial statements as a whole, and not to express an opinion on individual accounts or disclosures. Reading our auditor’s report is not intended to be a substitute for reading the financial statements, including the notes, in their entirety.

19

Appendix B Notes to the financial statements referred to in the audit reports

Note 1k) Impairment of long-lived assets

When events or circumstances indicate that impairment may exist, the Company reviews the carrying value of long-lived assets. Impairment exists when the carrying value of a long-lived asset or group of long-lived assets is not recoverable and exceeds the undiscounted future cash flows expected from its use and eventual disposition of the asset or group of assets. The amount of any impairment loss is the excess of the carrying value over fair value. The fair value of long-lived assets is determined based on discounted future cash flows. During fiscal 2013, impairment losses totaling $ 860,921 were found in this regard ($ 431,013 in 2012).

Intangible assets whose useful life is indefinite are tested for impairment on an annual basis or more frequently if events or circumstances indicate that impairment. Impairment exists when the carrying value of the intangible asset exceeds its fair value. During fiscal 2013, impairment losses totaling $ 125,881 were found in this area ($ 14,050 in 2012).

Note 5 Risks to liquidity and financial ratios

Over the past few years, the company has faced several operating challenges arising including the economic crisis, which resulted in lower revenue and gross margins and radiation.

Under the terms related to the credit facility described in Note 6 and certain liabilities described in note 7, the company is committed to meeting certain conditions as well as financial ratios. Dated January 31, 2013, the company was in default for certain financial ratios on its credit facilities and certain debts totaling $ 16,189,452. However, it has obtained waivers of the financial institution concerned. She also obtained an extension of the maturity of certain debts totaling $8,595,974 until August 31, 2014 and changes and flexibility applicable to credit facilities and financial ratios for these debts. Also, the company will have to renegotiate the credit facilities and debt totaling $8,595,974 expiring in August 2014. There is no assurance that such financing will be renewed at maturity in August 2014 or the financial ratios will respected at that date.

Note 14 Contingencies

In December 2011, the company was named as a defendant in a legal action for patent infringement by a competitor. It alleges that Alpha inc. infringed one of its patents by manufacturing and selling patented composite materials. The competitor claims to have suffered financial damages and claims more than $10 million in damages. The issues raised in the complaint are viewed by society as unfounded and unproven allegations will be vigorously contested, although no assurance can be given as to the outcome of these proceedings. The Company believes it has strong defenses to this request and therefore has not recorded any related liability.

20

References

Anderson, M. J. 1988. A comparative analysis of information search and evaluation behavior of professional and non-professional financial analysts. Accounting, Organizations and Society 13 (5):431-446.

Arnott, D. 2006. Cognitive biases and decision support systems development: a design science approach. Information Systems Journal 16 (1):55-78.

Bouwman, M. J. 1982. The use of accounting information: Expert vs. novice behavior. In Decision Making: An Interdisciplinary Inquiry, edited by G. Ungson and D. Braunstein. Boston. MA: Kent, 134–167.

Canham, M., and M. Hegarty. 2010. Effects of knowledge and display design on comprehension of complex graphics. Learning and Instruction 20 (2):155-166.

CFA Institute. 2011. Usefulness of the Independent Auditor’s Report: Survey to the CFA Institute Financial Reporting Survey Pool, 9 pages.

Chu, S., N. Paul, and L. Ruel. 2009. Using eye tracking technology to examine the effectiveness of design elements on news websites. Information Design Journal 17 (1):31-43.

Church, B. K., S. M. Davis, and S. A. McCracken. 2008. The Auditor's Reporting Model: A Literature Overview and Research Synthesis. Accounting Horizons 22 (1):69-90.

Collewijn, H. (1999). Eye movement recording. In: R. H. S. Carpenter and J. G. Robson (eds.), Vision Research: A Practical Guide to Laboratory Methods (pp. 245–285). Oxford: Oxford University Press

Crundall, D., P. Chapman, E. France, G. Underwood, and N. Phelps. 2005. What attracts attention during police pursuit driving? Applied Cognitive Psychology 19 (4):409-420.

Duchowski, A. T. (2002). A Breadth-First Survey of Eye Tracking Applications. Behavior Research Methods, Instruments, & Computers (BRMIC), 34(4), pp.455-470.

Einhäuser W., Koch C., Carter O. (2010). Pupil Dilation Betrays the Timings of Decisions, Frontiers in Neuroscience, 4 (18).

Elliott, W. B., F. D. Hodge, J. J. Kennedy, and M. Pronk. 2007. Are M.B.A. Students a Good Proxy for Nonprofessional Investors? The Accounting Review 82 (1):139-168.

Footprint Consultants. 2011. Study of the perception of statutory auditors’ justification of assessments, 44.

Goldberg, J.H. and Kotval, X., P. (1998) Eye Movement-Based Evaluation of the Computer Interface. In Kumar, S.K. (Ed.), Advances in Occupational Ergonomics and Safety, Amsterdam: IOS Press: 529-532.

International Auditing and Assurance Standards Board. 2011. Consultation Paper: Enhancing the Value of Auditor Reporting: Exploring Options for Change.

International Auditing and Assurance Standards Board. 2012. Invitation to Comment: Improving the Auditor’s Report. New York, NY: International Federation of Accountants.

21

International Auditing and Assurance Standards Board. 2013. Exposure Draft, Reporting on Audited Financial Statements: Proposed New and Revised International Standards on Auditing (ISAs). New York: International Federation of Accountants.

Jacob, R. (1995). Eye Tracking in Advanced Interface Design, in Virtual Environments and Advanced Interface Design, Oxford University Press, New York, pages 258-288.

Jacob, R. J. K., and K. S. Karn. 2003. Eye tracking in human-computer interaction and usability research: Ready to deliver the promises. In The mind's eye: Cognitive and applied aspects of eye movement research, edited by, Hyona, Radach & Deubel. Oxford, England: Elsevier Science BV.

Just, M. A., and P. A. Carpenter. 1976. Eye fixations and cognitive processes. Cognitive Psychology 8 (4):441-480.

Kessels, L. T. E., and R. A. C. Ruiter. 2012. Eye movement responses to health messages on cigarette packages. BMC Public Health 12 (1):352.

Lam, S. Y., A. W.-L. Chau, and T. J. Wong. 2007. Thumbnails as online product displays: How consumers process them. Journal of Interactive Marketing 21 (1):36-59.

Lindberg, T., and R. Näsänen. 2003. The effect of icon spacing and size on the speed of icon processing in the human visual system. Displays 24 (3):111-120.

Lohse, G. L. 1997. Consumer Eye Movement Patterns on Yellow Pages Advertising. Journal of Advertising 26 (1):61-73.

Maines, L. A., and L. S. McDaniel. 2000. Effects of Comprehensive‐Income Characteristics on Nonprofessional Investors' Judgments: The Role of Financial‐Statement Presentation Format. The Accounting Review 75 (2):179-207.

Mayer, R. E., and R. Moreno. 2003. Nine Ways to Reduce Cognitive Load in Multimedia Learning. Educational Psychologist 38 (1):43-52.

Mayer, R.E. (2001) Multimedia Learning Cambridge University Press, New York.

Miller, J. R., and L. M. Smith. 2002. The Effects of the Level of Assurance, Accounting Firm, Capital Structure, and Bank Size on Bank Lending Decisions. Journal of Accounting, Auditing & Finance 17 (1):51-71.

Mock, T. J., J. Bédard, P. J. Coram, S. M. Davis, R. Espahbodi, and R. C. Warne. 2013. The Audit Reporting Model: Current Research Synthesis and Implications. Auditing: A Journal of Practice & Theory 32 (Supplement 1):323-351.

Munafò, M. R., N. Roberts, L. Bauld, and U. Leonards. 2011. Plain packaging increases visual attention to health warnings on cigarette packs in non-smokers and weekly smokers but not daily smokers. Addiction 106 (8):1505-1510.

Poole, A. and Ball, L. J. 2005. Eye tracking in human-computer interaction and usability research: current status and future prospects. In Ghaoui, C. (Ed.), Encyclopedia of Human Computer Interaction, Idea Group

Public Company Accounting Oversight Board. 2013. Proposed Auditing Standards – The Auditor's Report on an Audit of Financial Statements when the Auditor Expresses an Unqualified Opinion; The Auditor's Responsibilities Regarding Other Information in

22

Certain Documents Containing Audited Financial Statements and the Related Auditor's Report; and Related Amendments to PCAOB Standards. PCAOB Release No. 2013-005. Washington, D.C.

Rayner, K. 1998. Eye movements in reading and information processing: 20 years of research. Psychological Bulletin 24 (3):372-422.

Sharif, B. and Maletic, J. (2010) An eye tracking study on the effects of layout in understanding the role of design patterns, 2010 IEEE International Conference on Software Maintenance, pp. 1-10

Viger, C., R. Belzile, and A. A. Anandarajan. 2008. Disclosure versus Recognition of Stock Option Compensation: Effect on the Credit Decisions of Loan Officers. Behavioral Research In Accounting 20 (1):93-113.

Wedel, M. and R. Pieters (2007). Informativeness of eye movements for visual marketing: Six cornerstones. Visual marketing: From attention to action, pp. 43–71.

23

Table 1: Summary of eye movement metrics

Eye and navigation metric used

Definition and construct Interpretation

Total fixations count Total number of fixations on the overall visual display (e.g., financial statement URL page) or for a particular area of interest (e.g., specific text area, for example section of Note 1-k). A fixation is defined as a short burst of stationary visual display where the eyes remain almost motionless while focusing on a particular point. A typical fixation lasts for 200-300 ms. approximately. The identification of fixation points is determined by the software and filter (i.e., algorithm) used. The IVT Fixation filter is used here (Tobii Studio user manuel, version 3.2)

High number of fixations on a particular area indicates that it is more important to the viewer than other areas (Poole and Ball, 2005). For a specific task, high number of fixations indicates the use of the area (Jacob and Karn, 2003).

Total fixation duration In millisecond (ms.) 1 minute = 60 000 ms.

The sum of time of all fixations on the overall visual display or for a particular area of interest.

Longer fixation duration may indicate difficulty in extracting information, or mean that the area is more engaging in some way (Just and Carpenter 1976).

Relative total fixation duration (relative total number of fixations)

Total fixation duration (count) for a particular area of interest, divided by the subject’s total fixation duration (count) on the full content areas (i.e., full pages excluding side navigation menu) of the complete financial statements over the full duration of the experiment. For Note 1-k, the subject’s total fixation duration (count) is scaled by the total fixation duration (count) recorded on the full text of Note 1.

Same as total fixation duration/count, controlling for subject unobserved characteristics, such as reading and financial statement analysis skill. For Note 1-k, the scaling on subject’s reading of the complete Note 1 further controls for differences on how specific sections of the financial statements can be read. This level of scaling is impossible for Notes 5 and 14 as the area of interest studied corresponds to the full text section of those notes.

Relative pupil size (first fixation)

Average pupil size for both eyes recorded over the first fixation on a particular area of interest, divided by the subject’s mean pupil size for both eyes recorded over the total time spent on the specific page containing the area of interest. This measure is “financial statement page” specific to

Pupil size varies by subjects and must be scaled to allow comparisons. Values above 1 indicate pupil size increase (dilation). Pupil dilation is a reflex and is attributed to effort and attention paid to the area of interest (Einhäuser et al., 2010). Measures of pupil size on the first fixation only are used as pupil dilation occurs over a very short time frame when first being exposed to a stimulus.

24

better control for factors impacting pupil size correlated with the display of a particular page (e.g., general luminosity of the content, random and minor variations in lighting condition, other content of interest in the page, etc.).

Number of “other pages” visited before

Number of page-visits to accessible URL pages during the task (i.e., financial statements, notes, auditor’s report and task evaluation page), before the first fixation was recorded for a particular area of interest. Multiple, non-continuous visits to the same page are added. Page-visits are counted after the subjects leave the auditor’s report page displayed at the beginning of the task. “Micro visits” are excluded from the count to limit the effect of noise/errors in subjects navigation (“clicks”) across URLs. Visits to URL pages with no recorded fixations on the main text or with total fixation duration below 500 milliseconds are excluded from the count.

Lower number other pages visited before fixating the area of interest indicates less time to access particular information. The lower the count of page-visits after reading the auditor’s report for the first time until fixating the relevant content of a note referred in the auditor’s report indicates greater integration of and interest in the information presented in the auditor’s commentaries/emphases (relative to the control group).

Time to first fixation (ms.)

Difference between the recorded time stamp from after the subjects finishes reading the auditor’s report (i.e., after clicking on “continue” on the first display of the auditor’s report) to the recorded time of the subject’s first fixation on a particular area of interest.

Similar to Number of “other pages” visited before (see above). Shorter time to first fixation indicates greater integration of and interest in the information presented in the auditor’s commentaries/emphases.

Proportion of subject with 0 fixations ("area not seen")

Number of subjects with no recorded fixation on a particular area of interest throughout the experiment, divided by the total number of subjects in a particular test group.

The lower the proportion, the more an area presents an interest to the subjects.

Number of URLs visits to Total page-visits to a particular group of pages: all page notes, excluding notes 1, 5 and 14 all financial statement pages, including all notes

but excluding the auditor’s report. Multiple, non-continuous visits to the same page are counted. Page-visits are counted after the subjects leave the auditor’s report page displayed at the beginning of the task.

Greater number of page-visits indicates attention to multiple sections of the financial statements. Moreover, greater page-visits may indicate less structured search strategies by subjects.

25

Table 2 Descriptive statistics of the participants

Group A Group B Group C Group D

Count 26 22 25 25

Sex (% Female) 39% 36% 44% 36%

Year (% 2nd year or MSc) 62% 59% 68% 60%

26

Table 3 First reading of the Auditor's report – average eye metrics per report format

Sections (number of words for scaling)

Group A Group B Group C Group D

Absolutea By wordsa Absolutea By wordsa Absolutea By wordsa Absolutea By wordsa

Common sections Introduction (65) 9 140 141 8 131 125 8 370 129 7 160 110

(40) (0.62) (34) (0.52) (37)*** (0.57) (33) (0.51) Management responsibility (60)

7 855 131 6 720 112 7 979 133 6 952 116 (33) (0.56) (27) (0.46) (38) (0.63) (32) (0.53)

Auditor's responsibility (232) 28 364 122 22 322 96 31 273 135 26 987 116 (120) (0.52) (90)*** (0.39) (137) (0.59) (121) (0.52)

Opinion (67) 6 268 94 5 719 85 4 262 64 5 404 81 (28) (0.42) (24) (0.35) (20)*** (0.29) (25) (0.37)

Commentaries/emphases Comm. Introduction (C = 10. D = 31)

3 661 366 4 268 138 (18) (1.76) (20) (0.63)

Comm./emph. - Note 1-k (C = 55. D = 93)

11 560 210 16 400 176 (47) (0.86) (71) (0.77)

Comm./emph. - Note 5 (B = 65. C = 52. D = 67)

8 251 127 8 992 173 12 273 183 (35) (0.54) (38) (0.74) (52) (0.78)

Comm./emph. - Note 14 (C = 44. D = 68)

12 420 282 14 976 220 (53) (1.21) (66) (0.98)

Comm. Add. Information (D = 102)

13 006 128 (60)*** (0.59)

27

Table 3 First reading of the Auditor's report – average eye metrics per report format (continued)

Sections (nb. words) Absolutea By wordsa Absolutea By wordsa Absolutea By wordsa Absolutea By wordsa

Total - common parts (424)

51 626 122 42 892 101 51 884 122 46 502 110 (222) (0.52) (175) (0.41) (232) (0.55) (210) (0.50)

Total commentaries/emphases (B = 65, C = 161, D = 361)

8 251 127 36 632 228 60 923 169

(35) (0.54) (157) (0.97) (270) (0.75)

Total all report (B = 489, C = 585, D = 785)

51 626 122 51 143 105 88 516 151 107 426 137

(222) (0.52) (210) (0.43) (388) (0.66) (480) (0.61) a Total fixation duration on first reading in ms. (1 minute = 60 000 ms.), fixation count in parenthesis. Total fixation duration and count for the first “reading” of the auditor’s report only; i.e., the first time the report is displayed. Subjects may return to the report at any time while navigating the financial statements and completing the task. The measures reported here do not include the time spent on reading the report during those later visits (minor). The total fixation duration and count on the first reading varies across groups from 87% to 91% of the total from all visits to the audit report.

b Absolute Total fixation duration (count) scaled by the number of words in the area of interest.

*** Subjects with 0 fixations – “are not seen”: Only 4 different subjects have “missed” reading the specific parts of the report identified above. Eye metric measures confirm that all subjects in groups B, C and D have seen and read the commentaries/emphases in the report.

28

Table 4 Impact of supplementary information in the audit report

Panel A Note 5

Aa (n= 26)

Expectation B-C-Da (n= 72)

Prob F (Chi-sq.)

Relative total fixation time 0.0306 < 0.0429 0.0191** (0.0248) (0.0272)

Relative total number of fixations 0.0303 < 0.0426 0.0156** (0.0245) (0.0261)

Relative pupil size (first fixation) 1.023 < 1.029 0.3416 (0.0394) (0.0533)

Number of other pages visited before 17.68 > 10.80 0.0005*** (9.16) (7.31)

Time to first fixation (ms.) 472 935 > 323 451 0.0048*** (224 266) (210 408)


26.9% > 11.1% (0.0588)†

Panel B Note 1-k

A-Ba (n = 48)

Expectation C-Da (n= 50)

Prob F (Chi-sq.)

n = 50 Relative total fixation time 0.1266 < 0.2046 0.0168**

(0.0152) (0.2222)

Relative total number of fixations 0.1204 < 0.2001 0.0148** (0.0154) (0.2212)

Relative pupil size (first fixation) 0.996 < 1.008 0.0536* (0.0305) (0.0476)

Number of other pages visited before 10.43 > 8.14 0.0375** (7.50) (5.99)

Time to first fixation (ms.) 397 109 > 289 046 0.0045*** (230 854) (186 493)


16.7% > 12.0% (0.3554)

29

Table 4 Impact of supplementary information in the audit report (continued)

Panel C Note 14

A-Ba (n = 48) Expectation

C-Da (n= 50)

Prob F (Chi-sq.)

Relative total fixation time 0.0235 < 0.0235 0.4703‡ (0.0148) (0.0131)

Relative total number of fixations 0.0228 < 0.0230 0.4980 (0.0131) (0.0122)

Relative pupil size (first fixation) 1.020 > 1.020 0.4318 (0.0289) (0.0363)

Number of other pages visited before 24.76 > 22.72 0.1652 (9.16) (9.40)

Time to first fixation (ms.) 789 303 > 658 432 0.0268* (336 450) (285 278)


6.3% > 6.0% (0.6415) a Group mean, standard deviation in parenthesis. ‡ All differences are of predicted sign, except Panel B “Relative total fixation time”.

***, **, * Significant difference of predicted sign between group means below the 1%, 5% and 10% level respectively (one-sided F-test).

† Significant difference of predicted sign in proportions below the 10% level (one-sided Fisher’s exact Chi-square test).

Eye and navigation metrics defined in Table 1.

30

Table 5 Impact of number of matters in supplementary information in the audit report - Note 5

Ba (n= 22) Expectation

C-Da (n= 50)

Prob F (Chi-sq.)

Relative total fixation time 0.0517 > 0.0390 0.0321**

(0.0277) (0.0263)

Relative total number of fixations 0.0520 > 0.0385 0.0208**

(0.0268) (0.0250)

Relative pupil size (first fixation) 1.045 > 1.020 0.0332 (0.0586) (0.0492)

Number of other pages visited before 8.62 < 11.86 0.0600* (6.37) (7.56)

Time to first fixation (ms.) 299 937 < 334 935 0.2716 (217 032) (208 724)


4.5% < 14.0% (0.2285)

a Group mean, standard deviation in parenthesis. All differences are of predicted sign.


† Significant difference of predicted sign in proportions below the 10% level (one-sided Fisher’s exact Chi-square test).


31

Table 6 Tests of differences in eye and navigation metrics - Financial statements (total) and all other notes

Panel A Impact of supplementary information

Aa (n= 26) Expectation

B-C-Da (n= 72)

Prob F (Chi-sq.)

Total fixation time - All other notes (ms.) 304 431 > 286 244 0.0071*** (161 245) (145 330)

Total fixations - All other notes (count) 1 355 > 1 266 0.0159** (679) (575)

Number of URLs visits to - All other note pages

21.9 > 18.9 0.0088*** (10.7) (8.2)

Total fixation time - Financial statements total, excluding auditor’s report (ms.)

659 020 > 632 395 0.0064***

(317 788) (255 125) Total fixations - Financial statements total, excluding Auditor’s report (count)

2 931 > 2 758 0.0169**

(1 290) (975) Number of URLs visited to - Financial statements total, excluding Auditor’s report

41.9 > 36.7 0.0099*** (15.9) (13.8)

Panel B Impact of number of matters in supplementary information

Ba (n= 22) Expectation

C-Da (n= 50)

Prob F (Chi-sq.)

Total fixation time - All other notes (ms.) 354 941 > 256 017 0.0046*** (174 509) (120 421)

Total fixations - All other notes (count) 1 498 > 1 165 0.0154** (635) (522)

Number of URLs visited to - All other pages

22.0 > 17.6 0.0289** (9.0) (7.5)

Total fixation time - Financial statements total, excluding auditor’s report (ms.)

762 963 > 574 945 0.0033*** (305 103) (208 316)

Total fixations - Financial statements total, excluding Auditor’s report (count)

3 153 > 2 584 0.0183** (1 036) (904)

Number of URLs visited to - Financial statements total, excluding Auditor’s report

41.3 > 34.7 0.0381** (15.9) (12.4)

a Group mean, standard deviation in parenthesis. All differences are of predicted sign.



32

Figure 1 Research Design

Matter referred to in the audit report Control

(A)

Emphasis 1 matter

(B)

Emphasis 3 matters

(C)

Commentary 3 matters

(D)

Note 1k 0 0 1 1

Note 5 0 1 1 1

Note 14 0 0 1 1