new findings on unconscious versus conscious thought in decision making: additional empirical data...

7/29/2019 New findings on unconscious versus conscious thought in decision making: additional empirical data and meta-an

1/13

Judgment and Decision Making, Vol. 3, No. 4, April 2008, pp. 292303

New findings on unconscious versus conscious thought in decision

making: additional empirical data and meta-analysis.

Felix Acker

School of Psychological Science

La Trobe University

Abstract

Ninety-eight Australian students participated in a functional replication of a study published by Dijksterhuis et al.

(2006). The results indicated that unconscious thought does not necessarily lead to better normative decision making

performance than conscious thought, which is contrary to the results found in Dijksterhuis et al. Since other studies

showed a positive, though statistically not significant, effect for unconscious thought, a meta-analysis comprising atotal of 17 experiments was conducted. It suggests that there is little evidence for an advantage to normative decision

making using unconscious thought. However, a discussion of potential moderators shows that further study would help

to identify situations in which unconscious thought is truly helpful and those in which it is not.

Keywords: unconscious thought, meta-analysis, normative decision making.

1 Introduction

In a series of studies, Dijksterhuis and colleagues (e.g.,

Dijksterhuis, 2004b; Dijksterhuis, Bos, Nordgren, & van

Baaren, 2006; Dijksterhuis & Meurs, 2006) established

the surprising and counterintuitive finding that uncon-

scious thought leads to better decision making perfor-mance for complex problems than conscious thought.

Unconscious thought is defined as cognitive and/ or

affective task-relevant processes that take place outside

conscious awareness (Dijksterhuis, 2004b, p. 586). A

second postulate is that the reverse is true for simple de-

cision: consciously thinking about them is better. This

paper is concerned only with the first.

A common denominator of all experiments described

by Dijksterhuis and colleagues is the experimental tech-

nique. Participants are split in two or three groups and

provided with a number of pieces of information about a

number of options to choose from, for instance four apart-ments or flatmates. Each option is described by the same

number of attributes, and usually these are attributes of

the choice option. For example, an attribute of all choice

options for apartments might be the size. Typically

these attributes are conceptualized dichotomously ei-

ther an apartment is spacious or it is not. All pieces of

information about all choice options are presented either

I am indebted to Mary Omodei and Geoff Cumming (both La Trobe

University), Ben Newell (UNSW) as well as one other anonymous

reviewer for helpful comments and critique. Address: Felix Acker,

School of Psychological Science, La Trobe University, 1 Kingsbury

Drive, Melbourne, 3078. Email: [email protected]

as lists (e.g., Newell, Wong, Cheung, & Rakow, submit-

ted), individually and randomized (e.g., this study), or in-

dividually in a fixed order (e.g., Phillips et al., 2007) for

a fixed amount of time. Prior to the information presen-

tation, participants are informed that they will have to a)

choose one of the options (e.g., Dijksterhuis, 2004b, Ex-

periment 1) or b) rate each option (e.g., Lerouge, submit-ted). Then, following the presentation of the information,

participants either immediately make a choice between

options or think about their choice for a fixed amount of

time (except in Payne, Samper, Bettman, & Luce, 2007),

or are distracted for the same amount of time before mak-

ing their decision.

The principal claim is that, when the number of fac-

tors that ought to be considered is high (as indicated by

the number of attributes), and the decision is therefore

complex, unconscious thought will lead to better deci-

sion performance than conscious deliberation. Better de-

cision making (which I call normative here) is definedas choosing, or ranking higher, options with more pos-

itive features. However, although most of Dijksterhuis

and colleagues experiments found an effect in the hy-

pothesized direction, such effects often failed to reach

statistical significance when comparing the unconscious

and conscious group directly. For example, in the first

reported experiment in Dijksterhuis 2004 study the crit-

ical comparison did not reach statistical significance (p 10 indicate that all information for a given option was presented simultaneously, the value >100

indicates that all information for all options were presented simultaneously.

The amount of variability between the effect sizes was

substantial (Q[df=16] = 54.994, p .000; I2

= 70.906).Only five out of the included 17 experiments returned re-

sults that can be described as statistically significant in

classical terms. Each of these five provided evidence for

the superiority of unconscious thought. They also had

the largest effect sizes but at the same time the smallest

sample sizes. Newer data that still await publication pro-

vided evidence conflicting with the unconscious thought

theory. The aggregate estimate shows a modest benefit

for unconscious thought, although, from a significance-

testing perspective, the confidence interval includes 0

and can thus be interpreted as non-significant support.

3.2.2 Moderator variables

Given the high level of study heterogeneity, several meta-

regressions were carried out to investigate the effect of

potential moderator variables. The statistical package

Comprehensive Meta AnalysisTM (Borenstein, Hedges,

Higgins, & Rothstein, 2008) was used for this purpose.

Gender ratio: Dijksterhuis (2004), in his discussion of

Experiment 1, found an interaction between gender and

thought condition. Males were choosing particularly well

following unconscious thought. Other studies have not

specifically investigated this point, but it was worthwhile

to follow up with a large set of data here. The vast ma-


9/13

Judgment and Decision Making, Vol. 3, No. 4, April 2008 Conscious versus unconscious thought 300

This study

Dijksterhuis (2004), Exp 1




Dijksterhuis et al. (2006), Exp 1

Dijksterhuis et al. (2006), Exp 2

Ham et al. (sfp), Exp 2

Ham et al. (sfp), Exp1

Lerouge (sfp), Exp 2 configural

Lerouge (sfp), Exp 2, featural

Newell et al. (sfp), Exp 1



Payne et al. (2007), Exp 1

Payne et al. (2007), Exp 2

Phillips et al. (2007)

Overall

1.0 0.5 0.0 0.5 1.0 1.5 2.0

Figure 4: Forest plot of studies displaying effect sizes and

95% confidence intervals.

jority of experiments exhibited a surplus of females, with

the exception of Payne et al. (2007). The experiments by

Lerouge (submitted) and Dijksterhuis et al. (2006) were

not included as no data on gender were available. The

regression analysis suggested that the gender ratio of a

study is a poor predictor of effect size (= 0.214, CI95[0.786, 0.357], SE= 0.291).

Item presentation duration: The analysis of item pre-

sentation duration focused only on those studies that

showed each piece of information individually. Stud-

ies (Dijksterhuis, 2004, Experiment 4; Ham et al., sub-

mitted, Experiments 1 and 2, Lerouge, submitted, both

conditions; Newell et al., submitted, Experiment 2) that

showed items list-wise or all simultaneously were ex-

cluded. The analysis showed a trend that longer pre-

sentation times per item led to less advantage for uncon-

scious thought, but this relationship was slight and did not

reach statistical significance (= 0.095, CI95 [0.232,

0.042], SE= 0.07).

Thought interval: All studies were included in this

meta-regression. Similar to the item presentation vari-

able, the results suggested that a longer interval between

information presentation and decision is favourable for

conscious thought rather than unconscious thought.

Again, though, the result did not reach statistical signif-

icance ( = 0.199, CI95 [0.445, 0.048], SE = 0.126)

and was strongly influence by Newell et al.s (submitted)

second experiment, which had a substantially longer in-

terval than all other studies.

Table 6: Effect sizes (g), standard errors (SE) and rela-

tive weights (w) for the experiments included in the meta-

analysis. The abbreviation sfp means submitted for pub-

lication.

Study name g SE w

This study 0.471 0.243 6.743

Dijksterhuis (2004), Exp 1 0.434 0.306 5.998




Dijksterhuis et al. (2006), Exp 1 0.968 0.390 5.054

Dijksterhuis et al. (2006), Exp 2 1.247 0.417 4.774

Ham et al. (sfp), Exp 2 0.883 0.352 5.469

Ham et al. (sfp), Exp 1 1.055 0.349 5.503

Lerouge (sfp), Exp 2 configural 1.116 0.326 5.765

Lerouge (sfp), Exp 2, featural 0.064 0.303 6.033

Newell et al. (sfp), Exp 1 0.171 0.336 5.650



Payne et al. (2007), Exp 1 0.722 0.393 5.025

Payne et al. (2007), Exp 2 0.483 0.340 5.604

Phillips et al. (2007) 0.251 0.222 6.984

Overall 0.251 0.137

Presentation format: A last moderator variable was

the effect of presenting all pieces of information either in-

dividually or in clusters (as lists for each choice option or

all simultaneously). This information was not available

for Phillips et al.s data, which was therefore excluded

from the analysis. The results showed that the aggregate

effect size was lower (g = 0.147, CI95 [0.037, 0.331],

SE = 0.094) for experiments that presented the items in-

dividually (n = 9) than for the overall estimate. On the

other hand, the results for studies that presented multiple

pieces of information at the same time (n = 6) suggested

a higher and most likely positive effect (g = 0.369, CI95[0.110, 0.627], SE= 0.132). This means that unconscious

thought may actually be helpful when much information

is presented simultaneously, but not when bits of infor-

mation are presented individually.

3.2.3 Unconscious thought versus immediate deci-

sion making

Altogether 13 out of the 17 data sets were included in

a meta-analysis comparing decision making after uncon-

scious thought with immediate decision making. The five

excluded data sets did not feature the immediate decision


10/13


conditions. These were Dijksterhuis et al. (2006, Exper-

iments 1 and 2), Payne et al. (2007, Experiment 1) and

Phillips et al. (2007). The results were similar to the

comparison of unconscious and conscious thought. There

was a modest trend in favour of unconscious thought (g

= 0.189, CI95 [0.05, 0.428]) but a significant amount

of heterogeneity across studies (Q[df=12] = 26.691, p =

.009; I2= 55.041). Further analyses to identify true mod-

erator variables were not carried out.

3.2.4 Conscious thought versus immediate decision

making

The same 13 data sets were used for the con-

scious thought versus immediate decision making meta-

analysis. The results showed only a very slight advantage

for conscious thought over immediate decision making

(g = 0.084, CI95 [0.72, 0.24]) with substantially moreagreement among experiments than for the other compar-

isons (Q[df=12] = 9.77, p = .636; I2= 0).

4 Discussion

4.1 Empirical study

The results of the present experiment are noticeably at

odds with the theory of unconscious thought. According

to the obtained data set, it is a better idea to consciously

think about different choice alternatives in order to ar-

rive at the best choice than letting the unconscious dothe work. Not only, it seems, does conscious thinking

lead to the better identification of the top choice, but it

also allows to differentiate between gradual choice op-

tions. With unconscious thought the choice is a muddy

one; it did not consistently help individuals to differen-

tiate between choice alternatives. Unconscious thought

failed to allow a clear distinction of the cars; contrary to

the expectations conscious thought did do just that.

While some alternative explanations for the diver-

gence from the expected results were investigated (re-

cency effects, weighing differences, scale usage differ-

ences), other factors in which the present study differed

from Dijksterhuis et al. (2006) may have been responsible

for the reversal of results. An obvious difference between

this and the Dijksterhuis study was that this experiment

was carried out with English material and Australian stu-

dents. It is difficult to imagine, however, how cultural

or linguistic variation could have had such a radical ef-

fect on decision making quality. Similarly, delivering the

information to a group rather than on an individual ba-

sis is unlikely to have had any impact, especially since

participants were engaged in the task and did not distract

one another during the task presentation or the thought

interval. One other noteworthy difference between this

experiment and the original study is the distractor task

used. Dijksterhuis et al. mainly used anagrams to keep

participants in the unconscious thought condition busy; a

word search task was used here. Anagrams have been

widely used in the study of incubation (see for exam-

ple Vul & Pashler, 2007) and seem to be positively re-

lated to the kind of processes activated during uncon-

scious thought. Hence, anagram solving might have had

a positive mediator function. Word search, on the other

hand, does not have much in common with unconscious

thought, as most people scan the array of letters system-

atically for the correct combinations. Hence it is more

akin to conscious thought. However, in some studies Di-

jksterhuis (2004b) and also Ham (Ham, Bos, & Doorn,

submitted) used the n-back task (Kane, Conway, Miura,

& Colflesh, 2007). This task puts high demand on ex-

ecutive functioning and also can hardly be conceived as

supporting unconscious thought. Nevertheless, the pos-sibility that the type of distractor tasks affects decision

making efficiency under unconscious thought conditions

warrants further study and could be used as a moderator

variable in future meta-analyses when more experiments

with distractor tasks other than anagrams are available. If

all these three alternative explanations can be discounted,

then the results of the present study provide strong evi-

dence that the true effect size for unconscious thought is

much smaller than assumed so far or that this particular

experimental approach is not very suitable to demonstrate

the unconscious thought effect reliably. Further support

for either of these two conclusions comes from the meta-analytic findings presented here.

4.2 Meta-analysis

The statistical synthesis of all available data provides

at best suggestive evidence in favour of unconscious

thought, but, on the basis of 888 subjects tested under

similar conditions, there is no convincing statistical evi-

dence. The true effect in the population may be anything

between a moderate benefit after unconscious thought

to a slight advantage following conscious thought. One

sign for caution is that the experiments with fewer par-

ticipants consistently generated substantially larger effect

sizes than the larger studies.

4.3 Moderator variables

Four moderator variables were investigated in the present

meta-analysis. Of these, only the presentation format

as either single item or list-wise, did help to explain

the variance between studies, which is substantial. On

the other hand, gender-ratio, presentation time per item

and thought interval length were very weak predictors

of effect. This weakness, however, may have partially


11/13


been due to methodological constraints. For example,

the moderator effect for gender ratio looked at the to-

tal number of males and females in the experiment. A

better estimator would be the ratio of males to females

within the specific conditions, since there may be inter-

actions that cannot be uncovered by looking at the to-

tal number for each gender alone. Secondly, there may

be higher order interactions between different moderators

that obscure any simple effects. This meta-analysis car-

ried out only simple meta-regressions, but there is reason

to believe that multiple meta-regression may find rela-

tionships, given that I2 turned out to be large. Before such

analysis is feasible, though, more studies are required to

ensure the statistical soundness of the analysis (Boren-

stein et al., 2008).

Beside the ones investigated here, further potential

moderators have been identified. Configural versus fea-

tural processing (Lerouge, submitted) or self-paced de-cision making (Payne et al., 2007) and perhaps the type

of distractor task are promising candidates. These will

require much more data do make them suitable for meta-

analysis. Further, the present selection of experiments

was restricted to normative choices, but this in itself may

be a factor affecting the efficiency of unconscious or

conscious thought. Future approaches should thus in-

clude studies that operationalise good decision making

with subjective degrees of decision satisfaction. Finally,

the meta-analytic confirmed effect of presentation format

should be investigated in a specifically designed study.

At this stage, with few experiments, the result may stillbe spurious, especially since higher order interaction of

moderators and task conditions are unclear.

4.4 Other comparisons

The primary focus of the present study was to illumi-

nate the current state of affairs regarding unconscious

versus conscious thought. The comparison between un-

conscious or conscious thought and immediate decision

can further help to draw some useful conclusions. On

the basis of 13 sets of data, there is no statistical evi-

dence to suggest that either form of thought following

stimulus presentation leads to better choices than imme-

diate decision making. This, again, is quite surprising

given prior evidence for conscious thought processing as

represented in, for example, the weighted-additive model

of decision making (Keeney & Raiffa, 1976). The fact

that the meta-analytic review is not able to distinguish

between the three alternative ways of decision making at

all reinforces two possibilities: 1., other variables such as

presentation format or even gender significantly interact

with decision making condition; or, 2., the general ap-

proach is not suitable to differentiate modes of thought

and might instead reflect other influences (see, for exam-

ple, Newell et al. [submitted], Experiment 4).

As shown by the present study, the early finding that

unconscious thought leads to normatively better decisions

than conscious thought (Dijksterhuis, 2004, Experiment

1), which provided the starting point for further devel-

opments of the theory of unconscious thought, is unsup-

ported when looked at from a meta-analytic perspective.

However, this insight became possible only through the

inclusion of many data sets that are yet to be published.

All currently published data, with the exception of the

present empirical study, showed uniform support for the

theory of unconscious thought. The actual variability in

results that was demonstrated by the inclusion of addi-

tional, unpublished data thus reinforces the need to get

more results out in the open.

4.5 Conclusion

In summary, the findings reported here in conjunction

with the ones found in Newell et al. (submitted), Payne

et al. (2007), Phillips et al. (2007) and Lerouge (submit-

ted) cast doubt on the unqualified claim that unconscious

thought is the superior way of processing information for

important and complex choices. So far the bulk of stud-

ies had provided confirmatory evidence for unconscious

thought in different contexts such as justice and consumer

choice, in the laboratory as well as in more naturalistic

studies, and with different dependent variables. Taken

together, these studies were certainly bolstering the di-

vergent validity of the theory of unconscious thought.

However, with the inclusion of new results, the theory

will have to explain the effects of moderator variables

as shown in Lerouge (submitted) or Payne et al. (2007)

and, at least for normative choices, will have to step away

from choice model that identifies the rational choice as

the one with most positive attributes (see Newell et al.)

if it is to retain internal validity and produce correct pre-

dictions. For the time being there is little guarantee that

unconscious thought will truly be a good aid when faced

with complex and important decisions.

References

Bekker, H. L. (2006). Making choices without deliberat-

ing. Science, 312, 1472.

Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Roth-

stein, H. R. (2008). Comprehensive Meta-Analysis

(Version 2.2.046) [Computer software]. Englewood,

NJ: Biostat.

Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Roth-

stein, H. R. (2008). Introduction to meta-analysis.

Manuscript in preparation.


12/13


DeCoster, J. (2004). Meta-analysis notes. Retrieved 01,

11, 2007, from http://www.stat-help.com/notes.html

Dijksterhuis, A. (2004a). [decision quality after different

amounts of unconscious thought]. Unpublished raw

data.

Dijksterhuis, A. (2004b). Think different: The merits of

unconscious though in preference development and de-

cision making. Journal of Personality & Social Psy-

chology, 87, 586598.

Dijksterhuis, A., Bos, M. W., Nordgren, L. F., & van

Baaren, R. B. (2006). On making the right choice: The

deliberation-without-attention effect. Science, 311,

10051007.

Dijksterhuis, A., Bos, M. W., van Baaren, R. B., & van

der Leij, A. (2007). Unconscious thought and the

weighting problem in decision making. Manuscript in

preparation.

Dijksterhuis, A., & Meurs, T. (2006). Where creativityresides: The generative power of unconscious thought.

Consciousness and Cognition, 15, 135146.

Dijksterhuis, A., & Nordgren, L. F. (2006). A theory of

unconscious thought. Perspectives on Psychological

Science, 1, 95109.

Dijksterhuis, A., van Baaren, R. B., Bongers, K. C. A.,

Bos, M. W., van Leeuwen, M. L., & van der Leij, A.

(in press). The rational unconscious: Conscious versus

unconscious thought in complex consumer choice.

Dijksterhuis, A., & van Olden, Z. (2006). On the ben-

efits of thinking unconsciously: Unconscious thought

increases post-choice satisfaction. Journal of Experi-mental Social Psychology, 42, 627631.

Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (in

press). G*power 3: A flexible statistical power analy-

sis program for the social, behavioral, and biomedical

sciences. Behavior Research Methods.

Grissom, R. J., & Kim, J. J. (2005). Effect sizes

for research: A broad practical approach. London:

Lawrence Erlbaum.

Ham, J., Bos, K. v. d., & Doorn, E. A. V. (submitted

for publication). Lady justice thinks unconsciously:

Evidence for more accurate justice judgments without

conscious thought.

Kane, M. J., Conway, A. R., Miura, T. K., & Colflesh, G.

J. (2007). Working memory, attention control, and the

n-back task: A question of construct validity. Journal

of Experimental Psychology: Learning, Memory, and

Cognition, 33, 615622.

Keeney, R. L., & Raiffa, H. (1976). Decisions with mul-

tiple objectives: Preference and value tradeoffs. New

York: Wiley.

Lerouge, D. (submitted for publication). Evaluating the

benefits of distraction on product evaluations: The

mindset effect. Journal of Consumer Research.

Newell, B. R., Wong, K. Y., Cheung, J., & Rakow, T.

(submitted for publication). Think, blink or sleep on it?

The impact of bodes of thought on complex decision

making.

Payne, J. W., Samper, A., Bettman, J. R., & Luce, M. F.

(2007). Boundary conditions on unconscious thought

in complex decision making. Manuscript in prepara-tion.

Phillips, Nathaniel D., Gonzlez-Vallejo, C., Bellezza,

F. S., Chimeli, Janna., Harman, J., Lassiter, G. D., &

Lindberg, Matthew J. (2007). [Comparing the effects

of distraction and deliberation in an online attitude up-

dating task]. Unpublished raw data.

Schmidt, F. L., Oh, I.-S., & Hayes, T. L. (in press). Fixed

versus random effects models in meta-analysis: Model

properties and an empirical comparison of differences

in results. British Journal of Mathematical and Statis-

tical Psychology.

Shanks, D. R. (2006). Making choices without deliberat-ing. Science, 313, 760.

Vul, E., & Pashler, H. (2007). Incubation benefits only

after people have been misdirected. Memory and Cog-

nition, 34, 701710.


13/13

Reproducedwithpermissionof thecopyrightowner. Further reproductionprohibitedwithoutpermission.

new findings on unconscious versus conscious thought in decision making: additional empirical data...

Documents