social psychology before and after the s

8/19/2019 Social Psychology Before and After the S

1/10

Abma - Social psychology & Stapel 28 Aug. 2013

1

Social psychology before and after the Stapel fraud caseRuud Abma

Utrecht UniversityThe Netherlands

On the 28 th of November 2012, the three committees that had investigated the work of theDutch social psychologist Stapel, published their joint report Flawed science: The fraudulentresearch practices of social psychologist Diederik Stapel (Levelt Committee, 2012). They hadestablished without any doubt that Stapel had committed fraud in 55 papers, all published ininternational journals in the period 2004-2011. Moreover, they found serious evidence offraud in 10 earlier papers. Finally, in 10 PhD theses that Stapel supervised one or morechapters were based on fraudulent data.

On the same day that the report was presented, Stapel appeared on Dutch television.He stated that he was sorry for the damage he had caused to his field, his colleagues and hisPhD students. His mental condition did not allow him to be interviewed by the press, but hereferred to an autobiographical book that was about to published: ‘Ontsporing’, which literallymeans ‘derailment’.

In this paper, I will discuss the meaning and significance of the Stapel fraud case, against the background of the post-war history of social psychology. The 1950’s and 1960’s saw a rise ofexperimental social psychology, followed by a critical intermezzo, in which basicepistemological, methodological and ethical issues were discussed. This did not, however,stop the development of experimental social psychology. The Stapel fraud case has renewedthe interest in current methodological problems of social psychology. What are the differenceswith the earlier wave of criticism?

Stapel’s fraud

Fraude ratio Stapel

ESHHS Würzburg 20133


2/10


2

The graph shows Stapel’s production of international (English) papers (black line) and theamount of fraudulent papers (dotted line). His high production in 1998 gave him his image of‘golden boy’ with a knack for successful experimental work. It also helped him to get a

professorship in Groningen. His extremely low production in 2003 drove him to falsify data,according to himself for the first time. When no one found out and instead everyone admired

his beautiful results (‘too good to be true’), falsification became a habit, and later on hisfabrication of complete data sets guaranteed a continuing high publishing record.

Stapel even succeeded in getting his paper ‘Coping with chaos’ published in Science ,on messy environments and stereotyping, which was completely made up (Stapel &Lindenberg, 2010). Neither his co-author Siegwart Lindenberg, a reputed sociologist, nor thereviewing editor of Science, the Dutch social psychologist Ap Dijksterhuis had any doubtsabout the truthfulness of the reported experiments.

Stapel published mainly in social psychology journals, his favourite being the Journalof Personality and Social Psychology , the leading journal in the field. More than half of his

publications in this journal (1996-2011) were based on falsified or fabricated data. The picturefor other journals is about the same (see Table). Neither his 70 co-authors, nor the journalreviewers were alerted by the sometimes quite obvious ‘mistakes’ and unlikely patterns inStapel’s papers.

J. of Personality and Social Psychology 13/24 European Journal of Social Psychology 11/19 Personality and Social Psychology Bulletin 8/15 Journal of Experimental Social Psychology 5/14 Social Cognition 5/14 British Journal of Social Psychology 3/4 Psychological Science 2/4

Basic and Applied Social Psychology 2/3

Sloppy science?Stapel had published fraudulent papers, co-authored by leading figures in the field, in themajor social psychology journals without being noticed at all. The Levelt committee thereforeraised the question ‘why this fraud and the widespread violations of sound scientificmethodology were never discovered in the normal monitoring process in science […]Virtually nothing of all the impossibilities, peculiarities and sloppiness mentioned in thisreport was observed by all these local, national and international members of the field, and nosuspicion whatever arose’. (Levelt committee, 53) Therefore they concluded that ‘there arecertain aspects of the discipline that should be deemed undesirable or even incorrect from the

perspectives of academic standards and scientific integrity’ (Ibidem, 54).

Confirmation biasThe main target of the committees was the so-called verification (or confirmation) bias: ‘Oneof the most fundamental rules of scientific research is that an investigation must be designedin such a way that facts that might refute the research hypotheses are given at least an equalchance of emerging as do facts that confirm the research hypotheses. Violations of thisfundamental rule, such as continuing to repeat an experiment until it works as desired, orexcluding unwelcome experimental subjects or results, inevitably tend to confirm theresearcher’s research hypotheses, and essentially render the hypotheses immune to the facts.’(Ibidem,.48)


3/10


3

The report subsequently presents a multitude of examples of verification bias inStapel’s publications. When confronted with these examples, several co-authors ‘defended theserious and less serious violations of proper scientific method with the words: that is what Ihave learned in practice; everyone in my research environment does the same, and so doeseveryone we talk to at international conferences’. (Ibidem, 48) Therefore the committees

concluded that ‘from the bottom to the top there was a general neglect of fundamentalscientific standards and methodological requirements’. (Ibidem, 53)

According to the committee, this situation was aggravated by certain practices byinternational journals: ‘For instance, a co-author stated that editors and reviewers wouldsometimes request certain variables to be omitted, because doing so would be more consistentwith the reasoning and flow of the narrative, thereby also omitting unwelcome results.Reviewers have also requested that not all executed analyses be reported, for example bysimply leaving unmentioned any conditions for which no effects had been found, althougheffects were originally expected. […] Not infrequently reviews were strongly in favour oftelling an interesting, elegant, concise and compelling story, possibly at the expense of thenecessary scientific diligence.’ (p.53)

As early as 1998, Stapel himself openly had advocated the practice of verification bias, in aspeech at the occasion of receiving the Jos Jaspars Award: ‘We design an experiment and goto our lab to test our conjectures. And then what happens? Our experiment fails. We don’tfind what we expected to find. Moreover, we find something we cannot explain. We tweakand fine-tune the experimental set-up until we find something we do comprehend, somethingthat works, something with a P-value smaller than .05. Champaign! Celebration! […]

I am sure that there are other ways of doing experimental social psychology.Sometimes, for example, our research is theory- rather than data- or observation-driven. My

point is that whatever way we arrive at our theories and hypotheses, the experiments and testswe design are made to verify, not to falsify our conjectures. The leeway, the freedom we havein the design of our experiments is so enormous that when an experiment does not give uswhat we are looking for, we blame the experiment, not our theory. (At least, that is the way Iwork). Is this problematic? No.

Our results are often paradigm-contingent. That is, we find what we are looking for because we design our experiments in such a way that we are likely to find what we arelooking for. Of course! Should we design our experiments such that we are unlikely to findsupport for our hypotheses? Should we try to prove ourselves wrong? No, for the best results,we should use the methods that are likely to work best. Use a spoon to eat your soup and acup to drink your tea. Not vice versa.’ (Stapel, 1998, xx)

Here, Stapel appears to be a good pupil of Festinger (1953, 156) who states that ‘there isalmost limitless room for the experimenter’s ingenuity to create a situation which will be bestfor his experimental purpose’. Fifteen years later, Aronson and Carlsmith (1968) suggestedthat ‘there are almost as many ways of building and conducting an experiment as there areexperimenters. (p.3 / 372).

Angry social psychologistsThe evocation by the Levelt committees of the ‘fundamental rules of scientific research’ andthe suggestion that social psychologists in their research tend to fall short of these standards,created an uproar in social psychological circles.


4/10


4

For instance, the emeritus professor of social psychology from Utrecht University, WolfgangStroebe, offered as his opinion that the report was an insidious attack on the whole of social

psychology. Stroebe agreed that the reviewer blindness for Stapel’s misconduct is shocking, but he added ‘sadly it is pervasive to all fields of science and not specific to social psychology. The use of this observation as an argument to attack the entire field of social

psychology shows therefore a breathtaking ignorance of the vast literature on scientificmisconduct.’ (DUB, 5 dec. 2012)

Interestingly, Stroebe added the following observation: ‘The major difference betweenthe Levelt/Noorth/Drenth Committees and the reviewers of Stapel’s manuscripts, whooverlooked discrepancies that were obvious to the clever committee members, is that thesereviewers were reviewing articles by a star scientist of unblemished reputation, whereas theCommittee already knew that most of this research was fraudulent.’ Apparently, Stroebe didnot presume that there was Stapel’s papers were submitted to blind reviewing.Stroebe ended his contribution by requesting an apology: ‘I would therefore suggest that themembers of the committee read the vast literature on scientific fraud, enriched by some social

psychological reading on “hindsight bias” (why people are always cleverer after the fact).They could then produce an amended version of their report, omitting all defamatorystatements about social psychology. And finally, they might offer their apology to the manyscientists they have insulted with their slanderous conclusions about social psychology.’

Stroebe’s article was followed by a long string of comments by social psychologists,most of whom agreed with him. For instance, Fritz Strack of Würzburg University stated:‘The report is an excellent example of "slodderwetenschap". It spreads sweepinggeneralizations without clear documentation while neglecting the diligence to which itsubjects and holds up social psychology. I doubt that its claims would pass peer review andeditorial scrutiny for any scientific journal.’

More officially, the European Association for Social Psychology also protested againstthe conclusions of the report about the research culture in social psychology: ‘The Leveltreport was published on November 28, but instead of providing the expected closure and awelcome insight into one of the darkest chapters of the history of social psychology, it bringsthe discipline as a whole into disrepute. For, in addition to describing Stapel’s fraudulentactivities, the report characterizes social psychology as a discipline with low academicstandards and limited scientific integrity.’ (EASP, 2012)

Not one of the angry social psychologists, however, responded to the critique that itwas the policy of the journals in this field that encouraged researchers to cut corners and‘photoshop’ their data and analyses.

A historical view

Confirmation bias is found in experimental social psychology, but is it limited to social psychology? Probably not (Stroebe, Postmes and Spears, 2012). But it certainly has a longtradition here.

In ‘Making social psychology experimental’, Kurt Danziger (2000) demonstrates howthe meaning of ‘experimentation’ has varied widely in psychology, but ended up in specifictypes of manipulative social psychological experimentation, in which ‘social’ was reduced to‘interpersonal’, whereby ‘humans functioned as ‘social stimuli’ only insofar as they weredirectly present to each other’ (Danziger, 2000, 333). These experiments were limited toexploring effects that were local, proximal, short term and decomposable. This constituted aradical break [by F. Allport] ‘with the more traditional conceptions of the social that wereoperative in the social sciences’ around 1900 (334). Kurt Lewin’s experimental study of

group phenomena in the 1940s can be seen as an intermezzo: his conception of theory as a


5/10


5

structural model became after his death in 1947 became replaced by Festingers ‘conceptualvariables’.

This tradition was especially st imulated by Leon Festinger: ‘The laboratoryexperiment should be an attempt to create a situation in which the operation of variables will

be clearly seen under special identified and defined conditions. It matters not whether such a

situation would ever be encountered in real life.’ (Festinger, 1953, 139) Festinger’s definitionof a ‘true’ experiment: ‘the investigator creates a situation with the exact conditions he wantsto have and in which he controls some, and manipulates other variables. He is then able toobserve and measure the effect of the manipulation of the independent variables on thedependent variables.’ (1953, 137). Festinger’s preference is for strong manipulations ofvariables, obtaining effects that are comparable with those in real life. To obtain these strongeffects it is a matter of course that the experimenter must use ‘deception, prevarication,misdirection of subjects, and the like.’ (1953, 170).

This reorientation brought social psychology into line with well-respected core areasof experimental psychology and thereby boosted its prestige.

Crisis in social psychology

During the 1970’s laboratory experimentation, which had become the method of choice insocial psychology research publications, was criticized from various perspectives: ethical,methodological and epistemological. It became a contested field, even among experimentalsocial psychologists themselves.

EthicalLong after he abandoned social psychology, Festinger (1980) reflected: ‘ethical issues neverseemed extraordinarily difficult to me […] I do not see the harm in temporarily deceiving

persons in order to study some important question.’

MethodologicalMilgram’s obedience studies constituted ‘an extreme version of this new form ofexperimentation insofar as his experiments were frequently illustrations rather than simpleinductive accounts of hypotheses tested even if, epistemologically, they were onlyillustrations of Milgram’s own cleverness.’(Stam et al, 374)

Ecological validityRobert Zajonc reflected in 1997 that ‘experiments had taken on a life of their own, andresearch had lost contact with everyday life. Events researched in one lab were designed to

explore effects not found in everyday life, but in other labs’ (Brannigan, 2004, 11) Individualism: [the social collapses into interpersonal which then becomes individualized]

These critiques for a large part reflect the self image of experimental social psychologists. Forinstance, Festinger (1953) saw subjects as barely sentient data-producing organisms, thatnevertheless had a seemingly endless ability to destroy the aims of the experiment. This wasechoed by Aronson and Carlsmith (1968), who depicted subjects as the Other – theunknowing and uncaring raw material of the scientific enterprise in social psychology. Aboveall ‘they could not be trusted – all manner of deceptions, manipulations, and checks must be

built into an experiment to guard against this Other. (Stam et al., 375)


6/10


7/10


7

requires humans both as communicators and recipients, while the methodology plays the roleof the rhetoric’ (Strack, 2012). Instead of supporting moralistic pseudo-methodologicalstandards, Strack favours a change in the content and not the rhetoric of the persuasivecommunication. Researchers should not try to please journalists and mass media by creatinga ‘wow!’ but instead focus on the underlying processes, the ‘how’ of psychological

phenomena.Also they should not focus on the exact replication of specific phenomena, but rather

on ‘the demonstration of those universal underlying mechanisms that are responsible for producing them.’ Nobody would expect that the relationship between cognitive contents and behaviour, for instance the elderly stereotype as in Bargh’s experiment, would be the sameover varying contexts. Strack’s main plea is to focus on creating a solid conceptual basis forsocial psychology research (including replications).

More fundamental are the suggestions done by Naomi Ellemers in her paper‘Connecting the dots: Mobilizing theory to reveal the big picture in social psychology (andwhy we should do this)’ (Ellemers, 2013). Like Strack she deplores the frequent publicationof ‘sexy’ attention grabbers and the increase of short papers being published. She is alsocritical about the emphasis on methodological purity (or ‘fetishism’) to counter sloppy scienceand questionable research practices: ‘Adding rules, checks and sanctions is a standardresponse displayed by organizations that try to cope with errors that have been made.However, empirical research has revealed that this is not necessarily the best way to learnfrom previous problems or to improve collective performance.’ (Ellemers, 2013, 3)

Instead, Ellemers advocates to examine whether and how specific findings relate to a broader theoretical perspective. Regaining a focus on the big picture is the only way forward,according to Ellemers. Along with this, she places great value on methodological pluralismand triangulation: examining the same phenomenon from different angles by combiningdifferent research approaches and types of measures: ‘This requires that we more directlyconnect highly rigorous (and for some reductionist) experimentation […] with more ‘messy’and ambiguous methods or field observations (to establish external validity)’. (Ibidem, 4)

Ellemers favours ‘meaningful research’ and suggests that short papers with singlestudy observations do not contribute much to the understanding of real-life concerns, such asthe pervasiveness of discrimination, the development and resolution of intergroup conflict, orthe tendency toward suboptimal decision making. Situations in which basic mechanismsoccur should be examined more closely, with an emphasis on contextual variables that are atstake. Why not make use of earlier grand theories of social psychology: Heider, Festinger,Tajfel or Allport? Or present-day authors such as Daniel Kahneman or Susan Fiske.

Ellemers concludes her paper with the advice to (a) combine different methods andmeasures, (b) writing more overview articles, drawing theoretical lines between research

traditions, and (c) transcend disciplinary boundaries – not just towards the ‘harder sciences’, but also towards the humanities and of course the colleagues within the faculties of socialsciences: sociologists, cultural anthropologists, political scientists and economists.

This reflects an earlier statement by one of the grand old men from the field, RobertZajonc about the lack of cumulativeness in social psychology: If one were to take anytextbook and randomly reshuffle the chapters, it would matter since ‘there is no compellingorder’. So in a century of psychology nothing accumulates. And ‘we have no consensus aboutthe core of our field’s subject matter’. (Brannigan, 2004, 11)

This might have to do with the nature of social psychological experiments. According tosociologist Augustine Brannigan, social psychological experiments are not true experiments

in the sense of the natural sciences. They are not tests in the strict sense designed to compareoutcomes on human subjects in experimental designs based on random assignment to


8/10


8

different treatment groups. Rather they are demonstrations or dramatizations with a pedagogical or moral lesson about everyday life. Social psychological experiments also borrow heavily from commonsense knowledge. According to Brannigan, they ‘simplyreiterate the obvious in abstract methodological form. The experiment gives the sense ofterrific scientific precision in the form of knowledge without actually discovering anything

substantively new.’ (Brannigan, 2004, 20)

Conclusion It seems as if social psychology is facing another crisis. The critique in the 1970’s for a large

part had a philosophical and ethical nature, and was voiced by people from outsidemainstream psychology. This time, it is experimental psychologists from other subdisciplinesand methodologists who conclude that experimental social psychology lacks epistemologicalrigour. At the same time, in the eyes of the public and the media, at least in The Netherlands,social psychology has lost its credibility as problem-solving branch of social science.

Whether the interdisciplinary, multi-method approach that Ellemers advocates willsucceed depends for a large part on the reward structures within the academic world, which inthe end boils down to finance structures. As long as the emphasis is on so-called fundamentalresearch and publication in international high ranking journals, and governments and other

parties are willing to keep on subsidizing this type of research, changes will not be very probable. If on the other hand more emphasis is put on societal relevance of research, social psychologists will be challenged more to make an effort to increase the ecological validity oftheir research – which also means: a broader theoretical and methodological scope.


9/10


9

References

Arkin, R.M. (2011) Most underappreciated. 50 prominent social psychologists describe theirmost unloved work. Oxford: Oxford University Press.

Aronson, E. & J.M. Carlsmith (1968) Experimentation in social psychology, in G. Lindzey &E. Aronson (Eds) The handbook of social psychology, Vol. 2, Reading MA: Addison-Wesley,1-79.

Brannigan, A. (2004) The rise and fall of social psychology. The use and misuse of theexperimental method. New York: Aldine de Gruyter.

Danziger, K. (2000) Making social psychology experimental. A conceptual history, 1920-1970, Journal of the History of the Behavioral Sciences, 36, 329-347.

European Association of Social Psychology (2012) [Comment on Levelt report]… European Journal of Social Psychology, 42,

Ellemers, N. (2013) Connecting the dots: Mobilizing theory to reveal the big picture in social psychology (and why we should do this), European Journal of Social Psychology, 43 (1), 1-8.

Festinger, L. (1953) Laboratory experiments. In: L. Festinger & D. Katz (Eds.) Researchmethods in the behavioral sciences, New York: Holt, 136-172.

Greenwood, J.D. (2004) The disappearance of the social in American social psychology.Cambridge: Cambridge University Press.

Kahneman, D. (2012) Open letter, 26 sept. 2012.

Lange, P.A.M. van, A.W. Kruglanski & E.T. Higgins (Eds.) (2011) Handbook of theories ofsocial psychology, vol 1 & 2. Sage: Thousand Oaks.

Lange, P.A.M. van, A.P. Buunk, N. Ellemers & D.H.J. Wigboldus (2012) Sharpeningscientific policy after Stapel. ASPO: Internal report.

Levelt Committee (2012) Flawed science: The fraudulent research practices of social

psychologist Diederik Stapel. Tilburg: Tilburg University.https://www.commissielevelt.nl/wp-content/uploads_per_blog/commissielevelt/2013/01/finalreportLevelt1.pdf

Morawksi, J. (2012) The importance of history to social psychology. In: A.W. Kruglanski &W. Stroebe (Eds.) (2012) Handbook of the history of social psychology. New York andLondon: Psychology Press, 19-41.

Moscovici, S. & I. Marková (2006) The making of modern social psychology: The hiddenstory of how an international social science was created. Cambridge: Polity Press.

Mummendey, A. (2012) Scientific misconduct in social psychology: Towards a currencyreform in science, European Bulletin of Social Psychology, 24 (1), 4-7.


10/10


10

Parker, I. (1989) The crisis in modern social psychology – and how to end it. Londen;Routledge.

Patnoe, S. (1988) A narrative history of experimental social psychology. The Lewin tradition.

New York: Springer.

Rodrigues, A. & R. Levine (Eds.) (1999) Reflections on 100 years of experimental social psychology. New York: Basic Books.

Stam, H., Radtke, H. and Lubek, I. (2000) Strains in experimental social psychology: Atextual analysis of the development of experimentation in social psychology, Journal of the

History of the Behavioral Sciences, 36, 365–82.

Stapel, D.A. (2000). Moving from fads and fashions to integration: Illustrations fromknowledge accessibility research. European Bulletin of Social Psychology , 12 , 4-27.

Stapel, D.A. & S. Lindenberg (2011) Coping with chaos. How disordered contexts promotestereotyping and discrimination, Science 332 (6026), 251-253.

Strack, F. (2012) The Wow and How of research in social psychology: Causes andconsequences, European Bulletin of Social Psychology, 24 (2), 4–8.

Stroebe, W. (2012) Open letter in DUB, 5 dec. 2012.

Stroebe, W. (2013) Scientific fraud: Lessons to be learnt, European Bulletin of SocialPsychology, 25 (1), 5-12.

Stroebe, W., T. Postmes & R. Spears (2012) Scientific misconduct and the myth of self-correction in science, Perspectives on psychological science, 7, 670-688.

Zajonc, R.B. (1997) One hundred years of rationality assumptions in social psychology. In: A.Rodrigues & R. Levine (Eds.) (1999) Reflections on 100 years of experimental social

psychology. New York: Basic Books, 200-214.

social psychology before and after the s

Documents