social psychology of creativity: a consensual assessment ... · assessment techniques used in some...

Social Psychology of Creativity:A Consensual Assessment Technique

Teresa M. AmabileBrandeis University

Although personality research has made much progress in developing an indi-vidual-difference psychology of creativity, the nature of this phenomenon canonly be fully illuminated if a social psychology of creativity is developed as well.Yet experimental studies of social and environmental influences on creativity areextremely rare. A major obstacle to such research is the criterion problem: thelack of a clear operational definition and an appropriate assessment methodology.For a variety of reasons, both the popular creativity tests and the subjectiveassessment techniques used in some previous creativity studies are ill-suited tosocial psychological studies of creativity. A consensual definition of creativity ispresented and, as a refinement of previous subjective methods, a reliable sub-jective assessment technique based on that definition is described. Several studiestesting the methodology in both artistic and verbal domains are presented, andthe advantages and limitations of this technique are discussed. The present meth-odology can be particularly useful for the development of a social psychology ofcreativity because of the nature of the tasks employed and the nature of thecreativity assessments obtained. In the discussion, creativity assessment is con-sidered as one instance illustrating the more general issue of the divergent aimsand methods of personality psychology and social psychology.

The. past 30 years have seen an enormousincrease in creativity research. Creativity ci-

Thls work was supported in part by Predoctoral Fel-lowship 5F31 MH05232-02 from the National Instituteof Mental Health and by a BRSG SO7044 awarded bythe Biomedical Research Support Grant Program, Di-vision of Research Resources, the National Institutes ofHealth.

The author gratefully acknowledges the help of severalcolleagues, students, and friends who worked on thestudies reported here: Carolyn Amabile, Phyllis Ama-bile, and Julia Steinmetz, who provided invaluable as-sistance in the design and execution of Study 1; ElliotEisner, who offered several suggestions for Study 1;Steven Berglas and Marie Handel, who collaborated onStudy 2; Margaret Stubbs, who collaborated on Study3; Shereen Brackfield, who collaborated on Study 6;Phyllis Goldfarb, who collaborated on Study 7; LisaHerman and Ronit Goldlust who helped conduct Study8; and the principals, teachers, and students of St. Jude'sSchool in Waltham, Massachusetts, and Charles ,E.Cashman School in Amesbury, Massachusetts, for theirinvaluable cooperation in Studies 2 and 3.

In addition, several colleagues offered very helpfulcomments on an earlier draft of this manuscript: Wil-liam DeJong, Maurice Hershenson, Robert Kidd, EllenLanger, Mark Lepper, Leslie McArthur, Ricardo Mor-ant, Harvey Pines, Lee Ross, Mark Snyder, and Mar-garet Stubbs.

Requests for reprints should be sent to Teresa M.Amabile, Department of Psychology, Brandeis Univer-sity, Waltham, Massachusetts, 02254.

tations, which accounted for less than .2% ofthe listings in Psychological Abstracts in1950, made up more than 1% of the citationsin 1980. Despite this proliferation of researchpublications on creativity, however, there hasbeen virtually no work on the social psy-chology of creativity. Between 1975 and1980, there were barely half a dozen articlesin the Journal of Personality and Social Psy-chology and the Journal of Experimental So-cial Psychology that dealt in some way withthe effects of social and environmental fac-tors on creativity. Instead, creativity researchhas been heavily personality oriented; studieson the description and identification of cre-ative personality and cognitive style havepredominated (e.g., Gough, 1979;Lott, 1978;Sobel & Rothenberg, 1980).

Development of a social psychology of cre-ativity could be important for both person-ality and social psychology, as well as for thestudy of creativity. Although a great deal ofresearch has already focused on the creativepersonality, an illumination of the types ofsocial variables that can positively or nega-tively influence creative performance wouldaid in the comprehensive description of cre-

Journal of Personality and Social Psychology, 1982, Vol. 43, No. 5, 997-1013Copyright 1982 by the American Psychological Association, Inc. 0022-3514/82/4305-0997S00.75

997

998 TERESA M. AMABILE

ative personality, its development, and itsmanifestation. Within social psychology, re-searchers have in recent years turned increas-ing attention to an exploration of the effectsof social factors on various aspects of cog-nitive and motor performance (e.g., Condry,1977; Deci, 1971; Diener & Dweck, 1978;Kruglanski, Friedman, & Zeevi, 1971; Lep-per, Greene, & Nisbett, 1973; McGraw &McCullers, 1979; Seta & Hassan, 1980; Wim-peris & Fair, 1979). As one important aspectof performance that has been virtually ig-nored in these investigations, creativity shouldbe integrated into social psychological theo-ries of motivation and cognition. Finally,although cognitive and characterological de-terminants of creativity unquestionably de-serve research attention, theories of the cre-ative process will be incomplete without anaccounting of social-situational determinantsas well.

Methodologically, studies of the social psy-chology of creativity can be grouped into twobroadly denned categories: archival and ex-perimental. An excellent example of the for-mer type is a program of research carried outover the past few years by Simonton (1975,1977a, 1977b, 1979). In these studies* datacollected from archival sources such as his-tories, anthologies, and biographical diction-aries are subjected to transhistorical time-se-ries analyses to test hypotheses about the ef-fect of social variables such as role modelavailability and political instability on cre-ative production. Despite the utility of thisapproach, however, it must be coupled withexperimental techniques to more clearlyidentify the causal influence of social factorson creativity and the mechanism of theiroperation (Simonton, 1975). It is such ex-perimental investigations that are so rare inthe literature of personality and social psy-chology.

Most of the experimental work on cre-ativity has used one of two assessment tech-niques. Some researchers have had judgesrate products (or the persons who producedthem) on scales of creativity. The vast ma-jority, however, have relied upon creativitytests—instruments that are usually similar inform and administration to conventional in-telligence tests. Despite these two rather well-established approaches to the assessment of

creativity, it can be argued that the criterionproblem still presents a major obstacle to suc-cessful and rigorous empirical work in allareas of creativity research and especially inthe social psychology of creativity. The defi-nition and assessment of creativity have longbeen the subject of disagreement and dissat-isfaction among psychologists. This problemhas been dealt with in a variety of ways bythe few researchers who have experimentallystudied social influences on creativity, butnone of them has explicitly addressed theadequacy of conventional assessment meth-ods in any detail.

Creativity Tests

Although several different batteries of cre-ativity tests have been developed, the mostwidely used (and the prototype for manyother tests) are the Torrance Tests of CreativeThinking (TTCT; Torrance, 1962). TheTTCT consist of a number of subtests, mostof which are verbal, although some involvedrawing. The child (or adult, depending onthe form used) is typically presented with oneor several subtests, such as the Unusual Usestest, in which the child is asked to think ofas many different uses as he can for a com-mon object; the Consequences Test, in whichthe child is required to suggest consequencesof several "impossible" events (e.g., peoplebecoming invisible at will); and ImaginativeStories, where the child is asked to write aninteresting and exciting story on a given topic(e.g., the dog that doesn't bark). Responsesto these test items are scored according to thenumber and variety of ideas expressed, aswell as the elaboration of ideas and their sta-tistical infrequency. Most other creativitytests are similar to the TTCT in form, con-tent, administration, and scoring (e.g., Guil-ford, 1967; Wallach & Kogan, 1965).

One conceptual problem with the use ofcreativity tests in empirical studies of creativeperformance is that their use is not clearlytied to an operational definition of creativity.There are, certainly, many definitions of cre-ativity. Although some theorists have ex-pressed the bold hope that an ultimate defini-tion will one day be articulated (Ghiselin,1963), it is generally recogni/ed that a searchfor specific, objectively identifiable features

CREATIVITY ASSESSMENT 999

common to all creative products will not befruitful. Instead, most definitions, while us-ing the creative product as the distinguishingsign of creativity, propose that the generalqualities of novelty and appropriateness dif-ferentiate creative from uncreative products(e.g., Barren, 1955; Bruner, 1962; Newell,Shaw, & Simon, 1962; Stein, 1974). In otherwords, the product or response must be un-usual, statistically infrequent, or completelyunique, and it must also be correct in thecontext of the problem or audience to whichit was addressed. In addition, some theoristsdefine creativity in terms of the response thatthe product produces in an observer. For ex-ample, Bruner (1962) saw the creative prod-uct as anything that produces "effective sur-prise" in the observer, along with a "shockof recognition" that the product or response,although novel, is entirely appropriate.

Despite the existence of such intuitivelyreasonable definitions, however, current as-sessment techniques depending on the cre-ativity tests are not closely linked to them.Indeed, it can be argued that the criterionproblem in creativity research has arisen inlarge part because most definitions do notinclude conceptualizations that are readilytranslated into useful assessment criteria, letalone ultimate criteria. It is not clear, for ex-ample, how one might assess novelty andappropriateness in any general way or howone might determine in any satisfactory waywhether an observer is experiencing "effec-tive surprise" and a "shock of recognition."As a result, the creativity tests and the studiesbuilt around them are operating in a defini-tional void. There is no clear, explicit state-ment of the criteria that conceptually un-derlie the assessment procedures.

A second problem is the basically subjec-tive nature of the purportedly objective scor-ing procedures in the creativity tests. Perfor-mance on these tests is rated according to thetest constructor's or scorer's intuitive assess-ment of what is creative and not accordingto objective criteria of novelty, appropriate-ness, satisfyingness, and so on. Motivatedperhaps by the apparent success of objectivetests of intelligence, creativity researchersmight have been too quick to attempt to ob-jectify the assessment of creativity. Almostcertainly, many creativity tests do measure

abilities that are important for creative per-formance, but it is unclear whether they areuseful for directly assessing something calledcreativity. It will be argued here that suchjudgments can ultimately only be subjective.

For the purposes of developing a socialpsychology of creativity, however, the mostcrucial feature of the creativity tests is thatthey were primarily developed as tools forpersonality research. They were expresslydesigned to be sensitive to individual differ-ences in performance in a wide variety ofdomains—verbal, artistic, scientific, andmathematical (e.g., Guilford, 1956, 1957a,1957b; Torrance, 1962). To the extent thatthey are able to detect subtle individual dif-ferences, however, these tests are problematicfor experimental studies of social and envi-ronmental influences on creativity. Althoughit has been suggested that creativity tests canbe influenced by social and environmentalfactors (e.g., Wallach & Kogan, 1965), thereis abundant evidence that many of these testsdo assess rather stable individual differences(e.g.,Gakhar&Luthra, 1973; Holland, 1968;Torrance, 1972a, 1972b). It is precisely thissuccess in identifying stable traits that ren-ders creativity tests less appropriate for stud-ies of environmental influences (Feldman,1980). Normally, social psychologists seek tocontrol for and, as much as possible, elimi-nate individual-difference or within-groupvariability in the crucial dependent measuresso that those measures can more easily detectthe "signal" of between-group differencesproduced by experimental manipulations(Carlsmith, Ellsworth, & Aronson, 1976).Thus, in developing a social psychology ofcreativity, it would seem unwise to rely uponassessment techniques that were expresslydesigned to reveal consistent individual dif-ferences.

Judgments of Products

The second, and much less common, ap-proach to creativity assessment has been theuse of judges to provide subjective ratings ofthe creativity of products or the persons whoproduced them (e.g., Domino, 1974; Helson& Crutchfield, 1970; Kruglanski et al., 1971;MacKinnon, 1962; Sobel & Rothenberg,1980). In one of the best known of the studies


using this approach, for example, Mac-Kinnon and his colleagues (1962) asked fiveuniversity professors of architecture to nom-inate the 40 most creative architects in theUnited States and to rate the creativity ofeach on a 5-point scale. The nominated ar-chitects were then invited to participate inextensive personality assessments. In otherstudies (e.g., Sobel & Rothenberg, 1980), sub-jects with special expertise or experience ina particular domain are asked to produceworks in that domain during the experimen-tal session; these products are then rated oncreativity by expert judges.

Studies that have employed subjectivejudgment have clearly avoided one of theproblems noted earlier with the creativitytests. In their use of judges' ratings, they pre-sent an assessment of creativity in which thesubjective nature of the measure is direct andunveiled, in contrast to the apparent objec-tivity of the creativity tests. In other respects,however, even these methods present diffi-culties. These studies, too, often suffer froma definitional void by failing to explictly ar-ticulate the definition of creativity guidingthe research or by presenting conceptual defi-nitions (usually including novelty and ap-propriateness as criteria) that are not directlytied to the assessment procedure. In addition,in their assessment procedures, these studiesoften fail to differentiate between the cre-ativity of the products and other constructs,such as technical correctness or aesthetic ap-peal. Moreover, the interjudge reliabilitymight be questioned in studies where the ex-perimenter presents judges with his or herown definition of creativity for them to applyor trains judges beforehand to agree with oneanother (e.g., Eisner, 1965).

Of primary importance for the presentdiscussion, however, are the ways in whichprevious subjective methodologies might beinappropriate for use in social psychologicalresearch. Although these techniques may nothave been specifically designed for person-ality research, those that require judges tomake assessments of a person on the basisof a large body of work over time are likelyto detect relatively stable characteristics andwould, therefore, certainly be better suitedto personality than to social-psychologicalresearch.

Even those assessment procedures thathave judges rate a single product, however,may be too sensitive to large and stable in-dividual differences in performance. In a re-cent conceptualization of the creative process(Amabile, in press), it was proposed that cre-ativity depends on three classes of factors:domain-relevant skills, such as knowledgeabout and talent in the domain, which de-pend on innate abilities and training; cre-ativity-relevant skills, such as cognitive style,work habits, and knowledge of creativity heu-ristics, which depend on personality char-acteristics and training; and task motivation,which depends on individual task preferencesand socially imposed constraints. A social-psychological study might, for example, in-vestigate decrements in creativity resultingfrom a decrease in task motivation by theoffer of a tangible reward. The assessment ofcreativity will be most sensitive to such taskmotivational effects if the influence of do-main-relevant and creativity-relevant skillscan be controlled or eliminated. Thus, to theextent that the task presented to subjectsdraws on special talents or experience-relatedskills—as the tasks in most previous subjec-tive-assessment studies do—the assessmentwill be insensitive to social-psychological ef-fects. Moreover, the high levels of skill re-quired by some tasks employed will limitpossible subject populations to those whohave considerable experience in that domain.

It appears, then, that the criterion problempresents a major obstacle to the investigationof social-environmental effects on creativity.The development of a social psychology ofcreativity that could be integrated with per-sonality and cognitive research might be fa-cilitated by a straightforward operational def-inition of creativity and by an assessmenttechnique amenable to social-psychologicalinquiry. The definition and the methodologydescribed here are presented to that end.

A Consensual Definition of Creativity

The creativity assessment technique de-scribed here is grounded in a consensual defi-nition of creativity—an explicitly opera-tional definition that implicitly underliesmost subjective creativity assessment meth-odologies:


A product or response is creative to the extent that ap-propriate observers independently agree it is creative.Appropriate observers are those familiar with the do-main in which the product was created or the responsearticulated. Thus, creativity can be regarded as the qual-ity of products or responses judged to be creative byappropriate observers, and it can also be regarded as theprocess by which something so judged is produced.

It should be noted, first, that this defini-tion, like most current definitions of creativ-ity, is based on the creative product ratherthan on the process or the person. Not onlyis a clear articulation of the creative processnot yet possible but, more importantly, anyidentification of a thought process as creativemust finally depend on the fruit of that pro-cess—a product or response. Similarly, evena clear specification of the personality traitsthat mark outstandingly creative individualswould have to be validated against theirwork. Thus, a product-centered operationaldefinition is clearly most useful for empiricalresearch in creativity.

Perhaps the most important feature of thisdefinition is its reliance on subjective criteria.In this way, it overcomes the difficulty ofattempting to specify ultimate objective cri-teria for identifying products as creative. In-deed, it can be argued that it is impossibleto articulate such ultimate criteria—that, justas the.judgment of attitude statements asmore or less favorable (Thurstone & Chave,1929) or the identification of individuals asphysically attractive (Walster, Aronson,Abrahams, & Rottman, 1966) depends on asocial context, so too does the judgment ofcreativity. Certainly, there must be particularcharacteristics of attitude statements of per-sons or products that observers systemati-cally look to in rating them on scales of fa-vorability or physical attractiveness or cre-ativity, but, ultimately, the choice of thosecharacteristics is a subjective one. This as-sertion of the necessarily subjective natureof creativity assessment must be emphati-cally underscored, given the strong predom-inance of creativity tests in empirical re-search and the tendency for many observersto regard these tests as pure, objective mea-sures of creativity.

This consensual definition of creativityalso avoids the translation problem notedearlier with traditional definitions. Although

the conceptualizations of novelty and appro-priateness in traditional definitions are un-doubtedly important in formulating theoret-ical models of creative performance (see Amrabile, in press), it is not clear how theseconcepts are to be translated into assessmentoperations. By contrast, the consensual defi-nition conceptually identifies creativity withthe assessment operations. It may indeed bepossible to identify particular objective fea-tures of products that correlate with subjec-tive judgments of creativity or to analyze thenature of subjective correlates of those judg-ments, but this definition makes it unnec-essary to attempt to specify those objectivefeatures or the characteristics of those sub-jective reactions beforehand.

The consensual definition rests on a pairof important assumptions. The first assump-tion is that it is possible to obtain reliablejudgments of the creativity of products, givenan appropriate group of judges. In otherwords, although creativity in a product maybe difficult to characterize in terms of specificfeatures, it is something that people can rec-ognize when they see it. Furthermore, peoplefamiliar with such products can agree withone another on this perception. Second, theassessment methodology that is built uponthe consensual definition assumes that thereare degrees of creativity—that observers canindeed say, at an acceptable level of agree-ment, that some products are more creativeor less creative than others. This assumptionof a continuous underlying dimension of cre-ativity is fairly standard in contemporary the-ory and research on creativity (e.g., Getzels& Jackson, 1962; Guilford, 1959; Nicholls,1972).

A Reliable SubjectiveAssessment Technique

Given the methodological requirements ofa social psychology of creativity and giventhe, consensual definition and the assump-tions about creativity asserted here, there areseveral features required of an appropriatesubjective assessment methodology. First,there are some requirements of the task itself.Because it is desirable for social-psychologi-cal research that there not be large individualdifferences in baseline performances on the


task, it should be one that does not dependheavily on certain specialized skills—such asdrawing ability or verbal facility—that someindividuals have undoubtedly developedmore fully than others. At the same time, thetask should be open-ended enough to permitconsiderable flexibility in responses. It is de-sirable that the task allow for novelty in re-sponse, since the independent variables willbe designed to influence creativity and since,conceptually, creativity has most often beenidentified with novelty of response. In addi-tion, of course, the task must lead to someproduct or clearly observable response thatcan be made available to appropriate judgesfor assessment.

There are also a number of requirementsfor the assessment procedure. First, the judgesshould all have some experience with thedomain in question, although the level ofexperience for all judges need not be iden-tical. Basically, the method requires that alljudges be familiar enough with the domainto have developed, over a period of time,some implicit criteria for creativity, technicalgoodness, and so on. It is important to notethat these judges need not themselves haveproduced work rated as highly creative; it istheir familiarity with the domain that is im-portant.

Second, the judges must make their as-sessments independently. Because the tech-nique relies on subjective judgments, partic-ularly the implicity shared subjective stan-dards of creativity that judges are proposedto have, these judges should not be trainedby the experimenter to agree with one an-other, and they should not be given specificcriteria for judging creativity.

Third, in preliminary work on developingthe technique for a given task, judges shouldbe asked to make assessments on other di-mensions in addition to creativity. Mini-mally, they should make ratings of the tech-nical aspects of the work and, if appropriate,its aesthetic appeal as well. This would thenmake it possible to examine the degree ofrelatedness or independence of those dimen-sions in subjective assessments of the productin question. Assessments of other aspects ofthe work would also allow for an analysis ofeffects of social and environmental factors onthose aspects, making a comparison with cre-

ativity effects possible. This is important be-cause theoretically there might be reasons topredict a differential effect of a given socialfactor on creativity and on technical perfor-mance.

Fourth, judges should be instructed to ratethe products relative to one another on thedimensions in question rather than ratingthem against some absolute standards theymight have for sculpture, poetry, and so on.This is important because for most studiesthe levels of creativity produced by the "or-dinary" subjects who participate will be verylow in comparison with the greatest worksever produced in that domain.

Finally, each judge should view the prod-ucts in a different random order and eachjudge should consider the various dimen-sions of judgment in a different random or-der. If all judgments were made in the sameorder by all judges, high levels of interjudgereliability might reflect method artifacts.

Once the judgments are obtained, ratingson each dimension should be analyzed forinterjudge reliability. In addition, if severalsubjective dimensions of judgment have beenobtained, these should be subjected to a fac-tor analysis to determine the degree of in-dependence between creativity and the otherdimensions. Finally, if the products lendthemselves to a straightforward identificationof specific objective features, these featuresmay be assessed and correlated with creativ-ity judgments.

Given the consensual definition of cre-ativity, the most important criterion for theresults of this assessment procedure is thatthe ratings be reliable. By definition, inter-judge reliability in this method is equivalentto construct validity; if appropriate judgesindependently agree that a given product ishighly creative, then it can and must be ac-cepted as such. In addition, it should be pos-sible to separate subjective judgments of cre-ativity from judgments of technical goodnessand from judgments of aesthetic appeal. Ob-viously, for some domains of endeavor, itmay be relatively difficult to obtain ratingsof aesthetic appeal and technical quality thatare not highly correlated with ratings of cre-ativity. However, it is important to demon-strate that it is at least possible to separatethese dimensions. Otherwise, the discrimi-


nant validity of the measure would be indoubt; judges might be rating something ascreative simply because they like it or be-

cause they find it to be technically well done.Judges' ratings can be used to determine

if the original task presented to subjects wasappropriate for the purposes of a social-psy-chological methodology. Certainly, if virtu-ally all of the subjects in a random sampleof a population are able to do the task andreport no technical difficulty in doing so (i.e.,in manipulating the materials, in finishingwithin a reasonable period of time, andxsoon), this suggests that the task was well-cho-sen for these purposes. If later judging of theproducts reveals a low correlation betweenjudged creativity and experience-relatedcharacteristics of the subjects (e.g., age, ex-perience with the particular type of materi-als), then the task can truly be considered asatisfactory one.1

Application to Artistic Creativity

In a program of research carried out overthe last 5 years, several studies have dem-onstrated that it is possible to develop a re-liable subjective assessment technique appro-priate for social-psychological studies of cre-ativity. Most of these examined artisticcreativity, an area that has often been ne-glected by traditional assessment methods.

Study 1

MethodSubjects and judges. Subjects were 22 girls, aged 7-

11 years, who each participated in one of two groupsessions held in their apartment complex in Palo Alto,California. Because this was the preliminary investiga-tion of this assessment technique, three different groupsof judges were used, varying in expertise with art: (a)psychologist judges—12 members of the Stanford Uni-versity psychology department (faculty and graduate stu-dents), (b) art teachers—21 members of an art educationcourse in the Stanford University School of Education(many of whom were elementary- and secondary-schoolart teachers), (c) artist judges—7 undergraduate andgraduate artists from the art department at StanfordUniversity, each of whom had spent at least 5 yearsworking in studio art.2

Materials. All subjects were given identical sets ofmaterials to work with: over 100 pieces of lightweightpaper in several different sizes, shapes, and colors (allarranged identically for each subject), a container ofglue, and a 15 inch X 20 inch (.38 m X .51 m) piece ofwhite cardboard.

Procedure. The girls were seated at widely spacedplaces at long tables, given their materials, and asked tospend some time making a design for the experimenter.3

The experimenter then demonstrated gluing shapes ontopaper, and the children practiced with some scrap ma-terials. None appeared to have any difficulty in workingwith the materials. Following this, the children were toldthat they could use the materials in any way they wishedto make a design that was "silly." This silliness themewas used as a way to obtain a relatively high baselinelevel of creativity and to reduce one source of vari-ability—the themes children might employ—that couldserve to make the judges' task more difficult. After 18minutes (at which time virtually everyone had finished),the children were asked to stop.

All judges were told that the designs had been madeby children in 18 minutes. Each psychologist judge wasindividually asked to rank the designs from most to leastcreative, using his or her own subjective definition ofcreativity. The art teachers were shown professionallymade slides of the 22 designs and were also asked to usetheir own subjective definitions of creativity in makingassessments. They were asked to assign each design toone of five categories immediately after viewing it: (a)very uncreative (b) rather uncreative (c) undecided (d)rather creative or (e) very creative.

The artist judges were recruited to spend 4 hours inan individual session judging the 22 designs on a varietyof dimensions. As with the other two groups of judges,the artists were allowed to inspect all 22 designs beforemaking any judgment on any one design and were toldthat the designs had been made by children in 18 min-utes. They evaluated each of the 22 designs on 23 dif-

1 Of course, in the extremes, experience-related vari-ables will be important regardless of the particular taskchosen. For this reason, care should be taken in elimi-nating extremes in ability level on the task in question.For example, if a verbal task was being used, it wouldbe inappropriate to include an age range of 5-15 yearsin the population for a given study, even if the task hadbeen chosen to minimize the importance of special ver-bal skills. Likewise, if an artistic task was being used witha population of college students, it would be inappro-priate to include students who had been doing studio artfor several years along with students who have no in-volvement in art, even if the task does not depend in anobvious way upon draftsmanship.

2 It is difficult to provide a clear rule for the numberof judges to be used. By the Spearman-Brown reliabilitycalculation (Nunnally, 1967), the mean judge correla-tion and the number of judges both contribute to theinterjudge reliability. Thus, to the extent that the judgingis a difficult task and the mean interjudge correlation islow, the number of judges should be increased. However,if the mean correlation is high, good reliabilities can beobtained with fewer judges.

3 Of the artistic creativity studies presented here, thisis the only one in which subjects worked in groups. Theywere, however, encouraged by the experimenters to "dotheir own thing" and not to copy or interfere with theother girls. The experimenters were present throughoutthe session and observed that there was very little imi-tation of ideas or techniques.


Table 1Factor Analysis on 23 Dimensions of ArtisticCreativity Judgment: Study 1

Factor loading

DimensionFactor 1:Creativity

Factor 2:TechnicalGoodness

Creativity cluster

CreativityNovel use of materialsNovel ideaEffort evidentVariation in shapesDetailComplexity

.68

.78

.55

.85

.72

.95

.91

-.23-.21-.18

.23-.04

.09-.30

Technical cluster

Technical goodnessOrganizationNeatnessPlanningRepresentationalismSymmetryExpression of meaning

.16-.08-.34

.10

.00-.34-.01

.54

.67

.51

.83

.95

.48

.92

Aesthetic judgment

Liking .22Aesthetic appeal -.04Would you display it? .22

.04

.14

.28

ferent dimensions including creativity, technical good-ness, and aesthetic appeal. A nonrestrictive definition ofeach dimension was provided at the top of its ratingsheet (e.g., "Creativity: Using your own, subjective defi-nition of creativity, the degree to which the design iscreative."). The judges were asked to keep the differentdimensions of judgment as separate from one anotheras possible, and they were instructed to rate the collagesrelative to one another on each dimension rather thanrating them against some absolute standard of art.

The artist-judges were presented with continuousscales for making each judgment. These scales had fiveequally spaced, reference points marked, three of whichwere labeled "high," "medium," and "low." The judgeswere told to make each judgment by placing an X any-where on the scale. They were asked to view the scaleas having equal spacing between the five referencepoints, and they were encouraged to make use of theentire range of the scale. On four of the dimensions(creativity, technical goodness, liking, and silliness), theartist-judges were also asked to assign each design to oneof three categories (high, medium, low) and to rank all22 designs from highest to lowest on the given dimen-sion.

The designs were arranged in a different random orderfor each artist-judge, and the dimensions were judgedin different random orders (e.g., all designs judged onbalance, then all judged on creativity, and so on). The

various judgment tasks were described to judges before-hand, and the judges were given loose time limits to helpthem pace themselves through the judgment tasks.

Results

Interjudge reliabilities (Winer, 1971) werecalculated for each of the three groups ofjudges.4 The reliability of the rankings oncreativity provided by the 12 psychologistjudges was .73; that of the creativity ratingsmade by the 21 art teachers was .88. Inter-judge reliabilities for the seven artist-judges,on the 23 dimensions of judgment were, onthe whole, quite good. The reliability of thecreativity ratings was .77 and that of the tech-nical goodness ratings was .72. In addition,16 of the 23 dimensions had reliabilitiesgreater than .80; the median reliability was.77. There was little systematic differencebetween the reliabilities of the scale ratings,the rankings, and the category groupings forthose four dimensions that were assessed inall three ways. Thus, in subsequent studies,only the scale ratings were used.

The agreement between different groupsof judges on their creativity assessments wasfairly good. The correlation between the psy-chologist-judges' mean creativity ranking foreach design and the artist-judges' mean cre-ativity ranking was .44, p < .05. The correla-tion between the art teachers and the artist-judges was even higher: .65, p < .01.

Several of the dimensions assessed by theartist-judges did, indeed, correlate with theirjudgments of creativity. In addition, how-ever, it appears that the creativity judgmentswere psychologically separate for the judgesfrom their assessments of technical compe-tence and aesthetic appeal. A factor analysisdone on the mean ratings of all 23 dimen-sions of judgment for each design (varimaxrotation) revealed two major factors thatwere essentially orthogonal to one another.

4 The technique of calculating interjudge reliabilitiespresented by Winer (1971) involves an analysis of be-tween-design and within-design variance in ratings, andthe result depends on the number of judges used. TheSpearman-Brown calculation of reliabilities (Nunnally,1967), which also depends on the number of judges,yields results for these analyses that are virtuallyidentical to those obtained by the analysis of variancetechnique. Because the Spearman-Brown calculationsare considerably simpler, they were used in subsequentstudies.


These appear to be a Creativity factor anda Technical Goodness factor. Many of the23 dimensions clustered rather neatly aboutthese two factors; loadings on the factors arepresented in Table 1. It is important to notethat liking for and aesthetic appeal of thecollages loaded low on both of these mainfactors, as did the rated "silliness" of the de-sign.

A number of objective measures weremade on the collages by two independentraters: number of pieces used, number of col-ors used, number of shape categories used(circles, squares, crescents, and so on), num-ber of pieces altered in some way (ripped,folded, and so on), and number of pieces thatoverlapped other pieces, in addition to sev-eral other objective features of the collages.Interrater agreement on these measures wasnearly 100%. Many of the objective featuresdid indeed correlate significantly with theartist-judges' ratings of creativity: number ofcolors, r = .48, p < .05; number of pieces,r = .64, p < .01; shape categories, r = .52,p < .05; pieces altered, r = .37, p < .05; andpieces overlapping, r = .62, p < .01.

Importantly, age of the child did not cor-relate significantly with any of the groups ofjudges' assessments of creativity, although itdid correlate with the rated technical good-ness of the collages, r = .46, p < .05.

Studies 2 and 3Method

These studies also employed children as subjects.Study 2 included 55 boys and 56 girls drawn fromGrades 2-6 at a parochial school in eastern Massachu-

- setts. Study 3 included 24 boys and 23 girls enrolled inGrades 1 and 2 at a nontraditional ''open" school ineastern Massachusetts. In Study 2, the judges were sixundergraduate studio art majors from Brandeis Univer-sity, each with at least 4 years of studio art experience.In Study 3, the judges were seven artists from BrandeisUniversity5 and seven nonartists (psychology graduatestudents, undergraduates, and elementary-school teach-ers from schools other than that in which the study wasconducted). Using the same materials as in Study 1, eachchild in these studies worked individually on making acollage for about 15 minutes. Following the standardprocedures, each judge rated each collage on creativityand technical goodness.

Results

In Study 2, the interjudge reliabilities were.77 for creativity and .72 for technical good-

ness. Again, the correlation between subjectage and rated creativity of the collage wasnonsignificant, r = .12, whereas the correla-tion between subject age and rated technicalgoodness was significant, r = .28, p < .01.There were no significant differences be-tween boys and girls in rated creativity,r(109) = .68.

In Study 3, the interjudge reliability forcreativity judgments was .81 for the artists,.83 for the nonartists, and .89 when all 14judges were considered together. The reli-ability for technical goodness judgments was.72 for the artists, .80 for the nonartists, and.83 for all 14 judges. For creativity judg-ments, the correlation between artists andnonartists was .69, p < .001. Over all judges,the correlation between creativity and tech-nical goodness judgments was .77.

Studies 4, 5, and 66

MethodThe subjects in each of these studies were college un-

dergraduates: 95 women at Stanford University in Study4,10 men and 10 women at Stanford University in Study5, and 40 women at Brandeis University in Study 6.Using the standard materials, each subject was given 15minutes to make a collage that "conveyed a feeling ofsilliness." In addition, in Study 5, half of the male andhalf of the female subjects were asked to "be as creativeas possible." ,

The judges were 15 artists with at least 5 years ofstudio art experience in Study 4, 14 nonartist under-graduate and graduate students in Study 5, and 10 stu-dents working on honors projects in studio art in Study6. All judges rated the collages from their study individ-ually, according to the standard procedure. In Study 4,the judges rated each collage on each of 16 differentartistic dimensions, which had been formed by elimi-nating from the 23 dimensions used in Study 1 thosethat had low interjudge reliabilities or had not clusteredwith the creativity or technical goodness judgments.

Results

The interjudge reliability of the creativityjudgments in Study 4 was .79. In general, thereliabilities of all the subjective judgments inthis study were quite high; 12 of the 16 di-mension reliabilities were above .80, and themedian reliability was .84. Only one dimen-sion of judgment, balance, failed to reach an

5 Some of these judges had also participated in Study 2.6 A more complete report of Study 4 can be found in

Amabile (1979).


acceptable reliability level. As in Study 1, afactor analysis was performed on the dimen-sions of judgment. With three exceptions, the16 dimensions clustered almost exactly asthey had in the earlier analysis. There weretwo nearly orthogonal factors: one composedof creativity, novel material use, novel idea,effort evident, variation of shapes, detail, andcomplexity; and one composed of neatness,organization, planning evident, balance, andexpression of meaning. The asymmetry di-mension (the negative of the symmetry judg-ments) loaded sufficiently low on the cre-ativity factor that it did not cluster with theothers. In addition, representationalismloaded nearly zero on both factors, and thesingle dimension of technical goodness loadedhigh not only on the technical factor but alsoon the creativity factor.

In Study 5, the judges rated the collagesonly on creativity; the reliability of thesejudgments was .93. Instructions to "be cre-ative" had no impact on the rated creativityof the collages (F < 1), but there was a trendtoward a sex difference; females made col-lages that were rated higher in creativity thanthose of the males, F(l, 16) = 4.39, p < .10.There were no significant differences betweenthe ratings assigned by female judges andthose assigned by male judges, t(l9) = 1.47.

Both creativity and technical goodnesswere rated in Study 6, and the reliabilitieswere .93 and .91, respectively. The correla-tion between the two dimensions was .70.

Discussion of Collage Studies

This set of studies on the development ofa reliable subjective method for assessing ar-tistic creativity has demonstrated severalstrengths of the methodology. Of primaryimportance, these studies have shown thatit is possible to obtain high levels of agree-ment in subjective judgments of creativity,even when the judges are working indepen-dently and have not been trained to agree inany way. Thus, according to the consensualdefinition of creativity presented earlier, thiscan be considered a valid measure of cre-ativity in these products.

It appears, however, that the levels of in-terjudge agreement for this task may dependto some degree on the magnitude of effort

required of judges. For artists, the correlationbetween the total number of dimensionsjudged and the mean interjudge correlationof the creativity judgments is —.43. For alljudge groups, this correlation is —.24. Like-wise, for artists, the correlation betweennumber of collages judged at a given timeand mean interjudge correlation of the cre-ativity judgments is -.33. For all judgegroups, this correlation is -.14. This is per-haps not surprising, because in Study 4,where each artist spent approximately 4hours rating 95 collages on each of 16 di-mensions, the mean interjudge correlationwas only .21; by contrast, in Study 6, whereeach artist-judge spent only about ¥2 hourrating 40 collages on 2 dimensions, the meaninterjudge correlation was .57. Possible fac-tors of judge fatigue and difficulty in main-taining consistent criteria throughout thejudging task should be taken into account byincreasing the number of judges used undersuch circumstances.

It is interesting that level of expertise ofthe judges appears not to matter as much asmight have been expected for this task. Overall six studies, there is no clear superiority ofartists over nonartists in interjudge reliabil-ity. The mean interjudge correlation is .35for the former group and .33 for the lattergroup. Moreover, nonartists and artists donot appear to be subjectively defining cre-ativity in very different ways. In those studieswhere both types of judges were used (Studies1 and 3), the degree of agreement betweenthe groups is quite high (.44 and .65 in Study1, .69 in Study 3). This, of course, raises thequestion of who is to be considered an expertfor the purposes of the present methodologyand, thus, who is to be considered an appro-priate judge. It appears that the only require-ment is a familiarity with the domain of en-deavor in which the product was made. Ap-parently, all the groups of judges used inthese collage studies shared some requisiteminimal level of familiarity with collages.Indeed, it is probably true that for some do-mains—judging cartoon captions, for ex-ample—any "ordinary" individuals with anaverage level of exposure to the written me-dia would be appropriate judges. On theother hand, there are other areas—for ex-ample, medical research or atonal music—in


which, to be an appropriate judge, an indi-vidual must have received special training inthe field. Otherwise, the judges' familiaritywith the domain would be in doubt and theirlevel of agreement would almost certainlysuffer.

These studies have also demonstrated thatsubjective judgments of the creativity of art-works can be separated from judgments oftheir aestheic appeal and technical goodness.In Study 1, aesthetic judgments loaded verylow on both the Creativity and the TechnicalGoodness factors in the factor analysis. Inboth Study 1 and Study 4 (the two studiesin which enough dimensions were judged towarrant a factor analysis), there was a clearseparation between the Technical Goodnessand Creativity factors. It must be noted, how-ever, that the single dimension labeled Tech-nical Goodness does not seem to consistentlycapture the technical component of subjec-tive judgments. Although the correlation be-tween the single Creativity and TechnicalGoodness dimensions was only . 13 and .26in Studies 1 and 2, respectively, it was .77,.68, and .70 in Studies 3, 4, and 6, respec-tively. Because the factor analyses did con-sistently yield a good separation between theCreativity and Technical Goodness factory,it may be that a scale labeled something otherthan Technical Goodness would better cap-ture the distinct set of features that includesneatness, organization, planning, and so on,and that ratings on such a scale would bemore consistently distinct from ratings on theCreativity scale.

Although they were independent of thetechnicaland aesthetic judgments, the cre-ativity ratings were correlated with severalother subjective judgments and a number ofobjective features of the collages as well. Aclearly delineated set of artistic dimensionsclustered closely with creativity in a factoranalysis of subjective judgments, and inStudy 1, several physical features of the col-lages correlated with assessments of cre-ativity.

It is of particular significance for a socialpsychology of creativity that it is possible, asdemonstrated in these studies, to find a cre-ativity assessment task that does not dependheavily on the special skills that some indi-viduals may have developed more highly

than others. The collage task was quite man-ageable for both adults and children and,among the children, there was no apparentrelationship between age and creative per-formance. Thus, this type of task could beused in experimental studies of social andenvironmental influences on creativity.

Application to Verbal Creativity

Study 7

MethodSubjects and judges. The subjects were 48 female

students enrolled in an introductory psychology class atBrandeis University. Two groups of judges assessed thefinished products, which were five-line stanzas. Group1 was composed of seven males and three females re-cruited from the English department of Brandeis Uni-versity; all had had at least three years' experience study-ing poetry at an advanced level and writing poetry them-selves. Several were published poets. Group 2 wascomposed of five published poets and five nonpoets wholived in Cambridge, Massachusetts. The five nonpoetswere of approximately the same age and background asthe poets.

Procedure. Each subject was given 20 minutes towork individually on writing a cinquain.7 A cinquain isa simplified form of unrhymed poetry consisting of 5lines: Line 1 is a single noun; Line 2 consists of twoadjectives describing the noun; Line 3 consists of threeverb forms relating to the noun; Line 4 contains anynumber of words (a phrase or sentence about the noun);Line 5 repeats the noun of Line 1. After the initial in-structions, subjects were presented with two examplesof cinquains. All subjects were provided with the firstline of the poem they were to write—the word joy—inan effort to reduce variability and to make the judgingtask somewhat easier.

The judges in Group 1 participated together in onesession. Of course, since the judging task involved read-ing, each judge was able to work completely indepen-dently. Indeed, the judges were not allowed to discussthe poems or the judging task at any time before thesession was completed. They were each given copies ofthe instructions that subjects had received and were toldthat the female undergraduate subjects had had 20 min-utes in which to write their poems. Each judge had copiesof the 38 poems and was allowed to read through all ofthem prior to making any judgments. These judges allrated each poem on creativity, using their "own subjec-tive definition of creativity." The judges in Group 2, alsoparticipating in one group session, rated each of 24poems on 14 different dimensions, including creativity,use of the poetic form, and liking.8 Scales similar to those

7 Pretesting indicated that 20 minutes was ample timefor subjects to finish.

8 Half of the original 48 poems were randomly chosenfor assessment by Group 2. This was done because thetask of rereading each poem for rating on each of the


Table 2Factor Analysis on 14 Dimensions of Judgment for Group 2: Study 7

Factor loading

DimensionFactor 1:Creativity

Factor 2:Style

Factor 3:Technical

ClarityAppropriatenessConsistency

Creativity cluster

CreativityNovelty of word choiceOriginality of ideaSophisticationRhythm

.91

.91

.88

.90

.78

.30

.25

.36

.31

.36

.21

.24

.06

.13

.10

Style cluster

.23

.47

.44

.89

.82

.81

.27

.20

.16

GrammarUse of form

Technical cluster

.19

.14.17.22

.90

.91

used in the collage judging were employed. Indeed, thejudging procedure for both Group 1 and Group 2 wasvirtually identical to that used in the collage studies.

Results

The interjudge reliability of almost all sub-jective judgments was quite high. The reli-ability of creativity judgments was .87 forGroup 1 and .90 for Group 2. For the 14dimensions rated by Group 2, 13 of the re-liabilities were above .70; the median reli-ability was .82. In addition, in Group 2, thecorrelation between the creativity judgmentsof poets and nonpoets was quite high: .80.A factor analysis (varimax rotation) of the14 dimensions rated by Group 2 did not pro-duce as clear a separation as was obtainedwith judgments on the collages. All of thedimensions loaded positively on the twomain factors, and some loaded rather highon both. However, there did seem to be somemeaningful clusters of dimensions (see Table2). Liking judgments did not cluster closely

14 dimensions was quite time consuming. This proce-dure was preferable to one in which each poem was ratedon all 14 dimensions before the next poem was rated onany dimension, however, because the former proceduremade it more likely that each judge would apply his orher subjective criteria for each particular dimensionmore consistently.

with the five creativity dimensions becausethe former loaded fairly high on both theCreativity and the Style factors. However, thecorrelation between judges' creativity ratingsand their liking ratings was .94.

Study 8

Method

Subjects in this study were 40 undergraduate femalesrecruited from an introductory psychology class at Bran-deis University. They wrote cinquains under the sameprocedure as had been used for Study 7. The judges weresix poets living in the Cambridge, Massachusetts area.9

These judges worked individually on the assessmenttasks in their homes; they were mailed copies of the 40poems and were asked to follow the same judging pro-cedures as had been followed in the group session forStudy 7. In addition, they were given written instructionsidentical to those used in the previous study. Each judgerated each of the 40 poems on creativity, clarity ofexpression, and use of poetic form.

Results

Although somewhat lower than the inter-judge reliabilities obtained in Study 7, twoof the three reliabilities here did meet ac-ceptable levels. The reliability of creativityjudgments was .77; poetic form judgments,

' Five of these six judges had served in Study 7.


.91; and clarity judgments, .62. Creativityjudgments correlated —.16 with poetic formjudgments and .38 with clarity judgments;both correlations are nonsignificant.

Discussion of Poetry Studies

This preliminary attempt to extend theproposed assessment technique to the verbaldomain was successful on many counts.First, and most important, subjective judg-ments of creativity of the poems were shownto have high interjudge reliability. In addi-tion, although the separation was not quiteas clear as it had been with the artistic judg-ments in Studies 1 and 4, there was a sepa-ration between the Creativity factor andother factors rated in a factor analysis of thesubjective judgments in Study 7. As notedearlier, although ratings of liking for thepoem did not cluster closely with the cre-ativity dimensions on that factor analysis, thesingle liking dimension was highly correlatedwith the single creativity dimension. Thus,it appears that although it may be possiblefor some types of products to obtain creativ-ity judgments that are clearly uncontami-nated by assessments of liking, creativityjudgments for other types of products maybe more tightly bound up with assessmentsof aesthetic appeal.

As with the collages, it appears that a veryhigh level of judge expertise is not requiredto ensure a high level of reliability in ratingsof the cinquains. The level of agreement increativity ratings between nonpoets and pub-lished poets in Group 2 of Study 7 was .80.This high level of agreement might havearisen because the cinquain is so simple andbecause most educated individuals in ourculture are familiar with simple poetic forms.As Nnoted earlier, higher levels of expertisewould probably be required to ensure reli-ability of judgments of very complex prod-ucts or products that would not be generallyfamiliar.

It is also important to note that, like thecollage-making task, the cinquain task ap-pears not to depend heavily on special skills.None of the subjects in either study expressedany difficulty in writing the poem or in com-pleting it easily in the time allotted. In ad-dition, these studies demonstrated that this

assessment technique can be adapted for verydifferent kinds of tasks. The collage makingis completely nonverbal, involving the ma-nipulation of paper and glue. In contrast, thetask of writing a cinquain requires the com-position of three original lines. It seems prob-able, therefore, that this methodology couldbe successfully employed for a variety oftasks in a wide range of domains and that itcould be generally useful in experimentalstudies on the social psychology of creativity.

General Discussion

Contrast to Previous Methods

In its basic approach, the methodologydescribed here is the conceptual reverse ofthat adopted by traditional objective creativ-ity tests. In those tests, component tasks andsubtasks are scored to yield a global assess-ment of an individual's creativity, an assess-ment that is ultimately based on the subjec-tive judgment of the psychometricians whodevised the subtasks or the raters who scorethem. Instead, the present technique beginswith a global, explicitly subjective assessmentof creativity. This global judgment is thenclearly demonstrated to be a reliable one.Once this is done, the judgment of creativitycan tentatively be broken down into com-ponent parts; it can then be examined to de-termine which other subjective judgmentsand, perhaps, which objective features of theproduct predict this judgment of creativity.As the studies presented here demonstrate,some progress toward this goal has alreadybeen made.

Moreover, the subjective assessment tech-niqu6 proposed here appears to have ecolog-ical validity as a measure of creativity. Usingmaterials that allow for considerable flexi-bility in response, subjects actually createsomething—a collage or a poem, for exam-ple—that "real world" creators might make.Then, using an approach that has clear prec-edents in social-psychological research (e.g.,Thurstone & Chave, 1929; Walster et al.1966), reliable subjective judgments of theproducts are obtained from appropriate ob-servers. Thus, not only does the task itselfmimic real-world performance but the as-sessment technique mimics real-world eval-uations of creative work.


The present methodology can be consid-ered a refinement of the subjective assess-ment techniques used in previous social-psy-chological and personality studies (e.g.,Domino, 1974; Helson &Crutchfield, 1970;Kruglanski et al, 1971; MacKinnon, 1962;Sobel & Rothenberg, 1980). In comparisonwith those techniques, the present method-ology can be more generally useful for social-psychological studies for two reasons. First,it calls for tasks that are structured to be rel-atively independent of skills such as writingor drawing ability. Second, in contrast tomany subjective assessment studies in whichjudges make global ratings of a person's cre-ativity, in this technique judges make ratingsof specific products. In addition, the reliabil-ity of creativity judgments across severalgroups of appropriate observers has beenfirmly established in the present method, inthe absence of any attempts to train judgesto agree. Finally, and perhaps most impor-tantly, the existence of a unique subjectiveconstruct called creativity has been demon-strated. Although judges were not providedwith a definition of creativity, but were in-stead asked to use their own subjective defini-tions, they consistently and reliably identifieda quality in both types of products that wasdistinct from technical execution. Moreover,for artworks, it was distinct from aestheticappeal as well.

It is important that this assessment meth-odology has been used in several studies in-vestigating substantive questions about cre-ativity. In particular, the methodology hasbeen successfully applied in studies of task-motivational influences on creativity. In onesuch investigation (Amabile, 1979), subjectsmaking collages either did or did not expectthat their work would be evaluated by ex-perts. Those expecting external evaluationnot only expressed a lower level of intrinsicinterest in the activity but also produced col-lages that were significantly lower on judgedcreativity than those produced by noneval-uation subjects. Another study (Goldfarb,Note 1) replicated this effect on the creativityof cinquains.10 Thus, the construct validityof this subjective method of creativity as-sessment has been reinforced; the methodprovides measures of a construct that be-haves as creativity was predicted to behaveon the basis of theoretical derivations.

Limitations of the Methodology

Despite its advantages, the present tech-nique of creativity assessment is not pro-posed as an ultimate and universally usefulmethodology. Indeed, it has some specificlimitations. First, if practical considerationsare primary, this method is decidedly im-practical in the short run. Choosing an ap-propriate task and an appropriate judge pop-ulation, having several individuals judge theproducts (sometimes on several dimensions),and doing statistical analyses are all time con-suming.

In addition, to the extent that the task cho-sen is one that does not depend heavily onparticular abilities and experience-relatedskills, the approach is probably not a usefulone for identifying a wide range of enduringindividual differences in creativity. However,because tasks can be chosen with this end inmind (i.e., with the inclusion of an appro-priate range of skills to be tapped), with mod-ification this method might prove to be a rea-sonable one even for such individual-differ-ence studies.

Also, it may be difficult to apply this as-sessment technique to products that are atthe frontiers of a particular domain of en-deavor. Consider, for example, revolutionarytheories in science or revolutionary works ofart. It would be difficult to apply this methodto assess the creativity of these products be-cause it is precisely their revolutionary naturethat makes it difficult for people, even sup-posed experts in their fields, to agree on thelevel of creativity evident. In fact, this prob-lem could be considered within the contextof the familiarity criterion proposed earlier.These products are so different that no oneis sufficiently familiar with the domain (per-haps because the products create their ownnew domain) to serve as an appropriatejudge.

Finally, a related caveat is that the reli-ability—and hence the validity—of the judg-ments obtained by this method are neces-sarily limited by historical time and place. Itis doubtful that a group of Italian Renais-

10 In addition, although their subjective assessmentmethodology was considerably less complex than thatoutlined here, Kruglanski et al. (1971) found negativeeffects of expected reward on the rated creativity of sto-ries and titles.


sance painters would agree to a great degreewith a group of contemporary American ar-tists in their creativity judgments of any givengroup of artworks from any era. Clearly, theshared subjective criteria of creativity in anydomain of endeavor do change over time anddo differ across cultures. This need not nec-essarily be considered a limitation, however.Many previous theorists have realized thatthe judgment of creativity is historically andculturally bound," and it seems unreasona-ble to expect that universal and enduring cri-teria—even subjective criteria—could everbe agreed upon.

Future Directions

Although the preliminary studies reportedearlier lay the groundwork for the assessmentmethodology proposed here, further refine-ment and extension of the technique are re-quired in a number of areas. First, attemptsshould be made to extend the subjective as-sessment methodology to other domains ofendeavor and to other types of tasks withinthe artistic and verbal domains already ex-plored. Second, a finer identification of ob-jective and subjective correlates of creativityjudgments may be possible. Third, the limitsof interjudge reliability should be explored,especially where the products being judgedare in relatively new domains or representtruly pioneering work. Fourth, since the re-sults presented earlier suggested that certaincharacteristics of the judging task—such asthe number of products or dimensions to bejudged—can influence interjudge agreement,it is important to determine what other fea-tures of the judging task might also influencereliability. Fifth, more work is needed on theidentification of appropriate judges for par-ticular types of products and on the influenceof judge characteristics on interjudge reli-ability. Sixth, because the studies presentedhere showed reasonable but not perfect sep-aration between creativity and other dimen-sions of subjective judgment, it is importantto determine under what circumstances andfor what types of products and domains ofendeavor creativity judgments will and willnot be clearly separated from assessments ofthe technical and aesthetic aspects of prod-ucts. Finally, attempts should be made todetermine whether this subjective assessment

technique could be usefully adapted not onlyfor the social psychology of creativity but alsofor the identification of individual differencesin creativity.

The arguments presented here on the mer-its of various creativity-assessment tech-niques for different types of psychological in-quiries raises a more general point about theaims and methods of personality and socialpsychology. Most simply stated, it is the pur-pose of personality psychology to identifystable characteristics and abilities and tostudy the ways in which they are related toother characteristics and abilities. Thus, per-sonality researchers generally look for con-sistency of behavior over time and across sit-uations and for evidence of a given trait ina variety of behavioral domains. For thesepurposes, measures that are particularly sen-sitive to stable baseline individual differencesare most useful. On the other hand, mostsimply stated, it is the purpose of social psy-chology to investigate the impact of socialand environmental factors on most peopleor on the "average" person. Thus, social psy-chologists are less concerned with within-group variability (and in fact seek to reduceit where possible) and more interested in sit-uationally induced between-group differ-ences in behavior. Such differences, althoughthey may be large and important ones, areusually of relatively brief duration whencompared to the enduring individual differ-ences that personality researchers seek toidentify. For their purposes, then, social psy-chologists require measures that are not par-ticularly sensitive to baseline individual dif-ferences but do allow room for variability inresponse. This difference between appropri-ate methods applies not only to creativityassessment but also to measurement in awide range of other behavioral domains thatare of interest to both social and personalitypsychology.

Rather than proposing a new and easilyapplicable creativity test, the consensual as-sessment technique instead presents a some-what time-consuming general methodologythat can produce clear and reliable subjective

" For example, Stein (i975, p. 253) defines creativityas "a process that results in a novel product or ideawhich is accepted as useful, tenable, or satisfying by asignificant group of others at some point in time."


judgments of creativity. In the long run,though, more than any single creativity test,this methodological approach may facilitatethe inclusion of creativity in a comprehensivetheory of social-psychological influences onperformance and the inclusion of social psy-chological influences in a comprehensive the-ory of creativity.

Reference Note1. Goldfarb, P. The effects of evaluation expectation and

coaction on verbal creativity. Unpublished manu-script, Brandeis University, 1978. (Available from T.Amabile, Department of Psychology, Brandeis Uni-versity, Waltham, Massachusetts 02254):

References

Amabile, T. M. Effects of external evaluation on artisticcreativity. Journal of Personality and Social Psychol-ogy, 1979, 37, 221-233.

Amabile, T. M. The social psychology of creativity: Acomponential conceptualization. Journal of Person-ality and Social Psychology, in press.

Barren, F. The disposition toward originality. Journalof Abnormal and Social Psychology, 1955, 51, 478-485.

Bruner, J. The conditions of creativity. In H. Gruber,G. Terrell, & M. Wertheimer (Eds.), Contemporaryapproaches to creative thinking. New York: AthertonPress, 1962.

Carlsmith, J., Ellsworth, P., & Aronson, E. Methods ofresearch in social psychology. Reading, Mass.: Addi-son-Wesley, 1976.

Condry, J. Enemies of exploration: Self-initiated versusother-initiated learning. Journal of Personality andSocial Psychology, 1977, 35, 459-477.

Deci, E. Effects of externally mediated rewards on in-trinsic motivation. Journal of Personality and SocialPsychology, 1971, 18, 105-115.

Diener, C, & Dweck, C. An analysis of learned help-lessness: Continuous changes in performance, strat-egy, and achievement cognitions following failure.Journal of Personality and Social Psychology, 1978,36, 451-462.

Domino, G. Assessment of cinematographic creativity.Journal of Personality and Social Psychology, 1974,30, 150-154.

Eisner, E. A typology of creative behavior in the visualarts. American Educational Research Journal, 1965,2, 125-136.

Feldman, D. Beyond universals in cognitive develop-ment. Norwood, N.J.: Ablex, 1980.

Gakhar, S., & Luthra, S. The test-retest reliability ofTorrance Tests of Verbal Creative Thinking in a sam-ple of ninth and tenth grade Indian children. Journalof the Indian Academy of Applied Psychology, 1973,10, 48-52.

Getzels, J., & Jackson, P. Creativity and intelligence:Exploration with gifted students. New York: Wiley,1962.

Ghiselin, B. Ultimate criteria for two levels of creativity.In C. Taylor & F. Barren (Eds.), Scientific creativity:Its recognition and development. New York: Wiley1963.

Gough, H. A creative personality scale for the adjectivecheck list. Journal of Personality and Social Psychol-ogy, 1979, 37, 1398-1407.

Guilford, J. P. The structure of intellect. PsychologicalBulletin, 1956, 53, 267-293.

Guilford, J. P. A revised structure of intellect. Report ofthe Psychological Laboratory, University of SouthernCalifornia, No. 19, 1957. (a)

Guilford, J. P. Creative abilities in the arts. PsychologicalReview, 1957, 64, 110-118. (b)

Guilford, J. P. Traits of creativity. In H. Anderson (Ed.),Creativity and its cultivation. New York: Harper,1959.

Guilford, J. The nature of human intelligence. NewYork: McGraw-Hill, 1967.

Helson, R., & Crutchfield, R. Mathematicians: The cre-ative researcher and the average PhD. Journal of Con-sulting and Clinical Psychology, 1970, 34, 250-257.

Holland, J. Test reviews. Journal of Counseling Psy-chology, 1968, 15, 297-298.

Kruglanski, A., Friedman, E., & Zeevi, G. The effectsof extrinsic incentives on some qualitative aspects oftask performance. Journal of Personality, 1971, 39,606-617.

Lepper, M., Greene, D., & Nisbett, R. Underminingchildren's intrinsic interest with extrinsic reward: Atest of the "overjustiflcation" hypothesis. Journal ofPersonality and Social Psychology, 1973, 28, 129-137.

Lott, B. Behavioral concordance with sex role ideologyrelated to play areas, creativity, and parental sex typ-ing of children. Journal of Personality and Social Psy-chology, 1978, 36, 1087-1100.

MacKinnon, D. The nature and nurture of creative tal-ent. American Psychologist, 1962, 17, 484-495.

McGraw, K.., & McCullers, J. Evidence of a detrimentaleffect of extrinsic incentives on breaking a mental set.Journal of Experimental Social Psychology, 1979,15,285-294.

Newell, A., Shaw, J., & Simon, H. The processes of cre-ative thinking. In H. Gruber, G. Terrell, & M. Werth-eimer (Eds.), Contemporary approaches to creativethinking. New York: Atherton Press, 1962.

Nicholls, J. Creativity in the person who will never pro-duce anything original and useful: The concept ofcreativity as a normally distributed trait. AmericanPsychologist, 1972, 27, 717-727.

Nunnally, J. Psychometric theory. New York: McGraw-Hill, 1967.

Seta, J., & Hassan, R. Awareness of prior success orfailure: A critical factor in task performance. Journalof Personality and Social Psychology, 1980, 39,70-76.

Simonton, D. K. Sociocultural context of individual cre-ativity: A transhistorical time-series analysis. Journalof Personality and Social Psychology, 1975, 32, 1119-1133.

Simonton, D. K. Creative productivity, age, and stress:A biographical time-series analysis of 10 classical com-posers. Journal of Personality and Social Psychology,1977, 55,791-804. (a)


Simonton, D. K. Eminence, creativity, and geographicmarginality: A recursive structural equation model.Journal of Personality and Social Psychology, 1977,35, 805-816. (b)

Simonton, D. K. Multiple discovery and invention:Zeitgeist, genius, or chance? Journal of Personalityand Social Psychology, 1979,^7, 1603-1616,

Sobel, R., & Rothenberg, A. Artistic creation as stim-ulated by superimposed versus separated visual im-ages. Journal of Personality and Social Psychology,198,0, 39, 953-960.

Stein, M. Stimulating creativity (Vol. 1). New York:Academic Press, 1974.

Stein, M. Stimulating creativity (Vol. 2). New York:Academic Press, 1975.

Thurstone, L., & Chave, E. The measurement of attitude.Chicago: University of Illinois Press, 1929.

Torrance, E. P. Guiding creative talent. EnglewoodCliffs, N.J.: Prentice-Hall, 1962.

Torrance, E. P. Career patterns and peak creative

achievements of creative high school students twelveyears later. Gifted Child Quarterly, 1972, 16,75-88. (a)

Torrance, E. P. Predictive validity of the Torrance Testsof Creative Thinking. Journal of Creative Behavior,1972, 6, 236-252. (b)

Wallach, M., & Kogan, N. Modes of thinking in youngchildren. New York: Holt, Rinehart & Winston, 1965.

Walster, E., Aronson, V., Abrahams, D., & Rottman,L. Importance of physical attractiveness in dating be-havior. Journal of Personality and Social Psychology,1966, 4, 508-516.

Wimperis, B,, & Farr, J. The effects of task content andreward contingency upon task performance and sat-isfaction. Journal of Applied Social Psychology, 1979,9, 229-249.

Winer, B. Statistical principles in experimental design.New York: McGraw-Hill, 1971.

Received August 28, 1981 •

Instructions to Authors

APA policy prohibits an author from simultaneously submitting the same manuscript totwo or more journals. Authors should prepare manuscripts according to the PublicationManual of the American Psychological ^Association (2nd edition) and the "Guidelines forNonsexist Language in APA Journals" (Publication Manual Change Sheet 2, American Psy-chologist, June 1977, pp. 487-494). Manuscripts must include an abstract of 100-175 words.Instructions on tables, figures, references, metrics, and typing (all copy must be double-spaced)appear in the Manual. Articles not prepared according to the Manual will not be reviewed.For further information on content, authors should refer to the editorial in the March 1979issue of this journal (Vol. 37, No. 3, pp. 468-469).

The reference citation for any article in any JPSP section follows APA's standard referencestyle for journal articles; that is, authors, article title, journal title, year of publication, volumenumber, and page numbers. (See Example 1, p. 122 of the Publication Manual, 2nd ed.) Thecitation does not include the section title.

Manuscripts should be submitted in quadruplicate (an original and 3 copies) to the ap-propriate section editor. All copies should be clear, readable, and on paper of good quality.Editors' addresses appear on the inside front cover of the journal. Authors should keep a copyof the manuscript to guard against loss.

Blind review is optional and must be specifically requested when a manuscript is firstsubmitted. In those cases, authors' names and affiliations must appear only on a separatetitle page, included with each copy of the manuscript. Footnotes containing the identity ofauthors or their affiliations must be on a separate sheet.

Section editors reserve the right to redirect papers among themselves as appropriate unlessan author specifically requests otherwise. Rejection by one section editor is considered re-jection by all, therefore a manuscript rejected by one section editor should not be submittedto another.

social psychology of creativity: a consensual assessment ... · assessment techniques used in some...

Documents