an empirical examination of the value of creativity ... · pdf filevalue of creativity support...

Creativity Support Systems

An EmpiricalExamination of theValue of CreativitySupport Systems onIdea Generation

By: Brenda MassettiSt. John’s University300 Howard AvenueStaten Island, NY 10301U.S.A.Massetti @sjuvm.stjohns.edu

Abstract

Because organizations seek more innovativeways to compete, the ability of their employeesto generate new and valuable ideas becomes afundamental survival skill. To the extent thatcomputer software might enhance the creativeperformance of individual users, organizationsmight ultimately apply such tools to enhance thecreative performance of their employees. A con-trolled laboratqry experiment was performed todetermine whether two popular creativity sup-port applications significantly enhanced the cre-ative performance of individual users. Theresults suggest that responses generated withsoftware support are significantly more noveland valuable than responses generated by penand paper. The results also question the previ-ous creativity research practice of not directlycontrolling for idea fluency prior to experimentalmanipulation. It is hoped the findings from thisinvestigation can be used to improve individualcreative performance, further research concern-ing factors relevant to creativity, and guidefuture ICSS development efforts.

Keywords: DSS, software quality, brainstorm-ing, software packages, interface charac-teristics, user satisfaction

ISRL Categories: AF0802, CB09, EI0206.01,EI0207, HA03, HD01

IntroductionTo respond effectively in today’s quickly chang-ing, highly complex business environment,management must depend on organizationalmembers’ mental capacities to generate newand meaningful ideas (Beckett, 1992;Herrmann,1993; Johnson, 1992; Kanter, 1982).Consequently, creativity has evolved into a fun-damental organizational resource useful inestablishing and maintaining competitive advan-tage (Coulson and Strickland, 1991; Everett,1983; Gillam, 1993; Kiechel, 1983).Organizations such as Microsoft and MinnesotaMining and Manufacturing, for example, claimthat cultivating creativity within their membershas led to innovations otherwise not possible(LaBarre, 1994; Morgan, 1993). Moreover,between 1988 and 1992, the number of firmsoffering creativity training programs hasincreased eightfold to 33 percent (Thierauf,1993). With industry leaders such asInternational Business Machines, Banc One,and Exxon Corporation regularly expending cor-porate dollars to nurture the creative spirit oftheir members, techniques aimed at enhancingcreativity are flourishing (Couger, 1995; deBono, 1993; Evans, 1991).

One relatively new set of tools intended to aug-ment the creative process is Creativity SupportSystems (CSS) (Abraham and Boone, 1994).These computer-based tools are generallyaimed at enhancing boundary-breaking, insight-ful thought during problem solving (Evans,1991; VanGundy, 1992; Winship, 1991). Forexample, some CSSs provide open-endedquestion-and-answer options for generatingnew points of view, while others provide morefocused structures for exploring ideas (Marakasand Elam, forthcoming). In addition, some pack-ages are designed to support individuals, whileothers are intended for group-oriented use(Dayton, 1991; Young, 1989).

Although the popularity of CSSs for individualuse appears to be growing, only a few con-

MIS Quarterly~March 1996 83


trolled examinations exploring their value havebeen performed (Rouse, 1989; Winship, 1991).For example, Proctor (1989) found that of 170subjects using Individual-Level CreativitySupport Systems (ICSS), over 50 percentclaimed to have generated at least one usefulinsight not considered prior to using the ICSS.However, Proctor’s investigation also found thata significant number of the subjects Were notenthusiastic about the software. In addition,reseamh by Watson (1989) and Roberts (1989)determined that students using an ICSS wereable to generate more ideas faster than stu-dents brainstorming without software. Whilethese findings have merit given the notion that anovel idea is more likely to emerge from manyrather than fewer ideas, neither study directlyconsidered the quality of ideas generated.

Notwithstanding the lack of clear empirical evi-dence concerning the value of an ICSS, there isalso little theoretical justification (Abraham andBoone, 1994; Elam and Mead, 1990). EachICSS appears to provide a different methodolo-gy for enhancing creativity with little more thananecdotal reasoning to justify the approach(Cohen, 1991; Young, 1989). However, if ICSS were to directly enhance creative perfor-mance, the organizational benefits could bemultifaceted. For example, organizationalmembers could use the ICSS for reinforcingtechniques learned in formal creativity training.Or, by matching ICSS tools to specific decisionneeds, the ICSS might enable management tobetter control creative performance. An organi-zational member unable to think in alternativeways on a particular issue might use the patternswitching or association tools of an ICSS torespond more variably.

To better understand the value of using anICSS, this paper proposes a theoretical modelreflecting a variety of factors believed directlyrelevant to an individual’s creative performance.Hypotheses concerning the effects of ICSSs oncreative performance are derived and investi-gated using a controlled laboratory experiment.

Theoretical foundation of the ICSS

Formally, an individual has performed creativelyto the extent that the response produced is notonly novel, but also meaningful and valuable fora given situation (de Bono, 1993; VanGundy,1982). Conceptually, the extent to which oneperforms creatively is influenced by a variety ofsocial, cultural, and historical factors (Ackoffand Vergara, 1981; Albert, 1983; Amabile andTighe, 1993; Couger, 1995; Tardif andSternberg, 1988). Within any given environ-ment, however, additional factors such as indi-vidual ability, the nature of the decision task, theamount of training, and the technology usedcould also affect creative performance (Elamand Mead, 1990; Evans, 1991; Torrance, 1988;Young, 1989). A graphical depiction of theseproposed relationships can be found inFigure 1.

Qualities relevant to an individual’s creativeability include physiological characteristics ofthe brain (Restak, 1993); distinctive cognitivecharacteristics (Guilford, 1968); and specialmotivational considerations (Csikszentmihaly,1988; Haines and Amabile, 1988). Generally,an individual’s natural creative potential is bio-logically determined and established early inlife. It is thought to directly affect creative perfor-mance and is not expected to vary significantlyover time (Amabile, 1991; Cox, 1983; Torrance,1988).

Through training, however, an individual’s cre-ative performance can be amplified or inhibited(de Bono, 1993; VanGundy, 1982; Walberg,1988;). Creativity training represents the individ-ual’s past knowledge and developmental historyconcerning his/her creative behavior (Couger,1995; Finke, et al., 1992; Jacobs, 1989).Depending upon the individual’s backgroundand training with respect to decision making,creative behavior may become more or lesspronounced over time (Tardif and Sternberg,1988; Walberg, 1988).

The nature of the decision task is another factorthat affects creative performance (Elam andMead, 1990; VanGundy, 1992). Categorically,creative responses result from two types ofmental processes: generative and exploratory(Finke, et al., 1992). Within the generative

84 MIS Quarter/v/March 1996


CreativityTraining

IndividualCreativeAbility

C reativePerformance

DecisionTask

ICSSTechnology

Figure 1. Theoritical Model of the Relationship Between IndividualCreativity Support Systems and Creative Performance

mode, divergent ways of thinking, includingremote association and pattern switching, pro-duce novel, unique concepts (Ackoff andVergara, 1981; de Bono, 1993; Young, 1989).In the exploratory mode, convergent thoughtsuch as elaboration or successive refinementreformulates a unique concept into a meaning-ful and valuable response (Ackoff and Vergara,1981; Guilford, 1968; VanGundy, 1982). Whileboth processes must occur for an individual toperform creatively (Guilford, 1968; Tardif andSternberg, 1988), the nature of the decisiontask defines which mode is likely to dominateresponse formation (Finke, et al., 1992).Depending on whether the stated task is aimedat the generative or exploratory mode, theresponse produced would tend to be morenovel or useful.

1991; Young, 1989). Theoretically, they influ-ence performance by providing the necessarysuggestions and cues for an individual to pro-duce a creative response (Robbin, 1990).Moreover, these tools are believed to ward offdebilitating distractions by focusing the individu-al’s thoughts toward creativity (Cohen, 1991;Finn, 1993). However, while arguments suggestICSSs ultimately enhance creative perfor-mance, it is also possible they have no impactor a negative impact on creativity. For example,operating a computer may make an individualso emotionally uncomfortable that his/her abilityto think in any fashion is limited.

Hypotheses formation

ICSSs are also thought to directly contribute toan individual’s creative performance (Evans,

Because creative performance has previouslybeen operationalized in a variety of ways, it is

MIS Quarterly~March 1996 85

Crea~vity Support Systems

useful to consider the expected impacts ofICSSs from multiple perspectives. One way isto consider the effect of ICSSs on idea fluency(Diehl and Stroebe, 1987; Young, 1989)o Ideafluency refers to an individual’s ability to gener-ate a number of different ideas in response to aspecific decision task (Guilford, 1968; Saletta,1978). It is typically considered an objective,quantitative measure of creative performanceand is widely used as a dependent variable inbrainstorming research (Ackoff.and Vergara,1981; Couger, 1995; Roberts, 1989; Watson,1989). Since ICSSs are specifically designed toprovide cues that heighten an individual’s cre-ative performance, and idea fluency is consid-ered an aspect of that performance, the follow-ing hypothesis is suggested.

HI: Use of ICSS technology will result in agreater number of ideas being pro-duced for a given decision task thanuse of conventional software supportor no software support.

Beyond providing mental triggers for quantita-tively enhancing creative thought, an ICSSshould also be capable of qualitatively improv-ing creative performance. Not only would oneexpect more ideas to be produced from usingan ICSS, but one would also expect these ideasto be more original and valuable than ideas pro-duced by some other means. A formal state-ment reflecting this notion follows,

H2: Use of ICSS technology will result inmore creative ideas being producedfor a given decision task than use ofconventional software support or nosoftware support.

To the extent that a computer program can sup-port creative thought, it theoretically has thepotential to enhance creative performance in aprecise way. Specifically, an ICSS focused ongenerative thinking (e.g., viewing a situation in completely different way) should be able tospawn more novel responses than an ICSSfocused on exploratory thinking’ (e.g., succes-sively refining an established perspective). And,an ICSS that emphasizes exploratory thoughtover generative should produce creative outputscontaining more situational value than novelty.

Hypotheses to reflect these implied relationshipsfollow.

H3: Use of a generative ICSS applicationwill produce ideas for a given decisiontask that are more novel than thoseproduced by an exploratory ICSS, con-ventional software support, or no soft-ware support.

H4: Use of an exploratory ICSS applicationwill produce ideas for a given decisiontask that are more valuable than thoseproduced by a generative ICSS, con-ventional software support, or no soft-ware support.

Methodology

The experiment entailed a 1 x 4 design wheresubjects completed the same task using one offour treatments: generative IOS$, exploratoryICSS, conventional software, and no software.Creativity training was held constant, and indi-vidual ability was monitored through testingbefore and after experimentation. The subjects’task responses were rated on creative merit byexperts in the task domain. Details concerningsubjects, software, variables, and proceduresare provided in the following paragraphs.

Subjects

The subjects were 44 MBA students taking agraduate course in management informationsystems at a large metropolitan university. Allwere employed in a full- or part-time capacity.They volunteered to participate and were ran-domly assigned to treatments after receivingcreativity training. None were familiar with anyof the software applications used prior to receiv-ing their respective treatments, But, all alreadyknew how to operate a computer for word pro-cessing and spreadsheeting purposes.

86 MIS Quarterly~March 1996


Software selection

There were three different types of software usedin the experiment. One, IdeaFisher, was selectedto represent an ICSS with a generative focus.This software contained an idea bank with over705,000 possible associations of topics, phrases,and words (Robbin, 1990). Using the idea bank,an individual could support the processes ofdivergent thinking and remote association bybrowsing through and pondering the listed topicsand ideas. In addition, IdeaFisher provided aquestion bank of over 3,000 questions designedto support thought processes such as flexibility,memory retrieval, and pattern switching. WhileIdeaFisher did not claim to inhibit exploratorythought, the main thrust of the software appearedto focus on tools to generate novel ideas.

The second, Ideatree, was selected to representan ICSS with an exploratory focus. Rather thanask open-ended questions or offer lists of genericideas, this package provided a means for usersto embellish, emphasize, and polish ideas.Specifically, the software enabled a user to typeconcepts into "idea-boxes," which could then belinked laterally or hierarchically. While the boxeshad a limit on the space available for phrasing,the user was free to append additional commentsto any idea-box through a word-processingoption (Cohen, 1991). Although Ideatree did notactively inhibit generative thought, it focused ondetailing, arranging, and coordinating ideas tomake them more meaningful and valuable.

The third, Harvard Graphics, was used as a soft-ware-control mechanism for any impacts thatmay result simply from computer use and notcreativity support. This conventional applicationwas selected because it mimicked some featuresoffered in formal ICSSs. For example, it readilyallowed users to create and arrange boxes,charts, and text-based outputs.

operationalized into idea fluency, novelty, value,and a creative performance score. Fluency wasdefined as the number of ideas generated byeach subject. Novelty was defined as the extentto which each response was rated as new,unique, and different. Value was defined as theextent to which each response was rated as real-istic or worthwhile. And the creative performancescore was generated by averaging novelty andvalue ratings across subjects’ responses.

A creativity inventory (from Hellriegel andSlocum, 1991) was given to each subject beforeand after the experiment. The inventory consist-ed of 36 items designed to assess an individual’sperceived self-confidence, need for individuality,abstract thinking ability, analysis capability,desire for task achievement, and degree of envi-ronmental control. Because these .have beenconsistently cited as characteristics of a creativeperson (Amabile and Tighe, 1993; Barron andHarrington, 1981; Tardif and Sternberg, 1988;Torrance, 1988), the inventory appeared to pro-vide a general determination of each subject’sability to perform creatively. Besides having facevalidity, the inventory seemed relatively simple toadminister. It was split into two 18-questioninstruments so that three questions about eachcharacteristic were present on each version. Theresponse options were constructed so that higherscores reflected individuals more predisposed toperform creatively.

A software satisfaction survey was also given toeach subject in the software treatments. Thesurvey consisted of 11 items constructed todetermine a subject’s attitude toward computers(four items); deference toward the software used(two items); sense of the software’s applicabilityand decision support (three items); and percep-tion of its ease-of-use and operability (two items).The survey also contained two open-endedquestions to gather additional opinions about thesoftware each subject used.

Variables and measures

The independent variable manipulated was thetype of creativity support each individual used togenerate creative responses. The dependentvariable was the subjects’ creative performanceon a decision task. Creative performance was

Decision task

The task required subjects to devise solutions tothe homeless problem faced by cities and soci-ety. This task was chosen based on Haines andAmabile (1988), which suggested that

MIS Quarterly/March 1996 87


decrease the potential for response bias, tasksrequiring no specific knowledge or trainingshould be used in creativity experimentation.Because homelessness reflects a problem thatany person living in a large metropolitan areawould have general awareness of and one wherecommon knowledge would suffice for solutionformation, the task seemed to meet these crite-ria. In addition, the task seemed distinctiveenough to hold the subjects’ attention but indefi-nite enough to support generative and explorato-ry thought equally well (Finke, et al~, 1992).

Judges

The judges were selected for their expertise ingovernmental policy issues such as homeless-ness. One was an attorney specializing inhuman rights, and the other was a former com-munity council president with a master’s degreein public administration. The judges made theirassessments independently and rated the ideasrelative to each other rather than to a standardthey might hold from, their work domains. Expert-level performance standards are typically toohigh for members of the general population toconsistently meet (Amabile and Tighe, 1993).Therefore, a negative rating bias might be intro-duced if an expert were to apply a domain-deter-mined standard. The judges were asked to rateeach idea on a scale of one (no merit for the fac-tor) to 10 (maximum merit for the factor).

Experimental procedures

The experiment occurred over two sessions. InSession 1, the creativity training session, sub-jects were given an introduction in how to thinkmore creatively. The session began with a lec-ture on how important creativity is for businessdecision making. Examples of how organizationshave used creativity to become more successfulwere given. Subjects were also given a descrip-tion of how the creative process occurs and weremade aware of biases both individuals and orga-nizations have with respect to creativity. Ways toovercome these obstacles were suggested, anddiscussion was encouraged. The session lastedapproximately an hour and included one exercise

in brainstorming and one in elaboration. Thepurpose of the session was to establish a consis-tent knowledge base conceming creativity.

After the discussion, each subject was given thefirst half of the creativity inventory to determinehis/her baseline ability to respond creatively. Thepreinventory measure was given at the end of thetraining session so that a distinction could later bemade between training and experimental effects.Had the preinventory been given before training,establishing whether training or treatment pro-duced a given outcome would be difficult.

The second session occurred one week afterthe first. In Session 2, subjects in each softwaretreatment received an initial overview of howtheir application functioned. Specifically, theidea and question bank features of IdeaFisher(novel-thought support tools) were taught subjects in the generative treatment. Subjects inthe exploratory treatment were taught the idea-box and note-pad options of Ideatree (refining-thought support tools). And conventional sub-jects received instruction in the organizationchart and text-writing options of HarvardGraphics because these features appeared tomimic the ICSS box and screen features.

The software applications were menu-driven,offered online help, and appeared straightfor-ward to use. For example, to use the idea bankfeature in IdeaFisher, one would highlight listedideas and then copy them to a supporting"screen-pad" with the click of a mouse. No spe-cific creativity training examples were used inSession 2. But, various suggestions for how touse the chosen functions were taken from eachapplication’s training manual. The training foreach software application took approximately40 minutes to complete. When a subjectacknowledged comfort in operating the featurespertinent to his/her application, that subject wasgiven a sheet of paper containing the decisiontask. Subjects were told to generate as manydifferent ideas as possible in the period allottedusing the software they had just learned.

Subjects in the control treatment were eachgiven a pen, blank paper, and the sheet contain-ing the assigned task. They, too, were told togenerate as many different ideas as possible inthe timeframe allotted. All subjects in all treat-



ments were allowed 30 minutes to complete theassigned task. All subjects were allowed to askquestions concerning the tools they used suchas "How do I change screens?" for the softwaretreatments, or "Can I write on the back of thepaper?." for the control treatment. Tool-focusedquestions were allowed to subvert confusioneffects that might occur from subjects notremembering how to use their applications. Butno questions concerning the assigned task wereallowed. At the end of the allotted time, subjectswere asked to either save their ideas on disk orturn them in on paper. All subjects then complet-ed the second half of the creativity inventory,and subjects in the software treatments alsocompleted the software satisfaction survey.

Data compilation techniques

Once the experiment was complete, the subjects’ideas were typed into a common word-processingformat. A common format was used so that anyvisual differences in the outputs generated (such

¯ as handwriting or font style) would not influencethe judges’ decisions. The process of differentiat-ing ideas from the responses given by the sub-jects occurred as follows. For the most part, thesubjects either numbered their ideas, separatedthem by spacing between blocks of text, placedthem into different boxes with any text-basedelaboration attached on the electronic note-pad,or simply used phraseology such as "another ideawould be." Where a response contained a lengthyexplication of similar content or was not distinctlyseparated, the entire response was included as asingle idea. Given that the judges were expertscapable of appropriately assessing any elabora-tion and that nuances in word and grammarchoice more accurately reflect a respondent’s trueintention, the ideas were transcribed as writtenwith only spelling errors and basic punctuationcorrected. Moreover, because the judges wereasked to rate the ideas relative to one another, allideas, even those of one subject that may haveappeared similar in content to the ideas of anoth-er subject, were given to the judges.

Each expert was given a list of 149 ideas andasked to rate each on novelty and value. Two ver-sions of the idea lists, one the reverse order of theother, were used so that fatigue or leaming effects

could be detected. The experts were provided asmuch time as they wanted and were encouragedto take breaks to avoid fatigue. Because so manyideas were generated, and because the expertswere professionals with pressing responsibilitiesoutside the cause of research, they only wentthrough the rating process once.

Results

Because of the volume of data analyzed, theresults have been divided into the following sec-tions: Creative Ability, Judges’ Reliability, IdeaFluency, Creative Performance, Generative andExploratory Support, and Software Satisfaction. Asummary of the statistics used for testing thehypotheses can be found in Table 1.

Creative ability

A formal extreme-scores test revealed that onepair of inventory scores from the control treatmentwas an outlier (Hildebrand and Ott, 1991). Afterviewing this subject’s responses to both versionsof the creativity inventory and the experimentaltask, it was decided to drop the subject from thestudy. A total of 43 subjects were left in the analy-sis. Table 2 presents the means and standarddeviations for the creativity preinventory and post-inventory scores.

Multivariate analysis of variance techniques wereused to compare subjects’ inventory scores beforeand after the experimental manipulation. No signif-icant differences were found between the meanscores at the .10 probability level.1 No systematicbias was revealed, and creative ability did notappear influenced by the treatment conditions.

Judges’ reliability

Interrater reliability was determined for eachmeasure of creativity assessed by the judges

Because of the exploratory nature of this investigation, the.10 probability level, a more inclusive criterion, was delectedfor hypothesis testing (Cohen, 1977).



Table 1. Summary of Key Statistics for Performance and Satisfaction Measures

Variable/ Wilks’Statistic/ lambdaEffectTested

Creative Ability 3.26

Fluency 40.91

Treatment 2.54

Treatment xFluency

Performance Satisfaction

F Ratios

CreativePerformance Novelty Value

6.88 3.48 6.02

78.75 51.38 70.01

3.90 2.56 4.00

Wilks’lambda F Ratios

Computer SoftwareComfort Likability

Ease i Decisionof Usei Support

2.15 6.31 7.36

Note: Only values that tested significant at the .10 probability I~vel or below are reported,

Table 2. Means and Standard Deviations of Creativity InventoryScores and Number of Ideas Generated

Instrument/ Creativity Creativity NumberGroup n Preinventory Post-Inventory of Ideas

Control 9

Generative 11

Exploratory 11

Conventional 12

Overall 43

45.89 46.44 2.22(8.30) (5.25) (1.13)43.09 44.82 3.73(5.17) (7.29) (2.53)43.82 44.82 3.91(7.80) (5.25) (1.44)45.92 42.50 3,25(7.91) (9.47) (1.48)44.65 44.51 3.30(7.22) (7.07) (1.65)

Notes~AII measures are means except where noted. Standard deviations appear in parentheses.The higher the value, the more the individual is predisposed to perform creatively.

(Huck, et al., 1974). The coefficient of correla-tion for novelty was .88 and for value, .74. Thehigh degree of agreement on novelty is logicalwith expert-level raters (Amabile and Tighe,1993). The more moderate agreement on valueis logical given the experts were involved in dif-ferent aspects of the problem domain.

Idea fluency

Hypothesis testing began by inVestigating H1,the impact of the experimental treatments onthe number of ideas generated by each subject.The means and standard deviations for thismeasure can be found in Table 2.

In preparing to perform a one-way analysis ofvariance, preliminary tests showed that thehomogeneity of variance assumption was violat-ed (Bartlett-Box F = 2.22, p < .10), while fre-quency distributions and normal plots on thenumber of ideas generated indicated the pres-ence of a bimodal distribution. Specifically, clus-tering existed around the low and high valuesfor idea fluency.

Although the creativity inventory results indicat-ed that subjects were dispersed throughout thetreatments in an unbiased manner, the invento-ries did not directly account for a subject’s abili-ty to generate ideas. In the psychological litera-ture, idea fluency has been characterized as acreative ability that remains relatively constant

90 MIS Quarterly/March 1996


over time (Guilford, 1950; Torrance, 1988; andWallach, 1983). Consequently, rather than havea random sample free of systematic bias, thetreatment groups contained individuals whoappeared to vary in their ability to generateideas.

To statistically adjust these conditions so thatthe hypotheses could be appropriately tested,an additional independent factor entitledFluency--with two levels (high and Iow)--wasformed (Dillon and Goldstein, 1984). Subjectswere placed into the high-fluency category ifthey generated more than the mean number ofresponses (i.e., four or more) and into the low-fluency category if they generated three orfewer responses. Because the subjects weredispersed throughout the treatment condiiionsin such a fashion that discernible fluency levelscould be formed, the analysis continued.2

Creative performance

A creative performance score was generated byaveraging each subject’s total novelty and valuerating scores.3 By using an averaging process,both qualitative and quantitative aspects of asubject’s performance could be simultaneouslyconsidered. Because creative ability was onlyindirectly controlled before the experimentalmanipulations, hypothesis testing began againusing analysis of covariance techniques.

2 Interestingly, creativity research has rarely attempted todirectly control for idea fluency before experimental manipu-lation (Ackoff and Vergara, 1981 ; VanGundy, 1992).Typically, research efforts have not only operationalizedidea fluency as a dependent measure of creative perfor-mance, but they have also used bias controls for creativeability that only indirectly consider an individual’s propensityto generate ideas quickly (Barron and Harrington, 1981;Domino, 1970; Joseph and Pillaj, 1986). Without establish-ing a subject’s fluency level prior to treatment and control-ling for this potential bias throughout experimentation, anyresults produced could be due to an individual’s propensityto generate ideas (either innately present or otherwiselearned) and not the intended experimental manipulation.

3 Total novelty and value rating scores were generated bysumming each judge’s ratings on value and novelty for eachsubject’s response set. These totals were then divided bythe number of ideas the given subject generated to dedve amean novelty and value score for each subject. These meanscores were then averaged across judges to create a totalnovelty and a total value rating score for each subject.

Specifically, an assessment of H1 and H2 wasmade using the creative performance score asthe dependent variable, the treatment conditionand degree of fluency present as independentfactors, and the creative ability preinventoryscore as a covariate. Adjusted mean scores forcreative performance can be found in Table 3.

The ability covariate was significantly and posi-tively related to creative performance (F1,34 6.88, p < .01). Two main effects were signifi-cant: treatment (F3,~4= 3.9, p = .02) and fluency(F1,~4 = 78.75, p < .01). No significant interac-tion effects were noted. In addition, fluencyaccounted for the largest percent of varianceexplained--57 percent--while treatmentaccounted for 9 percent, and creative abilityaccounted for 5 percent.

The main effect of fluency revealed that highlyfluent subjects generated ideas considered sig-nificantly more creative than those produced byless fluent subjects. And the Scheffe univariateadjusted mean comparisons for interpreting thetreatment effect indicated that subjects in thesoftware conditions performed significantlymore creatively than subjects in the no softwarecondition (t = -3.10, p = .01). However, no sig-nificant performance differences were notedbetween the software conditions.

H1 was not supported by these findingsbecause the Treatment x Fluency interactionwas not significant. Hence, subjects using ICSSsoftware did not generate more ideas than sub-jects using conventional software or no soft-ware. The findings did show partial support forH2, however, given that subjects using software.outperformed subjects using a pen and paper.

Generative and exploratory support

Multivariate analysis of covariance techniqueswere used to examine H3 and H4, with the nov-elty and value measures as dependent vari-ables, the treatment condition and degree of flu-ency present as independent variables, and thecreative ability preinventory score as a covari-ate. Adjusted means for both novelty and valuecan be found in Table 3.


¯ Creativity Support Systems

Table 3. Adjusted Means for Creativity, Novelty, and Value

Dependent Measure/Independent Factor n Creativity Novelty ValueControlHigh Fluency 2 14.69* 13.47" 16.59"Low Fluency 7 7.94 6.46 9.86Treatment Total 9 11.31"* 9.69*** 13.14"*

GenerativeHigh Fluency 5Low Fluency 6Treatment Total 11

24.20* 24.13* 24.62*8.61 7.18 9.80

16.40 15.66 17.21

ExploratoryHigh Fluency 6 23.36* 19.95" 27.22*Low Fluency 5 12.33 11.11 13.94Treatment Total 11 17.85 15.53 20.58

ConventionalHigh Fluency 4 21.17" 18.18" 24.16"Low Fluency 8 9.74 8.13 11.73Treatment Total 12 15.45 13.16 17.95

OverallHigh Fluency 17 20.85* 18.93* 23.15"Low Fluency 26 9.65 8.22 11.29Sample Total 43 15.25 13.58 17.22

Notes: All measures are means except where noted. Higher scores represent better performance.* High fluency outperformed Low fluency at a .01 probability level.

** The Computer groups outperformed the Control group at a .01 probability level.*** The Computer groups outperformed the Control group at a .02 probability level.

The test for covariation was significant (Wilks’lambda = 3.26, p = .05), with creative abilitypositively related to both novelty (F1,34 = 3.48,p = .07) and value (F1,34 = 6.02, p = .02). Theoverall multivariate test for fluency was signifi-cant (Wilks’ lambda = 40.91, p < .01) as werethe subsequent univariate comparisons for itseffect on novelty (F~,34 = 51.38, p < .01) andvalue (F~.34 = 70.01, p < .01). Thus, highly flu-ent subjects produced ideas that were judgedsignificantly more novel and valuable than thoseproduced by less fluent subjects.

The overall multivariate test for treatment wasalso significant (Wilks’ lambda = 2.54, p = .03),with the univadate comparisons for its effect on

novelty (F3.34 = 2.56, p = .07) and value (F3,34 4.00, p = .01) significant. In addition, Scheffemean comparisons.suggested that subjects in allthree software conditions produced ideas judgedsignificantly more novel (t = -2.40, p = .02) andmore valuable (t = -2.86, p = .01) than those pro-duced in the no software condition. But, subjectsin the software treatments did not significantlyoutperform each other on novelty or value. Nointeractions were noted in this analysis.

In general, these findings do not support H3and H4 since the generative application did notoutperform the exploratory on the perceivednovelty of ideas produced, and the exploratory



application did not outperform the generative onperceived value.

Software satisfaction

To investigate subjects’ satisfaction concerningthe software they used, composite measures ofcomputer comfort, software likability, betterdecision support, and ease of use were formedby averaging subjects’ responses across relat-ed items. The means and standard deviationsof these measures can be found in Table 4.

With no significant covariation, the compositesatisfaction measures served as dependent vari-ables, with the treatment condition and degreeof fluency present as independent factors in amultivariate analysis of variance. Overall, themain effect of treatment was significant (Wilks’lambda = 2.15, p = .02), indicating subjects’ sat-isfaction varied depending on the software theyused. Univariate comparisons revealed signifi-cant differences in perceptions of software lika-bility (F2,31 = 6.31, p < .01) and ease of use (F2,31= 7.36, p < .01). Specifically, individual meancomparisons showed that subjects using thegenerative application liked it significantly/essthan subjects using either the conventional (t 3.44, p = .01) or exploratory (t = 4.27, p = 02)application. The generative application was alsorated significantly more difficult to use thaneither the conventional (t = 3.36, p = .01) exploratory (t = 3.27, p = .01). Satisfaction withthe conventional or exploratory applications didnot differ. Moreover, fluency did not affect soft-ware satisfaction, perceptions of computer com-fort or better decision support did not varybetween the software conditions, and no interac-tions were noted.

ConclusionAlthough using experimental and statistical tech-niques to control for the decision task, creativitytraining, and creative ability precluded a compre-hensive examination of Figure 1, an indicationthat software support directly enhances an indi-vidual’s creative performance was noted.Moreover, since the number of ideas generated

did not differ between treatment conditions, andall subjects in the software conditions had priorexperience with computers, it is unlikely thatusing technology, per se, produced the perfor-mance enhancements. Rather, the finding thatsatisfaction with the generative software was rel-atively low while performance was high suggeststhe applications might have been providing deci-sion support and not simply charming subjectsinto performing more creatively.

However, because performance did not differbetween the software conditions, this experi-ment cannot explain exactly how the softwareenhanced creative performance. Both genera-tive and exploratory thought seemed equallywell supported, though the software featuresexamined were expected to provide differentialsupport. One potentially valuable use for ICSStechnology may be to frame thought in a partic-ular manner so that the ideas produced reflectspecific qualities. However, if all support fea-tures improve novelty and value equally well,then discerning which feature to use whenbecomes arbitrary. Moreover, because low rat-ings on likability and usability did not appear tonegatively affect performance, further study ofthe impact of ICSS design on creative perfor-mance seems appropriate.

Another limitation to this investigation is theone-time examination of the technology. Whilethe performance difference noted was favor-able to the technology, the efficacy of an ICSSto enhance creativity over time was not exam-ined. And potential relationships between ICSStechnology, creativity training, and the decisiontask were not fully explored. For example,although the technology appeared to enhanceperformance when training was held constant,ICSS may directly affect training efforts. Or, amore analytic decision task may hinder theability of ICSS to enhance creative perfor-mance.

Despite its limitations, a few useful insightshave been gained from this investigation. First,an empirical demonstration of ICSS technolo-gy’s positive effect on creative performanceoffers an encouraging confirmation for its popu-larity. Second, a formal consideration of factorsrelevant for creativity enhancement provides afocus for managerial efforts to improve creative

MIS Quarter/y/March 1996 93


Table 4. Means and Standard Deviations for Software Satisfaction

Fluency/Satisfaction/

Treatment High Low Total

ComputerComfort

Generative 2.35 2.46 2.41(0.91) (0.83) (0.87)

Exploratory 2.33 2.50 2.42(1.06) (0.66) (0.86)

Conventional 2.38 2.03 2.21(0.66) (0.31) (0.49)

Overall 2.29 2.35 2.31(0.60) (0.83) (0.71)

SoftwareLikability

Generative 2.90 3.08 2.99*(1.57) (0.80) (1.19)

Exploratory 2.17 2.00 2.09(0.52) (0.79) (0.66)

Conventional 1.88 1.56 1.72(0.63) (0.68) (0.66)

Overall 2.16 2.33 2.24(0.95) (0.98) (0.96)

DecisionSupport

Generative 3.33 3.61 3.47(0.82) (0.65) (0.74)

Exploratory 3.00 3.20 3.10(0.79) (0.65) (0.72)

Conventional 3.41 2.58 2.99(1.19) (0.46) (0.83)

Overall 3.07 3.22 3.14(0.69) (0.84) (0.76)

Easeof Use

Generative 2.70 3.42 3.06*(1.92) (0.67) (1.30)

Exploratory 1.58 1.80 .~1.69(0.67) (0.84) (0.76)

Conventional 1.75 1.69 1.72(0,50) (0.59) (0.55)

Overall 2.26 2.00 2.15(1.00) (1.20) (1.10)

Notes: All values are means except where noted. The standard deviations appear in parentheses. Thelower the value, the more positive the perception.*Significant at a probability level Ibelow .01.

performance. Third, the inclusion of idea fluencyas an independent factor affecting creative per-formance suggests a more thorough approachto researching creativity. These insights may

not only allow for better designs and applica-tions of,ICSS, but may also ultimately improvethe ability of organizations to creatively respondto their environments.



Acknowledgements

The author would like to thank ThomasAbraham, Larry W. Boone, Beata Lobert, andPatrick Lyons for their suggestions and supportduring the initial design and data collectionphases of this project. The author would alsolike to thank the reviewers for their very helpfulguidance in developing this research projectinto a paper.

References

Abraham, T. and Boone, L. W. "Computer-Based Systems and Organizational DecisionMaking: An Architecture to SupportOrganizational Innovation," CreativityResearch Journal, April/May 1994, pp.111-123.

Ackoff, R. L. and Vergara, E. "Creativity inProblem Solving and Planning: A Review,"European Journal of Operational Research(7:1), 1981, pp. 1-13.

Albert, R. S. "The Concept of Genius and ItsImplications for the Study of Creativity," inGenius and Eminence: The SocialPsychology of Creativity and ExceptionalAchievement, R. S. Albert (ed.), PergamonPress, Oxford, England, 1983, pp. 6-18.

Amabile,T. M. "Within You, Without You: TheSocialPsychology of Creativity and Beyond,"in Psychological Dimensions ofOrganizational Behavior, B. M. Staw (ed.),Macmillan, New York, NY, 1991,pp.537-558.

Amabile, T. and Tighe, E. "Questions ofCreativity," in Creativity: The Reality Club 4,J. Brockman (ed.), TouchStone, Simon andSchuster, New York, NY, 1993, pp. 7-27.

Barron, F. and Harrington, D. M. "Creativity,Intelligence, and Personality," AnnualReview of Psychology (32), 1981, pp.439-463.

Beckett, D. "Straining Training; TheEpistemology of Workplace Learning,"Studies in Continuing Education (14:2),1992, pp. 130-142.

Cohen, J. Statistical Power Analysis for theBehavioral Sciences, Academic Press, NewYork, 1977.

Cohen, L. Power Thinking: Top-Down Analysisfor the Information Age, Mountain HousePublishing, Waitsfield, VT, 1991.

Couger, J. D. Creative Problem Solving andOpportunity Finding, Boyd and Fraser,Danvers, MA, 1995.

Coulson, L. and Strickland, A. "AppliedCreativity," Executive Excellence (8), August1991, pp. 8-9.

Cox, C. M. "The Early Mental Traits of 300Geniuses," in Genius and Eminence: TheSocial Psychology of Creativity andExceptional Achievement, R. S. Albert (ed.),Pergamon Press, Oxford, England, 1983, pp.46-51.

Csikszentmihaly, M. "Society, Culture, andPerson: A Systems View of Creativity," inThe Nature of Creativity: ContemporaryPsychological Perspectives, R. S. Albert(ed.), Cambridge University Press,Cambridge, MA, 1988, pp. 325-339.

Dayton, D. "Idea Generators Spark NewSolutions," PC Week, March 18, 1991, pp.109-110.

de Bono, E. de Bono’s Thinking Course, Factson File, New York, NY, 1993.

Diehl, M. and Stroebe, W. "Productivity Loss inBrainstorming Groups: Toward a Solution ofa Riddle," Journal of Personality and SocialPsychology (53:3), 1987, pp. 497-509.

Dillon, W. and Goldstein, M. MultivariateAnalysis: Methods and Applications, JohnWiley and Sons, New York, 1984.

Domino, G. "Identification of Potentially CreativePersons From the Adjective Check List,"Journal of Consulting and ClinicalPsychology (35:1), 1970, pp. 48-51.

Elam, J. J. and Mead, M. "Can SoftwareInfluence Creativity?" Information SystemsResearch (1), March 1990, pp. 1-22.

Evans, J. R. Creative Thinking in the Decisionand Management Sciences, South-WesternPublishing, Cincinnati, OH, 1991.

Everett, E. "Improving Creativity: OneOrganization’s Approach," Public Manage-ment (65:2), 1983, pp. 7-8.

Finke, R. A., Ward, T. B., and Smith, S. M.Creative Cognition: Theory Research andApplications, The MIT Press, Cambridge,MA, 1992.



Finn, K. Making Creativity Happen: AnIntroduction to the Solution Machine, TheGemini Group, Bedford, NY, 1993.

Gillam, T. K. "Managing the Power ofCreativity," Bank Marketing, December 1993,pp. 14-19.

Guilford, J. P. "Creativity," AmericanPsychologist (5:9), September 1950, pp.444-454.

Guilford, J. P. Intelligence, Creativity, and TheirEducational Implications, Robert R. Knapp,San Diego, CA, 1968.

Haines, B. and Amabile, T. "The Conditions ofCreativity," in The Nature of Creativity:Contemporary Psychological Perspectives,R. Sternberg (ed.), Cambridge UniversityPress, Cambridge, MA, 1988, pp. 11-38.

Hellriegel, D. and Slocum, J. W. ExperiencingManagement, Annotated Instructor’s Edition:Management, Sixth Edition, Addison-Wesley,Reading, MA, 1991.

Herrmann, N. "Magic of Inspiration," ExecutiveExcellence, September 1993, p. 20.

Hildebrand, D. K. and Ott, L. Statistical Thinkingfor Managers: Third Edition, PWS-KentPublishing, Boston, MA, 1991.

Huck, S. W., Cormier, W. H. and Bounds, W. G.Reading Statistics and Research, Harperand Row, New York, NY, 1974.

Jacobs, R. D. "Creativity Through Interpretationand Its Implications for Education," Journalof Educational Thought (23:3), December1989, pp. 197-208.

Johnson, P. "The Mindful Use of Mental Capitalin Career Development," InternationalJournal of Career Management (4:2), 1992,pp. 8-14.

Joseph, A. and Pillaj, A. "Projective Indices ofCreativity," Indian Journal of ClinicalPsychology(13:1), 1986, pp. 9-14.

Kanter, R. "The Middle Manager as Innovator,"Harvard Business Review (60), July/August,1982, pp. 95-103:

Kiechel, W. "Getting Creative," Fortune (108),July 1983, p. 109.

LaBarre, P. "The Creative Revolution," IndustryWeek, May 16, 1994, pp. 12-19.

Marakas, G. and Elam, J. "CreativityEnhancement: Through Software or Process?"Management Science, 1994 (forthcoming).

Morgan, G. Imaginization, Sage, Newbury Park,CA, 1993.

Proctor, To "Experiments With Two ComputerAssisted Creative Problem-Solving Aids,"Omega, The International Journal ofManagement Science (17: 2), 1989, pp.197-200.

Restak, R. "The Creative Brain," in Creativity:The Reality Club 4, J. Brockman (ed.),TouchStone Book, Simon and Schuster,New York, NY, 1993, pp. 164-175.

Robbin, A. IdeaFisher m An Introduction, FisherIdea Systems, Irvine, CA, 1990.

Roberts, M. "Brainstorming by Computer,"Psychology Today, July/August, 1989, p. 51.

Rouse, N. E. "Brainstorming Software UnlocksCreativity," Machine Design, October 12,1989, pp. 100-102.

Saletta, P. What is Creativity, publication fromthe Office of Gifted and Talented, US Officeof Education, Department of Health,Education, and Welfare, Washington, DC,May 1978.

Tardif, T. Z. and Sternberg, R. J. "What Do WeKnow About Creativity?" in The Nature ofCreativity: Contemporary PsychologicalPerspectives, R. J. Sternber 9 (ed.),Cambridge University Press, Cambridge,MA, 1988, pp. 429-440.

Thierauf, R. J. Creative Computer Software forStrategic Thinking and Decision Making,Quorum Books, Westport, CT, 1993.

Torrance, E. P. "The Nature of Creativity asManifest in its Testing," in The Nature ofCreativity: Contemporary PsychologicalPerspectives, R. J. Sternberg (ed.),Cambridge University Press, Cambridge,MA, 1988, pp. 43-75.

VanGundy, A. IdeaPower, AmericanManagement Association, New York, NY,1992.

VanGundy, A. Training Your Creative Mind,Prentice Hall, Englewood Cliffs, N J, 1982.

Walberg, H. J. "Creativity and Talent asLearning," in The Nature of Creativity:Contemporary Psychological Perspectives,R. J. Sternberg (ed.), Cambridge UniversityPress, Cambridge, MA, 1988, pp. 340-361.

Wallach, M. A. "What Do Tests Tell Us AboutTalents," in Genius and Eminence: TheSocial Psychology of Creativity andExceptional Achievement, R..S. Albert (ed.),Pergamon Press, Oxford, England, 1983, pp.99-113.



Watson, D. L. "Enhancing Creative Productivitywith the Fisher Association Lists," Journal ofCreative Behavior, 1989, pp. 51-58.

Winship, S. "Packages Stimulate CreativeProcess, Buyers Say," PC Week (23:1),March 18, 1991, pp. 109-110.

Young, L. Decision Support and IdeaProcessing Systems, WC Brown, Dubuque,IA, 1989.

About the Author

Brenda I~lassetti is currently an assistant pro-fessor in the Management Department at St.John’s University in New York City. Shereceived her Ph.D. in information and manage-ment science from Florida State University. Herresearch emphasis is on exploring the impactsof communication and information technologieson individuals, organizations, and society.


an empirical examination of the value of creativity ... · pdf filevalue of creativity support...

Documents