the value and effectiveness of project management ... · the value and effectiveness of project...

THEVALUEANDEFFECTIVENESSOFPROJECTMANAGEMENT

SIMULATIONASACAPSTONEACTIVITYINANEXECUTIVEMBA

EDUCATIONALPROGRAM

by

JamesMichaelSzot

Athesis

submittedforthedegreeof

DoctorofPhilosophyinStrategy,Programme&ProjectManagement

SkemaBusinessSchool,Lille

March2013

PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE ii

THEVALUEANDEFFECTIVENESSOFPROJECTMANAGEMENTSIMULATIONASA

CAPSTONEACTIVITYINANEXECUTIVEMBAEDUCATIONALPROGRAMS

Athesissubmitted

by

JamesMichaelSzot

toSkemaBusinessSchool

inpartialfulfillmentoftherequirementsforthedegreeof

DoctorofPhilosophy

inthesubjectof

Strategy,Programme&ProjectManagement

ThisdissertationhasbeenacceptedforthefacultyofSkemaBusinessSchool

PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE iii

Certificate

Icertifythattheworkinthisthesishasnotpreviouslybeensubmittedfora

degreenorhasitbeensubmittedforadegreeexceptasfullyacknowledgedinthetext.I

alsocertifythatthethesishasbeenwrittenbyme.AnyhelpthatIhavereceivedinmy

researchworkandthepreparationofthethesisitselfhasbeenacknowledged.In

addition,Icertifythatallinformationandliteratureusedareindicatedinthethesis.

JamesM.Szot

PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE iv

Acknowledgement

Anundertakingofthisenormityisonlypossiblewiththesupportand

encouragementofothers.Attheriskofomittingsomeonewhodeservesmention,I’d

liketothankseveralgroupsofpeopleforsharingtheirknowledgeandinspiration.

Firstthanksgototheexecutiveeducationprojectmanagementgraduate

studentsatTheUniversityofTexasatDallaswhoparticipatedinthisstudy;JamesJoiner

ofTheUniversityofTexasatDallasforencouragingmetostartthejourney;Dr.

ChristopheBredillet,thenofESCLille,whowelcomedmeintotheprogram;thePhD

FacultyatSkemaBusinessSchool,especiallyDr.RalphMuellerandDr.PhilippeRuizfor

theirexcellentcoursescoveringresearchmethods,qualitativeandmixedmethods

research,andquantitativemethodsandresearchtechniques;Dr.FrankAnbari,Clinical

ProfessorofProjectManagementatDrexelUniversity,forhisguidanceandadvocacy;Dr.

RodneyTurnerforhisleadershipduringmylateryearsintheprogram;andexaminers

Dr.LynnCrawfordandDr.YoungHoonKwakwhoprovidedvaluablesuggestionsfor

editingthefinalversion.

ClosertohomeandmoreremovedintimeI’dliketothankDr.SuzanneStoutand

Dr.JohnWiorkowskiformyinitialtutelageinresearchmethodsandappliedstatistics

duringanearliergraduatedegreeprogramatTheUniversityofTexasatDallas.Lessons

learnedfromthemwhettedmyappetiteforresearchandsetthestageforthisendeavor.

Noneofthiswouldhavebeenachievablehaditnotbeenformyparents,Joseph

andGenevieveSzotwhobroughtmeintothisworld,instilledaworkethic,andmade

suretheysavedenoughmoneyformyundergraduatetuitionsoIcouldbethefirstinour

familytoearnacollegedegree.

Andlast,butcertainlynotleast,manythanksgotomywife,Patricia,forliving

withonemoredistractioninourwonderfullifejourneytogetherasIworkedonthis

researchorjettedofftoFranceforanotherseminar.

Allshortcomingsinthisresearchareattributabletomealone.

PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE v

Table of Contents

Certificate ................................................................................................................................... iii

Acknowledgement ..................................................................................................................... iv

List of Illustrations .................................................................................................................... viii

List of Tables ............................................................................................................................... ix

List of Acronyms and Abbreviations ........................................................................................... xi

Abstract .................................................................................................................................... xiii

Introduction ................................................................................................................................ 1

Background ............................................................................................................................. 2

Research Aim ........................................................................................................................ 15

SimProject – the simulation under study ............................................................................. 16

Approach and Methodology ................................................................................................ 17

Hypotheses Tested ............................................................................................................... 19

Organization of this Study .................................................................................................... 20

Literature Review ..................................................................................................................... 22

Historical Perspective ........................................................................................................... 22

Search Methodology ............................................................................................................ 23

Taxonomy ............................................................................................................................. 25

Definitions ............................................................................................................................ 27

Experiential Learning ............................................................................................................ 31

Simulation Games as Experiential Learning Activities (ELA) ................................................ 33

Simulation Game Effectiveness Research ............................................................................ 55

Project Management Simulation Gaming Research ............................................................ 70

Summary ............................................................................................................................ 108

Methodology .......................................................................................................................... 111

Research Paradigm ............................................................................................................. 111

PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE vi

Research Approach ............................................................................................................ 113

Research Design ................................................................................................................. 114

Survey Design ..................................................................................................................... 115

Pilot Study .......................................................................................................................... 119

Participants ......................................................................................................................... 121

Procedure ........................................................................................................................... 123

Data Processing and Analysis ............................................................................................. 126

Ethical Considerations ........................................................................................................ 129

Summary ............................................................................................................................ 129

Results .................................................................................................................................... 131

Initial data analysis ............................................................................................................. 131

Verification of assumption of parametric data .................................................................. 135

Scale Reliability ................................................................................................................... 142

Participant demographics .................................................................................................. 145

Descriptive statistics ........................................................................................................... 155

Hypothesis testing .............................................................................................................. 155

Qualitative Results.............................................................................................................. 160

Course Evaluation Survey Quantitative Data ..................................................................... 166

Summary of Results ............................................................................................................ 167

Discussion ............................................................................................................................... 169

Experiential Learning and the Project Management Simulation ....................................... 169

Discussion of Findings ........................................................................................................ 170

Limitations of this research ................................................................................................ 179

Summary ............................................................................................................................ 180

Conclusion .............................................................................................................................. 182

Contribution to Knowledge ................................................................................................ 183

PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE vii

Implications ........................................................................................................................ 184

Opportunities for further study ......................................................................................... 185

Appendix A Literature Search Journal Sources .................................................................... 187

Appendix B Presimulation Team Assignment ...................................................................... 190

Appendix C Presimulation Survey ........................................................................................ 209

Appendix D Postsimulation Survey ...................................................................................... 217

Appendix E Project Simulation Final Presentation .............................................................. 227

Appendix F SimProject – About the Simulation .................................................................. 228

SimProject, an Engaging Experience .................................................................................. 228

Playing the game. ............................................................................................................... 229

Appendix G Descriptive Statistics for Variable Components ............................................... 232

References .............................................................................................................................. 239

PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE viii

List of Illustrations

Figure1–UniversitieswithGAC‐accreditedprogramsbyregion................................................5

Figure2‐LiteratureReviewTaxonomy................................................................................................26

Figure3‐ResearchApproach.................................................................................................................113

Figure4‐Pretest‐posttestdesign..........................................................................................................114

Figure5‐Researchmodel........................................................................................................................115

Figure6‐Teamgendermix.....................................................................................................................146

Figure7‐Teamtechnical/non‐technicalbackgroundmix.........................................................146

Figure8‐Contacthoursofpriorprojectmanagementtrainingoreducation....................147

Figure9‐Priortraining/educationbyteam.....................................................................................148

Figure10‐Projectexperience................................................................................................................149

Figure11‐Professionalexperience.....................................................................................................149

Figure12‐Worldregionofstudentorigin........................................................................................150

Figure13‐Worldregionoforiginbyteam.......................................................................................151

Figure14‐Industrybycohortgroup...................................................................................................152

Figure15‐Industrybyteam...................................................................................................................152

Figure16‐Jobtitlebycohortgroup....................................................................................................153

Figure17‐Jobtitlebyteam.....................................................................................................................153

Figure18‐Annualincomebycohortgroup......................................................................................154

Figure19‐Annualincomebyteam......................................................................................................154

Figure20‐Comparisonofaverageknowledgeratings................................................................171

Figure21‐Comparisonofaverageconfidenceratings................................................................172

Figure22‐Comparisonofpre‐andpost‐simulationattitudesonsimulation....................176

PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE ix

List of Tables

Table1‐Projectmanagementcompetenceitems.............................................................................86

Table2‐Teamexperienceassessmentitems......................................................................................86

Table3‐Attitudevariablequestions...................................................................................................116

Table4‐Problem‐solvingquestions....................................................................................................116

Table5‐Presimulationdemographicvariables..............................................................................116

Table6‐Postsimulationdemographicvariables............................................................................117

Table7‐Postsimulationopen‐endedquestions.............................................................................117

Table8‐RelevantProjectManagementCorePhaseexitsurveyquestions.........................118

Table9‐Pilottestscalereliability........................................................................................................120

Table10‐Simulationparticipants........................................................................................................122

Table11‐Datareviewsummary...........................................................................................................134

Table12‐Presimulationquestionresponseratebyvariable...................................................134

Table13‐Postsimulationquestionresponseratebyvariable.................................................135

Table14‐CombinedGroupsBandDdatasettestsofnormality.............................................136

Table15‐Testsofnormalitysplitbycohortgroup.......................................................................137

Table16–Cohortgroupmeanscores.................................................................................................139

Table17‐Resultsofindependentsamplest‐testbetweencohortgroups..........................140

Table18‐ScaleK1‐ReliabilityStatistics...........................................................................................142

Table19‐ScaleK1‐presimulationknowledgeItem‐TotalStatistics....................................143

Table20‐Scalereliability‐GroupsBANDD(OC2010ANDOC2011)..................................143

Table21‐RevisedScaleReliability‐GroupsBANDD(OC2010ANDOC2011)...............145

Table22‐Pairedsamplesstatistics......................................................................................................156

Table23‐Pairedsamplescorrelations...............................................................................................157

Table24–Pairedsampletestresults..................................................................................................158

Table25‐Courseevaluationratings....................................................................................................167

Table26‐Summaryofresults................................................................................................................168

Table27–SimProjectattitudequestions...........................................................................................177

Table28‐Descriptivestatisticsforpresimulationknowledgevariable(K1)....................232

Table29‐Descriptivestatisticsforpresimulationconfidencevariable(C1).....................233

Table30–Descriptivestatisticsforpresimulationteamexperience(T1)..........................233

Table31‐Descriptivestatisticsforpresimulationteamexperience(TP1)........................234

PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE x

Table32‐Descriptivestatisticsforpresimulationgenericteamattitude(Tg1)...............234

Table33‐Descriptivestatisticsforpresimulationgenericsimulationattitude(SG1)...234

Table34‐Descriptivestatisticsforpresimulationprojectsimulationattitude(SP1)....235

Table35‐Descriptivestatisticsforpresimulationtechnicalknowledgeapplication(N1,

E1,P1)...............................................................................................................................................................235

Table36‐Descriptivestatisticsforpostsimulationknowledgeperception(K2).............236

Table37‐Descriptivestatisticsforpostsimulationconfidenceperception(C2).............236

Table38‐Descriptivestatisticsforpostsimulationteamexperience(T2).........................237

Table39‐Descriptivestatisticsforpostsimulationteamexperience(TP2)......................237

Table40‐Descriptivestatisticsforpostsimulationgenericteamworkattitude(SG2)..237

Table41‐Descriptivestatisticsforpostsimulationgenericsimulationattitude(SG2).238

Table42‐Descriptivestatisticsforpostsimulationprojectsimulationattitude(SP2)..238

Table43‐Descriptivestatisticsforpostsimulationtechnicalknowledgeapplication(N2,

E2,P2)...............................................................................................................................................................238

PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE xi

List of Acronyms and Abbreviations

AACSB AACSBInternational‐TheAssociationtoAdvanceCollegiateSchoolsof

Business(formerlytheAmericanAssociationofCollegiateSchoolsof

Business)

ABSEL AssociationforBusinessSimulationandExperientialLearning

AMA AmericanManagementAssociation

ANOVA AnalysisofVariance

ELA ExperientialLearningActivity

EMBA ExecutiveMasterofBusinessAdministration

GAC GlobalAccreditationCenterforProjectManagementEducationPrograms

(sponsoredbytheProjectManagementInstituteandsometimesreferred

toasthePMI‐GAC)

GUI GraphicalUserInterface

ISAGA InternationalSimulationandGamingAssociation

JASAG JapaneseAssociationofSimulationandGaming

MBA MasterofBusinessAdministration

MMR MixedMethodResearch

MS MasterofScience

NASAGA NorthAmericanSimulationandGamingAssociation

PMBOK®GuideTheProjectManagementInstituteStandard:AGuidetotheProject

ManagementBodyofKnowledge

PMI ProjectManagementInstitute

PMI‐GAC ProjectManagementInstitutesponsoredGlobalAccreditationCenterfor

ProjectManagementEducationPrograms

PMP ProjectManagementProfessionalcredentialawardedbytheProject

ManagementInstitute

PMT ProjectManagementTrainer,asimulatordevelopedbyDr.Avraham

Shtub

PTB ProjectTeamBuilder,asimulatordevelopedbyDr.AvrahamShtubbased

onexperiencewithPMT

SACS SouthernAssociationofCollegesandSchools

SAGSET SocietyforAcademicGamingandSimulationinEducationandTraining

PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE xii

SCH SemesterCreditHour(nominallyequatesto15hoursofcontacttimeand

another30hoursofindependentstudy)

SLO StudentLearningObjective

SPSS Asoftwareprogramforstatisticalanalysisoriginallystandingfor

StatisticalPackagefortheSocialSciences

PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE xiii

Abstract

Moststudentsrespondfavorablywhenaskedabouttheuseofasimulationgame

inabusinessschoolcourse.However,havingenjoyedthegamedoesn’tnecessarily

meanitwasagoodinvestmentofthestudent’stimeandtuitiondollars.Thisresearch

reviewsthecurrentstateofmeasuringtheeffectivenessofsimulationgameexperiential

learningandbuildsonseveraltechniquesfoundintheliteraturetoexaminetheuseof

theSimProjectsimulationgameinoneuniversity’sprojectmanagementgraduate

program.Thisresearchisimportantbecauseincreasingdemandforqualifiedproject

managershasresultedinsubstantialgrowthinthenumberofacademicproject

managementdegreeprogramsandincreasedemphasisonassuranceoflearningby

academicaccreditingbodiesandgovernmentalentitiesrequiresexaminationofthe

effectivenessofmethodsusedtotrainandeducateprojectmanagers.

Theliteraturereviewdiscussesthreerelevantbodiesofknowledge:learning

theory,simulationgameapplicationandsimulationgameeffectiveness.Mostliterature

onbusinesssimulationgameeffectivenessexplorestheuseofmarketing,strategyor

totalenterprisesimulationgames;relativelyfewarticlesreportresearchontheuseof

projectmanagementsimulationgames.Thescarcityofresearchontheuseofproject

managementsimulationgamesasapedagogicaltoolidentifiesagapinman’sknowledge.

Usingmixedmethodtechniquesandaunitofanalysisoftheindividual,this

researchtakesapostpositivismapproachtowardmeasuringthechangeinstudent

perceptions,attitudesandabilitiesresultingfromparticipationinaprojectmanagement

simulationgameandexploreswhattheylearnedfromtheexperience.Resultsare

comparedwithanotherresearcher’sfindingstopositgeneralizabilityofconclusions.

Thisresearchfoundfavorableattitudesoverallandsignificantincreasesin

studentperceptionsofknowledgeandtheabilitytoapplythatknowledge,significant

increasesinattitudestowardstheirteamandteamwork,andnosignificantincreaseina

favorableattitudetowardssimulationgames.Thoughnotaspecificlearningobjective,a

surprisingfindingwasnoincreaseintheabilitytodevelopandanalyzeschedule

networkdiagramsortointerpretearnedvalueperformancegraphs.

TheuseoftheSimProjectgamewasfoundtobeavaluablecomponentofthe

curriculumandaneffectiveuseofclasstime.Recommendationsincludeusing

PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE xiv

assignmentsorinterimbriefingsonanyconceptstheeducatorwantsreinforcedduring

theexperienceandtheuseofformalself‐assessmenttechniquestoenhancelearning.

PROJECTMANAGEMENTCAPSTONESIMULATIONVALUE 1

Introduction

Increasingdemandforqualifiedprojectmanagersandtheresultingexponential

growthinthenumberofprofessionalprojectmanagercertificationsanduniversity

projectmanagementdegreeprogramscombinedwithincreasedemphasisonassurance

oflearningbyacademicaccreditingbodiesandgovernmentalentitiesaffirmthe

importanceofexaminingtheeffectivenessofmethodsusedtotrainandeducateproject

managers.

Simulationgamesareoftenusedinundergraduateandgraduatemarketingand

strategicmanagementcoursesandarealsofoundinsomeprojectmanagementcourses.

Thepurposeofthisresearchistoinvestigatetheuseofaparticularcomputer‐based

projectmanagementsimulationgameasacapstoneexperientialactivityfollowinga

seriesofprojectmanagementcoursesinanexecutiveeducationMasterofBusiness

Administration(MBA)cohortprogram,oftenreferredtoasanExecutiveMBAorEMBA

withanemphasisinprojectmanagement.Usingasimulationinthismannercanrequire

uptohalfthecontacthoursofa3semester‐credit‐hour(SCH)courseandonecan’thelp

butwonderwhetherornotthisisaneffectiveuseoftimewithajustifiablereturnon

tuitiondollarinvestment.

Withoriginsfromwargamingboardgamesdatingbackto3000BC,theonsetof

businessgamingintheUnitedStatesiscreditedtotheAmericanManagement

Associationinthemid‐1950s(Wolfe,1993).While“asaneducationaltool,business

simulationgameshavegrownconsiderablyinuseduringthepast40yearsandhave

movedfrombeingasupplementalexerciseinbusinesscoursestoacentralmodeof

businessinstruction”(Faria,Hutchinson,Wellington,&Gold,2009),furtherresearchon

itseffectivenessisneededas“theempiricalresearchontheinstructionaleffectivenessof

gamesisfragmented,filledwithill‐definedterms,andplaguedwithmethodological

flaws”(Hays,2005).Further,“studiesontheeducationalmeritsofsimulationsoftenare

measuredontheaffectivedomain,notthecognitivedomaintheypurporttomeasure”(P.

H.Anderson&Lawton,2009)

Thisstudyreviewstheliteratureontheuseandevaluationofsimulationgames

asexperientiallearningactivitiesinmanagementcurriculaandusesseveralprevious

studiesasthebasistoexplorethevalueandeffectivenessofoneparticulardelivery


modelusingaparticularprojectmanagementsimulationgameasanexperiential

learningactivity.

Theoverarchingaimofthisresearchistodeterminehowstudentsperceivetheir

participationinaprojectmanagementsimulationgameandwhetherornotthisisa

valuedexperience.Mixedmethodresearch(MMR)techniquesareusedtoevaluate

studentperceptionsandanalysisabilitybeforeandafterexperiencingthesimulation

gameandtoexplorewhatstudentsbelievetheylearnedfromtheexperience.Although

theuseofMMRtechniquesisoftenunderpinnedbytheparadigmofpragmatism,the

philosophyofthisstudyispostpositivismasresultsarecomparedwiththefindingsof

anotherresearcherusingadifferentsimulationgameinadifferentacademicsettingto

positgeneralizability.

Thischapterdiscussestherelevanceofthisresearchinthecontextofthe

growingneedfordevelopmentofcompetentprojectmanagers,theneedforrelevant

projectmanagementeducationandthegapinman’sknowledgerelatedtoassessingthe

valueandeffectivenessofprojectmanagementsimulationgamesinaneducational

environment;describestheapproachtousingtheprojectmanagementsimulationgame

intheprogramunderstudyandexpandstheresearchaimintoresearchquestions;

introducesthesimulationgameusedinthestudy;discussestheapproachand

methodologybuiltonpriorstudiesofotherresearchers;statesthehypothesestestedin

thequantitativeaspectofthisstudy;anddescribestheorganizationofthisstudy.

Background

Wecareaboutprojectmanagementeducationbecausethereisagrowingneedto

developcompetentprojectmanagers.Thisisevidencedbytheexponentialgrowthof

membershipintheProjectManagementInstituteanditscertificationofProject

ManagementProfessionals(PMP®)andbythegrowthinthenumberofacademicproject

managementdegreeprograms.

The need for developing competent project managers.AccordingtoPMI®,the

ProjectManagementInstitute,(2010a),theneedforcompetentprojectmanagersis

immense:

InthePersianGulfandChinaSearegionsalone–whereentirecitiesarebeing

built,seeminglyovernight–ashortageof6millionskilledprojectprofessionals

isexpectedby2013.


PMI(ProjectManagementInstitute,2010a)furthersuggeststhereisatrainingand

educationgap:

Ofthe20millionpeopleparticipatinginprojectsworldwide,justonemillion

haveprofessionallyrecognizedformaltrainingonhowbesttoexecutethose

projects.

Inadditiontothetrainingandeducationgap,PMI(ProjectManagementInstitute,

2010b)claimsthereisanincreasingtalentgapduetoexperiencedprojectmanagers

beingclosetoretirementinadvancedmarketsandashortageoftrainedorexperienced

projectmanagersinmanyemergingeconomies.

Ifweaccepttheimpliedpremisethatprojectsaremorelikelytobesuccessfulif

theyareledbypeopletrainedandeducatedinprojectmanagement,takentogether

thesetwoclaimssuggesttheneedforprogramstotrainandeducateprojectmanagers.

Thetrainingandeducationprovidersappeartoberespondingtothisneed.Wehave

movedfromthebeliefthatprojectmanagement“hasfailedtocapturetheimaginationof

academics”(Maylor,2001)to“interestineducationandtraininginprojectmanagement

hasbeengrowingatanextremelyrapidpace”(Anbari,2010).Anbaribelievesthis

interestistheresultof“growingrecognitionoftheimportantcontributionsofproject

managementtoanorganizationalcompetitivepositioninthemarketplace,individual

careerprogress,economicandsocietaldevelopment,andremediesforshortcomingsof

projectoutputsandoutcomes.”Hisconclusionsappeartobesubstantiatedbythe

membership,certification,andaccreditationgrowthoftheProjectManagement

Institute.

Growth of project management as a professional discipline.Recognitionofproject

managementasaprofessionaldisciplineisevidencedbytheexponentialgrowthofthe

ProjectManagementInstitute(PMI),itscertificationofProjectManagement

Professionals(PMP®),anditsaccreditationofuniversityacademicprogramsinproject

managementbythePMIGlobalAccreditationCenterforProjectManagement(PMI‐GAC).

Foundedin1969byworkingprojectmanagers,PMIstates“ourprimarygoalisto

advancethepractice,science,andprofessionofprojectmanagementthroughoutthe

worldinaconscientiousandproactivemannersothatorganizationseverywherewill


embrace,value,andutilizeprojectmanagementandthenattributetheirsuccessestoit”

(ProjectManagementInstitute,2010d).

PMImembershipgrewslowlyafteritsfoundingand26yearslaterin1995,Iwas

membernumber59,000+andcertifiedPMPnumber5,300+.Morerapidgrowthbegan

inthelate1990s.Duringthesucceeding16yearsfrom1995to2012,PMIgrewtonearly

396,000activememberswithover500,000activePMPcredentialsasofNovember30,

2012.Whileittookover25yearstoaward5,000PMPcredentials,PMIcertifiesover

5,000insomemonths(ProjectManagementInstitute,2013b).

CandidatesforthePMPcredentialwithafour‐yearcollegedegreemust

documentatleast4,500hoursofprofessionalprojectmanagementexperienceleading

anddirectingprojecttasks,receiveaminimumof35contacthoursofformalproject

managementeducation,anddemonstratetheirabilitytoapplyprojectmanagement

conceptsandexperiencebypassingascenario‐basedexamination(ProjectManagement

Institute,2011).

Whiletherearemanyprovidersofferingexam‐prepcoursestofulfillthe35

contact‐houreducationrequirement,somepractitionersofprojectmanagementprefer

togobeyondmeetingtheminimumrequirementandinsteadattendanacademic

programleadingtoanacademiccertificateordegree.Whensearchingforan

appropriateprogram,manyincludereviewofPMI’swebsitelistingofaccreditedproject

managementdegreeprogramsaspartoftheirsearch.

Accreditation of project management academic degree programs.PMIestablished

itsGlobalAccreditationCenter(GAC)in2001as“anindependentacademicaccreditation

bodytoadvanceexcellenceinprojectmanagementeducationworldwidethrough

collaborationwithandsupportofacademicinstitutions,andthroughaccreditationof

academicprograms”(ProjectManagementInstitute,2009).TheGACreportsthatit

found2bachelor’sleveland9master’slevelacademicprojectmanagementprograms

activein1994,primarilyinthefieldofconstructionmanagement(ProjectManagement

Institute,2010c).In2009thisincreasedtoafindingof640degreeprogramsofferedby

456worldwideinstitutionsinmultiplefieldsofstudy.

Figure1depictsthegrowthinthenumberofuniversitieswithGAC‐accredited

programsinprojectmanagementbyregion(NA=NorthAmerica,EMEA=Europe‐Middle

East‐Asia,AP=AsiaPacific,LA=LatinAmerica).AsofOctober31,2012,therewere40


universitiesin13countrieswith86GAC‐accrediteddegreeprograms(Project

ManagementInstitute,2012).

FIGURE1–UNIVERSITIESWITHGAC‐ACCREDITEDPROGRAMSBYREGION

Despitethisgrowthinthenumberofacademicprogramsrelatedtoproject

management,PMIisconcernedtherearenotenoughacademicandtradeinstitute

programsinprojectmanagementtobridgethegap.Theyalsopointoutthatdeliveringa

successfulprojectmanagementeducationalprogramrequiresacarefulselectionof

instructors,curriculum,andstandards;combinedwithafirmcommitmentandquality

resources(ProjectManagementInstitute,2010b).

Projectmanagementeducationprogramsaretypicallyfoundintechnology

schoolsanduniversitycollegesofengineeringandmanagement(business).This

researchconsidersevaluationofoneaspectofeducationalquality,theeffectivenessof

usingaprojectsimulationgameasaformofexperientiallearninginoneuniversity’s

graduatemanagementprogram.

The value of management education.Althoughtherelevanceofmanagement

educationprogramsisoftencriticizedinthepopularpressbyrespectedacademicslike

JeffreyPfefferandHenryMintzberg(Starkey&Tiratsoo,2007),Yeaple(2006)found

MBAprogramsremainpopularbecausetheyprovidevalue:“Asurveyof3,771MBA

graduatesfindsthateightyearsaftergraduation,93%agreetheirgraduatemanagement

educationwasworththetimeandinvestment.”

SimilartothegrowthcurvesofPMImembershipandcredentialedPMPsisthe

growthofmaster’sdegreesinbusinessawardedbyU.S.universities.Theseincreased

fromabout3,000inthelate1950’stonearly130,000intheearly2000’s;withMBA’s

0

5

10

15

20

25

30

35

40

45

2002

2003

2004

2005

2006

2007

2008

2009

2010

2011

2012

LA

AP

EMEA

NA


nowaccountingfor25%ofthemaster’sdegreesawardedintheUnitedStates(Dennis&

Smith,2006).DennisandSmithnote“themanager’scareernowhastheattributesofa

profession”andentryintothecareerdependsonvalidationofasetofcredentials,the

MBAdegreebeing“theentryticketorunioncardforemployeesonthemanagement

track.”

Withover500,000activeProjectManagementProfessionals(PMPs)credentialed

byPMI,projectmanagersmayfindtheyneedanadvanceddegreeinproject

managementoranMBAtofurtherdeveloptheirknowledge,skillsandabilitiesandto

differentiatethemselvesfromthosePMPswhoseonlyprojectmanagementtrainingor

educationwasa35contact‐hourPMPexampreparationbootcamp.

Thechallengeforeducationalinstitutionsistogobeyondteachingthe

fundamentalsandtoprovidealearningexperiencethatpreparesstudentstoactually

manageprojects,leadprojectteams,anddealwithavarietyofethicaldilemmas.The

aimofthisresearchistodeterminewhetheraparticularprojectmanagementsimulation

gamehelpsdothisinagraduatelevelmanagementdegreeprogramdesignedspecifically

forworkingprofessionals.

InsharpcontrasttothecriticismsofferedbyPfefferandFong(2002)and

Mintzberg(2004),Yeaple(2006)notedthatpart‐timeprogramsprovidedhigher

financialgainstotheparticipantsthanfull‐timeprogramsatthesameschoolsbecause

theydidn’tinvolvetheopportunitycostoflostincomewhileattendingschool.Inan

informalsurveyofPMI‐GACaccreditedprojectmanagementmaster’sdegreeprograms,I

observedthatmanyarepart‐timeandtargetedatworkingadults.Inmyconversations

withpart‐timeEMBAstudents,Ifindthemtobeveryprotectiveoftheirtimeandto

expecttheircourseworktoberelevantandapplicable.Theeducationalvalidityand

theirperceivedreturnoninvestmentfromplayingasimulationgameisofconsiderable

importance,especiallyifthesimulationexperienceisusedasacapstoneandisoneof

theirlastexperiencesintheprojectmanagementphaseofaprogram.

Yeaple(2006)claimsthatgoingtobusinessschooldoesnotaddvalue,it

multipliesit–fortalentedandmotivatedstudentswhoapplythetoolslearnedwithskill

andenthusiasm.Heconcludes“thevalueofagivenschool’sMBAdegreeisthereforethe

productofitsincomingstudentqualitytimesthequalityofitseducationalprocess.”This

studyexploresthequalityofoneaspectoftheeducationalprocessforanEMBAprogram

withanemphasisinprojectmanagement:theuseofasimulationgameasanexperiential


learningcapstoneactivityattheconclusionofaseriesofprojectmanagementcore

courses.

Experiential learning and business games.Experientiallearningactivities(ELA)

andsimulationgamesareoftenfoundinMBAprograms.Therelevanceandimportance

ofexperientiallearningwasmadebyKolb(1984)inhisoftencitedseminalbookon

experienceasthesourceoflearninganddevelopment.Aformofexperientiallearning,

simulationgamesemphasizingtheapplicationoftoolsareidealfordevelopingproject

managers“providedtheyhighlightthemodusoperandiandproblemsolvingmethodsof

projectmanagers”(Hutcheson,1984).Simulationsinbusinesseducationhelpbridgethe

gapbetweenthetraditionaleducationalparadigmfocusedonthetransmissionof

knowledgeandthedisciplineswheretheapplicationofknowledgeintheperformanceof

realtasksisneeded(Larréché,1987).Overtheyears,“simulationshavebeenusedwith

increasingfrequencyinthedevelopmentofmanagementtalent”(Thornton&Cleveland,

1990).

Fariaetal(2009)reviewedallthearticlespublishedinSimulationandGaming

fromthefirstissuepublishedinMarch1970throughtheSeptember2008issueand

foundthat304outof1,115fullarticles(27%)coveredsomeaspectofbusiness

simulationgameeducationandlearning.Theyidentifiedninecentralthemesastowhy

educatorsusebusinesssimulationgames,tabulatedtheseinorderoffrequency

mentionedbydecade,andnotedthatthetopfivetopicsappearedinthetopfiveofeach

decade.Thesewere:

experiencegainedthroughbusinessgames

thestrategyaspectsofbusinessgames

thedecision‐makingexperiencegainedthroughbusinessgames

thelearningoutcomesprovidedbybusinessgames

theteamworkexperienceprovidedthroughbusinessgames(Fariaetal.,2009)

Theynotedthatifoneassumedacorrelationbetweenthearticlesappearingin

SimulationandGamingandthereasonswhyeducatorswereusingbusinessgames,the

reasonshave“remainedremarkablythesameduringthepast40years”(Fariaetal.,

2009).Theyalsofoundthat“inthe2000’sexperienceasanarticletopicjumpedfrom

thirdplacetofirst,andlearningobjectivesandoutcomesmovedfromfifthplaceto

third.”Fariaetal.attributethisshifttothetrendofaccreditingorganizations,suchas

AACSBInternational‐TheAssociationtoAdvanceCollegiateSchoolsofBusiness


(AACSB),toemphasizelearningoutcomemeasurement.ThePMI‐GACsimilarlyrequires

anoutcomes‐basedassessmentoflearning(ProjectManagementInstitute,2009).One

opportunityforfurtherresearchistodeterminewhatlearningoutcomesareintendedby

educatorsusingprojectmanagementsimulationsasexperientiallearningactivitiesin

theirdegreeprograms.

Whenaskedwhyasimulationgamewasoriginallyincludedinthecurriculum

understudy,thedegreeprogramfoundingdirectorsaid“forstudentstopulltheir

learningtogetherintoanintegratedandfunactivity.”Whilesuchanobjectivemightbe

viewedwithskepticismbyanaccreditationsitevisitor,itsuggestsblindacceptanceof

thevalueofsimulationgamesinacademicprograms.Thecurrentsyllabuscallsforthe

studentstodemonstratetheirabilitytoworkasateamtoplanandexecuteasimulated

project.

Non‐specificlearningobjectivesandqualitativeassessmentofthisformof

experientiallearningmaybethenorm.Fariaetal(2009)observethat:

Debriefinghasgrowntremendouslyasatopicofinterestinsimulationresearch

inthepastdecade.Thediscussionofthelearningintentofbusinesssimulation

exercisescoupledwithfeedbackfromthestudentsastowhattheyhave

experiencedandlearnedhasalwaysbeenacentralpartofbusinesssimulation

gamingresearchthroughthedecades.

AnotedcriticoftraditionalMBAprograms,HenryMintzberg(2004)arguesthat

“conventionalMBAprogramstrainthewrongpeopleinthewrongwayswiththewrong

consequences”andexecutive(EMBA)programs“takemoreexperiencedpeopleonapart

timebasis,andthendomuchofthesamething....traintherightpeopleinthewrong

waysandwiththewrongconsequences.Thatisbecausetheymostlyfailtousethe

experiencethesepeoplehave.”However,peopledorelyontheirexperienceswhen

playingasimulationgame,andwhileMintzbergarguesthattheartificialexperience

providedbycomputersimulationsandroleplayingareatthelowendofthe

authenticity‐methodscale,hedoesacknowledge“thebusinessgamecanhavean

appropriateroleinthebusinessschool”asaneffectivemethodforillustratingconcepts

andforstudentstolearnhowtoapplytheseconcepts.Hisconclusionisbusinessgames

areacapstoneforwhattheMBAteaches,buthisconcernisthetypicalMBAdoesnot


teachmanagement.Theintentofthisstudyisnottotakeissuewithhisconclusions,but

todeterminewhethertheuseofaparticularprojectmanagementsimulationgameasa

capstoneexperientiallearningactivityinagraduatemanagementprogramreturns

benefitscommensuratewiththetimespentplayingit.

Assurance of learning.Collegiateandprofessionalaccreditingagenciessuchas

theSouthernAssociationofCollegesandSchools(SACS)CommissiononColleges,AACSB

andGACallhavestandardsrelatedtoassuranceoflearning.Whiletheydonotspecify

howlearningistobeassured,theyrequireevidenceoflearningassessment,attainment,

andcontinuousimprovementforinitialaccreditationandonarecurringbasisthereafter.

SACS(2009)isaregionalaccreditingbodyfocusingontheoverallinstitutionand

referstoitsU.S.SecretaryofEducationrecognition“underTitleIVofthe1998Higher

EducationAmendmentsandotherfederalprograms”initsFederalmandaterequirement

dealingwithstudentachievementtoaffirm“theinstitutionevaluatessuccesswith

respecttostudentachievement”and“theinstitution’scurriculumisdirectlyrelatedand

appropriatetothepurposeandgoalsoftheinstitutionandthediplomas,certificates,or

degreesawarded.”UnderitsInstitutionalEffectivenessstandarditrequires“the

institutionidentifiesexpectedoutcomes,assessestheextenttowhichitachievesthese

outcomes,andprovidesevidenceofimprovementbasedonanalysisofresultsin…

educationalprograms,toincludelearningoutcomes.”

AACSB(2007)focusesondegreeprogramsandisinterestedinprogram‐level

learninggoalsthatarebroaderinscopethanindividualcoursegoals,noting“each

specializedmaster’sprogramwillneedauniquesetof4‐10learninggoals;however,

somemaybethesameacrossallsuchprograms.”Thesebroadlearninggoalsshould

addressbothgeneralknowledgeandskillssuchascommunicationsandethical

reasoningaswellasknowledgeandskillsthatdirectly“relatetomanagementtasksthat

formthebusinessfoundationofdegreerequirements.”Thesegoalsarerestatedasone

ormorelearningobjectiveswithmeasurableattributes.Forexample,theLearningGoal

of“ourgraduateswillbeeffectivecommunicators”mayhaveasoneofitsobjectives,

“studentswilldemonstrateanacceptablelevelofwrittencommunicationskills.”An

exampleAssessmentMeasureis,“anessayisrequiredforadmissionwhichisevaluated

forwritingcompetencies.”

SimilartoSACSandAACSB,PMI(2010c)states“theprogramshallhaveclearly

statedlearningoutcomes…thataredirectlyrelatedtothestatedmissionandobjectives


[oftheprogram].”Butitgoesfurtherandnotonlydefinesalistof14topicsthatwould

“normallybeexpectedtobepartoftheprogramorprerequisiteknowledge”butalso

lists5areasoffocuscontainingatotalof39learningoutcomesthatprogramsapplying

foraccreditationmustaddressinitsself‐evaluationreport.AkeycomponentoftheGAC

accreditationprocessisdescribingandbeingauditedonhowtheuniversityassures

studentsarepreparedtotakeresponsibilityforthetasksdescribedinthelearning

outcomes.

Sincesimulationgamesareexperientiallearningactivitiesattemptingtomodel

realityandmayrequireasignificantnumberofcoursestudyhourstoimplement,one

wouldexpectthattheywouldhelpsatisfytherequirementforassuranceoflearning.

Researchdemonstratingthelearningeffectivenessofprojectsimulationgamescanhelp

justifytheiruseaspedagogicaltooltopreparestudentstotakeresponsibilityforthe

tasksdescribedinlearningoutcomes.

Simulation games as experiential learning activities.Theuseofbusinessgames

andsimulationsasatoolforprovidingexperientiallearningisnotnew.Thefirst

practicalbusinessgameintheUnitedStateswasintroducedbytheAmerican

ManagementAssociation(AMA)overahalfcenturyagoin1956(Biggs,1990;Dillman&

Cook,1969;Meier,Newell,&Pazer,1969;Taylor&Walford,1978).AMA’sTop

ManagementSimulationwasimmediatelyrecognized“asanimportantnewapproachto

jobinduction”and“almostovernight,gamingbecameapopulartrainingactivityfor

universitiesworkinginthisfield”(Taylor&Walford,1978).FariaandWellington

(2004)foundthatthepercentageofrespondingAACSBmemberschoolsusingatleast

onebusinesssimulationgrewfrom71.1%in1962to97.5%in1998.Although,usage

rateswerehighestinstrategicmanagement/businesspolicyandmarketingcourses,

usagewasalsoreportedforthecategoriesoffinance,management,accounting,and

otherbusiness.

Faria(1998)foundthat,onaverage,25.1%ofthecourse’sgradewasbasedon

thesimulationgameand23.8%ofclasstimewasdevotedtoplayingit.Thiscontrasts

with25%ofthegradeand40%oftheclasstimefortheprogramstudiedbymy

research.Fariaalsofoundthatfacultyusingbusinesssimulationgamesratedthem

highestinperceivedteachingeffectiveness,overlectures,cases,andtextbooks;but

facultywhohadneverusedabusinesssimulationgamerankedthemlowest.Thiswould

suggestthatnotmuchhaschangedsinceBiggs(1990)wrote:


Giventherapidgrowthonemightassumethattheeducationalmeritsofbusiness

gamesarewellestablished.Thefactis,however,thattheireducationalmerits

havebeensubjecttoconsiderabledebate.Therearestudieswhichindicatethat

otherformsofpedagogyarejustaseffectiveormoreeffectivethanbusiness

games,whileotherstudiesfindthereversetobetrue.

Whilethecomparisonoftheuseofbusinessgamesoraparticularbusinessgame

versusotherpedagogiesisaninterestingopportunityforfurtherresearch,theaimof

thisresearchislimitedtoexploringwhetheraparticularsimulationgameascurrently

deployedisagooduseoftimeinanexistingacademicprogram.

Project management simulation games.Usingsimulationtodevelopproject

managersisalsonotnew.DillmanandCook(1969)describethedevelopmentand

evaluationofanon‐computer‐basedprojectmanagementsimulationthatwasusedto

developresearchanddevelopmentprojectmanagers.Itgrewoutofaneed“foramore

realisticandbroaderbasedexercise”thanwasbeingusedinanexistingtraining

program.Thedevelopersbelieved“increasedrealismandgreaterinvolvementby

participantswouldbeobtainedbytheuseofasimulatedsituationwhichdemanded

morecomplexskills,somedegreeofroleplaying,andincreasedopportunityfordecision

making.”Theirsimulationrequiredabouthalfofaone‐weektrainingprogram,roughly

equivalenttohalfofthecontacttimefoundinatypical3semestercredithour(SCH)

collegecourse.Theyfound“overallpositivereactionstowardtheuseofsimulationin

thetrainingprogram”andconcluded“simulationappearstobeaverypromisingtoolin

thetrainingofR&Dprojectmanagers.”Theseconclusionswerebasedonparticipant

feedbackratherthanmeasurementofperformancegainsversusspecificlearning

objectives.

Inaninterestingsidenote,Dr.Cook(1976)lateropinedinanAcademyof

ManagementJournalarticlethathumanrelationsandcommunicationswere

underrepresentedinasurveyheconductedonthe“currentstatusofproject

managementinstructioninAmericancollegesanduniversities.”Bothofthesetopicsare

importantaspectsintheplayingoftheprojectmanagementsimulationgameunder

study.


However,morethantwentyyearslater,KeysandBiggs(1990)makenomention

ofprojectmanagementsimulationgamesintheirchapterreviewingcomputerized

businessgamesavailablefromwell‐knownpublishersintheABSELGuidetoBusiness

GamingandExperientialLearning(J.W.Gentry,1990a).Discussedaretotalenterprise

businessgamesandfunctionalbusinessgamesforaccounting‐finance,marketing,

personnel‐humanresourcesandproduction/operations.Afurthersearchofthe

literaturerevealedveryfewarticlesrelatingspecificallytoprojectmanagement

simulationgames.

SimProject,thesimulationusedinthisstudy,firstbecamewidelyavailablefroma

well‐knownpublisherasanoptionalbundlewithGrayandLarson’ssecondedition

textbook,ProjectManagement:TheManagerialProcess(Gray&Larson,2003),remained

therethroughthefourthedition(Gray&Larson,2008)andwasdroppedfromthefifth

edition(Larson&Gray,2011)afterthesimulationauthorscancelledtheircontractwith

thepublisher(thesimulationisnowdirectmarketedbytheauthors).Anappendix

suggestinguseofthesimulationthroughoutthecourseofstudyappearedinthesecond

throughfourtheditionsofthetextbookandofferedexercisestohelpintegratethe

simulation“intothe‘fabric’oftheclass”to“bringhometheapplicationofthetoolsand

techniquesofprojectmanagement“(Gray&Larson,2008).

Severalarticleswerefoundexaminingtheuseofprojectmanagementsimulation

gamesinacademicprograms.Thesevariedinmethodsfromqualitativestudiesrelying

onpostsimulationstudentcommentsandinstructorobservations(S.Al‐Jibouri,

Mawdesley,Scott,&Gribble,2005;S.H.Al‐Jibouri&Mawdesley,2001;Collofello,2000;

L.S.Cook&Olson,2006;J.M.Cooper,2011;Dantas,Barros,&Werner,2004;Dillman&

Cook,1969)toquantitativestudiesfollowingamorerigorousresearchmethodology

(Davidovitch,Parush,&Shtub,2006;Davidovitch,Shtub,&Parush,2007;Davidovitch,

Parush,&Shtub,2008;Davidovitch,Parush,&Shtub,2009;Davidovitch,Parush,&

Shtub,2010;McCreery,2003;Pfahl,2004).ThesearediscussedintheLiteratureReview

chapter.

Oneevaluationtechnique,theuseofstudentperceptiongainsasthebasisfor

measuringeffectiveness,wasusedbyMcCreery(2003)inhisinvestigationintotheuse

ofacomputer‐basedprojectmanagementsimulationgameinagraduateproject

managementcourse.McCreeryfound“thesimulationexerciseimprovesparticipant

knowledgelevelsaswellastheabilityofparticipantstoapplythatknowledge.”


McCreery’sresearchwasconductedtoexploretheacademicuseofasimulation

originallydesignedforuseincorporateenvironmentsbyafor‐profittrainingcompany.

Buzzetto‐MoreandMitchell(2009)similarlyassessedtheeffectivenessofa

businesssimulationandfound“studentsoverwhelminglyfeltthesimulationhelped

themunderstandtheapplicationofkeyconceptsandlearnthedecisionmakingprocess

thatoccursinprofessionalbusinesspractice.”

Ahn(2008)investigatedtheuseofabusinessgameinanentrepreneurship

courseandfoundapositivecorrelationbetweenstudentperceptionsofthelearning

experienceandtheirgameperformancescores.

ThisresearchstudybuildsonMcCreery’s(2003)approachtomeasuretheself‐

reportedgainofknowledgeandabilitytoapplythatknowledgeasaresultofthe

simulationandexploresitscorrelationwithincreasedabilitytodevelopandanalyze

simpleschedulesandinterpretprojectearnedvaluedata.Italsoincludesanadaptation

ofBuzzetto‐MoreandMitchell’s(2009)andAhn’s(2008)questionnairestoassess

studentexpectationsandsatisfactionwiththesimulationgameexperience.

Why this research is important.Usingasimulationgameaspedagogycanrequire

manycontacthoursinacourse,henceitisimportanttoaffirmalearningbenefit

commensuratewiththetimeinvested.However,assessingtheeffectivenessofa

simulationgameasalearningtooliscomplicatedanddifficult.Gameresultsand

learningareafunctionofmanyvariablesincludingthegamemodel,participant

intelligenceandpersonality,instructorrole,gameadministrationprocess,numberof

periodsofplay,and,forteamsimulations,teamsizeandinteraction(Greenlaw&

Wyman,1973).

Earlyresearchcomparedsimulationgamingversusotherlearningmodes.

GreenlawandWyman(1973)comparedlearningtomeetcourseobjectivesusinggames

versusothermodesofteachingsuchascasestudiesandconsideredbothmultifunctional

generalbusinessgamessimulatingtopmanagementdecision‐makingandmono‐

functionalgamesemphasizingonlyonebusinessfunction.Theyobservedthat,although

theauthorsoftheresearchtheyreviewedwereveryenthusiasticabouttheirfindings,

“verylittle‘hard’researchhasbeendoneongaming–especiallyconcerningwhat

playerslearntomeetcourseobjectives,whichwasourprimeinterest.”Reasonsfound

forthisincludedthedifficultyindesigningavalidresearchmethodologywithmany

“soft”and“hard”variablestocontendwithandthebeliefthatgamesmaybeusedto


teach“intangible”conceptsandprovide“awareness”ratherthanteachspecificfactsand

relationships.

Despitetheconcernoverwhetherlearningreallyoccurswhileplayinga

simulationgame,Waggener(1979)foundthatstudentspreferexperientiallearning

techniquesoverthetraditionaltextbookapproachandprefersimulationsovercase

studiesasaformofexperientiallearning.

GreenlawandWyman(1973)furtherfoundthat“strong‘learning’ingamesmay

notnecessarilybereflectedbygoodgamingperformance”and“converselyabsenceof

learningmaynotalwaysbereflectedby‘poor’performance.”Somethingotherthan

simulationresultsmustbemeasuredtodetermineeffectivenessandlearning.In

contrasttotheconclusionsfoundinsomeoftheresearchtheyreviewed,Greenlawand

Wymanconcluded“theeffortandexpenditureswhichhavethusfarbeeninvestedin

developingbusinessgameshavenotbeenjustifiedbytheknowledgeofspecificallywhat

gamesteach,ifanything.”Ratherthancreatingnewsimulationgames,theysuggested

moreemphasisonresearchingexistinggamesisneeded.Thisstudymakesa

contributioninthatregard.

Taylor(1978)laterobservedthat“presentinformationconcerningthelearning

impactofsimulationisfragmentedandbasedmoreonhunchandgeneralimpression,

thanonsystematicvalidatedresearchstudy.”However,hefoundnostudiesclaiming

thatsimulationwasanyworsethananyothertechniqueinteachingfactualmaterial.He

alsosuggestedthatmanyusersofsimulationswouldnotwishtoevaluateitslearning

possibilitiesseparatelyfromtheotherlearningmodesinagiventeachingunit.Rather

“theywouldarguethatthesimulationactsasastimulustosubsequentlearningandthat

thisspin‐offinterestcanbeproperlyconsideredaspartofthebenefitofthetechnique,

eventhoughitmaybedevelopedthroughmoretraditionalmethodsoflearning.”

TwelveyearslaterKeysandWolfe(1990)“found60fairlyrigorousstudiesthat

provideevidencetobusinessgames’generalyetproblematiceducationalefficacy”and

observed:

Asequivocalasthesefindingsare,manyoftheclaimsandcounterclaimsforthe

teachingpowerofbusinessgamesrestonanecdotalmaterialorinadequateor

poorlyimplementedresearchdesigns.Theseresearchdefectshavecloudedthe


businessgamingliteratureandhavehamperedthecreationofacumulative

streamofresearch.

Followingtheirreviewoftheliteratureontheeducationalvalueofmanagement

games,KeysandWolfe(1990)observe“managementgameshavebeenfoundtobe

generallyeffectiveinthestrategicmanagementtypecourse”and“someusefulresearch

isemergingintheothermanagementareasthoughnotasextensiveasinstrategic

management.”Theyconclude,“Thereisagreatneedforcontinuityofresearchinthe

managementgamingarea,andforfreshnewapproachestoresearchdesignwheregaps

areapparent.”Myresearchhasfoundlittlewrittenspecificallyabouttheuseofproject

managementsimulationgamesineducationandisacontributiontothebodyofresearch

inthis“othermanagementarea”andprovides“somecontinuityofresearch”tothe

projectmanagementsimulationgameresearchreportedbyMcCreery(2003).

Research Aim

Thegapinman’sknowledgeisthevalueandeffectivenessofusingthePintoand

Parente(2003)SimProjectprojectmanagementsimulationgameasacapstoneactivity

inagraduateEMBAprogramwithanemphasisinprojectmanagement.Thepurposeof

thisstudyistodetermineiftheuseofthissimulationprovidesvaluetothestudentand

isaneffectiveuseofclasstime.Theresultswillbeofinteresttoprojectmanagement

educatorsusingorcontemplatingtheuseofaprojectmanagementsimulationgamein

theircurriculum.

IntheEMBAprogrambeingstudied,thesimulationisusedasacapstoneactivity

attheconclusionofa21SCHprojectmanagementcorecurriculum.Theapproachused

introducesthescopeofthesimulatedprojectalongwithsomeinitialplanningestimates

intheformofateamassignmentduringthefourthofsixsequentialprojectmanagement

coursesandtheprojectissimulatedasamajorcomponentofthesixthsequentialproject

managementcourse.

Duringthefourthsequentialcourse,studentteamsfirstdevelopabaseline

scheduleandbudgetbasedongiveninformation,thendevelopastaffingmanagement

planandrevisedscheduleandbudgetbasedontheiranalysisofavailableresources(see

AppendixBforteamassignmentinstructions).Mostofthisworkisdoneashomework

outsidetheclassroom.


Duringthesixthsequentialcourse,sixteenhoursofcontacttime(fortypercentof

thecontacthoursforthecourse)areusedtoexecutethesimulatedprojectanddiscuss

theresults.Duringexecution,theteamsattempttoobtaintheresourcesidentifiedin

theirstaffingmanagementplan,assignthemtotasks,monitorperformance,and

implementchangesasneededinanattempttocompletetheprojecton‐timeandwithin

budget.Thisisasubstantialportionofthecontacttimeforthesixthcourseanditis

importanttoknowwhetherornotthisactivityisavaluablelearningexperience.

Theresearchquestionsforthisstudyare:

Dostudentself‐assessmentsofprojectmanagementknowledgeand

confidenceintheirabilitytoapplythatknowledgeincreasefollowingthe

simulationexperience?Further,areMcCreey’s(2003)resultsrepeatable

inadifferentcontextwithadifferentsimulationgame?

Howdostudentopinionsregardingtheirteamexperienceintheprogram

andgroupworkingeneralchangeasaresultofthissimulation

experience?

Howdostudentopinionsregardingtheuseofsimulationsasalearning

toolchangeasaresultofthissimulationexperience?

Arestudentsbetterabletodevelopandinterpretschedulenetwork

diagramsandanalyzeearnedvaluedataasaresultofthesimulation

experience?

Whatdostudentsfindvaluableabouttheexperience?

Accordingly,theunitofanalysisforthisstudyistheindividualstudent.

Sinceincludingasimulationgameaspartofacoursecurriculumcanrequirea

substantialamountoftimeandadditionalcost,thisresearchshouldbeusefultoanyone

consideringtheuseofaprojectmanagementsimulationgameasalearningtoolin

professionaltrainingaswellasinacademiceducationprograms.Thisstudyis

consideredexploratoryandidentifiesopportunitiesforcontinuedresearch.

SimProject – the simulation under study

TheprogramunderstudyusesSimProject,developedbyDr.JeffreyPintoandDr.

DianeParenteofThePennsylvaniaStateUniversity,forthesimulationgameexperiential

learningactivity.Thissimulationisusedbytheprogrambecauseofitsmentioninthe

prefaceandappendixofaprogramtextbook(Gray&Larson,2008)andbecauseofits


successfulapplicationaspartofprofessionaldevelopmenttrainingprogram

implementedforacorporatecustomerpriortotheonsetofthisstudy.

Accordingtothedevelopers,SimProjectprovides“virtual‘first‐hand’experience

inmanagingprojects”(SimProfessionals,2009):

Computersimulationsencourageteamdevelopment,collaboration,global

thinking,andapredilectiontoconsidertheramificationsofdecisionsandtheir

effectonthebottomline–inotherwords,manyoftheskillsthatareusefulto

projectmanagersandteammembersinbusiness.Thepurposeofthissimulation

istotietogethermanyofthesalientchallengesofprojectmanagementinorder

togivestudentsthedeepestpossibleunderstandingofthecomplexitiesinvolved

inundertakingaproject.Thegoalofthesimulationwillbetohavestudents

manageaprojectfrominitiationtocompletion.Withinthisframeworkthe

studentwillneedtoemployanddevelopskillspertinenttopersonnelselection

andtraining,motivation,conflictmanagement,andstakeholdermanagement.

Studentswillberequiredtouseplanningandschedulingtechniques,suchas

workbreakdownstructures,PERT/CPM,scopedevelopment,andriskanalysis

(SimProfessionals,2009).

Thissimulationgameistypicallyplayedbystudentteamswhodevelopaplan

involvingfourtypesofdecisions:resourcehiringandrelease,resourcetrainingto

improveexpectedperformance,managerialactionstoinfluenceresourceperformance,

andassignmentofresourcestoactivities(Pinto&Parente,2003).Thestudentteams

analyzethegiveninformationforacommonproject,competeagainsteachotherfor

acquisitionofresourcesfromacommonresourcepool,assigntheseresourcestothe

simulatedproject’stasks,andreceivefeedbackonthesimulatedprojectteam’s

performance.Thesimulatorprovidesactualcostandtaskdurationinformation

followingeachsimulationroundandrankseachteam’sperformanceinfourcategories:

Cost,Time,Functionality,andStakeholderSatisfaction.

Approach and Methodology

Thisstudyusesmixedmethodresearch(MMR)techniqueswithanemphasison

buildingonthequantitativeresearchofMcCreery(2003),Ahn(2008),andBuzzetto‐

More&Mitchell(2009)tomeasurestudentperceptionsofvalueandeffectivenessofthe


simulationexperiencerelatedtoknowledge,confidence,teamworkandthepedagogical

useofsimulationgames.Thequalitativeaspectofthisstudyassesseswhatstudents

claimtheylearnedfromtheexperience.Togethertheresultsanswertheoverarching

researchquestionofhowstudentsperceivetheirexperienceintheprojectmanagement

simulationgameandwhattheybelievetheylearnedfromit.AlthoughMMRresearchis

mostoftenassociatedwiththeparadigmofpragmatism(Tashakkori&Teddlie,2010b);

thisresearchtakesasingleparadigmstanceunderpinnedbypostpositivismowingtoits

comparisonwiththeresultsofMcCreery(2003)andsuggestionofgeneralizability.

McCreery(2003)foundthatusingateam‐basedprojectsimulationaspartofa

graduatelevelcourseinprojectmanagement“improvesparticipantknowledgelevelsas

wellastheabilityofparticipantstoapplythatknowledge.”Ahn(2008)foundapositive

correlationbetweenstudents’simulationgameperformanceandtheirperceptionofthe

learningexperience.Buzzetto‐MoreandMitchell(2009)foundthatusingacapstone

simulation“enhance[d]theoveralleducationalandpersonaldevelopmentexperiences

ofminoritystudentsenrolledinhighereducationbusinessprograms.”

McCreery(2003)usedalongitudinalapproachofpre‐andpost‐simulationself‐

assessmentwithLikert‐typeratingscalestodetermineifthereweresignificantchanges

inself‐perceptionsofprojectmanagementknowledgeandabilitytoapplythis

knowledgeovertheperiodofexecutingtheprojectsimulationandpostsimulation

questionsregardingtheteamexperience.Ahn(2008)andBuzzetto‐MoreandMitchell

(2009)usedpostsimulationinstrumentstoassessperceptionsontheuseofsimulation

gamesasalearningtool.

Thequantitativeapproachusedinthisresearchexpandsonaspectsof

McCreery’s(2003)longitudinalself‐assessmentapproachbyaddinglongitudinal

adaptationsofMcCreery’steamworkquestionsandAhn’s(2008)andBuzzetto‐Moreand

Mitchell’s(2009)simulationperceptionquestions,andbyaddingpre‐andpost‐

simulationquestionsassessingstudentknowledgeandabilitytoperformproject

schedulecriticalpathanalysisandinterpretearnedvaluedata.

Thequalitativeapproachexaminespostsimulationquestionnaireresponses

regardingthesimulationexperienceandprogramfeedbacksurveysissuedattheendof

theProjectManagementCorePhaseandaftergraduation.Responsesarecategorizedto

identifyemergentthemesdescribingwhattheylearnedfromtheexperience.


Twoon‐campuscohortsofEMBAwithanemphasisinprojectmanagement

studentsaretheprimaryfocusofthisstudy.Datawerealsocollectedandanalyzedfor

twogroupsofsimilaradultlearners.Thefirstgroupparticipatedinacorporate

professionaldevelopment(non‐academic)programwithcontentsimilartotheEMBA

projectmanagementcorecoursesbutwithouttherigorofacademicassessment.This

groupservedasapilotfortheresearchmethodology.Theothergroupparticipatedina

singleprojectmanagementoverviewcourseinanotherprogramanditsdatawillbeused

infutureresearchstudies.Allgroupscompletedthesamepresimulationpreparatory

exerciseandencounteredasimilarsimulationexperience.

Hypotheses Tested

Thequantitativeapproachtestssevenhypothesesclaimingparticipantswill

perceiveordemonstrateasignificantincreaseintheirperceptions,attitudesand

abilitiesafterparticipatingintheprojectmanagementsimulationgameexperiential

learningactivity.

Hypotheses1and2explorewhetherMcCreey’s(2003)findingofsignificant

gainsinprojectmanagementknowledgeandabilitytoapplythatknowledgeare

repeatablewithadifferentsimulationinadifferentsetting.

H1: Participantswillassesstheirprojectmanagementknowledgelevelhigher

aftercompletingthesimulationgame.

H2: Participantswillassesstheirabilitytoapplyprojectmanagement

knowledgehigheraftercompletingthesimulationgame.

Hypotheses3and4explorewhethertherearesignificantdifferencesin

participants’opinionsoftheirsimulationteamandofteamprocessesingeneralafter

experiencingthesimulation.

H3: Participantswillreportahigheropinionoftheirteamexperienceafter

completingthesimulationgame.

H4: Participantswillreportahigheropinionofgroupprocessesingeneralafter



ExpandingontheworkofAhn(2008)andBuzzetto‐MoreandMitchell(2009),

Hypotheses5and6explorewhethersignificantgainsinopinionontheuseof

simulationsaslearningactivitiesandtheprojectmanagementsimulationinparticular

arefound.

H5: Participantswillreportahigheropinionontheuseofsimulationsasa

learningtoolaftercompletingthesimulationgame.

H6: Participantswillreportahigheropinionofthisspecificproject

managementsimulationaftercompletingthesimulationgame.

Hypothesis7exploreswhethersignificantgainsintheabilitytosolvetypical

projectmanagementcriticalpathandearnedvaluemanagementanalysisproblemsare

found.

H7: Participantswillbebetterabletosolveprojectscheduleandearnedvalue

analysisproblemsaftercompletingthesimulationgame.

Organization of this Study

Thischapterdemonstratestherelevanceofthisresearchbydiscussingthe

growingneedforcompetentprojectmanagementprofessionals,theneedforrelevant

projectmanagementeducation,andthegapinman’sknowledgerelatedtomeasuring

thevalueandeffectivenessofprojectmanagementsimulationgamesinaneducational

environment.Fiveresearchquestionsareidentified,andfollowinganintroductionof

thesimulationgameunderstudyandtheresearchapproach,thesevenhypotheses

associatedwiththequantitativeresearchquestionsarelisted.

TheLiteratureReviewchapterdescribesthesearchmethodology,providesa

historicalperspectiveonkeydefinitions,discussesthetheoreticalunderpinningof

experientiallearningandtheuseofsimulationgamesasexperientiallearningactivities,

exploresbusinesssimulationgameandprojectmanagementsimulationgameresearch

toidentifypracticesrelatingtoeffectivenessevaluation,andconcludesadditional

researchisneeded.

TheMethodologychapterdiscussesthephilosophicalapproachtakenand

providesdetailsontheresearchapproach,researchdesign,surveydesign,pilotstudy,


participantselection,anddataprocessingandanalysis.Itconcludeswithastatementof

ethicalconsiderations.

TheResultschapterreportstheresultsofacheckofthedataforcompleteness

andoutliersandthetreatmentusedformissingdata,validatestheassumptionof

parametricdata,describesthediversityofthestudentparticipants,analyzesthedata

basedonthehypothesesunderstudy,summarizesthequalitativefindings,andreports

additionalquantitativedatafromacourseevaluationsurvey.

TheDiscussionchapterdiscussestheimplicationsandlimitationsofthese

findings.

TheConclusionschaptersummarizesthisresearchandidentifiesopportunities

forcontinuedstudy.


Literature Review

Thischapterprovidesahistoricalperspectiveontheuseofsimulationgamesas

experientiallearningactivitiesinacademicprograms,explorestheresearchtoidentify

thepracticesrelatingtoeffectivedeliveryandevaluationofsimulationgamesas

experientiallearningactivitiesinanacademicenvironment,andidentifiesaneedfor

additionalresearch.Followingabriefhistoricalperspectiveanddescriptionofmy

searchmethodology,Iprovideareviewoftherelevantliteraturebeginningwitha

definitionoftermsfollowedbysectionsdiscussingexperientiallearningtheory,theuse

ofsimulationgamesasexperientiallearningactivities,anoverviewofbusiness

simulationgamingresearchandadetailedlookatprojectmanagementsimulation

gamingresearch.Thechapterconcludeswithadiscussionofthegapinman’s

knowledge.

Historical Perspective

Althoughthevalueofplayandgamesineducationandtrainingcanbedatedback

tothewritingsofPlato,mostpublishedworkoneducationalsimulationandgaming

datesfromthe1960s(Megarry,1978).Megarryobservesthatthetechniquesdescribed

inthepublicationsshereviewedfrom1968‐1977arederivednotfrompriortheoretical

writings,butratherfromthebusinessmanagementtrainingdevelopedin1956bya

researchgroupoftheAmericanManagementAssociationcitingtheinfluenceofthe

ancienttraditionofmilitarygaming.

Amorerecentreviewofthepast40yearsofdevelopmentsinbusiness

simulationgamingbyFariaetal.(2009)confirmedthisancestryanddiscussedthe

evolutionofbusinessgamingfrommanuallyscoredsimulationgamesoflimiteddecision

andfeedbackcomplexitytosimulationgameswithincreasinglycomplexdecisionsand

moredetailedfeedbackhostedonserversaccessedviatheWorldWideWeb.The

desirabilityofthistrendtointernethostedsimulationgamesissupportedbythe

researchofAshleigh,Ojiako,ChipuluandWang(2012)whofoundprojectmanagement

studentsdesire“ablendoflearningthatresidesontheintersectionof‘transferable

skills’and‘e‐learningenvironments.’”Despitethispreferencefortheuseoftechnology

andpedagogicalinnovation,“thefundamentalreasonsastowhyeducatorsusebusiness

simulationgameshavenotchangedmuchduringthepast40years”(Fariaetal.,2009).

Fariaetal.(2009)furthernote,“asgameshavebecomemorecomplexbecauseof


advancesincomputingpower,theneedforgroupdiscussionanddecisionmakingto

understandandmanagethiscomplexityhasbecomegreater.”

IncontrasttoKeys(1977)earlyfindingthattheinstructorshouldplaya

significantroleinguidingthegamelearning,thisneedforgroupprocess/teamwork“has

transferredfarmoreofthelearningresponsibilityofbusinessgamestothegame

participantswhilemakingthegameslessdependentonactiveinstructoroperationand

manipulation”(Fariaetal.,2009).Asaresultofthishands‐offrolefortheinstructor,the

postsimulationgamedebriefdiscussionhasbecomeagrowingresearchinterestduring

the2000’s.Thisincreasedinterestislikelytheresultofoutcome‐basedlearning

measuremandatesofaccreditingbodiesandatendencyforthesimulationgameto

becomethecenterpieceofthebusinesscourse(Fariaetal.,2009).

ConsistentwiththisbackgroundandalamentbyAndersonandLawton(2009)

thatmuchoftheresearchontheeffectivenessofusingbusinesssimulationsas

experientiallearningactivitieshasbeenbasedonmeasuringparticipantorinstructor

perceptionsoflearningratherthanonactual,direct,objectiveevidenceoflearning,my

literaturesearchfoundaneedforadditionalresearchonevaluatingtheeffectivenessof

businesssimulationsingeneralandprojectmanagementsimulationsascapstone

activitiesinparticular.Ifoundnogenerallyacceptedprocessforevaluating

effectivenessofbusinesssimulationgames,verylittleonprojectmanagementsimulation

games,andconfirmedAndersonandLawton’sobservationthateffectivenessmeasures

tendtorelyonsubjectiveassessmentofparticipantperceptionsandsatisfactionwiththe

experience.Whilemostagreedirectmeasurementoflearningispreferred,thisisoften

difficultbecauselearningobjectivesarenotalwaysclear,teamdynamicsareoften

involved,andlearningcanoccurwhilelosingthegame(Greenlaw&Wyman,1973).

Search Methodology

Aninitialsearchoftheliteratureusingkeywords“managementeducation,

projectmanagement,experientiallearning,businesssimulation,simulationgames,

effectiveness,MBA,andassessment”resultedinafindingofseveralhundredarticles,

Ph.D.dissertationsandbookstoreviewforrelevance.Primarysearchengineswerea

universitylibraryhostedManagementPOWERSEARCHwhichsimultaneouslysearches

AcademicSearchComplete,BusinessSourceComplete,EconLit,andRegionalBusiness

News;EducationPOWERSEARCHwhichsimultaneouslysearchesAcademicSearch

Complete,EducationResearchComplete(ERIC),andProfessionalDevelopment


Collection;SCOPUS;GoogleScholar;ProQuestfordissertationsandtheses;andthe

university’scatalogkeywordsearch.Relatedarticleswerefoundinover60journals

(listedinAppendixA;boldfontindicatescitedreference)indicatingbroadinterestinthe

useofsimulationgamesasapedagogicaltoolforexperientiallearning.Onearticleby

McCreery(2003)intheInternationalJournalofProjectManagement,“Assessingthe

valueofaprojectmanagementsimulationtrainingexercise,”provedparticularly

valuableasitprovidedafoundationformyresearchdesign.

ThejournalSimulationandGaming:AnInternationalJournalofTheory,Design

andResearch(oftencitedasSimulation&Gaming)wasidentifiedastheleadingacademic

journalcontainingarticlesdiscussingtheuseofsimulationgamesinanacademic

environment.Theprimarysponsorofthisbi‐monthlypublication,theAssociationfor

BusinessSimulationandExperientialLearning,alsopublishesitsannualconference

proceedingsinDevelopmentsinBusinessSimulation&ExperientialLearning(ABSEL,

2011).Oneoftheirstatedmaingoals“toaugmenttechniquesusedfortheassessmentof

educationandthedevelopmentoflearningtheory”iscongruentwiththeintentofstudy.

PursuitofanotherABSELgoal,“tofacilitatecommunicationonaglobalscale

amongspecialistsdesigningandusingbusinesssimulationsandexperiential

methodologies,”isevidencedbytheirlistedtiestootherorganizationswithaninterest

insimulationandgaming(ABSEL,2011).“TheseincludetheInternationalSimulation

andGamingAssociation(ISAGA),theJapaneseAssociationofSimulationandGaming

(JASAG),andtheNorthAmericanSimulationandGamingAssociation(NASAGA).”ISAGA

andNASAGAareco‐authorsofSimulationandGaming.Anotherorganization,theSociety

forAcademicGamingandSimulationinEducationandTraining(SAGSET),published

PerspectivesonAcademicGamingandSimulation¸theproceedingsoftheirannual

conferences,andajournal,Simulation/GamesforLearning;butthesedonotappeartobe

inwidecirculationasaninitialreviewoftheirwebsite(SAGSET,2011)referred

inquiriesforpasteditions,ifavailable,tooneofitsmembersandthecurrentwebsite

(SAGSET,2013)onlylistsoneessayandnoneofthesejournalsaspublications.

Foundedin1974,ASBEListheleadingsocietyencouragingacademicresearchin

theuseofsimulationgamesinacademicbusinessmanagementprogramsandthe

contentsofSimulationandGamingissueswerealsoreviewedmanuallyforappropriate

articlesthatmayhavebeenmissedbysearchengines.ABSELpublishedGuideto

BusinessGamingandExperientialLearningin1990toprovide“thedefinitive,hands‐on


guideforbusinesseducatorswishingtousegamesandexperientialexercisesto

maximumeffect”(J.W.Gentry,1990a).Thisseminalpublicationincludedchaptersby

nineteenexpertcontributorsandintroducesABSELasanorganizationandsimulation

gamingasanexperientiallearningactivity;discussesgamedevelopment,futuretrends,

non‐gameexperientialexercises;andconcludeswiththreechaptersdiscussingthe

evaluationofexperientiallearning.Worksbytheseauthorsandotherscitedbythem

werealsoexploredforcontentrelevanttothisstudy.

ABSELconferenceproceedingsarepublishedinDevelopmentsinBusiness

SimulationandExperientialLearninganditspredecessor,DevelopmentsinBusiness

SimulationandExperientialExercises,thesecondmostcitedpublicationsinthisstudy.

Taxonomy

AsdepictedinFigure2,theliteratureusedinthisstudyisgroupedintothree

broadcategories:LearningTheory,SimulationGameApplicationandSimulationGame

Effectiveness.LearningTheoryencompassesthetheoreticalunderpinningofKolb’s

(1984)seminalfindingsonexperientiallearning,Mayer’s(2002)discussionofrote

versusmeaningfullearningandKrathwohl’s(2002)descriptionoftherevisionto

Bloom’sTaxonomyofEducationalObjectives.SimulationGameApplicationincludesthe

argumentsforandagainsttheuseofsimulationgamesasexperientiallearningactivities

ineducationalprogramsandguidelinesfortheiradministration.SimulationGame

EffectivenessisdividedintoBusinessManagement,wheremostoftheresearchtakes

placeusingtotalenterprise,strategyormarketingmanagementsimulationgames,and

ProjectManagement,whereverylittleresearchhasbeendoneandeverystudyfoundis

cited.

TheBusinessManagementresearchcanbefurthercategorizedintoarticles

discussingevaluationmethodsandtheiradequacy,theuseofsimulationgamesto

achievecourseobjectives,earlyexamplesofrigorouslearningresearchjustifying

simulationgamesasapedagogicaltool,proposedframeworksforfutureresearchand

examplesofstudiesattemptingtomeasuresomeaspectofthebenefitofincludinga

simulationgameinanacademiccourseofstudy.Assessmentoflearningisfoundtobea

complicatedandcontinuingtopicofcontroversyamongresearchersleadingtocallsfor

additionalresearch.


Muchoftheprojectmanagementresearchlacksrigorinmethodologyand/or

reporting.NotableexceptionsarethearticlesbyDavidovitchetal.(2006;2007;2008;

2009;2010),McCreery(2003)andPfahl(2004).

Mystudybuildsonthisknowledgeandmakesacontributiontowardsclosingthe

knowledgegaprelatedtotheeffectiveuseofprojectmanagementsimulationgamesin

academicprograms.

FIGURE2‐LITERATUREREVIEWTAXONOMY

Project Management Capstone

Simulation Effectiveness

LearningTheory

Meaningful LearningMayer, 2002

Revised Bloom’s Taxonomy

Krathwolh, 2002

Experiential LearningKolb, 1984

Simulation GameEffectiveness

Simulation GameApplication

Business Management

Project Management

Rigorous Learning ResearchKeys, 1977

Relation to Course Objectives

Greenlaw & Wyman, 1973

EvaluationGosenpud, 1990;Gosen(pud) &

Washbush, 2004, 2010

Quantitative PrePost Perception

Cook and Olson, 2006McCreery, 2003

Quantitative PrePost Simulation ScoreDavidovitch et al., 2006, 2007, 2008,

2009, 2010

Qualitative Post Perceptions

Collofello, 2000Dillman & Cook, 1969

Usage and HistoryFaria et al., 2009; Faria & Wellington,

2004

DebriefingLederman, 1984; Thatcher, 1990; Thiagarajan, 1992

Promoting LearningGentry, 1990;

Hertel & Millis, 2002; Jones, 1987

Student AssessmentAnderson & Lawton, 1988, 1992, 2007;Salas et al, 2009

Multi‐source Framework

Stainton, Johnson & Borodzicz, 2010

MethodologyExamples

Quantitative PosttestAnderson, 2005;

Ardobor &Daneshfar, 2006;

Baglione & Tucci, 2010;Buzzetto‐More &Mitchell, 2009

Mixed Method PosttestGraziano, 2003;

Lainema & Lainema, 2007

QualitativeGreen, 2004

QuantitativePretest‐PosttestGamlath, 2009;

Klein, 1980, 1984;Seethamraju, 2011;

Smalt, 1999; Wellington et al., 2012;

Williams & Williams, 2011

Mixed Method PrePost

Pfahl, 2004

Quantitative Post Simulation ScoreMartin, 2000

Mixed Method PostAl‐Jibouri &

Mawdesley, 2001; Al‐Jibouri et al., 2005;

Cooper, 2011

Quantitative Post Perceptions

Dantas, Barros & Werner, 2004


Definitions

Althoughsimulationshavebeenusedineducationforover50years,“thereare

nogenerallyaccepteddefinitionsofaneducationalsimulationoritsmanyvariations”

(Hertel&Millis,2002).Thissectionintroducesseveralofthedefinitionsfoundinthe

literatureandconcludestheearlydefinitionshavestoodthetestoftime.

Experiential Learning.Whilecautioning“whatthestudenttakesawayfroma

particularexperienceisoftenidiosyncratictohis/herperceptionsoftheexperience,and

issomewhatoutsidethecontroloftheinstructor,”Gentry(1990b)discussesseveral

definitionsand“criticalcomponentstoexperientiallearning”andconcludes:

Experientiallearningisparticipative,interactive,andapplied.Itallowscontact

withtheenvironment,andexposurestoprocessesthatarehighlyvariableand

uncertain.Itinvolvesthewholeperson;learningtakesplaceontheaffectiveand

behavioraldimensionsaswellasonthecognitivedimension.Theexperience

needstobestructuredtosomedegree;relevantlearningobjectivesneedtobe

specifiedandtheconductoftheexperienceneedstobemonitored.Students

needtoevaluatetheexperienceinlightoftheoryandinlightoftheirown

feelings.Andprocessfeedbackneedstobeprovidedtothestudent(andpossibly

supersede)theoutcomefeedbackreceivedbythestudent.

Gentry(1990b)developedthisdescriptionbyconsideringthepriorworkofan

AACSBtaskforceexploringappliedandexperientiallearningcurriculumdevelopment

andprior“definitionalworkbyHoover(1974)atthefirstABSELconference”and“ina

subsequentpaper(HooverandWhitehead1975,p.25),”combinedwith“theoverall

experientiallearningtaskstructureproposedbyWolfeandByrne(1975).”Gentrynoted,

“Thisprocess‐orientedapproachissomewhatsimilarinnaturetothoseproposedby

Kolb(1984)andLewin(1951),”seminalauthorsonthetopicofexperientiallearning.

Thesecharacteristicsofexperientiallearningarealsocongruentwiththeconcept

ofmeaningfullearningdescribedbyMayer(2002)inhiscomparisonofthreelearning

outcomes(nolearning,rotelearningandmeaningfullearning)andtheirrelationshipto

thecognitiveprocessesoftheRevisedBloom’sTaxonomy.Inthiscomparison,rote

learnerscanrecallinformationbutareunabletotransferthisknowledgetonew


situationswhereasmeaningfullearnerscannotonlyrecallinformation,theycanapply

thisknowledgetonewproblemsanddifferentlearningsituations.AccordingtoMayer:

Meaningfullearningoccurswhenstudentsbuildtheknowledgeandcognitive

processesneededforsuccessfulproblemsolving.Problemsolvinginvolves

devisingawayofachievingagoalthatonehasneverpreviouslyachieved;thatis,

figuringouthowtochangeasituationfromitsgivenstateintoagoalstate

(Mayer,1992).Twomajorcomponentsinproblemsolvingare(a)problem

representation,and(b)problemsolution,inwhichastudentdevisesandcarries

outaplanforsolvingaproblem(Mayer,1992).

Mayer(2002)differentiatesrotelearninganditsassociationwithRemember,the

simplestcognitivecategoryintheRevisedBloom’sTaxonomy,frommeaningfullearning

anditsrelationshipwiththefiveothercognitivecategoriesthatare“increasinglyrelated

totransfer(Understand,Apply,Analyze,Evaluate,andCreate).”Remember,describedas

“retrievingrelevantknowledgefromlong‐termmemory”categorizesonlytwocognitive

processesinBloom’srevisedTaxonomy(RecognizingandRecalling),whereastheother

fivecategoriescontain17cognitiveprocesses(Understand:Interpreting,Exemplifying,

Classifying,Summarizing,Inferring,Comparing,Explaining;Apply:Executing.

Implementing;Analyze:Differentiating,Organizing,Attributing;Evaluate:Checking,

Critiquing;Create:Generating,Planning,Producing)(Krathwohl,2002).Allofthesemay

berequiredwhenparticipatinginanexperientiallearningactivitysuchasabusiness

managementorprojectmanagementsimulationgame.

ThisstudyacknowledgesthecongruenceofGentry’s(1990b)descriptionwith

theseminalworkofKolb(1984),Mayer’s(2002)descriptionofMeaningfulLearningand

theRevisedBloom’sTaxonomyasdescribedbyKrathwohl(2002),andconsiders

experientiallearningtobeaninstructor‐facilitatedlearningexperiencewherethe

studentslearnbyreflectingontheiractiveinvolvementinanexperientialactivity

requiringtheuseofanalyticalskillstoconceptualizetheexperienceanddecisionmaking

andproblemsolvingskillstointeractwithandapplyrelevantconceptsduringthe

activity.ThisdefinitionalsoalignswithGentry’s(1997)latershortdefinition:

“Experientiallearningistheaffective,cognitive,andbehavioralchangeinastudent

pursuanttosomestructuralexperiencedesignedtofacilitatethesechanges.”


Game.Heyman(1975),anearlyauthoronclassroomuseofsimulationgames,

definesagameas“acontestinwhichpeopleagreetoabidebyasetofrules”and

comments“theveryword‘game’meansfuntostudents–andmostotherpeople–which

maybeachangefrommanyschoolactivities,onereasonwhygamesarepopularin

classrooms.”

HertelandMillis(2002)distinguishsimulationsfromgamesbynoting“games

ofteninvolveelementsoffantasyandmake‐belief...[with]rulesofplay[that]...are

likelytobefixedandrigid,unlikethemorefluidandoftenspontaneousguidelinesof

[educational]simulations”which“typicallyplacestudentsintrue‐to‐liferoles.”

Forthepurposesofthisresearch,Heyman’s(1975)definitionofgameis

sufficient.

Simulation.Asimulationis“animitationorsimplificationofsomeaspectof

reality”(Heyman,1975).Heymanadds“amoreelaboratedefinitionis:Asimulationis

anactivitywhoserulestendtogenerateinthetotalbehavioroftheparticipantsamodel

ofsomerealworldprocess.”

HertelandMillis(2002)prefernottoengagein“semanticdebates”onthe

“variouspermutationsofsimulations,games,roleplaying,andotherinteractive

pedagogies”andsuggestthat“educationsimulationstypicallyplacestudentsintrue‐to‐

liferoles,andalthoughthesimulationactivitiesare‘realworld,’modificationsoccurfor

learningpurposes.”

Forthepurposesofthisresearch,Hyman’s(1975)formerdefinition,“an

imitationorsimplificationofsomeaspectofreality”issufficientasthenotionofrulesin

themoreelaboratedefinitioncomeintoplaywhenconsideringthedefinitionfora

simulationgame.Asimulationcanbealiveroleplayoraninteractivecomputermodel.

Simulation game.Combininghisdefinitionsforsimulationandgame,Heyman

(1975)definesasimulationgameas:“Anactivitythatcombinesthecharacteristicof

both‘game’and‘simulation;’itimitatessomepartofrealityandisacontext.Simulation

games,thereforearebothenjoyableandeducational,withtheemphasisonlearning”

(Heyman,1975).

HertelandMillis(2002)prefertoavoidcombiningtermsinto“hyphenated

horrors”andadvise“theusetowhichateacherputsateachingtechniqueismore

importantthanwhatthetechniqueiscalled.


Thetermssimulationandgameareoftenusedinterchangeablyinresearch

articlesandmaybefoundwithadescriptivemodifiersuchasbusiness,marketing,

strategicmanagement,totalenterprise,orprojectmanagement.

Forthepurposeofthisresearchthetermsimulationgamewillbeusedtoreferto

anexperientiallearningactivitywithrulesofplaythatsimulatesaspectsofthereal

worldtoachieveeducationallearningobjectives.

Inthecaseofgraduatebusinessmanagementeducation,simulationgamesare

oftencomputerizedtoprovideadynamicandrobustlearningenvironment.Biggs

(1990)describescomputerizedbusinessgamesasonesinwhich“gameplayers

(participants,students)assumetheroleofdecision‐makersinorganizations.”Headds:

Frequently,thecomplexityofthegameissuchthattheparticipantsaregrouped

intoteamsofthreeormoremembers.Theteams’decisionareasmaycoverthe

totalfirmorafunctionalunitofafirm,dependingonthefocusofthesimulation.

Theplayersareprovidedwithaplayer’smanualwhichpresentsthe“rulesofthe

game,”describestheenvironment,andgivesastartingpointforthefirm.The

startingpointisusuallythesameforeachfirmintheindustry.Theparticipants

submitasetofdecisionsfortheirfirmtothegameadministrator(theinstructor

ortrainerorhis/herdesignee).Eachsetofdecisionsusuallyrepresentsa

quarteroftheyearorayearoftheoperationofthefirm.Thegames

administrator,usingthecomputer,processesthedecisionsandreturnsthe

resultstotheparticipants.Theparticipants,giventheircurrentsituation,

prepareanothersetofdecisionswhicharethenprocessedbythegame

administrator.Thefactthatparticipantsmakedecisionsforanumberof

decisionperiodsforcesthemtolivewiththeconsequencesoftheirprevious

decisions.

Theoutput(results)receivedbyparticipantsgenerallyconsistsofatleast

abalancesheetandanincomestatement.Frequentlyatleastonepageof

supplementaloutputisprovidedeachfirmandinsomeinstancesagreatmany

pagesareprovided.

Thisdescriptionisrelevantandeasilyadaptedfortheprojectmanagement

simulationunderstudy.Ratherthansimulatetheoperationofthetotalfirmora


functionalunitofthefirm,thedecisionareascoveraprojectbeingundertakenbythe

firm.Eachteamhasthesamestartingpoint,i.e.,thesameprojectandobjectiveswitha

resourcepoolcommontoallteams.Eachsetofdecisionsrepresentsaphaseofthe

projectratherthanacalendarquarteroryearandteamsmustlivewiththe

consequencesofdecisionsmadeduringearlierphases.Theoutputconsistsofschedule,

costandresourceefficiencyresultsforeachworkperiod.

Experiential Learning

WritingintheforewordofDavidKolb’sseminaltextonexperientiallearning,

notedcontemporaryleadershipscholarWarrenBenniscreditsKolbwithbeingthefirst

toconclusivelydemonstrate“thatlearningisasocialprocessbasedoncarefully

cultivatedexperiencewhichchallengeseverypreceptandconceptofwhatnowadays

passesfor‘teaching’”(Kolb,1984).Kolbdrawson“theintellectualoriginsof

experientiallearningfromtheworksofJohnDewey,KurtLewin,andJohnPiaget”to

describe“theprocessofexperientiallearningandproposesamodeloftheunderlying

structureofthelearningprocessbasedonresearchinpsychology,philosophy,and

physiology.”Hecreditsthe“greatRussiancognitivetheoristL.S.Vygotsky,thatlearning

fromexperienceistheprocesswherebyhumandevelopmentoccurs”(Kolb,1984).

Kolb(1984)regardsDewey,LewinandPiaget“astheforemostintellectual

ancestorsofexperientiallearningtheory”andidentifiesthecommoncharacteristicsof

theirlearningmodelstodescribethenatureofexperientiallearningas“theprocess

wherebyknowledgeiscreatedthroughthetransformationofexperience.”His

descriptionemphasizes“theprocessofadaptationandlearningasopposedtocontentor

outcomes...thatknowledgeisatransformationprocess,beingcontinuouslycreatedand

recreated,notanindependententitytobeacquiredortransmitted”andthat“learning

transformsexperienceinbothitsobjectiveandsubjectiveforms”(Kolb,1984).

Basedonhisresearchofindividualityinlearningandtheconceptsoflearning

styles,Kolb(1984)proposedatwo‐dimensionalperspectiveoflearningstylewitha

learningspacedefinedbythedimensionsabstract/concreteandactive/reflectivebut

cautionedthatthismodelonlyrepresentedqualitativedifferencesinelementary

learningorientations,adding“tofullyappreciateaperson’sapproachtolearning,we

needtounderstandhisorherpositiononathirddimension,thatofdevelopment.”Asit

relatestolearninganddevelopmentinhighereducation,Kolbfound“anyeducational

program...canbeviewedashavingdegreesoforientationtowardeachofthefour


learningmodesintheexperientiallearningmodel,labeledasaffective,perceptual,

symbolic,andbehavioral,toconnotetheoverallclimatetheycreateandtheparticular

learningskillormodetheyrequire.”

Intheaffectivelycomplexlearningenvironment,learnersarelikelytoexperience

whatitistobeaprofessionalinthefieldunderstudyby“engaginginactivitiesthat

simulateormirrorwhattheydoasgraduates,ortheyareencouragedtoreflectuponan

experiencetogeneratetheseinsightsorfeelingsaboutthemselves(Kolb,1984).”

Perceptivelycomplexlearningenvironmentsrequireconsideringatopicfrom

differentperspectivesandindifferentways.“Ifataskisbeingdoneoraproblemis

beingsolved,theemphasisismoreonhowitgetsdone,theprocess,thanonthesolution.

...Learnersarethusfreetoexploreothers’ideas,opinions,andreactionsinorderto

determinetheirownperspective”(Kolb,1984).Kolbstatestheteacher’srolehereisto

emphasizeinquiryandreflectiononresultsasaguidetofutureactionsratherthanto

evaluatebasedontheattainmentofthecorrectsolution.

Symbolicallycomplexlearningenvironmentsinvolve“tryingtosolvea[n

abstract]problemforwhichthereisusuallyarightanswerorabestsolution”(Kolb,

1984).Thelearnerisguidedandconstrainedbyrulesandtheteacheristheaccepted

expert,timekeeper,taskmaster,andscheduleenforcer.“Successismeasuredagainstthe

rightorbestsolution,expertopinion,orotherwiserigidcriteriaimposedbytheteacher

oracceptedinthefieldofstudy”(Kolb,1984).Kolbfoundthatlearnerspreferringthis

learningenvironmenttendtoprefertheoryreadingsandthinkingaloneandthatgroup

exercisesandsimulationshindertheirabilitytolearn.

Behaviorallycomplexlearningenvironmentsemphasizeapplyingknowledgeor

skillstoapracticalproblemthatneednothaveacorrectorbestanswer;“butitdoes

havetobesomethingthelearnercanrelateto,value,andfeelsomeintrinsicsatisfaction

fromhavingsolved”(Kolb,1984).Thiscouldbeareal‐lifeproblem,case,orsimulation

withafocusoncompletingthetaskwherethelearnerisresponsiblefordecidingona

courseofactionandmanaginghisorhertimewithintheconstraintsofpossible

checkpointsandadeadline.

Theuseofasimulationgameasanexperientiallearningactivityinan

educationalprogramhasastrongorientationtotheaffective,perceptualandbehavioral

learningmodeswherestudentslearnbydoingtasksrelatedtoworkintheirprofessional

field,oftenonteamswheretheyalsolearnfromeachother.


Simulation Games as Experiential Learning Activities (ELA)

Theuseofbusinesssimulationgameshasgrownsubstantiallyinacademic

programsand“reachedtherelativepointofsaturationinvariousAmericanbusiness

courseapplications”(Wolfe,1993).Durablelearningispromotedacrossallphasesof

thesimulationexperiencebystudentsstructuringtheinformationtheyreceiveandact

onintoknowledge(Zantow,Knowlton,&Sharp,2005).“Withrespectto[student]

attitudes,thereappearstobelittletorisk,andmuchtogain,fromintegratingbusiness

simulationsintobusinessschoolprograms”(P.H.Anderson&Lawton,2009).Although

therehavebeendramatictechnologicalchangesenhancingmultipledimensions(listed

byFaria,etal.,2009,as“realism,accessibility,compatibility,flexibilityandscale,

simplicityofuse,decisionsupportsystemsandcommunication”),thebasicexperiential

learningprocessforapplyingeducationalsimulationgameshasn’tchangedmuchover

thedecadesfromtheonedescribedbyTaylorandWalford(1978):

1.Participantstakeonroleswhicharerepresentationsofrolesintherealworld,

andthenmakedecisionsinresponsetotheirassessmentofthesettinginwhich

theyfindthemselves.

2.Theyexperiencesimulatedconsequenceswhichrelatetotheirdecisionsand

performances.

3.Theymonitortheresultsoftheiractionsandreflectontherelationship

betweentheirowndecisionsandtheresultantconsequences.

Interrelationshipsbetweenalargenumberoffactorscanbedisplayed,

visiblymanipulatedandadjusted.Asthesituationdevelopssonewstrategies

needtobeformulatedandadopted.

Thisdescriptionofthelearningprocessisverysimilartotheonedescribedby

Fariaetal.(2009)over30yearslater:

Asvehiclesforinstruction,businesssimulationsremainaspowerfultodayas

theywerewhenfirstintroduced.Theyallowfordynamicbusinessdecision

makingwhereplayersformastrategyandthencarryoutaseriesofdecisionsto

implementthestrategy.Gameparticipantsreceivefeedbackthatdemonstrates

theconsequencesoftheirdecisions,andtheparticipantsareabletoevaluate


theirstrategiesand,ifnecessary,reformulatetheirstrategies.Theexperience

gainedfromtherepeatediterationsofdecisionperiodsprovidesdirectfeedback

toplayers,fromwhichtheyareabletolearn.

Whathaschangedisthecomplexityofthegamesresultingfromadvancesin

computingpowerandtheavailabilityoftheinternet.Teamworkremainsimportant,but

nowlessofaresearchinterestthandebriefing,as“theneedforgroupdiscussion...to

understandandmanagethiscomplexityhasbecomegreater”(Fariaetal.,2009).Faria

etal.note“thewhyofbusinessgameusagehasremainedremarkablythesameduring

thepast40years”andalthoughtheresearchinterestin

debriefinghasgrowntremendouslyinthepastdecade....thelearningintentof

businesssimulationexercisescoupledwithfeedbackfromthestudentsasto

whattheyhaveexperiencedandlearnedhasalwaysbeenacentralpartof

businesssimulationgamingresearchthroughthedecades.

Thissectionsummarizestheseminalargumentsforandagainstusingsimulation

gamesasexperientiallearningactivitiesandreviewsguidelinesforadministeringa

businesssimulationgameinanacademicenvironment.

The case for and against simulations.Theuseofsimulationsasanexperiential

teachingtechniquegainedpopularitybecausetheymadetheeducationalprocessmore

effectivebyrelatingacademicactivitiestotherealworld(Heyman,1975).Simulations

weave“substance‐specificinformationintoreal‐lifeproblemsinmeaningfulwaysthat

studentscanunderstand”(Hertel&Millis,2002).Awell‐designedsimulationgamecan

simultaneouslyteachtheoryandprovidepracticalapplicationpractice(Salas,Wildman,

&Piccolo,2009).Assuch,theyaresuperiortootherlearningstrategiesfordeveloping

complexcompetencies.Simulationsbasedonmodelsthatarecarefullyconstructedand

realisticcanprovidestudentswithalaboratoryenvironmentthatcompressestimeand

facilitatesexperimentingundercontrolledconditions(Meieretal.,1969).While

acknowledgingthat“simulationscancreativelyfocusonexperimentation,prediction,

andevaluation(Cunningham,1984),”HertelandMillis(2002)stressthe“general

educationalgoalsof(a)transferofknowledge,(b)skilldevelopment,and(c)the

applicationofbothknowledgeandskills.”


Educationalsimulationscanalsohelpbridgethegapsbetweendisciplines.

Educationsimulationsprovideuniquestructuresforintegratingsubstantive

principlesaswellasdoctrinesandskills.Thiscapabilitymakesthemparticularly

usefulforcapstonecourses[emphasisadded]thatattempttobringtogetherthe

essentiallearningobjectivesofmultiplecourseswithinanacademicor

professionaldiscipline....simulationsshouldbeconsideredasacentral

structurearoundwhichacapstonecourseevolves.(Hertel&Millis,2002)

Anattractiveaspectofsimulationsisthatthey:

createasocialenvironmentinwhichparticipantsbecomeinvolvedwith

whateverresourcestheyhave.“Individualization”and“startingfromwhereyou

are”happennaturallyinsimulations.Becauseparticipantswilljumpinattheir

ownlevels,theselectionofanappropriatesimulationisnotasdifficultasthe

selectionofappropriatebooks,films,orothereducationalmedia.(Heyman,

1975)

Advantages.Studentsparticipateinsimulationgamesonafairlyequalbasis

withoutregardforpasteducationandachievements.Thefunaspectofsimulation

gamesmakesthemexcitingandmotivating.Withtheemphasisonaction,studentsare

drawnintoanofteninterdisciplinaryanddynamicscenarioandgivetheirfullattention

tomaking“almostreal”decisionsastheyplayaroleand“sometimesexcitedly[become]

involvedwithothersinnegotiating,persuading,andresistingpersuasion”(Heyman,

1975).Thedynamicnatureofsimulationgameshelpsstudentsbridgethegaptoreality

bymovingbeyondtheapplicationofconventionalwisdomandcookbooksolutions.

“Issuesmustbetreatedontheirownmerits,alternativestrategiesmustbedevisedand

attempted,resultsobservedandconclusionsdrawn,onthebasisofdirectexperience”

(Taylor&Walford,1978).Theintegrativenatureofalarge‐scalesimulationgamemay

providestudentswithagreaterclarityofthesystemsaspectsofacomplextopicthanis

achievablefromthe“divideandconquerapproach...usedinmostlecture‐based

courses”(J.W.Gentry,1990b).


Studentsreportaheightenedinterestinlearningaftercompletingasimulation

game.Althoughthiseffectisdifficulttomeasure,manyeducatorsbelievethisis

sufficientreasonfortheircontinueduseanddevelopment(Taylor&Walford,1978).

Heyman(1975)notes:

Thewheelinganddealingthatsimulationsrequiregivestudentsintensive

practiceinverbalandwrittencommunication.Thiskindofcommunicationin

simulationsdoesnotcausetheapprehensioninherentin“beingcalledon,”orin

makingapresentation.Thisdiffusingofpublicperformanceisagreathelpfor

theshyorquietstudent,whateverhisorherskills....Almostallsimulationscan

berepeatedwiththesameparticipantsandtheywilllearnsomethingneweach

time.Theknowledgegainedfromthepreviousplay(pluschanceaspectsofthe

game)willmakethesimulationadifferent,yetstillproductive,experiencewhen

itisusedagain.

Learningfromsimulationsoccursatdiverselevels.The“highflyersare

motivatedtoprogresstoevengreaterheightswithoutadverseeffectonthelessgifted

who,inturn,learnfromtheirpeers”(Taylor&Walford,1978).

Anotheradvantage(thatsomeeducatorsmayviewasathreat)isremovingthe

student‐educatorpolarizationandchangingtheroleoftheeducatorfrom“sageonstage”

tothatoffacilitator,andperhapsinterpreter,duringtheconductofsimulationgames.

Theself‐monitoringaspectofmanysimulationgamesmakestheeducator’sroleto

inspire,stimulate,motivate,andhelpstudentslearn;ratherthantoteach,direct,order

andjudgeasstudentsmakedecisions,observetheconsequences,andmakefurther

decisionsbasedontheirevaluationoftheconsequences(Heyman,1975;Taylor&

Walford,1978).

Simulationgamescanalsobeusedasdiagnostictoolsinconjunctionwithother

instrumentsandexperientiallearningactivities.Forexample,studentswith3‐5yearsof

workexperienceandanundergraduatebusinessdegreefromanAACSB‐accredited

institutionattendingaone‐yearMBAprogramatBabsonCollege,participateina3‐day

businesssimulationaspartofa2‐weekprogram‐beginningresidencydesignedto

provideabaselineoftheirfunctionalskillsandidentifyspecificareasforimprovement.


Otheractivitiesduringthistwoweekresidencyincludeapersonalcareerassessment

exercise,team‐buildingexercises,andtwogroupprojects(Schlesinger,1996).

Disadvantages.Asidefrompotentiallimitedavailabilityandcost,somemayview

theextratimerequiredtobecomefamiliarwiththegameandtointegrateitintothe

curriculumasnotjustifiedbythebenefits.Anotherargumentagainstissomestudents

maynotfindthemrelevant.Otherargumentsaresimulationstaketimetoconduct,may

createdisruptivenoise,maynotbeeasytostageduetofacilitylimitations,andmaybe

difficulttoaccuratelygrade.Animportantconcernisthat,whileemotionalinvolvement

ofstudentsisgoodfromamotivationalperspective,emotionsmaygetoutofcontroland

requiretheeducatortoinitiateamoderatingintervention.(Heyman,1975;Taylor&

Walford,1978;Wolfe,1990).

Views of simulation game users, former users, and never users.Supportingthese

earlyobservationsbyHeyman(1975)andTaylorandWolford(1978),Fariaand

Wellington(2004)surveyedbusinessschoolfacultymembersandfound,consistentwith

Faria’s(1998)earlierresearch,that“across1,085respondents...30.6%werecurrent

businessgameusers,17.1%wereformergameusers,and52.3%wereneverusers.”Of

thecurrentusers,theyfoundadisproportionatelyhighnumberoffullandassociate

professors.Theysuggestthattheadditionaltimerequiredtosuccessfullyincorporatea

simulationintoacoursemaybeviewedasnon‐justifiablebyyoungerfacultymembers

whoareundersignificantpressuretopublishorperish.Anotherreasongivenis

youngerfacultymembersarelessfamiliarwithsimulationgamesandlesslikelytohave

receivedpromotionalmaterialsfromsimulationgamesuppliers.Inthisstudy,Fariaand

Wellington(2004)found:

Thetopthreeperceivedadvantagesofsimulationgamesforstudentsarethe

sameforcurrentandformergameusers–simulationsprovideexperientialor

active,participativelearning;simulationsintegratethedifferentfunctional

businessareas;andsimulationsallowfortheoryapplication.Never‐usersreport

thetopgameadvantagesasprovidingexperientiallearning,realism,and

allowingfortheoryapplication.Withrespecttoadvantagesforinstructors...

simulationsprovideinteractive/dynamicexercisesandtheyallowfortheory

application....Otherimportantsimulationgameadvantages[are]...simulations

interestandmotivatestudents,gamesintegratethefirm’sfunctionalareas,[and]


simulationsprovideameasureofstudentcomprehensionandsubjectmatter

understanding.

Theprimarylearningobjectivegivenbycurrentusersforselectingabusiness

simulationgamewastogivestudentsdecision‐makingexperience(Faria&Wellington,

2004);otherobjectivesincluded“integratingtheorywithpractice,introducingstudents

tobusinessplanning,andgivingstudentsexperiencewithgroupdecisionmakingand

teamwork.”FariaandWellingtonfoundthat75%ofallgameusersstronglyfeltthey

achievedthelearningobjectivesusingsimulationgames.Never‐userscitedtherequired

preparationtime,poorfitofknownsimulationgameswiththecourse,lackof

informationaboutavailablesimulationgamesandpreferenceforanalternatepedagogy

asreasonsfornotusingsimulationgamesintheircourses.

Theoverallimpressioniseducatorsbelievetheadvantagesoutweighthe

disadvantagesandsimulationsareavalue‐addedexperientiallearningactivity(Faria&

Wellington,2004;J.W.Gentry,1990a;Hertel&Millis,2002;Heyman,1975;Jones,1987;

Taylor&Walford,1978;Wolfe,1997).However,Wolfe(1990)cautions:

Theuseofgamesbyinstructorsmaybeself‐servingtotheextentthatcertain

instructorsmadethecoursesmoreinterestingtothemselves,madethemfeel

theywereaccomplishingsomething,anditrelievedthemofhavingtodeliver

lectures.

Administration guidelines.Whatstudentsactuallylearnfromthesimulation

experienceisafactoroftheirperceptionsoftheexperience,somethingtheinstructor

cannotcontrol;however,theinstructordoeshavecontrolovertheexperiential

environmentandthepedagogyusedtocreatethatexperienceandisresponsiblefor

providingahigh‐qualityexperience(J.W.Gentry,1990b).Otherthantheimportanceof

thepostsimulationgamedebriefdiscussionbeinghighlightedinconjunctionwiththe

emergenceofcomputer‐basedsimulationgames,theguidelinesforusingasimulation

gameasanexperientiallearningactivitydonotappeartohavechangedmuchsincethe

earlytextsadvocatingtheiruse(Fariaetal.,2009).Accordingly,thissectionincludes

someofthisearlyworkinthediscussionofthesimulationenvironmentandadvicefor

directing,debriefingandgradingsimulationgameexperientiallearningactivities.


The simulation environment.Studentsengagedinasimulationareactivelearners

whoshouldself‐discovertheconceptsandprinciplesbuiltintothesimulationmodeland

experiencedduringtheactivity(Dukes&Seidner,1978).Intheiranthologydescribinga

varietyofsimulationgames,DukesandSeidnersuggestthatoneofthebestwaysto

insurelearningoccursasaresultofparticipatinginasimulationgameistodesignthe

game“insuchawayastomakethedesiredlearningrequisitetoplayingthegame,andto

provideforatleastsemicontrolledreinforcementforappropriateplayerbehavior.”This

designshouldtakethepurposeofthegameintoconsideration.Ifcognitiveskill

developmentisthegoal,acomputer‐basedsimulationand/orrigidsetofgamerules

maybeappropriate;however,interpersonalexchangesmaybemoreappropriateifthe

objectivesofthesimulationgamefallwithintheaffectivedomain(Dukes&Seidner,

1978).

Jones(1987)concursinhishandbookontheuseofsimulationsbyteachersand

trainers:“Intheactionpartofthesimulationthereisnoteacher....Asimulationmustbe

anon‐taughtevent...ifitistaughtthenitisnotasimulation.”Further,teachersmust

resistthetemptationtohelptheparticipantssucceed.Inasimulationgame,asinreal

life,studentswilllearnfromtheirmistakesandinthecaseofasimulationgame,

“mistakesareinevitableanddesirable...thegreaterthedisasterthegreaterthe

learning”(Jones,1987).Jonesemphasizesthatforlearningtooccurit’simportantfor

thestudentstobeperformingfunctionalrolesinthesimulationandforthemtohave

sufficientinformationtoperformasaprofessionalinthatrole.

Learningdoesnotnecessarilyneedtooccurduringtheexecutionofthe

simulationandfrequentlyoccursaftertheeventduringthedebriefdiscussion.The

postsimulationdebriefprovidestimeforadiscussionaboutwhathappened.However,

stagingthesimulationgamewithdiscreetworkperiodsfollowedbyreflectiontimeprior

toenteringthenextroundofdecisionshelpsenhancelearning.“Learningfrom

experiencemustallowtimeforreflectiononthatexperience,andtheopportunitytotry

again.Instantenlightenmentisnomoreanessentialfeatureofsimulationsthanitisof

lifeoutsidetheclassroom”(Jones,1987).

Learning objectives.UseofthebroadlearningobjectivesfoundbyFariaand

Wellington(2004)maybecriticizedbythoselookingforspecific,measurableoutcomes

becausetheyaregeneralinnatureanddifficulttoassess;however,“identifyingand


specifyingoutcomesofexperientiallearningisparticularlyproblematic”(Gosenpud,

1990).Gosenpudexplains:

Usuallytheteacherordesignerofanexercisedefineswhatistobelearned.

However,thelearneroftenlearnsthingsnotintendedbythedesigner,andoften

thisunintendedlearningismorevaluablebecauseitisrelevanttothelearner.

Theproblemforassessingexperientiallearningisthatevaluation,definedbythe

designer,maymisstherealworthoftheexperientialexperiencebecausewhatis

valuabletothelearnerisdefinedbythelearnerandmayhavenothingtodowith

thedesigner’sintention.

Inaddition,inexperientiallearningintendedoutcomesareoftenvague

sincethefocusonexperientiallearningisusuallyonverycomplex,abstract

phenomena.

Jones(1987)agreesbroadlearningobjectivesmaybesufficientandiscriticalof

the“Americanapproachtoeducation,whichtendstobebasedonstep‐by‐steplearning

aimedatresultsthatcanbequantifiedbymeansofobjective(andfrequent)testing.”

JonesdoesnotbelieveitisessentialforasimulationgameELAtohaveclearlydefined

educationalobjectives:“Someofthemostfamousandeffectivesimulationsare

educationallyambiguous,andtheprocessisusuallyfarmoreimportantthantheend

product.”HertelandMillis(2002)agree:“Education—particularlyhighereducation—

shouldbedesignedonthebasisofapplicationofknowledge,interactionwithideasand

people,experience,feedback,andreflection.Simulationsarestructuredpreciselyon

thesepremises.”Thepostgamedebriefdiscussionthereforeisacriticalelementofthe

overallexperienceandservestoclarifywhatoccurredduringthesimulationgameand

toassesswhatwaslearnedwithregardstothesevagueorself‐discoveredobjectives.

Theimportanceoftheeducator’sroleinleadingthisdiscussionandtheneedforstrong

discussion‐leadingskillsshouldnotbeunderestimated(Dukes&Seidner,1978).

AndersonandLawton(2003)observethatmostsimulationgamesareusedafter

studentsshouldhavelearnedthefundamentalconceptsofthediscipline,eitherearlierin

thecurrentcourse;orinthecaseofacapstonecourse,inanearliercourse.Inbothof

thesecases,thelearningobjectivesemphasizetheapplicationofthecourseconceptsina

mannerthatistransferabletotherealworld.However,theyhavesuccessfullyuseda


simulationgameatthebeginningofanintroductoryundergraduatecoursewith

alternateobjectives“toprovidestudentswithevidencethattheylackknowledgeoftools

requiredofaneffectivemarketingmanager”and“tomotivatestudentstoacquirethe

toolstheyneedtoimprovetheirperformance.”

Directing a simulation game.Onceasimulationgamehasbeencarefullyselected

tofulfilloneormorecourselearningobjectives,it’simportanttosharethelearning

objective(s)withthestudentsandtoinformthemofhowtheirindividualperformance

willbeevaluated.Aprereadoftheplayer’smanualcanbeassuredbyconductingashort

quizpriortobeginningthegame.Researchhasshownthatstudentswholearnthe

simulationbyreadingthemanualorfromtheirpeersperceivesignificantlymore

transferorlearningasaresultofthesimulationexperiencethaniftheinstructor

explainstheoperationofthesimulationtothem(B.W.Mayer,Dale,Fraccastoro,&Moss,

2011).Priortostarting,ashortdiscussioncanbeheldtoreviewthesimulationandits

rules,clarifyconfusingoroftenmisunderstoodaspectsofitsoperation,andanswer

proceduralquestions;ifthesimulationgamewillbeplayedbyteams,theidealnumber

formostgamesappearstobefromthreetofive(Fritzche&Cotter,1990).Apractice

periodconductedbeforeofficialplaybeginscanresultinstudentsreportingtheywere

moreeffectiveandthesimulationgameexperiencewasmorevaluablethanifnopreplay

practicewasallowed(Snow,Gehlen,&Green,2002).Heyman(1975)offersthisbasic

ruleandguidancefordirectingthesimulation:

Saynomorethanthefewwordsnecessary:beforethegame,tostartit;during

thegame,tokeepitrunning;afterthegame,tokeepthediscussiongoing.The

simulationitselftellstheplayerswhattheyneedtoknowinorderforthegame

toworkeffectively.Runthesimulation,notthestudents....Youarenotthereto

teach...thesimulationdoestheteaching.Youwillseeitduringthegame,and

laterinthepostgamediscussion,whenthestudentswillrevealwhattheyhave

learned.Anattempttoteachwillonlyinterferewiththesimulation....Agame

director,unlikeateacher,doesn’tsayhowtobehaveinagame,nordoesheor

shecommentonstudents’decisionsinorafterthesimulation.

HertelandMillis(2002)agree:“Simulationsarenotteachingeventsinthe

normalcontextofaninstructor‐studentrelationship.Inmostcases,onlybyremoving


theinstructorfromthedirectactionwillstudentsfullyassumetheirrolesandtake

chargeofthescenarioaction.”Somewhatcountertothisadvice,FritzscheandCotter

(1990)suggestitmaybeappropriatefortheeducatortoemphasizecertainprinciples

thoughtheuseofsupplementalcasesorexercises;however,thismaybeintendedto

occurbetweensimulationruns,notduringthem,andshouldnotconstitutecoaching.

Debriefing.Thepostsimulationdebriefisanimportanttooltohelpassure

learning(Fariaetal.,2009;Hertel&Millis,2002;Heyman,1975;Peters&Vissers,2004).

Itcanbeconductedindividually,witheachteam,ortogetherwithalltheteams.The

educatorlistensforindicationsoflearningandcanuseaseriesofpreplannedquestions

relatedtothelearningobjectivestofacilitatethisdiscussion(Pate&Parker,1973).

Conductingitwithalltheparticipantstogetherofferstheadvantageoflearningfrom

eachotherratherthanfromamorelimitedperspective(Fritzche&Cotter,1990).When

asimulationgameisusedasaneducationalexperientiallearningactivity,thedebrief

discussionoftenfocusesonthestudents’performancesinthegame.Thejointdiscussion

approachcanhelpmakethelimitationsofaparticularapproachsalienttothestudents,

butit’salsoimportanttodiscusstheconnectionbetweenthegameandtherealworld

(Peters&Vissers,2004).

Scholarship gap.Crookall(2010)expressesconcernthatproperdebriefingisnot

beingdoneasmuchasitshouldbe,“despiteanentireS&G[Simulation&Gaming]

symposiumin1992beingdevotedtothetopic,guesteditedbyoneofdebriefing’smost

articulateproponents,LindaLederman.”CrookallandFariaetal.(Fariaetal.,2009)

believethescholarshipgapcreatingthisconcernischangingintherightdirection,buta

searchofSimulation&Gamingshowsmostrecentarticlesmentioningdebriefingcitethe

importanceofit,butdonotoffernotnewscholarshiponthetopic.

Despitethegenerallyrecognizedassumptionthatthedebriefprocessisa

primarilyfacilitatoroflearningfromsimulationgames,Lederman(1992)andPetersand

Vissers(2004)foundlittlewrittenonthetheoreticalandconceptualunderpinningofthe

debriefingprocess,andwhattheydidfindcamefromoutsidethedomainofsimulation

gaming.Thisistroublingbecause:

Feedbackiscriticalforproperlearningtotakeplaceafteranexperience.The

studentshouldnotbeallowedtoconcludewhatwaslearnedwithoutreceiving

feedback;thereistoomuchevidencethathumanbeingsdonotdothisproperly.


Thedebriefingsessioniscrucial[emphasisadded].Studentsneedtoarticulate

theirperceptionofwhatwaslearned,andtheinstructorneedstoputthingsinto

abroaderperspective.(J.W.Gentry,1990a)

Learning process.Thedebriefingsessionshouldbeconsideredalearningprocess

inandofitselfandtheroleoftheeducatoristoleadthestudentsthroughapurposeful

discussionoftheirexperienceplayingthesimulationgame(Lederman,1984;Lederman,

1992).Itshouldbetreatedasadistincteventandseparatedfromtheplayingofthe

simulationgame.Thiscanbedonebytakingashortbreakifbothoccurduringthesame

classperiod.Thedebriefshoulddiscussthestudents’personalfeelingsaboutthe

experience(e.g.,elation,embarrassment,anger),therolesplayedandwhathappened

duringthesimulation,andwhattheylearnedfromtheexperience,includingwhatcanbe

tiedtotheirrealworld(Hertel&Millis,2002).

Twoassumptionsare(1)theexperienceaffectedthestudentsinameaningful

way,and(2)processingofthatexperience,usuallyintheformofadiscussion,isneeded

“toprovideinsightintothatexperienceanditsimpact”(Lederman,1992).

Lederman(1984)describesthepostsimulationgamelearningprocessas:

Astructured,guidedmethodforbringingmeaningtotheexperienceandfor

learningfromthatmeaning.Itinvolvestalkingabouttheexperiences,analyzing

them,evaluatingthem,andintegratingthemintoone’scognitiveandconscious

database.Itisthepartoftheprocessinwhichthestudentsreflectuponthe

experiences,andtheimplicationsofthoseexperiences,fortheworldexternalto

theclassroom.Itisthepartoftheprocessinwhichwhatisbeingcreatedfor

studentsisanewwayofseeingtheworld,ofperceivingitandmakingsenseofit.

Theinstructorguides;studentscreateforthemselves,basedoninsights,and

fromthisrelatewhattheyareseeingnowtowhattheykneworthoughtbefore.

Lederman(1984)advisesthat“knowledgethatistheproductofexperienceis

highlysubjective;itistheproductoftheinteractionbetweentheindividualandthe

experience”and“ratherthanconsideringthe[postsimulationdiscussion]processa

simpledebriefing,itisbetterdepictedastheCognitiveAssimilationofExperience


(Lederman,1983),orthatpartofthemethodologyinwhichwhatthestudenthas

experiencedisintegratedintosomecognitivecomprehension.”

Procedures and design.Debriefingsaregenerallyconductedasguideddiscussions

withproceduresbasedonclasssizeandavailabletime.Withasmallnumberof

students,everyonecanparticipateinthediscussion.Withlargergroups,volunteerscan

besolicitedwithcaregiventonotallowafewpeopletodominatethediscussion.For

shortsimulations,atleast25%ofthetotaltimeallottedtothesimulationplusdebrief

maybeneededforthediscussion.Writtenreflectionontheexperiencemayalsobe

requiredandthiscantaketheformofajournalwithentriesaftereachoftheactivity

periodsinmultiphasesimulations(Hertel&Millis,2002).

Regardlessofitsfrequency,writtenreflectionshelpstudentsintegratetheir

simulationexperiencewiththeirpriorknowledgeandexperiencesandforce

themtoorganizetheirthoughtsandemotions,enablingmoreproductive

participationinagroupdiscussion.Instructorsrequirethesewrittenreflections

formanyreasons:tocapturedetailsofwhathappenedandstudentrationalefor

them,toallowstudentstoshareanyemotionsengenderedbythesimulation,and

togivestudentsanopportunitytonotepersonalissuesorquestionsthatthey

mightnotmentioninagroupsetting.(Hertel&Millis,2002)

Thesewrittenreflectionscantaketheformofself‐diagnosticlearninglogswhich

arereferredtoduringthedebriefdiscussionandcanalsobereviewedbytheeducatoras

partofthegradingprocess(Hertel&Millis,2002).Thelogreportsanswerstoquestions

suchas:

(a)Whichcourseconceptsorprincipleswereusefultoyouinworkingthe

currentproblemorissue?Wheredidyoulearnthemfrom?—fromindependent

research?fromyourtextbook?fromotherstudents?fromthesimulationcoach?

fromoutsideexperts?(b)Whichcourseconceptorprinciplethatyoupreviously

learneddidthisnewconceptorprinciplebuildon?(c)Ifyouexperienced

difficultyorwereunabletoworktheproblemorissue,whatinformationor

knowledgewouldhaveenhancedyourabilitytoworkit?Wherecouldyouhave

obtainedthisknowledge?(Hertel&Millis,2002)


Severalmodelsorframeworksareavailabletoassisteducatorswithdebrief

discussiondesign(Hertel&Millis,2002;Heyman,1975;Lederman,1984;Lederman,

1992;Thatcher,1990;Thiagarajan,1992);thesearediscussedbelow.

Heyman(1975)offeredsomesimpleadviceforfacilitatingthepostgame

debriefingdiscussion:

Letitflow,letitgo.Atthestart,therewillprobablybetenminutesor

morewhenitwillrangeallovertheexperience;allowthistohappen.

Thesharingoffeelings–andreactiontoothers’feelingsisvery

important....

Ifitisnecessary...tostartthediscussion,asktheplayerswhatkindsof

feelingstheyexperiencedduringthesimulation...

Thepostgamediscussioncancovermanyquestionsconcerningthe

simulationanditsrealworldcounterpart.

o Wasthegametruetolife(ingeneralandinspecificaspects)?...

o Howwouldyoumodifythesimulationtoimproveit?...why?...

Ifstudentsturntoyoufor“therightanswer,”yourbestbetistoreplyin

ahighlytentativemanner.Afterall,therealworldiscomplex.These

questionsareclearsignsofstrongmotivationtolearnmore;listen

carefullyandusethatmotivationtoleadyourstudentsintomore

activities....

Commentsonindividualorteamperformancewillnotcontributetothe

students’learningorsatisfaction.

LedermanandRuben(1984)developedanappliedframeworkforthesystematic

assessmentofcommunicationgamesandsimulationsconsistingofthreecriteria:

validity,reliabilityandutility.Lederman(1984)expandedonthisframeworktoinclude

discussionquestionsrelatedtoeachofthesecriteria;validityisconsideredfromtwo

perspectives,“facevalidity(correspondencewithrealworldcounterparts)andconstruct

validity(correspondencebetweenconceptsandtheactivitydesignedtopresentthose

concepts).”Reliabilityquestions“[guide]studentsthroughareviewoftheactivity:how

ithappened,whattheydid,and,finally,theimplications.”Utilityis“acomparison


betweenengagementcostsintermsoftime,money,energy,andemotionalexpenditures

versustheoutcomesorbenefits.”Evaluatingutilityinvolvesacost‐benefitanalysisofthe

students’perceivedexpendituresversustheirperceivedbenefits.Ratherthanlecture,

instructors“engageinastructureddialogwithstudents—adialoguedesignedtoelicit

everypossibleconnectionbetweentheexperienceanditscognitivecomprehension.”

Inalaterwork,Lederman(1992)reviewedtheliteratureondebriefingand

presentedamodelforsystematicassessmentofthedebriefingprocessconsistingoffive

setsofquestionsdealingwithlearningobjectives,situationalconstraints,debriefing

strategy,implementation,andtheprocessingexperience.

Similarly,Thiagarajan(1992)developedaseven‐phaseproceduralmodelfor

debriefing,withthephasescategorizedbythequestions:“Howdoyoufeel?...What

happened?...Doyouagree?...Déjàvu?...Whatwouldyoudodifferently?...What

if…?...Canyouimprovethisactivity?”Kriz(2010)discusseddebrieffacilitationusing

thisphasedapproachandofferedrelatedstructuredquestionsthatmightbesuggested

foruseinthestudent’slearningdiary.

HertelandMillis(2002)builtontheworkofLederman(1992,1984)and

Thiagarajan(1992)andsuggestthesedebriefquestionsformultiple‐rolesimulations

thatareeasilyadaptabletoteam‐basedsimulations:

Dealingwithemotions:

1. Howdoyoufeel?

2. Howdidyoufeelwhen_______did_______?

3. Didyoufeelingschangeduringthesimulation?

Describingsimulationaction:

1. Whathappened?Howdidtheactionunfold?

2. Whatwasthescenarioabout?

3. Whatwasthemajorinterestofyourrole?

4. Whatdidyoudo?

Personalizingtheaction:

1. Whydidyoutakeacertainaction?

2. Howdid(anotherrole’saction)causeyouto(re)act?

3. Whatweretheimplications(ofyouraction)?

4. Whatwereyourgreatestsuccessesandfrustrations?


5. Whatwereyourgreatestobstacles;howdidyouovercomethem?

6. Whatwouldyoudodifferentlynexttime?Why?

Applythesimulationtopastandfuturelearning:

1. Whatwerethekeyissues?

2. Howdidthescenariorelatetothetypicalissuesofthecoursediscipline?

3. Whatdidyoulearn?

4. Howwastheexperienceworthwhile?

5. Whatwouldhaveoccurredifotherdecisionshadbeenmade?

6. Whatwouldyoudodifferentlynexttime?Why?

Applyingthesimulationtotherealworld:

1. Howdoesthesimulationcomparewithreal‐worldbehaviors?Give

examples

2. Wasitpredictable?Why?

3. Whatreal‐lifeissuesweremissinginthescenario?Whateffectdidthis

have?

Thatcher(1990)describesdebriefingaswhere“reflectiontakesplaceandfrom

whichchangewilloccur,becauseitisthepartoftheactivitywhichfocusesonthe

complexprocesseswhichtookplaceineachindividualandinthegroupasawhole.”

Thatcherrecognizedthatreflectionalsooccursduringtheplayingofthesimulation

gameandadaptedKolb’s(1984)learningcyclediagramtodepictthatevaluationand

reflectionoccursafterresultsareobtainedfollowingeachdecisionpointduringa

simulationgame.Thislearningfromreflectionwhileplayingthegamediffersfromthe

learningthatoccursfromthereflectionduringthepostsimulationdebriefdiscussion.

Thatcher(1990)noted:

Abt(1968)identifiedthreedifferenttypesoflearningwhichareoften

presentinasimulation:

1. Learningthefacts,expressedinthegamecontextanddynamics(byfacts

aremeantnotonlyfacts,concepts,andgeneralizationsbutskills)

2. Learningtheprocessessimulatedbythegame

3. Learningtherelativecostsandbenefits,risksandpotentialrewardsof

alternativestrategiesofdecisionmaking


Missingfromthislistislearningabouttheinterpersonalskillsaprojectmanager

needstobesuccessful,e.g.“leadership,teambuilding,motivation,communication,

influencing,decisionmaking,politicalandculturalawareness,negotiation,trustbuilding,

conflictmanagementandcoaching”(ProjectManagementInstitute,2013a),thatoften

occurswhileplayingateam‐basedsimulationgame.

Thatcher(1990)describedthedebriefingprocessasaseriesofstagesthatcanbe

separateorintegrated.Thesearerelatedtodifferentkindsofknowledge,different

learningstyles,andtothedifferentpartsofthereflectionprocess:

1. Identifyingtheimpactoftheexperienceoneachindividual—thisoften

involvesthedevelopmentofself‐knowledge.Thismaybeovertorcovert,

dependingonhowthreateningtheexperiencehasbeentoeachofthe

participants.Forsomeoftheparticipantstheidentificationremainsa

personalprocess,whichtheymayrevealatalatertimethantheactual

debriefing.

2. Identifyingandconsideringtheprocesseswhichweredevelopedinthe

simulation.

3. Clarifyingthefacts,concepts,andprincipleswhichwereusedinorrelated

tothesimulation.

4. Identifyingthewaysinwhichemotionwasinvolvedinorfiguredinthe

simulationforeachindividualandforthegroupasawhole,

5. Identifyingthedifferentviewswhicheachoftheparticipantsformedof

thenatureoftheprocessesandtheexperience.Thisidentificationisthe

processbywhichalltheparticipantsbegintoexplorethecomplexityof

thesysteminwhichtheywereparticipants.(Thatcher,1990)

Thatcher(1990)advises“eachofthesestagesisanimportantelementinthe

processofreflectionbywhichthereallearningtakesplaceforeachindividual”and“the

debriefingcanbeorganizedinanumberofways,fromaninformaldiscussion,througha

structureddiscussion,tosomeformofwrittenreportorcommentaryontheexperience.”

Hefindsitusefultodistributea“carefullydesignedresponsequestionnaire”foreach

studenttocompletepriortothedebriefingdiscussion.Thishelps“ensureeachofthe


participantshasconsideredthesignificantpointsrelatedtothesimulationindividually

beforeageneraldiscussiontakesplaceandsomeimportantpersonalexperienceislost

intheheatofthegeneraldiscussion.”

Thecommonthemeinallthesemodelsisstudent‐definedmultidimensional

reflectivelearning.Althoughthestudentssharedacommonexperience,eachwilltake

awayhisorherownpersonalizedconclusionsfromtheexperience.Theroleofthe

educatoristoencouragethisself‐reflectionandtoenhanceitwiththereflectionofthe

otherstudentsduringthedebriefdiscussion.Thiscanbeaguideddiscussionledbythe

educatorusingpreplannedquestionsimmediatelyfollowingtheactivityoritcanrequire

apredebriefdiscussionrefectionandpreparationperiodforthestudent/teamtoreview

whathappenedduringthesimulationgameandtoreflectontheresultsandwhatthey

learnedfromtheexperiencepriortodiscussingitwiththerestoftheclass.(Hertel&

Millis,2002;Heyman,1975;Lederman,1984;Lederman,1992;Thatcher,1990;

Thiagarajan,1992)

Instructor skill.Forsimulationgame‐basedlearningtobepedagogically

legitimateratherthanjustafeelgoodexperience,educatorsmustassistthestudentsin

refiningtheirknowledgebasedontheexperience.Doingsorequiresexperience

guidanceskillsthatgobeyondnormalclassroomskillstoenablethemtohelpstudents

“articulateacognitiveunderstandingoftheexperientialactivitiesinwhichtheyhave

engaged”(Lederman,1984).Theseexperienceguidanceskillsinclude“tolerancefor

ambiguity,...abilitytoobserveandinterpretbehavior,...abilitytoformquestionsand

listentoanswersaboutbehaviors,...abilitytoselectappropriatedirectivenessor

nondirectivenessinworkingwithstudents,...asenseoftiming,andsoundjudgmentcalls

(Lederman,1984).Whilethese“experienceguidance”skillsmayhaveapplicabilityin

anyclassroom,Ledermanstressesthesearearequirementinanexperience‐based

learningclassroom.

Instructorswhousesimulationsarenotonlyresponsibleforthelearning

outcomesofthoseexperiences,theyarealso“responsibleforthehumanoutcomes—the

feelingsthatstudentshaveaboutthemselvesandothersasaconsequenceof

engagementintheactivityanddiscussionofthatengagement”(Lederman,1984).

Accordingly,instructorsmusttakeasupportiveratherthanjudgmentalapproachto

createanon‐threateninglearningenvironment.


Assessment.Assessmentoflearningismorecomplicatedthanmerelyreviewing

theresultsfromperformancemeasuresthatmaybebuiltintothesimulationgame.It’s

verypossiblethatthelowestscoringstudentorteamlearnsthemostbecauseofthe

adversitytheydealtwith,andinthecaseofteamactivities,whattheylearnedabout

workingwithothersinstressfulsituations;therefore,theemphasisshouldnotbeon

whowonbutonwhattheylearnedfromtheexperience(P.H.Anderson&Lawton,2007;

P.H.Anderson,Lawton,&Wellington,2008;Chin,Dukes,&Gamson,2009;Dukes&

Seidner,1978;Gosenpud,1990;Taylor&Walford,1978;Teach&Murff,2007).“Much

moreresearchneedstobeconductedinthe‘whatislearnedbyplayingbusinessgames’

genreandmorespecificallythelinkbetweenthecomplexityofataskandlearning”

(Teach&Murff,2009).

Assessmentcanincludetheobservationofbehaviorandinterpersonalskills

duringtheactivityandindividualretrospectiveinterviewsthatsupplementthegroup

debriefaftertheactivity(Hertel&Millis,2002;Jones,1987).“Assessmentofallaspects

ofthesimulation,includingactivities,learning,student‐studentinteractions,evaluation

procedures,comfortlevels,andsoon,areessential”(Hertel&Millis,2002).This

assessmentcancomefrommultiplesourcesincludingdatafromquestionnaires,

uninvolvedclassroomobservers,focusgroups,andinstructorself‐reflection.Salas,

Rosen,HeldandWeissmuller(2009)concurandofferasetof21performance

measurementsbestpracticesbasedontheoriesofhumanperformanceandpractice

methodsfoundintheliteraturethatcanbeconsideredwhenplanninganassessment

strategy:

Performancemeasurementusuallyworksbestwhen:

1. Itcapturesmultipledimensionsofperformanceatappropriatelevels

ofanalysis....

2. Event‐basedtechniquesareusedtocapturedataatmultiplelevelsof

analysis.

3. Multiplemeasuresfromvarioussourcesarecaptured.

4. ASystematicplanisinplacetointegratedatafrommultiplemeasures..

..

5. Itcapturestheprocessesofperformance....


6. Expertmodelsofthetaskareusedasstandardsagainstwhichto

compareandevaluateperformanceprocessesandoutcomes.

7. Thecollectionandtransmissionofobjectivemeasuresisautomated.

8. Measuresaredescriptiveofperformance.

9. Performancecanbecomparedtostandardsfordesiredlevelsof

performance.

10. Itisdiagnostic—whenitprovidesinsightintothecausesofperformance....

11. Itallowsforperformancediagnosistobepartiallyorfullyautomated.

12. Itallowsforperformanceevaluationtobepartiallyorfullyautomated....

13. Thereisflexibilitydesignedintoembeddedmeasures(different

measurescanbesubstituted)....

14. Itcapturesabroadspectrumofmeasuresandthecontextof

performance....

15. Observersaretrainedtohighlevelsofreliability.

16. Observersuseprotocols....

17. Itsupportslearning....

18. ItallowsfortheautomatedandmanualcreationofAAR[AfterAction

Review]aidsfortrainingremediationandfeedback....

19. Itenablesautomatedscaffoldingandperformance‐basedcoaching....

20. Whenisdrivesreal‐timecorrectivefeedback....

21. Itisintegratedwithtrainercontrolsandfeedbackgeneration.(Salas

etal.,2009)

AndersonandLawton(1988)offeradifferentperspective.Theylisted11

techniquesthatcanbeusedtoevaluateperformanceonasimulationexerciseand

postulatedtheirabilitytomeasurelearningobjectivesaccordingtoBloom’staxonomy.

Nineofthesetechniqueswereratedstronginonetothreeofthetaxonomycategories.

InorderofprogressionfromstrengthinBasicKnowledgethroughObjectiveKnowledge,

theyare:

Examsonsimulationrulesandprocedures

Examsonreadingsimulationoutput

Evaluationofawrittenplan

Abilitytopredictresults


Performancerelativetoimplementationoftheteam’splan

Identificationofandrecoveryfrommistakes

Relativerankingonsimulationresults

Analysispaper

Oralpresentation

Theremainingtwoevaluationmethodslistedare:

Examsonconceptualissues

Peerevaluations

AndersonandLawton(1988)notedtheabilityofanexamonconceptualissues

tomeasurelearningobjectivesinanyofthesecategorieswilldependonitsdesignandis

instructor‐dependent;peerevaluationswereratedmoderateinthelowerthree

categories(BasicKnowledge,Comprehension,andApplication)andweakinthehigher

categories(Analysis,ObjectiveSynthesis,andObjectiveEvaluation).Theyconcluded:

Nooneofthesemethodsprovidesacomprehensivemeasureofhowcompletelya

studenthasmasteredthelearningobjectivesforacourse.Somelearning

objectivesarejusteasiertomeasurethanothers,justassomeknowledgeis

easiertodemonstratethanothers....Manyofthesemethodsevaluate[team]

performanceratherthanindividualperformance.Unlesstheinstructorchooses

togivethesamegradetoallteammembers...somemethod(s)ofindividual

performanceevaluationmustbeutilized....Acafeteriaapproach...would

providethemostcomprehensiveappraisal.(P.H.Anderson&Lawton,1988)

Inresearchassessingtherelationshipbetweenfinancialperformanceona

businesssimulationandothermeasuresofstudentlearning,AndersonandLawton

(1990)foundveryfewsignificantrelationshipsbetweenthesimulationperformance

scoreandeightoftheothermethodsusedtoassesslearning,anobservationpreviously

madeinotherstudies(Greenlaw&Wyman,1973;B.Keys,1977;Wolfe,1985).Thisled

themtoconcludethatperformancescoresonabusinesssimulationarenotavalid

measureofstudentknowledgeorcomprehension.Theyalsofoundlittlesupportfortheir

priorpostulatedhierarchyofassessmentmethods,suggestingthestrengthofa

particularmethodforevaluatingalearningobjectiveaccordingtoBloom’staxonomy

maybesituationalratherthanabsolute.Thisdoesnotmeanlearningisnotoccurring,


onlythatnocorrelationbetweensimulationscoresandotherassessmentmeasureswas

found.ResearchbyFariaandWhiteley(1990)foundthat“certaintypesoflearningcan

beimprovedbyparticipatinginasimulationgame.”.

Consistentwiththisadvice,multiplemethodsaretypicallyusedtoassessstudent

performance.Inasurveyofsimulationusers,AndersonandLawton(1992)foundmost

respondentsusedanaverageof4.4methodstomeasurestudentperformancein

simulationgames.Theyfurtherfoundthesimulationgamesinuselastedfromeightto

16classperiods,primarilyinadvancedbusinesspolicy,management,ormarketing

courseswithstudentteamsofthreetofivemembers.Onaveragethesimulationgrade

countedfor30%ofthestudent’scoursegrade.Teamperformancerelativetotheother

teamswasusedasanevaluationmethodinalmostallcoursesaccountingfor,on

average,40%ofthesimulationgrade.Thiswasfollowedbyevaluationoftheteam’s

writtenplanin77%ofthecaseswithanaveragegradeimpactof16%andateam‐

writtenpaperanalyzingitsperformancein62%ofthecaseswithanaveragegrade

impactof23%.Noneoftheothermeasuresaccountedformorethan17%ofthe

simulationgrade.Peerevaluationwasusedtheleast(13.9%)andonaverageaccounted

for13.8%ofthesimulationgrade.

HertelandMillis(2002)stresstheimportanceofembeddingassessmentsinto

thesimulationexperience.Thesecanbeoralpresentationsand/orwrittenpaperssuch

asmanagementbriefingsandbusinessproposalsthatcanbeassessedandgradedina

mannersimilartootherassignments.Theyidentifyninecriteriathatmaybeconsidered

forgradingstudentperformance:

1. Demonstratedunderstandingofsubstantiveissues.

2. Understandingandproperuseofprocess.

3. Representationofroleinterests.

4. Demonstratedinitiative.

5. Qualityofwrittenwork.

6. Qualityoforalpresentations.

7. Demonstratedabilitytoworkwithothers.

8. Demonstratedleadership.

9. Effectivetimemanagement.(Hertel&Millis,2002)


Grading.Thediscussionontheimportanceofthepostgamedebriefemphasized

itsroleinhelpingstudentsself‐discoverwhattheylearnedfromtheexperience.Tothe

extentthattheylearnedorreinforcedconceptsortechniquesthatcanbetested,this

testingisbetterdonelaterasaseparateactivityandthesimulationgameactivityitself

shouldbeungradedorgradedonthebasisofthethoughtfulnessofthefeedback

provided(Chinetal.,2009;Heyman,1975).Further,“simulationcoursesshouldinvolve

noncompetitivegradingpractices:Studentspittedagainstoneanotherhavenoreasonto

cooperatetoenhancethelearning–andhencethehigherachievement–offellow

students”(Hertel&Millis,2002).

Chinetal.(2009)recommendconductingashortungradedsurveyatthe

conclusionofthesimulationgameandbeforethedebriefdiscussionto“capturethoughts

fromtheparticipantsbeforetheyare‘contaminated’bygroupdiscussion.”Theyfurther

notethatthecomparisonofdatafromapretest‐posttestapproachcanprovideamore

directassessmentoftheimpactandthatthenatureofthequestions,skills‐basedor

attitudinal,willdependontheobjectivesoftheactivity.

HertelandMillis(2002)cautionagainstgradingstudentsindividuallyfortheir

participationduringgroupactivitiesbecauseofthedifficultyinvolvedwithmonitoring

allgroupactivityallthetime.Theysuggestpeerandself‐assessmentcanbeusedto

determineafairgradeclaiming“peerassessmentcanbecomemoremeaningfulwhen

studentshaveinputandownershipovertheprocess”andcanbeimplementedbyhaving

studentscompleteafeedbackformconsideringtaskperformanceandgroupskillsafter

allgroupmeetings.

Whenitcomestoassigninggrades,Heyman(1975)cautions:

Itisimpossibletogradesimulations,andifonetiesthescoresgainedinthegame

togrades,thesimulationitselfissignificantlychanged.Inanycasegamesare

excellentmotivators,sothatonepurposeofgradingisalreadyfulfilled.

HertelandMillis(2002)citeresearchpreferringcriterion‐referencedgrading

where“allstudentswhoreachaspecifiedlevelofcompetencyearnthedesiredand

appropriategrade”and“amasteryorientationtowardlearningcanpromotepositive

motivation.Studentsneedasenseofcontroloverthefinaloutcome.Theyshouldfeel


theyhaveresponsibilityfortheirowngradesandcantakepositivestepstoimprove

them.”

Rubricscanbedevelopedtoaidingradingsimulationgameartifactsand

presentations(Hall&Ko,2008).BasedonaStateofFloridarequirementforaproject

managementstudentlearningobjective(SLO)inacurriculum,onewasdevelopedfor

evaluatingtheprojectmanagementskillsandpracticesemployedincompletinga

capstonecoursesimulationproject(Hornyak,Peach,Bowen,Moes,&Wheeler,2006).

Summary.Thissectionpresentedtheadviceforadministeringsimulationgames

inaneducationalenvironmentandhighlightedtheindividualnatureofthelearningthat

occurs.Inadditiontolearningbydoing,individualreflectionisimportant.The

concludingdebriefdiscussionhelpsmakethislearningsalient.Itisdifficulttograde

learningfromexperientialactivitiesandtheuseofmultipleassessmentmethodsis

recommended.Mostsimulationgamesaremoreorientedtotheaffective,perceptual

andbehavioralmodesoflearningratherthantowardthesymbolicmode;therefore,

assessingtheeffectivenessofasimulationgameexperiencewilllikelybemore

subjectivethanobjectiveandcanbenefitfromtheuseofgradingrubrics.

Simulation Game Effectiveness Research

Researchandevaluationintotheeffectivenessofsimulationgamesisanareaof

controversy,bothinthesimulationliteratureandforeducationingeneral,especially

whenconsideringwhetherthemethodologyandexperimentaldesignisappropriateand

sufficienttojustifytheconclusions(Jones,1987).

MuchoftheliteratureonassessmentfoundinSimulationandGamingwas

writtenduringthe1970sand1980stoprovetheeducationaleffectivenessofsimulation

games(Chinetal.,2009;Graf,2001).Recentarticleshavediscussedthelackof

standardsfor,anddifficultyof,measuringtheeffectivenessofsimulationgamesas

experientiallearningactivities(P.H.Anderson&Lawton,2009;Gosen&Washbush,

2004).

GosenandWashbush(2004)reviewedresearchandscholarshipdealingwith

assessmentofexperientiallearningapproachesandfoundattemptstoassessthevalidity

ofexperientiallearningandtomeasuretheeffectivenessoftheapproach.Whilethey

foundempiricalresearchsupportingboththevalidityandeffectivenessofexperiential

learning,theyalsofound“along‐standingtrendofnotmeetingthehighestofresearch

designandmeasurementstandards.”Althoughrigorousstudiesexist,


theredoesnotexistenoughconsistentresearchfrommethodologicallysound

studiesacrossmultiplegamestoconcludesimulationsarevalid....Thisisnotto

saysimulationsarenotvalid,butinsteadthereisnotenoughevidenceto

concludetheyareactuallyvalid(Gosen&Washbush,2004).

Whiletheyagree“learningisacomplexconstruct,hardtopindownand

thereforedifficulttomeasure,”GosenandWashbush(2004)believeassessmentof

effectivenessisimportantandcallformorerigor;however,Jones(1987)suggeststhis

maynotbepracticalasmanyauthorshavepointedoutthedifficultiesoftesting

hypothesesrelatedtoshowingthat“simulationsproducegreatergainsincritical

thinking,decision‐makingandproblem‐solvingthandootherlearningmethods,orsome

suchsimilargeneralization.”Forexample:

DavisonandGordon(1978)pointoutthatnoevaluatoryinstrumentscanreadily

encompassthemanydifferentdimensionsofbehaviourandexperienceinvolved,

andTwelker(1977)emphasizestheproblemscausedbythegreatdifferences

betweenindividualsimulationsandbetweentheconditionsinwhichtheyare

used.Inthejargonofresearch,thereareboundtobeagreatmanyuncontrolled

variables.(Jones,1987)

Thecallformorerigorousresearchisnotnew.AndersonandLawton(1997)

observedthatmostsimulationgameresearchisbasedontheconstructsofperceptions

andattitudesfordependentvariablesbecauseweknowhowtomeasurethemand

advocatedfortheuseofmoreobjectivedependentvariablesandcontrolmeasuresto

accountfortheinfluenceofmoderatingvariablestoimproverigor.Theysuggesttwo

modelstoestablishlearningoccurs:(1)pre‐postdesignsmeasuringchangesincognition

orbehaviorforeachindividualparticipantand(2)after‐onlywithcontrolgroup

experimentsexposingparticipantstoalternatepedagogiesandcomparingtheaverage

difference.

Theoperationalizationof“learning”posesdifficultmeasurementproblems,

however,manydifferentmeasureshavebeenusedtorepresent“learning,”


including:simulationgameorexerciseperformance,objectivetestperformance,

coursegrade,andself‐reports.Infact,thedefinitionandmeasurementof

learningremainsanelusivegoal.Confoundingoccursinmostinstances.For

example,simulationorexerciseperformancemeasuresbothknowledgeofthe

conceptsandanunderstandingofthegameitself.Theuseofstandardizedtests

dealingwiththegeneralcoursetopicremovesthisproblem,butintroduces

others.Onesuchproblemoccurswhentheteachingmethodologyisusedwith

oneobjectiveinmind,andthetestmeasuresperformancenotdirectlyrelatedto

it.(J.V.Gentry&Burns,1981)

GreenlawandWyman(1973)found“thedevelopmentanduseof[business]

gamesexpandedenormously”intheyearsfollowing“introductionofthefirstpractical

businessgamein1957.”Theyreviewedresearchstudiesthatfocusedon“gamelearning

tomeetcourseobjectives”andfoundthat“althoughgameshavebeenanextremely

popularandwidespreadteachingtool,verylittle‘hard’researchhasbeendoneon

gaming–especiallyconcerningwhatplayerslearntomeetcourseobjectives.”They

observed,“Developingafirst‐ratelearningresearchwhereboth‘soft’and‘hard’

variablesarenumerous(asingames)isanextremelydifficulttask.”Further,

Asseveralauthorssuggested,gamesmaybemoreeffectiveinteachingcertain

“intangible”concepts,suchasMcKenny’s(1962)planningconceptsorStrotheret

al.’s(1996)“awareness,”thaninteachingexplicitfactsandrelationshipsas

foundbyMoore(1967).Thisraisesacriticalpoint:stronglearningingamesmay

notnecessarilybereflectedbygoodgamingperformance;conversely,absenceof

learningmaynotalwaysbereflectedby“poor”performance.(Greenlaw&

Wyman,1973)

Keys(1977)examinedwhatheconsideredmain‐streambusinesssimulation

gamesresearchbyonlyreviewing“articleswhichutilizedefinitecriteriaforthe

measurementoflearningandprofessionallyacceptableresearchtechniques.”Hefound

addingabusinesssimulationgamecanprovidesuperiorresultscomparedtocourses

withouttheiruse,butgame‐onlyclassesweresuperiortocase‐onlyclassesonlywhen

“significantinstructorguidanceisprovidedinthegame‐onlyclass.”Incontrasttomore


recentadviceintheliteratureonlettingstudentslearnfromtheexperiencewithout

interferencefromtheinstructor,headvocatedtheinstructorshould“playanactiveand

significantroleinguidinggamelearning.”Thisdifferingadviceislikelytheresultof

Keysonlylookingatstudiesthatmeasuredlearningbyperformanceonwritten

examinationsandcasestudyessaysandnottakingintoconsiderationthebehavioraland

intangibleaspectsofassessment.

Stainton,JohnsonandBorodzicz(2010)advocateforassessinglearningbasedon

“datasourcedfromablendofvariedresearchinstruments,suchaswrittenreflective

accounts,questionnaires,semistructuredinterviews,anddirectobservation.”

Inarecentattempttocorrelatesimulationgameperformancescoreswith

learning,andperhapsaddingfueltothecontroversyabouttheuseofsimulationscores

tomeasurelearning,GosenpudandWashbush(2010)conductedanexploratorystudy

correlatinggradesearnedonareflectivepaperexplainingwhythestudentscoredas

theydidonthegamewiththegameperformancescoresandfoundasignificantand

positivecorrelation.Thisfindingisbasedononestudyof28studentsusingatotal

enterprisesimulationandmoreresearchissuggested.

Withthisforewarningaboutthedifficultyandcontroversyofevaluatinglearning

fromsimulationgames,thenextsectionsexamineexamplesoftypicalmethodologies

usedtoevaluatetheeffectivenessofbusinesssimulationgames.

Postsimulation perceptions.MostoftheresearchIfoundwasbasedonasurvey

ofstudentperceptionsandattitudesatthecompletionofaneventorcourse.While

studentperceptionsandattitudesarenotproofoflearning,theycanservetomeasure

satisfactionwhichcanserveasaproxyfortheperceivedvalueoftheeventorcourse.

“Studentattitudesareoftenpostulatedasaninterveningvariablebetweenthepedagogy

andlearning;highlymotivatedstudentsarelikelytolearnmore”(J.V.Gentry&Burns,

1981).Thesesurveysareusedbysimulationgamingresearcherstocombineperception

responsesintovariousconstructsmeasuringperceptionsofknowledge,confidence,

teamdynamics,simulationexperiencesatisfaction,andlearningmodepreference.

Resultsarereportedand,insomecases,correlatedwithdemographiccategoriesand/or

simulationperformancescoresand/orpostsimulationcourseexamstoshowsupportfor

relatedhypotheses.Asexemplifiedbelow,methodsrangefromasimplecalculationof

theconstructmeanstorigorousstatisticalanalysisofthedata.Insomecases,qualitative


dataisalsoobtainedfromresponsestonarrativequestionswhicharethencategorized,

analyzedandusedtotriangulatefurthersupportfortheresearchhypotheses.

Qualitative approach example.Green(2004)analyzedtheresponsestostudent

writingsonthequestions:“WhatdidIlearn?WhatdidIlearnthatIneededtolearn?

WhatdoI‘take‐away’fromthisexperience?”andfoundstudentsvaluedteam

experience,realizedtheimportanceofhavingastrategy,appreciatedtheintegrationof

thecoursematerial,andacknowledgedmoreself‐awarenessabouttheircompetencies

andrisktolerance.

Quantitative approach example.Buzzetto‐MoreandMitchell(2009)useda

postsimulationsurveytoassessstudentimpressionsaboutthesimulationexperience

anditsimpactonlearning.Inthisstudy,studentteamscompetedagainstteamsfrom

otheruniversitiesandindustrybyplayingsixroundsoftheCapstone®Business

Simulation.Studentsalsocompletedonesimulationroundasindividuals,keptadiary

wheretheyreflectedontheirdecisionsandprocesses,andweresubjecttoapeer

assessmentandrelatedcomprehensiveexam.Apostsimulationonlinesurveywas

administeredwithperceptionquestionresponsesusinga5‐pointLikert‐typescale

rangingfromstronglydisagreetostronglyagree.Themeanresponsetoeachquestion

wasreportedinthearticlewithitsstandarddeviation.Thereportedmeansrangedfrom

3.2to4.04andweredeemedproofofafavorablereaction(therewasnoindicationof

significancetestingcomparedtoaneutralresponse).Buzzetto‐MoreandMitchellalso

comparedtherankingofstudentteamsbasedonthesimulationperformancescoreand

concludedtheyperformedsuccessfully.Resultsfromthecomprehensiveexamwerestill

underanalysisandnotdiscussed.Whilethisstudyprovidedinformationonstudent

perceptions,itdidnotassessiftherewasanydirectimpactonlearning.Thequestions

usedformeasuringstudentattitudesaboutsimulationscouldbeusefulinafurtherstudy

onstudentperceptionsofsimulationgames.

Quantitative approach example.AdoborandDaneshfar(2006)usedanalysisof

postsimulationsurveyresultstoexploretherelationshipbetweenperceivedlearning

andperceptionsofsimulationrealism,simulationeaseofuse,teamtaskconflict,team

emotionalconflictandsimulationperformancescore.Allsurveyquestionsusedaseven‐

pointLikert‐typeratingscaleandthecontinuoussimulationperformanceresultswere

codedfromonethroughseven,e.g.ascorefrom60to64wascodedaoneandascore

from96‐100wascodedaseven.Constructswereformedfromthequestionsfor


learning(21questions),taskconflict(4questions),emotionalconflict(4questions),ease

ofuse(3questions)andrealism(7questions).Cronbach’salphawasabove0.91forall

theconstructsexcepteaseofusewhichwas0.77.

Forty‐nineteams,withatleastfourstudentsperteam,attendingasenioryear

undergraduatebusinesscoursespentanaverageofsevenhoursperweekplayingThe

BusinessStrategyGameoverthecourseofasemester.Eachteammanagedasimulated

businessintheathleticfootwearindustry,hadtheoptiontomanufactureandsellonfour

continents,hadplantstooperate,aworkforcetomanage,andhadtodealwithinventory,

distribution,salesforecasting,andcurrencyexchangefluctuations.Anaverageof2.2

studentsperteamvoluntarilycompletedthequestionnaireforatotalof109studentsin

thesample.Thesimulationprojectcomprised60%ofthecoursegrade;therewereno

incentivesorpenaltiesassociatedwithparticipatingornotinthesurvey.(Adobor&

Daneshfar,2006)

Afactoranalysisofthelearningconstructrevealedthreefactorsrelatedto

problem‐solvingskills,teamworkskills,andself‐as‐managerskills.Adoborand

Daneshfar(2006)usedthesefactorsinaregressionanalysiswiththeotherconstructsto

observe:

Realismwaspositivelyassociatedwithallthreedimensionsoflearning

supportingthehypothesisthatahigherlevelofperceivedrealismis

associatedwithahigherlevelofindividuallearning

Realismwasnotsignificantlyassociatedwithgroupperformance

Easeofusewaspositivelyassociatedwiththemanagerialproblem‐

solvingdimensionoflearning,butnotwiththeteamworkandself‐as‐

managerlearningdimensions

Easeofusewaspositivelyassociatedwithgroupperformance

Taskconflictwaspositivelyassociatedwiththeteamworkandself‐as‐

managerlearningdimensionsoflearning,butnotwiththemanagerial

problem‐solvingdimension

Taskconflictwaspositivelyassociatedwithteamperformance

Emotionalconflictwasnegativelyassociatedwiththemanagerial

problemsolvingdimensionoflearningindicatingthatahigherlevelof

emotionalconflictisassociatedwithlowerlevelsofindividuallearning


Therewasnosignificantrelationshipbetweenemotionalconflictandthe

teamworkorself‐as‐managerdimensionsoflearning

Higherlevelsofemotionalconflictwereassociatedwithlowerlevelof

groupperformance

Groupsize,demographics,andattitudetowardgroupworkcontrol

variableswereallshowntonothaveasignificanteffectontheresults

Thisstudyisanexampleofusingrigorousstatisticalanalysistechniquesto

analyzeperception‐baseddata.Whiletheresultsdonotprovethattransferablelearning

occurred,theydosuggestthatrealismanduser‐friendlinessinasimulationare

importantfeaturestoconsiderandgroup/teamdynamicsplayanimportantroleinthe

successofusingsimulationgamesinanacademicprogram.

Quantitative approach example.Citingthemixedresultsofpriorresearchby

others,Anderson(2005)exploredtherelationshipbetweenindividualperceptionof

teamdynamicsandstudentaffectforthesimulationexerciseandtheinfluenceofteam

dynamicsonsimulationperformanceresultswithanintroductorymanagementclass

containing220students.Studentswererandomlyassignedtothree‐personteamsand

playedtheCapsimFoundationsimulationgameforfourweeksduringthesemester.

Studentsinputcross‐functionalbusinessdecisionsinroundsrepresentingoneyearof

operations.Studentsreceivedextracreditontheirfinalexamiftheychosetocomplete

thesurvey;172studentsprovidedusableresponsesindicatingtheirlevelofagreement

with21statementsusinga5‐pointLikert‐typescalefromstronglydisagreetostrongly

agree.Thedependentvariable,studentaffectforthesimulationexperiencewasbased

ononeitem:“IlikedtheCapsimsimulation.”Theotherdependentvariable,simulation

performance,wasa10‐pointscalebasedonaweightedcombinationofsimulation

generatedbusinessmetrics.Independentvariableconstructswerestudent’sperception

ofgroupcohesiveness(2items),teaminterdependence(5items),teamheterogeneity(3

items),teamopportunisticthinking(5items)andteamhypothesis‐driventhinking(5

items).Researchhypothesesweretestedrelatedtotheimpactofstudentperceptionsof

teamcohesion,teaminterdependence,teamheterogeneity,teamopportunisticpractices

andteamhypothesis‐driventhinkingonstudentaffectforthesimulationgameand

simulationperformance.Studentpriorworkexperience(5‐pointscale)anddegree


major(businessornon‐business)wereusedtocontrolfortheireffectswhiletestingthe

hypothesesusingtwo‐stephierarchicalregression.Anderson(2005)found:

Studentswhoperceivetheirteamstobecohesiveandindependenthavestrong

affectfortheexercise,butthisdoesnottranslateintostrongperformance.

However,studentswhoperceivetheirteamstohavelowcohesiveness,low

heterogeneity,highopportunisticpractices,andhighhypothesis‐driventhinking

experiencehigherperformance.

Thisresearchprovideseducatorswithinsightintotheeffectsofteamdynamics

onstudentaffectforasimulationexerciseandonteamperformanceasmeasuredbythe

simulationgame.Itdidnotassociatesimulationperformancewithlearninganddidnot

discussthestudentaffectmeanscoreof3.1andstandarddeviationof1.35.Ifthe

distributionwasnearnormal,manystudentswereindifferentontheiraffectforplaying

thegameor,ifbimodal,therewerenearlyasmanystudentsthatdidnotlikethe

experienceastherewerethatdid.Sincemostresearchersfindthatstudentslike

simulationgames,furtherresearchanddiscussionofthisresultwouldbebeneficial.

Quantitative approach example.BaglioneandTucci(2010)usedanalysisof

postsimulationsurveyresultstoconcludethatPharmaSim,amarketingcomputer

simulation,improvedtheunderstandingofmarketingconceptsfor130undergraduate

studentsfromfiveclasses.Studentperceptionswithsurveyedwitha26item

questionnaireusinga10‐pointExcellent(10)toPoor(1)scalefortheresponses.

Individualquestionresponsesweretestedforsignificanceusingaone‐samplet‐test

againstthescalemidpoint(5.5)representinganeutralresponse.Allquestions,except

two,yieldedsignificantandfavorableresponses.Studentswereneutralonthetwo

questionsregardingPharmaSimbeingexcitingandfun.

Questionsweresummedbycategorytocreateconstructsforoverallsimulation

evaluation,simulationdescription,simulationknowledge,andinstructor

knowledge/integration.TheseconstructswereusedinLatentClassClusterAnalysis

where48%ofthestudentswerefoundtohavehadanoverallpositiveassessmentofthe

simulation,includingitseducationalvalue;founditexciting,challengingandfun;and

believeditincreasedtheirknowledgeinallthecategories.Thesecondcluster

representing40%ofthestudentswereneutraloverallandontheeducationalvalueof


thesimulationandfounditchallenging,butunexciting,unpredictable,hard,andnot

enjoyable;theyreportinggainingknowledgeinonly2of13learningareas.Thefinal

clusterrepresenting13%ofthestudentsevaluatedthesimulationpoorlyandforits

educationalvalue.Furtheranalysisfoundthatthisclusterhadlessinstructorinteraction

thantheothers.(Baglione&Tucci,2010)

BaglioneandTucci(2010)concludedthat“studentsimprovetheirknowledge

andevaluatethesimulationexperiencehigherwhentheyandfacultyspendadequate

timeonit.”

Thisexampledemonstratesamethodofsignificancetestingthatisnotroutinely

seeninthedataanalysisofothersimulationgameresearchstudies,perhapsbecausethe

scoresinthosestudiesarenotnearthescalemidpointandsignificanceisassumed.The

useofclusteranalysisdemonstratedtheriskofrelyingonaveragestoassumethat

everyonerespondedfavorably.

Mixed method example. LainemaandLainema(2007) examinedtheuseof

RealGameinacorporatelearningenvironmentwithparticipantsselectedfrommiddle

management.Althoughthiswasnotanacademicprogram,theparticipantsweretypical

ofwhatonewouldfindinanexecutiveMBAprogram.Thesimulationisdesignedto

modelthebusinessoperationsandfunctionsofamanufacturingcompanyinan

environmentwithuptoeightcompaniescompetingformarkets,suppliersandfunding.

Eachcompanywasmanagedbyateamofthreeparticipants.Thepurposeofthetraining

was“toproduceanauthenticdecision‐makingenvironmentandtogiveaholisticviewof

howabusinessorganizationworkstoproduceprofit”(Lainema&Lainema,2007).

Followingthesimulation,participantscompletedapostsimulationsurvey

comprisedof13closedquestionsusingsevenpointLikert‐typescales(Poor/Excellent

andDisagree/Agee)andeightunstructuredopen‐endedquestions.Mostofthe

unstructuredquestionswereansweredfirst.Themeandatawasreportedfortheclosed

questionstodemonstratethefavorableimpressionsoftheparticipants.These

impressionsweresupportedbyquotationsfromtheopenquestionsleadingtheauthors

toconclude:“Ifwewishtostrengthenthebusinessknowledgeofkeypersonnelin

companies,RealGame,andsimilartypesofsimulation,seemstoofferawell‐receivedand

anefficientlearningtoolforit,”adding,“Whileitisdifficultifnotimpossible,tomeasure

whatpeoplehavelearned,wecanaskwhatusetheythinktheywillhavefortheirnew

skillsandknowledge”(Lainema&Lainema,2007).


Mixed methods example.Inhisdoctoraldissertation,Graziano(2003)useda

mixedmethodapproachofsurveysandfocusgroupinterviewstofind“remarkable

consistencyintheperceptionsofstudents,thattheymakeprogressintheidentified

learningoutcomesand,indoingso,developthecontentknowledgeandpersonal

developmentskillsthatenhancetheirpreparationfortheworldofwork.”Hisstudy

evaluatedtheuseofVirtualEnterprise,“aninterdisciplinary,technology‐driven,

simulationprograminwhichstudentsdevelopandmanageglobalenterprisesina

virtualbusinessenvironment.”Objectivesforthisuseofthesimulationincluded

providinganopportunityforstudentstoseetherelationshipbetweencoursework

learningandtheworkplace.

TheVirtualEnterprisesimulationwasoriginallyintroducedintotheTourismand

HospitalityDepartmentcurriculumatKingsboroughCommunityCollege(NewYork)on

acceptanceofthepremiseofthevalueofexperientiallearning.Adaptationsweremade

toreflectthebackgroundsofthestudents.Studentswereaskedtokeepaweeklylogand

identifycompetenciesrelatedtoeachactivityperformed.Attheconclusionthey

providedanarrativestatementoftheactivitiestheyperformedandthelearningthat

resulted.Forthefirstseveralyears,programevaluationconsistedofanecdotalevidence

andstudenttestimonialsastoitsvaluefollowedlaterbyaslightlymoreformal

assessmentoftheprogram’slearningoutcomes,basedprimarilyonfacultyobservation

anddiscussionswithstudents.

Graziano(2003)usedaquantitativesurveyofpostsimulationperceptions

(reportedasmeanscoresandpercentperceivingmoderatetoverygreatprogress)

supplementedbyqualitativeinterviewdatatotriangulateandfindstudentsagreedthey

madeprogressrelativetothoselearningoutcomestoanswertheresearchquestion:

“Whataretheperceptionsof...studentsabouttheirlearning...”Anadditionalthree

researchquestionsexploredcorrelationsbetweentheperceptionresultsand

demographicfactorssuchastakingtheprogrammorethanonce,beingtaughtby

differentinstructors,attendingremediationclasses,courselength,passingareading,

writingand/ormathematicsexamination,andwiththecoursegradeandstudentGPA.

ANOVAwasusedtoexplorethecorrelationsand,whileinteresting,theseresultswere

ancillarytothequestionofoveralleffectiveness.

Longitudinal methods/pretest‐posttest designs.Avarietyoflongitudinalmethods,

mainlyinvolvingpretest‐simulation‐posttestapproaches,arealsofoundintheliterature


onmeasuringtheeffectivenessofbusinesssimulationgames.Mostusea

preexperimentalpretest‐postdesignandsomeuseanexperimental‐controldesignto

demonstratesignificantgainsaftercompletingthesimulation(Campbell&Stanley,

1963).Whiledesirable,theuseofacontrolgroupisoftennotpracticalbecauseitwould

denysomestudentsthefulllearningexperienceencounteredbytheexperimentalgroup.

Studentattitudesandperceptionsareoftenmeasuredinthisresearchand,dependingon

thenatureoftheresearch,maybecorrelatedwithobjectivemeasurementssuchas

cognitiveexamscoresandthesimulationperformancescores.Manyusemixedmethods

triangulationbyincludingaqualitativeanalysisofstudentresponsestoquestionsabout

theexperience.Thefollowingpretest‐posttestexamplesarecategorizedbythenatureof

theprimarydependentvariableusedintheresearch.

Simulation performance score.Gamlath(2009)foundbusinesssimulationgame

performancewasduetoskillandnottoluckbycomparingtheresultsofalearning

roundandanexamroundfortwodifferentsimulationsusedintwosequentialcourses

withthesameteamsplayingeachgame.Allstudentsparticipatingreceivedapassing

gradeforsubmittingacceptableplanningdocumentsanddecisionjournals,andwere

awardedextrapointsbasedontheirsimulationperformancescores.Thesimulation

graderepresented10%ofthecoursegrade.Gamlathobservedthatallteamsperformed

similarlyinthebeginning;thehigherperformingteamstendedtoexperimentmore

duringthelearningroundandaskedtheinstructorfewerquestionsthanthelower

performingteams.ThisissimilartothefindingsofDavidovitchetal.(2006;2007;2008;

2009;2010)wheretheyfoundsimulationperformancescoresweresignificantly

improvedbyprovidingtheabilitytoreviewandredopastdecisions.Thesimulation

performancescorecanbeareliablemeasureoflearningifthegame‐playingskills

developedaretransferabletorealworlddecision‐making.Gamlathalsofoundno

correlationbetweenstudentperformanceonthesimulationandothertraditionalforms

ofassessmentleadinghimtoconcludethatdifferentskillsarebeingassessed.

Exam scores.Smalt(1999)foundusinganaccountingsimulationnearthe

beginningofanacademictermwaseffectiveinimprovingperformanceonstudentfinal

examinationscores.Performancewasmeasuredbysummingallthetestscores

administeredduringthetermforthetwogroupsbeingcompared.Onegroup

experiencedthesimulationgameearlyduringthetermandthecontrolgroupdidnot

experiencethesimulationgame.AChi‐Squareanalysisofdemographicdataindicated


thegroupswerehomogeneousbasedonacademicmajor,gender,andrace.Independent

t‐testsrevealednosignificantdifferencesbetweengroupsfortheotherindependent

variablesexceptacomparisonofthepretestofbasicaccountingknowledgescores

revealedthecontrolgrouphadabetterunderstandingofaccountingprinciplesthanthe

groupabouttoexperiencethesimulationgame.

Performanceofthesimulation‐experiencinggroupwasfoundtobesignificantly

betteronthefinalexamination.Postcoursesurveyanalysisindicatednosignificant

improvementinstudentperceptionsaboutaccountingasamajor,thequalityof

instruction,theaccountingcourseingeneral,orthequalityofclassroominteractions.

Theseperceptionswerenotconsideredsurprisingsinceoneinstructorconductedthe

bulkofthecourseandthesimulationwasexperiencedatthebeginningofthecourseand

ledbyadifferentinstructor.However,“89.9%ofexperimentalgroupparticipants

indicatedthatexposuretothesimulationhadapositiveimpactontheirattitudetoward

accountingingeneraland93.9%indicatedthegamehadimprovedtheirunderstanding”

(Smalt,1999).

Self‐assessed knowledge and competence.Seethamraju(2011)askedstudentsto

self‐assesstheirknowledgeandthecompetencegainedbeforeandafterthesimulation

usingaquestionnairethatwasadministeredinweekthreeofthecourse,justbeforethe

simulationgame,andinweek12,justaftercompletion.Fortyquestionswithresponses

usingafive‐pointLikert‐typescalefromverylowtoveryhighweregroupedintofour

constructsrepresentingfourknowledgedimensions:Processconcepts&terminology,

processsignificance&awareness,processmanagementandanalysis,andSAPskills.An

additional18questionswereaskedonthepostsimulationquestionnaireusinga5‐point

scalefromstronglydisagreetostronglyagreetoassessperceptionsandattitudesrelated

totheadministration,benefits,organizationandenvironmentofthegame.Thisscale

wasbiasedtowardsagreementasavalueof3correspondedtoagreeratherthana

neutralneitheragreenordisagree.Constructdefinitionsandthenumberofrelated

questionsweregivenforallthesedependentvariables.Examplesoftheunderlying

questionswerementioned;alistofquestionsusedisavailableonrequestfromthe

author.Participantsalsorespondedtonarrativequestionsaskingaboutthebestand

challengingaspectsofthegameandtooffersuggestionsforimprovingthegamedesign

anditsadministration.


Seethamraju(2011)usedpairedsamplest‐teststocomparethebeforeandafter

ratingsandfoundasignificantgaininself‐reportedknowledgeandabilityacrossthe

fourknowledgedimensions.Meandifferenceincreasesforthefourknowledge

dimensionconstructsrangedfrom1.54to3.8andweresignificantforp<=.01.

Postsimulationattitudescoresrangedfrom3.19to4.11forthefourconstructsand3.87

overall;allintheagreetostronglyagreerange.Thisoverallscorewasdeemed

significantlybetterthantheoverallratingof3.2observedforothersimilarcourses

withoutsimulation;thedetailsofthisanalysiswerenotprovided.Contentanalysisof

thenarrativefeedbackwasregardedas“generallypositive.”Seethamrajunotedstudent

commentspraisingtheaddedbenefitofdevelopinggroupworkskillsandobserved:

Participantsappreciatedthelearningvalueofthegameandreportedlygained

moresignificantskillsandknowledgethanfrompreviousteachingmethods.

Participantsingeneralfeltthegameisfun,excitingandinspiredlearningwitha

focusonprocessesratherthanIT,andassistedtheminappreciatingcross‐

functionalissuesandinterdependencies.(Seethamraju,2011)

Keyword counts and exam scores.Klein(1980;1984)usedananalysisof

keywordcountsmeasuredonmid‐termandfinalexamstoconcludetheuseofan

internationalbusinesssimulationtointroducetheconceptsofbusinessriskwas

moderatelymoreeffectivethanlecturealone.However,a68‐itemmultiplechoicetest

administeredbeforeandafterthesimulationshowednosignificantimprovement

betweentheexperimentalandcontrolgroupsandtheargumentwasmadethatthisis

theresultofproblemsassociatedwithmeasuringexperientiallearning,notthatno

learningoccurred:

Learningissaidtohavetakenplaceifanewbehavior,anewattitude,oranew

interestorvalueispresent.Theinferencehereisthatoneshoulddetectlearning

bycomparingbeforeandafterbehavior.

Withaprocessascomplexasasimulationgame,thetaskofisolatingthat

whichislearnedfromthetotalinvolvementoftheindividualisindeedcomplex.

Thedevelopmentofatestinstrumentwhichmeasuresspecificsbecomes

extremelydifficultinanexperientiallearningenvironment....


Inasimulationgame...questionsthataskforspecificresponsesmaybe

inappropriate.Therespondentsmaynotbeabletoverbalizearesponse

concerningaspecialconcept,yettheycanapplyittoadecision‐makingproblem.

...questionssuitedtothetextandlectureenvironmentmaynotbeappropriate

totheexperientiallearningmedium.(Klein,1984)

Kleinconcludedthat“simulationdoesindeedsensitizeitsuserstocritical

internationalbusinessissuesthatareanintegralpartofthegame’sdecision‐making

tasks(Klein,1980)andtheuseofa“simulationgameprovidesanopportunityto

exercisetheskillsandknowledgeacquired.Itprovidesaviewfromadifferentbridge”

(Klein,1984).

Attitude measures.WilliamsandWilliams(2011)measuredpre‐andpost‐

simulationattitudesandbehaviorsofstudentgroupswhoeitherplayedaMultiple

IdentificationTheorygameaboutinternationalconflicttwiceorwerepartofacontrol

groupthatwatchedadocumentaryaboutconflictthenplayedthegameonce.

Measurementsweretakenbefore,midway(aftercompletionofthefirstgameor

documentaryviewing)andfollowingcompletion.Thecontrolandexperimentalgroups

didnotdiffersignificantlyatthestartoftheexperiment.Followingtheexperiment,the

experimentalgroupindicatedasignificantchange;thecontrolgroupdidnotindicatea

significantchangeafterviewingthedocumentary,butdidindicateasignificantchange

afterplayingthegame.Thissupportedthehypothesisthatattitudeswouldchangeasa

resultofplayingthegame.Dependent(matchedpairs)t‐testswereusedtodetermine

significance.

Managerial and personality traits and decision‐making styles.Wellington,

HutchinsonandFaria(2012)usedapretest‐posttestquasi‐experimentinvolving460

undergraduatestudentsfromaPrinciplesofMarketingcoursetoconductexploratory

researchontheeffectofMerlin:AMarketingSimulationonstudents’managerialand

personalitytraitsanddecisionmakingstyles.Thestudentswerefromtwodifferent

iterationsofthecoursewhichranindifferentsemesters.Twopretestswere

administered.Thefirsttoassessgeneralmanagerialandpersonalitytraitsbeforethe

studentswereassignedasindividualstosimulationcompanies.Thesecondafter

studentswereassignedtosimulationcompaniesandreceivedalectureonthe

simulationtoassessdecisionmakingstyleandadditionalmanagerialandpersonality


traitsinthecontextofthesimulation.Thepostsimulationposttestrepeatedand

combinedthequestionsfromthetwopretests.Atotalof133questionsusingLikert‐

typescalesweredrawnfromtheliteraturetoconstruct18managerialandpersonality

traitvariables.Simulationperformancewasratedhighorlowbasedonranked

performancevs.theotherparticipants.RepeatedmeasuresMANOVAwasusedto

simultaneousexaminechangesovertimeandinrelationtosimulationperformance.

Wellingtonetal.(2012)foundanumberofstatisticallysignificantchangesand

relationships,buttheirfindingsareanexampleofwhensignificantdoesnotmeana

substantialchange.Forexample,statisticallysignificantchangeswerefoundforthe

variableBig5–Conscientiousnessbuttheaveragechangeforthehighperformance

groupwasfromapretestscoreof3.65toaposttestscoreof3.59,adecreaseof0.06ona

5‐pointscale;thelowperformancegroupchangedfrom3.50to3.42,adecreaseof0.08.

Sowhiletheresearchindicatesstudentconscientiousnessdecreasedsignificantlyasa

resultofplayingMerlin,thecasualobservermightnotfindthisdecreasetobe

meaningful.

Summary.Researchintotheeffectivenessofusingbusinesssimulationgamesas

experientiallearningactivitiesisconfoundedbythecomplexityofdefiningand

measuringlearning(P.H.Anderson&Lawton,2009;Chinetal.,2009;J.V.Gentry&

Burns,1981;Gosen&Washbush,2004;Greenlaw&Wyman,1973;Jones,1987;Stainton

etal.,2010).Someresearcherscontinuetousethesimulationscoreasadependent

variablerepresentinglearning,butmanyagreethatwinningthegameandscoringwell

onothersimulator‐generatedmetricsdoesnotprovelearning.Itispossibletolearn

whilescoringpoorlyandlow‐scorersmayactuallylearnmore.Lessonslearnedcan

includeself‐awarenessandinterpersonalskillsinadditiontoneworrefinedtechnical

knowledgeandskillrelatedtothetheoreticalunderpinningsofthesimulationmodel.

Researchmethodsintotheuseofbusinesssimulationgameshavetendedtorely

ofpostsimulationassessmentsofobjectivemeasuressuchassimulationandexamscores

andonstudent‐reportedperceptionsandattitudes.Longitudinalresearchmeasuring

presimulationandpostsimulationconstructshasbeenfoundusingbothobjectiveand

perceptivemeasures.Therecenttrendappearstobetowardconductingmore

longitudinalstudiestoshowlearningacrossthesimulationratherthantocompare

simulationandnonsimulationmodeswhichmaybedifficulttoarrangeinanacademic

degreeprogram.


Despiterepeatedcallsforgreaterrigorinmeasuringlearning,perceptionsare

stillwidelyusedtoconstructvariablesinsimulationgamingresearchbecausetheyare

easilymeasuredandbecausesucceedingexamscoresmaynotmeasurewhatthe

simulationgameintendedtoteach.Thisseemsreasonableconsideringtheteam‐based

simulationgameenvironmentismorestronglyrelatedtotheaffective,perceptualand

behaviorallearningmodesthantothecognitivelearningmode(Kolb,1984).More

discipline‐basedresearchiscalledfor.

Thenextsectioninthischapterexplorestheavailableresearchintotheuseof

projectmanagementsimulationgamesasexperientiallearningactivitiesinacademic

programs.Duetothesmallnumberofstudiesfoundandtheirapplicabilitytothisstudy,

eachisdiscussedinmoredetailthanwasdoneforthebusinesssimulationstudy

examplesprovidedinthissection.

Project Management Simulation Gaming Research

Researchontheuseofbusinesssimulationgamesinacademicprograms

indicatesthemostextensiveuseisingeneralmanagement,marketingand

strategy/policycourses,andtoalesserextentinmanagementscience,finance,and

accountingcourses;thereisnomentionofprojectmanagementsimulationgames(Faria

&Wellington,2004).However,15articleswerefoundexaminingtheuseofproject

managementsimulationgamesinacademicprograms.These15variedinmethodsfrom

qualitativeandquantitativestudiesrelyingonpostsimulationcomments/surveysand

instructorobservations(S.Al‐Jibourietal.,2005;S.H.Al‐Jibouri&Mawdesley,2001;

Collofello,2000;L.S.Cook&Olson,2006;J.M.Cooper,2011;Dantasetal.,2004;Dillman

&Cook,1969)tomorerigorousquantitativepretest‐posttestandexperimental‐control

groupstudieswithadescribedresearchmethodology(Davidovitchetal.,2006;

Davidovitchetal.,2007;Davidovitchetal.,2008;Davidovitchetal.,2009;Davidovitchet

al.,2010;McCreery,2003;Pfahl,2004).Thissectionreviewsthesestudiesand

recognizesaneedforadditionalresearchontheuseofprojectmanagementsimulation

gamesinacademicprograms.Thelessrigorousstudiesarediscussedfirstin

chronologicalorder,followedbythemorerobuststudies.Thefivearticlesby

Davidovitchetal.arepresentedinorderoftheirwritingtoprovidebettercontinuity.

Dillman and Cook (1969). Thearticle“SimulationinthetrainingofR&Dproject

managers”istheearliestreferencefounddiscussingprojectmanagementsimulationas

anELA.AtthetimeofitswritingandpresentationattheFebruary8,1969meetingof


theAmericanEducationalResearchAssociation,Dr.DillmanwasaPh.D.student

associatedwiththeEducationalProgramManagementCenteratTheOhioState

UniversityinColumbus.Ohio.HecurrentlyisthePrincipalofDillmanAssociatesinLos

Angeles,CAandanaffiliateoftheCerritosPsychologicalCenter.Dr.Cookwasafull

professorintheCollegeofEducationDepartmentofPsychology.

Thisarticleispreviouslydiscussedonpage11followingtheheadingProject

ManagementSimulationGames.Itincludeselevenreferences,sixoftheserelatedto

simulation,threetoprojectmanagement,andoneeachtosocialsystemsanalysisand

teachingproblemslaboratory.

Followingtheintroductionoftheneedforamorerealisticexercise,Dillmanand

Cook(1969)brieflyintroduceprojectmanagementandsimulationandthendescribethe

developmentprocessforthesimulationexerciseandmaterials.Evaluationwasbasedon

qualitativeanalysisofresponsestotenopen‐endedsurveyquestions.Responseswere

categorizedaspositive,negative,neutral,suggestion(novalueattribution),positive‐plus

suggestion,negative‐plussuggestion,ornoresponse.Mostofthequestionswererelated

tothesimulationadministrationandfeedbackforimprovement,i.e.,game

administrationandorganization,gamerealism,sessiontimelength,information

provided,correlationwithpriorinstructionalsessions,valueofroleplaying,staff

feedback,andclarityofdesireddeliverables.DillmanandCook(1969)found:

Themostpositivereactionsweretowardtherealismofthesimulationexercise

...andthecorrelationofthesimulationsessionswiththepreceding

instructionalsessions....Themostnegativereactionsweretothetimelengthfor

eachsession...theexplicationoftherolestobeplayedandthevalueofrole

playing...andthefeedbackfromstaff....Thelargestnumberofneutral

responseswereobtainedontheitemsconcerningthetimelengthofeachsession.

Onaseparatecritiqueform,thesimulationreceivedthehighestmentionasbeing

especiallyfacilitativetolearning.(Dillman&Cook,1969)

Whilefarfrombeingconsideredarigorousacademicarticle,theresultsindicate

theparticipantsfoundvalueintheprojectsimulationELAduetoitsrealismand

reinforcementoflearning.


Collofello (2000).CollofellodescribesacollaborativeeffortbetweenArizona

StateUniversityandMotorolaUniversitytodevelopasoftwareprojectmanagement

trainingcourseincorporatingaSystemDynamicssimulatorofthesoftwaredevelopment

process.Dr.CollofelloisaProfessorintheDepartmentofComputerScienceand

EngineeringatArizonaStateUniversityinTempe,Arizona.

Hisarticlecites15references,oneontrainingeffectiveprojectmanagersandthe

restrelatedtosoftwaredevelopment,modelingorprocesssimulation.Sevenofthe

citationsarefromAmericanProgrammer(laterrenamedCutterITJournal),a

practitionerjournal.Threeoftheotherreferencesaretosoftwareengineering

textbooks,andtwoaretoproceedingsfromsoftwareprocesssimulationmodeling

workshops.

Followingabriefintroductionlistingmethodstoimprovesoftwareproject

managementskillsandadvocatingforexplorationviaasystemdynamicssimulation

model,Collofello(2000)providesanoverviewofsystemdynamicsmodelingandhowa

simplemodelcandescribetheinterrelationsbetweentimespentonqualitytasks,

undiscovereddefectsandabilitytomeetschedule.Hethenbrieflydescribesthe

softwareprojectmanagersimulatorandfivecourseexercisesthatuseit.

TheinitialdeploymentwasinaMotorolaUniversityprofessionaldevelopment

coursethatraninthefallof1999andspringof2000withallparticipantsreportingthe

“useofthesimulatoraddedsignificantlytothevalueofthecourse”(Collofello,2000).

Collofelloclosedbystatinga“web‐basedgraduatelevelsoftwareprojectmanagement

courseisalsobeingdevelopedatASU[ArizonaStateUniversity]utilizingthesimulator.”

Whilenotanacademicresearcharticle,thisarticleisanexampleofthe

acceptanceofthevalueofsimulationasanELA.Thismessagecouldhavebeen

strengthenedwithamorerigorouspostcourseevaluationprocessataminimumand

greatlyenhancedwithaproperlyconstructedresearchstudy.

Martin (2000).MartindescribesasimulationcalledContractandConstructand

itsapplicationinteachingprojectmanagementatthegraduateandundergraduatelevel

atWarwickBusinessSchoolwhereheisalecturerinInformationSystemsand

Computing.

Twenty‐sixreferencesarecited:11relatedtobusinessandinformationsystem

simulationsandsimulationgames,eightrelatedtoprojectmanagement(twoofthese


fromatrainingperspective),sixrelatedtogeneralmanagement(onerelatedto

education),andoneonteachingpreparation.

Followinganintroductionofprojectmanagementskilldevelopmentandthe

benefitsofsimulation‐basedtraining,Martin(2000)describesthesimulationandits

primaryuseinteachingprojectmanagementinanMBAcourse.Thisisfollowedbya

discussionoftheunderlyingsimulationengine(database,schedulingengine,user

interface,simulationgeneration)andadiscussionofpossibleenhancementsand

implementationvariations.

Thesimulatedprojectisan18‐activitychemicalplantconstructionprojectwitha

simulateddurationof94weeksandabudgetof₤7million.Studentsaredividedintofive

groups.Eachgroupistoldtoeitherfocusonacost,schedule,qualityormoraleobjective,

ortobalanceallfouroftheseobjectives.Afterreceivingabriefingdocumentandashort

presentationaboutthesimulatedproject,studentsanalyzethegiveninformation,select

contractorsforthe18activities,andexecutethesimulatedproject.Duringexecution

theyreceiveprojectstatusinformationandrespondtoeventsaffectingthecost,

duration,quality,moraleandsafetyoftheproject.Performanceisassessedineachof

thesefiveareas.Safetyisimportantnomatterwhattheprimaryobjectiveandallgroups

mustobtainthesameminimumacceptablesafetyscore.Witheachgrouphavinga

differentprimaryobjective,avarietyofstrategiesareimplementedprovidingdiverse

resultsandarichdebriefdiscussionafterconclusionofthesimulationgame.(Martin,

2000).

Performanceagainsteachofthesefiveobjectivesisscoredonacumulativebasis

throughouttheprojectandanoverallscoreiscalculatedbyaveragingthefiveelements

withadoubleweightgivenfortheemphasizedobjective.Afterthesimulation,students

prepareashortpresentationandreflectonwhathappened,howsuccessfultheywereat

implementingtheirstrategy,andwhattheylearned.Theeducatorthenleadsa

discussioncomparingperformanceacrossthegroupsandnotingtheirdifferent

objectives.(Martin,2000).

Martin(2000)presentedatableofhypothesesandanalysisofscoresgainedby

59groupsdemonstratingthatstudents,asexpected,scorebetterintheirareaof

strategicfocus.Itisnotclearhowmanystudentsactuallyparticipatedinthisstudyorif

theyworkedindependentlyorinteams.StudentreactiontotheELAisreportedasbeing


generallyfavorable,butnoindicationisgivenastohowthiswasdetermined.Martin

(2000)summarizedthekeylearningpointsas:

Understanding,identifyingandmanagingthenatureofthecriticalpath...

Practiceisdifferentfromtheory....

Decisionshavetobemadeunderuncertainty.

Theconsequencesofdecisionshavetobefaced.

Planningisacoreactivityinprojectmanagement.

Martin(2000)reportsthesimulationwasalsousedsuccessfullyinanon‐

academicprofessionaldevelopmentsettingatanelectricitygeneratingcompanywith

teamsofpeople,butnodetailswereprovided.

Thisarticledescribedthesimulationanditsapplicationatahighlevel;however,

itislackingthedetailsandrigorneededtobeconsideredascholarlyarticleortobe

reproducedbyanotherresearcher.Itsmaincontributionisprovidinganexampleofa

projectmanagementsimulationgamethatisbeingusedinbothacademicand

commercialprograms.

Al‐Jibouri and Mawdesley (2001).Al‐JibouriandMawdesleydescribethe

developmentandinitialuseofaninternethostedsimulationgamewithgraphicaluser

interface(GUI)toteachprojectplanningandcontrol.Bothauthorsareaffiliatedwith

universitiesinWesternEurope.Theyprovideashorthistoryofconstruction

managementgamesandthebenefitsofusinggamesfollowedbytheconsiderationsand

objectivesfordevelopingagame;adescriptionofthesimulatedproject(arockandclay

filldamwiththeplayertakingontheroleofthecontractor’sprojectmanager

responsibleforplanning,resourceselectionanduse,controlandreportingto

management);adescriptionofthestructure,mainfeaturesandGUI;andabrief

descriptionofitsusein“anundergraduatecourseinplanningandcontrolandasa

demonstrationtoolinanintensivecourseonplanningandcontrolforpracticing

engineersandconstructionmanagers.”Thearticleisprimarilyadescriptionofthe

developmentandinitialuseofthesimulation.Althoughsomequalitativeassessment

dataisprovided,itisnotarigorousstudyevaluatingitseffectivenessasalearningtool.

Al‐Jibouri&Mawdesley(2001)citethirteenreferences:fivearticlesdescribing

constructionmanagementandschedulinggames,threearticlesprovidingan


introductiontobusinessandmanagementgames,twoconstructionmanagement

textbooks,twoarticlesonmodeling,andonearticleoneffectiveteaching.Elevenof

thesereferencesarecitedintheintroductionandhistoryofconstructionmanagement

gamesectionsandtheremainingtwoarecitedinthesectiondiscussingreasonsfor

developinggames.

Studentsworkedinsmallteamsof2to4playersintheundergraduatecourse

andinitiallytendedtoignoretherisksandbeoptimisticintheirplanning.When

executiondidn’tgoaccordingtoplan,theyfirstignoredcostswhileattemptingtohold

scheduleandlaterengagedinformalre‐planninginanattempttoalsocontrolcost.

Despiteonlyfairresultsinmeetingthecostandschedulegoalsoftheproject,qualitative

analysisofthenarrativequestionnaireresponses,informaldiscussionfeedbackfromthe

studentsandtheobservationofthesupervisingacademicsprovidedevidencethatthe

studentsmade“substantialimprovementsduringthegame”bylearninghowtoinclude

projectriskmanagementintheirplanningandcontrolactivities.(S.H.Al‐Jibouri&

Mawdesley,2001)

Fewdetailswereprovidedabouttheuseofthesimulationintheintensivecourse

forpracticingmanagersandfeedbacksuggestedthesimulationgame“shouldbeused

aloneinthefutureandthatthe‘lecture’sessiononthesubjectwasnotrequired”(S.H.

Al‐Jibouri&Mawdesley,2001).

Al‐JibouriandMawdesley(2001)concludethat“throughcarefulconsiderationof

theuserinterfaceandselectionofthecorrectprojectitispossibletobuildagamewhich

addssignificantlytothetraditionalcoursestructureforsuchatopic.”Theyobservethat

bothstudentsandpracticingengineersappearedtoliketheexperientiallearning

providedbythegameand“particularlyvaluedtheexperienceofbeingina‘reallife’

situationwheretheycanmakedecisionsandseetheconsequencesofsuchdecisions.”

Onassessingtheeffectivenessofthislearningexperience,Al‐JibouriandMawdesley

opine:

Itwouldbedifficulttodetermineifanymethodofeducation/trainingismore

successfulthanothers.Measuresof‘successfulness’and‘effectiveness’ofa

particularmethodwillbeanamalgamoffactorsandresultsthataresometimes

difficulttoisolate.However,itisbelievedthattheobservationofandfeedback


fromparticipantssuggestthegameoffersbenefitsinsomeareasovertraditional

methods.

Assessmentoftheperformanceofthestudentsonthegamehasshown

considerableimprovementsduringthegamewhichindicatesalearningprocess

thatisnormallydifficulttomeasureintraditionalteachingmethodssuchas

lecturesandexams.

Noquantitativeevidenceofthis“considerableperformanceimprovementwhile

playingthegameisprovided”andanunresolvedissueishowtoassessstudent

performanceforcoursegradingpurposes.Al‐JibouriandMawdesley(2001)suggest

threepossiblemethods:(1)assessingtheteampresentationsduringthereviewswith

management,(2)assessingperformancewhileplayingthegame,and(3)administeringa

traditionalexamattheconclusionofthecoursebasedongame’slearningobjectives.

Thisarticleprovidedadescriptiveoverviewofaprojectmanagementsimulation

gameusedbynon‐competitiveteamsinundergraduateandpostgraduateconstruction

managementprograms.Theinternet‐hostedsimulationishighlightedbyagraphical

userinterfacewithpictures,videoclips,animation,textualandgraphicreports,and

eventstriggeredbyplayerdecisions.SimilartotherecommendationbyClarkandMayer

(2008)thatmoreisnotnecessarilybetter,70%ofthestudentsdidnotrecommend

addingmorevideotothegameandstudentopinionsweresplit50:50onwhethermore

animationshouldbeadded.Thoseformorevideo“thoughtitwouldbeinteresting”(not

thattheywouldlearnmore)andthoseopposedthoughtmorevideoandanimation

wouldwastetimeandnotaddanythingimportanttotheexperience(S.H.Al‐Jibouri&

Mawdesley,2001).Thisarticledidnotincludestatisticsdescribingtheclasssizeand

composition,numberofcoursesthesimulationwasusedin,etc.

Notablefindingsfromthequalitativeanalysisofthepostsimulation

questionnairedataare75%ofthestudentsthoughtlectureswereneededtosupplement

thesimulationtoprovidethebackgroundtheoryonprojectcontroland90%suggested

theycouldlearnbetterfromthegamethanlecturesthattheoriginalplanisrarelyideal

andunexpectedeventswilloccurandrequirere‐planning(S.H.Al‐Jibouri&Mawdesley,

2001).Althoughthisstudycouldbeimprovedbyapplyingamorerigorousassessment

methodology,thegenericpostsimulationnarrativequestionsmightbeusefulinfuture


research.Otherthanidentificationoftheneedtodevelopperformanceassessment

measures,Al‐JibouriandMawdesleymadenorecommendationsforfutureresearch.

Dantas, Barros and Werner (2004).Dantasetal.believe“theinadequateuseof

projectmanagementtechniquesinsoftwareprojectscanbetracedtothelackofefficient

projectmanagementeducationstrategies,wherelearningbyexperienceandmotivation

arekeyissues.”TheydevelopedaprojectmanagementgamebasedonaSystems

Dynamicsmodelofasoftwareprojectandpresenttheresultsoftwoexperimental

studiesconductedtodemonstrate“theusefulnessofgamesinprojectmanagement

trainingprograms.”Dantasetal.areaffiliatedwiththeUniversidadeFederaldoRiode

JaneiroinBrazil.

Seventeenreferencesarecited:fiverelatingtoSystemsDynamics,fiveto

softwareprojectsandmanagementstrategies,fourtotheuseofsimulationsinsoftware

engineeringtraining,andoneeachtoandradogy(adultlearningstrategies),digital

game‐basedlearningandsimulationeffectiveness.

Followingadiscussionoftheineffectivenessofeducatingadultlearnerswith

content‐centricapproachesandtheadult‐learnerpreferenceforexperientialand

application‐basedlearning,Dantasetal.(2004)brieflydescribethedevelopmentand

operationoftheirSystemsDynamics‐basedsimulationmodelfollowedbyasummaryof

theirlearningevaluation.

Thestudywasconductedwithstudentsstudyingsoftwaredevelopmentattwo

universitiesinBrazil.Therewere7studentsinoneclassand8intheother.Allreceived

20minutesoftraininginprojectmanagement,notablyfunctionpointestimating,andin

theuseofthesimulationpriortoplayingthesimulationgame.Theonlyresultsprovided

wereinasingletablesummarizingthestudentopinionsonfivedimensionsoftheELA

experience:“PMSkill”and“InterestinPM”withtheresponseoptions:“raised...

indifferent...reduced;”“Game‐basedTraining”withtheresponseoptions:“good...

indifferent...bad;”“PresentedLessonsLearned”withtheresponseoptions:“all...none

...lots...few;”and“Wasthetrainingfun?”withtheresponseoptions:“yes...no...

much...little.”Onlyonestudentcompletedthegamewithsuccess;however,the

studentfeedbacksummarizedtheexperienceas“motivating,dynamic,practicaland

enjoyable.”(Dantasetal.,2004)

Thisarticlediscussedasimulationgamedevelopedbytheauthorstoaidinthe

developmentofsoftwareprojectmanagersanddemonstratedthatprojectmanagement


learnersreactfavorablytotheinclusionofaprojectmanagementsimulationgameinan

educationalprogram.Limitationsofthisstudyincludethesmallsamplesizesand

researchapproachlimitedtoanalyzingpost‐ELAattitudestowardprojectmanagement

andthesimulationgame.

Al‐Jibouri, Mawdesley, Scott and Gribble (2005).Inthissequeltothe2001article

byAl‐JibouriandMawdesley,Al‐Jibourietal.(2005)reportinadifferentjournalon

usingthesamesimulationmodelofanearthmovingprojectasamanagementgamefor

teachingconstructionprojectmanagement.Asbefore,playingthegamerequired

studentstoplan,makedecisionsunderuncertainty,anddealwithenvironmentaleffects

andfinancialresults.Thefirstthreeauthorsarecivilengineersteachingincivil

engineeringdepartmentsorschoolsatuniversitiesinTheNetherlands,theUnited

KingdomandAustralia.ThefourthauthoristheTeachingFellowforstaffatthe

Australianuniversityresponsibleforimprovementofteachingandlearning.

Ninereferencesarecitedinthisarticle,fourrelatedtoconstructionmanagement

andschedulinggames,threerelatedtoconstructionmanagement/costcontrolmodels,a

genericarticleonbusinessandmanagementgames,andamanualforacomputer

managementgame.The2001articlebyAl‐JibouriandMawdesleyisnotcitedeven

thoughtheopeningparagraphintroducingtheimportanceofprojectplanningand

controlisanear‐verbatimcopy.

Inthisarticle,Al‐Jibourietal.(2005)skiptheintroductoryhistoryof

constructionmanagementgamesandreasonsfordevelopinggamesfoundinthe2001

articleandprovideatwoparagraphdiscussionaboutthedifficultyinprovingthe

effectivenessofthetechniquestaughtinthesecoursesandthedifficultyofassessingthe

effectivenessandefficiencyofthemethodusedtoteachthem.Thelegitimacyof

simulationasapedagogicaltoolisthensuggestedbythetwosucceedingparagraphs

describingthenationalcivilengineeringgamerunannuallybyLoughboroughUniversity

intheUK(withthewinningteamawardedaprizeandpublicityintheprofessional

press)andanotherconstructionmanagementgameusedinindustrialratherthan

academicenvironments.Withtheexceptionofanupdatetoacitationontheindustrial

game,thesetwoparagraphsareidenticaltoparagraphsfoundintheun‐cited2001

articlebyAl‐JibouriandMawdesley.


Similartothe2001article,thisarticledescribesthesimulationgameand

providesexamplescreenshotsofthesiteplan,decisionscreenandtextandgraphical

reportscreens.Thissectionisre‐writtenbutcontainsessentiallythesameinformation.

Differentinthisarticleisthe“experiencewiththegame”section.Ratherthana

summaryoftheresultstoallthepostsurveyquestionnaireresponsesasinthe2001

article,presentationofresultsislimitedto18examplesofnarrativeresponsestoonly

onepostsimulationsurveyquestionasking,“Whatifanythingdidyoulearnfromthe

game,whichcouldnothavebeenlearnedmoreeasilyfromtraditionallectures?Please

givereasonsforyourstatements”(S.Al‐Jibourietal.,2005).Theseweresummarizedas:

Themainpointsreportedbythestudentsinclude:

Thedifferencebetweentheoryandpractice

Theimportanceofobtainingrealisticratherthanoptimisticdata

Theimportanceofcontrol

Theneedforplanningandcontrolevenwhenfacedwithanuncertain

world(S.Al‐Jibourietal.,2005)

Asectionnotfoundinthe2001articlebyAl‐JibouriandMawdeslydescribesthe

useofasummaryplayerperformancemeasurebasedonalinearcombinationofthe

differencesbetweencurrentandplannedexpenditure,income,[cash?]balance,clay

heightandrockheight.Theresultsfor11teamsof3playerseachareshownasagraph

oftheaverageofthismeasureforthe11teamsandasagraphshowingtheprogressof

eachteamrelativetotheirownworstscore.Usingthisscore,Al‐Jibourietal.(2005)

found,onaverage,teamperformanceisgoodduringtheopeningsimulationweeksofthe

gamewhenthesimulatedweatherisgoodanduncertaintyeffectsareminimal.Thisis

followedbyalossofcontrolandmuchworsescoreswhenuncertaintiesaffect

performance.Thentheaverageperformancemeasureshowsanimprovementtrend

throughthecompletionoftheproject.Thegraphofindividualgroupperformance

relativetotheirownmaximumperformancemeasureshowsthatsomegroups

performedmuchbetterthanotherswiththebestgroupsabletoconsistentlyimprove

theirperformancescorethroughoutthesecondhalfoftheprojectandtheworstgroups

onlyabletostabilizetheirperformance.Theauthorsgavenoindicationofwhetherthis

measureaffectedthecoursegrade.(S.Al‐Jibourietal.,2005).


Alsonewtothisarticleisadescriptionoftheuseofthegameasarecruit‐select

toolbyaconstructionservicescompany.Inthisapplicationstudentswereinvitedto

attendafoursessiononeeveningperweekcourseconsistingoflecturesandthegame.

Thelectureswereprovidedbyseniorpeoplefromthecompanyontopicsrelevanttoits

operation.Observationofstudentsplayingthegameprovidedvaluableinsightsthat

wereusedbythecompanyinmakingemploymentdecisions.

Thecompanywere[sic]verypleasedwiththegameespeciallyinthewaythatit

demonstratedthecomplexityofrealprojectsandtheneedtoconsidersomany

apparentlyunrelatedtopicsinordertomanageit.Theycommentedonhowwell

itenabledthemtoobservethebehaviouroftheparticipantsinanear‐realistic

environment.(S.Al‐Jibourietal.,2005)

Thisarticleprovidedanecdotalevidenceontheefficacyofusingaconstruction

projectmanagementsimulationgameandanexampleofaquantitativeassessmentof

learningtechniqueforasingleclassof33students.Similarresultsfromrepeateduseof

thistechniquewouldstrengthenthevalidityoftheresultsandconclusions.Thisuseofa

runningperformancemeasurederivedfromthesimulationresultsisatechniquefor

furtherinvestigation.

Ifweconsidertheadditionoftheperformancemeasurequantitativeanalysisand

thediscussionconcerningtheuseofthesimulationgameasanemployeeselectiontool

tobethecoreofthisarticleandthatthesubstantialrepeatofpreviouslypublished

materialasnecessarytounderstandthisnewcontribution,thentheauthorsmaynotbe

technicallyguiltyofself‐plagiarism;butitwouldhavebeenbettertocitetheprior2001

articlebyAl‐JilbouriandMawdesleyatthebeginningofthis2005article(American

PsychologicalAssociation,2010).

Cook and Olson (2006).CookandOlsondescribetheuseofateamexperiential

learningactivity(ELA)simulatingaskyscraperconstructionprojectusingspaghettiand

miniaturemarshmallowsasapredecessortothelectureonprojectmanagementinan

operationsmanagementcourse.Dr.OlsonisDepartmentChairoftheOperationsand

SupplyChainManagementDepartmentattheUniversityofSt.ThomasinMinneapolis,

MNandDr.CookisanAssociateProfessorintheDepartmentofManagementatDePaul

UniversityinChicago,IL.Theirarticleprovidesanoverviewoftheexercise,


administrationadviceforinstructors,andadiscussionofevidenceofeffectiveness.

Theirguidanceincludesstressingtheneedtoestablishthebasisfortheexercisewiththe

studentsandtotieittothelecturelearningobjectivesthroughouttheactivity.

Fourteenreferencesarecited;sevenarearticlesrelatedtoteachingtechniques

(twooftheseonexperientiallearningandtwooncriticalthinking),twodescribingthe

useoftowerbuildingexercisesinmanagementeducation,oneonmakingfaststrategic

decisions,andtextbooksonorganizationalpsychology,experientiallearningandusing

MicrosoftProject,andTheGuidetoProjectManagementBodyofKnowledge(PMBOK®

Guide).

CookandOlson(2006)usedaquasi‐experimentalpretest‐posttestresearch

designwherestudentswereaskedtoratetheiragreementusinga7‐pointtypeLikert

scale(from1=stronglydisagreeto7=stronglyagree)withthestatement“Ifgiventhe

examtoday,Iwouldreceivefullcreditforthequestionlisted...”followedbythe

learningobjectivesforthecourse.Thearticlelistedsevenlearningobjectivesforthe

projectmanagementmodel;theauthorsreportedtheresultsoft‐testsforthefour

objectivesrelatedtoboththelectureandtheELAshowingasignificantincreaseinthe

meanscoreforallfourofthesequestions.Thepretestwasadministeredatthe

beginningofthecourseandtheposttestonthelastdayofthecourseandincludeda

similarassessmentofalltheothernon‐projectmanagementmodulesinthecourse.

AttributiontotheELAwasmadebyaskingstudentstoratetheoverall

effectivenessofthesimulationexerciseona5‐pointLikert‐typescale.Providingno

details,CookandOlson(2006)reported:

Morethantwo‐thirdsofthestudentsreportedthatitwaseffectiveinhelping

themunderstandtheclassmaterial.Acrossthevarioussections,the

overwhelmingmajorityofstudentsfoundtheexercisetobeanextremely

effectivemeanstoteachprojectmanagement.

Theauthorsconcedethat“althoughthedatademonstratelearning,itisdifficult

toisolatethespecificattributesoftheclassexerciseversustheensuingclasslecture”and

“theattributesspecifictotheclassexerciseandclassinstructionmaybeconfounded”(L.

S.Cook&Olson,2006).


Thisarticleconcludedthatbothstudentsandfacultybelieve“theuseofanELA

canbeaveryeffectivemethodtopromotehigherlevelsoflearning”(L.S.Cook&Olson,

2006).However,theinternalvalidityoftheseresultsissubjecttotheeffectsof

extraneousvariablessuchashistoryandmaturationduetothetestsnotbeing

administeredimmediatelybeforeandaftertheELAandbecausetheposttestis

administeredattheendofthecourseafterboththeELAandthesucceedinglecture

withoutalecture‐onlycontrolgroup(Campbell&Stanley,1963;T.D.Cook&Campbell,

1979).

Cooper (2011). CooperevaluatedthesingleplayeruseoftheHarvardBusiness

PublishingProjectManagementSimulation:Scope,Resources,Scheduleinan

undergraduateclassofmanagementmajorsconcentratinginprojectleadership.Dr.

CooperisanAssociateProfessorintheManagementandFacilitiesDepartmentat

WentworthInstituteofTechnology.Thesimulationisusedinthefirsttwooffour

coursesrequiredforaBachelorofSciencedegreeinmanagementwithaconcentration

inprojectleadership.

Sevenreferencesarecited:threearticlesdiscussingsimulationfidelityand

assessment,threeonprojectmanagement,andthesimulationfacilitator’sguide(J.M.

Cooper,2011).

Cooper(2011)reportsonusingthesimulationwithaclassof34undergraduate

studentsintwoin‐classandthreeout‐of‐classassignments.Evaluationconsistedof

analyzingthreeitems:(1)studentresponsestoaquestionnaireadministeredfollowing

thesimulationexercises,(2)studentdiscussionboardpostsabouthowtokeepaproject

onschedule,and(3)answerswritteninresponsetoanopen‐endedfinalexamquestion.

Mostoftheitem(1)postsimulationquestionsrelatedtotherealismofthesimulation

andthebelievabilityoftheresults.Studentsidentifiedmanythingstheylikedaboutthe

simulationandtoaquestiondirectlyaddressingthelearningexperience,71%of

respondentsagreedthatthesimulation“provide[d]avalidlearningexperienceand/or

assessmentoflearning.”Fourteenpercentwereneutralandfifteenpercentdisagreedor

stronglydisagreed.Cooper(2011)noted:

Studentsdidnotapproachthesimulationasacomputergame.Itisimportantfor

anacademicsimulationthatoutcomesarenotdeterminedbychanceorluck;

insteadusersexperiencetheconsequencesoftheiractions.Aneffective


simulationexperienceshouldnotseemlikerollingdice!Participantsneedto

taketheirroleseriously.Halfofthestudentsagreedthatthesimulationresults

werenotdeterminedbyluckorchance[24%wereneutral;26%

disagreed/stronglydisagreed].

Studentswerelesscertainaboutthesimulation’sbelievabilityandrealism.

Cooper(2011)believesthisisduetothestudents’perceivedinabilitytoeasilyinfluence

themoraleofthesimulatedprojectteam.

Thesimulationisscoredwitha1000pointmaximumwith725pointsavailable

formeetingtargetsand275pointsavailableforexceedingthem.Studentsarepermitted

toplaythescenariosmultipletimesforeachassignmentandbecomefamiliarwiththe

simulationlogicandcasualrelationships,especiallytheinterrelationshipbetween

scheduleandbudgetandthedifficultyoffinishingbothontimeandonbudget.These

insightssuggestthesimulationiseducationallyvalidasdescribedbyFeinsteinand

Cannon(2002).(J.M.Cooper,2011)

Thisarticleprovidedanexampleofusingpost‐ELAstudentfeedbacktoassess

theefficacyofaprojectmanagementsimulationgame.Basedonthisfeedback,Cooper

(2011)concludesthescoringdashboardbuiltintothesimulationandtheabilityof

studentstoredoeachsimulationassignmentmultipletimessupportsskilldevelopment.

McCreery (2003).McCreeryevaluatedtheintegrationofasimulationfroma

three‐daycommercialprojectmanagementtrainingprogramintoaonesemesterproject

managementcourse.Dr.McCreeryisanAssociateProfessorofOperationsand

InnovationManagementandDirectoroftheMasterofGlobalInnovationManagement

PrograminthePooleCollegeofManagementatNorthCarolinaStateUniversity.The

computersimulationwasManagingbyProject,aproductofDavis&Dean,Inc.Davis&

Deanisaglobalprojectmanagementeducationcompanywhoclaimstohavetrained

morethan75,000projectmanagersaroundtheworld(Davis&Dean,2011a).Deanand

Davisprominentlyfeaturethisarticleontheirwebsiteandclaim“[McCreery]found

significantlearninginallsixteenprojectmanagementareas”(Davis&Dean,2011b).A

moreaccurateclaimwouldrelatetoMcCreery’sfindingofsignificantperceptionsof

learning.


Tenreferencesarecited,fiveonprojectmanagementandprojectmanager

competencymodels,twoonexperientiallearning,twoonsimulationandoneisthe

manualfortheSPSSstatisticalanalysissoftwarepackage.

Thisquasi‐experimentalstudyusedapretest‐posttestmethodologyinvolving

twoseparategraduate‐levelprojectmanagementclassestaughtinoneacademicyear

andatotalof63students(29inoneclass,34intheother;datawerecombinedforthe

twoclasses).Projectmanagementconceptsweretaughtduringthefirsttenweeksofthe

semester,thesimulationexercisewasadministeredduringweeks11through15,and

thecoursewassummarizedandwrappedupinthefinalweek16.Priortoattending

classduringweek11,studentscompleted1‐2hoursofout‐of‐classpreparatoryreading

regardingthesimulation.Classtimeamountingto3‐3.5hoursperweekduringweeks

11‐15wasdevotedsolelytoperformingthesimulation.(McCreery,2003)

Studentsweregroupedintoteamsofthreetofivepeople.Eachstudentteam

managedasimulatedprojectcontaining24activitiesandabaselinescheduleof22

weeksand$800,000budget.Teammemberscollaboratedtomakedecisionsinvolving

resourceallocation,training,quality,andproblemresolutionforeachsimulationweek.

Performancemetricsarecostminimization,scheduleattainmentanddeliverablequality.

(McCreery,2003)

Researchquestionswerenotexplicitlystatedbutdescribedasinvolvingthe

assessmentofknowledgeincreaseandabilitytoapplythatknowledgeasaresultof

participatingintheELAandexploringtheseinrelationtothestudents’experiencelevel

andqualityofthestudents’teamprocessduringtheexercise.McCreery(2003)tested

thefollowinghypotheses:

H1(a)Participantswillassesstheirlevelofprojectmanagementknowledgeas

greateraftercompletingtheexercisethentheywillpriortoperformingthe

exercise....,

H1(b)Participantswillassesstheirabilitytoapplyprojectmanagement

knowledgeasgreateraftercompletingtheexercisethentheywillpriorto

performingtheexercise....,

H2(a)Priortoperformingtheexercise,thelevelofprojectmanagement

knowledgewillbegreaterforparticipantswithhighamountsofprojectwork

experiencethanforparticipantswithlowamountsofprojectworkexperience....,


H2(b)Frompreexercisetopostexercise,themagnitudeofimprovementinthe

levelofprojectmanagementknowledgewillbegreaterforparticipantswithlow

amountsofprojectworkexperiencethanforparticipantswithhighamountsof

projectworkexperience....,

H3(a)Priortoperformingtheexercise,theabilitytoapplyprojectmanagement

knowledgewillbegreaterforparticipantswithhighamountsofprojectwork

experiencethanforparticipantswithlowamountsofprojectworkexperience....,

H3(b)Frompreexercisetopostexercise,themagnitudeofimprovementinthe

abilitytoapplyprojectmanagementknowledgewillbegreaterforparticipants

withlowamountsofprojectworkexperiencethanforparticipantswithhigh

amountsofprojectworkexperience....,

H4(a)Participantsoflowperformingteamswillshowsmallerimprovementsin

levelofknowledgeandabilitytoapplythatknowledgethanwillparticipantsof

highperformingteams....,

H4(b)Participantsinvolvedinanegativeteamprocesswillshowsmaller

improvementsinlevelofknowledgeandabilitytoapplythatknowledgethan

willparticipantsinvolvedinapositiveteamprocess.

Dataonknowledgeandabilitytoapplythatknowledgewerecollectedviapretest

andposttestquestionnaireswithidenticalquestionsusingasevenpointLikert‐type

scalerangingfromextremelylow(1)toextremelyhigh(7).Studentsrespondedtothe

instructions“assessyourcurrentlevelofknowledgeineachofthefollowingareas”and

“assesshowconfidentyouareinyourabilitytoapplythisknowledgeineachofthe

followingareas”forthe12projectmanagementcompetenceitemslistedinTable1.

Teamperformancedatawerecollectedfollowingthesimulationwithstudents

respondingtotheinstruction“assessyourteamexperiencethroughoutthesimulation

exercise”usingasevenpointLikert‐typescalewithchoicesrangingfromstrongly

disagree(1)tostronglyagree(7)fortheteamexperienceitemslistedinTable2.

Additionaldatacollectedincludedthestudents’yearsofprojectworkexperience.

(McCreery,2003)


TABLE1‐PROJECTMANAGEMENTCOMPETENCEITEMS

1. Project organizational design 2. Project planning in general 3. Assessing and managing project risks 4. Estimating project scope 5. Sequencing of project activities 6. Estimating project activity times 7. Project budgeting 8. Allocating project resources 9. Project change management 10. Designing project performance measures 11. Performing as an effective project leader 12. Building team consensus 13. Negotiating for resources and budgets 14. Using earned value concepts 15. Evaluating the performance of project personnel 16. Managing project uncertainty

TABLE2‐TEAMEXPERIENCEASSESSMENTITEMS

1. The workload was fairly balanced across all team members 2. Team members cooperated well throughout the exercise 3. Our team worked through the exercise in an efficient manner 4. Team members all participated equally in the team decision making process 5. Our team maintained a pleasant working atmosphere 6. Our team worked out disagreements in an equitable manner 7. Team members were highly motivated to perform well in the exercise 8. Overall, I am satisfied with my team experience 9. I would be willing to work with my team on an actual project in the future

Improvement in knowledge and ability to apply that knowledge.McCreery(2003)

foundthatstudentsstartedtheexercisewithself‐perceptionsofknowledgeandability

thatwereneitherextremelyhighnorextremelylow.Mostvalueswerebetween3and5

onthe7‐pointscaleindicatingmoderateknowledgeandabilitywithroomfor

improvement.Hesubtractedpostsimulationmeansforeachitemfrompresimulation

meansandfoundthepostsimulationmeansincreasedby1.0unitormoreforallitems.

Theseincreasesweredeterminedtobestatisticallysignificantbyperformingt‐testsfor

pairedsamplesoneachitem.McCreeryconcludedhypothesesH1(a)andH1(b)were

supportedandstudentssignificantlyimprovedtheirlevelofprojectmanagement

knowledgeandtheirabilitytoapplythisknowledgeasaresultofparticipatinginthe


simulationgame.McCreery(2003)affirmedthisconclusionusingpostexercise

discussionswithmultiplestudents:

Thereisgeneralconsensusamongparticipantsthattheexerciseprovidesamuch

neededcontextwithinwhichtoimproveprojectmanagementskills.Bymaking

numerousdecisionsinasimulatedprojectenvironment,participantsareableto

“maketheconnection”betweentheoryandpractice,therebygainingadeeper

understandingofhowtomanageanactualproject.

Effects of project work experience.McCreery(2003)constructedknowledgeand

abilityvariableswerefromthegrandmeansofthe16knowledgeitemsandnineteam

experienceitems,respectively.Studentswererankedaccordingtotheiryearsofproject

managementworkexperienceandgroupedintoexperiencequartiles.Thelowranking

quartilehad14of15studentswithnoprojectmanagementworkexperienceand

remainingstudenthad0.2yearsofexperience;thehighrankingquartileaveraged10.7

yearswitharangefrom6to26.5years.Atabularpresentationofthesegrandmeansof

knowledgeandabilityshowedthemostexperiencedquartilehadhigherpresimulation

knowledgeandabilityscoresthantheleastexperiencedquartile(4.13and4.24versus

3.42and3.64,respectively).

Agraphicaldisplayofthedataforeachconstructitemshowedthat“invirtually

allinstances,moreexperiencedparticipantsenterintotheexercisewithhigherlevelsof

knowledgeandabilitytoapplythatknowledgethanlessexperiencedparticipants”

(McCreery,2003).ANOVAtestsoneachofthe16presimulationitemswithap‐valueof

0.1asacutofffoundsignificantdifferencesbetween9of16knowledgeitemsand4of16

abilityitems.Noresultsofstatisticallycomparingthegrandmeansoftheknowledge

andabilityconstructswerereported.TheconclusionwashypothesisH2(a)wasstrongly

supportedandhypothesisH3(a)wasmoderatelysupported:“Highlyexperienced

participantsappeartocomeintotheexercisewithhigherperceivedlevelsofproject

managementknowledgethandothelesserexperiencedparticipantsand,toalesser

degree,greaterabilitiestoapplythatknowledge”(McCreery,2003).

Thetabulardisplayofgrandmeanknowledgeandabilityalsoshowedtheleast

experiencedquartileexperiencedagreaterknowledgegainfollowingthesimulation

thandidthemostexperiencedquartile(1.46versus1.01),buttheincreaseinabilityto


applythatknowledgewasroughlythesameforbothofthesequartiles(1.06and1.02).

Thegraphicalfiguresshowedthesedifferencesforeachconstructitem.Supportfor

hypothesisH2(b)wasfoundtobemoderateatbest:ANOVAtests(p<=0.1)showedfive

ofthe16knowledgeitemstohavesignificantdifferences;oneoftheseitemswasinthe

oppositedirectionthanexpected.McCreey(2003)notedtheseresultsmaybeaffected

bythesmallsamplesizeof15studentsineachquartilegroup.Assuggestedbythe

abilityconstructdifference,HypothesisH3(b)wasnotsupported:ANOVAtests(p<=

0.1)foundthreeof16differencestobesignificantandtwoofthemintheopposite

directionthanexpected.McCreery(2003)concluded:

Lessexperiencedparticipantswillmakegreatergainsinlevelsofknowledge...

[and]tendtoclosetheprojectmanagementknowledgegapbytakingpartinthe

exercise....[all]participantsmadeequivalent,substantialgainsintheabilityto

applytheirprojectmanagementknowledge.”

Effects of team performance and team process.Tostudytheeffectsofteam

performance,McCreery(2003)calculatedacompositeteamperformancescoreusingthe

simulatedprojectcostandscheduleperformanceresultstorankthe16teamsandused

ANOVAtocomparetheimprovementsmadebythe4teamsinthelowestquartileagainst

theimprovementsmadebythe4teamsinthehighestscoringquartile.Hypothesis

H4(a)wasnotsupported;onlyoneofthe16knowledgeitemsandnoneoftheability

itemsshowedastatisticallysignificantimprovement.McCreery(2003)concluded“there

isnosubstantialevidencetosupporttheviewthatobjectiveteamperformancehasan

impactontheeducationalvalueoftheexercise.”

Tostudytheeffectsofteamprocess,McCreery(2003)calculatedateamprocess

scoreusingthenineteamexperienceassessmentitemslistedinTable2andrankedthe

teamsfromlowesttohighest.Heobservedthatteammembers’assessmentsofthe

qualityoftheirteamprocessvariedwidely,evenwithinteams.AgainusingANOVAtests

withthehighestandlowestscoringquartiles,hefoundnocorrelationbetweenteam

processscoresandimprovementsinknowledgeorintheabilitytoapplythat

knowledge.WhilethisindicatednosupportforhypothesisH4(b),henotedthatmost

studentsexperiencedaquitepositiveteamexperiencewithcompositeteamprocess

scoresover6.00andthatthelowestteamquartilehadameanscoreof5.01.“Itleavesas


anopenquestionwhetheranextremelynegativeteamprocesswouldaffectthe

educationalvalueoftheexercise”(McCreery,2003).McCreeryconcluded,“giventhe

dataavailableinthisstudy,theeducationalvalueofthesimulationisnotsignificantly

affectedbyeitherateam’sobjectiveperformanceorbythequalityoftheteamprocess.”

Summary.McCreery’s(2003)researchdemonstratedtheeducationalvalueof

usingaprojectmanagementsimulationasanELAwithamixofstudentsfrom

managementandengineeringgraduateprogramswithvaryingdegreesofpriorwork

experience.Theuseofp<=0.1forsignificancetestingsuggeststheresultswerenotas

robustashopedforwhenconductingresearch;however,thefindingthatlearningoccurs

regardlessofhowwelltheteamperformedwhileexecutingthesimulationisvaluable.

Additionaldiscussionoftheinternalthreatstovalidityincludingtheself‐assessmentof

knowledgeandabilitywouldbeuseful.Opportunitiesforfutureresearchinclude

demonstratingtheimprovementsfoundusingthismethodologyarenotahaloeffectand

theyresultintransferableimprovementsintheabilityofstudentstoperformasproject

managersintherealworld.

Pfahl, Laitenberger, Ruhe, Dorsch and Krivobokova (2004).Pfahletal.present“the

resultsofatwicereplicatedexperimentthatevaluatesthelearningeffectivenessofusing

aprocess[SystemsDynamics]simulationmodelforeducatingcomputerscience

studentsinprojectmanagement.”TheauthorsareaffiliatedwiththeUniversityof

KaiserslauterninGermany(wheretheexperimentwasfirstperformed),theUniversity

ofOulu(Finland),andtheUniversityofCalgary(Canada)andconsiderthisresearchto

beexploratoryinnature.

Thirtyreferencesarelisted(citationswerefoundfor26),14relatedtosoftware

engineeringandSystemsDynamicsmodeling,ninerelatedtoresearchmethods,three

relatedtomanagementeducationwithsimulatorsandcasestudies,threerelatedarticles

listingPfahlastheprimaryauthor,andoneondesigningandevaluatingacomputer‐

basedlearningenvironment.

Theresearchapproachwasapretest–posttestcontrolgroupexperiment.

Learningeffectivenesswasevaluatedbycomparingwithin‐subjectposttesttopretest

scoresandbycomparingscoresbetweentheexperimentalandcontrolgroups.The

experimentalgrouptrainingmoduleincludedtheuseofaSystemsDynamics(SD)model

simulatingthetypicalbehaviorofasoftwaredevelopmentproject;thecontrolgroup

trainingincludedtheuseofthepopularCOCOMOsoftwarecostestimationmodel.The


trainingmodulewascomposedofprojectplanningandcontrolcoursematerial.(Pfahl,

2004)

Pfahletal.(2004)usedfourconstructsderivedfromthepretest/posttest

questionnaire,eachrepresentedbyadependentvariable(Y.1–Y.4),tomeasure

performanceandtwoadditionalconstructsfromaseparatesessionevaluation

questionnaire(Z.1andZ.2)tomeasuresubjectiveperceptions:

Y.1 Interestinsoftwareprojectmanagementissues(‘Interest’)

Y.2 Knowledgeabouttypicalbehaviourpatternsofsoftwaredevelopment

projects(‘Knowledge’)

Y.3 Understandingof‘simple’projectdynamics(‘Understandsimple’)

Y.4 Understandingof‘complex’projectdynamics(‘Understandcomplex’)

Z.1 Availabletimebudgetversustimeneed[tocompletethetreatment](‘Time

Pressure’)

Z.2 Sessionevaluation

Thenatureofthequestionscomprisingtheconstructsisdiscussedinthearticleand

examplequestionsareprovidedintheappendixforallconstructs.Thereaderisreferred

toPfahl’sPh.D.thesistoviewthecompletesetofquestions.

Theexperimentalhypotheseswereintroducedas:

1. Thereisapositivelearningeffectinbothgroups(A:experimentalgroup,

B:controlgroup),i.e.post‐testscoresaresignificantlyhigherthanpre‐

testscoresforeachdependentvariable.

2. ThelearningeffectingroupAishigherthaningroupB,eitherwith

regardtotheperformanceimprovementbetweenpre‐testandpost‐test

(relativelearningeffect),orwithregardtopost‐testperformance

(absolutelearningeffect).(Pfahl,2004)

Thesearerestatedasthefollowingnullhypotheses:

H0,1:Thereisnodifferencebetweenpre‐testscoresandpost‐testscores

withinexperimentalgroupAandcontrolgroupB.


H0,2a:Thereisnodifferenceinrelativelearningeffectivenessbetween

experimentalgroupAandcontrolgroupB

H0,2b:Thereisnodifferenceinabsolutelearningeffectivenessbetween

experimentalgroupAandcontrolgroupB.(Pfahl,2004)

Studentswererandomlyassignedtothetwogroupsandparticipatedas

individuals.Thestudentsdidnotknowthepretestandposttestquestionswereidentical

andtheywerenotprovidedwiththecorrectanswersuntilaftertheconclusionofthe

experiment.Theinitialexperimentwasconductedontwodayswithinaweekofeach

otherandstartedwith12graduatecomputersciencestudentsattheUniversityof

Kaiserslauternandfinishedwithninestudents.Thesecondinstancewaswith12

graduateandpostgraduatestudentsattheUniversityofOulu,Finland,andthethird

instancewaswith13seniorundergraduatestudentsattheUniversityofOulu,Canada.

Thesecondandthirdinstanceswereconductedonasingleday.Studentsparticipating

inthesecondandthirdrunningoftheexperimentwereenrolledinavarietyoftechnical

andmathematicaldegreeprograms.ThefirstrunoftheexperimentattheUniversityof

Kaiserslauternlasted130minuteswith45minutesallocatedtothetreatment.

Treatmenttimewasincreasedto80minutesfortheothertworunsattheUniversityof

OuluandattheUniversityofOuluafterstudentsattheUniversityofKaiserslautern

indicatedtheydidn’thaveenoughtime.(Pfahl,2004)

Atreatmentstartswithbothgroupsreceivinga3‐5minuteintroductiontothe

maintasksofasoftwareprojectmanager,typicalplanningandcontrolproblems

experienced,andtheneedtomaketrade‐offswhenmanagingthetripleconstraintofthe

project.FortheexperimentalgroupusingtheSD‐modelprojectsimulator,this

introductionwasfollowedbya15‐30minuteintroductiontotheprinciplesdominating

softwareprojectperformanceandarole‐playinteractionwiththeSDmodel.Duringthe

roleplay,thestudentactsastheprojectmanagerandispresentedwithaprojectplan

thatgreatlyexceedsthedeadline.Theprojectmanagermustdecidewhichactions

(basedonthepreviouslyintroducedprinciples)totakeinordertocompletetheproject

ontime.Tobesuccessful,theprojectmanagermustinvokeactionsinvolvingmorethan

oneoftheprinciples.Therole‐playisfollowedbyashortdiscussionofthedifferent

possiblesolutionswithanexplanationoftheadvantages/disadvantagesofeach.The

experimentalgroupthenreceivesa15‐30minuteintroductiontoprojectmanagement


planningmodels,includingdetailedexplanationsoftheSDmodel,followedby12‐15

minutesofapplicationexampleswithshortexercisesusingthemodel.Thecontrol

groupreceivesa30‐60minuteintroductiontoprojectmanagementplanningmodelsand

detailedexplanationsofusingCOCOMOintheintermediatemodefollowedby12‐15

minutesofapplicationtimewithshortexercisesusingthemodel.(Pfahl,2004)

Forhypothesistesting,theresultsofone‐waypairedt‐testforH0,1andone‐sided

t‐testforindependentsamplesforH0,2aandH0,2barereported.Pfajhletal.(2004)note

thatatestfordatanormalityrevealedthatanormaldistributioncouldnotbeassumed

andadditionalnon‐parametrictestswereperformed(Wilcoxonmatchedpairtestfor

H0,1andMann‐WhitneyUtestforH0,2aandH0,2b).Resultsofthesetestsdidnotrevealany

differencesandonlytheparametrictestresultsarereportedinthearticle.Further

discussedistheneedtosetα=0.1forsignificancetestingduetothesmallsamplesize

andthemeta‐analysistechniquesusedtointegratethefindingsfromthethreestudies.

Descriptivestatisticsandresultsofthestatisticalanalysesarepresentedindetail(data

tablesanddiscussion).InsummaryPfahletal.(2004)found:

EvidencefortheassumptionthatthetrainingsessioninvolvingtheSD

modelinsteadofCOCOMOplusperformingarole‐playsignificantly

increasesinterestinthetopicofprojectmanagement....

TraininginvolvingSDmodelandrole‐playyieldssignificantlybetter

scoresforvariableY.2(knowledgeaboutempiricalpatternsinsoftware

projects)thanusingCOCOMOwithoutrole‐play....

Noconsistentsignificantdifferencebetweenexperimentalandcontrol

groupscouldbeobservedregardingvariableY.1,Y.3,andY.4(interestin

thetopicofprojectmanagement,understandingofsimpleproject

dynamics,understandingofcomplexprojectdynamics)....

Qualitativedataindicatestheinclusionofrole‐playswithSDmodelsin

projectmanagementeducationisperceivedasahighlyusefulexercise....

withregardstousefulness,entertainment,difficulty,andclarity.

Summary.Thiswasthemostcompletereportofaresearchstudyfoundamong

thearticlesrelatedtousingprojectmanagementsimulationasanELA.Itdemonstrated

theuseofapretest‐posttestcontrolledexperimentmethodologyforevaluatinglearning.


Itsdesignusingalessrobustbutstillinterestingandgenerallyrecognizedcost

estimatingtoolasthereplacementfortheSDsimulatorinthecontrolgroupmayhave

confoundedtheresults;however,evidencesupportingtheuseofaSystemsDynamics

simulatorforteachingsoftwareprojectmanagementwasfound.

Davidovitch, Parush and Shtub (2006).Davidovitchetal.researchthe

performanceofparticipantsusingdifferenthistory‐keepingmodesofaproject

managementsimulatordevelopedbyDr.Shtubandfind“thesimulatoras[an]

educationalinnovationimprovesstudentlearningandperformance.”

Atthetimeofwritingthisarticle,Mr.DavidovitchwasaPh.D.candidateinthe

IndustrialEngineeringandManagementfacultyattheTechnionIsraelInstituteof

TechnologyunderthesupervisionofDr.Parush,associateprofessorofpsychologyat

CarletonUniversityinOttawa,CanadaandDr.ShtubatTechnion.Dr.Davidovitchis

currentlyemployedasaprogrammanagerattheIsraelMinistryofDefense.Dr.Shtubis

theprincipaldeveloperofProjectManagementTrainer(PMT)anditssuccessor,the

ProjectTeamBuilder(PTB)simulator,andhaswrittenbooksonitsuse.Thestated

purposeofthesesimulatorsis“toprovideadynamic,stochastic,simulatedenvironment

forpracticingthetoolsandtechniquesofprojectmanagementandtosupportthe

creationofsharedunderstandingamongprojectteams”(Shtub,2012).

Davidovitchetal.(2006)usedKolb’sExperientialLearningTheory(Kolb,1984)

andaconstructivistapproachastheoreticalframeworkstoexaminetheeffectofthe

PTMSimulatorasanexperientiallearningtoolonadaptivetransferofknowledgeby

“teachingandtrainingonetaskandthenexaminingtheperformanceofadifferenttask.”

Thirty‐sevenreferencesrelatedtoprojectmanagement,learningtheory,and

simulationarecitedintheintroductionsectionofthearticlesupportingdiscussionson

teachingandlearningprojectmanagement,simulation‐basedlearning,theProject

ManagerTrainersimulator,andtheimpactoflearninghistoryonlearningwith

simulations.Fiveofthesereferencessupportthenotionofallowingthelearnerssome

controloverthelearningenvironmenttogivethemamoreactiveroleinconstructing

theacquiredknowledgeandtherebyenhancinglearning.Anadditionalfivearticleson

curriculumdevelopmentandlearningtheoryarecitedinthemethodologyand

discussionsections.(Davidovitchetal.,2006)

Theresearchwasdesignedtotestthehypothesis“thatauser‐controlledor

manuallearninghistorywillhaveagreaterpositiveimpactonsimulator‐basedlearning


ascomparedwiththeautomatichistoryrecording”(Davidovitchetal.,2006).ThePMT

simulatorallowsuserstoreviewpriordecisionsandtore‐runthesimulationfroma

priorhistoryrestorepoint.Inadditiontocomparinglearningbetweenthemanualand

automatichistoryrecordingmodes,thisresearchalsotestedthehypothesis“thatif

historykeepingandreviewinghasanimpactonlearning,thenparticipantswholearned

withsuchamechanismshouldstillperformbetterwhenthismechanismisremoved”

(Davidovitchetal.,2006).

Thiswasaccomplishedbyconductingtheexperimentintwophases.InPhaseI:

BasicLearning,studentswereassignedtooneoffivegroups:automatichistorywith

undo,automatichistorywithoutundo,manualhistorywithundo,manualhistory

withoutundo,andno‐history[controlgroup].Allweregiventhesame16activity

projectscenariotorunthreetimes.InPhaseII:Transfer,studentsinallfivegroupswere

givena24activitymultipleprojectscenariowithnohistorykeepingandnoundo.The

projectsscenariosarepredefinedbythesimulator.Studentsaregiventaskinformation

(durationdistributionandpredecessors),resourceinformation(type,availability,cost

workingandcostidle)andtargetprojectcost.Performancegoalsaretofinishontime

andmaximizeprofit.Abonusisaddedtoprofitforearlyfinishesandapenaltyis

deductedfromprofitforlatefinishes.Thedependentvariablesusedintheresearch

wereprofitattheendofthesimulationrunandelapsedtimetocompletethesimulation

run.Therandomnumbergeneratorsinthesimulationengineweresettogeneratethe

samenumbersforallthestudentstohelpassuredifferencesinresultswerebasedsolely

onstudentdecisions.(Davidovitchetal.,2006)

Thefiveexperimentalgroupswererandomlypopulatedwithstudentsfromthe

poolof98fourthyearengineeringstudentswithnopriorprojectmanagement

experience.Agesrangedfrom18to35.Groupsizesvariedfrom17to21students;the

authorsdidnotdiscusswhythegroupsizesweren’t19and20.Studentmotivationwas

stimulatedbymakingthecoursegradepartiallydependentonthesimulatedprofit

achieved.(Davidovitchetal.,2006)

Dataanalysisconsistedofcomparingthemeansoftheprofitandsimulationrun

timeresultsgraphicallyandbyperformingt‐testsandAnalysisofVariance(ANOVA)on

them.AgraphofprofitresultsforthethreesequentialPhaseIsimulationrunsshoweda

clearandconsistentincrease;thecontrolgroupdidnot.Nosignificantdifferencewas


foundamongthefivemeanprofitsforthefirstPhaseIrunindicatingtheexperimental

andcontrolgroupsstartedwithsimilarknowledge.Insummary,forsimulatedprofit:

Havingahistorykeepingmechanismhadasignificantimpactonperformance

comparedwithnohistorykeepingmechanism.Inallconditionswheretherewas

ahistorymechanism,meanprofitsincreasedsignificantlybetweensimulation

runs,whereastherewasnoimprovementwithouthistory.Inaddition,the

highestmeanprofitattheendofthethirdsimulationwasfoundforthemanual

historykeepingascomparedwiththeautomatichistorykeeping(bothwithundo

capability....Havingtheabilitytoundo(restartthesimulationfromanypointin

apreviousrun)hadasignificantimpactonperformance.Forbothhistory‐

keepingmodes,manualandautomatic,themeanprofitsweresignificantlyhigher

thantherespectivemanualandautomaticmodeswithoutundo.(Davidovitchet

al.,2006)

Forsimulationrunduration,Davidovitchetal.(2006)foundmeanrundurations

werelongerinthefirstrunandlongestforthegroupswithmanualhistorykeeping.The

abilitytoundodidnotaffectrundurationduringthefirstrun.Secondandthirdmean

simulationrundurationsdecreasedsignificantlyforeachofthefivegroups.Mean

simulationrundurationsweresignificantlylongerforthemanualhistorywithundo

groupduringthesecondandthirdruns.

DuringPhaseIIallgroupscompletedthefourthsimulationrunusingthesame

multipleprojectscenariowithouthistorykeepingandwithouttheabilitytoundo.The

meanprofitoftheexperimentalgroupwasfoundtobesignificantlyhigherthanthe

controlgroup.ThegroupwiththehighestPhaseIIsimulationrunprofitwasthegroup

thatoperatedinPhaseIwithmanualhistorykeepingandtheabilitytoundo.From

highesttolowestprofit,thePhaseIIgrouprankingwas(1)manualhistorywithundo,

(2)automatichistorywithundo,(3)manualhistorywithoutundo,(4)automatichistory

withoutundo,and(5)nohistory[control].Thedifferencesbetweentheautomaticand

manualhistorygroupsweresignificantinbothcasesofabilitytoundoornotundo.No

significantdifferenceswerefoundforthemeansimulationrundurationamongthe

groups.(Davidovitchetal.,2006)


Theexperimentalgroupsachievedsignificantlyhigherprofitandshoweda

betterlearningprocess...whilethecontrolgrouphadverylittlelearning....The

findingsshowabetterlearningprocessfortheparticipantswithmanualhistory

mechanismcomparedtotheparticipantswithautomatichistorymechanism,and

withoutanysignificantdifferencebetweensimulation‐rundurations.Moreover,

whileobservingtheMP[PhaseIImultipleproject]scenarioperformancewhich

reflectedtransferabilities,itwasfoundthetrendpersistedandperformancewas

betterforthegroupthatpreviouslyhadthemanualhistorykeepingmechanism.

Thesefindingssupportthehypothesis...thatmanuallearninghistoryrecording

willhaveagreaterpositiveimpactonsimulator‐basedlearningascompared

withautomatichistoryrecording.Itcanbeassumedthathavingthemanual

historykeepingmechanismforcesthelearnertobettermonitortheirown

progress...[and]mayhavecontributedtobetterunderstandingoftheprinciples

involvedwithprojectmanagementandconsequentlyprovidedthelearnerwitha

bettercapabilitytotransferandhandlemorecomplexscenarios....[Further,]

havingtheabilitytoundoinsimulator‐basedlearningcannotonlyimprove

performance,butactuallysupportandimprovelearning.(Davidovitchetal.,

2006)

Davidovitchetal.(2006)discussedtheseresultsinthecontextofseveral

learningtheoriesandconcluded“thehistorymechanismenabledthebuildingofstorage

strengthbyprovidingthelearnerswithcontinuousaccesstothelearnedmaterial....

[and]probablyprovidedmoreexposuretopossiblesubjectmatterelements.Thisin

turnnotonlystrengthenedthestorage,butmadeitpossibletobetterunderstandthe

processunderlyingthelearntmaterial.”

Unlikethepreviousarticlesreviewedinthisstudy,thisarticlebyDavidovitchet

al.(2006)wentbeyonddescribingasimulationgameanddocumentingthatthe

studentslikedit.Followinggenerallyacceptedacademicresearchprocesses,

Davidovitchetal.demonstratedthevalueofprovidingstudentswithahistory‐keeping

mechanismfortheirusewhensimulatingaprojectanddemonstratedthattransferable

learning(formaximizingsimulatedprofit)isoptimalwhenthehistory‐keepingis

implementedmanuallyandstudentsareallowedtoundoandchangedecisionsmade

whilerunningasimulation.Unanswerediswhetherornotthestudentsarelearning


applicableprojectmanagementskillsthattransfertotherealworldormerelylearning

howtooptimizethesimulation.

Davidovitch, Parush and Shtub (2008).Buildingontheresearchdocumentedin

their2006article,Davidovitch,ParushandShtub(2008)discussthelearning‐forgetting‐

relearningprocessanddescribetheirexperimentalresearchevaluatingtheeffectiveness

andefficiencyofthisprocessinaprojectmanagementsimulationenvironment.In

additiontotheirprior2006article,anarticlebyDr.ShtubdescribingthePMTsimulator,

andthePMBOK®Guide,twenty‐ninereferencesarecitedrelatedtolearningtheories.

Themainresearchquestion“washowhistoryrecordinginsimulator‐basedlearning

impactsthelearning‐forgetting‐relearningprocess.”Quantitativemethodssimilarto

thoseusedinthe2006studywereusedtotesthypothesesrelatedtotheforgetting

phenomenon,thelengthofthebreakperiodandhistorymode(automaticormanual),

andbasiclearning.

Theexperimentaldesignusedthreeindependentvariables:(1)historyrecording

mode(automatic,manual,none[control),(2)breakperiod(twoweeksorfourweeks),

and(3)simulationrun(describedbelow).Thereweresixexperimentalgroupstofully

crossthethreehistoryconditionswiththetwobreakperiodconditions.(Davidovitchet

al.,2008)

Thetwo‐meetingexperimentwasconductedwith66secondyearMasterof

Engineeringstudentsranginginagefrom25to50.Participationwasvoluntaryand

participantsreceivedagradebonusbasedontheresultsoftheirsimulationruns(with

theapprovaloftheIsraelInstituteofTechnologyResearchEthicsBoard).Studentsran

fouridenticalsimulation‐runsofasingleprojectscenarioduringthefirstmeetingand,

followingabreakofeithertwoorfourweeks,reranthesamesingleprojectscenario

followedbytwicerunningamorecomplicatedmultipleprojectscenario.Thesimulator

manualhistorywithundofeaturewasavailableforthestudentsuseduringthefirst

meeting(PhaseI:Basiclearning[simulationrunsonethroughfour])butnotduringthe

secondmeetingafterthebreak(PhaseII:Forgettingandrelearning[simulationrunfive]

andPhaseIII:Transfer[simulationrunssixandseven]).Asinthe2006study,the

sequenceofrandomnumbersgeneratedforthestochasticmodelinginthesimulatorwas

thesameforallstudents.Dependentvariableswereagainprofitandscenariorun

durationandt‐testsandANOVAwereusedtotestforsignificantdifferencesbetweenthe

datameans.(Davidovitchetal.,2008)


Asbefore,inPhaseI(basiclearning),nosignificantdifferencewasfoundamong

thegroupsafterthefirstsimulationrunindicatingnoinitialimpactfromthe

experimentalmanipulationsandthatallstudentsstartedwiththesameknowledgeand

experience.Onceagain,thegroupswithhistoryshowedconsistentsignificant

improvementsinprofitoverthenextthreesimulationrunsduringthefirstmeetingwith

themanualhistorygroupscoringsignificantlybetter.Thecontrolgroupshowedno

patternofconsistentgrowth.Meansimulationtimedecreasedsignificantlyforeachof

thegroupsoverthefourrunsandnosignificantdifferencewasfoundbetweenthe

automaticandmanualhistorygroupsandbetweenthesegroupsandthecontrolgroup.

(Davidovitchetal.,2008)

AfterthePhaseII(forgettingandrelearning)fifthsimulationrun,themeanprofit

ofthecontrolgroupremainedsignificantlylowerthanthehistorygroups(roughlyhalf)

andthemeanprofitwassignificantlylessforthegroupswiththefourweekbreakthan

forthosewiththetwoweekbreak.Forthetwo‐weekbreakgroups,therewasa

significantdifferencebetweentheautomaticandmanualhistorygroups;forthefour‐

weekbreakgroups,therewasnosignificantdifferencebetweenthem.Therewere

significantdifferencesbetweenthemeansimulationrundurationforthetwo‐week

breakgroupswiththemeantimeofthemanualhistorygroupbeinglowest,the

automatichistorygroupthesecondlowest,andthecontrolgroupthemost.The

simulationruntimesforthefourweekbreakgroupswereallsignificantlylongerthan

thetwoweekbreakgroups;however,inthiscasetherewasnosignificantdifference

betweenthemanualandautomatichistorymodegroups.Bothfourweekbreakhistory

groupsweresignificantlyshorterthanthecontrolgroup.(Davidovitchetal.,2008)

AfterPhaseIII(transfer)andthefinaltwosimulationruns(simulationrunssix

andseven)withthemultipleprojectscenario,asubstantialandsignificantincreasewas

observedforthegroupswithhistoryfromrunsixtorunseven;nosignificantincrease

wasobservedforthecontrolgroups‐controlgroupprofitdecreasedslightlyfromrun

sixtorunseven.Manualhistorygroupprofitwassignificantlyhigherthantheautomatic

historygroupforthetwo‐weekbreakgroupsbutnotforthefour‐weekbreakgroups.

Meansimulationruntimesdecreaseddramaticallyforallgroupsfromrunsixtorun

seven.ComparingthefifthrunofPhaseIIwiththesixthruninPhaseIIIyieldedno

significanteffectsorinteractions.(Davidovitchetal.,2008)


Insummary,thestudentsingroupswiththesimulationhistorykeepingfeature

performedsignificantlybetterthanthecontrolgroupsandthemanualhistorygroups

performedsignificantlybetterthantheautomatichistorygroups.Withtheexceptionof

simulationrunsix,thecontrolgroupsimulationrundurationswereallgreaterthanthe

rundurationsoftheexperimentalgroups.Significantprofitdifferenceswereobserved

betweenthetwo‐weekbreakmanualandautomatichistorygroupsbutnotthefour‐

weekbreakgroups.Davidovitchetal.(2008)foundtheseresultsconsistentwith

literaturedescribingtheforgettingphenomenonandstressedtheimportanceof

designingsimulationsthatprovidestudentswiththeabilitytoreviewhistoryandideally

tocontrolwhenhistoryiscaptured.

Inconcludingthisarticle,Davidovitchetal.(2008)didacknowledgethe

shortcomingsofmeasuringtransferoflearningwithinthesimulatorratherthantothe

“realworld”andsuggestfurtherresearchinto“howwellthePMTsimulationfacilitates

‘true’projectmanagement”andhowwellit“promote[s]expertiseinreal‐lifesituations.”

Thisarticlewasanothergoodexampleofprojectmanagementsimulationgaming

researchthatstartedwitharesearchquestionandhypothesesandusedgenerally

acceptedacademicresearchprocessestoanswerthequestion.Ifskillslearnedplayinga

simulationgamearetransferabletotherealworldandimprovementinplayingthegame

bybeingabletocapturehistoryandundopriordecisionstotakeadifferenceapproach

therebyimprovesperformanceplayingthegame,thenskilltransfertotherealworld

maybeimproved;however,moreresearchisneededtosupportthishypothesis.

Davidovitch, Parush and Shtub (2007).Althoughpublishedearlierthanthe2008

articledescribedabove,thisarticleactuallysucceededitinsubmissionandbuildsonit

byreportingontheresultsofanadditionalexperimentinvolvingtheperformanceof

studentteamsformedfromthesamegroupofstudentsthatparticipatedinthe

individualstudentsimulationexperiment.

Thearticleaddsbriefdiscussionsoftheforgettingcomponentofthelearning

process,team‐basedlearning,problem‐basedlearning,andtheimportanceofdebriefing.

Thesediscussionsaresupportedbycitationsofeighteennewreferencesandthree

referencesrepeatedfromthe2008article.(Davidovitchetal.,2007)

Theresearchquestionforthemultiuserexperimentaddressedtheimpactofthe

historyrecordingmodeanddebriefingonteamlearning.Twohypothesesweretested:

(1)meanprofitwillbesignificantlyhigherforteamsusingthehistorymode;(2)mean


profitwillbehigherfortheteamsparticipatingindebriefingthanforteamsthatdon’t

debrief.Theprocedureforselectingtheteams,thenumberofstudentsperteam,andthe

natureandlengthofthedebriefsessionswerenotdescribed.(Davidovitchetal.,2007)

Meanprofitisagainusedasameasureoflearningandisthedependentvariable.

Independentvariablesarehistoryrecordingmode(withhistory,withouthistory),

debriefing(with,without),andsimulationrun(therewerethreesimulationruns).The

twohistoryrecordingconditionswerefullycrossedwiththetwodebriefingconditions

tocreatefourexperimentalgroups.Allfourgroupsweregiventhesamesingle‐project

managementscenarioandthesamescenariowasrunthreetimes.(Davidovitchetal.,

2007)

Atableofresultsshowedthesameperformancerankingaftereachsimulation

run.Onaverage,thegroupofteamswithbothhistory‐keepinganddebriefsscoredthe

highest.Secondhighestwasthegroupwithdebrief/nohistoryfollowedbythegroup

withhistory/nodebrief.Thecontrolgroupwithoutbothscoredthelowest.Although

therankingwasconsistentthroughout,nosignificantdifferenceswerefoundforeither

historykeepingordebriefingafterthefirstrun.Thisindicatedthatallteamsstarted

withsimilarknowledgeandexperience(thetestusedtoconfirmthiswasnotdescribed).

UsingANOVAwithrepeatedmeasures,significantdifferenceswerereportedbetween

thefourgroupsforboththedebriefingfactorandforthehistoryfactor.(Davidovitchet

al.,2007)

Theperformanceofparticipantsthatusedthehistorykeepingmechanismswas

significantlybetter.

Theperformanceofthesystemsengineeringstudentsthatusedthe

debriefingprocesswassignificantlybetter.Theinteractionbetweenthehistory

factorandthedebriefingprocessfactorwasnotfoundassignificant.

...Theuseofthedebriefingprocessforteamlearningisfoundalso

powerfulinthelearningprocess.(Davidovitchetal.,2007)

Theprecedingquotationcontainstheonlycommentsinthediscussionsection

abouttheteam‐basedexperiment.Thediscussionsectionprimarilysummarizedthe

resultsobtainedfromtheindividuallearnerexperimentpreviouslydescribedin

Davidovitchetal.(2008)emphasizingthepowerofthesimulatorhistorymechanism.


Theadditionofresearchintotheuseofsimulationbyteamsisimportantbecausethisis

atypicalmodeofELAwithsimulationgamesingraduatebusinessmanagement

programs.Moredisclosureofthemethodologyusedfortheteam‐basedexperimentin

thisstudywouldbeuseful.Additionalresearchtodemonstratethatskillslearned

playingthissimulationgamearetransferabletotherealworldwouldalsobeusefulas

wouldresearchshowingthosewhoimprovemorewhileplayingthisgamearemore

effectiveprojectmanagersintherealworld.

Davidovitch, Parush and Shtub (2009).Buildingontheirpriorresearchonthe

effectivenessofusingsimulatorsinprojectmanagementacademiccoursesandthe

impactofsimulatorhistoryrecordingfeaturesonlearningandforgetting(Davidovitch

etal.,2006;Davidovitchetal.,2008),Davidovitch,ParushandShtub(2009)examinethe

impactofprojectmanagementsimulatorfunctionalfidelityonstudentperformance.

Twenty‐sevenreferencesarecitedintheintroductionsectionofthisarticle:

threeareusedtodefineprojectmanagementandtocreateananalogybetweenlearning

projectmanagementandlearningtoflyanairplane,threetocitepriorarticlesontheuse

ofthePMT(ProjectManagementTrainer)simulator,onependingarticleonthenext

generationPTB(ProjectTeamBuilder)simulator,2onjournal‐keepingenhances

reflectivelearning,fiveonthevalueofgivinglearnerssomecontroloverthelearning

environment,and2onsimulatorfidelityanditsinfluenceonlearningtransfer.No

citationwasfoundforthetwenty‐eighthandfinalreferencelisted,thePMBOK®Guide.

TwoadvancedfeaturesofthepreviouslydescribedPMTsimulatorwere

employedinthisstudy:(1)theabilitytosplitactivityexecution,i.e.theabilitytostartan

activity,stopworkingonitforaperiodoftime,andresumeworkonitlater;and(2)the

abilitytohireanddismissresources.Studentswithaccesstotheadditionalfeatures

receivedinformationontheminimum/maximumnumberofresourcesallowedandthe

coststohireanddismiss.(Davidovitchetal.,2009)

Theexperimentaldesignforthisstudyusedcumulativeprofitattheendofthe

simulationrunasthedependentvariableandthreeindependentvariables:(1)

functionalfidelity(abilitytousetheadvancedactivitysplitting/resourcemanagement

featuresornot)(2)historyrecordingmode(automatic,manual,none),and(3)undo

(abilitytoundoornot).Thestudyusedtengroups,fullycrossingallexperimental

conditions.(Davidovitchetal.,2009)


Thepriortwo‐phaseresearchapproach(Davidovitchetal.,2006;Davidovitchet

al.,2008)wasused.InPhaseI:BasicLearning,alltengroupsreceivedthesamesingle‐

projectscenariotorunthreetimesand.ThiswasfollowedbyPhaseII:Transfer,where

alltengroupsweregivenanewmultiple‐projectscenarioandnogroupwasabletokeep

historyorundo.Learnerconditionswereasbefore:thesimulationisstochasticin

executionwithregardtoactivitydurationsandcosts(allstudentsreceivethesame

sequenceofrandomnumbers);studentswererequiredtocompletethesimulated

projectbyitsduedate(withpenaltiesassessed/bonusespaidforlate/earlycompletion)

andtomaximizeprofit.(Davidovitchetal.,2009)

Noneofthe199participatingfourthyearengineeringstudentswithagesranging

from18to35hadpriorpracticalprojectmanagementexperience.Studentswere

randomlydividedintothetenexperimentalandcontrolgroups.Studentsreceived

verbalandwritteninformationontheuseofthesimulator,thescenarios,andtheprofit

performancemeasure(which,inpart,affectedtheircoursegrade).(Davidovitchetal.,

2009)

Aswasreportedinthepriortwostudies(Davidovitchetal.,2006;Davidovitchet

al.,2008),Davidovitchetal.(2009)found:nosignificantprofitdifferencesafterthefirst

runverifyingtheassumptionthatallstudentsstartedwithsimilarknowledge;all

experimentalgroups(withhistory)achievedasignificantincreaseinprofitfor

subsequentPhaseI:BasicLearningsimulationrunswhilethecontrolgroups(no

history/noundo)didnot;experimentalgroupswithundoperformedsignificantlybetter

thanthosewithout.Theoptiontousetheenhancedfeaturestoimprovetheprofitwas

foundtobe“meaningful”(Davidovitchetal.,2009):

TheeffectoftheEnhancedcapabilitiesfactoronthehistory‐keepingmodedoes

notexist,whiletheperformancesremainbetterforusingthemanualhistory‐

keepingmodeandusingundocapabilities.

ThiscommentandtheassociatedreportingoftheANOVAresultsarenotclear.A

reviewoftheprovidedgraphindicatedthatprofitwashigherwhenusingenhanced

techniquesforthemanualhistory‐keepinggroupsandinconsistentfortheautomatic

history‐keepinggroup(profitimprovedfortheautomaticgroupwithoutundoand

worsenedfortheautomaticgroupthatcouldundo).Therewasnonoticeabledifference


forthecontrolgroupwithnohistory/undoafterthefirstandsecondruns,butthemean

profitwasnoticeablyhigherforthecontrolgroupwithaccesstotheadvancedfeatures

afterthethirdrun.

FollowingPhaseII:Transfer,whereallgroupsaredeniedaccesstohistoryand

cannotundo,studentswhohadaccesstohistoryinPhaseIperformedsignificantly

betterthanthosewhodidn’thaveaccesstohistoryinPhaseI;studentswithprioraccess

tomanualhistory‐keepingperformedbetterthanthosewithpriorautomatichistory‐

keeping;andstudentswithpriorabilitytoundodidsignificantlybetterthantheir

counterpartswithoutthispriorability(Davidovitchetal.,2009).Alltheseresultsare

consistentwiththepriorfindingsofDavidovitchetal.(2006;2008).Whencomparing

theeffectofaddingtheabilitytousetheadvancedfeatures,areviewofthetableof

resultsindicatedprofitwassignificantlyhigherforthegroupswiththeabilitytousethe

enhancedfeaturesiftheydidnothavetheabilitytoundotheirpriorwork;thegroups

withundocapabilityperformedslightlybetterthantheircounterpartsiftheydidnot

haveaccesstotheenhancedfeatures.Davidovitchetal.(2009)describethisasfollows:

Theenhancedprojectmanagementcapabilitiesfactorkeepstheperformances

andtheresultsofthesignificantdifferencesbasedonthehistory‐keepingmode.

Moreover,theimprovedperformancesintheinitiallearningphasearekeptfor

thetransfertoadifferentscenariophasewhenconsideringtheenhanced

capabilitiesfactor....Theresultsindicatethatusingadvancedproject

managementcapabilitiesenhancedlearningforallusergroups....foradvanced

capabilities,thecomparisonbetweenhavinghistorymechanismwithundoand

withoutundo,duringbasiclearningphase,showedbetterperformancesfor

thosehavingundocapability.Thefindingssuggesthavingtheabilitytoundoin

simulator‐basedlearningcanactuallysupportandimprovelearningandnotonly

improveperformance....Theuseofadvancedresourcemanagementcapabilities

enhancesthelearningofprojectmanagement.

Summary.Thisarticleisanotherexampleofprojectmanagementsimulation

gamingresearchthatstartswitharesearchquestionandhypothesesandusesgenerally

acceptedacademicresearchprocessestoanswerthequestion.Thisarticlestrengthened

theconclusionsreachedintheauthors’priorstudiesabouttheeffectsofsimulation


history‐keepingandundoonlearning.Theconclusionsaboutthebenefitofaddingthe

advancedfeaturestothesimulatorappeartobebasedonmixedresults.More

discussionwouldbeusefulofthePhaseIIobservationthatthegroupswithhistoryand

theabilitytousetheseadvancedfeaturesandundounderperformedthecounterpart

groupswithoutaccesstotheadvancedfeatures.

Additionalresearchdemonstratingthatskillslearnedplayingthissimulation

gamearetransferabletotherealworldwouldbeusefulaswellasresearch

demonstratingthatthosewhoimprovemorewhileplayingthisgamearemoreeffective

projectmanagersintherealworld.

Davidovitch, Parush and Shtub (2010).Inthepriorarticles,Davidovichetal.

(2006;2007;2008;2009)discussedresearchonusingthePMTsimulatorasanELAtool

forindividuallearners.Thisarticledescribestheuseofasimilarresearchparadigmwith

amultiuserversionofthesimulatorcalledProjectTeamBuilder(PTB)toexplore

trainingprojectmanagerstocollaborateonresourceusageinasimulatedmatrix

organizationrequiringthesharingofresources.

Fifty‐threereferencesarecitedintheirintroductionanddiscussionofthe

importance,challengesandmechanismsofwhatDavidovitchetal.(2010)term

“collectiveprojectlearning.”Thisdiscussionincludesthebenefitofgroupintegrationof

knowledgewhere“gapsinindividualknowledgecanbefilledinbytheknowledgeof

otherteammembers,”theneedinamatrixorganizationforcollaborationinanoften

competitiveenvironment,benefitsandtypicaldeficienciesofpostprojectreviews,key

successfactorsforandbarriersagainstimprovingprojectlearning,Kolb’s(1984)

experientiallearningmodel(emphasizingthe“criticalroleofreflectiveobservation”)

anditsextensiontotheKolbTeamLearningExperience(asdescribedbyKayes,Kayes,

andKolb(2005a;2005b)).Anadditionalsevenreferencesarecitedinthediscussionof

results.

Theexperimentaldesignforthisstudyagainusedcumulativeprofitattheendof

thesimulationrunasthedependentvariableandmeasureofeffectivenessformanaging

theproject.Independentvariableswere:(1)historyrecording(using,notusing),(2)

teamdebriefingprocedure(structured,informal),and(3)experience(graduatestudent

withprojectmanagementexperience,undergraduatewithoutprojectmanagement

experience).Thestudyusedeightgroupsofteams,fullycrossingallexperimental

conditions.Eachteamwascomposedofthreeteammembers[assumedtoberandomly


selectedbasedontheauthors’pastarticles(Davidovitchetal.,2006;Davidovitchetal.,

2007;Davidovitchetal.,2008;Davidovitchetal.,2009),butnotspecifiedinthisarticle].

InPhaseI:BasicTeamScenario,eachteammembermanagedasingleseven

activityprojectwithresourcesassignedfromaresourcepoolcommontothethreeteam

members(21totalactivities).Eachscenariowasrunfourtimes.InPhaseII:Transfer,a

moredifficultmultipleprojectscenariowaspresentedwithone16activitysub‐project

andtwoeightactivitysub‐projects(32totalactivities).Bothscenarioshadduedates

andbonus/penaltyawardsforearly/latecompletion.Theauthorsdidnotstatewhether

thethreeteammembersagainmanagedonesubprojecteachinPhaseIIorwhetherthey

collectivelymanagedallthreeasasingleprojectmanagerteam.(Davidovitchetal.,

2010)

Atotalof132studentsparticipatedassubjectsinthestudy:60graduate

engineeringstudentsintheexperiencedgroupand72undergraduateengineering

studentsintheinexperiencedgroup.Agesrangedfrom18to45andstudentswereboth

maleandfemale(nofurtherdescriptivestatisticswereprovided).Thisimpliesthere

were20teamsintheexperiencedgroupsand24intheinexperiencedgroups.History

keepingisdescribedinthepriorarticles(Davidovitchetal.,2006;Davidovitchetal.,

2007;Davidovitchetal.,2008;Davidovitchetal.,2009).Thegroupswithdebriefing

wereabletochatviaatoolembeddedinthesimulationwhilecompletingarunandused

awrittenoutlinetoconductastructureddebriefingaftercompletingtherun.An

appendixtothearticledescribedthedebriefingprocedure.Groupswithoutdebriefing

werepermittedtodebriefinformally,it’snotclearwhetherornottheyhadaccesstothe

chatfeatureduringarun.(Davidovitchetal.,2010)

Agraphofmeanprofitresultsfortheeightgroupsshowedatrendof

improvementoverthefivesimulationrunswiththefourgroupswithdebriefingscoring

thehighestafterrunstwothroughfive;thehighestscoringgroupafterrun5wasthe

experiencedgroupwithhistoryanddebriefing.Forthefirstrun,nosignificant

differencesbetweenthefactorswerefoundexceptformeanprofitwith/withouthistory.

Thisverifiedthatallstudentsstartedwithsimilarknowledge,andunliketheprior

studies,thehistoryfeaturewasfoundtoenhancelearningduringthefirstrun.ANOVA

withrepeatedmeasurestestsindicatedthatforthePhaseI:BasicLearningsequenceof

foursimulationruns:


Theexperienceoftheexperiencedgroupenhancesteamlearningforall

fourruns....

Thehistorymechanismenhancesteamlearningforthefirstrunaswell

asfortheadditionalthreeruns....

Thedebriefingmechanismenhancesteamlearning....

Learningisbetterforexperiencedprojectmanagersgroupshaving

historymechanismthanexperiencedprojectmanagerswithouthistory

mechanismcapabilitiesduringthefourruns....

Learningisbetterforexperiencedprojectmanagersgroupshaving

debriefingprocedurethansuchexperiencedprojectmanagerswithout

debriefingprocedureduringthefourruns....[and]

Learningisbetterforusershavinghistorymechanismthatuses

debriefingprocedurethanforusershavinghistorymechanismwithout

debriefingprocedure,duringthefourruns.(Davidovitchetal.,2010)

Davidovitchetal.(2010)reportedthatfollowingPhaseII:Transfer,themean

profitoftheexperiencedgroupswashigherthanthemeanprofitoftheinexperienced

groups;however,areviewoftheplotofmeanprofitresultsvs.runontheprovided

graphshowstheinexperiencedgroupwithdiscussionandnohistoryoutperformingthe

experiencedgroupwithdiscussionandnohistory.Thisobservationisnotdiscussedin

thearticle.

AnalysisofPhaseII:TransferANOVAtestresultsledtothefollowingsummaryof

results:

Theexperiencedprojectmanagersgroupsenhanceteamlearningduring

thetransfertoadifferentscenariophase....

Thehistoryfactorenhancesteamlearningduringthetransfertoa

differentscenariophase[note:thehistorymechanismwasnotavailable

tostudentsduringthisphaseintheearlierresearchperformedbythe

authors....(Davidovitchetal.,2006;Davidovitchetal.,2007;Davidovitch

etal.,2008;Davidovitchetal.,2009).Thisarticledidnotstipulate

whetherornotthehistorymechanismwasavailableforuseduringthe

simulationrunofPhaseIIduringthisstudy]


Thedebriefingfactorenhancesteamlearningduringthetransfertoa

differentscenariophase....

Nosignificantinteractionwasfoundbetweenexperiencedproject

managers’factorandhavinghistorymechanism....

Nosignificantinteractionwasfoundbetweenexperiencedproject

managers’factorandusingdebriefingprocedure....[and]

Nosignificantinteractionwasfoundbetweenhavinghistorymechanism

andusingdebriefingprocedure.Davidovitchetal.(2010)

Theselastthreefindingsappeartocontradictastatementinthediscussionof

resultsthat“theinteractionoftheexperiencedgroupwiththehistorymechanismwas

significant.”WhilethiswasfoundtobethecaseduringPhaseI:BasicLearning,itdoes

notappeartobethecaseduringPhaseII:Transfer.Furtherdiscussionofthese

observationswouldclarifythis.

Summary.ThiscontrolledexperimentbyDavidovitchetal.(2010)demonstrated

thatthepriorfindingofenhancedperformanceofindividualstudentshavingaccesstoa

historymechanismasindividualsusingthePMTsimulatorisalsothecaseforteamsof

studentscollaboratingonresourcesharingusingthePTBsimulator.Thisstudyfurther

demonstratedtheadvantageofteamsusingastructuredprocesstodebrieffollowing

completionofaprojectrun.

Project management research summary.Mostarticlesdiscussingtheacademic

useofsimulationgamesdescribefindingsrelatedtotheuseofmarketing,strategy

managementortotalenterprisemanagement.Anunderresearchedareaistheuseof

projectmanagementsimulations.Onlyfifteenarticlesresearchingprojectmanagement

simulationwerefoundinavarietyofjournalsandproceedingsandnoneinSimulation

andGamingorDevelopmentsinExperientialLearning,thejournalandproceedingsof

ABSEL.All15ofthesearticlesmadeaknowledgecontribution,butonlysevencouldbe

consideredacademicallyrigorous,andfiveofthissevendocumentedaseriesofstudies

bythesameresearchers.ExceptforthisseriesbyDavidovitchetal.(2006;2007;2008;

2009;2010)whichequatedlearningeffectivenesstotheamountofsimulatedprofit,a

questionablepractice,alltheprojectmanagementsimulationstudiesmeasuredstudent

perceptionsandattitudes.Ofthese,onlyMcCreery(2003)andPfahl(2004)reported

doingthisinanacademicallyrigorousmanner.Bothusedacombinationofpretest‐


posttestmethodswithsignificantfindingsandaqualitativeanalysisofstudent

commentstoconcludetheuseofthesimulationwaseffectiveandaddedvaluetothe

educationalexperience.Acommonthemeamongstallthearticlesisstudentslikedthe

overallexperience,thecontextitprovidedforimprovingskillsandtheopportunityit

providedtopracticetheirdecision‐makingskills.

Summary

Simulationgamingasanexperientiallearningactivityisunderpinnedbythe

seminalworkofKolb(Kolb,1984)buildingonthe“intellectualoriginsofexperiential

learningfromtheworksofJohnDewey,KurtLewinandJohnPiaget.”Theuseof

businesssimulationgamesinacademicprogramshasgrowndramaticallysincethe

AmericanManagementAssociationdevelopedasimulation‐basedtrainingcoursein

1956(Fariaetal.,2009).Despitethisgrowthandtheassociatedresearchattestingtoits

effectiveness,controversyremainsaboutthebestwaytoevaluatethiseffectivenessand

therearerepeatedcallsformorerigorousresearchonthemethodsusedtoadminister

simulationgamesandonassessmentofthelearningthatresults(P.H.Anderson&

Lawton,2009;Crookall,2010;Fariaetal.,2009).Thisisespeciallytrueforproject

managementsimulationgameswhereveryfewarticlesarefoundintheliteraturethat

canbeconsideredacademicallyrigorous.

Oneareawherethereappearstobeagreementisonthedifficultyofassessing

learningbecausesimulationgameloserscanlearnmorethanthewinnersandvaluable

lessonscanbelearnedthathavenothingtodowiththeeducator’sintention(Gosenpud,

1990;Gosenpud&Washbush,1994;Greenlaw&Wyman,1973).Accordingly,the

statementofbroadlearningobjectivesforsimulationgameactivitieshasbeendeemed

acceptable,ifnotdesirable(Hertel&Millis,2002;Jones,1987)withstudentsdefining

whattheyfindmostvaluable(Gosenpud,1990).Theimportanceofdebriefingtohelp

makethislearningsalienthasbeenemphasized(Lederman,1984;Lederman,1992)and

reemphasized(Crookall,2010).DespiteAndersonandLawton’s(2009)callforless

assessmentonperceptionsoflearninginsimulations,thissuggeststhemeasurementof

perceptionsoflearningmaybeasimportantasfindingvalidmeasuresofcognitive

learningandmoreresearchintowhatislearnedisappropriate(Teach&Murff,2007;

Teach&Murff,2009).RespondingtoAndersonandLawton’scall;Cronan,Leger,Robert,

BabinandCharland(2012)comparedtheresultsobtainedfromapostsimulationself‐

assessmentsurveywithresultsfroma60questionobjectiveexaminationandfoundself‐


assessedmeasureresultscorrelatedwithobjectivemeasures.Thissuggestsself‐

assessmentsurveysarevalidproxiesforlearning.

AlthoughtheprojectmanagementsimulationresearchbyDavidovitchetal.

(2006;2007;2008;2009;2010)isquiterigorous,itsfocusonasinglemetric,simulated

projectprofit,doesnotnecessarilydemonstratewhatthestudentshavelearnedfrom

thisexperienceotherthantheabilitytodobetteratplayingthegame.Thereisno

discussionofwhatprojectmanagementskillsarebeingimprovedandwhetherthese

improvedskillsaretransferabletothepracticeofmanagingrealprojects,apoint

concededintheir2008article.Theirresearchisimportantbecauseitdemonstrates

rigorousresearchpracticesandtheuseofaprojectmanagementsimulationgame

performancemetricasadependentvariableinacademicresearch.However,theriskof

solelybasingconclusionsoflearningeffectivenessonsuchametricisthatitdoesnot

takeintoaccountthatlearningcanoccurwhilelosingthegame.Asnotedearlier,

GreenlawandWyman(1973)foundthat“strong‘learning’ingamesmaynotnecessarily

bereflectedbygoodgamingperformance”and“converselyabsenceoflearningmaynot

alwaysbereflectedby‘poor’performance.”GosenandWashbush(2004)concluded

simulation“performanceisnotaproxyforlearning,anditisinappropriatetoassess

simulationsusingperformanceasameasureoflearning.”

Amoreappropriatestartingpointforresearchintothevalueandeffectivenessof

usingaprojectmanagementsimulationasacapstoneexperientiallearningisthe

researchbyMcCreery(2003).Hisresearchmeasuredstudentperceptionsofproject

managementknowledgeandtheabilitytoapplythatknowledgebeforeandafter

experiencingaprojectmanagementsimulationgameandfoundsignificantincreasesin

both.Areplicationofthisaspectofhisstudyusingadifferentsimulationinadifferent

venuewouldaddtothebodyofknowledgeand,ifsimilarresultsarefound,suggest

generalizability.ToexpandonMcCreery’swork,questionsfromtheresearchconducted

byBuzzetto‐MoreandMitchell(2009)andAhn(2008)canbeadaptedtothe

longitudinalmethodologyusedbyMcCreerytoadditionallyresearchchangesinstudent

perceptionsofsimulationgamesandMcCreery’smethodcanbesimilarlyadaptedto

studyperceptionchangesrelatedtoteam‐basedlearningactivities.Tobegintoaddress

AndersonandLawton’s(2009)advocacyformoreobjectivemeasures,assessmentsof

cognitivelearningrelatedtoprojectmanagementschedulingandprogressreportingcan


alsobeadded.TorespondtoTeachandMurff’s(2009)callforresearchonwhatis

learned,qualitativequestionscanbeusedtoprobeforthis.

Thischapterreviewedtheliteraturerelatedtousingsimulationgamesas

experientiallearningactivitiesinacademicbusinessandprojectmanagementdegree

programs.Followingadiscussionofexperientiallearningandrecommendationsfor

conductingsimulationgames,examplesoftypicalresearchwereexploredforboth

businessmanagementandprojectmanagementgames.Additionalresearchisneeded

andastartingpointforadditionalprojectmanagementresearchisidentified.The

methodologyforconductingthisresearchisdescribedinthenextchapter.


Methodology

Thisresearchisastudyofoneuniversity’sapproachtousingaparticularproject

managementsimulationgameasacapstoneexperientiallearningactivityinitsgraduate

MS/MBAprogramforexperiencedprofessionals.Itspurposeistodetermineifthis

applicationofaprojectmanagementsimulationgameisregardedasavaluableand

effectiveuseofclasstime.ThemethodologybuildsontheapproachtakenbyMcCreery

(2003)usingadifferentprojectmanagementsimulationandadifferentdelivery

approachatadifferentuniversity.ThisstudyreplicatesMcCreery’sapproachto

measurechangesinperceptionsofprojectmanagementknowledgeandtheabilityto

abilitythisknowledgebycomparingpresimulationandpostsimulationsurvey

responses.Inadditiontothisreplication,changesinattitudesandperceptionsof

simulationgamesandteamdynamicsareexploredbyaddingadaptionsofMcCreery’s

teamexperiencequestionsandBuzzetto‐MoreandMitchell’s(2009)andAhn’s(2008)

simulationsatisfactionquestions.Analysisofpostsimulationqualitativedataisusedto

supportthefindings.Similarfindingswillsuggestthegeneralizabilityofresultstoother

deliveryapproachesusingprojectmanagementsimulationgamesinanacademic

environment.

Thischapterdiscussestheresearchparadigm,researchapproach,research

design,surveyinstruments,pilotstudy,researchparticipants,administrationprocedure,

dataprocessingandanalysisprocedure,andethicalconsiderations.

Research Paradigm

Thisresearchusesmixedmethodstoanswertheresearchquestionofwhetheror

nottheuseofaprojectmanagementsimulationasacapstoneactivityinaparticular

executivelevelMBAprogramisavaluableandeffectiveuseofclasstimeandtuition

dollars.Mixedmethodsresearchisaninquiryapproach“thatcombinesorassociates

bothqualitativeandquantitativeformsofresearch”(Creswell,2009).Theuseof

concurrent(parallel)mixedmethodsproceduresprovidesforamorecomprehensive

analysisoftheresearchproblem(Creswell,2009;Mertens,2010).Sidebysideanalysis

ofthequantitativeandqualitativeresultsallowseachtoreinforceeachother(Creswell,

2009),e.g.,inthecaseofthisstudy,findingsofsignificantgainsinknowledgeand

confidenceintheabilitytoapplythatknowledgearefurtherunderstoodandmaybe

reinforcedbythelearningthemesthatemergefromthequalitativeanalysis.


Conventionalwisdomholdsthattheresearcher’sviewoftheworldandhis

chosenparadigmunderpinsthechosenresearchapproach;however,theuseofmixed

researchmethodscanbeproblematicfromaphilosophicalperspective.Mostmixed

methodresearchersstatepragmatismfortheirparadigmbutnotallresearchers“agree

withparadigmpluralismasastartingpoint”(Tashakkori&Teddlie,2010a).Thisis

furtherexasperatedbydisagreementregardingtheneedtoevenacknowledgean

underlyingparadigmandtheroleparadigmsserveintheresearchprocess(Mertens,

2010).“Thea‐paradigmaticstancestatesthat,formanystudiesconductedwithinreal

worldsettingsespeciallyinappliedfields,paradigmsorconceptualstancesare

unimportanttopractice(e.g.,TashakkoriandTeddlie,2003)”(Tashakkori&Teddlie,

2010b).

Thedebateregardingwhichparadigmservesasthefoundationformixed

methodsresearchhasresultedintheidentificationofseveralconceptualstancesin

additiontothesingleparadigmstancewhichincludespragmatism,criticalrealism,and

thetransformativeparadigm(Tashakkori&Teddlie,2010b).TashakkoriandTeddlie

describefourothers:thesubstantivetheorystanceclaimsthattheoriessuchascritical

racetheoryorattributiontheoryaremoreimportantthanphilosophicalparadigms;the

complementarystrengthsstancestatesthatthedifferentmethodsshouldbekeptas

separateaspossibletocapitalizeonthestrengthofeachparadigmaticpositionsuchas

constructivismorpostpositivism;themultipleparadigmsstancearguesthatasingle

paradigmdoesnotapplytoeachinstanceofmixedmethodsresearch,thebestparadigm

isresearchsituationspecific;andthedialecticstanceassumesallparadigmsareuseful

andcanbecombinedinasinglestudytoprovideamorethoroughanswertothe

researchquestion.

Whilepragmatismisfrequentlycitedalongwithrealismasparadigmsformixed

methodsresearch,theunderlyingphilosophyofthisstudyisbestdescribedbythe

paradigmofpostpositivismowingtoitsprimaryuseofquasi‐experimentalquantitative

methodsandcomparisonofresultswithanotherstudysuggestingthepossibilityof

generalizabilitytoonereality.Althoughaqualitativemethodisusedinthisstudyto

reinforceorrefutetheconclusionsfromthequantitativestudy,thismethodcollected

andanalyzednarrativeresponsedatainadispassionate,objectivemannersuggestingan

epistemologymoreassociatedwithapostpositivismparadigmthanaconstructivistor

pragmaticparadigm(Mertens,2010).Thissimultaneouscollectionofquantitativeand


qualitativedatawiththeoreticaldriveorprioritygiventothequantitativedataanalysis

supplementedbyqualitativeanalysiscanbereferredtobythenotation“QUAN+qual”

wherethecapitalletters“indicatethetheoreticaldriveorpriority(coremethods)given

inastudy”(Creswell,2010).

Research Approach

TheQUAN+qualapproachtakeninthisresearchisdepictedinFigure3.The

quantitativeapproachtothisstudyadministersthesamesurveyquestionsbeforeand

afterthestudentsexperiencetheprojectmanagementsimulationgame.Thequalitative

approachsupportsorcontradictsthesefindingswithananalysisofnarrativeresponses

toadditionalpostsimulationsurveyquestions.

FIGURE3‐RESEARCHAPPROACH

Quantitative method.Thequantitativeapproachtothisstudyusesapretest‐

posttestpreexperimentaldesignasillustratedinFigure4.Datawascollectedfromfour

groupsofstudents.Sequencedintime,GroupAwasapilotstudyconductedwithanon‐

academiccorporateclasstopilotthemethodologyandGroupsBandDwereayearapart

inthegraduateprogramofinterest.GroupCwascomposedofstudentsfromasystems

engineeringandmanagementgraduateprogramwithasingleprojectmanagement

course.GroupAresultsarediscussedinthischapterandGroupBandDresultsare

discussedintheResultschapter.AnalysisofGroupCresultsisoutsidethescopeofthis

studyandanopportunityforfurtherresearch.GroupsA,B,CandDarelaterreferredto

asCohortPD,OC2010,SEM2011andOC2011,respectively.


FIGURE4‐PRETEST‐POSTTESTDESIGN

Group A: O1A ‐‐‐‐‐‐‐‐‐‐ X ‐‐‐‐‐‐‐‐‐‐ O2A

Group B O1B ‐‐‐‐‐‐‐‐‐‐ X ‐‐‐‐‐‐‐‐‐‐ O2B

Group C O1C ‐‐‐‐‐‐‐‐‐‐ X ‐‐‐‐‐‐‐‐‐‐ O2C

Group D O1D ‐‐‐‐‐‐‐‐‐‐ X ‐‐‐‐‐‐‐‐‐‐ O2D

O1 = Presimulation survey

O2 = Postsimulation survey

X = Simulation game

Studentperceptionsaresurveyedimmediatelypriortobeginningthesimulation

gameandagainimmediatelyaftertheconcludingdebriefdiscussion.Analysisofthedata

isperformedtodeterminethesignificanceofthedifferences.

Qualitative methods. Open‐endedquestionsinthepostsimulationsurveyand

presentationsmadeduringthesimulationdebriefprovidequalitativedataforfurther

assessmentofparticipantattitudestowardsthevalueofthesimulationgameexperience.

Additionalqualitativedataiscollectedfollowingcompletionofthe21SCHProject

ManagementCorePhaseoftheMS/MBAgraduateprogramtoassesssatisfactionwith

theoverallprojectmanagementlearningexperience.Thislatterdataisscannedfor

commentsregardingthesimulationexperienceandananalysisofthisdataisincluded

withthequalitativeassessmentresults.

Research Design

Variables.Perceptionsmeasuredareprojectmanagementknowledge(K1

[presimulation]andK2[postsimulation]),confidenceintheabilitytoapplythat

knowledge(C1andC2),usefulnessofsimulationgamesingeneral(SG1andSG2),

usefulnessofthesimulationgameexperienced(SP1andSP2),attitudetowards

teamworkingeneral(TG1andTG2),andcurrentteamsatisfaction(T1,TP1,T2and

TP2).Alsoassessediseachstudent’sabilitytosolvefundamentalproblemsinvolving

thecriticalpathmethodandearnedvaluemanagement(E1,N1,P1andE2,N2andP2,

whereP1=E1+N1andP2=E2+N2).Theconstructsforthesevariablesandthe

demographicdatacollectedisdiscussedlaterinthischapterinthesectiondiscussingthe

surveyinstruments.


Method.FollowingMcCreery’s(2003)approach,t‐testsformatchedpairs

(dependentt‐tests)determinethestatisticalsignificanceofdifferencesbetweenthe

presimulationandpostsimulationmeasurementsforeachofthecohortgroups.

RelationshipstotheresearchhypothesesofthisstudyaredepictedinFigure5.The

resultsofstatisticaltestsareconsideredsignificantforp<.05.

Qualitative Analysis.Thestatisticalresultsarecomparedwithacategorizationof

thequalitativefindingscollectedduringthepostsimulationdebriefsessionsand

reportedonthepostsimulationandprogramevaluationsurveys.

FIGURE5‐RESEARCHMODEL

Survey Design

SurveyquestionsarebasedonthepriorworkofMcCreery(2003),Buzzetto‐

MoreandMitchell,(2009),andAhn(2008).Thepresimulationsurvey,foundin

AppendixC,iscomposedof74questions.Fiveofthesequestionsaredemographicand

58combinetoformtheattitudeandperceptionvariables.Table3liststhevariablesand

theirassociationwiththe58attitudequestions.


TABLE3‐ATTITUDEVARIABLEQUESTIONS

Variable (1=presimulation; 2=postsimulation) Questions

Knowledge Self‐Assessment (K1, K2) 16 questions: 1A‐1F

Knowledge Application Confidence (C1, C2) 16 questions: 2A‐2F

Teamwork Attitude (McCreery) (T1, T2) 9 questions: 3A‐3I

Current Team Satisfaction (TP1, TP2) 10 questions: 3A‐3I, 4N

Teamwork Attitude (Generic) (TG1, TG2) 2 questions: 4C, 4K

Generic Simulation Attitude (SG1, SG2) 6 questions: 4A, 4B, 4E, 4G, 4I, 4L

This Simulation Attitude (SP1, SP2) 8 questions: 4D, 4F, 4H, 4J, 4M, 4O, 4P, 4Q

Theremaining11questionscomprisetheknowledgeapplicationtestquestions

relatedtonetworkdiagramsandearnedvaluemanagement.Table4showsthe

associatedvariablesandtheirrelationshiptothequestions.Thenetworkanalysis

questionsareadaptedfromanexercisefoundinthe“DevelopingaProjectPlan”chapter

ofoneofstudents’projectmanagementtextbooks(Gray&Larson,2008).Theearned

valuequestionsaskthestudenttoanalyzetheprogresspresentedinafigurefoundin

the“ProgressandPerformanceMeasurementandEvaluation”chapterofthesametext

(Gray&Larson,2008).

TABLE4‐PROBLEM‐SOLVINGQUESTIONS

Variable (1=presimulation; 2=postsimulation) Questions

Network Analysis Ability (N1, N2) 4 questions: 5A‐5D

Earned Value Analysis Ability (E1, E2) 7 questions: 6A‐6G

Problem‐solving Ability (P1, P2) P1 = N1 + E1, P2 = N2 + E2

Table5liststhedemographicdatacollectedinthepresimulationsimulation.

TABLE5‐PRESIMULATIONDEMOGRAPHICVARIABLES

Years Project Experience

Years Work Experience

Gender

Prior Formal Project Management Training (type and duration)

Background (technical/non‐technical)

Question sources.The16knowledgeself‐assessmentquestions(1A‐1F),16

knowledgeapplicationconfidencequestions(2A‐2F)and9of10currentteam

satisfactionquestions(3A‐3I)areidenticaltothoseusedbyMcCreery(2003).


Simulationattitudequestions4Athrough4Mareslightre‐phrasingsofBuzzetto‐

MoreandMitchell’s(2009)questionstoadaptthemforuseinpretestandposttest

surveys.Questions4Othrough4QarefromasurveyusedbyAhn(2008).Inbothof

thesecases,theresponsescalewasexpandedfromfivepointstosevenpointstobe

consistentwiththescalesusedbyMcCreery(2003):

Questions1and2:1=ExtremelyLowto7=ExtremelyHigh

Questions3and4:1=StronglyDisagreeto7=StronglyAgree.

Thepostsimulationsurveyquestionsareidenticaltothepresimulationsurvey

questionsexceptthe“thissimulation”attitudequestionsarere‐phrasedtoaskabout

impressionsofthesimulationjustcompletedversuspreviouslyaskingabout

expectationsfortheupcomingsimulation.Forexample,thepresimulationquestion,“I

believethissimulationwillbefunandexciting”changedto“thissimulationwasfunand

exciting”inthepostsimulationsurvey.Thefivedemographicquestionsofthe

presimulationsurveywerereplacedbythesixdemographicquestionslistedinTable6

andthefiveopen‐endedquestionslistedinTable7.Thepostsimulationsurveyisfound

inAppendixD.

TABLE6‐POSTSIMULATIONDEMOGRAPHICVARIABLES

Industry

Job Title

Education Level

Age

Global Origin

Annual Income

TABLE7‐POSTSIMULATIONOPEN‐ENDEDQUESTIONS

What did you like about the just‐completed simulation experience?

What did you learn from this experience?

Excluding changes to the computer simulation itself, what changes do you think could be made to the overall simulation experience to better prepare you to manage projects?

What changes do you recommend be made to the computer simulation itself?

Elaborate on any other thoughts you have regarding learning project management with a computer‐based simulation game.

Other qualitative data sources.Inadditiontothepresimulationand

postsimulationsurveys,qualitativedataisderivedfromthesimulationdebriefteam


presentationsthatprecedetheadministrationofthepostsimulationsurveyandfromthe

ProjectManagementCorePhaseexitsurvey.Duringthedebrief,eachteampresentsits

finalstatusreporttomanagementdiscussingitsmetricsandvariances,aprojectaudit

reportdescribingtheteamstrategyandlessonslearned,andrespondstothequestion,

“WhatdidyoulearnaboutprojectmanagementfromparticipatingintheSimProject

simulation?”Thehandoutdescribingthedeliverablesassociatedwiththisteamdebrief

presentationinfoundinAppendixE.Table8listsrelevantopen‐endedquestionsfrom

theProjectManagementCorePhaseexitsurvey.

TABLE8‐RELEVANTPROJECTMANAGEMENTCOREPHASEEXITSURVEYQUESTIONS

If you were given an option to change only 3 things in the Project Management Core Program, what would those 3 things be, and why would you change them?

If you were given an option to retain only 3 things in the Project Management Core Program, what would those 3 things be, and why would you keep them?

Survey Structure.Toeasethementalburdenontherespondentandtoprovidea

clearframeofreference,questionsweregroupedintosectionsbasedontheiroverall

topicandscaletypewitheachsectionpresentedononepageinamatrixformat.While

someauthorsofe‐mailandweb‐basedresearchbooksdiscouragetheuseofmatrix

questionsbecauseofextraworkwithinascreen(Schonlau,Elliott,&Fricker,2002),

othersemphasizethegeneralneedtogivetherespondentthesenseofasmooth

progression,thusfacilitatingarapidprogressionthroughthequestionnairebygrouping

questionswiththesamescaletogether(Czaja&Blair,2005).AlreckandSettle(1995)

advisethatgroupingitemsbybothtopicandscaletypeis“theidealsituationbecause

therearebothalogicalsequenceandahighdegreeoftimeandspaceeconomy.”Cooper

andSchindler(2006)emphasizetheimportanceofprovidingastrongandobviousframe

ofreferenceforthequestionswithclearlydefinedtransitionsbetweensectionsto

delimitchangesinframeofreference.

Groupingquestionsbytopic,whichinseveralcaseshereisalsogroupingthe

questionsconstructingavariable,cancauseconcernoverorderingeffectswith

respondentsselectingthesameratingforallanswers.Areviewofthedatacollected

showedthisrarelyoccurred,andwhenitdid,itwasneverforallsectionsofthesurvey.

Sincethetotalquestioncountwasabove70,followingtheaforementionedadvicethatit

isbettertoimprovethecooperationoftherespondentsandreducetheircognitive


burdenbyorderingthequestionsbytopicinsteadofrandomizingthemappearstobe

valid.

ConsistentwithguidelinesofferedbyCzajaandBlair(2005)basedonthework

ofDillman(2000),thepapersurveysweresevenpagesinlengthwithsufficientlyspaced

precodedresponsecategoriesarrangedvertically.Topicalquestionsusingthesame

scaleweregroupedtogetheronthesamepage.Toavoidconfusion,noreversingof

questionswasdone.

Thenetworkanalysisknowledgeapplicationquestionspresentedthe

respondentwithatableofsevenactivitieswithpredecessoranddurationdataandasked

twoquestionsaboutthecriticalpathandtwoquestionsaboutactivityslacktime.Each

questionwasscoredaoneforacorrectanswerandazeroforanincorrectanswer.A

respondentansweringallfourquestionscorrectlyearnsascoreoffour.

Theearnedvalueknowledgeapplicationquestionspresentedtherespondent

withacumulativeearnedvaluegraphshowingPV,EV,AC,BAC,andEACforaprojectin

progressandaskedsevenquestionsrequiringanalysisandinterpretationofthe

graphicaldata.Correctanswersscoredaoneandincorrectanswersscoredazero.A

respondentansweringallsevenquestionscorrectlyearnsascoreofseven.

Pilot Study

Thesurveywaspilotedusingthe19participantsinGroupA,acorporate

professionaldevelopmentseminarserieswithsimilarcontenttotheproject

managementmaster’sdegreeprogramofinterest.UnlikeMcCreery’s(2003)results,

whichwerebasedon63participants,thematched‐pairt‐testsyieldedsignificant

increasesforsome,butnotall,individualquestionmeans;similartohisresults,these

testsdidyieldsignificantdifferencesfortheknowledgeandconfidencetoapplythat

knowledgegrandmeanscomprisingtheKandCvariablesunderstudy.

Thereliabilityofusingthesegrandmeans,exceptgenericteamworkattitude

(TG)whichisthemeanresponseofonlytwoquestions,wasconfirmedasthescales

representedbytheK,C,TP,SG,andSPvariableshadCronbach’sα>.8.Thesevaluesare

listedinTable9.


TABLE9‐PILOTTESTSCALERELIABILITY

Cronbach's α

Variable Description 1

(Pre‐sim.) 2

(Post‐sim.)

K Knowledge .934 .938

C Confidence to apply knowledge .948 .963

TG Teamwork attitude .771 .606

TP Current team satisfaction .95 .96

SG Generic simulation attitude .941 .883

SP This simulation attitude .895 .887

Onaverage,participantsreported:

Significantincreaseinprojectmanagementknowledge(M=5.29,SE=

0.15)fromtheirpresimulationperceptions(M=4.80,SE=0.16),t(18)=‐

7.72,p<.05,r=.88.

Significantincreaseinconfidenceinabilitytoapplyprojectmanagement

knowledge(M=5.16,SE=0.18)fromtheirpresimulationperceptions(M

=4.80,SE=0.16),t(18)=‐3.07,p<.05,r=.59.

Favorableattitudes,butnosignificantincreaseswereseenfortheteamand

genericsimulationperceptions.Onaverageparticipantsreported:

Nosignificantincreaseingeneralteamworkattitude(M=5.53,SE=

0.196)fromtheirpresimulationperceptions(M=5.58,SE=0.173),t(17)

=0.399,p>.05,r=.10.

Nosignificantincreaseincurrentteamsatisfaction(M=5.526,SE=

0.285)fromtheirpresimulationperceptions(M=5.500,SE=0.232),

t(18)=‐.122,p>.05,r=.03.

Nosignificantincreaseingeneralattitudetowardstheuseofsimulations

asalearningactivity(M=5.15,SE=0.201)fromtheirpresimulation

perceptions(M=5.13,SE=0.19),t(17)=‐0.138,p>.05,r=.03.

However,onaverage,participantsdidreportasmallbut

Significantincreaseinfavorableattitudetothisparticularsimulation

experience(M=5.35,SE=0.211)fromtheirpresimulationperceptions

(M=5.1,SE=0.190),t(17)=‐1.968,p<.05,r=.43.


Ontheschedulenetworkandearnedvalueanalysisproblems,participants

demonstrated

Nosignificantincreaseintheirabilitytosolveschedulenetworkanalysis

problems(M=2.68,SE=0.316)fromtheirpresimulationability(M=

2.58,SE=0.336),t(18)=‐1.000,p>.05,r=.22.

Significantincreaseinintheirabilitytosolveearnedvalueanalysis

problems(M=3.6316,SE=0.406)fromtheirpresimulationability(M=

2.68,SE=0.297),t(18)=‐2.282,p<.05,r=.47.

Significantincreaseintheiroverallabilitytosolveanalysisproblems(M

=6.3158,SE=0.490)fromtheirpresimulationability(M=5.26,SE=

0.512),t(18)=‐2.727,p<.05,r=.54.

Itisnoteworthythatthisprofessionaldevelopmentcohortsimulationexperience

includedaninstructor‐led20minutediscussionoftheearnedvaluemanagement

methodologywiththeentireclassduringoneoftheinterimprogressreportingmeetings

andonewouldexpecttheretobeasignificantincreaseinabilitytosolveearnedvalue

analysisproblems.InkeepingwithHeyman’s(1975)advicethat“youarenotthereto

teach...thesimulationdoestheteaching,”instructorledtutorialdiscussionsofthistype

wereavoidedduringthesimulationactivityfortheMS/MBAcohortsthatarethesubject

ofthisstudy.

Participants

Acombinedtotalof60peopleparticipatedinthefourgroupsexperiencingthe

simulationgame.Table10liststhesimulationdates,totalclasstimeallottedtothe

simulationgame,participantcountsandcontacthoursofinstructionpriortostartingthe

simulationgameforeach.GroupA,thecorporatemanagers,participatedinthe

previouslydescribedpilotstudy.GroupsBandDarethesubjectsofthisstudy;GroupC

datawascollectedforafuturestudy.Individualclasssizesaresmallduetothe

specializednatureoftheseprograms.Intheacademicgroups,studentswereassignedto

teamstomaximizediversityatthefirstclassmeeting;theteamsinthecorporate

programwerecomprisedofmanagersfromdifferentfunctionalareaswithincommon

businesscomponents.Inallcases,studentsworkedtogethertocompleteteam

assignmentsbeforebeginningtheprojectsimulation.


TABLE10‐SIMULATIONPARTICIPANTS

Class Simulation dates /

Total class time allotted Number of Participants

Contact Hours of Instruction

Prior to Starting

Simulation

Corporate managers (single company) Group A (PD)

March 23‐25, 2010 20 hours

19 176

Project Management Graduate Program Group B (OC2010)

June 10‐12, 2010 16 hours

14

220

Systems Engineering and Management Graduate Program Group C (SEM2011

April 15, 22, 29, 2011 12 hours

13 28

Project Management Graduate Program Group D (OC 2011)

June 10‐11, 2011 July 15, 2011 16 hours

14

204

Total number of participants 60

Pilot study participants.Thepilotstudywasconductedattheconclusionofa

professionaldevelopmentprogramheldfor19cross‐functionalmanagerschargedwith

developingastrategytoimprovetheprojectmanagementpracticesandperformanceat

theirenergyservicescompany.Thisprogramwassimilartotheacademicgraduate

programunderstudyindurationandcontent,butwithouttheassessmentrigorrequired

forgraduatecoursework.Classeswereheldfrom8AMto5PMinahotelmeetingroom

ontwoconsecutivedayspermonthfor11monthswithlunchandbreaksnacks

provided.Thetwelfthmonth’ssessionswereheldatthecorporateheadquartersto

provideeachteamwithitsownbreakoutroomfortheconductofthesimulation.All19

managersparticipatedinthispilot.Onaverage,theyreported12.3yearsofwork

experienceand7.9yearsofexperiencewithprojects;therewere14menand5women.

Project management graduate program participants.Thegraduateprogram

understudyischaracterizedbysmallclasssizesduetoitsspecializednatureand

premiumtuition.Twenty‐eightstudentsfromtheprojectmanagementgraduate

program(14inGroupBand14inGroupD)completedfiveprecedingproject

managementcoursesandanintegratedcourseinorganizationalbehaviortotaling17

SCHofacademicstudy.Topicalcontentwassimilartothatreceivedbythecorporate

managers(GroupA)butwithsubstantialacademicrigor.Thethirdclass(GroupC)was

attheconclusionofasingleprojectmanagementoverviewcoursefromasystems

engineeringandmanagementgraduateprogramwithsimilaradultlearners.Although

notfromtheprogramunderstudy,thisthirdinstanceisofinterestbecauseofits


similaritytotheprogramstudiedbyMcCreery(2003)andisasubjectforpotential

futureresearch.

Classesinthegraduateprojectmanagementprogram(GroupsBandD)areheld

ononeconsecutiveThursday‐Friday‐SaturdaypermonthintheExecutiveEducation

AreaoftheSchoolofManagementbuildingfrom8AMto5PM;breakfast,lunchand

breaksnacksareprovided.Studentscomplete9monthsofcourseworkpriorto

participatinginthesimulationgame.

Duetothesmallclasssizes,randomsamplingisnotperformed;allstudentsare

invitedtoparticipateinthestudypriortothestartofthesimulation.Noincentivesto

participateorpenaltiesfornotparticipatingareassessed.Theinstructorleavesthe

roomandisnotawareofwhodidordidnotparticipateinthesurvey;however,mostdo

participate.

Procedure

Inthissimulationgame(brieflydescribedinAppendixF),studentteamstakeon

theroleoftheprojectmanagementcoreteamresponsibleforselectingandmanaging

theresourcesassignedtoasimulatedproductdevelopmentproject.Thissection

providesachronologicaldescriptionoftherelatedpresimulationinstructionand

preparatoryassignments,presimulationsurveyadministration,simulation

administration,andpostsimulationsurveyadministration.

Presimulation instruction.Studentsparticipatinginthegraduateprogram

leadingtoeitheraM.S.orMBAwithanemphasisinprojectmanagementcompleteover

200contacthoursofprojectmanagementandintegratedorganizationbehavior

courseworkpriortobeginningexecutionofthesimulatedproject.Theyareintroduced

tothesimulationgamethreemonthspriortoitsexecutionduringtheirfourthsequential

projectmanagementcoursecoveringprojectplanningandexecutionintheformofan

assignment.Theprimarypurposeoftheassignmentisforstudentstoapplythe

conceptsofprojecttime,resourceandcostmanagementbyfirstdevelopingabaseline

projectscheduleandbudgetfromgivenstandards,thenbydevelopingastaffing

managementplanandarevisedbaselinescheduleandbudgetbasedonanalysisof

availableresourcesandtheircharacteristics.Thesecondarypurposeoftheassignment

isforthestudentstobecomefamiliarwiththesimulationscenarioandtoprepareaplan

inadvanceofexecutionday.ThisassignmentisavailableforreviewinAppendixB.

Aftersubmittingthebaselineplanfromgiveninformation,studentsreceivethecorrect


solutiontouseasthestartingpointforcompletingparttwooftheassignment;theteams

presentthisrevisedplaninasimulatedmeetingwiththesponsoramonthlaterinclass.

Presimulation survey administration.Instructionsforcompletingthesurveysare

givenverballybytheinstructoralongwithassurancesofanonymityandvoluntary

participation.Tohelpassureanonymity,theinstructorleavestheroomduring

completionandcollectionofthesurveysisadministeredbyastaffmemberorstudent.

Thepresimulationsurveyisadministeredatthebeginningoftheclassperiodpriortoa

reviewofpresimulationquizandtheteamsbreakingouttobeginthesimulationgame.

Allstudentscompletedthepresimulationsurvey.

Simulation administration.Thesimulationisadministeredfollowingthe

philosophythatstudentsshouldlearnfromtheexperienceandnotfromthefacilitator.

Instructorinteractionwiththeclassoccursprimarilyduringthesimulationkick‐off

briefing,duringinterimteamstatusreviewswithmanagementandduringthecombined

finalpresentationtomanagementandsimulationexperiencedebrief.Studentsalso

interactbrieflywiththeinstructorduringtheprocessinganddisplayofresultsattheend

ofeachsimulatedworkperiod.Exceptforthepreviouslymentionedreviewofearned

valuemanagementduringthepilot,theinstructordoesnotteachorconsultduringthe

administrationofthesimulation.

Simulation time allocation.Sixteenhoursofclasstimeareallocatedtoplaying

anddebriefingthesimulationgame.Thistimeisdividedintofourfour‐hourclass

modules.Morningmodulesrunfrom8AMtonoonandafternoonmodulesrunfrom1to

5PM.Playbeginsinanafternoonmodulestartingat1PMandendingat5PM.

Module one – the first four hours.Duringthefirstmodule,studentsfirst

participateinapresimulationkick‐offbriefingheldfortheentireclass,thenmeet

privatelyasteamstoreviewtheirstrategyandentertheirresourcebidsforthefirst

roundofpreplay.Onceallbidsarein,studentsreturntotheclassroomwherethe

instructorinitiatesthesimulatorpreplayroundandstudentsreceivetheresultsoftheir

preplaybids.Additionalpreplayroundsareheldasneededforteamsthatlostbidsand

stillneedtohireresourcesforthefirstworkround.Preplayendswhenteamsare

satisfiedtheyareadequatelystaffedandtheinstructorinitiatesthefirstworkround.

Oneortwoworkroundsarecompletedduringthefirstfour‐hourmodule.


Thekick‐offmeetingisusedbytheinstructorto:

Administerthepresimulationsurvey

Describehowthesimulationwillbeadministered

Reviewtheresultsofthepresimulationquizasarefresheroftherulesof

thegame

Answeranyquestionsregardingtherulesoradministration

Instructtheteamstobeginroundoneofpreplaybyenteringtheir

resourcebids

Thevariationinthenumberofworkperiodscompletedduringthissessionisthe

resultofsometeamsneedingmorethanonepreplayroundtoobtaintheirstarting

resources.Forthetwocohortgroupsunderstudy,GroupBrequired2preplayrounds

andGroupDrequired3preplayrounds.Asaresult,GroupBcompletedonesimulated

workperiodduringthisopeningsessionandGroupDcompletedtwo.Thisfirstfour‐

hourmoduleendswithallteamsmeetingtogetherwiththeinstructortoviewtheresults

ofthejust‐endedsimulatedworkperiodandtoreceivetheassignmenttopreparea

statuspresentationformanagementthatwillbedeliveredatthebeginningofthenext

modulesessionfollowinga30minuteteamworkperiod.Inkeepingwiththecapstone

natureofthisassignment,teamsmustdecideforthemselvesbasedonpriorlearningand

experienceswhattheyaregoingtopresentandhowtheywillformatthepresentation.

Nospecificationsortemplatesaregivenforthispresentationtomanagement.

Module two – the next morning.Thesecondmodulebeginswithashortteam

workperiodtocompletetheirpresentationpreparationandtoenterdecisionsforthe

nextworkperiodround.Thisisfollowedbya“meetingwithmanagement”withthe

instructoragainplayingtheroleastheprojectsponsor.Teamspresenttheirstatus,

receiveencouragementbutnoguidance,andviewthecompletionresultsforthenext

simulatedworkperiod.Threetofouradditionalworkperiodroundsarecompleted

duringthisfour‐hoursession,eachwithtypically30minutestoreviewresults,discuss

neededchangestotheplan,andenterdecisionsforthenextsimulatedworkperiod.

Afterreceivingtheresultsofthefinalsimulatedworkperiodofthemodule,studentsare

againadvisedtoprepareanotherinterimstatuspresentationformanagementwith

contentandformatoftheirchoosingthatistobedeliveredatthebeginningofthethird

four‐hourmodule.


Module three – schedule variation.Dependingonthecourseschedule,thethird

four‐hoursimulationmodulebeginsafterlunchonthesameday(GroupB)oramonth

laterwhenstudentsreturntocampus(GroupD).Thisdifferenceisduetoachangein

coursescheduletoprovidemoretimeforcompletingassignmentsrelatedtothe

nonsimulationportionofthecourse.Theextendedformatisplannedtobethenormfor

futurecohorts.Duringthethirdmodule,studentscompleteanadditionalfivesimulated

workperiods.Followingthereceiptofresultsforthesimulatedworkperiodconcluding

thissession,studentsreceivethespecificationsforthefinalpresentation.Thefinal

presentationisacombinedfinalprojectstatusreport,projectauditreportandstatement

oflessonslearnedwithadefinedoutline.Thehandoutwiththerequirementsforthis

presentationisshowninAppendixE.

Module four – who won?Thefourthandfinalfour‐hourmodulebeginswitha

shortworkperiodtocompletedecisionsforthenextroundor,inthecasewherethe

simulationwascompletedattheendofthepriorsession,alongerworkperiodto

completethefinalpresentation.Intheformercase,theremainingroundsarecompleted

withinthefirsttwohoursofthemodule,anhourisallowedtopreparethefinal

presentation,andthefinalpresentationsanddebriefareconductedduringthefourth

hour.Inthelattercase,studentsareinformedofthetimeofthefinalpresentationand

arepermittedtoscheduletheirclassarrivalandteampreparationtimeastheyseefit.

Duringthesefinalstatuspresentations,theinstructorallowsthestudentstoself‐

facilitateandonlyasksprobingquestionsofthepresentingteamiftheotherteamsdo

not.Followingthelastpresentation,theinstructorshareshisobservationsonwhat

occurredandfacilitatesadebriefdiscussion.

Postsimulation survey administration.Thepostsimulationsurveyisadministered

immediatelyfollowingthepostsimulationteampresentationsanddebriefdiscussion.

Theinstructoragainleavestheroomtohelpassureanonymity.Onestudentdidnot

completethepostsimulationsurveyandanothertwosurveyswereunusablebecausethe

studentdidnotincludetheanonymousidentificationinformationrequiredtoallowa

longitudinalstudyontheirindividualresponses.

Data Processing and Analysis

Thisstudyusesquantitativemethodstoacceptorrejecttheresearchhypotheses.

Theseconclusionsaresupplementedbythethemesfoundinthequalitativedataand,

togetherwiththeresultsofthequantitativeanalysis,triangulatetoanswerthe


fundamentalquestion:Istheuseofaprojectmanagementsimulationasacapstone

activityaneffectivetechniqueforaddingvaluetoanexecutiveMBAprogram

specializinginprojectmanagement?Thissectiondescribestheresearchmethodsused.

Avoiding sample selection bias.Sinceweareconcernedwiththeeffectivenessof

asimulationgameinaparticularprogramusingaspecificdeliverymodel,andthe

populationofinterestissmallandidentifiable,acensusofallstudentsparticipatingin

thesimulationgameistaken.Thiseliminatesthepossibilityofsampleselectionbiasand

isnecessarybecausesamplingmighteliminateimportantcasesfromthestudyand

credibilityrequiresconsiderationofallmembersofthetargetpopulation(D.R.Cooper&

Schindler,2006).Inthecaseofstudentsattendingacourseinaclassroom,dataiseasy

tocollectandmoststudentsarewillingtoparticipateiftheiranonymityisassured.

Inspectionofthedatashowswhethertheelementsarequitedifferentfromeachother

andsuggestwhethersamplingmightbepossibleforlargerpopulationsofinterest

(Burns&Burns,2008).

Initial data review. Dataarereviewedforextremevaluesandmissingdata

elements.Inthecaseofamissingdataelement,theaverageansweroftheother

respondentsisenteredforthemissingdata(Burns&Burns,2008;Fowler,2009).

Data reliability.Thisstudyusesparametrictestsbasedonthenormal

distributionandaccordingly,datamustbecheckedtoverifytheymeettheassumptions

ofparametrictests:normallydistributeddata,homogeneityofvariance,intervaldata

andindependence(Field,2009).Dataarecheckedfornormaldistributionusingthe

Kolmogoroc‐SmirnovandShapiro‐Wilktests.Homogeneityofvariancebetweenthe

cohortgroupsistestedusingLevene’stest.Theattitudinalsurveydataresponsesare

treatedasintervalandvalidforparametrictesting;thesurveyscalesusedimplyan

equaldifferencebetweeneachrating(Field,2009).

Independenceisassumedsince,inthecaseofthepresimulation,participantsdo

notseethequestionnairebeforetheyareaskedtocompleteit.Afterwardstheyare

immediatelydistractedfromdiscussingitbytheneedtogetstartedwithreviewingtheir

teamstrategyforexecutingthesimulationprojectandplacinginitialbidsforresources.

This,combinedwithnotknowingthesimilarityofthepostsimulationquestionnaire,

suggeststheirpostsimulationresponseswillnotbeinfluencedbyotherparticipants.

Thereliabilityofusingthegrandmeanoftheattitudequestionstocreatethe

variablespreviouslylistedinTable3isassessedbycalculatingCronbach’sAlpha.


Cronbach’sAlphavaluescloseto1indicateahighdegreeofinternalconsistencyand

reliabilityamongthesetofquestionresponses(Sekaran,2003).Acceptablelevelsfor

Cronbach’sAlphaare0.7andaboveandcoefficientsof0.9andaboveareconsideredto

indicateexcellentstrengthofassociation(Burns&Burns,2008).

Hypothesis testing.Dependentt‐testsareusedtotestforsignificantdifferences

betweenpresimulationandpostsimulationmeasures.Theresultsareconsidered

significantforp<.05.

Qualitative analysis.Narrativedataiscollectedfromthesimulationgame

participantsduringtheirfinalpresentationanddebriefdiscussions,fromtheir

postsimulationsurveyqualitativequestionresponses,andfromcourseandprogram

evaluationsurveys.Thesedataarecategorizedtoreportemergentthemesand,inthe

caseoftheexitsurveys,favorableandunfavorablecommentsarecountedandreported.

Postsimulation team presentations.Followingcompletionoftheproject

simulationgame,eachteampreparesandpresentsashort(20minute)PowerPoint

presentationcontainingthefinalprojectstatusreporttothesponsor,aprojectaudit

reportdiscussinglessonslearnedonthesimulatedproject,andasummaryofwhatthey

learnedaboutprojectmanagementfromparticipatinginthesimulation.Thisisfollowed

byaninstructor‐leddebriefdiscussionoftheoverallexperience.Presentationand

debriefdiscussioncontentiscategorizedandemergentthemesarereported.

Postsimulation survey open‐ended question responses.Thepostsimulationsurvey

askedparticipantstorespondtofouropen‐endedquestions:

1. Whatdidyoulikeaboutthejust‐completedsimulationexperience?

2. Excludingchangestothecomputersimulationitself,whatchangesdoyou

thinkcouldbemadetotheoverallsimulationexperiencetobetter

prepareyoutomanageprojects?

3. Whatchangesdoyourecommendbemadetothecomputersimulation

itself?

4. Elaborateonanyotherthoughtsyouhaveregardinglearningproject

managementwithacomputer‐basedsimulationgame.

Answerstoeachquestionwerecategorizedandanalyzedtoidentifyemergent

themes.


Course and program evaluation surveys.Course,ProjectManagementCorePhase,

andgraduateexitsurveysareexaminedforcommentsrelatedtothesimulation.These

commentsarenotedandcountedasappropriate.

Ethical Considerations

Studentswereadvisedthatparticipationinthisstudywasvoluntaryand

anonymous.Theconsentprocesswascomprisedofaverbalstatementpriorto

beginningthesimulationsimilarto:

Iamconductingresearchintotheeffectivenessofprojectmanagement

simulationgamesandwouldappreciateitifyouwouldparticipateinthis

researchbycompletingashortsurveybeforeandafterthesimulation.

Participationisentirelyvoluntaryanddoesnotaffectyourcoursegradeinany

way.Tohelpprotectyouranonymity,Ionlyneedyoutoindicatewhichteamyou

areamemberofandtoassignanindividualcode,knownonlytoyou,thatwill

allowcomparingpresimulationandpostsimulationsurveyresponsesby

anonymousindividual.Ifyouarenotcomfortableansweringanyofthe

demographicquestions,youmayleavethemblankastheyaretheretodescribe

thecompositionofthesampleandarenotvitalforthisresearch.Completingthe

surveywillindicateyourconsent.Ifyouchoosenottoparticipateyoumaytakea

breakuntilwereconvenein30minutestobeginthesimulation.

IcompletedtheNationalInstitutesofHealth(NIH)web‐basedtrainingcourse

“ProtectingHumanResearchParticipants”andtheresearchplanwasreviewedand

approvedbytheInstitutionalReviewBoardofTheUniversityofTexasatDallaswhereI

amemployedandwheretheresearchwasconducted.

Summary

Thisresearchusesamixedmethodsapproachtostudyoneuniversity’sapproach

tousingaprojectmanagementsimulationgameasacapstoneexperientiallearning

activityinitsgraduateMS/MBAprogramforexperiencedprofessionals.The

quantitativeapproachbuildsonthepriorworkofMcCreery(2003)andallowsfor

comparisonstobemadetopositgeneralizabilityaboutchangesinperceptionsof

knowledgeandabilitytoapplythatknowledge.Additionaladaptationsprovidefor


determinationofchangesinattitudesrelatedtoteamworkandsimulationgamesas

experientiallearningactivities.Additionalquestionsprovideformeasuringgainsinthe

abilitytoanalyzeprojectnetworkandearnedvaluedata.Thequalitativeapproach

examinespostsimulationandpostcoursesurveydatatoidentifyemergentthemesof

whatstudentslearnedfromtheexperiencetohelpexplainandreinforcethequantitative

findings.Apilotstudyvalidatedthemethodologyandtheprocedureforconductingthe

study


Results

Thischapterdescribesandanalyzesthequantitativedataobtainedfromthe

presimulationandpostsimulationsurveysandthequalitativedataobtainedfromthe

postsimulationdebriefsession,postsimulationsurvey,andtheProjectManagementCore

Phaseexitsurvey.Thefirstsectiondescribestheresultsofacheckofthedatafor

completenessandoutliersandthetreatmentusedformissingdata.Thesecondsection

checkstheassumptionsofparametricdata,importantbecauseofthesmallsamplesize.

Thethirdsectiondescribesthediversityofthestudentparticipants.Thefourthsection

analyzesthedatabasedonthehypothesesunderstudy.Thefifthsectionsummarizes

thequalitativeresults.Thesixthsectionreportsadditionalquantitativedatafroma

courseevaluationsurvey.

Presimulationandpostsimulationsurveyquantitativedataareanalyzedusing

SPSSforWindowsPASWStatistics18.0.2.Theresultsofstatisticaltestsareconsidered

significantforp<.05.

Initial data analysis

Theinitialanalysisofthesurveydataconsistedofcalculatingtheresponserate

andcheckingforandtreatingextremevaluesandmissingdataelements.Noextreme

datavalues(outliers)werefound.

Response rate.AsdiscussedintheSamplePopulationsectionofthe

MethodologyChapterandsummarizedinTable10,60peoplein4cohortgroups

participatedinthesimulationgamewithpresimulationandpostsimulationsurveys.

GroupAwasthepilotgroupcomprisedofparticipantsinacorporatetrainingprogram

withcontentandcontacthoursthataresimilartothetwocohortgroupsofinterestin

thegraduatedegreeprogramwithemphasisinprojectmanagement.Thepilotgroupis

categorizedasPD(ProfessionalDevelopment)inthedata.GroupsBandDarefromthe

graduatedegreeprogramunderstudyandarecategorizedinthedataasOC2010and

OC2011.Datawasalsocollectedforparticipantsattendingasingleexecutivegraduate‐

levelacademiccourseinprojectmanagement(GroupC)whocompletedsimilar

academicassignmentstoGroupsBandD,butexperiencedfewercontacthoursof

presimulationprojectmanagementstudy.GroupCcompletedthesimulationactivityin

thetimeperiodbetweenthesimulationactivityforGroupsBandDandiscategorizedas

SEM2011inthedata.AlthoughGroupCisnotthesubjectofthisresearch,thedatawas


availableandsomeobservationsandcommentsareincludedtosuggestopportunities

forfutureresearch.Simulationactivityforallgroupswasfacilitatedbythesame

instructor.

Ahighresponseratewasexpectedsincethiswasanin‐classcensusofsimulation

participants(Fowler,2009);however,theuseofapapersurveyallowedrespondentsto

skipquestionsornotprovidethepropercodetoallowmatched‐pairlongitudinal

analysisofthedata.Onehundredpercentofsimulationparticipantscompletedthe

presimulationsurvey.AllparticipantsinGroupsAandB(PDandOC2010)completed

thepostsimulationsurvey.OneGroupC(SEM2011)participantwasabsentonthelast

dayofsimulationanddidnotcompletethesurvey.OneGroupD(OC2011)participant

didnotcompleteasurvey.TwoGroupDparticipantsfailedtoentertheirunique

identifieronthepostsimulationsurveythuspreventinglongitudinalpairingand

renderingthepostsimulationsurveysuselessforthisstudy.Surveyresponserates

variedfrom79%to100%andarelistedinTable11.

Aninitialcheckofthedatatoverifyitsusabilityrevealed10instancesofasingle

missingdataelementwithinaparticipant’sattitudinalresponse,5inthepresimulation

surveyand5inthepostsimulationsurvey.FourofthesemissingelementsareinGroups

BandD,thetwogroupsunderstudy.With58attitudinalquestionspersurveyand60

and56respondentsforthepre‐andpost‐simulationsurvey,respectively;thisisan

overallmissingdatarateof.0014%forthepresimulationsurveyand.0015%forthe

postsimulationsurvey.Thereasonforincompletedatacanbeaccidentalordeliberate

foravarietyofreasons,buttheotherdatafromthatparticipantdoesn’thavetobe

ignored(Field,2009).Toavoidexcludingrespondentsfromtheanalysisbecauseof

these10casesofamissingdataelement,theaverageansweroftheotherrespondentsis

enteredforthemissingdata(Burns&Burns,2008;Fowler,2009).

Presimulation survey data review.Allparticipantscompletedthepresimulation

surveyinallfourcohortgroups.Inthetwocohortgroupsfromthegraduateprogram

understudy(GroupsBandD,OC2010andOC2011),oneGroupDrespondentdidnot

answeronequestion,anelementoftheteamperformance(TP1)variable.

Inthesingle‐courseacademiccohort(GroupC–SEM2011),oneindividualdid

notreplytothreeoftheattitudinalquestions,onequestioneachintheelementsofthe

knowledge(K1),confidence(C1)andteamperformance(TP1)variables.Asecond


individualinGroupD(SEM2011)didnotrespondtooneattitudinalquestion,anelement

ofthegenericsimulationperception(SG1)variable.

Inallfivecasesofamissingdataelementinthepresimulationsurvey,themean

oftheotherresponsesforthatquestionwassubstitutedforthemissingelementtoavoid

reducingthesamplesizeduetorandommissingdata(Burns&Burns,2008;Fowler,

2009).

Postsimulation survey data review.OneGroupA(PD)persondidnotcomplete

anyofthe17simulationattitudequestionsandthisindividualwasexcludedfromthe

matched‐pairt‐testpilotanalysisinvolvingthetwosimulationperceptionvariables.

InGroupB(OC2011),onemissing“confidenceintheabilitytoapplyproject

managementknowledge”questionresponsewasreplacedbythemeanoftheother

responsesforthatquestionandonepersonskippedallthenetworkandearnedvalue

analysisquestions.Sinceitisreasonabletoassumethatthislatterpersondidnotforget

howtosolvetheseproblemsbutchosetonotrepeatdoingso,theassociated

presimulationnetworkandearnedvalueanalysisscoreswerecopiedtotheassociated

postsimulationvariables.Thisassumesthepersonlearnednothingtoimprovethis

scoreduringthesimulation,apossiblereasonforchoosingtonotcompletethese

sectionsofthesurvey.

Randommissingdataelementswerereplacedwiththemeanforone

respondent’sknowledgeself‐assessmentquestioninGroupC(SEM2011)andGroupD

(OC2011),andonegenericsimulationattitudequestioninGroupD.OneGroupCand

twoGroupDrespondentsdidnotcompletethescheduleandearnedvalueanalysis

problems.Onceagain,theassumptionistheycanstillanswerthesetotheextentthey

didbeforethesimulationandtheirpresimulationresponsesarecopiedintothe

associatedpostsimulationdata.

Table11summarizestheresultsofthedatareviewandtheactionstaken.Table

12andTable13summarizethemissingdatavaluesanalysisbyvariable.


TABLE11‐DATAREVIEWSUMMARY

Group

Presimulation Survey Postsimulation Survey

Responses (rate)

Missing Data Treatment

Usable Responses(rate) Missing Data Treatment

A (PD) 19

(100%) None n/a

19 (100%)

1 person skipped all 17 simulation questions

Omitted from matched‐pair t‐tests for simulation perceptions (SG and SP)

B (OC2010) 14

(100%) None n/a

14 (100%)

1 person skipped network and earned value questions

Include in analysis copying presimulation score of 6

1 – C2 Replace missing data element with mean of other responses

C (SEM2011) 13

(100%)

1 – K1 1 – C1 1 – TP1 1 – SG1

Replaced missing data element with mean of other responses

12 (92%)

1 – K2 1 – TP2

1 person skipped network and earned value questions

Include in analysis copying presimulation score of 5

D (OC2011) 14

(100%) 1 – TP1

11 (79%)

1 – K2 1 – SG2

Replace with mean of other responses

2 people skipped network and earned value questions

Include in analysis copying presimulation scores of 1 and 2

TABLE12‐PRESIMULATIONQUESTIONRESPONSERATEBYVARIABLE

Variable Questions

Number of Possible Responses

Number of

Missing Values

Percent Missing Values

Knowledge Self‐Assessment (K1)

1A‐1F (16) 960 1 .001

Knowledge Application Confidence (C1)

2A‐2F (16) 960 1 .001

Current Team Satisfaction – McCreery (T1)

3A‐3I (9) 540 2 .004

Current Team Satisfaction (TP1)

3A‐3I, 4N (10) 600 2 .003

Teamwork Attitude (TG1) 4C, 4K (2) 120 0 0

Generic Simulation Attitude (SG1)

4A, 4B, 4E, 4G, 4I, 4L (6) 360 1 .003

This Simulation Attitude (SP1)

4D, 4F, 4H, 4J, 4M, 4O, 4P, 4Q (8) 480 0 0


TABLE13‐POSTSIMULATIONQUESTIONRESPONSERATEBYVARIABLE

Variable Questions

Number of Possible Responses

Number of

Missing Values

Percent Missing Values

Knowledge Self‐Assessment (K2)

1A‐1F (16) 896 2 .002

Knowledge Application Confidence (C2)

2A‐2F (16) 896 1 .001

Current Team Satisfaction – McCreery (T2)

3A‐3I 504 1 .002

Current Team Satisfaction (TP2)

3A‐3I, 4N (10) 559 1 .002

Teamwork Attitude (TG2) 4C, 4K (2) 110 0 0

Generic Simulation Attitude (SG2)

4A, 4B, 4E, 4G, 4I, 4L (6) 330 1 .003

This Simulation Attitude (SP2)

4D, 4F, 4H, 4J, 4M, 4O, 4P, 4Q (8) 440 0 0

Verification of assumption of parametric data

Normal distribution.ResultsoftheKolmogorov‐SmirnovandShapiro‐Wilktest

fordistributionofdataareshownforthecombineddatasetofGroupsBandD(OC2010

andOC2011),thetwogroupsinthegraduateprogramunderstudy,inTable14andfor

thesplitdatasetinTable15.Bothtestsindicatethedataareassumedtobenormalfor

theK1‐K2andC1‐C2matchedpairt‐testsforasignificantincreaseinperceived

knowledgeandabilitytoapplythisknowledge.Forexample,forthepresimulationself‐

perceptionofknowledge(K1),D(25)=.069,ns(p=.200).

Inthecombineddataset,atleastoneofeachoftheotherpre‐postsimulation

test‐pairingvariablesissignificantlynon‐normal,p<.05,andisindicatedbyhighlight

shading.

Thesplitdatasetresultsindicatethatifthecohortgroupsareconsidered

independently,accordingtotheKolmogorov‐Smirnovtest,theadditionalpairingsofT1‐

T2,TP1‐TP2,SG1‐SG2,SP1‐SP2andN1‐N2areassumedtousenormaldata,p<.05.

WiththeexceptionofSG2(p=.038),theShapiro‐Wilktestalsoindicatesthedatafor

thesevariablesmaybeassumedtobenormal,p<.05.


TABLE14‐COMBINEDGROUPSBANDDDATASETTESTSOFNORMALITY

Tests of Normality

OC Kolmogorov‐Smirnov Shapiro‐Wilk

Statistic df Sig. Statistic df Sig.

K1 .069 25 .200 .981 25 .911 C1 .116 25 .200 .967 25 .581 T1 .141 25 .200 .926 25 .069 TP1 .173 25 .052 .925 25 .068 TG1 .160 25 .097 .930 25 .086 SG1 .153 25 .132 .925 25 .066 SP1 .199 25 .012 .930 25 .089 N1 .218 25 .003 .815 25 .000 E1 .205 25 .008 .889 25 .011 P1 .211 25 .005 .894 25 .013

K2 .135 25 .200 .955 25 .320 C2 .165 25 .076 .939 25 .141 T2 .135 25 .200 .903 25 .022 TP2 .140 25 .200 .901 25 .019 TG2 .177 25 .042 .916 25 .042 SG2 .163 25 .086 .884 25 .009 SP2 .139 25 .200 .954 25 .306 N2 .170 25 .059 .888 25 .010 E2 .165 25 .076 .922 25 .056 P2 .170 25 .060 .944 25 .178


TABLE15‐TESTSOFNORMALITYSPLITBYCOHORTGROUP

Tests of Normality

Cohort

Kolmogorov‐Smirnova Shapiro‐Wilk

Statistic df Sig. Statistic df Sig.

K1 OC2010 .153 14 .200 .933 14 .334

OC2011 .147 11 .200 .938 11 .492

C1 OC2010 .108 14 .200 .983 14 .989

OC2011 .232 11 .102 .904 11 .206

T1 OC2010 .183 14 .200 .893 14 .089

OC2011 .164 11 .200 .949 11 .633

TP1 OC2010 .180 14 .200 .902 14 .120

OC2011 .188 11 .200 .930 11 .411

TG1 OC2010 .249 14 .019 .887 14 .073

OC2011 .177 11 .200 .930 11 .413

SG1 OC2010 .176 14 .200 .890 14 .081

OC2011 .248 11 .057 .886 11 .125

SP1 OC2010 .203 14 .122 .923 14 .242

OC2011 .132 11 .200 .942 11 .547

N1 OC2010 .214 14 .081 .830 14 .012

OC2011 .230 11 .106 .822 11 .018

E1 OC2010 .252 14 .016 .871 14 .044

OC2011 .245 11 .065 .811 11 .013

P1 OC2010 .269 14 .007 .843 14 .018

OC2011 .190 11 .200 .921 11 .325

K2 OC2010 .149 14 .200 .914 14 .178

OC2011 .127 11 .200 .951 11 .653

C2 OC2010 .191 14 .180 .943 14 .464

OC2011 .194 11 .200 .903 11 .203

T2 OC2010 .182 14 .200 .910 14 .159

OC2011 .175 11 .200 .892 11 .146

TP2 OC2010 .192 14 .171 .896 14 .099

OC2011 .186 11 .200 .895 11 .161

TG2 OC2010 .157 14 .200 .914 14 .177

OC2011 .249 11 .056 .872 11 .081

SG2 OC2010 .152 14 .200 .926 14 .267

OC2011 .215 11 .166 .846 11 .038

SP2 OC2010 .115 14 .200 .943 14 .456

OC2011 .206 11 .200 .941 11 .538

N2 OC2010 .152 14 .200 .916 14 .190

OC2011 .226 11 .123 .864 11 .064

E2 OC2010 .232 14 .040 .891 14 .084

OC2011 .148 11 .200 .946 11 .590

P2 OC2010 .160 14 .200 .923 14 .247

OC2011 .209 11 .195 .913 11 .268


Homogeneity of the cohort groups.InspectionofTable16showsthattheself‐

perceptionscoresare,onaverage,slightlyhigherforGroupB(OC2010)thanforGroupD

(OC2011).Table17reportstheresultsofLevene’stestforequalityofvariancesandt‐

testsfortheequalityofmeans.

Forallvariablesinthiscombineddataset,thevariancesarefoundtobeequal.

Forexample,theperceptionofknowledgevariableK1,F(1,26)=1.66,ns(p=.209).

Forallvariables,exceptpresimulationSG1andSP1,therewerenosignificant

differencesbetweenthemeans,p>.05.ForSG1,onaverage,studentsinGroupBhada

morefavorableviewofgenericsimulationsasalearningtool(M=5.94,SE=0.228)than

thoseinGroupD(M=4.71,SE=0.339).Thisdifferencewassignificantt(26)=3.003,p<

.05.ForSP1,onaverage,studentsinGroupBalsohadamorefavorableviewofthe

simulationusedinthecourse(M=5.78,SE=0.194)thanstudentsinGroupD(M=4.96,

SE=0.291).Thisdifferencewassignificantt(26)=2.346,p<.05.

Correspondingsignificantdifferencesarenotfoundfortherelated

postsimulationvariables,SG2andSP2.Followingthesimulation,onaverage,studentsin

GroupBstillhadamorefavorableviewofgenericsimulationsasalearningtool(M=

5.48,SE=0.222)thanthoseinGroupD(M=5.01,SE=0.422).Thisdifferencewasnot

significantt(23)=1.050,p>.05.ForSP2,onaverage,studentsinGroupBalsohada

morefavorableviewofthesimulationusedinthecourse(M=5.47,SE=0.203)than

studentsinGroupD(M=5.02,SE=0.388).Thisdifferencewasnotsignificantt(23)=

1.094,p>.05.


TABLE16–COHORTGROUPMEANSCORES

Group Statistics

Cohort N Mean

Std. Deviation

Std. Error Mean

K1 OC2010 14 5.29 .577 .154

OC2011 14 4.78 .756 .202

C1 OC2010 14 4.96 .710 .190

OC2011 14 4.68 .902 .241

T1 OC2010 14 5.52 1.292 .345

OC2011 14 5.46 1.101 .294

TP1 OC2010 14 5.52 1.293 .346

OC2011 14 5.44 1.115 .298

TG1 OC2010 14 5.93 .829 .221

OC2011 14 5.39 1.003 .268

SG1 OC2010 14 5.94 .854 .228

OC2011 14 4.71 1.267 .339

SP1 OC2010 14 5.78 .728 .194

OC2011 14 4.96 1.089 .291

N1 OC2010 14 2.21 1.672 .447

OC2011 14 2.14 1.512 .404

E1 OC2010 14 2.00 1.468 .392

OC2011 14 1.36 1.737 .464

P1 OC2010 14 4.21 2.424 .648

OC2011 14 3.50 2.710 .724

Years of experience with projects

OC2010 14 5.25 6.296 1.683

OC2011 14 8.86 8.347 2.231

Years working as a professional

OC2010 14 10.75 7.552 2.018

OC2011 14 14.57 7.939 2.122

K2 OC2010 14 5.62 .661 .177

OC2011 11 5.35 .661 .199

C2 OC2010 14 5.46 .839 .224

OC2011 11 5.33 .746 .225

T2 OC2010 14 5.87 .997 .266

OC2011 11 5.51 1.379 .416

TP2 OC2010 14 5.83 1.074 .287

OC2011 11 5.48 1.423 .429

TG2 OC2010 14 6.07 .703 .188

OC2011 11 5.95 .850 .256

SG2 OC2010 14 5.48 .832 .222

OC2011 11 5.01 1.399 .422

SP2 OC2010 14 5.47 .758 .203

OC2011 11 5.02 1.287 .388

N2 OC2010 14 2.21 1.369 .366

OC2011 11 2.36 1.502 .453

E2 OC2010 14 2.07 1.639 .438

OC2011 11 2.55 1.968 .593

P2 OC2010 14 4.29 2.016 .539

OC2011 11 4.91 2.773 .836


TABLE17‐RESULTSOFINDEPENDENTSAMPLEST‐TESTBETWEENCOHORTGROUPS

Independent Samples Test

Levene's Test for Equality of Variances t‐test for Equality of Means

F Sig. t df Sig. (2‐tailed)

Mean Differ‐ence

Std. Error Differ‐ence

95% Confidence Interval of the Difference

Lower Upper

K1 Equal variances assumed

1.660 .209 2.019 26 .054 .513 .254 ‐.009 1.036

Equal variances not assumed

2.019 24.310 .055 .513 .254 ‐.011 1.038

C1 Equal variances assumed

1.873 .183 .902 26 .375 .277 .307 ‐.354 .907


.902 24.634 .376 .277 .307 ‐.356 .909

T1 Equal variances assumed

.893 .353 .143 26 .888 .065 .454 ‐.868 .997


.143 25.362 .888 .065 .454 ‐.869 .998

TP1 Equal variances assumed

.541 .469 .175 26 .863 .080 .456 ‐.859 1.018


.175 25.451 .863 .080 .456 ‐.860 1.019

TG1 Equal variances assumed

1.168 .290 1.540 26 .136 .536 .348 ‐.179 1.251


1.540 25.104 .136 .536 .348 ‐.180 1.252

SG1 Equal variances assumed

2.109 .158 3.003 26 .006 1.226 .408 .387 2.065


3.003 22.793 .006 1.226 .408 .381 2.071

SP1 Equal variances assumed

3.372 .078 2.346 26 .027 .821 .350 .102 1.541


2.346 22.673 .028 .821 .350 .097 1.546

N1 Equal variances assumed

.640 .431 .119 26 .907 .071 .603 ‐1.167 1.310


.119 25.740 .907 .071 .603 ‐1.168 1.311

E1 Equal variances assumed

.222 .641 1.058 26 .300 .643 .608 ‐.606 1.892


1.058 25.296 .300 .643 .608 ‐.608 1.894

P1 Equal variances assumed

.059 .810 .735 26 .469 .714 .972 ‐1.283 2.712


.735 25.681 .469 .714 .972 ‐1.284 2.713

Yrs of exper. with projects

Equal variances assumed

1.679 .206 ‐1.291

26 .208 ‐3.607 2.794 ‐9.351 2.137


‐

1.291 24.176 .209 ‐3.607 2.794 ‐9.372 2.158


Independent Samples Test

Levene's Test for Equality of Variances t‐test for Equality of Means

F Sig. t df Sig. (2‐tailed)

Mean Differ‐ence

Std. Error Differ‐ence


Lower Upper

Years working as a prfssnl

Equal variances assumed

.120 .732 ‐1.305

26 .203 ‐3.821 2.928 ‐9.841 2.198


‐

1.305 25.935 .203 ‐3.821 2.928 ‐9.842 2.199

K2 Equal variances assumed

.001 .978 1.023 23 .317 .272 .266 ‐.278 .823


1.023 21.621 .318 .272 .266 ‐.280 .825

C2 Equal variances assumed

.187 .670 .410 23 .685 .132 .322 ‐.535 .799


.416 22.589 .681 .132 .318 ‐.526 .790

T2 Equal variances assumed

.639 .432 .775 23 .446 .368 .475 ‐.614 1.350


.745 17.608 .466 .368 .494 ‐.671 1.407

TP2 Equal variances assumed

.544 .468 .695 23 .494 .347 .499 ‐.685 1.379


.672 18.160 .510 .347 .516 ‐.737 1.431

TG2 Equal variances assumed

.026 .872 .376 23 .710 .117 .310 ‐.525 .759


.368 19.342 .717 .117 .318 ‐.548 .781

SG2 Equal variances assumed

3.113 .091 1.050 23 .305 .472 .449 ‐.457 1.400


.989 15.419 .338 .472 .477 ‐.542 1.485

SP2 Equal variances assumed

3.711 .066 1.094 23 .285 .450 .412 ‐.401 1.302


1.029 15.324 .319 .450 .438 ‐.481 1.382

N2 Equal variances assumed

.528 .475 ‐.260 23 .798 ‐.149 .575 ‐1.340 1.041


‐.257 20.575 .800 ‐.149 .582 ‐1.361 1.063

E2 Equal variances assumed

.683 .417 ‐.657 23 .517 ‐.474 .721 ‐1.966 1.018


‐.643 19.432 .528 ‐.474 .738 ‐2.015 1.067

P2 Equal variances assumed

3.845 .062 ‐.651 23 .521 ‐.623 .957 ‐2.603 1.356


‐.627 17.686 .539 ‐.623 .995 ‐2.716 1.469


Scale Reliability

Scalereliabilityisassessedusingtheinternalconsistencymethodby

determiningCronbach’sαforthequestionscomposingeachvariable.Underlying

statisticsreviewedforeachscaleincludetheCorrectedItem‐TotalCorrelationand

Cronbach’sAlphaifItemDeletedreportedintheItem‐TotalStatisticsTable.Corrected

Item‐TotalCorrelationvalueslessthan.3indicateaweakcorrelationwiththeoverall

scaleandsuggestthatitembedroppedfromthescale.ItemswithCronbach’sAlphaif

ItemDeletedvaluessubstantiallygreaterthantheoverallαsuggestthatitembedeleted

toimprovetheoverallscalereliability(Field,2009).

Table18andTable19showtheSPSSoutputfromthesecalculationsfortheK1

variable.AllCorrectedItem‐TotalCorrelationvaluesaresubstantiallygreaterthan.3and

allCronbach’sAlphaifItemDeletedvaluesarelessthantheoverallCronbach’sAlphaof

.924,indicatingallitemsarerelevantintheconstructbeingmeasured.Thesame

inspectionprocesswasfollowedfortheothervariables.Scalereliabilityforthevariables

usedinthisstudyissummarizedinTable20.

TABLE18‐SCALEK1‐RELIABILITYSTATISTICS

Reliability Statistics

Cronbach's

Alpha

Cronbach's

Alpha Based on

Standardized

Items N of Items

.924 .928 16


TABLE19‐SCALEK1‐PRESIMULATIONKNOWLEDGEITEM‐TOTALSTATISTICS

Item-Total Statistics

Scale Mean

if Item Deleted

Scale Variance if

Item Deleted

Corrected Item-Total Correlation

Squared Multiple

Correlation

Cronbach's Alpha if Item

Deleted

K1_ORGDESGN 75.86 113.534 .727 .727 .917

K1_PROJPLAN 75.25 111.750 .830 .855 .914

K1_PROJRISK 75.64 113.720 .545 .722 .922

K1_ESTSCOPE 75.50 112.111 .650 .760 .919

K1_SEQACT 74.93 115.106 .668 .668 .919

K1_ESTDUR 75.43 116.180 .680 .736 .919

K1_BUDGETNG 75.82 108.967 .758 .839 .915

K1_RESALLOC 75.25 109.083 .831 .898 .913

K1_CHGMGT 75.64 113.794 .622 .650 .920

K1_DESPERFMEAS 75.96 112.036 .579 .852 .922

K1_EFFLDRSHP 75.21 120.101 .581 .797 .921

K1_TEAMCONS 75.04 116.925 .566 .832 .921

K1_NEGOT 75.39 116.766 .542 .746 .922

K1_EVM 76.18 109.411 .608 .759 .921

K1_EVALPERS 75.18 115.560 .565 .830 .921

K1_MNGUNCERT 75.75 119.231 .503 .665 .923

TABLE20‐SCALERELIABILITY‐GROUPSBANDD(OC2010ANDOC2011)

Cronbach's α


(Pre‐sim.) 2

(Post‐sim.)



T Teamwork attitude (McCreery) .927 .946


TG Teamwork attitude (generic) .562 .672

SG Generic simulation attitude .928 .939

SP This simulation attitude .840 .869

ReviewoftheCronbach'sαscalereliabilityscoresindicatepotentialissueswith

thevariablesTGandSP.


TGisagenericteamworkattitudevariablecomprisedofthemeanoftwo

questionsbasedonBuzzetto‐MoreandMitchell’s(2009)postsimulationsurvey:

GroupProjectshelppreparestudentstobeabletoworkinprofessional

groupsinthefuture

Ienjoyworkingingroups

Thesearetheonlyquestionsinthesurveyaddressingteamsandteamworkthat

arenotspecifictotheteamofparticipantsperformingthesimulation.Otherthan

evaluatingeachitemindividually,thisisnottreatableandthegenericteamworkattitude

scaleremainswithrelativelylowreliability.

Cronbach’sαforSP1canbeimprovedfrom.840to.943andSP2canbe

improvedfrom.869to.925byeliminatingthequestions“Ibelievethissimulationwillbe

difficult”and“Thissimulationwasdifficult”fromthepresimulationandpostsimulation

surveys,respectively.TheCorrectedItem‐TotalCorrelationfortheseitemswere

‐.253and‐.026,respectively.Accordingly,anadditionalanalysisisperformedwiththis

itemremovedfromtheSPscaleandconsideredindependently.

McCreery’s(2003)teamworkassessmentvariables(T1andT2)are

strengthenedbytheadditionofthecurrentteamquestionfromBuzzetto‐Moreand

Mitchell’s(2009)survey,“Iwouldbewillingtoworkwithmyteamonanactualproject

inthefuture.”Consequently,theconstructTPwillbeusedratherthanTforassessing

teamworkattitudesrelatedtotheparticipant’ssimulationteam.

ThescalereliabilitysummarywiththisrevisedconstructstructureforSPis

showninTable21.


TABLE21‐REVISEDSCALERELIABILITY‐GROUPSBANDD(OC2010ANDOC2011)

Cronbach's α


(Pre‐sim.) 2

(Post‐sim.)



T Teamwork attitude (McCreery) .927 .946


TG Teamwork attitude (generic) .562 .672

SG Generic simulation attitude .928 .939 SP‐D This simulation attitude minus

difficulty question .943 .925

Withtheexceptionofthepreviouslydiscussedgenericteamworkattitude

construct(TG)basedononlytwoquestions,theresultsofCronbach’sαanalysissupport

theconclusionthatthedefinedsurveymeasuresallhavehighreliabilities,Cronbach’sα

>.9(Burns&Burns,2008).

Participant demographics

StudentsareassignedtoteamswhentheybegintheProjectManagementCore

Phaseofthegraduateprogram.Theseassignmentsaremadewiththeintentto

maximizediversityontheteamswithregardstogender,workexperience,ethnicity,and

industry.Inadditiontodemographicdatarelatedtothesecharacteristics,demographic

datawascollectedinthepre‐andpost‐simulationsurveysregardingprofessional

background(technicalornon‐technical),program‐startingeducationallevel,prior

projectmanagementtraining/education,andannualincometofurtherdeterminethe

diversityoftheparticipants.Theintentofthisresearchisnottoexplorecorrelations

withthesedemographiccategories;however,thedataprovidesforthispossiblefuture

research.

Gender and technical background.Thetwocohortgroupsunderstudycontained

identicalgender(64%male/36%female)andtechnical/non‐technicalbackground

(57%technical/43%non‐technical)distributions.Allteamshadatleastonewoman

(seeFigure6)andamixoftechnicalandnon‐technicalbackgrounds(seeFigure7).


FIGURE6‐TEAMGENDERMIX

FIGURE7‐TEAMTECHNICAL/NON‐TECHNICALBACKGROUNDMIX

Educational background.Priortobeginningtheirgraduatestudies,60%ofthe

studentsineachcohorthadexperiencednoformalprojectmanagementtrainingor

education.Thosewithpriortrainingoreducationhadexperiencedmorethan26contact


hours,butonlyonepersonreportedmorethan125contacthours.Figure8depictsthis

forthetwocohortgroupsunderstudy.

Priorprojectmanagementtrainingandeducationwasnotoneofthefactors

consideredwhenassigningteamsatthestartoftheprogram;however,Figure9shows

theteamsarealsodiverseinthiscategory.Exploringrelationshipsbetweenteam

compositionandteamperformanceisanopportunityforfutureresearch.

FIGURE8‐CONTACTHOURSOFPRIORPROJECT

MANAGEMENTTRAININGOREDUCATION


FIGURE9‐PRIORTRAINING/EDUCATIONBYTEAM

Years of experience with projects and years of professional experience.Overall,

participantsreportedanaverageof7yearsofexperienceworkingwithprojectsand12.7

yearsworkingasaprofessional.TheexperiencedistributionsareshowninFigure10

andFigure11.

Onaverage,GroupB(OC2010)participantshad3.6feweryearsofexperience

withprojects(M=5.25,SE=1.68)thanGroupD(OC2011)participants(M=8.86,SE=

2.23).Thisdifferencewasnotsignificantt(26)=‐1.291,p>.05;however,itdid

representamediumsizeeffect,r=0.24.

Onaverage,GroupB(OC2010)participantshad3.8feweryearsofprofessional

workexperience(M=10.75,SE=2.02)thanGroupD(OC2011)participants(M=14.57,

SE=2.12).Thisdifferencewasalsonotsignificantt(26)=‐1.305,p>.05;r=0.25.


FIGURE10‐PROJECTEXPERIENCE

FIGURE11‐PROFESSIONALEXPERIENCE


Region of origin.Studentsoriginatedfrommultipleregionsoftheworld,but

primarilyfromNorthAmerica.Thisdemographicdatawascollectedonthe

postsimulationsurveyanddataismissingforfourGroupD(OC2011)students.The

mixtureofworldregionoforiginbycohortgroupandbyteamisshowninFigure12and

Figure13.

FIGURE12‐WORLDREGIONOFSTUDENTORIGIN


FIGURE13‐WORLDREGIONOFORIGINBYTEAM

Prior education level.Fourstudents,twoineachofthecohortgroupsreported

previouslyearningamaster’sdegree.Allotherstudentsenteredtheprogramwith

bachelor’sdegrees.

Industry and job title.Studentdiversityofworkindustrybycohortgroupis

showninFigure14andbyteaminFigure15;diversitybyjobtitleisshowninFigure16

andFigure17,respectively.Industrieswritteninforthe“other”categorywerefinance,

semiconductors,retail,technologyanddesign/architecture.Jobtitleswritteninforthe

“other”categorywereanalyst,accountmanager,teamleadandprojectdesigner.

Annual income.Studentdiversityinannualincomebycohortgroupisshownin

Figure18andbyteaminFigure19.


FIGURE14‐INDUSTRYBYCOHORTGROUP

FIGURE15‐INDUSTRYBYTEAM


FIGURE16‐JOBTITLEBYCOHORTGROUP

FIGURE17‐JOBTITLEBYTEAM


FIGURE18‐ANNUALINCOMEBYCOHORTGROUP

FIGURE19‐ANNUALINCOMEBYTEAM


Descriptive statistics

AppendixGcontainstablesshowingthedescriptivestatisticsforeachvariable

anditscomponentquestions.Thisdatawascollectedusing7‐pointLikert‐typescaleson

thesurveyinstrumentandtreatedasintervaldata.Higherscoresindicatehigherself‐

assessmentsofprojectmanagementknowledge,abilitytoapplythatknowledge,

strongerfavorableattitudesrelatedtoteamsandsimulations,ormorecorrectanswers

ontheabilitytoanalyzeschedulenetworksandearnedvalueperformance.

Hypothesis testing

Thissectionreportstheresultsofthequantitativeanalysisofthedatarelatedto

testingthehypothesesposedbytheresearchquestions.Testingofthesehypotheses

usesdependentt‐teststocomparetheresultsobtainedfromthepresimulationand

postsimulationsurveys.Theresultsofthesedependentt‐testsarepresentedinTable22

(descriptivestatistics),Table23(correlations)andTable24(testresults).Sinceweare

lookingforincreases,itisappropriatetouseone‐tailedsignificance,obtainedby

dividingthereportedtwo‐tailedsignificancebytwo.Effectsizesassociatedwiththese

testsarecalculatedusingtheformula,r=SQRT(t2/(t2+df).(Field,2009).

Do participant knowledge and confidence to apply knowledge self‐assessments

increase following the simulation experience?

H1: Participantswillassesstheirprojectmanagementknowledgelevelhigher

aftercompletingthesimulationgame.

Onaverage,participantsassessedsignificantlygreaterprojectmanagement

knowledgeaftercompletingthesimulation(M=5.5,SE=0.132)thanbeforestartingthe

simulation(M=5.01,SE=0.148),t(24)=4.942,p<.01,r=.71.

H2: Participantswillassesstheirabilitytoapplyprojectmanagement

knowledgehigheraftercompletingthesimulationgame.

Onaverage,participantsassessedsignificantlygreaterconfidenceintheirability

toapplyprojectmanagementknowledgeaftercompletingthesimulation(M=5.4,SE=


0.157)thanbeforestartingthesimulation(M=4.75,SE=0.162),t(24)=5.639,p<.01,r

=.75.

TABLE22‐PAIREDSAMPLESSTATISTICS

Paired Samples Statistics

Mean N

Std. Deviation

Std. Error Mean

Knowledge K2 5.50 25 .661 .132

K1 5.01 25 .739 .148

Confidence C2 5.40 25 .786 .157

C1 4.75 25 .810 .162

Team Experience (McCreery)

T2 5.71 25 1.168 .234

T1 5.39 25 1.188 .238

Team Experience (enhanced)

TP2 5.68 25 1.225 .245

TP1 5.38 25 1.199 .240

Teamwork (generic)

TG2 6.02 25 .757 .151

TG1 5.56 25 .939 .188

Generic Simulation Attitude

SG2 5.27 25 1.117 .223

SG1 5.35 25 1.187 .237

Project Simulation Attitude

SP2 5.28 25 1.026 .205

SP1 5.39 25 .955 .191

Project Simulation Attitude ‐ difficulty

SP2_D 5.51 25 1.156 .231

SP1_D 5.47 25 1.113 .223

Project Simulation Difficulty Question

S2_HARD 3.64 25 1.604 .321

S1_HARD 4.76 25 1.985 .397

Network Analysis Problem

N2 2.28 25 1.400 .280

N1 2.24 25 1.640 .328

Earned Value Analysis Problem

E2 2.28 25 1.768 .354

E1 1.80 25 1.658 .332

Overall Problem‐Solving Score

P2 4.56 25 2.347 .469

P1 4.04 25 2.638 .528


TABLE23‐PAIREDSAMPLESCORRELATIONS

Paired Samples Correlations

N Correlation Sig.

Knowledge K2 & K1 25 .752 .000

Confidence C2 & C1 25 .737 .000

Team Experience (McCreery) T2 & T1 25 .743 .000

Team Experience (enhanced) TP2 & TP1 25 .775 .000

Teamwork (generic) TG2 & TG1 25 .482 .015

Generic Simulation Attitude SG2 & SG1 25 .304 .140

Project Simulation Attitude SP2 & SP1 25 .426 .034


SP2_D & SP1_D

25 .442 .027


S2_HARD & S1_HARD

25 .417 .038

Network Analysis Problem N2 & N1 25 .714 .000

Earned Value Analysis Problem E2 & E1 25 .546 .005

Overall Problem‐Solving Score P2 & P1 25 .643 .001


TABLE24–PAIREDSAMPLETESTRESULTS

Paired Samples Test

Paired Differences

t df Sig. (2‐tailed) Mean

Std. Dev.

Std. Error Mean


Lower Upper

Knowledge K2 ‐ K1 .493 .499 .100 .287 .699 4.942 24 .000

Confidence C2 ‐ C1 .654 .580 .116 .414 .893 5.639 24 .000

Team Experience (McCreery)

T2 ‐ T1 .316 .845 .169 ‐.032 .665 1.872 24 .073

Team Experience (enhanced)

TP2 ‐ TP1 .297 .814 .163 ‐.039 .632 1.823 24 .081

Teamwork (generic) TG2 ‐ TG1 .460 .877 .175 .098 .822 2.623 24 .015

Generic Simulation Attitude

SG2 ‐ SG1 ‐.079 1.360 .272 ‐.640 .483 ‐.289 24 .775

Project Simulation Attitude

SP2 ‐ SP1 ‐.110 1.063 .213 ‐.549 .329 ‐.517 24 .610


SP2_D ‐ SP1_D

.034 1.199 .240 ‐.461 .529 .143 24 .888


S2_HARD ‐ S1_HARD

‐1.120 1.965 .393 ‐1.931 ‐.309 ‐2.850 24 .009

Network Analysis Problem

N2 ‐ N1 .040 1.172 .234 ‐.444 .524 .171 24 .866

Earned Value Analysis Problem

E2 ‐ E1 .480 1.636 .327 ‐.195 1.155 1.467 24 .155

Overall Problem‐Solving Score

P2 ‐ P1 .520 2.124 .425 ‐.357 1.397 1.224 24 .233

How do participant opinions regarding their team experience in the program and

group work in general change as a result of this simulation experience?

H3: Participantswillreportahigheropinionoftheirteamexperienceafter


SimilarresultswereobservedusingboththeconstructsbasedonMcCreey’s

(2003)teamattitudequestions(T1andT2)andtheenhancedconstructsaddingthe

teamworkquestion(TP1andTP2)fromBuzzetto‐MoreandMitchell’s(2009)survey.

UsingtheconstructbasedonMcCreey’s(2003)teamattitudequestions,on

average,participantsreportsignificantlygreatersatisfactionwiththeirteamexperience


intheprogramaftercompletingthesimulation(M=5.71SE=0.234)thanbefore

startingthesimulation(M=5.39,SE=0.238),t(24)=1.872,p<.05,single‐tailed,r=.36.

UsingtheenhancedteamexperienceconstructbasedonMcCreey’s(2003)team

attitudequestionsandBuzzetto‐MoreandMitchell’s(2009)teamexperiencequestion,

onaverage,participantsreportedsignificantlygreatersatisfactionwiththeirteam

experienceintheprogramaftercompletingthesimulation(M=5.68SE=0.245)than

beforestartingthesimulation(M=5.38,SE=0.240),t(24)=1.823,p<.05,single‐tailed,

r=.35.

H4: Participantswillreportahigheropinionofgroupprocessesingeneralafter


Onaverage,participantsassessedsignificantlygreaterpositiveopinionongroup

workingeneralaftercompletingthesimulation(M=6.02,SE=0.151)thanbefore

startingthesimulation(M=5.56,SE=0.188),t(24)=2.623,p<.01,singletailed,r=.47.

How do participant opinions regarding the use of simulations as a learning tool

change as a result of this simulation experience?

H5: Participantswillreportahigheropinionontheuseofsimulationsasa

learningtoolaftercompletingthesimulationgame.

Onaverage,therewasnosignificantchangeofopinionontheuseofsimulations

asalearningtoolaftercompletingthesimulation(M=5.27,SE=0.223)thanbefore

startingthesimulation(M=5.35,SE=0.237),t(24)=‐.289,p>.05,r=.06.

H6: ParticipantswillreportahigheropinionofthesimProjectsimulationafter


Onaverage,therewasnosignificantchangeofopinionontheprojectsimulation

gameusedinthecourseaftercompletingthesimulation(M=5.51,SE=0.231)than

beforestartingthesimulation(M=5.47,SE=0.223),t(24)=‐.143,p>.05,r=.03.


Thesemeansscoresareslightlyhigherthanthoseobtainedwiththeoriginal

constructthatincludedtheopinionquestiononsimulationdifficulty.Analysisusingthe

initialconstructalsoindicatedtherewasnosignificantchangeofopinionontheproject

simulationgameusedinthecourseaftercompletingthesimulation(M=5.28,SE=

0.205)thanbeforestartingthesimulation(M=5.39,SE=0.191),t(24)=‐.517,p>.11,r

=.06.

Consideringthesinglequestionregardingperceiveddifficultyofthesimulation,

therewasasignificantdecreaseintheperceptionofhowdifficultthesimulationwas

aftercompletingthesimulation(M=3.64,SE=0.321)thanhowdifficultthesimulation

wasgoingtobebeforestartingthesimulation(M=4.76,SE=0.397),t(24)=‐2.850,p<

.01,r=.50.

Are students better able to develop and interpret schedule network diagrams and

earned value data as a result of the simulation experience?

H7: Participantswillbebetterabletosolveprojectscheduleandearnedvalue

analysisproblemsaftercompletingthesimulationgame.

Onaverage,therewasnosignificantchangeinabilitytoanswertheschedule

networkanalysisquestionsaftercompletingthesimulation(M=2.28,SE=0.280)than

beforestartingthesimulation(M=2.24,SE=0.328),t(24)=.171,p>.05,r=.03.

Onaverage,therewasnosignificantchangeinabilitytoanswertheearnedvalue

analysisquestionsaftercompletingthesimulation(M=2.28,SE=0.354)thanbefore

startingthesimulation(M=1.80,SE=0.332),t(24)=1.467,p>.05,r=.29

Therewasnosignificantchangeintheoverallproblem‐solvingscoreafter

completingthesimulation(M=4.56,SE=0.469)thanbeforestartingthesimulation(M=

4.04,SE=0.528),t(24)=1.224,p>.05,r=.24

Qualitative Results

Thissectionsummarizesthequalitativedatacollectedfromthesimulationgame

participantsduringtheirdebriefpresentationanddiscussions,fromtheirpostsimulation

surveyqualitativequestionresponses,andfromcourseandprogramevaluationsurveys.

Postsimulation team presentations.Teamstendedtoincludegenericlessons

learnedaspartoftheirpresentationoftheproject‐specificauditandalsoincluded


simulation‐specificobservationsduringtheiranswertowhattheylearnedaboutproject

managementingeneral.Accordingly,bothsectionsoftheteampresentationswere

analyzedforsalientthemesrelatingtothepracticeofprojectmanagementinthereal

world.Projectresourcemanagementwascommentedonthemost,followedby

teamworkandmonitoring&control.

Theme 1: Resource management is critical.Itisimportanttostartwith

appropriateandcapableresourcesandtohavethemhiredwhenneeded.It’salso

importanttobesureresourcesareassignedtoalltasks;otherwise,lesscapable

resourcesmaybeassignedtocompletethetasktokeeptheprojectmovingandthiscan

betime‐consumingandexpensive.Donotsecuremoreresourcesthanareneededas

thistoocanbeexpensivewhentheyarepaidforidletimeorassignedtotasksthatare

incompatiblewiththeirbackground.Multipleresourcecross‐functionalassignmentson

ataskcanbeeffectiveiftherightskilltypesandpersonalitiesarecombined.The

converseisalsotrue;assigninganadditionalresourcewithoutappropriateskillstoa

taskwillnotproducethedesiredschedulecompressionandwillalsonegativelyaffect

cost.

Theme 2: Teamwork is important. Presenterscommentedontheneedforthe

studentteamstoapplytheirsocioculturalskillstoworktogetheronplanningforthe

simulationandplayingthegame,especiallywhenworkingthroughthestressofthe

unknown.Theneedforteammemberinvolvement,planningaheadandcommunication

wasstressed.Theuseofvisualaidsduringteamworkingsessionswasfoundtobe

useful.Thesimulationgamehighlightedtheimportanceofconsensusbuildinginteam

decision‐making;italsodemonstratedthefrustrationthatoccurswhenconsensusis

slowly,ornot,reached.Inthesecasesitishelpfultohaveadesignatedteamleaderwho

considersalltheopinions,makesadecisionandmovestheteamon;however,thiscan

leaveteammembersfeelingdisenfranchisedandunmotivatedforfutureperformance

andtheteamleadermustbeawareofthisandactaccordinglytominimizethiseffect.

Theme 3: Monitor results and adapt as needed. Thesimulationreinforcedthe

valueofplanningaheadandadaptingtoanewstrategyifthecurrentonefails.Havinga

planinthebeginningprovidesabaselinetomeasureperformanceagainst.Change

controlsareneeded.It’simportanttomonitortrendsrelatedtothebaselineandto

differentiatebetweenessentialandinessentialdata.Ittakesjustonewrongdecision

duringaphasetohugelyimpactthesimulation;therefore,itisimportanttoanalyze


decisionsineveryphaseandmakeadjustmentswhenevernecessary.Periodresults

needtobereviewedthoroughlyandusedasthebasisforfuturedecisions.Itiseasierto

shortentheschedulethanreducecost.

Postsimulation survey open‐ended question responses. Thepostsimulation

surveyaskedparticipantstorespondtofouropen‐endedquestions:

1. Whatdidyoulikeaboutthejust‐completedsimulationexperience?

2. Excludingchangestothecomputersimulationitself,whatchangesdoyou

thinkcouldbemadetotheoverallsimulationexperiencetobetter

prepareyoutomanageprojects?

3. Whatchangesdoyourecommendbemadetothecomputersimulation

itself?

4. Elaborateonanyotherthoughtsyouhaveregardinglearningproject

managementwithacomputer‐basedsimulationgame.

Answerstoeachquestionwerecategorizedandanalyzedtoidentifyemergent

themes.

What they liked about the just completed simulation experience. Resource

managementreceivedthemostcommentsfollowedbynear‐equalnumbersofcomments

relatedtocompetition,planning,teamworkandanalysis/tools.Lessfrequently

mentionedweretheaction‐orientedpracticeofprojectmanagementtechniques,the

abilitytoexperiencethewholeprocess,experiencingthepressureofbeingtheproject

manager,thedebriefdiscussions,decision‐makingpractice,andchangemanagement.

Resource Management. Elevenresourcemanagementcommentsrelatedtothe

needtoanalyzeeachresource’scharacteristics,assignthemappropriatelytotaskswhile

takingintoaccounttheirefficiencyanditsrelationshiptothebaselinescheduleand

budget,considerdiversityeffects,andtoobserveandrespondtotheirperformanceon

tasks.

Competition. Eightcompetitioncommentsincludedthefunofcompetingagainst

theotherteamsforresources,comparingperformanceresultsaftereachworkperiod,

andlearningfromeachotherduringtheprojectstatusmeetingswiththesponsorandat

theconclusionofthesimulation.

Planning. Eightparticipantsreportedlikingthepresimulationplanningexercise

topreparethebaselineschedulebasedonnominalresourcesandastrategyincluding

thestaffingmanagementplanandaccompanyingrevisionstothenominalbaseline.The


simulationgameexperienceemphasizedtheimportanceofplanningandhavingan

executionstrategy;studentsenjoyedseeinghowtheirstrategiesplayedoutduringthe

simulatedprojectexecution.

Teamwork. Eightparticipantscommentedfavorablyonworkingasateamtoplan

andmanagetheexecutionofthesimulatedproject.Inadditionto“greatteam

experience”andsimilarcomments,moreverbosecommentsonteamworkincluded“[it]

showedtheimportanceofteamworkinordertosucceedinaproject”and“Ialsoenjoyed

splittingthetaskswithmyteam.”

Analysis/Tools. Sevenparticipantsmentionedenjoyingusingthetoolsanddatato

analyzetheprojectresultsandusetheresultsofthisanalysistoguidedecisionmaking.

Onecommented,“ItmademerealizethatIneedtolearntointerpretthedataIam

presentedwith.”

Recommended changes to the experience excluding the simulation itself.

Recommendationsforchangestotheadministrationoftheexperiencegroupedintofour

categories:none(9),teamworktime(6),practice(6),andotherunrelatedsuggestions

(11).

None. Ratherthanwrite“none”orleavetheanswerblank,oneofthenine

respondentsinthiscategorywrote,“IfeelsohappyIgottoparticipateinthissimulation

asitallowedmetohavealittlebitofwhatrolesaprojectmanagerplaysandhowhis

decisionsaffectsthesuccessoftheproject.”

Teamwork time.Threeoftherespondentssuggestedmoretimeshouldbe

allowedfortheworkperiodsbetweenmilestones,onesuggestedlesstime,one

preferredfixedtimesthatdidn’tflextoincreasepressureontheteams,andone

preferredfewermilestoneswithoutsuggestingwhethermoretimeshouldbeallowedfor

theremainingworkperiods.

Practice.Fiverespondentssuggestedafewpracticeroundsbeplayedbefore

startingthegameandonesuggestedtheentiregamebereplayedto“seeifyoucan

improve.”

Other.Othersuggestionsweretohaveliveinstructionontopicsthatarecovered

intheuserguide(twomentions),assignanexperiencedchampiontoeachteam,conduct

moreprojectreviewswiththesponsor,requirespecificformatsforthein‐progress

projectreviewswiththesponsor,includemoreissues/events,compareperformance

againstthemodifiedteambaseline,andchangeteamassignmentsduringthesimulation.


Recommended changes to the simulation. Themostfrequentcommentregarding

changingthecomputersimulationitselfwas“none”ornoresponsebyninerespondents

followedbysixcommentseachinthecategoriesof“userguide”and“reporting.”Four

respondentscommentedontherulesregardingreleaseofresources,twoontheuseof

events,twoonresourceassignmentflexibility,andtherewerefiveotherunrelated

comments.

User guide.Userguidesuggestionsincludedmorescreenshotsofthesimulation

“sowearenotscramblingtolearnallthesoftwareinadditiontoinputtingproject

managementdecisions,”theadditionofjobdescriptionsofthesimulatedresource

categories,clarificationonthepartialassignmentofresources,morediscussiononthe

functionalitymetric,andmoreinformationingeneral.

Reporting.Reportingsuggestionswereforaneasierwaytoproducechartsand

graphs,aCostPerformanceIndex(CPI)report,trendreportsshowingpriorworkperiod

results,EarnedValueManagement(EVM)reports,separationoftrainingcostsfromtotal

projectcosts,andmorewrittenfeedback.

Rules.Fourrespondentssuggestedmodifyingtheresourcehiring/firingprotocol

toeliminatetheunrealisticpracticeofbiddingonmoreresourcesthanareneededand

allowingtheimmediateterminationofthebackupsiftheteamwonallthebids.

Events.Onerespondentwouldliketohaveseentheinclusionofmoreeventsand

onerespondentwouldhavepreferredteam‐specific“eventsbasedonindividualteam

dynamics.”

Other.Othercommentsrequestedawarningmessagetopreventdecision

submittalifnoresourcesareassignedtoatask,moreneedforresourcetraining,fewer

workperiods,adecisionsummaryscreenpriortosubmittal,andconsiderationofvirtual

teamdiversityincalculations(thislastsuggestionisbuiltintothesimulationand

discussedintheuserguide).

Other thoughts about learning project management with a computer‐based

simulation game.Responsesconcerningotherthoughtsgroupedintofourcategories:

favorable(11comments),noneornoresponse(6),unrealistic(2),andother(6).

Favorable.Favorablecommentsmadebyelevenoftherespondentsincluded:

“IfeltlikeIlearnedalot.Irealizedwhatdatawasmoreimportantthan

others.ThoughImayhavethoughtsomethingwasimportantearly,Iwas

forcedtoreevaluatewhatinfowasuseful.”


“Itprovidedanexcellentbaselineexperiencetocomponentsofproject

management.”

“Greatexperience;canrelatetoreallifePMexperience”

“Greatalternativeto‘reallife’projects.Someitemswereunrealisticbut

mostwererealistic.”

“Ihavelearntalotespeciallyinthehumanresourceareaofproject

management”

“Greatidea.Givesstudentsasclosetoa‘realworld’experienceasyou

can.”

Threeoftherespondentswithsimilarfavorablecommentsalsosuggested

playingmoresimulationgameswouldbebeneficial.

Unrealistic.Tworespondentscommentedthatthesimulationwasnotrealistic,

referringtothemannerinwhichresourcesarehiredandreleasedandtheabilityto

“game”thesystembybiddingonmoreresourcesthanneedingandimmediately

releasingonesthataren’tneeded:

“Itisnotreallyarealworldexperiencesinceinarealworldexperiencea

projectmanagercannothireandfirepeopleinthesameperiodiftheyare

notneeded.”

“Simulationwillneverbeanaccuratereflectionoftherealworld.There

willalwaysbeawayto‘game’thesystem.Iseenowayaroundthis

limitation.”

Other.Oftheothercomments,onerespondentwrote,“Iwouldratherhave

learnedonarealprojectwithrealpeople.”Tworespondentswouldhavepreferredto

workaloneonthesimulation,onefoundtheexperience“somewhatexhausting,”another

wroteonly“planahead,”andonecommentedthatseniorprojectmanagersmaynotget

asmuchbenefitfromthesimulationgameexperienceasjuniorprojectmanagers.

Course evaluation survey responses. Duetoaproceduralerror,nocourse

evaluationsurveywasissuedforthefirstcohortgroup.Thesecondcohortgroup

completedacourseevaluationsurveythatexplicitlyasked,“Whatcommentswouldyou

liketomakeaboutthesimulation?”Fiveofsixresponseswereveryfavorable,echoing

previouscommentsfromthepostsimulationsurveyresponsesandoneresponsealso

repeated“simulationisgoodforjr.projectmanager,butnotforsr.projectmanager”and

added,“shouldaddmorediscussionshowtolinksimulationtoreallifeproject.”


Program evaluation survey responses. Programevaluationsurveysissuedasthe

completionofthe21SCHProjectManagementCorePhaseoftheprogramincludethe

questions:

1. Ifyouaregiventheoptiontochangeonly3thingsintheProject

ManagementCoreProgram,whatwouldthose3thingsbe,andwhy

wouldyouchangethem?

2. Ifyouaregiventheoptiontoretainonly3thingsintheProject

ManagementCoreProgram,whatwouldthose3thingsbe,andwhy

wouldyoukeepthem?

Bothcohortgroupswereaskedtocompletethissurvey.Norespondentsto

question1mentionedchangingoreliminatingthesimulation.Fourof21respondents

mentionedretainingthesimulationinresponsetoquestion2.

Graduate program exit survey responses.StudentsgraduatingwithaMasterof

ScienceorMasterofBusinessAdministrationdegreeareaskedtocompleteasurvey

shortlyaftertheirgraduationtoprovidefeedbackontheiroverallgraduateeducation

experience.ResponsestothissurveytendtoconcentrateonthemorerecentBusiness

andMBACorePhasesoftheprogramratherthantheProjectManagementCorePhase.

Onthequestionofwhataspectsoftheprogramwillyourememberthemost,

respondentsalmostunanimouslycitedthetwoweekinternationalstudytripwithafew

favorablecommentsabouttheclassenvironmentandprofessors.

Respondingtothequestionofsuggestionsforimprovingtheprogram,one

respondentreactedfavorablytotheuseofsimulationsintheprogram,“Wehavedone2

simulationprojects,1inProjectManagementand1instrategicManagement.Iwould

liketoseemoresimulationprojectthroughouttheprogram.”Therewerenonegative

commentsaboutusingsimulations.

Course Evaluation Survey Quantitative Data

Beginningwiththesecondcohortgroup,thecourseendingsurveyforthecourse

withthecapstonesimulationgameaskedrespondentstoindicatetheiragreementwith

fivestatementsrelatedtothesimulation.Thequestionsandtheresponsesshownin

Table25indicateafavorableviewontheuseofthesimulationgameinthecourse.


TABLE25‐COURSEEVALUATIONRATINGS

OC2011 SA (5)

A (4)

N (3)

D (2)

SD (1)

Rating(mean)

The simulation increased my knowledge of fundamental project management principles

3 2 1 1 4

The simulation experience improved my ability to work with and lead project teams

5 0 1 1 4.3

My team worked together effectively on the simulation 4 3 4.6

I enjoyed the simulation experience 5 2 4.4

I recommend this simulation be kept in the program 5 1 1 4.6

Summary of Results

Table26summarizestheresultsofthequantitativehypothesistesting.H1

throughH4aresupportedindicatingstudentsreportedsignificantincreasesinproject

managementknowledge,abilitytoapplythatknowledge,teamexperienceattitude,and

attitudetowardsfuturegroupwork.H5andH6arenotsupportedindicatednochange

inattitudeaboutusingsimulationgamesasexperientiallearningactivities,eitherin

generalorforthissimulationexperienceinparticular.H7wasnotsupportedindicating

noincreaseinabilitytosolveprojectscheduleornetworkanalysisproblems.

Analysisofteampresentationanddebriefcommentsandnarrativeresponsesto

surveyquestionssupporttheconclusionthatthesimulationgamewaseffectiveand

addedvalue.Thisconclusionisreinforcedbyresponsestospecificcourseevaluation

ratingsrelatedtothesimulationgameexperience.


TABLE26‐SUMMARYOFRESULTS

Hypothesis Result Significance (one‐tailed)

H1: Participants will assess their project management knowledge level higher after completing the simulation game.

Supported P < .01

H2: Participants will assess their ability to apply project management knowledge higher after completing the simulation game.

Supported P < .01

H3: Participants will report a higher opinion of their team experience after completing the simulation game.

Supported P < .05

H4: Participants will report a higher opinion of group processes in general after completing the simulation game.

Supported P < .01

H5: Participants will report a higher opinion on the use of simulations as a learning tool after completing the simulation game.

Not supported ns, p = .388

H6: Participants will report a higher opinion of the simProject simulation after completing the simulation game.


H7: Participants will be better able to solve project schedule and earned value analysis problems after completing the simulation game.



Discussion

Thischapterdiscussestheresultsofthisstudyintothevalueandeffectivenessof

usingaprojectmanagementsimulationgameasacapstoneexperientiallearningactivity

inagraduateexecuteMS/MBAdegreeprogramwithinemphasisinproject

management.ThechapterbeginswithadiscussionofKolb’s(1984)perspectiveson

experientiallearninginthecontextofusingtheSimProjectprojectmanagement

simulationgameasanexperientiallearningactivity.Thisisfollowedbyadiscussionof

theresultsandacontrastwithpriorsimulationgamingresearch.Thechapterconcludes

withadiscussionofthelimitationsofthisstudyandasummaryofthefindings.The

implicationsofthisresearchandrecommendationsforfutureresearcharediscussedin

thenextchapter.

Experiential Learning and the Project Management Simulation

FavorablereactiontoplayingSimProjectisconsistentwithKolb’s(1984)finding

thatadultlearnerspreferexperientiallearningmethodsthatallowthemtotestthe

relevanceandapplicationofideasinacontextthatfacilitatescomparisonwiththeir

“ownaccumulatedexperienceandwisdom....Fortheseadults,learningmethodsthat

combineworkandstudy,theoryandpracticeprovideamorefamiliarandtherefore

moreproductivearenaforlearning.”Kolbidentifiedfourlearningmodesinhis

experientiallearningmodel:affective,perceptual,symbolicandbehavioral(thismodelis

describedintheLiteratureReviewchapter).

Theaffectivelycomplexlearningenvironmentrequiresactivitiessimilartothose

performedasaprofessionalorreflectionontheexperiencetogenerateinsightsandself‐

discovery(Kolb,1984).BothoftheseaspectsarecharacteristicofplayingSimProject:

thesimulationoftheprofessionalprojectmanagerexperienceduringthecourseofthe

gameandthereflectionthatoccursaspartoftheperiodicprogramreviewsandduring

thedebriefsessionafterthegame.

Perceptivelycomplexlearningenvironmentsrequireconsideringatopicfrom

differentperspectivesandindifferentways(Kolb,1984).Thismodeoflearningisalsoa

componentofusingSimProjectasanexperientiallearningactivity.Theuseofteams,

selectedtomaximizetheirdiversity,asthemakersofgamedecisionshelpsassurethe

considerationofdifferentperspectiveswhileplanningforandplayingtheproject

simulationgame.Theinstructor‐facilitatedperiodicprogressreviewsandformal


“reportstomanagement”helpstimulateinquiryandreflectionasaguidetofutureaction

andcircumventanyparticipant’sdesiretomerelyevaluatebasedonthecorrectnessof

thedecisions.

Symbolicallycomplexlearningenvironmentsinvolveabstract“problem[s]for

whichthereisusuallyarightanswerorabestsolution”(Kolb,1984)wherethelearner

isguidedandconstrainedbytherulesandtheteacheristheacceptedexpert.Thismode

sharessome,butnotall,characteristicswiththeSimProjectexperience.The

performanceofprojectworkandthehumanresourcesthatperformitarebothabstract

sincetheyaresimulatedonthecomputer.Also,therearerulesofthegameanda

performancegoaltoachieve.However,herethebestsolutionisnotknownandthe

instructorplaysafacilitativeratherthanexpertrole.

Behaviorallycomplexlearningenvironmentsemphasizeapplyingknowledgeor

skillstoapracticalproblemthatneednothaveacorrectorbestanswer;“butitdoes

havetobesomethingthelearnercanrelateto,value,andfeelsomeintrinsicsatisfaction

fromhavingsolved”(Kolb,1984).Thefocusofbehaviorallycomplexlearningison

completingthetaskandthelearnerbeingresponsiblefordecidingonacourseofaction

andmanaginghisorhertimewithintheconstraintsofpossiblecheckpointsanda

deadline.Thisisanaccuratedescriptionoftheenvironmentencounteredforthis

applicationofSimProjectasanexperientiallearningactivity.

Inthecontextofthisstudy,theSimProjectexperiencehasastrongbehavioral

learningmodeandexhibitsstrongcharacteristicsoftheaffectiveandperceptualmodes,

andtoalesserdegree,thesymbolicmodewherelearnerpreferencestendtobefor

thinkingaloneandavoidinggroupexercisesandsimulations.TeamsplayingSimProject

cansomewhatmitigatethepreferenceforsymboliclearningbydividingthetasksofeach

workperiodamongtheteammemberssuchthattheneedforindividualthinkingcanbe

providedfor.Whilemostteamswereobservednotpreferringtosubdividethework,

dividingdecisionperiodtaskswasobservedinseveralinstanceswhereteamshadoneor

morememberswithstrongpreferencesforsymboliclearning.

Discussion of Findings

Althoughseveralvariablesdidnotmeetalltheassumptioncriteriafor

parametrictests,mostdidandallwereassumedtobeusablesince“mostparametric

testsarerelativelyinsensitivetoslightviolationsofassumptions...anddotendtobe

usedifpossible.”(Burns&Burns,2008).


Research Question 1 – Support for Hypotheses 1 and 2.Researchquestion1asked

whetherparticipantperceptionsofprojectmanagementknowledgeandconfidencein

theirabilitytoapplythisknowledgeincreasedasaresultofthesimulationgame

experience.Hypothesis1statedperceptionsofprojectmanagementknowledgewould

increaseandHypothesis2statedperceptionsinabilitiestoapplythisknowledgewould

increase.

Finding 1.Onaverage,participantsassessedsignificantlygreaterproject

managementknowledgeaftercompletingthesimulation(M=5.5,SE=0.132)than

beforestartingthesimulation(M=5.01,SE=0.148),t(24)=4.942,p<.01,r=.71.This

0.49increaseinthemeanissmallerthanthe1.26significantincreaseobservedby

McCreery(2003)inhisstudyusingadifferentprojectmanagementsimulationgameas

anelementinagraduateprojectmanagementcourseatNorthCarolinaStateUniversity’s

CollegeofManagement.However,asshowninFigure20,thestudentsinMcCreery’s

studystartedwithaloweraverageself‐assessmentandhadmoreroomtogrow,perhaps

becausethiswastheirfirstcourseinprojectmanagementandtheywerenotinaproject

managementspecialtyprogram.

FIGURE20‐COMPARISONOFAVERAGEKNOWLEDGERATINGS

McCreery(2003)reportedthattheknowledgeincreasewasgreaterfortheleast

experiencedquartileofstudents(1.46forthosewithnooralmostnoprojectwork

experience)andlessforthemostexperiencedquartile(1.01forthoseaveraging10.7

yearsofexperience).Inbothofthesecases,theincreasesweregreaterthanfoundinthis

1.00 2.00 3.00 4.00 5.00 6.00 7.00

McCreery (2003)

This Study

Self‐Assessment of Knowledge1 = Extremely Low 7= Extremely High

Post

Pre


studybutthepostsimulationknowledgescoreswereless(4.88[lowexperience]and

5.14[highexperience]versus5.5[thisstudy]).Thisisnotunexpectedasthestudentsin

thisstudywerecompletingtheirsixthcourseinprojectmanagementinaproject

managementspecialtyprogramwithareportedaverageof12.7yearsofwork

experienceandsevenyearsofexperienceworkingwithprojects.Withthisexperience

andadditionalcoursework,itisnotsurprisingthatthesestudentswouldreportahigher

thanmid‐rangeself‐assessmentofknowledgepriortotheELA.

Thepostsimulationaverageratingof5.5indicatesafairlyhighself‐assessmentof

knowledge.Sincethereisalwaysmoretolearn,onewouldnotexpectverymany

studentstobeboldenoughtoratetheirknowledgeasextremelyhighandscores

between5and6arereasonable.

Finding 2.Onaverage,participantsassessedsignificantlygreaterconfidencein

theirabilitytoapplyprojectmanagementknowledgeaftercompletingthesimulation(M

=5.4,SE=0.157)thanbeforestartingthesimulation(M=4.75,SE=0.162),t(24)=

5.639,p<.01,r=.75.Again,asshowninFigure21,theparticipantsinthisstudy

reportedasmallerincreasebutahigheraverageratingofconfidenceintheirabilityto

applyprojectmanagementknowledgethanMcCreery(2003)observedinhisstudy

(unliketheknowledgerating,McCreeryfoundnosignificantdifferenceintheaverage

increasebetweenlowexperienceandhighexperiencequartiles[1.02and1.06,

respectively]).

FIGURE21‐COMPARISONOFAVERAGECONFIDENCERATINGS

1.00 2.00 3.00 4.00 5.00 6.00 7.00

McCreery (2003)

This Study

Self‐assessment of confidence in ability to apply knowledge1 = Extremely Low 7 = Extremely HIgh

Post

Pre


Whilethe0.65increaseinperceptionsoftheabilitytoapplyprojectmanagement

knowledgeisnotasdramaticasfoundbyMcCreery(2003),theratingof5.4indicatesa

fairlyhighself‐assessmentofconfidence.

Conclusion.Consideringthelargeeffectsizesoftheseresultsandthethemes

expressedinthequalitativedata,itappearsstudentsbelievetheirprojectmanagement

knowledgeandconfidenceintheirabilitytoapplythisknowledgeincreasedasaresult

oftheSimProjectexperientiallearningactivity.Inusingasimulationgameasacapstone

activity,itisnotsurprisingtoseemodestgainsinknowledgeandconfidence.Large

gainswouldimplystudentsstartedwithlittleornoknowledgeandconfidence.This

mightbethecaseifthesimulationwasusedatthebeginningofthecoursewithaless

experiencedgroupofstudents;butinthisstudy,anysignificantgainwithexecutive

educationstudentsismeaningful.

Research Question 2 – Support for Hypotheses 3 and 4.Researchquestion2

exploredhowparticipantopinionsregardingtheirteamexperienceintheprogramand

groupworkingeneralchangedasaresultofplayingthesimulationgame.Hypothesis3

statedparticipantswouldreportahigheropinionoftheirteamexperienceafter

completingthesimulationgame.Hypothesis4statedparticipantswouldreportahigher

opinionofgroupprocessesingeneralaftercompletingthesimulationgame.

Finding 3.Satisfactionwiththestudents’currentclassteamwasassessedusing

thenine“teamexperienceassessment”itemsfromMcCreey’s(2003)study.Unlike

McCreery’sstudywhichonlyassessedthequalityofthestudentteamexperience

followingthecompletionofthesimulationgame,thisstudypolledsatisfactionboth

beforeandafterthesimulationgame.AlsounlikethestudentsinMcCreery’sstudy,

thesestudentshadalreadyparticipatedwiththeirclassteamonmultipleassignments

overaspanofninemonthsbeforeworkingtogetherasamanagementteamtoexecute

thesimulatedproject.Aspartofthepresimulationsurvey,studentsassessedtheirprior

teamexperienceonthepriorexercisesandactivitiesintheprogram.Followingthe

simulation,theywereaskedtoassesstheirteamexperiencethroughoutthesimulation

exerciseusingthesamequestionsandratingscale(1=stronglydisagree;7=strongly

agree):

Theworkloadwasbalancedacrossallteammembers

Teammemberscooperatedwellthroughouttheprogram

Ourteamworkedinanefficientmanner


Teammembersallparticipatedequallyintheteamdecisionmaking

process

Ourteammaintainedapleasantworkingatmosphere

Ourteamworkedoutdisagreementsinanequitablemanner

Teammemberswerehighlymotivatedtoperformwellintheteam

exercisesandactivities

Overall,Iamsatisfiedwithmyteamexperience

Iwouldbewillingtoworkwithmyteamonanactualprojectinthefuture

[AppendicesCandD]

Averagescoreswerefavorablebothbeforeandafterthesimulationand,on

average,participantsreportedsignificantlygreatersatisfactionwiththeirteam

experienceintheprogramaftercompletingthesimulation(M=5.71SE=0.234)than

beforestartingthesimulation(M=5.39,SE=0.238),t(24)=1.872,p<.05,single‐tailed,

r=.36.Theaverageincreaseinagreementwiththesurveystatementsof0.32suggestsa

slightlyhigherlevelofsatisfactionwiththeirteamexperienceplayingthesimulation

gamethanontheotherpriorteamassignmentsintheprogram.Thisscoreislessthan

the6.29compositeprocessmeanscorereportedbyMcCreery(2003).Onepossible

explanationforthishigheraverageisMcCreery’steamsstartedwithoutaneighttonine

monthlegacyofworkingtogetherandhadfeweropportunitiestobecomedissatisfied

witheachother.

Finding 4.TwoquestionsfromBuzzetto‐MoreandMitchell’s(2009)study

comprisedtheconstructusedtoassessgenericgroupworkattitudes:

Groupprojectshelppreparestudentstobeabletoworkinprofessional

groupsinthefuture

Ienjoyworkingingroups[AppendicesCandD]

Onaverage,participantsassessedsignificantlygreaterpositiveopinionongroup

workingeneralaftercompletingthesimulation(M=6.02,SE=0.151)thanbefore

startingthesimulation(M=5.56,SE=0.188),t(24)=2.623,p<.01,singletailed,r=.47.

This0.46increaseto6.02ona7‐pointscale(1=stronglydisagree;7=stronglyagree)

indicatesahighlevelofagreementwiththesetwogenericteamworkattitudequestions

andamorefavorableattitudeaftercompletingthesimulationgame.


Conclusion.Thegraduatedegreeprogramunderstudyemphasizesleadership

andgroupworkduringtheprojectmanagementcorephaseanditisreassuringandnot

surprisingthatopinionsarehighlyfavorableaboutgroupworkandworkingwiththeir

assignedprojectteams.A0.46pointincreaseingenericteamworkattitudeand0.32

pointincreaseinattitudetowardworkingwiththeirspecificteamaftercompletingthe

simulatedprojectsuggesttheprojectsimulationgameexperiencereinforcedandslightly

strengthenedtheirattitudesaboutlearningonteamsinanacademicenvironmentand

ontheplayingofsimulationgamesinparticular.Consideringthisincreaseandthe

teamworkthemesexpressedinthequalitativeresults,itappearsstudentopinionoftheir

teamexperienceandgroupworkingeneralwasfavorableandimprovedasaresultof

participatinginthesimulationgame.Thisisespeciallymeaningfulconsideringtheonly

statedlearningobjectiveinthecoursesyllabusisto“demonstrateyourabilitytoworkas

ateamtoplanandexecuteasimulateproject.”

Research Question 3 – Lack of Support for Hypotheses 5 and 6.Researchquestion

3asked,“Howdoparticipantopinionsregardingtheuseofsimulationsasalearningtool

changeasaresultofthissimulationexperience?”Hypothesis5stated,“Participantswill

reportahigheropinionontheuseofsimulationsasalearningtoolaftercompletingthe

simulationgame”andhypothesis6stated,“Participantswillreportahigheropinionon

theSimProjectsimulationaftercompletingthesimulationgame.”Neitherhypothesisis

supportedbythisresearch.

Finding 5.Thepresimulationandpostsimulationsurveyquestionsforassessing

genericsimulationattitudesareadaptedfromapostsimulationsurveyconductedby

Buzzetto‐MoreandMitchell(2009)andusedthe7‐pointscale(1=stronglydisagree;7=

stronglyagree):

Simulationsallowstudentstoseehowcourseconceptsareappliedinreal

worldpractice

Simulationsprovidevaluablereal‐worldexperience

Computersimulationshelponebetterunderstandthedecisionmaking

processthatoccursinprofessionalpractice

Simulationscanmakeclassmorefun

Simulationshelpstudentsbuildprofessionalskills

Iwouldliketoseemoresimulationsinfutureprograms[AppendicesC

andD]


AsshowninFigure22,studentsindicatedamoderatelystrongagreementwith

thestatementsindicatingfavorableattitudestowardstheuseofsimulationinthe

classroombothbeforeandafterthesimulationgameexperience.However,onaverage,

therewasnosignificantchangeofopinionontheuseofsimulationsasalearningtool

aftercompletingthesimulation(M=5.27,SE=0.223)thanbeforestartingthe

simulation(M=5.35,SE=0.237),t(24)=‐.289,p>.05,r=.06.Althoughdisappointing

fromtheperspectivethatthesimulationexperiencedidnotresultinanincreasein

favorableattitudetowardtheuseofsimulationgamesasanexperientiallearningmode,

thisfindingsuggeststhattherewasnotahalo‐effectaccountingforthesignificant

increasespreviouslydiscussedforhypotheses1–4.Studentsstartedthesimulation

gamewithafairlyhighopinionontheuseofsimulationgamesasateachingtooland

retainedthisopinionattheendofthesimulation.

FIGURE22‐COMPARISONOFPRE‐ANDPOST‐SIMULATIONATTITUDESONSIMULATION

Finding 6.ThequestionsforassessingattitudesaboutSimProjectarebasedon

postsimulationsurveysconductedbyBuzzetto‐MoreandMitchell(2009)andAhn

(2008).AsshowninTable27,thepresimulationsurveysusedslightlyre‐phased

questionstocorrectthecontext.

1 2 3 4 5 6 7

simProject difficulty

simProject

Generic

Rating on use of simulation as learning tool1 = strongly disagree 7 = strongly agree

Post

Pre


TABLE27–SIMPROJECTATTITUDEQUESTIONS

Presimulation Survey Postsimulation Survey

This simulation will be a valuable learning experience This simulation was a valuable learning experience

This simulation will help me understand key course concepts

This simulation helped me understand key course concepts

I think I will be pleased with my performance on the simulation

I am pleased with my performance on the simulation

I think I will enjoy the computer simulation I enjoyed the computer simulation

I believe this simulation will help me feel more confident about my abilities to manage projects

As a result of the simulation, I now feel more confident about my abilities to manage projects

I believe this simulation will be educational This simulation was educational

I believe this simulation will be fun and exciting This simulation was fun and exciting

I believe this simulation will be difficult This simulation was difficult

Onaverage,therewasnosignificantchangeofopinionregardingSimProject

aftercompletingthesimulation.Thiswasthecaseforboththeconstructexcludingthe“I

believethissimulationwillbedifficult”question[(M=5.51,SE=0.231)versus

presimulation(M=5.47,SE=0.223),t(24)=‐.143,p>.05,r=.03]andfortheconstruct

includingit[(M=5.28,SE=0.205)versuspresimulation(M=5.39,SE=0.191),t(24)=

‐.517,p>.11,r=.06].AsshowninFigure22,studentsbeganthesimulationgamewith

fairlyfavorableexpectationsandtheseexpectationswereconfirmedasmetbythe

postsimulationsurveyresults.

Consideringthequestionregardingperceiveddifficultyofthesimulationthat

wasremovedfromtheSP(thissimulation)variableconstruct,studentsperceiveda

significantdecreaseinhowdifficultthesimulationwasaftercompletingthesimulation

(M=3.64,SE=0.321)fromhowdifficultthesimulationwasgoingtobebeforestarting

thesimulation(M=4.76,SE=0.397),t(24)=‐2.850,p<.01,r=.50.Onepossible

explanationistheystartedthesimulationwithasmallamountofapprehensionthatwas

relievedbyafavorableexperiencebuttherearenoqualitativefindingsspecifically

supportingthisconclusion.

Conclusion.Althoughthesetwofindingsindicatenosupportforthehypotheses

relatedtomorefavorableattitudesaboutsimulationgamesingeneralandtoSimProject

inparticular,thefactthatattitudesstartedoffmoderatelypositive,andremainedso,

reinforcestheiruseinanacademicprogramandthelackofasignificantincreaseshould

notnecessarilybeaconcern.Hadtherebeenanunfavorableattitudeorasignificant

decreaseinratingsfollowingthesimulationgameexperience,thenthiswouldsuggesta

problemrequiringfurtherinvestigationandmitigation.


Basedontheobservedintensityofactivityduringtheteamworkingsessionsand

noqualitativethemessuggestingstudentsthoughtmanagingthesimulatedprojectwas

easy,thedecreaseindifficultyratingdoesnotappeartoindicatethesimulationwasnot

achallengingandrewardingexperience.

Research Question 4 – Lack of Support for Hypotheses 7.Researchquestion4

askedwhetherstudentswerebetterabletodevelopandinterpretschedulenetwork

diagramsandanalyzeearnedvaluedataasaresultofcompletingthesimulation

experience.Duringtheirpriorcoursework,studentscompletedindividualassignments

requiringthemtoproduceandanalyzeaschedulenetworkdiagramandtosolveseveral

earnedvalueanalysisproblems.Sinceallstudentssuccessfullycompletedthese

assignments,thepresimulationassessmentprovidesanindicationoftheirretentionof

theseabilitiesandthepostsimulationassessmentindicatesiftheyimprovedonthem

duringthecourseofthesimulation.

Fortheschedulenetworkanalysisproblem,studentsaregivenatablelisting

sevenactivitieswiththeirpredecessorsanddurationsandaskedtoidentifytheactivities

onthecriticalpath,theprojectduration,andtheslacktimefortwoactivities.Each

correctanswerreceivesonepointandaperfectscoreis4.

Fortheearnedvalueproblem,studentsaregivenagraphicaldisplayof

cumulativeearnedvaluedataforaprojectinprogressandaskedtoassesshowwellthe

projectisperformingrelativetoscheduleandbudgetandsixadditionalquestionsare

askedrequiringknowledgeofearnedvaluevariancesandindices.Eachcorrectanswer

receivesonepointandaperfectscoreis7.

Theseschedulenetworkandearnedvaluescoresarecombinedfortheoverall

problem‐solvingabilityscorewhichcanrangefromzeroto11.

Resultshereweresurprisinganddisappointingfromboththeretentionof

knowledgeandabilityperspectiveandfromthelearningfromthesimulation

perspective.

Finding 7. Onaverage,therewasnosignificantchangeintheoverallproblem‐

solvingscoreaftercompletingthesimulation(M=4.56,SE=0.469)thanbeforestarting

thesimulation(M=4.04,SE=0.528),t(24)=1.224,p>.05,r=.24.Thiswasalsothe

caseforthecomponentquestions:

Onaverage,therewasnosignificantchangeinabilitytoanswerthe

schedulenetworkanalysisquestionsaftercompletingthesimulation(M


=2.28,SE=0.280)thanbeforestartingthesimulation(M=2.24,SE=

0.328),t(24)=.171,p>.05,r=.03.

Onaverage,therewasnosignificantchangeinabilitytoanswerthe

earnedvalueanalysisquestionsaftercompletingthesimulation(M=

2.28,SE=0.354)thanbeforestartingthesimulation(M=1.80,SE=

0.332),t(24)=1.467,p>.05,r=.29

Reviewofthequestionnaireresponsesrevealedatendencyforstudentstonot

doacompleteforwardandbackwardpassanalysisofthenetworkdiagramresultingin

missingoneoftheactivityslacktimequestionsandtoleaveoffthenegativesignwhen

answeringtheearnedvaluevariancequestions;otherscomplainedaboutnothaving

accesstotheformulas.

Conclusion.Thepoorscoresonthepresimulationquestionssuggestabitof

carelessnessandperhapsalackofknowledgeretentionwhenansweringthenetwork

analysisquestions,andalackofretentionoftheearnedvalueconceptsfromprior

coursework.Thetimemanagementmodulewasdeliveredthreemonthspriortothe

presimulationquestionnairebeingissuedandtheearnedvaluemodulewastwomonths

prior.Itappearsongoingpracticeoron‐the‐jobapplicationisneededfortheseconcepts

tostayfreshinmind.Italsoappearsstudentsdonotrefreshthemselvesonthis

knowledgewhileplayingthegameeventhoughtheteamsaremanagingschedulesand

preparingperformancereportstomanagement.Sincethepilotgroupdidexperiencea

significantearnedvaluescoreincreasewhenashort“chalktalk”washeldduringoneof

thestatusreviews,thisshouldbeconsideredforfutureiterationsifrefreshmentof

earnedvalueconceptsistobeoneofthelearningobjectives.Anotheroptionwouldbeto

requireapplicationoftheconceptsinthereportstomanagementduringthesimulation

game.

Limitations of this research

Thesizeandnatureoftheprogramunderstudyresultedinsurveyingand

observingasmallnumberofstudents(28).Ideallymorestudentswouldhavebeen

availabletoincreasethestatisticalpoweroftheresults.Thesamplesizeusedisnearly

sufficienttodetectlargeeffects(r=.5)withastatisticalpowerof.8,butlargersample

sizesareneededtodetectsmallereffectsizeswiththesamestatisticalpower(Field,

2009).


Ethicalconsiderationsrequiredtheuseofapreexperimentalpretest‐posttest

methodologyratherthanatrueexperimentwithacontrolgroup.Atruecontrolgroup

wouldrequiredoingnothingwithsomestudentswhileothersexperiencedthe

simulationgame.Substitutinganalternateactivitywouldnotbeatruecontrolsituation

and,asPfahl(2004)experienced,wouldconfoundtheresults.

Summary

ThereisastrongrelationshipbetweenusingSimProjectasanexperiential

learningactivityandKolb’s(1984)affective,perceptualandbehavioralmodesof

experientiallearningandalessstrongrelationshipwiththesymbolicmodeoflearning.

Theuseofthissimulationrespondstolearnerswithapreferenceforanyofthese

learningmodes;however,learnerswithastrongpreferenceforsymboliclearningmay

belesssatisfiedthantheothers.

Aftercompletingtheprojectmanagementsimulationgame,studentsperceiveda

significantincreaseinprojectmanagementknowledge,confidenceintheirabilityto

applythisknowledge,favorableattitudetowardsworkingwiththeirteamandfavorable

attitudetowardsgroupwork,ingeneral.Theincreasesinknowledgeandconfidence

perceptionsweresmallerthanseenbyMcCreery(2003)butthesedifferencesare

explainedbythedifferencesinclasscomposition.Onaverage,McCreery’sstudentswere

lessexperiencedandwereattendingtheirfirstprojectmanagementcoursewhilethe

studentsinthisstudyweremoreexperiencedandattendingtheirsixthproject

managementcourse.Consequently,McCreery’sstudentsreportedlowerknowledgeand

confidencescorespriortoparticipatinginthesimulationandcomparablescoresafter

completingthesimulation.Thisdifferenceinstartingpointmayalsobeduetoproject

managementbeingacasualareaofinterestforMcCreery’sstudentswhereasitisa

primarypointofinterestforthestudentsintheprogrambeingstudied.After

completing15SCHofprojectmanagementandanintegrated3SCHorganization

behaviorcoursefocusedonprojectleadership,itisnotsurprisingthatstudentsinthis

studywouldreporthigherlevelsofknowledgeandconfidencepriortobeginningthe

simulation.

Nosignificantattitudeimprovementswerefoundforthegenericuseof

simulationgamesasexperientiallearningactivitiesorforthespecificuseofSimProject

asaprojectmanagementELA,butattitudesweremoderatelyfavorableandremainedso

acrosstheexperience.Thislackofchangemaybeviewedasacontrolandstrengthens


thefindingofsignificancefortheincreasesinperceivedknowledgeandconfidenceand

improvedattitudetowardsteamworkandtheirteam.

Nosignificantimprovementintheabilitytodevelopandanalyzeaschedule

networkdiagramortointerpretagraphicalpresentationofcumulativeearnedvalue

datawasfound.Althoughthisisdisappointingtotheinstructor,itisnotsurprising

consideringthisimprovementisnotastatedlearningobjectiveforplayingthegameand

nothingwasexplicitlybuiltintotheexperiencetopromoteitsoccurrence.

Thenextchapterfurthersummarizesthisstudy,discussesitsimplicationsfor

practiceandrecommendstheconductofadditionalresearch.


Conclusion

Theuseofsimulationsinacademicbusinesscourseshasgrownconsiderably

sinceonewasdevelopedfortrainingbytheAmericanManagementAssociationin1956;

theyarenowoftenthecentralmodeofinstructioninsomecourses(Fariaetal.,2009).

Theaimofthisstudyistodetermineifonesuchuse,theapplicationoftheSimProject

projectmanagementsimulationgameasacapstoneexperientiallearningactivityina

particularprojectmanagementgraduatedegreeprogram,providesvaluetothestudent

andisaneffectiveuseofclasstime.

Areviewoftheliteraturefoundanactiveassociationencouragingandreporting

researchontheuseofsimulationgamesinacademicbusinessmanagementprograms.

Thisreviewexploredthreerelevantbodiesofknowledge:learningtheory,simulation

gameapplicationandsimulationgameeffectiveness.Mostliteratureonbusiness

simulationgameeffectivenessexplorestheuseofmarketing,strategyortotalenterprise

simulationgames;relativelyfewarticlesreportresearchontheuseofproject

managementsimulationgames.Thescarcityofresearchontheuseofproject

managementsimulationgamesasapedagogicaltoolidentifiesagapinman’sknowledge.

Thereissubstantialcontroversyintheacademiccommunityonhowto

determinesimulationeffectivenessduetothedifficultyindefiningwhatpeopleactually

learnfromsimulationgamesandhowtomeasureit;asaresult,therearerepeatedcalls

formorerigorousresearchonthemethodsusedtoadministersimulationgamesandon

theassessmentoflearningthatresults(P.H.Anderson&Lawton,2009;Crookall,2010;

Fariaetal.,2009).Oneareaofagreementisthatthesimulationgameresultsor

performancescores(e.g.,simulatedmarketpenetration,profit,scheduleperformance,or

stakeholdersatisfaction)arenotvalidindicatorsoflearningandvaluablelessonscanbe

learnedthathavenothingtodowiththeeducator’sintention(Gosenpud,1990;

Gosenpud&Washbush,1994;Greenlaw&Wyman,1973).Consequently,when

researchingtheeffectivenessofbusinesssimulationgames,studentperceptionsof

learningarefrequentlyusedasaproxyformoredirectmeasuresoflearningand

longitudinalpreexperimentalandpostsimulationexperiment‐controlmethodsare

recommendedforresearchingeffectiveness(P.H.Anderson&Lawton,1997).The

literaturesearchrevealedonly15articlesexaminingtheuseofprojectmanagement


simulationgamesinacademicprograms;one,byMcCreery(2003),wasappropriateto

buildonforthisstudy.

Contribution to Knowledge

Thisstudycontributestotheknowledgeonusingprojectmanagement

simulationgamesinacademicprogramsbybuildingonMcCreery’s(2003)longitudinal

approachofmeasuringincreasesinperceptionsofknowledgeandabilitytoapplythat

knowledgeintwoways:(1)Byaddingasimilarlongitudinalstudyofattitudestoward

teamworkandsimulationgamesbyadaptingpostsimulationsurveyitemsusedby

McCreery,Buzzetto‐MoreandMitchell(2009)andAhn(2008),and(2)Byexploring

whetherstudentsimprovetheirabilitytodevelopandinterpretschedulenetwork

diagramsandinterpretearnedvalueperformancegraphsasaresultofparticipatingin

theprojectsimulationgame.Analysisofpostsimulationnarrativequestionresponses

helpsusunderstandwhatstudentsbelievetheylearnedfromtheexperience.

ThisstudyconfirmedMcCreery’s(2003)findingthatplayingaproject

managementsimulationgameimprovesstudentself‐perceptionsofprojectmanagement

knowledgeandconfidenceintheabilitytoapplythatknowledgeandfurtherfoundthat

studentattitudestowardsteamworkingeneralandtheirclassroomteaminparticular

improvedsignificantlyasaresultoftheexperience.Categorizationofnarrativeresponse

datasupportthesefindingsand,takentogether,bothsupporttheconclusionthatit

appearsstudentsfoundtheprojectmanagementsimulationgametobeavaluable

experienceandaneffectiveuseofclasstime.

Althoughopinionsontheuseofsimulationgamesasalearningpedagogywere

notstrengthenedasaresultofplayingtheprojectmanagementsimulationgame,they

werefavorablebeforetheactivityandremainedso.Thislackofchangeisviewedasa

controlandstrengthensthefindingofsignificancefortheincreasesinperceived

knowledgeandconfidenceandimprovedattitudetowardsteamworkandtheirteam.

Asurprisingfindingwasnochangeinstudentabilitytoconstructandanalyze

schedulenetworkdiagramsandtointerpretgraphicalearnedvaluedataasaresultof

participatinginthesimulationexperience.Whilethesewerenotexplicitlystated

learninggoalsforthesimulationgameintheprogrambeingstudied,onewouldexpect

thesetechniquestobeappliedduringthecourseofthesimulation,therebyrequiring

recallandreinforcementofpriorlearning.


Implications

ConsistentwiththefindingsofKeys(1977),theimplicationsofthislastfinding

suggestthatratherthanfollowconventionalwisdomandrelyonthesimulation

experiencetodotheteaching,instructorinterventionand/orenhancedguidelinesfor

studentsmaybenecessaryifsimulationgamesareintendedtoreinforcespecific

learningoutcomes,evenwhenthesimulationgameisbeingplayedasacapstoneactivity

andpriorlearningisassumed.

Fortheprogramunderstudy,criticalpathnetworkanalysisandearnedvalue

managementtechniquescouldunobtrusivelybestrengthenedbyrequiringtheteamto

answerspecificquestionsabouttheirscheduleandearnedvalueperformanceduringthe

interimreviewswithmanagement,ratherthanrelyingonthemtodecidewhatis

relevantandreportingonit.Areviewofcriticalpathmethodsandearnedvalue

techniquescouldthenbeincludedduringaninterimreviewsessionifneeded.

Thisstudyalsodemonstratesthatifacapstonesimulationgameexperienceisto

beanythingotherthanafunactivitytowrapupacourseorprogramphase,itis

importanttoclearlyidentifylearningobjectivesforitsuseandtodevelopandapply

measurementsoftheintendedlearning.Inthecaseoftheprogramunderstudy,the

statedlearningobjectivewastotietogethereverythinglearnedinthepriorcoursework

andtodemonstratetheabilitytoworkasateamtoplanandexecuteasimulatedproject.

Sincetheemphasisisontheteamexperience,basedontheresultsofthisstudy,this

objectiveisbeingmet.Thereisanopportunitytoreinforcepriorlearningbyproviding

moreguidanceonwhatshouldbereportedduringtheinterimreviewswith

management;however,suchguidanceactscountertothenormthatteamsfigureoutfor

themselveswhatshouldbereportedbasedontheirpriorexperiencesandcoursework.

Itisuptotheeducatortodecideonanappropriateapproach.

Anotherrecommendationistoconsidertheincorporationofamoreformalself‐

assessmentoflearningthatrequiresmorethoughtthanisnormallygivento

postsimulationsurveyquestionsthatareusuallycompletedinahurryattheendofclass.

Thiscouldbeintheformofarequiredjournalthatiskeptduringtheconductofthe

simulationand/oraformalreportonwhatwaslearnedfromtheexperienceafterits

conclusion.


Opportunities for further study

Furtherstudyintowhatstudentsthinktheylearnfromprojectmanagement

simulationgameswouldservetoidentifyopportunitiestoenhancethislearningandto

identifyneedsthatarenotbeingmet.

Littleisknownabouttheextenttowhichprojectmanagementsimulationsare

usedinacademicprograms,howtheyaredeployedandhoweffectivenessismeasured.

Astudyoftheuseofprojectmanagementsimulationgamesinotherprogramswould

createadialoguewithintheprojectmanagementacademiccommunityandfoster

improvementintheconductofprojectsimulationgamesbysharingtechniquesand

lessonslearned.Suchastudycouldinclude:

Nameandsupplierofthesimulationgame

Howdeployed(e.g.,distributedthroughoutcourse,capstoneatendofa

singlecourse,capstoneatendaseriesofcourses)

Projectmanagementprinciplesappliedduringthegame(e.g.,simProject

requiredconsiderationofresourcecharacteristicsandteamdynamicsto

estimateactivitydurations,biddingtoacquireresourcesandestablish

theircost,afixedworkbreakdownstructureandactivitysequencewith

noopportunityforfasttrackingorscopemodification,12fixed‐scope

workperiodswithvariabledurationandcostbasedontheresources

acquiredandtheassignmentstrategy,andreportstomanagementbased

onmonitoringandcontrolproceduresestablishedbythestudentteams)

Natureandtimespentonanypregamesetupassignments

Playerteamsize

Totaltimespentplayingthesimulationgame

Numberofgamingsessions

Sessionlength

Venue:in‐classorvirtualbetweenclasssessions

Learningobjectives

Assessmentmethod(s)forstudentlearningandsatisfaction

Gradeweight(e.g.,25%ofcoursegrade)

Summaryofassessmentfindings

Instructorroleduringthegame


Timingandnatureofgroupdiscussions(e.g.interimstatusreview,final

statusreview,endingdebrief)

Lessonslearnedontheconductofprojectmanagementsimulation

games

Alternatively,thisstudycouldbeexpandedtoresearchtheuseofallformsof

experientiallearninginprojectmanagementcoursework,e.g.studentinitiatedcharity

projects,industryconsultingprojects,etc.

Inconclusion,thisstudyprovidesevidencethatstudentsinanEMBAprogram

emphasizingprojectmanagementvaluetheinclusionoftheSimProjectproject

managementsimulationgameinthecurriculumandfinditaneffectivepedagogicaltool

toincreasetheirprojectmanagementknowledgeandconfidenceintheirabilitytoapply

thatknowledge,andtoimprovetheiralreadyfavorableattitudetowardsteamwork.

SimilarityofresultstothepriorstudybyMcCreery(2003)suggestthatthesefindings

maybegeneralizabletootherprojectmanagementsimulationgamesandothergraduate

businessschoolprograms.Inadditiontoenjoyinghands‐onpracticewiththetoolsof

projectmanagementinacompetitiveenvironment,studentslearnimportantlessons

relatedtoprojectplanning,resourcemanagement,teamwork,performancemonitoring

andcontrol.

Educatorsconsideringtheadditionofaprojectmanagementsimulationgameto

theircurriculumshouldbemindfuloftheimportanceofdebriefsessionsandconsider

requiringformalself‐assessmentsoflearningthataremoresubstantialthanshort

answersurveyquestions.


Appendix A

Literature Search Journal Sources

AcademyofManagementJournal

AcademyofManagementLearning&Education

AcademyofManagementReview

AdministrativeScienceQuarterly

Agriculture&AppliedEconomicsAssociation

AmericanPsychologist

BritishJournalofEducationalPsychology

CommunicationEducation

Computers&Education

ComputerSimulationandLearningTheory

DevelopmentsinBusinessSimulationandExperientialExercises

DevelopmentsinBusinessSimulationandExperientialLearning

Economist

Education+Training

EducauseReview

Engineering,ConstructionandArchitecturalManagement

EuropeanManagementJournal

FinancialPracticeandEducation

HumanResourceDevelopmentQuarterly

IEEETransactionsonEducation

IEEETransactionsonEngineeringManagement

InformationandSoftwareTechnology

InnovationsinEducationandTeachingInternational

InterdisciplinaryJournalofE‐LearningandLearningObjects

InternationalJournalofEngineeringEducation

InternationalJournalofProjectManagement

JournalofBusiness

JournalofBusinessResearch

JournalofBusinessEducation

JournalofCollegeTeaching&Learning


JournalofContingenciesandCrisisManagement

JournalofEconomicEducation

JournalofEducationforBusiness

JournalofEngineeringEducation

JournalofEuropeanIndustrialTraining

JournalofExperientialLearningandSimulation

JournalofInformationSystemsEducation

JournalofInformationTechnology

JournalofInternationalBusinessStudies

JournalofManagement

JournalofManagementDevelopment

JournalofManagementEducation

JournalofMarketingEducation

JournalofTeachinginInternationalBusiness

JournaloftheConstructionDivision

JournalofResearchonTechnologyinEducation

ManagementLearning

MMAFallEducators’Conference–2007

OntheHorizon

PersonnelReview

PerspectivesonAcademicGamingandSimulation

PhysicsEducationResearchConference–2006

PMIGlobalCongressProceedings

PMIToday

ReviewofBusinessResearch

ReviewofEducationalResearch

SchoolLeadershipandManagement

SimulationandGaming

SimulationModelingPracticeandTheory

SoftwareEngineering&KnowledgeEngineering,ProceedingsoftheAnnual

InternationalConferenceon

SystemsEngineeringandModeling,InternationalConferenceon

TeachingSociology


TheoryintoPractice

TrainingandDevelopmentJournal

TrainingMediaReview


Appendix B

Presimulation Team Assignment

UTDPMPlastics–ProjectPlanningAssignment

BackgroundInformation

UTDPM Plastics is a 10‐year old plastics manufacturing company located in North Central

Texas. They specialize in developing parts for industrial use using injection molding and

extrusion technologies. Their specific specialty lies in the area of developing made‐to‐order

parts for the automotive after‐market, although their product catalog includes products used

within many industries, both marketed directly to consumers and those sold to

manufacturers and retailers.

UTDPM Plastics is a privately owned company with next fiscal year projected revenues of $30

million. In recent years, the company has begun to broaden its capabilities by developing in‐

house design and engineering expertise. This approach has allowed the company to expand

its business opportunities by developing products for other firms that lack specific knowledge

of plastics engineering and/or manufacturing. The impetus to expand in‐house engineering

capabilities at UTDPM has been identified by upper management as a necessary means for

continuing to enhance business opportunities and generating revenue in this highly

competitive marketplace.

The goal of UTDPM is to continue to develop in‐house engineering and new product

development to a level that will provide a sustainable competitive advantage for the firm

over competitors in both the local and national markets. This includes the objective to re‐

engineer existing products and develop new products in a cost‐effective manner while

meeting customer needs. UTDPM also seeks to use exclusivity agreements and patents to

protect its revenue generation for these products. And, recognizing the value of project

management procedures, UTDPM committed to improving their new product development

execution through superior project management three years ago.

You have been hired to replace the experienced project leadership team that led the move to

superior project management. Unfortunately for UTDPM, they decided to cash in on their

experience and explore opportunities at a competing firm. Your new management is

concerned that this competing firm may capture a large share of UTDPM’s target market if


the product launch they were planning before their departure is delayed or over‐budget.

Here’s what you know about the project they were planning:

ProjectObjectiveStatement

Demonstrate UTDPM’s new product development prowess by capitalizing on a new

commercial market opportunity with product launch 260 work‐days or less after initiation at

a cost not to exceed $380,000.

Milestones

The project is divided into 12 work periods, each ending with an associated milestone. Some

milestones describe multiple rather than unique events, but management has instructed you

to stay with these milestones as they represent phase boundaries requiring an approval to

proceed. This means the start of non‐critical path sequential tasks may be delayed or

become critical because of a phase boundary.

WBSandWBSDictionary

The departed team identified 58 project tasks and grouped them into 9 functional work

packages including a project management work package containing 12 tasks, one for each

work period:

1. Market Assessment

2. Procurement

3. Supplier Quality

4. Design

5. Engineering

6. Engineering Quality

7. Manufacturing

8. Commercialization

9. Project Management

This structure is similar to past projects and management has approved this structure as part

of the scope baseline for this project. You will not be able to add, delete or edit the

description of these tasks. Your preference is to refer to these “tasks” as “activities” to be

PMBOK® Guide compliant, but you realize that “task” is part of the corporate culture and

management has signaled that you shouldn’t diddle with culture until you’ve proven

yourselves. Since some milestones are related to the completion of several tasks, you decide

to group all the milestones together in the WBS under a tenth summary task called


“Milestones” (you hesitate to call this a work package because no work is performed here).

You note that each milestone will have at least two predecessors: a project management

task and a task that is probably on the critical path.

OtherTaskInformation

The departed project management team also estimated durations for each task based on

historical information and expert input. You have been instructed to go with these estimates

as you don’t have a better source of information nor time to go looking. You note that the

duration estimating assumption was effort‐driven based on one resource per task working

full time. For a worker of normal competence, eight hours of planned work effort will be

completed in one day.

The prior team also mapped each task into one of the twelve periods (you’d rather call them

“phases” but keep this thought to yourself after the feedback on “tasks” vs. “activities”).

Since the milestones are being treated as phase boundaries you also make a note to make

sure you there is a predecessor link to the prior milestone if needed to keep the task starts in

the proper period. This seems to be as far as the prior project management team progressed

as you’ve been unable to find any other work product related to developing a project

schedule.

Fortunately you bumped into the former leader of the departed project management

planning team at a local PMI® chapter meeting and, not wanting to burn any bridges, she

sends you her notes to help you complete your planning effort. You look these over and

decide that, combined with the WBS information you received from your new boss, you have

enough information to prepare a baseline schedule and budget.

Notesfromtheformerprojectmanager

With the following exceptions, tasks within each work package are performed in

sequence

o “Identify vendors” and “Develop and Issue RFQ” can start in parallel

o “Train sales team” and “Advertising campaign” and “Show functional model

at trade show” can start in parallel (if there are no other dependencies)

o “Assess RFQ responses and select vendors” starts after ”Qualify supplier”

completes and Milestone 10


The project begins with “Market Assessment.” Its completion is designated as

Milestone 1 and is followed by the start of “Design” and “Commercialization” in

Period 2

“Engineering” follows “Design”

The other work package starts are a bit more complicated:

o “Procurement” and “Engineering Quality” can start after the completion of

“Release pre‐production specifications” and Milestone 5

o “Supplier Quality” can begin after the completion of “Issue sample

(production equivalent)”

o “Manufacturing” can start in Period 8 after the completion of “Perform

supplier process capability” and “Build functional model”

“Issue sample (production equivalent)” requires the completion of both “Identify

vendors” and “Develop and Issue RFQ.” Its work needs to be performed during

Period 7.

“Perform supplier process capability” is a predecessor of

o “Approve sample parts”

o “Design validation activities”

o “Test prototype”

o “Process engineering plan”

o “Show functional model at trade show”

o “Milestone 7”

Starting “Identify testing requirements” also needs the completion of “Develop

marketing program”

“Release pre‐production specifications” is a predecessor of

o “Identify vendors”

o “Develop and issue RFQ”

o “Issue sample”

o “Build functional model”

o “Evaluate design specifications”

o “Develop testing protocol for prototype”

o “Milestone 5”

“Build functional model” is a predecessor of


o “Design validation activities”

o “Test prototype”

o “Process engineering plan”


o Milestone 6

“Design Transfer activities” also needs the completion of “Evaluate results of tests

and identify weaknesses”

“Product release meetings” also needs the completion of “Design transfer activities”

“Develop production plan” also needs the completion of “Validation design review”

and “Evaluate results of tests and identify weaknesses”

“Develop production control plan” also needs the completion of “Qualify supplier”

“Contracting for deliveries” also needs the completion of “Assess RFQ responses and

select vendors”

“Production pilot test” also needs the completion of “Product release meetings”

The following depend on the completion of “Develop marketing program”

o “Identify testing requirements”

o “Train sales team”

o “Advertising campaign”


o “Milestone 2”

“Product launch” requires the completion of

o “Production release”

o “Train sales team”

o “Advertising campaign”


With this information, the table of milestones, and the estimated durations, you are

confident you can quickly validate the desired schedule. You recall the need to make sure all

tasks are scheduled in the proper time period. This may require adding a milestone as a

predecessor to some tasks.


ResourcePool

Management provided a list of available resources and advised these are procured though a

bidding process with other project teams just before initiation of the execution phase. You

have some concern about this but realize you need to get over it as this is life in the fast lane.

Since each person you add to the team is charged against your project budget, your goal

should be to fully utilize any resource you hire. There is no overtime. Each resource’s

personal characteristics and the nature of the task determine whether they can complete the

task with the estimated amount of effort. Training and managerial actions are available to

influence their characteristics.

The cultural background of resources for project teams impacts on cohesion and team

performance. A diverse group is more effective than one with minimal diversity; however, a

group that is too diverse may be dysfunctional.

No more than two resources may be assigned to any task.

Hired resources become available for assignment in the succeeding time period. For

example, if you successfully bid on a resource prior to executing Period 1, they will be

available for assignment to tasks in Period 2. There will be a Period 0 with multiple rounds

for hiring the initial team. A released resource is immediately removed from the team and

not available for assignment in that period and not available for rehire until two periods later

(if not hired by another project team in the interim).

Training

Management also provided information on available training in case your planned resources

need development. This is a good thing as some of the resources look like they may need

some development and, due to the bidding process, there is no guarantee you can hire the

best resources available.

ManagerialActions

A list of managerial actions available during execution is also provided. You remember from

your MBA courses that doing some of these things may be a good idea when leading people.

These may be applied in an attempt to motivate, punish, or develop the team.


YourAssignment

Using the given information:

1. Prepare a project schedule using the critical path method and determine the size of

the project completion buffer in work‐days.

2. Prepare a baseline budget using this CPM schedule and the estimates provided by

the departed project management team. Calculate the size of the management

reserve/projected overrun.

3. Prepare a project staffing plan based on your analysis of the resource requirements

and the available resources. Since you are competing for resources, a backup plan

may be in order!

4. Prepare a schedule and cash flow forecast assuming you obtain all your desired

resources at the planned bid rates. Be sure to adjust work/durations as appropriate.

5. Analyze variances versus the baseline schedule and budget. Identify any needed

gap‐closing actions. Put them in your plan and redo 4 and 5. Finishing early and

under‐budget is a good thing, late and/or over‐budget is not.

6. Prepare a summary presentation suitable for reviewing the baseline and your

forecast with management. Be prepared to explain all variances as the prior project

management team had an outstanding reputation with management and your team

is relatively unproven.

Deliverables

Post three files on Blackboard:

1. MS Project file

a. Set Baseline with CPM schedule and baseline budget from (1) and (2) above

b. Active plan reflects resource assignments and associated changes from (3) –

(5)

2. Staffing Plan (MS Office or PDF document) showing hiring, training, release, etc. plan

by period. Initial staffing for Time Period 1 occurs during Time Period 0. List plans for

Time Periods 0 – 11.

3. MS PowerPoint presentation from (6) above. Do not show the people you plan to

hire or discuss your bidding strategy as the other teams will be bidding against you

during Time Period 0.


Period‐endingMilestones(PhaseBoundaries)

All are also preceded by the project management activity for the prior period. All but

Milestone 12 are followed by the project management activity for the next period. No work

on succeeding tasks may commence until all work preceding the milestone is completed and

approved. You may assume approval is automatic and consumes no time or budget.

Preceding Tasks Milestone Succeeding Tasks

Business Evaluation 1 Design and Development Plan

Develop Preliminary Marketing Plan

Design specs.

Develop marketing program 2

Identify testing requirements

Train sales team

Advertising campaign

Risk Analysis

Train sales team

Advertising campaign

3 Design labeling

Initial engineering specs. 4 Design verification activities

Release pre‐production

specifications 5

Identify vendors

Develop and issue RFQ

Build functional model

Evaluate design specifications

Identify vendors

Develop and issue RFQ

Build functional model

Evaluate design specifications

6

Issue sample (production equivalent)

Develop testing protocol for

prototype

Perform supplier process

capability

Develop testing protocol for

prototype

7

Approve sample parts

Design validation activities

Test prototype

Process engineering plan

Show functional model at trade show

Approve sample parts

Design validation activities

Test prototype

8

Validation design review

Evaluate results of tests and identify

weaknesses


Process engineering plan

Show functional model at trade

show

Approve model design

Evaluate results of tests and

identify weaknesses

9

Qualify supplier

Design transfer activities

Develop production plan

Qualify supplier

Product release meetings

Develop production plan

10

Assess RFQ responses and select

vendors

Develop production control plan

Contracting for deliveries 11 Submit production purchase order

Product launch 12 Celebrate!!!


WBSDictionary

Work Package Task Name Task Description

Phase(Work Period)

Est. Work (hours)

Est. Labor Rate

1 Market Assessment

1 Evaluate market Conduct full market research study to identify market segments, pricing, and final confirmation of product features 1 96 $50

1 Develop Business opportunity

Identify key customers and gaining preliminary commitments or contracts in order to secure a baseline contract to justify continuation of the project 1 112 $90

1 Customer preference study

Interviewing and conducting focus groups and surveys to identify most desirable product characteristics 1 168 $50

1 Business evaluation (NPV, etc.)

Project screening used to identify costs, including revenue streams and net cash flows, for the viability assessment 1 32 $125

2 Procurement

2 Identify vendors Create a viable vendor pool for all material and service requirements, including performance criteria such as delivery, material or service quality, and pricing 6 56 $50

2 Develop and issue RFQ Identify all purchased materials and service requirements and develop requests for quotation for each requirement. Issue RFQ 6 48 $50

2 Issue sample (production equivalent)

Issue purchase order for sample quantities to be used in first run production plan 7 40 $75

2 Assess RFQ responses and select vendors

Evaluate all supplier responses to RFQ and notify those selected 11 80 $50

3 Supplier Quality

3 Perform supplier process capability

Assessment by quality control and procurement of suppliers’ capability with respect to product characteristics, delivery, timeliness, and pricing 7 112 $50

3 Approve sample parts Quality control and manufacturing test and approve production equivalent sample orders for raw materials and parts 8 64 $75

3 Qualify Supplier Using results from sample parts assessment, formally notify suppliers, plant representatives and procurement of all suppliers qualified to bid 10 80 $50



Phase(Work Period)

Est. Work (hours)

Est. Labor Rate

for contracts for materials and services

4 Design

4 Design and development plan

High level structural design of the product, including plans and schedules for product completion 2 48 $50

4 Design specs. Detailed technical drawings and schematics for the product, including all equipment needs to create the final product 2 176 $50

4 Identify testing requirements

Detail critical product specifications, acceptable tolerances and product liability limits 3 80 $50

4 Risk analysis Identify significant product usage risk and adherence to product standards. Include an assessment of acceptable levels of product tolerance. 3 80 $125

4 Design labeling Developing design labeling and packaging for the finished product 4 40 $50

4 Approve design Final assessment of product design characteristics matched to preliminary customer specifications 4 32 $50

5 Engineering

5 Initial engr. specs. Converting product design specifications into engineering templates 4 40 $50

5 Design verification activities

Validate the consistency of product functionality, product design and engineering plans 5 56 $75

5 Verification design review

Formal review with engineering, design, and marketing to finalize product design 5 32 $50

5 Release pre‐production specifications

Formal approval and sign‐off on preliminary product specifications for review and comment 5 80 $50

5 Build functional model Develop product prototype 6 144 $75

5 Design validation activities

Develop protocol for verification of product design 8 40 $50

5 Validation design review

Perform desk check (structured walk through) of product design 9 32 $125

5 Approve model design Evaluate results from design review and secure final approval from engineering, design and manufacturing 9 32 $75



Phase(Work Period)

Est. Work (hours)

Est. Labor Rate

5 Design transfer activities

Develop the process to support the transfer the product to manufacturing

10 56 $75

6 Engineering Quality

6 Evaluate design specifications

Conduct quality assessment, including quality engineering, on product designs. Create upper and lower control limits for product component manufacturing

6 80 $50

6 Develop testing protocol for prototype

Identify specific testing protocol for each product specification, document for repeatability and benchmarking

7 64 $50

6 Test prototype Perform tests to valid all product characteristics and identify significant deviations from upper and lower control limit boundaries

8 80 $50

6 Evaluate results of tests and identify weaknesses

Evaluate results of prototype tests from a product quality perspective, identify characteristics outside of control limits, and implement plan for correction

9 48 $50

6 Product release meetings

Gain required sign‐off approval from representatives from engineering, manufacturing, design, and quality control

10 24 $125

7 Manufacturing

7 Process engineering plan

Convert engineering and design specifications to an operations plan for plant work flow and design for manufacturing

8 120 $50

7 Develop production plan

Identify the specific machine and manpower resources needed to produce the requirements for the product

10 48 $50

7 Develop production control plan

Develop the schedule for raw materials, shipping, and packaging against the sales forecast and requirements plan

11 68 $50

7 Approve production parts

Assess and approve first‐run production of product components 11 40 $50

7 Contracting for deliveries

Specify exact terms for schedules and quantities of manufacturing supplies, including quantity release schedule

11 64 $50

7 Submit production purchase order

Issue detailed production requirements for production pilot test 12 16 $50

7 Production pilot test Test production run within normal plant operations, staffing, and 12 40 $50



Phase(Work Period)

Est. Work (hours)

Est. Labor Rate

resource requirements for operational stability

7 Debugging production system

Identify and correct any significant deviations from process operations and product outcomes

12 32 $50

7 Production release Issue formal sign‐off from manufacturing to accept product into the production system

12 24 $50

8 Commercialization

8 Develop preliminary marketing plan

Develop timetable, responsibilities and costs for creating and implementing the marketing program

2 40 $90

8 Develop marketing program

Creating a plan to identify customers by segment, promotional programs, pricing structures, and distribution channels

2 120 $90

8 Train sales team Specific product training for sales personnel with the purpose of having them knowledgeable regarding the product during conversations with potential customers

3 176 $50

8 Advertising campaign Develop detailed advertising plan, including media schemes, scripts, and public relations activities (trade shows and trade journal promotion)

3 224 $50

8 Show functional model at trade show

Design display and coordinate delivery and presentation of prototype at selected trade shows

8 24 $90

8 Product launch Plan for and implement the formal announcement of the new product 12 24 $125

9 Project Management

9 Project Management Overseeing all project tasks for the period, developing status reports and project updates, and maintaining project team staffing and commitment

1 200 $125


2 112 $125


3 112 $125


4 104 $125


5 120 $125



Phase(Work Period)

Est. Work (hours)

Est. Labor Rate


6 72 $125


7 88 $125


8 24 $125


9 32 $125


10 40 $125


11 104 $125


12 80 $125

Training and managerial action budget of $25,000 is allocated at $3,125 per period for periods 1‐8


ResourceInformationandMetrics

Standard Rate is the rate paid on their last project. Resources may reject your bid if it is below what they think they should be paid. Training is the amount of previous instruction in related skills. You may send resources for additional training if you think it would enhance their ability to perform work effectively. Skill is degree of expertise. Skill levels can be enhanced through additional training. Experience relates to the length of time the individual has worked in their field. More experienced people tend to be more efficient and (at least initially) adapt faster to working on project teams. Education relates to the level and relevance of education completed. Work Ethic is the set of principles that individuals have about performing the job. A stronger work ethic means that the project team member is disposed to work more diligently. Reputation is the general belief about an individual’s character. It may also be described as the state of being well thought of. The better the individual’s reputation, the easier it is to hire other team members, retain top management support, and keep stakeholders happy. Public Relations skills include employee communications, media relations, advertising, and community relations. They are the ability of a person to present an appropriate “face” to external stakeholders. Flexibility is a measure of the adaptability of a person to a change in circumstance and the ability to handle changes. Interpersonal Skills are goal‐directed behaviors conducted in a face‐to‐face environment. They enable a person to relate to and interact with others.


Resource Name Category Age Gender Cultural

Background Stan

dard Rate

Overtim

e Rate

Training

Skill Level

Experience

Education

Reputation

Work Ethic

Public Relations

Flexibility

Interpersonal

Skills

Brandon Plude Engineer 24 Male American $57 $86 78 96 59 62 71 63 70 40 58

Christian Cepeda Engineer 26 Male American $46 $69 59 82 59 61 57 86 58 32 65

Darren Mohammad Engineer 27 Male American $48 $72 75 80 56 51 68 71 67 33 55

Ja Hauser Engineer 28 Female American $50 $75 83 89 50 66 71 62 72 36 61

Jose Fernandez Engineer 28 Male Spanish $51 $76 79 85 60 54 71 75 70 35 59

An Wu Junior Marketing Specialist 21 Female Chinese $53 $80 43 78 58 50 39 53 68 66 66

Bing Heilbronner Junior Marketing Specialist 26 Male German $58 $87 50 80 56 49 43 51 54 52 58

Ceandro Apellániz Junior Marketing Specialist 24 Male Spanish $47 $70 53 64 46 45 33 62 66 72 61

Felipe Garcia Junior Marketing Specialist 29 Male Spanish $48 $72 51 71 46 45 41 72 54 60 52

Fernando Bleakley Junior Marketing Specialist 24 Male American $45 $68 58 62 52 65 37 54 49 63 68

Guy Geer Junior Marketing Specialist 25 Male American $56 $84 48 77 54 47 41 49 52 50 55

Hekirou Ohitsuji Junior Marketing Specialist 27 Male Japanese $58 $87 41 56 49 63 37 59 57 63 64

Kurt Darcangelo Junior Marketing Specialist 28 Male American $47 $70 46 64 52 53 39 55 67 56 70

Natasha Cremin Junior Marketing Specialist 25 Female American $47 $70 40 65 54 46 39 56 51 66 70

Tani Ashida Junior Marketing Specialist 26 Female Japanese $56 $84 52 82 55 61 42 50 64 54 53

Armand Oswald Junior Product Designer 22 Male German $55 $82 46 69 42 52 43 55 41 33 49

Bud Zimmermann Junior Product Designer 22 Male German $55 $82 40 69 36 53 51 56 47 34 43

Gregory Treglia Junior Product Designer 22 Male American $41 $62 44 49 42 55 43 55 33 43 52

Kyung‐Yul Zheng Junior Product Designer 23 Male Chinese $51 $76 34 60 39 58 51 66 46 45 41

Madelene Emel Junior Product Designer 22 Female American $56 $84 38 67 33 56 51 49 40 39 40

My‐Trang Ding Junior Product Designer 22 Female Chinese $49 $74 39 58 44 56 57 65 35 39 47

Nadine Bathurst Junior Product Designer 23 Female German $51 $76 41 61 45 58 59 67 36 41 49

Sean Stanger Junior Product Designer 28 Male American $59 $88 45 50 35 44 41 61 45 43 48

Virgen Couch Junior Product Designer 21 Female American $54 $81 36 64 47 54 49 71 38 38 58

Wayne Brousseau Junior Product Designer 27 Male American $58 $87 33 48 39 43 47 59 38 42 59

Alejandro Córdoba Marketing Manager 36 Male Spanish $81 $122 75 75 81 78 90 85 83 81 64

Ana Garcia Marketing Manager 34 Female Spanish $76 $114 81 70 96 73 97 79 61 67 70

Baron Heilbronner Marketing Manager 24 Male German $102 $153 70 100 67 74 89 82 57 94 88


Resource Name Category Age Gender Cultural

Background Stan

dard Rate

Overtim

e Rate

Training

Skill Level

Experience

Education

Reputation

Work Ethic

Public Relations

Flexibility

Interpersonal

Skills

Darren Babbitt Marketing Manager 31 Male American $76 $114 90 76 96 82 77 91 75 71 85

Melodee Pledger Marketing Manager 35 Female American $79 $118 84 73 67 76 100 82 63 70 72

Carrie Frankfurter Operations Specialist 33 Female German $56 $84 72 70 74 41 66 76 60 61 86

Chigemi Washi Operations Specialist 31 Male Japanese $49 $74 75 76 56 48 65 85 64 50 67

Coronie Alonso Operations Specialist 30 Female Spanish $46 $69 71 72 81 45 62 81 60 70 63

Emery Zimmermann Operations Specialist 32 Male German $54 $81 81 91 82 59 73 73 74 59 64

Genji Nguyen Operations Specialist 31 Female Chinese $54 $81 64 86 75 49 56 86 55 57 79

Jane Barto Project Manager 37 Female American $143 $214 84 100 99 60 66 80 86 100 88

Kurt Nordine Project Manager 40 Male American $138 $207 76 98 73 50 70 98 79 99 89

Leonard Rockholt Project Manager 47 Male American $111 $166 79 79 77 62 98 81 82 90 93

Tyrone Leigh Project Manager 39 Male American $148 $222 69 75 89 62 90 83 68 83 79

Tyrone Rutten Project Manager 42 Male American $145 $218 80 100 78 54 75 70 83 100 94

Adelle Hitzig Quality Engineer 26 Female German $87 $130 90 74 86 63 93 95 34 17 51

Chigemi Yagi Quality Engineer 33 Male Japanese $81 $122 80 95 68 58 100 90 29 17 45

Jessie Taffe Quality Engineer 33 Male American $72 $108 92 85 89 51 76 90 34 16 52

Monique Korbar Quality Engineer 28 Female American $60 $90 78 100 76 65 98 76 29 21 44

Stephen Asuncion Quality Engineer 36 Male American $65 $98 100 82 80 58 81 93 24 23 52

Alder Heilbronner Senior Product Designer 29 Male German $86 $129 79 91 65 71 69 80 59 69 51

Danielle Stupp Senior Product Designer 23 Female American $73 $110 79 91 71 61 55 65 46 55 57

Guy Slain Senior Product Designer 27 Male American $81 $122 85 85 77 49 54 82 48 65 61

Mike Deblanc Senior Product Designer 28 Male American $69 $104 70 78 76 70 64 73 57 58 67

Stephen Adams Senior Product Designer 25 Male American $74 $111 67 78 55 61 58 68 51 60 65


Training

Training Name Description

Cost per

resource Duration

Maximum

Seats

Benchmarking Students will identify project management processes and develop strategies for obtaining information on best practices and implementing these strategies in their organizations.

$800 4 days 3

Financial Skills Refresher Use of NPV and IRR for project selection and portfolio management Successful completion of this course will enable students to be conversant in project selection criteria and to prepare cost‐benefit analysis.

$600 3 days 5

Interpersonal Skills Students will learn effective leadership techniques, group behavior and decision making. They will also practice persuasive communications, conflict resolution, and influence tactics.

$600 3 days 7

Introduction to Planning Elements in effective scope management, scheduling, resource management, and risk assessment.

$1,000 5 days 4

Market Research Tools An introduction into focus group administration, business‐to‐business survey methods, and data collection and analysis.

$400 2 days 2

Negotiation Techniques Basic skills and practice of effective negotiation. $600 3 days 5

Principles of Quality Introduction of Deming's Principles of Total Quality Management, statistical process control, and Taguchi quality cycles.

$600 3 days 5

Process Engineering Intermediate level course to learn best practices of organizational process design and improvement. Kaizen principles will be discussed.

$600 3 days 2

Project Evaluation Intermediate level course in project tracking and control techniques. $1,000 5 days 5

Project Management 101 Basic project management, covering a survey of the major relevant skills and knowledge to manage or perform effectively on projects.

$1,000 5 days 5


ManagerialActions

Managerial Action Name Description Cost per resource

Company Sponsored Family Event Company hosts a picnic for team members and their families. $75

Disciplinary Action Project Manager applies formal disciplinary sanctions for employee's behavior. Notation is placed in personnel file.

$0

Management Recognition Award Top management sends letters to all team members on project accomplishments. $50

Milestone Celebration Team celebrates completion of current milestone. $1,000

Monetary Bonus Project Manager announces nominal monetary award for excellent performance. $2,500

One‐on‐one Chat Project Manager calls in team member for informal discussion and corrective suggestions. $0

Pizza Party Project Manager throws a pizza party for the entire team. $10

Verbal Warning Project Manager applies informal disciplinary action without notation in file but warning if repeat performance occurs.

$0


Appendix C

Presimulation Survey

Thank you for participating in this survey! To help assure your responses are anonymous; please enter a unique identifier that is not traceable to you as an individual:

________________________ Which Team are you on? ________ Team A ________ Team B ________ Team C ________ Team D Enter this information in response to questions 12 and 13 and retain this sheet as you will need to enter this same information on the post‐simulation survey.


1. Assess your current level of knowledge in each of the following areas:

1 = Extremely Low ‐‐‐ 7 = Extremely High

A. Project organizational design 1 2 3 4 5 6 7

B. Project planning in general 1 2 3 4 5 6 7

C. Assessing and managing project risks 1 2 3 4 5 6 7

D. Estimating project scope 1 2 3 4 5 6 7

E. Sequencing of project activities 1 2 3 4 5 6 7

F. Estimating project activity times 1 2 3 4 5 6 7

G. Project budgeting 1 2 3 4 5 6 7

H. Allocating project resources 1 2 3 4 5 6 7

I. Project change management 1 2 3 4 5 6 7

J. Designing project performance measures 1 2 3 4 5 6 7

K. Performing as an effective project leader 1 2 3 4 5 6 7

L. Building team consensus 1 2 3 4 5 6 7

M. Negotiating for resources and budgets 1 2 3 4 5 6 7

N. Using Earned Value concepts 1 2 3 4 5 6 7

O. Evaluating the performance of project personnel 1 2 3 4 5 6 7

P. Managing project uncertainty 1 2 3 4 5 6 7


2. Assess how confident you are in your ability to effectively apply your knowledge in each of the following areas:

1 = Extremely Low Level of Confidence ‐‐‐ 7 = Extremely High Level of Confidence


















3. Assess your team experience on exercises and activities in this program prior to today

1 = Strongly Disagree ‐‐‐ 7 = Strongly Agree

A. The workload was balanced across all team members 1 2 3 4 5 6 7

B. Team members cooperated well throughout the 1 2 3 4 5 6 7 program

C. Our team worked in an efficient manner 1 2 3 4 5 6 7

D. Team members all participated equally in the team 1 2 3 4 5 6 7 decision making process

E. Our team maintained a pleasant working atmosphere 1 2 3 4 5 6 7

F. Our team worked out disagreements in an equitable 1 2 3 4 5 6 7 manner

G. Team members were highly motivated to perform well 1 2 3 4 5 6 7

in the team exercises and activities

H. Overall, I am satisfied with my team experience 1 2 3 4 5 6 7

I. I would be willing to work with my team on an actual 1 2 3 4 5 6 7 project in the future


4. Assess your view of using simulations as a learning tool


A. Simulations allow students to see how course 1 2 3 4 5 6 7 concepts are applied in real world practice

B. Simulations provide valuable real‐world experience 1 2 3 4 5 6 7

C. Group projects help prepare students to be able to 1 2 3 4 5 6 7 work in professional groups in the future

D. This simulation will be a valuable learning experience 1 2 3 4 5 6 7

E. Computer simulations help one better understand 1 2 3 4 5 6 7 the decision making process that occurs in professional practice

F. This simulation will help me understand key 1 2 3 4 5 6 7 course concepts

G. Simulations can make class more fun 1 2 3 4 5 6 7

H. I think I will be pleased with my performance on the 1 2 3 4 5 6 7 simulation

I. Simulations help students build professional skills 1 2 3 4 5 6 7

J. I think I will enjoy the computer simulation 1 2 3 4 5 6 7

K. I enjoy working in groups 1 2 3 4 5 6 7

L. I would like to see more simulations in future 1 2 3 4 5 6 7 programs

M. I believe this simulation will help me feel more 1 2 3 4 5 6 7

confident about my abilities to manage projects

N. I would be willing to work with my team on an 1 2 3 4 5 6 7 actual project in the future

O. I believe this simulation will be educational 1 2 3 4 5 6 7

P. I believe this simulation will be fun and exciting 1 2 3 4 5 6 7

Q. I believe this simulation will be difficult 1 2 3 4 5 6 7


5. For the following network, develop an activity‐on‐node project network. Complete the forward and backward pass, compute the activity slack, and identify the critical path.

Activity Predecessor Time (weeks)

A None 4

B A 5

C A 4

D B 3

E C,D 6

F D 2

G E,F 5

A. What activities are in the critical path?

B. How many weeks are in the critical path?

C. What is the slack time for activity C?

D. What is the slack time for activity F?


6. Respond to the questions below based on the following graphical project information.

A. How is this project performing (select the best answer)?

1. Ahead of schedule and under budget

2. Ahead of schedule and over budget

3. Behind schedule and under budget

4. Behind schedule and over budget

5. I have no idea

As of “Today” in the chart”

B. What is the value in dollars of the schedule variance? __________

C. What is the value in dollars of the cost variance? __________

D. What is the value of the schedule performance index? __________

E. What is the value of the cost performance index? __________

F. What is the value of the to‐complete performance index? __________

G. What will the value of the SPI be at the end of the project? __________


7. How many years of on‐the job experience have you had as a project manager/director, project coordinator, project team member, and other project‐related participant? ______ years 8. How many years have you worked as a professional (this is typically the time since you graduated from college with a bachelor’s degree)? ______ years 9. What is your gender?

Male Female

10. What formal project management training have you had prior to this course? Include an estimate of the number of contact or classroom hours. 11. How would you describe your background? Technical Non‐technical 12. What team are you a member of? 13. Print a unique identifier that will facilitate correlation with a post‐simulation questionnaire.


Appendix D

Postsimulation Survey

1. Assess your current level of knowledge in each of the following areas:

1 = Extremely Low ‐‐‐ 7 = Extremely High


















2. Assess how confident you are in your ability to effectively apply your knowledge in each of the following areas:

1 = Extremely Low Level of Confidence ‐‐‐ 7 = Extremely High Level of Confidence


















3. Assess your team experience throughout the simulation exercise


A. The workload was fairly balanced across all team 1 2 3 4 5 6 7 members

B. Team members cooperated well throughout the 1 2 3 4 5 6 7

exercise

C. Our team worked in an efficient manner 1 2 3 4 5 6 7

D. Team members all participated equally in the team 1 2 3 4 5 6 7 decision making process

E. Our team maintained a pleasant working atmosphere 1 2 3 4 5 6 7

F. Our team worked out disagreements in an equitable 1 2 3 4 5 6 7 manner

G. Team members were highly motivated to perform 1 2 3 4 5 6 7

well in the exercise

H. Overall, I am satisfied with my team experience 1 2 3 4 5 6 7

I. I would be willing to work with my team on an 1 2 3 4 5 6 7 actual project in the future


4. Assess your view of using simulations as a learning tool


A. Simulations allow students to see how course 1 2 3 4 5 6 7 concepts are applied in real world practice

B. Simulations provide valuable real‐world experience 1 2 3 4 5 6 7

C. Group projects help prepare students to be able to 1 2 3 4 5 6 7 work in professional groups in the future

D. This simulation was a valuable learning experience 1 2 3 4 5 6 7

E. Computer simulations help one better understand 1 2 3 4 5 6 7 the decision making process that occurs in professional practice

F. This simulation helped me understand key course 1 2 3 4 5 6 7 concepts

G. Simulations can make class more fun 1 2 3 4 5 6 7

H. I am pleased with my performance on the simulation 1 2 3 4 5 6 7

I. Simulations help students build professional skills 1 2 3 4 5 6 7

J. I enjoyed the computer simulation 1 2 3 4 5 6 7

K. I enjoy working in groups 1 2 3 4 5 6 7

L. I would like to see more simulations in future 1 2 3 4 5 6 7 programs

M. As a result of the simulation, I now feel more 1 2 3 4 5 6 7

confident about my abilities to manage projects

N. I would be willing to work with my team on an 1 2 3 4 5 6 7 actual project in the future

O. This simulation was educational 1 2 3 4 5 6 7

P. This simulation was fun and exciting 1 2 3 4 5 6 7

Q. This simulation was difficult 1 2 3 4 5 6 7


5. For the following network, develop an activity‐on‐node project network. Complete the

forward and backward pass, compute the activity slack, and identify the critical path.

Activity Predecessor Time (weeks)

A None 4

B A 5

C A 4

D B 3

E C,D 6

F D 2

G E,F 5

A. What activities are in the critical path?

B. How many weeks are in the critical path?

C. What is the slack time for activity C?

D. What is the slack time for activity F?


6. Respond to the questions below based on the following graphical project information.

A. How is this project performing (select the best answer)?

1. Ahead of schedule and under budget

2. Ahead of schedule and over budget

3. Behind schedule and under budget

4. Behind schedule and over budget

5. I have no idea

As of “Today” in the chart”

B. What is the value in dollars of the schedule variance? __________

C. What is the value in dollars of the cost variance? __________

D. What is the value of the schedule performance index? __________

E. What is the value of the cost performance index? __________

F. What is the value of the to‐complete performance index? __________

G. What will the value of the SPI be at the end of the project? __________


7. What did you like about the just‐completed simulation experience?

8. Excluding changes to the computer simulation itself, what changes do you think could be made to the overall simulation experience to better prepare you to manage projects?


9. What changes do you recommend be made to the computer simulation itself?

10. Elaborate on any other thoughts you have regarding learning project management with a computer‐based simulation game.


11 What best describes the industry you work in?

General Construction

Oil, Gas, Petrochemical or Natural Resources

Telecommunications

IT/IS

Pharmaceuticals

Management Services

Banking

Consulting

Other: ___________________________

12. What best describes your JOB TITLE?

Project Manager

Functional Manager

Cost Estimator/Scheduler

Finance/Accounting

Sales and Marketing

Training, Mentoring or Consulting

Human Resources

Engineering/Technical Support

Procurement/Purchasing/Expediting

Other: ____________________________

13. What is your highest level of formal education?

High School

Junior College

Bachelors Degree

Masters Degree

PhD


14. What was your PMP practice exam score?

Less than 34.5

34.5 – 38.4

38.5 – 41.9

42.0 – 45.9

Greater than 46

15. What best describes the part of the world you originate from?

North America

Central/South America

European Union

Russia/Eastern Europe

Middle East

Africa

Asia Pacific

16. What is your total gross income (in U.S. dollars) over the past twelve months from you primary source of employment (e.g., salary plus cash bonuses before taxes and other withholding)?

Less than $20,000

$20,001 ‐ $40,000

$40,001 ‐ $60,000

$60,001 ‐ $80,000

$80,001 ‐ $100,000

$100,001 ‐ $120,000

$120,001 ‐ $140,000

More than $140,000

17. What Team were you on?

18. What is the unique identifier you assigned to the pre‐simulation survey?


Appendix E

Project Simulation Final Presentation

ProjectSimulation–FinalPresentation

FinalStatusReport

Final Metrics: Time/Cost/Functionality/Stakeholder/Overall

Baseline Completion Date vs. Actual Completion Date

Baseline Budget vs. Actual Cost

Variance Explanations

ProjectAuditReport

Original strategy – what was your original approach?

How did you divide responsibilities on your team?

What worked?

What didn’t work?

What would you do differently the next time?

Some things to consider o Were the right people and talents hired? o Did you establish appropriate planning and control systems? o Did the project conform to plan? Why or why not?

What lessons did you learn that you would pass on to future project teams?

TheSimulation

What did you learn about project management from participating in the SimProject simulation?


Appendix F

SimProject – About the Simulation

SimProject, an Engaging Experience

Inmakingthecaseforeducationalgames,Quinn(2005)suggeststhefocus

shouldbeondesigninglearnerenvironmentsandexperiencesratherthanjustcontent,

emphasizingyouhaveto:

[Place]thelearnersintoacontextwheretheyhavetomakedecisions,

understandwhythosedecisionsareimportant,wanttomakethosedecisions,

andknowthatthereareconsequencesofthosedecisions.Ifyoudon’t,you’re

likelytobedoomingyouraudiencetoineffective,andreally,user‐abusive

learning....[Learnersshouldbeput]intothepositionofmakingdecisionsthat

makethecontentmeaningfuland…alignedwithhowtheyneedtoapplythe

knowledge.

SimProject,thesimulationusedinthisstudy,doesthisbyrequiringtheanalysis

ofavailableresourcesandthecreationofastaffingmanagementplanwhichisthenused

asthebasisforresourceselectionduringthesimulation.Followingeachworkperiod,

performancefeedbackisanalyzedandcomparedwiththeoriginplananddecisionsare

madeforthenextworkperiod.Thissimulationwasdevelopedtoprovide“virtual‘first‐

hand’experienceinmanagingprojects”byitsdevelopers,Dr.JeffreyPintoandDr.Diane

ParenteofThePennsylvaniaStateUniversity(SimProfessionals,2009).Theyprovide

thisoverviewofSimProject:

Withtherapidriseofcomputersandadvancedtechnologyintheclassroomhave

comenewpedagogicalapproachestoteaching.Oneofthemostpowerfulofthese

approachesistheuseofcomputersimulationstodemonstrate“realworld”

businesspractices,theinter‐relatednessofvariousbusinessfunctions,andthe

roleofcompetitivedecision‐makinginbusiness.Computersimulations

encourageteamdevelopment,collaboration,globalthinking,andapredilection

toconsidertheramificationsofdecisionsandtheireffectonthebottomline—in

otherwords,manyoftheskillsthatareusefultoprojectmanagersandteam


membersinbusiness.Thepurposeofthissimulationistotietogethermanyof

thesalientchallengesofprojectmanagementinordertogivestudentsthe

deepestpossibleunderstandingofthecomplexitiesinvolvedinundertakinga

project.Thegoalofthesimulationwillbetohavestudentsmanageaprojectfrom

initiationtocompletion.Withinthisframeworkthestudentwillneedtoemploy

anddevelopskillspertinenttopersonnelselectionandtraining,motivation,

conflictmanagement,andstakeholdermanagement.Studentswillberequiredto

useplanningandschedulingtechniques,suchasworkbreakdownstructures,

PERT/CPM,scopedevelopment,andriskanalysis.Thetopicalcoveragewillhave

approximatecoincidencewiththeProjectManagementInstitute’sBodyof

Knowledge(PMBOK),insuringthatthestudentsgainexposuretothosetopics

recognizedbythekeyprofessionalorganizationforprojectmanager.(Pinto&

Parente,2003)

Playing the game.

Thissimulationgameistypicallyplayedbystudentteamswhodevelopaplan

involvingfourtypesofdecisions:resourcehiringandrelease,resourcetrainingto

improveexpectedperformance,managerialactionstoinfluenceresourceperformance,

andassignmentofresourcestoactivities(Pinto&Parente,2003).Thestudentteams

competeagainsteachotherforacquisitionofresourcesfromacommonresourcepool

basedontheiranalysisofgiveninformation:

Fixedworkbreakdownstructurewithfixedsequencingofactivities

Eachactivityassignedtooneoftwelveprojectphasesorworkperiods

Estimateddurationsandcostbudgetsforeachactivity

Resourcepoolwithsufficientresourcesofeachcategoryforallteams,but

notwithequalperformancecharacteristics

Trainingcatalogofavailablecourses,durations,andcosts

Managerialactioncatalogwithcosts(Pinto&Parente,2003)

Thestudentteamsdecidewhichresourcestobidonbasedonanalysisofthe

giveninformationwhichincludesthefollowingdemographicinformationandmetrics

foreachavailableresource(Pinto&Parente,2003):


Demographicinformation

o Resourcename

o Category(Engineer,JuniorMarketingSpecialist,JuniorProduct

Designer,MarketingManager,OperationsSpecialist,Project

Manager,QualityEngineer,SeniorProductDesigner)

o Standardandovertimepayrates(overtimeisnotallowed)

o Age

o Gender

o Culturalbackground(American,Chinese,German,Japanese,

Spanish)

Metrics

o Training

o Skilllevel

o Experience

o Education

o Reputation

o WorkEthic

o PublicRelations

o Flexibility

o InterpersonalSkill

Resourcemetricinformationisgivenasapercentagebetweenzeroand100.

Resourceefficiencyontasksisbasedonanundisclosedformulainvolvingthevaluesof

thesemetrics,theresourcedemographicsandthecompositionofthesimulatedproject

team(Pinto&Parente,2003).Typicallytaskstakelongerthanthegivenduration

estimateifonlyoneresourceisassignedtothetask.

Priortothestartofthesimulationthestudentteamsengagein“preplay”tobid

onandobtaintheirstartingresources.Intheeventoneormoreteamsdoesnotwin

theirbidsandobtaintheneededresources,thepreplayroundisrepeateduntileach

teamissatisfiedtheyhaveacquiredtheresourcestheyneedtogetstarted.Oncethe

virtualprojectteamsarestaffed,thestudentteamsentertheirdecisionsandthe

instructoractivatesthesimulationengineforthefirstworkperiod.


Atthecompletionofeachsimulatedworkperiod,thestudentteamsreceive

projectsummaryinformationandstatisticsandtaskactualsinformation.Theproject

summaryinformationincludes:workperiodfinishdate,projectbeginningbudget,work

periodcosts,anycostadjustmentsresultingfromtheoccurrenceofunforeseenevents,

remainingbudgetandpercentagescoresforteamefficiency,cohesion,compositionand

longevity.Projectstatisticsreporttheteam’sperformancerelativetotheotherstudent

teamsaspercentilesinfourcategories:cost,time,functionalityandstakeholder

satisfaction.Regardlessofhowwelleachteamisdoing,theteamwiththebest

performanceineachofthesefourcategoriesisshownwithascoreof100%andtheteam

withtheworstperformancereceivesascoreof0%.Teamsinthemiddlereceiveascore

indicativeoftheirstandingrelativetotheotherteams.Forexample,allteamscould

havefinishedtheworkperiodaheadofthebaselineschedule.Theteamfinishingthe

earliestwouldreceiveatimescoreof100%andtheteamfinishinglatestwouldreceivea

scoreof0%eventhoughtheybothfinishedearlyandregardlessofthedifferenceintheir

finishdates(Pinto&Parente,2003).

Taskactualsperformancedataisprovidedforeachresourcebyactivitylisting

percentagesforeffectiveness,allocation,andefficiency;hoursworked;andcost.

Changesinresourcemetricvaluesindicatetheresultsoftraining,managerialactions,

eventsandteammorale(Pinto&Parente,2003).

Followingeachworkperiod,studentteamsreviewtheirresults,adjusttheirplan

asrequired,andsubmittheirfourtypesofdecisionsforthenextworkperiod(resource

bidsandreleases,training,managerialactionsandtaskassignments).

Thereportedwinnerofthesimulationgameistheteamreceivingtheoverall

percentilescoreof100%.


Appendix G

Descriptive Statistics for Variable Components

TABLE28‐DESCRIPTIVESTATISTICSFORPRESIMULATIONKNOWLEDGEVARIABLE(K1)

Descriptive Statistics

N Minimum Maximum Mean Std. Deviation Variance K1_ORGDESGN 28 3 7 4.68 .945 .893

K1_PROJPLAN 28 4 7 5.29 .937 .878

K1_PROJRISK 28 3 7 4.89 1.197 1.433

K1_ESTSCOPE 28 2 7 5.04 1.138 1.295

K1_SEQACT 28 3 7 5.61 .916 .840

K1_ESTDUR 28 4 7 5.11 .832 .692

K1_BUDGETNG 28 2 7 4.71 1.182 1.397

K1_RESALLOC 28 3 7 5.29 1.084 1.175

K1_CHGMGT 28 3 7 4.89 1.066 1.136

K1_DESPERFMEAS 28 3 7 4.57 1.260 1.587

K1_EFFLDRSHP 28 4 7 5.32 .670 .448

K1_TEAMCONS 28 4 7 5.50 .923 .852

K1_NEGOT 28 3 7 5.14 .970 .942

K1_EVM 28 2 7 4.36 1.393 1.942

K1_EVALPERS 28 3 7 5.36 1.026 1.053

K1_MNGUNCERT 28 3 6 4.79 .833 .693

K1 28 3.50 6.25 5.033 .710 .504


TABLE29‐DESCRIPTIVESTATISTICSFORPRESIMULATIONCONFIDENCEVARIABLE(C1)


N Minimum Maximum Mean Std. Deviation Variance C1_ORGDESGN 28 2 7 4.46 1.036 1.073

C1_PROJPLAN 28 3 7 5.25 1.110 1.231

C1_PROJRISK 28 2 6 4.50 1.291 1.667

C1_ESTSCOPE 28 2 7 4.86 1.297 1.683

C1_SEQACT 28 3 7 5.11 1.031 1.062

C1_ESTDUR 28 2 7 4.82 1.156 1.337

C1_BUDGETNG 28 2 6 4.50 1.171 1.370

C1_RESALLOC 28 3 7 5.11 1.197 1.433

C1_CHGMGT 28 2 7 4.75 1.236 1.528

C1_DESPERFMEAS 28 2 7 4.43 1.200 1.439

C1_EFFLDRSHP 28 4 7 5.25 .799 .639

C1_TEAMCONS 28 4 7 5.29 1.049 1.101

C1_NEGOT 28 3 7 4.93 1.016 1.032

C1_EVM 28 1 7 3.93 1.538 2.365

C1_EVALPERS 28 3 7 5.14 1.079 1.164

C1_MNGUNCERT 28 3 7 4.75 1.005 1.009

C1 28 3.25 6.19 4.817 .809 .654

TABLE30–DESCRIPTIVESTATISTICSFORPRESIMULATIONTEAMEXPERIENCE(T1)


N Minimum Maximum Mean Std. Deviation Variance T1_WRKLD 28 1 7 5.07 1.844 3.402

T1_COOP 28 4 7 5.82 1.218 1.484

T1_EFFIC 28 4 7 5.68 1.056 1.115

T1_EQPART 28 1 7 4.50 1.915 3.667

T1_PLEAS 28 1 7 5.79 1.475 2.175

T1_DISAGR 28 4 7 5.86 .970 .942

T1_MOTIV 28 1 7 5.39 1.524 2.321

T1_SATIS 28 2 7 6.00 1.122 1.259

T1_WILLNG 28 2 7 5.32 1.847 3.411

T1 28 3.1 7.0 5.5 1.2 1.388


TABLE31‐DESCRIPTIVESTATISTICSFORPRESIMULATIONTEAMEXPERIENCE(TP1)



T1_COOP 28 4 7 5.82 1.218 1.484

T1_EFFIC 28 4 7 5.68 1.056 1.115

T1_EQPART 28 1 7 4.50 1.915 3.667

T1_PLEAS 28 1 7 5.79 1.475 2.175

T1_DISAGR 28 4 7 5.86 .970 .942

T1_MOTIV 28 1 7 5.39 1.524 2.321

T1_SATIS 28 2 7 6.00 1.122 1.259

T1_WILLNG 28 2 7 5.32 1.847 3.411

S1_TEAM 28 2 7 5.39 1.663 2.766

TC1 28 3.1 7.0 5.482 1.1858 1.406

TABLE32‐DESCRIPTIVESTATISTICSFORPRESIMULATIONGENERICTEAMATTITUDE(TG1)


N Minimum Maximum Mean Std. Deviation Variance S1_GRP 28 3 7 5.50 1.202 1.444

S1_EGRPS 28 3 7 5.82 1.056 1.115

TG1 28 4 7 5.66 .943 .890

TABLE33‐DESCRIPTIVESTATISTICSFORPRESIMULATIONGENERICSIMULATIONATTITUDE

(SG1)


N Minimum Maximum Mean Std. Deviation Variance S1_APPL 28 2 7 5.32 1.565 2.448

S1_VAL 28 2 7 5.18 1.517 2.300

S1_DEC 28 2 7 5.21 1.618 2.619

S1_FUN 28 3 7 5.57 1.103 1.217

S1_HELP 28 2 7 5.11 1.449 2.099

S1_MORE 28 3 7 5.57 1.289 1.661

SG1 28 2.833 7.000 5.327 1.230 1.513


TABLE34‐DESCRIPTIVESTATISTICSFORPRESIMULATIONPROJECTSIMULATIONATTITUDE

(SP1)


N Minimum Maximum Mean Std. Deviation Variance S1_VAL 28 2 7 5.68 1.467 2.152

S1_UND 28 3 7 5.61 1.286 1.655

S1_PERF 28 2 7 5.11 1.286 1.655

S1_ENJOY 28 2 7 5.39 1.524 2.321

S1_CONFID 28 2 7 5.18 1.565 2.448

S1_EDUC 28 3 7 5.79 1.197 1.434

S1_FUN 28 3 7 5.43 1.200 1.439

S1_HARD 28 1 7 4.75 1.974 3.898

SP1 28 3.125 7.000 5.36607 1.000537 1.001

TABLE35‐DESCRIPTIVESTATISTICSFORPRESIMULATIONTECHNICALKNOWLEDGE

APPLICATION(N1,E1,P1)


N Minimum Maximum Mean Std. Deviation Variance N1 28 0 4 2.18 1.565 2.448

E1 28 0 6 1.68 1.611 2.597

P1 = N1 + E1 28 0 9 3.86 2.549 6.497


TABLE36‐DESCRIPTIVESTATISTICSFORPOSTSIMULATIONKNOWLEDGEPERCEPTION(K2)


N Minimum Maximum Mean Std. Deviation Variance K2_ORGDESGN 25 3 7 5.57 .955 .913

K2_PROJPLAN 25 3 7 5.72 .891 .793

K2_PROJRISK 25 3 7 5.32 1.108 1.227

K2_ESTSCOPE 25 3 7 5.52 .963 .927

K2_SEQACT 25 5 7 6.08 .640 .410

K2_ESTDUR 25 3 7 5.36 1.114 1.240

K2_BUDGETNG 25 3 7 5.20 .957 .917

K2_RESALLOC 25 4 7 5.88 .927 .860

K2_CHGMGT 25 3 7 5.40 .957 .917

K2_DESPERFMEAS 25 3 7 5.00 1.041 1.083

K2_EFFLDRSHP 25 4 7 5.76 .879 .773

K2_TEAMCONS 25 4 7 6.00 .816 .667

K2_NEGOT 25 3 7 5.56 .917 .840

K2_EVM 25 2 7 4.84 1.405 1.973

K2_EVALPERS 25 3 7 5.48 .963 .927

K2_MNGUNCERT 25 4 7 5.32 .690 .477

K2 25 4 6 5.50 .661 .437

TABLE37‐DESCRIPTIVESTATISTICSFORPOSTSIMULATIONCONFIDENCEPERCEPTION(C2)


N Minimum Maximum Mean Std. Deviation Variance C2_ORGDESGN 25 3 7 5.52 1.005 1.010

C2_PROJPLAN 25 3 7 5.68 .988 .977

C2_PROJRISK 25 3 7 5.12 1.054 1.110

C2_ESTSCOPE 25 3 7 5.56 1.083 1.173

C2_SEQACT 25 4 7 5.84 .850 .723

C2_ESTDUR 25 4 7 5.40 .957 .917

C2_BUDGETNG 25 3 7 5.04 1.020 1.040

C2_RESALLOC 25 3 7 5.80 1.000 1.000

C2_CHGMGT 25 3 7 5.28 1.173 1.377

C2_DESPERFMEAS 25 2 7 4.96 1.207 1.457

C2_EFFLDRSHP 25 4 7 5.76 .831 .690

C2_TEAMCONS 25 3 7 5.68 .945 .893

C2_NEGOT 25 4 7 5.54 .865 .749

C2_EVM 25 2 7 4.48 1.558 2.427

C2_EVALPERS 25 3 7 5.48 1.005 1.010

C2_MNGUNCERT 25 3 7 5.32 .988 .977

C2 25 4 7 5.40 .786 .618


TABLE38‐DESCRIPTIVESTATISTICSFORPOSTSIMULATIONTEAMEXPERIENCE(T2)



T2_COOP 25 2 7 5.92 1.187 1.410

T2_EFFIC 25 3 7 5.80 1.190 1.417

T2_EQPART 25 1 7 4.92 1.681 2.827

T2_PLEAS 25 3 7 6.08 1.256 1.577

T2_DISAGR 25 2 7 6.00 1.323 1.750

T2_MOTIV 25 2 7 5.92 1.288 1.660

T2_SATIS 25 3 7 6.00 1.291 1.667

T2_WILLNG 25 2 7 5.40 1.780 3.167

T2 25 2.67 7.00 5.7111 1.16843 1.365

TABLE39‐DESCRIPTIVESTATISTICSFORPOSTSIMULATIONTEAMEXPERIENCE(TP2)



T2_COOP 25 2 7 5.92 1.187 1.410

T2_EFFIC 25 3 7 5.80 1.190 1.417

T2_EQPART 25 1 7 4.92 1.681 2.827

T2_PLEAS 25 3 7 6.08 1.256 1.577

T2_DISAGR 25 2 7 6.00 1.323 1.750

T2_MOTIV 25 2 7 5.92 1.288 1.660

T2_SATIS 25 3 7 6.00 1.291 1.667

T2_WILLNG 25 2 7 5.40 1.780 3.167

S2_TEAM 25 1 7 5.36 1.912 3.657

TP2 25 2.5 7.0 5.676 1.2248 1.500

TABLE40‐DESCRIPTIVESTATISTICSFORPOSTSIMULATIONGENERICTEAMWORKATTITUDE

(SG2)


N Minimum Maximum Mean Std. Deviation Variance S2_GRP 25 3 7 5.92 .997 .993

S2_EGRPS 25 5 7 6.12 .726 .527

TG2 25 4 7 6.02 .757 .573


TABLE41‐DESCRIPTIVESTATISTICSFORPOSTSIMULATIONGENERICSIMULATIONATTITUDE

(SG2)


N Minimum Maximum Mean Std. Deviation Variance S2_APPL 25 2 7 5.04 1.338 1.790

S2_VAL 25 2 6 4.44 1.356 1.840

S2_DEC 25 2 7 5.03 1.209 1.461

S2_FUN 25 3 7 6.08 1.115 1.243

S2_HELP 25 2 7 5.36 1.221 1.490

S2_MORE 25 2 7 5.70 1.399 1.958

SG2 25 3 7 5.27 1.117 1.247

TABLE42‐DESCRIPTIVESTATISTICSFORPOSTSIMULATIONPROJECTSIMULATIONATTITUDE

(SP2)


N Minimum Maximum Mean Std. Deviation Variance S2_VAL 25 2 7 5.56 1.356 1.840

S2_UND 25 2 7 5.24 1.300 1.690

S2_PERF 25 2 7 5.60 1.500 2.250

S2_ENJOY 25 2 7 5.92 1.352 1.827

S2_CONFID 25 1 7 4.76 1.690 2.857

S2_EDUC 25 2 7 5.60 1.384 1.917

S2_FUN 25 3 7 5.88 1.092 1.193

S2_HARD 25 1 7 3.64 1.604 2.573

SP2 25 3 7 5.28 1.026 1.053

TABLE43‐DESCRIPTIVESTATISTICSFORPOSTSIMULATIONTECHNICALKNOWLEDGE

APPLICATION(N2,E2,P2)


N Minimum Maximum Mean Std. Deviation Variance N2 25 0 4 2.28 1.400 1.960

E2 25 0 6 2.28 1.768 3.127

P2 25 1 9 4.56 2.347 5.507


References

References

AACSBAccreditationCoordinatingCommittee,&AACSBAccreditationQuality

Committee.(2007).AACSBassuranceoflearningstandards:Aninterpretation.

Retrieved5/27,2011,from

http://www.aacsb.edu/publications/whitepapers/AACSB_Assurance_of_Learning.p

df

ABSEL.(2011).AboutABSEL.Retrieved7/21,2011,from

http://absel2011.wordpress.com/about

Adobor,H.,&Daneshfar,A.(2006).Managementsimulations:Determiningtheir

effectiveness.TheJournalofManagementDevelopment,25(2),151‐168.

Ahn,J.(2008).Applicationofexperientiallearningcycleinlearningwithabusiness

simulationgame.(UnpublishedDoctorofEducation).ColumbiaUniversity,

Al‐Jibouri,S.,Mawdesley,M.,Scott,D.,&Gribble,S.(2005).Theuseofasimulationmodel

asagameforteachingmanagementofprojectsinconstruction.TheInternational

JournalofEngineeringEducation,21(6),1195.

Al‐Jibouri,S.H.,&Mawdesley,M.J.(2001).Designandexperiencewithacomputergame

forteachingconstructionprojectplanningandcontrol.Engineering,Construction

andArchitecturalManagement,8(5/6),418.

Alreck,P.L.,&Settle,R.B.(1995).Thesurveyresearchhandbook(2nded.).Chicago:

Irwin.


AmericanPsychologicalAssociation.(2010).Publicationmanualoftheamerican

psychologicalassociation(6th[correct2printinged.).Washington,DC:American

PsychologicalAssociation.

Anbari,F.T.(2010).Distanceeducationinprojectmanagement.Paperpresentedatthe

PMIGlobalCongress2010NorthAmerica,Washington,D.C.

Anderson,P.H.,&Lawton,L.(1988).Assessingstudentperformanceonabusiness

simulationexericse.DevelopmentsinBusinessSimulation&ExperientialExercises,15

Anderson,P.H.,&Lawton,L.(1990).Therelationshipbetweenfinancialperformance

andothermeasuresoflearningonasimulationexercise.DevelopmentsinBusiness

Simulation&ExperientialExercises,17

Anderson,P.H.,&Lawton,L.(1992).Asurveyofmethodsusedforevaluatingstudent

performanceonbusinesssimulations.Simulation&Gaming,23(4),490‐498.doi:

10.1177/1046878192234010

Anderson,P.H.,&Lawton,L.(1997).Demonstratingthelearningeffectivenessof

simulations:Whereweareandwhereweneedtogo.DevelopmentsinBusiness

Simulation&ExperientialLearning,24

Anderson,P.H.,&Lawton,L.(2003).Theopimaltimingforintroducingbusiness

simulations.DevelopmentsinBusinessSimulation&ExperientialLearning,30

Anderson,P.H.,&Lawton,L.(2007).Simulationperformanceanditseffectivenessasa

PBLproblem:Afollow‐upstudy.DevelopmentsinBusinessSimulation&Experiential

Learning,34


Anderson,P.H.,&Lawton,L.(2009).Businesssimulationsandcognitivelearning:

Developments,desires,andfuturedirections.Simulation&Gaming,40(2),193‐216.

doi:10.1177/1046878108321866

Anderson,P.H.,Lawton,L.,&Wellington,W.J.(2008).Goalorientationandsimulation

performance.DevelopmentsinBusinessSimulation&ExperientialLearning,35,329‐

335.

Anderson,J.R.(2005).Therelationshipbetweenstudentperceptionsofteamdynamics

andsimulationgameoutcomes:Anindividual‐levelanalysis.JournalofEducationfor

Business,81(2),85‐90.

Ashleigh,M.,Ojiako,U.,Chipulu,M.,&Wang,J.K.(2012).Criticallearningthemesin

projectmanagementeducation:Implicationsforblendedlearning.International

JournalofProjectManagement,30(2),153‐161.doi:

10.1016/j.ijproman.2011.05.002

Baglione,S.L.,&Tucci,L.A.(2010).Usingamarketingcomputersimulationtoimprove

learninginaprinciplesofmarketingcourse.ReviewofBusinessResearch,10(4),47‐

57.

Biggs,W.D.(1990).Introductiontocomputerizedbusinessmanagementsimulations.In

J.W.Gentry(Ed.),Guidetobusinessgamingandexperientiallearning(pp.23‐35).

EastBrunswick,NJ/London:Nichols/PB.

Burns,R.B.,&Burns,R.A.(2008).BusinessresearchmethodsandstatisticsusingSPSS.

ThousandOaks,CA;London:SAGE.


Buzzetto‐More,N.,&Mitchell,B.C.(2009).Studentperformanceandperceptionsina

web‐basedcompetitivecomputersimulation.InterdisciplinaryJournalofE‐Learning

andLearningObjects,5,73‐90.

Campbell,D.T.,&Stanley,J.C.(1963).Experimentalandquasi‐experimentaldesignsfor

research.Chicago:R.McNally.

Chin,J.,Dukes,R.,&Gamson,W.(2009).Assessmentinsimulationandgaming:Areview

ofthelast40years.Simulation&Gaming,40(4),553‐568.doi:

10.1177/1046878108327585

Clark,R.C.,&Mayer,R.E.(2008).E‐learningandthescienceofinstruction:Proven

guidelinesforconsumersanddesignersofmultimedialearning(2nded.).San

Francisco,CA:Pfeiffer.

Collofello,J.S.(2000).University/industrycollaborationindevelopingasimulation‐

basedsoftwareprojectmanagementtrainingcourse.Education,IEEETransactions

on,43(4),389‐393.

Cook,D.L.,&Granger,J.C.(1976).Currentstatusofprojectmanagementinstructionin

americancollegesanduniversities.TheAcademyofManagementJournal,19(2),

323‐328.

Cook,L.S.,&Olson,J.R.(2006).Thesky'sthelimit:Anactivityforteachingproject

management.JournalofManagementEducation,30(3),404‐420.

Cook,T.D.,&Campbell,D.T.(1979).Quasi‐experimentation:Design&analysisissuesfor

fieldsettings.Boston:HoughtonMifflin.


Cooper,D.R.,&Schindler,P.S.(2006).Businessresearchmethods(9thed.).Boston:

McGrawHill.

Cooper,J.M.(2011).Crisismanagementandprojectleadershipskills:Assessingthe

educationalvalidityofaprojectmanagementsimulation.Unpublishedmanuscript.

Creswell,J.W.(2009).Researchdesign:Qualitative,quantitative,andmixedmethod

approaches[null](3rded.).ThousandOaks,CA:Sage.

Creswell,J.W.(2010).Mappingthedevelopinglandscapeofmixedmethodsresearch.In

A.Tashakkori,&C.Teddlie(Eds.),Handbookofmixedmethodsinsocial&behavioral

research(2nded.,pp.45‐68).ThousandOaks,CA:ThousandOaks,CA:SAGE

Publications.

Cronan,T.P.,Léger,P.,Robert,J.,Babin,G.,&Charland,P.(2012).Comparingobjective

measuresandperceptionsofcognitivelearninginanERPsimulationgame.

Simulation&Gaming,43(4),461‐480.doi:10.1177/1046878111433783

Crookall,D.(2010).Seriousgames,debriefing,andSimulation/Gamingasadiscipline.

Simulation&Gaming,41(6),898‐920.

Czaja,R.,&Blair,J.(2005).Designingsurveys:Aguidetodecisionsandprocedures.

ThousandOaks,CA:SagePublications,Inc.

Dantas,A.R.,Barros,M.O.,&Werner,C.M.L.(2004).Asimulation‐basedgamefor

projectmanagementexperientiallearning.Proceedingsofthe16thAnnual

InternationalConferenceonSoftwareEngineering&KnowledgeEngineering(SEKE),

Banff,Canadá,


Davidovitch,L.,Parush,A.,&Shtub,A.(2006).Simulation‐basedlearninginengineering

education:Performanceandtransferinlearningprojectmanagement.Journalof

EngineeringEducation,95(4),289‐299.

Davidovitch,L.,Parush,A.,&Shtub,A.(2008).Simulation‐basedlearning:Thelearning–

forgetting–relearningprocessandimpactoflearninghistory.Computers&

Education,50(3),866‐880.doi:DOI:10.1016/j.compedu.2006.09.003

Davidovitch,L.,Parush,A.,&Shtub,A.(2009).Theimpactoffunctionalfidelityin

simulator‐basedlearningofprojectmanagement.InternationalJournalof

EngineeringEducation,25(2),333‐340.

Davidovitch,L.,Parush,A.,&Shtub,A.(2010).Simulator‐basedteamtrainingtoshare

resourcesinamatrixstructureorganization.EngineeringManagement,IEEE

Transactionson,57(2),288‐300.

Davidovitch,L.,Shtub,A.,&Parush,A.(2007).Projectmanagementsimulation‐based

learningforsystemsengineeringstudents.SystemsEngineeringandModeling,2007.

ICSEM'07.InternationalConferenceon,17‐23.

Davis&Dean.(2011a).Davis&dean:Amilestonemanagementcompany.Retrieved

01/04,2013,fromhttp://www.davisdean.com

Davis&Dean.(2011b).Whattheresearchsays.Retrieved01/05,2013,from

http://www.davisdean.com/what‐the‐research‐says.html

Dennis,E.E.,&Smith,S.P.(2006).Findingthebestbusinessschoolforyou:Lookingpast

therankings.Westport,CT:Praeger.


Dillman,D.H.,&Cook,D.L.(1969).SimulationinthetrainingofR&Dprojectmanagers.

[Washington,D.C.]:DistributedbyERICClearinghouse,.

Dukes,R.L.,&Seidner,C.J.(Eds.).(1978).Learningwithsimulationsandgames.Beverly

Hills,Calif.:SagePublications.

Faria,A.J.(1998).Businesssimulationgames:Currentusagelevels‐‐anupdate.

Simulation&Gaming,29(3),295‐308.doi:10.1177/1046878102238606

Faria,A.J.,Hutchinson,D.,Wellington,W.J.,&Gold,S.(2009).Developmentsinbusiness

gaming:Areviewofthepast40years.Simulation&Gaming,40(4),464‐487.doi:

10.1177/1046878108327585

Faria,A.J.,&Wellington,W.J.(2004).Asurveyofsimulationgameusers,former‐users,

andnever‐users.Simulation&Gaming,35(2),178‐207.doi:

10.1177/1046878108322225

Faria,A.J.,&Whiteley,T.R.(1990).ANEMPIRICALEVALUATIONOFTHEPEDAGOGICAL

VALUEOFPLAYINGASIMULATIONGAMETNAPRINCIPLESOFMARKETING

COURSE.DevelopmentsinBusinessSimulation&ExperientialExercises,,1753‐57.

Feinstein,A.H.,&Cannon,H.M.(2002).Constructsofsimulationevaluation.Simulation

&Gaming,33(4),425‐440.doi:10.1177/1046878102238606

Field,A.P.(2009).DiscoveringstatisticsusingSPSS(3rded.).London:SAGE.

Fowler,F.J.(2009).Surveyresearchmethods(4thed.).ThousandOaks,CA:Sage

Publications.


Fritzche,D.J.,&Cotter,R.V.(1990).Guidelinesforadministeringbusinessgames.InJ.W.

Gentry(Ed.),Guidetobusinessgamingandexperientiallearning(pp.74‐89).East

Brunswick,NJ/London:Nichols/PB.

Gamlath,S.L.(2009).Fieldtestingtwosimulationgames:Dowinnerswinconsistently?

OntheHorizon,17(4),388‐396.

Gentry,J.V.,&Burns,A.C.(1981).Operationalizingatestofamodeloftheuseof

simulationgamesandexperientialexercises.DevelopmentsinBusinessSimulation&

ExperientialExercises,8,48‐52.

Gentry,J.W.(Ed.).(1990a).Guidetobusinessgamingandexperientiallearning.East

Brunswick,NJ/London:Nichols/GP.

Gentry,J.W.(1990b).Whatisexperientiallearning?InJ.W.Gentry(Ed.),Guideto

businessgamingandexperientiallearning(pp.9‐20).EastBrunswick,NJ/London:

Nichols/PB.

Gentry,J.W.,&Burns,A.C.(1997).Thoughtsaboutthemeasurementoflearning:The

caseforguidedlearningandassociatedmeasurementissues.Developmentsin

BusinessSimulation&ExperientialLearning,24,241‐246.

Gosen,J.,&Washbush,J.(2004).Areviewofscholarshiponassessingexperiential

learningeffectiveness.Simulation&Gaming,35(2),270‐293.doi:

10.1177/1046878108322225


Gosenpud,J.(1990).Evaluationofexperientiallearning.InJ.W.Gentry(Ed.),Guideto

businessgamingandexperientiallearning(pp.301‐329).EastBrunswick,

NJ/London:Nichols/PB.

Gosenpud,J.,&Washbush,J.(1994).Whatsimulationusersthinkplayersshouldbe

learningfromsimulations.DevelopmentsinBusinessSimulation&Experiential

Learning,21,96‐99.

Gosenpud,J.,&Washbush,J.(2010).Therelatonshipbetweenlearningandperformance

inatotalenterprisesimulation:Revisitedandnewdata.DevelopmentsinBusiness

Simulation&ExperientialLearning,37,95‐98.

Graf,L.A.(2001).ABSEL'scontributionstoexperientialLearning/Experientialexercises:

Thedecadeofthe1970s.Simulation&Gaming,32(1),40‐65.doi:

10.1177/1046878102238606

Gray,C.F.,&Larson,E.W.(2003).Projectmanagement:Themanagerialprocess(2nd

ed.).NewYork:McGraw‐Hill/Irwin.

Gray,C.F.,&Larson,E.W.(2008).Projectmanagement:Themanagerialprocess(4thed.).

NewYork:McGraw‐Hill/Irwin.

Graziano,R.(2003).Thevirtualenterprisesimulation:Students'perceptionsofan

experiential,activelearningstrategyforbusinessandcareereducation.Hofstra

University)..(UMINumber:3088556).


Green,J.C.(2004).Studentreactionstotheuseofacomputer‐basedsimulationasan

integratingmechanismforaMBAcurriculum.DevelopmentsinBusinessSimulation

andExperientialLearning,,31286‐289.

Greenlaw,P.S.,&Wyman,F.P.(1973).Theteachingeffectivenessofgamesincollegiate

businesscourses.Simulation&Games,4(3),259‐294.

Hall,O.P.,&Ko,K.(2008).MeasuringMBAlearningoutcomesusingbusinesssimulations

Workingpaperseries,PepperdineUniversity.

Hays,R.T.(2005).Theeffectivenessofinstructionalgames:Aliteraturereviewand

discussion.(No.TechnicalReport2005‐004).Orlando,FL:NavalAirWarfareCenter

TrainingSystemsDivision.

Hertel,J.P.,&Millis,B.J.(2002).Usingsimulationstopromotelearninginhigher

education.Sterling,VA:StylusPublishing,LLC.

Heyman,M.(1975).Simulationgamesfortheclassroom.Bloomington,Ind.:PhiDelta

KappaEducationalFoundation.

Hornyak,M.J.,Peach,E.B.,Bowen,A.,Moes,W.,&Wheeler,R.(2006).Examiningproject

managementinbusinesssimulations:Studentandfacultyviews.Developmentsin

BusinessSImulationandExperientialLearning,,33107‐117.

Hutcheson,J.(1984).Educatingprojectmanagersfortheconstructionindustryin

australia.InternationalJournalofProjectManagement,2(4),220‐224.doi:

10.1016/0263‐7863(84)90039‐5


Jones,K.(1987).Simulations:Ahandbookforteachersandtrainers(2nded.).London:

KoganPage.

Kayes,A.B.,Kayes,D.C.,&Kolb,D.A.(2005a).Developingteamsusingthekolbteam

learningexperience.Simulation&Gaming,36(3),355‐363.doi:

10.1177/1046878108322225

Kayes,A.B.,Kayes,D.C.,&Kolb,D.A.(2005b).Experientiallearninginteams.Simulation

&Gaming,36(3),330‐354.doi:10.1177/1046878108322225

Keys,B.(1977).Reviewoflearningresearchinbusinessgaming.ComputerSimulation

andLearningTheory,3,173‐184.

Keys,B.,&Wolfe,J.(1990).Theroleofmanagementgamesandsimulationsineducation

andresearch.JournalofManagement,16(2),307‐336.

Keys,J.B.,&Biggs,W.D.(1990).Areviewofbusinessgames.InJ.W.Gentry(Ed.),Guide

tobusinessgamingandexperientiallearning(pp.48‐73).EastBrunswick,

NJ/London:Nichols/PB.

Klein,R.D.(1980).Anempiricalevaluationoftheeffectivenessofasimulationgamein

teachinginternationalbusiness.(Doctoraldissertation,GeorgiaStateUniversity).

(ProquestDissertationandThesesdatabase).(UMINumber8120885).

Klein,R.D.(1984).Addinginternationalbusinesstothecoreprogramviathesimulation

game.JournalofInternationalBusinessStudies,15(1),151.

Kolb,D.A.(1984).Experientiallearning:Experienceasthesourceoflearningand

development.EnglewoodCliffs,NJ:Prentice‐Hall.


Krathwohl,D.R.(2002).Arevisionofbloom'staxonomy:Anoverview.Theoryinto

Practice,41(4),212‐218.

Kriz,W.C.(2010).Asystemic‐constructivistapproachtothefacilitationanddebriefing

ofsimulationsandgames.Simulation&Gaming,41(5),663‐680.

Lainema,T.,&Lainema,K.(2007).Advancingacquisitionofbusinessknow‐how:Critical

learningelements.JournalofResearchonTechnologyinEducation,40(2),183‐198.

Larréché,J.(1987).Onsimulationsinbusinesseducationandresearch.Journalof

BusinessResearch,15(6),559‐571.doi:10.1016/0148‐2963(87)90039‐7

Larson,E.W.,&Gray,C.F.(2011).Projectmanagement:Themanagerialprocess(5thed.).

NewYork:McGraw‐Hill/Irwin.

Lederman,L.C.(1984).Debriefing:Acriticalreexaminationofthepostexperience

analyticprocesswithimplicationsforitseffectiveuse.Simulation&Gaming,15(4),

415‐431.doi:10.1177/0037550084154002

Lederman,L.C.(1992).Debriefing:Towardasystematicassessmentoftheoryand

practice.Simulation&Gaming,23(2),145‐160.doi:10.1177/1046878192232003

Lederman,L.C.,&Ruben,B.D.(1984).Systematicassessmentofcommunicationgames

andsimulations:Anappliedframework.CommunicationEducation,33(2),152.

Martin,A.(2000).Asimulationengineforcustomprojectmanagementeducation.

InternationalJournalofProjectManagement,18(3),201‐213.doi:10.1016/S0263‐

7863(99)00014‐9


Mayer,B.W.,Dale,K.M.,Fraccastoro,K.A.,&Moss,G.(2011).Improvingtransferof

learning:Relationshiptomethodsofusingbusinesssimulation.Simulation&

Gaming,42(1),64‐84.doi:10.1177/1046878110376795

Mayer,R.E.(2002).Roteversusmeaningfullearning.TheoryintoPractice,41(4,Revising

Bloom'sTaxonomy),226‐232.

Maylor,,Harvey.(2001).Beyondtheganttchart:Projectmanagementmovingon.

EuropeanManagementJournal,19(1),92.

McCreery,,JohnK.(2003).Assessingthevalueofaprojectmanagementsimulation

trainingexercise.InternationalJournalofProjectManagement,21(4),233.

Megarry,J.(1978).Retrospectandprospect.InR.McAleese(Ed.),Perspectiveson

academicgaming&simulation3().London:K.Page.

Meier,R.C.,Newell,W.T.,&Pazer,H.L.(1969).Simulationinbusinessandeconomics.

EnglewoodCliffs,N.J.,:Prentice‐Hall.

Mertens,D.M.(2010).Researchandevaluationineducationandpsychology:Integrating

diversitywithquantitative,qualitative,andmixedmethods(3rd.ed.).LosAngeles:

Sage.

Mintzberg,H.(2004).ManagersnotMBAs:Ahardlookatthesoftpracticeofmanaging

anddevelopingpeople.SanFrancisco:Berrett‐Koehler.

Pate,G.S.,&Parker,H.A.(1973).Designingclassroomsimulations.Belmont,Calif.:Lear

Siegler/Fearon.


Peters,V.A.M.,&Vissers,G.A.N.(2004).Asimpleclassificationmodelfordebriefing

simulationgames.Simulation&Gaming,35(1),70‐84.doi:

10.1177/1046878108322225

Pfahl,,Dietmar.(2004).Evaluatingthelearningeffectivenessofusingsimulationsin

softwareprojectmanagementeducation:Resultsfromatwicereplicated

experiment.InformationandSoftwareTechnology,46(2),127.

Pfeffer,J.,&Fong,C.T.(2002).Theendofbusinessschools?lesssuccessthanmeetsthe

eye.AcademyofManagementLearning&Education,1(1),78‐95.

Pinto,J.K.,&Parente,D.H.(2003).SimProjectinstructorsmanualV1.2.NewYork:

McGrawHill/Irwin.

ProjectManagementInstitute.(2009).Handbookofaccreditationofdegreeprogramsin

projectmanagement(3rded.).NewtownSquare,PA:ProjectManagementInstitute.

ProjectManagementInstitute.(2010a).AboutPMI.RetrievedJuly14,2010,from

http://www.pmi.org/AboutUs/pages/About‐PMI.aspx

ProjectManagementInstitute.(2010b).Thepowerofprojectmanagement.Retrieved

8/18,2010,fromhttp://www.pmiteach.org/your‐free‐

downloads/Power_of_Project_Management_WhitePaper.pdf

ProjectManagementInstitute.(2010c).Theprojectmanagementinstituteglobal

accreditationcenterforprojectmanagementeducationprograms(GAC).Retrieved

7/14,2010,fromhttp://www.pmi.org/CareerDevelopment/Pages/Global‐

Accreditation‐Center.aspx


ProjectManagementInstitute.(2010d).Projectmanagementinstitute:Whoweareand

whatwedo.Retrieved08/30,2010,from

http://www.pmi.org/AboutUs/Pages/FactSheet.aspx

ProjectManagementInstitute.(2011).Projectmanagementinstituteprofessional(PMP)

handbook.Retrieved3/18,2011,from

http://www.pmi.org/en/Certification/~/media/PDF/Certifications/pdc_pmphand

book.ashx

ProjectManagementInstitute.(2012).DirectoryofGACaccreditedprograms.Retrieved

October31,2012,fromhttp://www.gacpm.org/Directory‐of‐GAC‐Programs.html

ProjectManagementInstitute.(2013a).Aguidetotheprojectmanagementbodyof

knowledge(PMBOKguide)(5thed.).NewtownSquare,PA:ProjectManagement

Institute,Inc.

ProjectManagementInstitute.(2013b,PMIfactfile.PMIToday:ASupplementtoPM

Network,27(1)

Quinn,C.N.(2005).Engaginglearning:Designinge‐learningsimulationgames.San

Francisco,CA:Pfeiffer.

SAGSET.(2011).Publications.Retrieved7/21,2011,fromhttp://www.sagset.org

SAGSET.(2013).Publications.Retrieved1/20,2013,fromhttp://www.sagset.org

Salas,E.,Rosen,M.A.,Held,J.D.,&Weissmuller,J.J.(2009).Performancemeasurement

insimulation‐basedtraining:Areviewandbestpractices.Simulation&Gaming,

40(3),328‐376.doi:10.1177/1046878108322225


Salas,E.,Wildman,J.L.,&Piccolo,R.F.(2009).Usingsimulation‐basedtrainingto

enhancemanagementeducation.AcademyofManagementLearning&Education,

8(4),559‐573.

Schlesinger,P.F.(1996).Teachingandevaluationinanintegratedcurriculum.Journalof

ManagementEducation,20(4),479‐499.

Schonlau,M.,Elliott,M.N.,&Fricker,R.D.(2002).Conductingresearchsurveysviae‐mail

andtheweb.SantaMonica,CA:RAND.

Seethamraju,R.(2011).Enhancingstudentlearningofenterpriseintegrationand

businessprocessorientationthroughanERPbusinesssimulationgame.Journalof

InformationSystemsEducation,22(1),19‐29.

Sekaran,U.(2003).Researchmethodsforbusiness:Askillbuildingapproach(4thed.).

Hoboken,NJ:JohnWiley&Sons.

Shtub,A.(2012).ProjectmanagementsimulationwithPTBprojectteambuilder.New

York:Springer.

SimProfessionals.(2009).Aboutthesimulation.RetrievedMarch18,2012,from

http://www.simprojectonline.com/AboutUs.aspx

Smalt,S.W.(1999).Integrationofagameintoacollegeaccountingprinciplescourse:

Studentperformanceandstudentperceptions.(DoctoralDissertation,TheUnion

Institute).(ProQuestDissertationsandThesesdatabase).(UMINo.9958873).


Snow,S.C.,Gehlen,F.L.,&Green,J.C.(2002).Differentwaystointroduceabusiness

simulation:Theeffectonstudentperformance.Simulation&Gaming,33(4),526‐

532.doi:10.1177/1046878102238606

SouthernAssociationofCollegesandSchoolsCommissiononColleges.(2009).The

principlesofaccreditation:Foundationsforqualityenhancement(4thed.).Decatur,

Georgia:SouthernAssociationofCollegesandSchoolsCommissiononColleges.

Stainton,A.J.,Johnson,J.E.,&Borodzicz,E.P.(2010).Educationalvalidityofbusiness

gamingsimulation:Aresearchmethodologyframework.Simulation&Gaming,

41(5),705‐723.doi:10.1177/1046878109353467

Starkey,K.,&Tiratsoo,N.(2007).Thebusinessschoolandthebottomline.Cambridge,UK:

CambridgeUniversityPress.

Tashakkori,A.,&Teddlie,C.(2010a).Currentdevelopmentsandemergingtrendsin

integratedresearchmethodology.InA.Tashakkori,&C.Teddlie(Eds.),Handbookof

mixedmethodsinsocial&behavioralresearch(2nded.,pp.803‐826).Thousand

Oaks,CA:ThousandOaks,CA:SAGEPublications.

Tashakkori,A.,&Teddlie,C.(2010b).Overviewofcontemporaryissuesinmixed

methodsresearch.InA.Tashakkori,&C.Teddlie(Eds.),Handbookofmixedmethods

insocial&behavioralresearch(2nded.,pp.1‐41).ThousandOaks,CA:Thousand

Oaks,CA:SAGEPublications.

Taylor,J.L.,&Walford,R.(1978).Learningandthesimulationgame.BeverlyHills,Calif.:

SagePublications.


Teach,R.D.,&Murff,E.R.T.(2007).Assessingparticipantlearninginabusiness

simulation.DevelopmentsinBusinessSimulationandExperientialLearning,,3476‐

84.

Teach,R.D.,&Murff,E.R.T.(2009).Learninginhibitorsinbusinesssimulationsand

games.DevelopmentsinBusinessSimulationandExperientialLearning,,36191‐197.

Thatcher,D.C.(1990).Promotinglearningthroughgamesandsimulations.Simulation&

Gaming,21(3),262‐273.doi:10.1177/1046878190213005

Thiagarajan,S.(1992).Usinggamesfordebriefing.Simulation&Gaming,23(2),161‐173.

Thornton,G.C.,&Cleveland,J.N.(1990).Developingmanagerialtalentthrough

simulation.AmericanPsychologist,45(2),190‐199.doi:10.1037/0003‐

066X.45.2.190

Waggener,H.A.(1979).Simulationvscasesvstext:Ananalysisofstudentopinion.

JournalofExperientialLearningandSimulation,1,113‐118.

Wellington,W.J.,Hutchinson,D.,&Faria,A.J.(2012).Anexploratorystudyoftheimpact

ofasimulationexerciseonthemanagerialandpersonalitytraitsandthedecision

makingstylesofmarketingstudents.DevelopmentsinBusinessSimulationand

ExperientialLearning,,39132‐140.

Williams,A.J.,&Williams,R.H.(2011).Multipleidentificationtheory:Attitudeand

behaviorchangeinasimulatedinternationalconflict.Simulation&Gaming,42(6),

733‐747.doi:10.1177/1046878111429228


Wolfe,J.(1985).Theteachingeffectivenessofgamesincollegiatebusinesscourses:A

1973‐1983update.SimulationGaming,16(3),251‐288.

Wolfe,J.(1990).Theevaluationofcomputer‐basedbusinessgames:Methodology,

findings,andfutureneeds.InJ.W.Gentry(Ed.),Guidetobusinessgamingand

experientiallearning(pp.279‐300).EastBrunswick,NJ/London:Nichols/PB.

Wolfe,J.(1993).Ahistoryofbusinessteachinggamesinenglish‐speakingandpost‐

socialistcountries:Theoriginationanddiffusionofamanagementeducationand

developmenttechnology.Simulation&Gaming,24(4),446‐463.doi:

10.1177/1046878193244003

Wolfe,J.(1997).Theeffectivenessofbusinessgamesinstrategicmanagementcourse

work.Simulation&Gaming,28(4),360‐276.

Yeaple,R.N.(2006).DoesitpaytogetanMBA?.Orlando,FL:TheCenterforManagement

Education.

Zantow,K.,Knowlton,D.S.,&Sharp,D.C.(2005).Morethanfunandgames:

Reconsideringthevirtuesofstrategicmanagementsimulations.Academyof

ManagementLearning&Education,4(4),451‐458.

the value and effectiveness of project management ... · the value and effectiveness of project...

Documents