the value of istqb certification v3 - rice consulting value of istqb...the value of istqb...

©2011-2016,RiceConsultingServices,Inc.

The Value of ISTQB Certification and Training

A White Paper by Randall W. Rice, CTAL (Full), CTFL-AT, CMT, CTAL-SEC

July 17, 2017

Rice Consulting Services, Inc. 1608 SW 113th Pl

Oklahoma City, OK 73170

405-691-8075

www.riceconsulting.com


2

Introduction Somethingisseriouslywrongwiththecurrentqualitylevelsinsoftware.Itseemsthattherearestoriesinthemainstreamnewsmediaalmostdailyaboutsoftware“glitches”.Oneofthese“glitches”todatehascosttheNasdaqstockexchangeover$72milliontocompensateforaperformancedefectencounteredduringtheFacebookIPO.Indescribingtheremediationofthisproblem,CNBCreportedthatNasdaq,“…hasalsocreatedtwonewpositions(chiefinformationofficerandglobalheadofmarketsystems),anengineeringteamdedicatedtomonitoringandanalyzingdailysystemperformance,andaqualityassuranceorganizationfocusedontestingthetradingsystems.

"Werecognizethatthecornerstoneofamarketisinvestorconfidence,"Greifeldsaid.(NasdaqCEORobertGreifeld)1

Theissueintoday’scomputingsystemsistrustingthatourpersonalandprivatedatawillbehandledcorrectlyandsecurely.However,thedatashowsthatsensitiveinformationisnotbeinghandledwiththecareitdeservesduetohighlevelsofsoftwaredefectsandacasualattitudetowardsoftwarequality.Overthelast40yearsormore,therehasbeenenormousworkdonetoresearchandunderstandwhatittakestodeliverreliablesoftware.Thisinformationhasbeentranslatedintopracticesandotherformstomakeitpossibletohavehighlytrustedsystems.Theproblemistoomanycompaniesfailtoseethevalueinpreventingproblems.Instead,theyarewillingtosubjecttheircustomerstorisk,losemassiveamountsofmoneyinfixingandremediatingdefectsandpayfinesforthecarelesshandlingofdata.TheCurrentStateofTraininginGeneralTheUnitedStateshasbeenlosinggroundinworkereducationforyears.Wenowranknearthebottomof23countriesintechnicalskills.2Thisisanalarmingfactandamajorslidefromwherewewereduringthequalitymovementofthe1990’s.

“Thesharplackofskillsamongworkersshowsthat‘employersneedtodomoreforcontinuingworkereducation,’anarea,saysPeterCappelli,professorofmanagementattheUniversityofPennsylvania'sWhartonSchoolofBusiness.Investmentinemployeeeducationhasbeenneglectedinrecentyearsascompanieshavebeenlesswillingtotrainupcandidatesforfearoflosingthemtocompetitors.Asaresult,programslikeapprenticeships,whicharemorecommoninEurope,arenearlydefunctintheU.S.,andnewhiresareexpectedtobefullypreparedtotakeovertheirnewduties.

‘Andcompaniesdon'twanttospendthemoney,’Cappelliadds.Nordotheywanttopayhigherwagestorecruitpeoplewiththeneededskills,hesays.”3

1http://www.cnbc.com/id/1007369152http://skills.oecd.org/skillsoutlook.html3Americanworkersarewaybehind:BlameU.S.employers–Forbesonline-http://management.fortune.cnn.com/2013/10/25/american-skills-gap/


3

However,it’snotallbadnews.Itseemsthatsomesmartcompanieshavedecidedthereisvalueinskill-buildingandareinvestingintheirpeopleagain.

“Abouthalfofbusinessessaytheyplantotrainnewhiresthisyear,upfrom39%in2013,accordingtoarecentsurveybyCareerBuilder.

"Trainingbudgetsthatwerediminishedornonexistentduringtherecessionarestartingtomakeacomeback,"saysMattFerguson,CEOofCareerBuilderandauthorofTheTalentEquation.Thefirmsurveyed1,025employersincomputers,healthcare,manufacturing,engineeringandfinance—fieldswithmanyopenskilledpositions.”4

TheValueofTrainingandCertification

Akeyquestionthatshouldbeaskedandansweredaboutanyskill-buildingeffort–training,mentoring,certification,etc.–is,“whatisthevaluetotheorganizationandtotheindividual?”.TheISTQB(InternationalSoftwareTestingQualificationsBoard)programforsoftwaretestercertificationhasbeeninexistencesince2002,andwasintroducedintheU.S.in2004bytheASTQB(AmericanSoftwareTestingQualificationsBoard).Thereareothersoftwaretestcertifications,suchasthoseofferedbytheQAIGlobalInstituteandtheInternationalInstituteofSoftwareTesting(IIST).ThispaperfocusesonlyontheISTQBcertificationandit’suniquevalueproposition.Additionally,thevalueoftraininginsoftwaretestingisalsoshown.Therearestrongcriticsoftheideaofsoftwaretestcertification.Thoseopinionscanbefoundeasilyontheweb.Thepurposeofthispaperisnottorefutethosearguments,butrathertoshowthevalueofcertificationforthoseconsideringit.Thethesisofthispaperisthatthereisvaluetobuildingastrongbasisofknowledgeinsoftwaretesting,basedonprovenpractices.Theterm“bestpractices”isalsocontroversialtosomepeople,butitisinterestingtoobservethatmanydisciplinesandprofessions–medical,engineering,etc.havepublishedbestpractices.Thesoftwareindustry,andmorespecifically,thesoftwaretestingfieldseemstobeparticularlycriticaloftheideaofbestpractices.Whateveryouchoosetocallit,theabilitytodoqualityworkinanyendeavordependsonpeopleunderstandingwhattheyaredoing,andusingthemosteffectivepracticestoperformthework.Certificationsseektocodifythisknowledgeinawaythatcanbeassessed.Itisimportanttounderstandthatanycertificationortrainingismarket-driven.Withoutthedemandforcertificationandtraining,nonewouldexist.Ultimately,itisuptoacompanyorindividualtoseekcertificationandtrainingbasedonthevaluetheyseeinit.Therearenorequirementsintheindustryforasoftwaretestertobecertifiedand/ortrained(e.g.,licensing).Ifthecasefornocertificationhasmoremeritthanthecaseforcertification,thenthecommunityofsoftwaretestersandtheirmanagerswillagreeandthemarketforcertificationwilldisappear.

4Moreemployersspendtoraiseworkers'skills–USAToday,March9,2014


4

However,thetrendforobtainingtestcertificationsissharplyincreasing.Inaddition,therequirementforjobcandidatesinsoftwaretestingisalsoincreasing.Since2004,thepaceofcertificationshascontinuedtoincreaseatastronggrowthrateintheU.S.Currently,over21,500testersholdtheCTFLintheU.S.andover450,000peopleinternationallyholdtheCTFLin70+countries.ItappearsthattheISTQBprogramisthecertificationofchoicebasedonnumbersofpeoplecertified. Basis of Findings TheinformationcontainedinthispaperisbasedinactualresultsseenfromadoptersoftheISTQBprogramintheircompanies.Also,datafromASTQB-sponsoredsurveysareshown.WhilethesesurveysaretakenfromthosealreadyinvolvedintheASTQBprogram,itishelpfultoseetheattitudesfrompeoplewhosponsorand/orholdISTQBcertifications.Inaddition,CapersJoneshasconductedresearchthatspansthousandsofsoftwareprojectsandorganizations.Hisfindingsindicatethat“Asynergisticcombinationofdefectprevention,pre-testdefectremoval,andformaltestingbycertifiedpersonnelcantop99%indefectremovalefficiencywhilesimultaneouslyloweringcostsandshorteningschedules.”5Disclaimer Theauthor,RandallW.Rice,CTAL(Full)iscurrentlyTreasureroftheSoftwareTestingQualificationsBoard.HewasalsoafounderoftheCSTEprogramfromtheQualityAssuranceInstitute(QAI)in1997.RiceConsultingconductstrainingintheISTQBFoundationandAdvancedLevelcertifications. Basis of Certification TheISTQBcertificationprogramhasFoundation,AdvancedandExpertlevels–alldefinedbycommonEnglishlanguageinternationalsyllabi:CTFL–CertifiedTesterFoundationLevel,40-questionmultiple-choiceexam,1hour,$250examfee.Formaltrainingisnotrequiredbuthighlyrecommended.CTAL–CertifiedTesterAdvancedLevelTherearefiveadvancedlevelcertifications.Threeoftheseformthe“core”–AdvancedTestManager(ATM),AdvancedTestAnalyst(ATA)andAdvancedTechnicalTestAnalyst(ATTA).Whenall3coreadvancedcertificationshavebeenachievedbyaperson,theyhavetheCTAL(Full)designation.Therearetwospecialtyadvancedcertifications–AdvancedSecurityTester(CTAL-SEC)andAdvancedTestAutomationEngineer(CTAL-TAE).Eachofthesecertificationshasdifferinglevelsofexaminationandcourselengths.LiketheCTFL,formaltrainingisnotrequiredtositfortheexam,butitisrecommended.Also,thecertification,onceachieved,isforlife.Thereisa$200examfee,3yearsofverifiablefull-5http://www.ifpug.org/Documents/Jones-SoftwareDefectOriginsAndRemovalMethodsDraft5.pdf


5

timeexperienceinsoftwareorsystemstesting,development,qualityassurance,engineeringorarelatedfield,andmustholdISTQBCertifiedTester,FoundationLevel(CTFL).ExpertLevelCurrently,therearetwoexpertlevelsofISTQBcertification–ExpertinTestManagementandExpertinTestProcessImprovement.AgileCertificationsTheCTFL-ATistheFoundationLevelforAgileTesting.Thisisaone-hourexamwith40questions.Thereisapre-requisiteofholdingtheCTFL.AdvancedlevelsforAgileTestercertificationareindevelopment.Theroadmapshownbelowshowsthesyllabicurrentlyavailable,plusthoseinactivedevelopment.Asyoucansee,theISTQBprogramisdevelopingintoaveryrobustcoverageoftestingareas.


6

The Value of ISTQB Certification Thereareothersoftwaretestercertifications,sowhatisthevalueofISTQBcertifications?Builds a Common Vocabulary of Software Testing Overtheyears,eachcompanyandeachtesterhavecometousevarioustest-relatedtermsintheirownways.Thisleadstomisunderstandingsandmiscommunicationbetweenpeopleandteamsthatcancauseprojectdelaysandmisseddefects.WehaveseenthisonmanyconsultingprojectsperformedbyRiceConsulting.Forexample,onegroupmayfailtoperformacertaintypeofleveloftestingbecausetheydidn’trealizeeverythingthedesiredtestinginvolvedduetoalackoftestterminologyknowledge.Onetestmanagerofover200peoplecreditstheISTQBFoundationLevelCertificationforasmoothprojectandearlydelivery,mainlyduetohavingacommonbasisofterminology.6 Conveys Common Practices for Software Testing TheISTQBsyllabiarenotteststandardsortestprocesses,buttheyarebasedonabroadconsensusofinternationaltestexpertsinallindustriesandteststandardsfromavarietyofsources–IEEE,ISO,BSIandothers.Byunderstandingthisbroadroadoftesting,yourteambecomesawareofprovenpracticesthatareeffectiveinsoftwaretestingandwhentouseonetechniqueoveranother.AstestmanagerandadjunctprofessorMarcRenewrites,“Certificationacrossanorganizationgetsalltestersonthesameplayingfieldwithterminology,approaches,methodology,andgeneraltestingprinciples.Certificationsfocusonfoundationaltestingknowledgeandsoundtestingpractices.Furthermore,theyallowalltesterstohaveasharedunderstandingaboutthematurityoftestingprocessesintheorganization.Certificationalsoprovidesacompetitiveedge–allowingskilledtesterstofindtherightdefectswiththerightamountofeffort.”7Usingeffectiveprocessesfortestingisamajorsteptowardhighlevelsofdefectdetection,bettersoftwareandhappierusers. Establishes a Framework for Skill Building Therearemanytopicsavailablefortraininginsoftwaretesting,butwhichonesarebestforthedevelopmentofyourteam?TheISTQBcertificationprogramlaysoutastructureoftraining,withwell-definedsyllabiandlearningobjectivesdevelopedbyinternationaltestingexperts.TheISTQBsyllabiformsastrongbasisoftopicsandtimingsthatcanbeappliedatcorporateanduniversitylevels.8

6JoeGance,formertestmanageratEchoStarpresentedthisinformationatadebateofsoftwaretestcertificationsattheASTconferenceinSeattlein20077http://www.astqb.org/why-istqb/director-quality-services-pub.php8HighereducationusingtheISTQBsyllabiincludeVillanovaandRhodeIslandCollege.


7

ThisstructurestartswithFoundationLevelasthebasis,thengrowsintobranchesofAdvancedLevelsforTestManagers,TestAnalystsandTechnicalTestAnalysts.SpecialistareasareinplaceforAgiletesting,AdvancedSecurityTestingandAdvancedTestAutomation.Asmentionedearlier,ExpertLevelsareinplacefortopicssuchasTestManagementandTestProcessImprovement. Builds Credibility and Professionalism for Your Team Asatesterortestmanager,peoplemustfindyoucredible,ortheywon’tbelieveyourmessage.Whiletestcertificationdoesnotmeansomeoneisagreattester,itdoesattesttoalevelofknowledgetheyhaveachieved.InarecentASTQBsurveyofcertifiedtestersandtestmanagers:• 96%ofmanagersfeelthatprofessionalcertificationhelpstodemonstrate

professionalcompetency.• 92%ofmanagersfeelthatprofessionalcertificationhelpstogainprofessional

recognition.• 94%ofmanagersfeelthatasoftwaretesterismorevaluabletotheorganization

afterreceivingprofessionalcertification.• 98%ofmanagersbelievethatcertificationhelpsincareerdevelopment.• 93%oftestersbelievethatISTQBAdvancedLevelCertificationhelpstesterstogain

professionalrecognition.Inaddition,94%believeithelpstesterstodemonstrateahigherlevelofprofessionalcompetency.9

Has International Recognition and Consistency Withother440,000CTFLsworldwideandover20,000CTFLsintheUnitedStates,theISTQBprogramisthemostrecognizedandadoptedtestingcertificationintheworld.Infact,theISTQBcertificationprogramis3#intheworldbehindthePMIandITILcertificationprogramsintermsofindividualscertifiedtodate.Thereareover100countriesthatparticipateintheISTQB.YoucanrestassuredthattheISTQBcertificationswillbeknownbyothers.ManyinternationalcompaniesareadoptingISTQBcertificationasawaytogetconsistenttrainingofalltheirtestteamsworldwideonacommonsyllabus.Transfers Practical Knowledge for Software Testing TheISTQBcertificationsrequirethatyoudemonstratetheabilitytoactuallyperformkeylearningobjectives.AtRiceConsulting,ourgoalisnotonlytopreparepeoplefortheISTQBexams,butalsotoimpartpracticaltestingknowledgethatwillbeusedforyearstocome.Wedothisbyincludingvalue-addedtopicsandallmajorcoursepointsarereinforcedwithexercises.

9ASTQBNewsletter,March2011andJune2011


8

TrainingisnotrequiredtotaketheISTQBexams,howeveritisstronglyencouraged.TheentireISTQBprogramdependsontrainingproviderstopromoteitandtodeliverthetrainingservices.Thisallowscandidatestochoosethetrainingvendorandmethodthatbestmeetstheirneeds.ThegoalofachievinganISTQBcertificationisnottogetlettersafteryourname–itistobuildsoftwaretestingknowledge.Thisknowledgecanbeobtainedthroughself-studyand/orthroughformaltraining.Demonstrates Investment in People and the Testing Process Oneofthekeyindicatorsofmanagementsupportofsoftwaretestingisthelevelofinvestmentinpeople,processesandtools.Managementsponsorshipofsoftwaretestingcertificationandtrainingisatangiblewaytoshowthatmanagementunderstandsandvaluesskillbuilding.Achallengeinbuildinggoodteamsiskeepingthebestpeople.SkilldevelopmentisoneofthefivemajorthingspeoplevalueintheirjobaccordingtoHumanResourcesexpertSusanHeathfield.“Employeeswanttolearnnewskills,developtheircapabilities,andgrowtheirknowledgeandcareers.Makingdevelopmentalopportunitiesavailabletoeachemployeedemonstratesyourcommitmenttohelpingthemdeveloptheircareers.Theyappreciatethis.”10 The Value of Formal Training Learning the Subtleties of ISTQB Terminology and Approaches Peoplewhohavebeeninthetestingfieldformanyyearsmayunderstandtopicsfromtheirexperienceandprevioustraining.However,peoplewithover30yearsexperiencehavefailedtheCTFLexam–notbecausetheylackknowledge,butbecausetheylacktheISTQBperspective.Smalldetailsinunderstandingcanmakethedifferencebetweenpassingandfailingtheexam.Thesyllabiprovidesthelearningobjectivesandwhatshouldbecoveredinatrainingcourse.Theydonotconveytheknowledgebehindthetopicaloutline.Thatiswhereformaltrainingcomesintoplay.Instructor Guidance and Feedback Agoodinstructorwillmakesurealllearningobjectivesarecoveredandthatyourquestionsareanswered.Withself-study,youarelargelyonyourown.Terminologyandprocessdetailsareexplainedsothecandidatewillbeabletounderstandtheconceptsandapplythemonthejob.Focused Attention

10http://humanresources.about.com/od/managementtips/qt/four_factors_b4.htm


9

Whetherinliveclassroomtrainingorine-learning,youarefocusedonthetopicofsoftwaretestingforaperiodoftime.Thisismorethanjustreadingabook.(Which,bytheway,readingbooksontestingissomethingfewtestersdo.)


10

Making the Best Use of Your Exam Investment Acandidatecanre-taketheexamatanytimeiftheydon’tpass.However,eachexamsittingis$250.Withtraining,youhaveamuchbetterchanceofpassingtheexam,plusyougetthebenefitoftestingtrainingwhichcanbeusedonthejobforyearstocome.Practical Application Anytypeofsoftwaretestingtrainingshouldincludepracticalexercisestoreinforcethekeyconcepts.Peopleonlyretain50%ofwhattheyhearandsee,but90%ofwhatwesayanddo.11AccreditedISTQBtrainingmustincludeexercisesforeachtrainingobjectivethatrequirescandidatestoperformontheexam. Analysis of Return on Investment Somepeoplemaysaythatsoftwaretestcertificationsandtrainingareexpensive.Thisbegsthequestion,comparedtowhat?Let’stakeanexampleofacompanythathas10testerstobetrainedandcertified.Atthepresenttime,eachreleaseofsoftwareatthecompanyaverages20post-releasedefectsfoundbyusersandnotbytesters.

11EdgarDale,Audio-VisualMethodsinTechnology,Holt,ReinhartandWinston


11

ResearchfromRiceConsultingshowsitisnotunusualforaproductiondefecttocostover$4,000tofind,fix,re-testandre-implement.Twentydefectsatthiscostwouldbearound$80,000perrelease.12At2016rates,aRiceConsultingISTQBFoundationLevelcoursefor10peoplewouldcostaround$13,000,includinginstructortravelexpenses.Ifthetrainingresultedinadefectreductionof20%(therewouldbe16post-releasedefectsfound),thedefectcostinthisexamplewouldbe$64,000forasavingsof$16,000.Thiswouldmorethanpayforthetrainingandcertification.However,thisjustconsidersthecostsavingsforonereleasecycle.Iftheimprovementeffortscontinueforfuturereleases,thecostsavingswouldbeseenineachrelease.Thiscouldjustifyevenmoretrainingandmoreprocessimprovement.Inaddition,it’snotuncommontooptimizetestingprocessesandreducethecostoftestingbythebetteruseoftechniques,skillsandtools.A10%costsavingsisonthelowsidefortestimprovementprocessesconductedbyRiceConsulting.Ifthetestingbudgetis$500,000peryear,testprocessoptimizationcouldsave$50,000peryear,whilefindingmoredefectsthanbefore!Thatkindofcostsavingscanpayforalotoftraining,certificationsandotherthings(suchastools).Thiskindofreturnoninvestmentisnotautomaticfromcertificationandtraining.Managementleadershipandsupportarerequiredtoimproveprocessesandleadpeopleincreatingbettersoftwareusingtheinformationgainedfromcertificationandtraining.Summary Thevalueofanythingdependsonwhatsomeoneseesinaproductorservice.Whenitcomestotestcertificationsandthetrainingneededtoachievethem,youarenotbuyingtrainingandcertifications.Youarebuyingconsistencyofsoftwaretestingknowledge,higherlevelsofcredibility,reduceddefects,moreefficientandeffectivetesting,andhappierend-users.Ofallthetestcertificationprograms,theISTQBprogramisthemostrecognizedandadopted,withover330,000certifiedtestersworldwide.IntheU.S.,thereareover12,000CTFLs,whichmeansthisisthestrongestprogrambyfarintheU.S.

12CapersJoneshasanexcellentpaperonthecostofdefectsatwww.semat.org/pub/Main/PubsandRefs/a_short_history_of_the_cost_per_defect_metric.doc.Asof2009,Joneswritesthatindustryinformationshowedthecostofpost-productionataround$5,000each.


12

Ifyoucomparethecostofcertificationandtrainingwiththecostofsoftwaredefects,itiseasytoseehowthecertificationandtrainingpayforthemselvesintermsofbettersoftwareandmoreefficienttestprocesses.Thechoiceisclear.Companiescancontinuetorisktheircustomer’sinformationandcorporateprofitsbyignoringthevalueofsoftwaretestingeducationandlivingwiththeimpactofpoorqualitysoftware,ortheycaninvestinbuildingknowledgeinsoftwaretesting,preventdefectsandincreaseprofits. About the Author

RandyRiceisaleadingauthor,speakerandconsultantinthefieldofsoftwaretestingandsoftwarequality.Rice,anISTQBCertifiedTester–AdvancedLevel(Full),andaFellowoftheLifeOfficeManagementAssociation,hasworkedwithorganizationsworldwide to improve the quality of their information systemsandoptimizetheirtestingprocesses.Mr. Rice has over 38 years experience building and testing mission-criticalprojectsinavarietyofenvironments.

RandyisadirectoroftheAmericanSoftwareTestingQualificationsBoardandhasbeenpublishedbyBetterSoftware, the JournaloftheQualityAssuranceInstitute,CrosstalkandEnterpriseSystemsJournal.HeisaregularspeakeratinternationalconferencesonsoftwaretestinginNorthAmericaandEurope,andisalsopublisherofTheSoftwareQualityAdvisor.Heisco-authorwithWilliamE.Perryofthebooks,SurvivingtheTopTenChallengesofSoftwareTestingandTestingDirtySystems.RandyservesasaDirector for theAmericanSoftwareTestingQualificationsBoard(ASTQB).HealsoservedaschairoftheQualityAssuranceInstitute’sInternationalSoftwareTestingConferencefrom1995 – 2000 andwas a foundingmember of the Certified SoftwareTest Engineer (CSTE)certificationprogram.Asauthorof2booksandover60softwaretestingtrainingcourses,RandyhashadtheprivilegeoftrainingthousandsofsoftwaretestersthroughoutNorthAmerica. [email protected]