case study: oi sa’s journey on how to structure agile testing in a bimodal it and devops approach
TRANSCRIPT
World®’16
Oi’s JourneyonHowtoStructureAgileTestinginaBimodalITandDevOpsApproachAlcino Vieira- ITManager- OI
DO5X46S
DEVOPS
2 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
©2016CA.Allrightsreserved.Alltrademarksreferencedhereinbelongtotheirrespectivecompanies.
Thecontentprovidedinthis CAWorld2016presentationisintendedforinformationalpurposesonlyanddoesnotformanytypeofwarranty. The informationprovidedbyaCApartnerand/orCAcustomerhasnotbeenreviewedforaccuracybyCA.
ForInformationalPurposesOnlyTermsofthisPresentation
3 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Abstract
RestructurethemodusoperandiofITandOperationsforAgileTestingapproachwithinanITchainwaterfall,withrestrictionsontime-to-market,ensurestabilityinproductionenvironments,datamassconstraintsandtestsnotorientedtobusinessprocesses.
Withthisfocus,atOISAherebypresentaproposalforrestructuringandrepositioningoftheQualityprocessandteam,thinkingofAgileTesting,throughdisruptivemethodologyinorganizationalculture,usingCAsolutions:CAServiceVirtualization,CATestDataManagementandAgileRequirementsDesigner.
AlcinoVieira
OIQualityAssuranceITManager
4 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Agenda
WHOISOI?
ITOI:BIGNUMBERS
NEXTCHALLENGES
CHALLENGES
ADDRESSINGTHECHALLENGES
ACHIEVEMENTS
1
2
3
4
5
6
5 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
1.WhoisOi:AConvergentTelecomServicesCompany
OiisoneofthebiggestcompaniesinBrazil,employing,directlyandindirectly,morethan147,000people.It’sstartedin1998duringtheprivatizationoftheBraziliantelecommunicationssector.Thepioneeramongitscompetitors,thecompanyoffersconvergenttelecomservices,ithasmorethan72millioncustomersaroundthecountry.
Oihasoneofthebiggestandmostdiversetelecomnetworksintheworldthatreachesalmostallthe5.570Braziliancities.OialsohasthebiggestWi-FinetworkofallofLatinAmerica,withmorethan1millionhotspots.ThebiggestfibernetworkinBrazil,whichhasabout330,000kilometers,alsobelongstoOi.
RESIDENCIALSERVICES
B2B&M2MSERVICES
PERSONALMOBILE
FIX
MOBILE
BROADBAND
PAYTV
6 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
2.ITOI:BigNumbers
+8000integrationtestcasesperformedmanuallyin30applicationsand 3200defectslogged
+600ChangeRequests/monthdeployedinproduction,where240Infrastructure,360Applications,being100fromincidents,outsidereleasetimeline
~1500ProjectsinprogressacrossITchain
196 OwnStores630 Franchiseowned1.700 PAPs(doortodoor)30.000 Callcenterpositions
42BICDRsprocessed| 9.821 servers14MM invoicedclientes|37MMderecharges
+550applications
Regressiontestswith1500 testcasesp/monthin7application:1000 automated|500manually
7 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
3.ChallengesInadditiontothecurrentchallenges,newchallengeshavearrived,andalreadyarereality…
§ Digitaltransformation|AgileTesting§ DevOps|Bimodal§ ContinuousIntegration|ContinuousDelivery
Testorientedtobusiness§ Currentlyissystems-oriented,withfocusonreducing
redundancies,increasingqualityandefficiency.§ TraceabilitybetweenTestsMapandMonitoringMap§ NewresponsibilityofQALeader/BusinessAnalyst
TestDataand ServiceVirtualization§ Reducetimeandcost,increasedsecurityandquality§ Maintenanceoftestscripts
IncreasedTestCoverage§ +applications|+testcases|+teststypes§ inlesstime
ReduceandIntegratetheReleasePipeline
§ AgilewithWaterfall– legacy
§ Frombimonthlytomonthlyorbiweekly
…and,allthishavetointegratedwithOI
ecosystem
CHALLENGES
TestAutomation§ Acceptance§ Functional§ Integration§ Unit
ReleaseMgmt.Automation
8 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
§ Digitaltransformation|AgileTesting§ DevOps|Bimodal§ ContinuousIntegration|ContinuousDelivery
4.AddressingTheChallenges(1/2)
Testorientedtobusiness§ Currentlyissystems-oriented,withfocusonreducing
redundancies,increasingqualityandefficiency.§ TraceabilitybetweenTestsMapandMonitoringMap§ NewresponsibilityofQALeader/BusinessAnalyst
TestDataand ServiceVirtualization§ Reducetimeandcost,increasedsecurityandquality§ Maintenanceoftestscripts
IncreasedTestCoverage§ +applications|+testcases|+teststypes§ inlesstime
ReduceandIntegratetheReleasePipeline
§ AgilewithWaterfall– legacy
§ Frombimonthlytomonthlyorbiweekly
…and,allthishavetointegratedwithOI
ecosystem
CHALLENGES
TestAutomation§ Acceptance§ Functional§ Integration§ Unit
ReleaseManag.Automation
SOLUTIONSinprogress§ QALeader workingtogetherBusinessAnalystandCustomer
duringrequirementsphasetodrawtheReq.DiagraminARD§ QAandHPALMstrategyredesignedtobebusiness-oriented
andintegratewithCAARD
§ TheintegrationdevelopmentareabecomesresponsibleforSVanddedicatedcontractortodevelopandmaintainthevirtualservices
§ ThekeydevelopmentsuppliersarebeingtrainedtoconsumeSVservicesduringtheconstructionphase
§ Automationofintegrationandregressiontests§ IncreasedfunctionalcoverageinworkingwithOperations,
SoftwareFactoriesandespeciallyBusinessAreas
§ Review currentreleasepipeline(newdemands/correctives)§ StructurenewenvironmentsforQA§ IntegrateWaterfallandAgilepipelines– Bimodal§ AutomatedtheRelease Process
§ Scrumteamsweredefined§ Acrossfunctionalstructurewasdefined§ New toolsarebeingused/tested(Jenkins,Selenium,Docker,
GIT,Rally)§ TheentireITlifecycleisbeingreviewed
9 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
CAAgileRequirementsDesigner
PROJECT002
CORRECTION (PKE)001
PROJECT003
PROJETOn
PROJECT002
4.AddressingTheChallenges(2/2)
CAServiceVirtualization
CATestDataManager
Clarity PPM
RFT
ALMUFT
LoadRunner
CUSTUMER BA
QA
BigPicture
Architecture+Solution+QAStrategy
Code+Unit/System
Tests
SIT+UAT+Regression Production
RELEASEPIPELINE
Requirements
10 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
5.ACHIEVEMENTSTDM|SV|Automation|ARD|Transformation
GestãodeRelease
TestDataManagement § Automationoftestcaseswithhigherrecurrencein2016|saved ~U$300k
Automatedthe5 mostcommontestcasein2016:§ CREATEANACCOUNT§ CREATEACCOUNTCUSTOMER§ CREATEACCOUNTBILLING§ CREDITANALYSIS§ BILLINGPROFILECREATION 80h tospecify
andbuildComparativeruntimes:
§ 6,000hofexecutionin2016§ ~400manualtestcases§ Avg.Cost:~U$300,000
Volume Manual AutomatedScripts TDM
1Customer 15Hours 20Min 40Seconds
100Customers 1,460Hours 30Hours 1Hour
11 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
5.ACHIEVEMENTSTDM|SV|Automation|ARD|Transformation
GestãodeRelease
TestDataManagement
ServiceVirtualization
TestAutomation
OrientedtoBusiness/ARD
Transformation
§ ~300automatedserviceslast2months(withonly2analysts)§ StructuringateamdedicatedtoSV
§ IntegrationTest– 30%automated|saved~U$150,000in2016§ RegressionTest– 90%automated|saved~U$260,000in2016
§ Structure-orientedBusinessandtraceabilitywithmonitoringmapofcriticalchain(operations)
§ IntegratedEcosystem– HPALM,CAPPM,CAARD,CASVandCATDM§ COEStructured– Responsibleforthedevelopment&maintenanceofscripts§ BoardofDirectorswascreated§ 3ScrumTeamswerestructured– Digital,IntegrationandPortal
§ Automationoftestcaseswithhigherrecurrencein2016|saved ~U$300k
12 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
6.NextChallenge– IntegrateTheTwoWorld
CAAgileRequirementsDesigner
PROJECT002
CORRECTION (PKE)001
PROJECT003
PROJETOn
PROJECT002
CAServiceVirtualization
CATestDataManager
Clarity PPM
RFT
ALMUFT
LoadRunner
CUSTUMER BA
QA
Architecture+Solution+QAStrategy
Code+Unit/System
Tests
SIT+UAT+Regression Production
RELEASEPIPELINE
Requirements