devops – iib continuous build and deploy automation using ... · 1 devops – iib continuous...
TRANSCRIPT
1
DevOps–IIBcontinuousbuildanddeployautomationusingGitHub,JenkinsandJFrogArtifactory.
SurendrakumarJebakaniPerficientInc.
2
1 OVERVIEW............................................................................................................................................................................................................3
1.1 BASICFUNCTIONALITIES..............................................................................................................................................................................4
1.2 BUILDENVIRONMENTS................................................................................................................................................................................4
2 DEVELOPMENT.....................................................................................................................................................................................................5
2.1 GITHUBCONFIGURATION............................................................................................................................................................................5
2.2 IIBTOOLKITCONFIGURATION(UsingeGitplugin,youcancloneGitHubrepositoriesintoyourIIBtoolkitworkspace.)..........................13
3 JENKINS...............................................................................................................................................................................................................25
3.1 ANTBUILDSCRIPT......................................................................................................................................................................................25
3.2 JENKINSCONFIGURATION.........................................................................................................................................................................26
3.3 SCMConfiguration.....................................................................................................................................................................................31
4 JFrogArtifactoryConfiguration..........................................................................................................................................................................38
5 CONTINUOUSBUILDANDDEPLOYMENT...........................................................................................................................................................43
3
1 OVERVIEW
ThisdocumentdemonstrateshowcontinuousbuildanddeployautomationisachievedforIBMIntegrationBusv9deployment
usingopensourceGitHub,JenkinsandJFrogArtifactory.
4
1.1 BASICFUNCTIONALITIES• FullDeployment–LatestcodeispulledfromGitHubrepositories,andeachIIBapplicationsareassembled,packed,override
propertiesappliedthendeployedtorespectiveIntegrationServer(ExecutionGroup)onIIBnodes.
• AssembleandDeploybyApplicationName–LatestcodeforthegivenApplicationispulledfromGitHubrepository.Itis
thenassembled,packed,overridepropertiesappliedthendeployedtorespectiveIntegrationServer(ExecutionGroup)onIIB
nodes
• DeployOnly–LatestbarfileistakenfromGitrepositoryandaftertheenvironmentspecificbaroverrideapplied,the
overriddenbarfileisdeployedtorespectiveIntegrationServer(ExecutionGroup)onIIBnodes
• AssembleandBuild–LatestcodeforthegivenApplicationispulledfromGitHubrepository.Itisthenassembled,packed,
andcompiled.ThenewlycreatedbarfileispublishedtotheArtifactory.Inthisoption,deploymentwillnothappen.
• DeployfromArtifactory–Worksonlywith“DeployOnly”option,whenthisoptionisselectedbarfileisdownloadedfrom
Artifactory(bydefaultlatestbarfileforthegivenapplicationisdownloaded.Barfilecanalsobedownloadedbyspecifying
thetargetbuildnumber)
• ContinuousBuildandDeployment
o ScheduledBuildTrigger–thebuildjobgetstriggeredautomaticallybyascheduleratthescheduledtime.
o BuildwhenachangeispushedtoGitHub–thebuildjobgetstriggeredautomaticallyeverytimewhenthereisa
changepushedbysomeonetoIIBGitHubrepositories.
1.2 BUILDENVIRONMENTS• IBMIntegrationBusv9.0.0.6–Devplatform
• Jenkinsv2.0–forcontinuousintegrations
5
• GitHub-forversioncontrol(eGitplugincanbeinstalledontopofIIBToolkitforsourcecodemanagement.i.epull/push
changesfrom/toGitrespository)
• ApacheANTv1.9.7–forbuildautomation
• JFrogArtifactory4.13.2–ArtifactRepositoryManagerforarchivingthebuildartifacts
2 DEVELOPMENT
2.1 GITHUBCONFIGURATIONo Createmultiplerepositories,oneperIIBapplicationandoneforallthecommoncomponents(errorhandling,commonesql
procedure/functions,loggingetc).
6
o CreateadedicatedrepositoryforIIBbuildprocesstostoreandmanagethebuildrelatedartifacts.
o ClicktheGreenNewbutton
o Haveanamingstandardfornamingtherepositories.
7
o MaketherepoPublicandInitializetheREADMEfile.
8
9
o ClicktheREADME.mdfiletoedit.
10
o Fillinthe“EditFile”tabwithinfousingthetextbelowasatemplate.
#IIBAccountServiceRESTfulAPIserviceforgetting/updatingaccountinfo-AccountService_APP-ApplicationcontainingAccountServiceAPI*Thefollowingareusedbytheservicebutlocatediniib-commonrepository*-Common_CICS_LIB-LibrarycontainingCICSHTTPcallersubflow-Common_LIB-Librarycontainingerrorhandlersubflows-CommonUtils_LIB-Librarycontainingcommonutilcode*Anapplicationisacontainerforalltheresourcesthatarerequiredtocreateasolution.AnapplicationcancontainIBM®IntegrationBusresources,suchasflows,messagedefinitions,libraries,andJARfiles.*
11
*Alibraryisalogicalgroupingofrelatedcode,data,orboth.Alibrarytypicallycontainsreusablehelperroutinesandresourcessuchassubflows,ESQLmodules,messagedefinitions,maps,andJavautilities.Alibraryisusefultogrouptogetherresourcesofthesametypeorfunction,forreuseoreaseofmanagement.Librariesareoptional.*
o ClickCommitChanges
12
o Reposhouldbesuccessfullycreated.Clickthe“Cloneordownload”buttonandthenthe“CopytoClipboard”button
tocopytheHTTPsURL.
13
2.2 IIBTOOLKITCONFIGURATION(UsingeGitplugin,youcancloneGitHubrepositoriesintoyourIIBtoolkitworkspace.)
o SwitchtoIIBToolkitandgototheGitRepositoryExploringPerspective
14
o ClicktheCloneRepoButton
o PastetheURLintheURIboxandputinyourGitUsernameandPassword
15
16
o ClicktheNextButton
17
18
o ClickBrowseandselectalocalfolderthatyouhavechosentostoreallyourGitRepositories.ClickFinish.
o TherepositoryshouldnowshowinyourGitPerspective.
19
o TosharetheIIBproject(application,libraryetc)toGitrespository:
o Rightclickontheprojectandselect“Team”and“Shareapplication”.
20
o Rightclickontheprojectandselect“Team”and“ShareProject”.
21
o Select“Git”andclickNext.
22
o Selecttherepositoryfromthedrop-downlistandclickFinish.
23
o TopushthechangestoGitHub:
o Makethechangesinthemessageflowcode,afterthechangesareunit-testedlocallyandreadyforDEV/QA
deployment:
o Youwillseeallthechangedartifactsinthe“GitStaging”view.
o Selectthechangedfilesunder“UnstagedChanges”in“GitStaging”andmoveto“StagedChanges”bydrag-and-drop.
o Addacommentin“CommitMessage”andclickthe“Commit(Ctrl+Enter)”
24
o Asyousoonasyoucommittedthechange,youwillseeachangependingtobepushed(↑) incorrespondingGit
repository.AtthistimethechangesareonlycommittedlocallyandnotpushedtotheGithubserverrepository.
o Rightclickontherepositoryandselect“PushtoUpstream”topushthechangestoGithubserverrespository.
25
o ConfirmthechangeinGitHubbyloggingintohttps://github.com/<yourcompanyname>/<repositoryname>
3 JENKINS
3.1 ANTBUILDSCRIPTo AntneedstobeinstalledontheJenkinsserver.
o Anantbuild.xmliscreatedwithmultipleanttargettaskstoperformthefollowingtasks
o GetApplicationList–For“FullDeployment”,IIBapplicationlististakenfromtheJenkinspropertyfile
o assembleApp–GettheIIBcodefromJenkinsworkspace(pulledfromGitrepositories)
26
o mqsipackagebar–TocreatedeployableBARfiles.(mqsicreatebarcouldnotbeusedasIIBToolkitisnotinstalledon
buildJenkins/IIBservers)
o mqsiapplybaroverride–Toreplacetheenvironmentspecificconfigurablevaluestakenfromthebaroverrideproperty
files
o mqsideploy.deploybar–TodeploytheoverriddenbarfiletorespectiveExecutionGroup(IntegrationServer)on
multipleIIBnodes
o copyFiles–TocopythedeployablebarfilefromoneIIBservertootherIIBserversfordeployment
o cleanupFiles–Tocleanupthefiles/directoriesfromIIBserverafterthedeployment
3.2 JENKINSCONFIGURATION§ Jenkins is used to perform the typical build server work, such as doing continuous/official/nightly and on-demand
builds.
§ Login to Jenkins server to create a new build job for IIB. Click “New Item”, enter the name of the build project in
“Enter an item Name”, and select “Freestyle project” and then click “ok”. (There is a build project “iib-build-job”
created in Jenkins for continuous IIB build deployment purpose)
27
§ You can parameterize the project with parameters that you want to input into the build. These parameters will be
passed to the build job when you start the build.
SelectthePROJECT_NAMEinwhichtheApplicationresides.
o APPLICATION_NAME
§ SelecttheApplicationyouwishtodeploy.
28
29
o DEPLOYMENT_OPTION
§ Therearethreedeploymentoptionsgiven.
§ ASSEMBLE_AND_DEPLOY-PullsthesourcecodefromGitHubrepository,builds,appliesbaroverridesandthen
deploys.NewlybuiltbarfilewillbepublishedtoArtifactory.
§ ASSEMBLE_AND_BUILD-PullsthesourcecodefromGitHubrepository,buildsbutDOESN’Tdeploy.Newlybuilt
barfilewillbepublishedtoArtifactory.
§ DEPLOY_ONLY-GetsthebarfilefromGitHub,appliesbaroverridesandthendeploys.
o DEPLOY_FROM_ARTIFACTORY?
§ Whenthe“DEPLOY_ONLY”optionischosen,alsochoosewhethertodeployfromGithuborArtifactory
§ YES-TodeploythebarfilefromArtifactory.NO-TodeploythebarfilefromGitHub.
o ARTIFACTORY_BUILD_NO
30
§ IfyouchooseYESforDEPLOY_FROM_ARTIFACTORY?,thenentereithertheArtifactoryBuildNumberofthe
barfileor“LATEST”togetthelatestbarfile.
o ENV_NAME
§ NameoftheIIBenvironment,DEVorQA
o BRANCH_NAME
§ GitHubrepositorybranchname
o USER_NAME
§ IIBserverloginusername(ADusername)
o PASSWORD
§ IIBserverloginpassword(ADpassword)
31
3.3 SCMConfigurationo MakesurethatMultipleSCMpluginininstalledinJenkins.
o Select“MultipleSCMs”optionunder“SourceCodeManagement”andaddalltheIIBGitrepositories.
32
§ BuildConfiguration
o Inthe“Build”section,click“Addbuildstep”andselect“InvokeAnt”fromthedrop-downlist.
o Fill-uptheinformationasbelow
§ AntVerion–NameoftheAntinstallation.
33
§ Targets–Notneededifyouareusingcustombuildfile.Ifnot,specifytheanttargettaskshere.
§ BuildFile–Thelocationofcustombuildfile.ItispulledfromGit“iib-build-process”repository
§ Properties–Youcanaddthepropertieswithkey-valuepairinthissection.
o
§ FULL_DEPLOYMENT
o ItisusedfordeployingalltheIIBapplicationstorespectiveEG/ISrunningonmultipleIIBnodes.
34
o MakesuretoaddtheapplicationnameintheAPPLICATION_NAMElistfirst.
o MakesuretoaddtheGitrepositoriesinJenkins“MultipleSCMs”under“SourceControlManagement”.
o Clickthe“iib-build-job”projectfromthelistofJenkinsprojects
o Clickthe“BuildwithParameters”fromthelistedoptionsattheleftside.
35
o SelecttheoptionslikeaboveandinputyourusernameandpasswordinUSER_NAMEandPASSWORD
o Click“Build”totriggerthebuilddeployment.
o Youwillseeanewbuildjobinprogressunder“BuildHistory”,clickonthebuildjobtoseetheprogress
36
o Youcanalsoclickthe“workspace”toseethecomponentspulledfromGitrepositoryintoJenkinsworkspace
37
o “ConsoleOutput”showsthebuildlogsstep-by-step.Attheenditshowswhetherthebuildissuccessornot.
38
§ ASSEMBLE_AND_DEPLOY
o ThisoptionisusedforbuildanddeployingaparticularIIBapplication.(Sourcecodeforthatgivenapplicationis
pulledfromGitrepository)
§ SelecttheIIBapplicationnamefromthedrop-downlistintheAPPLICATION_NAMEinputparameter.
§ Select“ASSEMBLE_AND_DEPLOY”intheDEPLOYMENT_OPTIONinputparameter
§ SelecttheenvironmentinENV_NAME
§ Afterinputtingtheusername/password,clickbuild.
§ DEPLOY_ONLY
o Thisoptionisusedjustfordeployingpreviouslydeployedbarfile.(BarfileistakenfromGitrepositoryiib-build-
process/build_process/barfileBaseLocation/)
o Thebarfileisoverriddenwithenvironmentspecificconfigurablevaluestakenfrompropertyfiles(iib-build-
process/build_process/BarFileProperties/DEV/).
o Overriddenbarfileisthendeployed.
4 JFrogArtifactoryConfigurationo MakesurethattheJFrogArtifactoryplugininstalledinJenkins.
o IntheJenkinsjobconfigurationpage,under“BuildEnvironment”select“Generic-ArtifactoryIntegration”
39
o MaketheArtifactoryconfigurationasbelow:
Upload:
40
Download:
41
o Makesurethat“Captureandpublishbuildinfo”isselectedunder“MoreDetails”
o
o InAction:
Uponcompletionofdeployment–SeetheJenkinsjoblogstogettheURLoftheArtificatorytoseeanddownloadthe
newlybuiltanddeployedbarfile.
o TodeploythebarfilefromArtifactory:
42
43
5 CONTINUOUSBUILDANDDEPLOYMENT• “BuildTriggers”with“PollSCM”option
o Usingthisoption,thebuildjobgetstriggeredautomaticallybyascheduleratthescheduledtime.
o Intheschedulebox,putascheduletiming(Ex:H2211-12*-Thiswilltriggerthebuildjob1stdayofeverymonthat
10:00PM.)
o Bydefault,“FULL_DEPLOYMENT”optionwillbeexecuted(configurablesocanbechanged)
• “BuildwhenachangeispushedtoGitHub”option
o Usingthisoption,thebuildjobgetstriggeredautomaticallyeverytimewhenthereisachangepushedbysomeoneto
IIBGitHubrepositories.
o GettheJenkinshookURL
§ InJenkins,gotoManageJenkinsàConfigureSystem,lookfor“ManuallymanagehookURL’sinthe“GitHub
WebHook”section.ThehookURLwillbeshownwhenyou’reclickingonthequestionmarkiconontheright
44
side.Usually,itconsistsofyourhostnameandthe/web-hook/postfix.Example:https://<your-domain-
name>/web-hook/
o AddthehookURLinGitHubrepository
§ GototheIIBrepositoryforwhichyouwanttoenable“BuildwhenachangeispushedtoGitHub”.
§ Under“Settings”and“Webhook&Services”,click“Addservice”inthe“Services”section.
§ SelectJenkins(GitHubplugin)fromthedrop-downlist.
§ Andclick“Addservice”
§ Youcanclick“TestService”toconfirmeverythingiscorrectlyconfigured.