ibm db2 devops experience for z/os...ibm db2 devops experience for z/os § where db2 for z/os meets...
TRANSCRIPT
AgendaProduct introduction & positioning
Architecture
Primary capabilities
Walkthrough / Demo
Power of the APIs
IBM Db2 DevOps Experience for z/OS
§ Where Db2 for z/OS meets DevOps§ Enabling self-service, on-demand test environments create, modify, destroy§ Db2 Schema Database-as-a-Service§ Database-as-code, versioned source code management of DDL§ Site rules, Limits, & Environment control to manage this dynamic environment§ User Interface and API support§ Ready to be integrated into your DevOps pipeline via samples / REST
The State of DevOps - DORA§ The study is focused on code COMMIT to DEPLOY
- Area of low variability, sustained optimization§ Upstream processes are considered more creative
- More variability and difficult to optimize
ContinuousIntegration
ContinuousDelivery
ContinuousDeploymentProvisioning Design Iterate
IBM Db2 DevOps Experience for z/OS
Upstream
Move Db2 towards Continuous Integration / Delivery
5
• Large release cycles (months, quarters)• Slow delivery to customers/marketplace• Integration is expensive & disruptive• Problems can have a huge blast radius
• Shorter cycles baked into Dev process• Faster delivery to customers/marketplace• Measured / manageable integration• Contain problems to smaller scopes• Lower stress associated with release
delivery
Where does the Db2 DevOps Experience for z/OS fit
Db2 for z/OS
§ Allows Db2 for z/OS to participate in existing DevOps pipelines
§ Db2 DDL as code- Application and Infrastructure have
been managed as code- Now Db2 fits this model
§ On demand, self service provisioning and deployment of Db2 objects and data
§ Enable the upstream and downstream portions of DevOps
§ With controls to enable object handling, limits, and approvals for deployment
§ UI and REST API support§ Fits into customer’s existing DevOps
tooling
Db2 DevOps Experience for z/OS Architecture Summary
7
What is Zowe?
8
• Open source software licensed under the Eclipse Public License version 2.0• Extensible framework for connecting applications / tools to mainframe data and
applications• Delivered from the Open Mainframe Project
- Part of the Linux Foundation• Partnership with IBM, Rocket, and CA
1 Common look & feel –lower barriers to entry
2 Appeal to bring new skills to the platform
3 Simplify tasks & interaction
Zowe Capabilities• Virtual Desktop Environment on z/OS
• Browser based, graphical• Runs on z/OS
• GUI-based applications• File Transfer Application• Code Editor• Workflow Application• Jupyter Notebook
• CLI (Brightside, CA)• Security / Accessibility / Internationalization
• Leverages z/OS security• Single sign on (Roadmap item)
Database (DDL) as Code
§ Database as code (versioned DDL):– Logical groupings of Db2 objects (in support of Applications)
§ Unites with:– Application version control– Infrastructure as code
§ Fuels provisioning request & change deployments
‹#›
Db2 CatalogDDL SCM
MasterBranch UserBranches
State-based vs. Migration-based Approaches
11
State Based§ Source control system of record
– Established from snapshot of DB§ DDL stored as version control text files§ Has a Compare engine
– Indicate desired state– Engine optimizes change for target
Migration Based§ Database system of record§ Capture state at beginning of project§ Maintain series of sequenced migration
scripts§ Use culmination of scripts to achieve
desired state
Different Approaches to Database (DDL) as Code
DDLSource of
recordTarget
Analysis & Compare
Upgradescript
DatabaseSource of
record
State 1 State 2 State 3 State 4 State n
On-demand, Self-service, Developer Driven
§ Drive Database needs in the Developers cadence by the Developer§ Provision an Instance as needed within the Sprint § Fail fast, Deprovision the Instance and (perhaps) try again§ Deploy changes to the Instance as needed§ Can submit changes for consideration to include in the master branch
– Pull Request12
X
ProvisioningModification
Pull Request / Merge
Deprovisioning
Management/Administrative Directives
§ Environment definitions to control where Provisioning takes place§ Provisioning Instance Limits§ Administration of Application via Teams § Storage Limits monitoring Teams, Applications, Users, and Environments§ Site Rules for naming, definitions, placement§ Data Steward roles for approving database changes
13
Subsy
stems
Environments
Teams
Appli
catio
ns
Instances Lim
its
Users
Site Rules
EnvironmentsSubsystems
Applications
InstancesTeamsUsers
Site RulesLimits
IBM Db2 DevOps Experience for z/OS for z/OS
14
• A platform for the definition administration and operation of DevOps services in support of Db2 for z/OS.
• Register the Subsystem & establish Environments which will be the targets for provisioning operations
• Establish the Users, Teams, and Subsystems that will participate in DevOps
• Provision instances of the Application into defined Environments
- Will create an Instance of the Application tables- Optionally copy data into the provisioned Instance objects
• Define Site Rules / Limits to automatically manage the DevOps Experience
• Define Applications as set of Db2 for z/OS objects- Establishes a master branch in Git for those objects- Application hierarchy- Contains Db2 objects from the same Subsystem
• Applications are provisioned into Instances
Users
Teams
Subsystems
Environments
Applications
Site Rules / Limits
Instances
Adm
inis
tratio
nO
pera
tion
IBM Db2 DevOps Experience for z/OS for z/OS
15
• DDL for the Provisioned instance can be modified and deployed to the Instance for testing
• Deprovision object to start over (fail fast)
• Commit changes to the master branch when preparing to deploy to other landscapes (promotion via legacy processes)
- The UI/API will issue a Git pull request where Approvals can be added to the process before being accepted
- Once accepted, the master branch and source Db2 objects are updated
Users
Teams
Subsystems
Environments
Applications
Site Rules / Limits
Instances
Adm
inis
tratio
nO
pera
tion
Db2 DevOps Example Flows
16
Applications
Master branch in gitUser branches in git
Provision
X Deprovision
Pull Request
Instances
Source MergeDDL Edit
§ Subsystems are registered
§ Users, Teams set up and assigned Environments
§ Site Rules defined
A Snapshot / Restore helpful for
self-serve reset Instance Merge
Repo
DevOps In a Shared Environment
§ Distributed environments provisioning can be distinct from infrastructure, up§ This could also be the case with z/OS with a virtual environment (zD&T)§ But it’s more likely that the HW, z/OS, and z/OS resources (Db2, storage, etc.) will be shared§ Important elements of DevOps in a shared environment:
– Registration of participating Db2s & Db2 objects– Control where provisioning activities will take place with limits:
§ Expanded, fenced authorities for Developers– Namespace management for Instance separation – Rules for naming, placement, definitions– Storage monitoring– Easy visibility to rules, metrics, etc.
17
EnvironmentsSubsystems
Applications
InstancesTeamsUsers
Site RulesLimits
HW, z/OS, z/OS Resources
Application
Instances Instances
Zowe Log In
18
• Zowe login screen: <hostname>:<zlux-proxy/node server port>• Log in with your TSO ID for that LPAR
Starting the DevOps Experience
19
• Plugins appear across the bottom & in the Application Menu• After the Db2 DevOps Experience is installed, Pin to taskbar
ApplicationMenu
”Pinned”plugins
Pull Requests, Instances, & the Hamburger• What’s a Pull Request? What’s an Instance?• The remainder of this presentation describes all components Administrator Menu
User Menu
Db2 DevOps Experience for z/OS - Subsystems
21
• Open the Db2 DevOps Experience• Use the -> Manage -> Subsystems
• Subsystems are shown by default when Manage is selected• 2 Subsystems are registered for DevOps control• Additional Subsystems can be Discovered or Defined as needed
Db2 DevOps Experience for z/OS – Teams & Environments
22
• View/Add Teams (tiles contain the assigned Environments)• Teams can be edited to assign Environments, Limits defined, and Site Rules created
for Schema and Database names• Instance limits are hard limits• Honoring DB & Schema rule (default to DB* & SC*) • 1 DB & 1 Schema name will be generated per source DB & Schema name
• Recommend: Assign more than 1 Team Admin• An Admin cannot approve their own Pull Request
Db2 DevOps Experience for z/OS – Users
23
• The GET API call shows how to get the Team ID assigned by the DevOps Experience• The PUT assigns the User
Team ID & Name Modify Team assignments
Db2 DevOps Experience for z/OS – Storage• Storage Limits are soft limits specified in full GB increments• Can be set for Teams, Environments, Users, and Applications• Default is no limit
Application Discovery
25
This vs. This
Db2 DevOps Experience for z/OS – Applications• The Sample DB Application has already been defined• Click View to look at the objects included in this Application• From the View page, click Provision new instance to create a set of test objects based on the
Application definition• Recommend: Lean towards creating small application definitions
• A Db2 object can only be defined in 1 DOE Application• Allows for granular Instance provisioning.
• Recommend: See the Best Practices Design topic
Db2 DevOps Experience for z/OS – Provision Instance• Give the Instance a name (Sample DB Inst1)• Validate / Select the Team to assign and the Environment to host• Click Provision, and watch the progress bar cycle while the Instance is being
processed• 2 progress bars are displayed during object provisioning• A single progress bar indicates that data is being copied
Db2 DevOps Experience for z/OS – Modify an Instance• The Instance tile displays the Instance name; source Application name;
Environment, Subsystem, and Schema into which the objects have been provisioned- Note that the tile ellipsis can be expanded to deprovision this instance
• Click the Instance tile Details button, which provides options to deprovision, snapshot (future support), and issue a pull request
- A DDL Editor is also available here to view or modify the Instance object DDL
Added columnIndicated changeApply to Instance
Db2 DevOps Experience for z/OS – Pull request• Developer (Instance owner) can request that the changes to this Instance be
accepted into the Application master branch (and source Db2)• Comments can be entered• Additional Approvers (from the same team) can be added
• Team Admins are automatic Approvers• The DIFFerences can be viewed
Db2 DevOps Experience for z/OS – Pull Request• The Pull Request will appear on your main DevOps page as a request awaiting
approval
Db2 DevOps Experience for z/OS – Pull Request Acceptance• An an Application Approver, I see requests awaiting my view on the main page• Review the comments & differences of this PR• The PR can be set to Needs work, Approve, or can be Declined
• Needs work leaves the PR open for addition work• Decline closes out the PR
Db2 DevOps Experience for z/OS – Pull Request Acceptance• PR Approval enabled the Merge capability• Merge (which can be initiated by the Approver or Developer)
• Accepts the changes into the Git master branch• Modified the Source Db2 objects
Db2 DevOps Experience for z/OS – Full API Support• A robust set of REST APIs to the DevOps
Server• All shown administration and operation
activities could be performed via APIs- As depicted for User administration
• A Swagger json file is delivered with the product for documentation
• Current categories:- Users (zss security API)- Teams- Subsystems- Applications- Instances- Discovery Plan- Administration (job management)- Storage Limits
Gold MasterApplication definition source
Database DevOps Best Practice Design
34
Production
Clone intoGold
Provision intoInstance
Environments
Commit toMaster
(pull request)
Promotion(Traditional)
*Note: Promotion likely would go through other landscapes before reaching Production
DevOps Experience – CI/CD Integration• Sample Jenkins pipelines have been developed for Provisioning & Deprovisioning tasks
• Provisioning by Application name• Deprovisioning by Instance name
• This model can be extended to other use cases or CI/CD tools
DevOps Experience – CI/CD Integration• Parameters are taken into the first job (step) and passed as property files
• This job is assigned a webhook Token (DOEprovision) allowing it to be started via REST• Applications, Teams, & Environments retrieved and translated from names to UUIDs• The POST request to create the Instance (provisioning) is constructed & executed
Last pipeline execution
Review console output
<ipaddr or domain>:<port#>
DevOps Experience – CI/CD Integration• Parameters are taken into the first job (step)
• This job is assigned a Token (DOEprovision) allowing it to be started via REST• DOEServerPort formatted as <ipaddr or domain>:<port#>
• Applications, Teams, Environments retrieved and translated from names to UUID• The POST request to create the Instance (provisioning) is constructed & executed
<userid>:<password>
webhookToken
Overridepipeline
parameters
IBM Db2 DevOps Experience for z/OS
• Where Db2 for z/OS meets DevOps• Enabling self-service, on-demand test environments create, modify, destroy• Db2 Schema Database-as-a-Service• Database-as-code, versioned source code management of DDL• Site rules, Limits, & Environment control to manage this dynamic environment• User Interface and API support• Ready to be integrated into your DevOps pipeline via samples / REST
IBM Db2 DevOps Experience for z/OS
39
• A platform for the definition, administration, and operation of DevOps services in support of Db2 for z/OS.
• Announced 10/8/2018 at the IBM Data Technical Summit• https://www.ibm.com/developerworks/community/blogs/897a7c98-57af-4523-9cfa-
07ebc3f996b4/entry/Announcing_IBM_Db2_DevOps_Experience_for_z_OS?lang=en• Availability date of 11/23/2018• Accountment letter
• https://www-01.ibm.com/common/ssi/rep_ca/0/897/ENUS218-470/ENUS218-470.PDF• Based on Zowe
• https://zowe.org/home/• Runs as a plugin to the Virtual Desktop Infrastructure in Zowe
• Knowledge Center• https://www.ibm.com/support/knowledgecenter/en/SSGPWX_1.1.0/doe110_welcome.html
40