what's new and exciting in tools for db2 for...

© 2016 IBM Corporation

What's new and exciting in Tools for DB2 for z/OS

Elisabetta Curci [email protected] Analytics on System Z Platform Sales Representative Italy

DB2 User Group Italia - 2016

© 2016 IBM Corporation2

Please Note:

� IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion.

� Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision.

� The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract.

� The development, release, and timing of any future features or functionality described for our products remains at our sole discretion.

Performance is based on measurements and projections using standard IBM benchmarks in acontrolled environment. The actual throughput or performance that any user will experience will varydepending upon many factors, including considerations such as the amount of multiprogramming in theuser’s job stream, the I/O configuration, the storage configuration, and the workload processed.Therefore, no assurance can be given that an individual user will achieve results similar to those statedhere.


Agenda

� IBM DB2 z/OS Tools Today

� IBM DB2 z/OS Tools Directions 2016

Some specific new tools

� Data Server Manager–What is DSM– Integrating Products and Timelines–Directions–Features

� Infosphere System Z Connector for Hadoop


IBM DB2 z/OS Tools and z Analytics today • DB2 Administration tool

• DB2 Administration toolkit SAP

Edition• DB2 Automation Tool

• DB2 Automation Tookit SAP Edition

• DB2 Bind Manager• DB2 Buffer Pool Analyzer

• DB2 Change Accumulation Tool

• DB2 Cloning Tool• Db2 High Performance Unload

• DB2 Log Analysis Tool

• Db2 Object Comparison Tool

• DB2 Object Restore• DB2 Path Checker

• DB2 Perform Toolkit SAP Edition

• QMF z/os• DB2 Query Monitor

• DB2 Query Workload Tuner

• DB2 Recovery Expert

• DB2 Sort

• DB2 SQL Performance Analyzer• DB2 Table Editor

• DB2 Utilities Enhancement Tool

• DB2 Utilities Suite• DB2 Query Workload Tuner

• Infosphere Classic Federation

• Infosphere Classic Replication Server• Infosphere family for data governance

• IS Data Architect

• IS Information Server

• IS Optim Archive• IS Optim Test Data Management

• IS Data Replication

• Tivoli Omegamon XE for DB2 PM/PE• DB2 Analytics Accelerator

• DB2 Analytics Accelerator Loader

...and more !


DB2 Tools Delivered Q32015 – Q12016

� DB2 Administration Tool/Object Comparison Tool v11.2– More support for IBM DB2 Analytics Accelerator– Enhanced Change Management and usability features

� DB2 Cloning Tool v3.2– Ability to clone a subsystem at the data set level instead of the volume level,

reducing the footprint of the clone– Clone table spaces from image copies – NO DOWNTIME!

No fast-replication storage requirement – Consistent cloning across z/OS LPARS allows you to easily clone

large applications to a new LPAR at a consistent point-in-time

� DB2 Recovery Expert v3.2– Mass application recovery: Prioritize mass recovery

based on application priorities (no fast-replication storage requirement)– Redirected recovery can backup object from source (table, table space,

database) and restore to different target (database, LPAR– NO DOWNTIME IMPACT to SOURCE!!!

– Backout errant transactions to ensure errors are removed from applications

6

DB2 Utilities Solution Pack

Automate and enrich the IBM utilities

DB2 Automation ToolDB2 High Performance UnloadDB2 SortDB2 Utilities Enhancement Tool

DB2 Fast Copy Solution Pack

Superior avail & cost for copy, backup & recovery operations

DB2 Cloning ToolDB2 Recovery Expert

DB2 Administration Solution Pack

Manage objects & schema

DB2 Administration ToolDB2 Object Comparison ToolDB2 Table EditorOptim Configuration Manager

DB2 Performance Solution Pack

Master the performance lifecycle

DB2 SQL Performance AnalyzerTivoli Omegamon XE for DB2 PEOptim Query Workload TunerDB2 Query Monitor

IBM DB2 z/OS Solution Packs Today

Complete solutions for all needs

Simplified install and maintenance

Build intelligence into when and how actions are performed

Optimize performance and resource utilization associated with DBA activities to reduce TCO

DB2 Tools Solution Packs


Directions for DB2 Tools


Strategic Directions for DB2 Tools

� Self-managing DB2 systems– Extensive set of alert thresholds & recommended actions– Advanced action resolution (including calendaring)– Integration of performance tools for true intelligent decision making– Move from automation to true autonomic capability

� Simplification & Modernization of Administration– Autonomics– Advanced graphical interfaces– Convergence of tools

� Cloud & Database Provisioning– Install, migrate, decommission subsystems– Simple, automatic provisioning of tools in support of new systems– Exploit z/OSMF support for workflows– RESTful APIs to support marketplaces

� Analytics (including Enterprise Spark)– Enable access to all z data for the purpose of analytics– Identify opportunities to support Data Scientist & other use cases– Leverage streams & Spark for improved efficiency (e.g. continuous distribution statistics –

RUNSTATS elimination)


DB2 Tools Portfolio Focus in 2016

� All– DB2 12 Toleration & Exploitation!– Portfolio refresh coincident with DB2 VNEXT (Versions, Releases, PTFs)

� Administration Solution– Change Management enhancements (CM Batch, Multi-target Change)– Improved Accelerator support– Integration of provisioning capability into Administration Pack– Expose Admin functions via REST APIs (for UCD or DSM integration)

� Utilities Solution– New version of HPU for performance improvements and resource optimization– DB2 Sort: Support for variable length block level data exchange– Extend autonomics (additional data sources, REORG management, statistics management)

� Analytics Solution– Extend Loader data sources based on client demand– QMF analytics capabilities enhancements

� TCz– GUI enhancements including workflows to facilitate configuration

� Performance Solution– Populate autonomics model with performance data (OMPE & QM)– Populate IDAA with performance data to facilitate analytics (QM)– New DSM-based QWT and other products


IBM Data Server Manager


IBM Data Server Manager

Deliver a Simplified User

Experience

Common integrated web console

Deliver familiar capabilities

from the Database Tools• Single installer and

integrated repository

• Provides enterprise view of your environment

• Guided workflow and analysis

• Database Administration, Query Tuning, Configuration Management

7


Key development directions mapping Key clients requirements

Make it easier to use; provide workflow, diagnostic smarts, actionable recommendations

Streamline tools, installs, repositories

Manage databases across the enterprise

Provide better performance and scalability

one stop shop tooling

Mar 30, 2015 Dec 11, 2015

� eGA: IBM Data Server Manager V1.1.0

� Next Generation Core Tooling for DB2 (OPM, OQWT, OCM, Data Studio)

- Initial DB2 for LUW – Performance Management and DB Administration

� BigInsights - Simplicity in up and running for BigSQL -

- Initial BigInsights – BigSQLAdministrationa and Management

-

� Good customer reception

� IBM Data Server Manager V1.1.1

� Extending the value of Next Generation Tooling

- Second drop for DB2 for LUW, closed most gaps in differences between old tools and new tools

� IBM Data Server Manager V1.1.2

� Expanding the value of Next Generation of DB2 Tooling to z

-Third drop with support for DB2 for z/OS

-DB2 Query Workload Tuner for z/OS V5.1 based on DSM – standalone or in Performance Solutions Pack for z/OS

-Admin Solutions Packs for z/OS updates with DB2 Configuration Manager for z/OS V4.1 based on DSM

-- Platform adds for DB2 LUW

Data Server Manager: Overall Timeline

Jun 30, 2015

� BigInsights expanding platforms -

- Updates for Linux Platforms

2Q 2016 4Q 2016

� IBM Data Server Manager V1.1.2.1

� DB2 Configuration Manager z/OS:

� DB2 Query Tuner for z/OS

– Migration from Eclipse based OQWT to DSM Based Query Tuner

– Capture Query environment

– Integration – Query Monitor. Performance Solution Pack

� Participating in DB2 Cypress for z/OS ESP

� Beta OMPE V540 OPM based EI

� Management Console functions for DB2 for z/OS (including Utility Solution Pack Autonomics) integrated into DSM

Data Server Manager: Overall Timeline3Q 2016March 30 2016


Where can you find Data Server Manager today

DB2 Performance SolutionPack for z/OS

DB2 UtilitySolutionPack for z/OS

Data Server Manager Base Edition

FREE!- Use to upsell to paid products

DB2 AdminSolutionPack for z/OS


• OCM 3.1.x is part of previous releases of the z Admin Solution Pack• Integrated with z Admin Tool

• OCM 3.1.x is part of DB2 Connect Unlimited Advanced Edition for z/OS• DSM 3.1.x makes DB2 Connect management easy

• Great value for our customers

How can I get DB2 Configuration Manager?


DB2 Configuration Manager for z/OS V5.1 (inside DB2 Admin Soln Pack

only )

DB2 Query Workload Tuner for z/OS V5.1 (standalone, DB2 Perf Soln

Pack)

Features of the Data Server Manager Based Tools At A Glance

Data Server Manager Base

• Connect to DB2 for z/OS V10 and V11.

• Database object navigation, viewing

object detail, and linking to related

objects.

• Database object dependency display.

• Data browsing and editing.

• Basic database object operations, such

as creation of tables, indexes,

constraints, and tablespaces; dropping of

tables, indexes and constraints; altering

tables.

• Showing system privilege from the

perspective of Group/User, Role, or SQL

object. Choose:

-"Group/User" to see the role and the

relative object privilege for a user

account;

- "Role" to see the role a user account

belongs to and its relative object

privilege;

- "SQL object" to see a specific object

and users or roles that have the relative

authority.

• Launching of visual explain and tune

query on the SQL editor

• A new tuning wizard to capture SQL

statements from multiple sources

• Several Tuning advisors provide

recommendations for:

• Statistics Advisor

• Index Advisor

• IDAA Advisor

• Understand problem query or workload

• Access plan graph

• Query formatting and annotation

• Tuning Report

• Test Candidate Index

• Access Plan Comparison

• Index Impact Analysis

• Track Configuration Changes

• Configure zParm

• Compare and Clone Configurations

• Manage Application Profile

• Manage alias

• Manage clients

DB2 Utility Solution Pack V2.2

• Customizable profiles for identifying

object situations and generating resolving

JCL (reorg, copy, runstats, etc)

• Automatic prioritization of object

situations and their resolving actions

• Ability to define maintenance windows for

enabling active autonomics, allowing

DB2 to self manage utility runs

• Notification support (email, text, WTO) on

selected autonomic events

• Graphical trend analysis of historical RTS

• Capture of utility history, recording utility

output, time, duration, etc.


Modernize Query Tuning / Increase IDAA value

• Improve performance of applications leveraging DB2z • Simplified, modern look and feel to database query tuning

and also carried through in the DB2z Performance Pack• Get more value out of your IDAA investments through

evaluating IDAA eligible workloads • New features such as index impact analysis

Easy Database Administration

• Quick and simple way to navigate database objects, view

data and edit, write SQL and get results back

Simplify Best Practices Configuration

• Easier to manage database clients connecting to DB2 for z/OS systems, and manage what they are doing

• Increase value of the DB2z Administration Pack

Integrated Tool for DB2 for z/OS -Management Console and DSM Integration

• Common strategy for DB2 for z/OS customers who want simple access - one tool - to manage the health of the enterprise in a modernized fashion via web browser

Introducing Data Server Manager Supporting DB2 for z/OS


Data Server Manager - Key Features


3-step setup


Basic Navigation : New Tuning Wizard

Select “New to choose the

Database for Tuning

AbstractHelp users navigate between panels more conveniently

Launch pointOptimize -> Query Tuning


Manage Databases Using the Database Explorer

Explore the catalog

Explore database

object properties

Manage database objects


Develop and Run SQL Scripts

Validate SQL

• Customize and filter result output

• Save execution results

• Manage scripts

• Explain SQL • Tune SQL


Create and schedule jobs

2

Schedule a Create Table

job from Explore

Databases

Scheduled commands

from Explore Databases

Alternatively, create a

script and schedule the

job


Analytics

Query workload tuner for DB2 for z/OS overview

Web-based user interfaceWeb-based user interface

DSM common infrastructure

Query Tools and AdvisorsQuery Tools and Advisors Workload Tools and AdvisorsWorkload Tools and Advisors

Repositorydatabase

MonitoredDB2 for z/OS



Access Plan

Graph

Index

Advisor

Summary

Report

Formatter Annotation Service SQL

Overview

Page

Statistics

Advisor

Summary

Report

Statistics

Advisor

Index

Advisor

Test

Candidate Index

Index

ImpactAnalytics

IDAA

Advisor

Access Plan

Comparison Access Plan Comparison

Configuration Report

Expert-Enabled Query Tuning Out of the Box

� Identify query candidates from numerous

sources

– DB2 catalog

– Dynamic statement cache

– User defined repository

� Facilitate analysis

– Query formatting

– Query annotation

– Access path visualization and

annotation

� Get expert tuning advice

– Improve statistics quality

– Improve database design

– Improve accelerated queries

–


Tune SQL or Workload

� Review access plan graph and annotations

� Advisors streamline tuning process

– Single Query Statistics Advisor

– Workload Statistics advisor

– Index advisor

– IDAA advisor

� Use what-if analysis to determine what impactindexes have on a workload

� Use what-if analysis to determine what statements are

eligible for acceleration

Run scripts to implement

advise or share results using tuning report

New!


Analytics

Access Plan GraphRecommendation Integration


Analytics

Workload Index impact analysis

� Abstract

– Analyze the impact of the recommended indexes to other SQL statements

� Launch point

• Select a tuning job -> View Results -> Index script -> Test Candidate Indexes


Analytics

Workload Index impact analysis cont.

Analysis scope

– Statement Cache

– Packages

– Existing jobs


Analytics

Index impact analysis cont.


Migrating workloads from QWT 4.1.x to DSM


Data Server Manager : Useful web sites

� Download

http://www.ibm.com/developerworks/downloads/im/dsm/index.html

� Youtube

https://www.youtube.com/watch?v=x17hpPi9OkE&list=PLKMYHhPjiLIB2cuz7I_f-VWa9E-

APn-vF


IBM InfoSphere System z Connector for Hadoop

Whether you run Hadoop on z or off z … z/OS Data is important

Transaction and Log Data are consistently PART OF the analysis!

Evidence of the value of combining traditional data

with Hadoop analytics

Source : Gartner Inc, Research Note G00263798- Survey Analysis: Big Data Investment Grows but Deployments Remain Scarce in 2014 . Nick Heudecker, Lisa Kart,Published 09/09/2014

Extract, Transform, and Load Big Data With Apache Hadoop - Whitepaper:

https://software.intel.com/sites/default/files/article/402274/etl-big-data-with-hadoop.pdf

“By most accounts 80%

of the development effort in a big data project goes into data integration

goes towards data analysis.”…and only 20%

Most Hadoop initiatives involve- collecting, - moving,- transforming, - cleansing, - integrating, - exploring, and - analysingvolumes of disparate data sources and types.

Big Data Integration Is Critical For Success With Hadoop


For most, Big Data Integration Must Reach Back to z Systems

• 80% of transaction data is stored or originates on IBM z SystemsMuch of this is because of the scale of the 3,000+ mainframe systems

• Average annual downtime for System z applications is 5 minutes!

• Latest System z hardware capable of running 8,000 virtual linux images

• EAL5+ encryption & cryptographic hardware securing data in motion & at rest

Empowering System z Data Integration with Hadoop & Big Data Initiatives

� Bring together data from both realms � Provide up-to-date information

� Preserve core transactional data security � Minimize overhead and TCO

Z Systems

Open Systems

VSAM IMSLogs

DB2 Oracle

DB2

SQL Server

IDMS Datacom

Teradata

InfoSphere System z Connector for Hadoop (Passport PID 5725S33)

IBM IOP & BigInsights

Big Data

NoSQL

Cloud

Linux file

system &

ETL

Processes

Network

endpoint

IBM System z

Connector for

Hadoop


IBM InfoSphere System z Connector for Hadoop

� Easy to Setup and Use:

– Installs from Linux

– Drag-and-drop interface – no programming required

– COBOL/PL1 copybook parsing & presentation to automate data reformatting

– Automated Code Page Conversion

� Stellar Performance:

– Expect to see dramatic z/OS MIPS usage reduction over home grown & ETL

– Users report three and four times throughput increases

� Broadening Support:

– Multiple Hadoop distributions – on and off the mainframe

– Multiple sources: DB2, VSAM/QSAM, IMS, IDMS, Datacom, JDBC, Log files

– Multiple destinations: Hadoop, Linux File Systems, MongoDB, streaming endpoints

� Feature Rich:

– Define multiple data transfer profiles

– Filter and translate data & columns on the fly, on the target

– Secure pipe (SSL) – RACF and TopSecret integration

– Preserves metadata, landing mainframe data into targets

Key Facts

Behind the Scenes – Real life client flow semplification

40

Transactionaldata

MQFTE

Befo

reA

fter

Transactional data

Staging Staging

Mainframe processing(storage on disc and tape)

Code

System z

Connector

Mainframe • Point & Click Solution (no programming required)• Integrate with RACF mainframe security & scheduling• MIPS consumption 1/10 vs previous/traditional

approach• 4X Speed increase moving detailed data for analysis• No staging + cost saving on storage & software

10 TB daily

• Unacceptable delay with data volume increase• Exceed MIPS consumption & batch window• Security Challenges (unnecessary copies)• Outsourcing labor cost


InfoSphere System z Connector for Hadoop

41• Simplicity, Speed of deployment (no z/OS install), Instant value

through point and click.

• Security (RACF & Top Secret access method so no duplicated profiles),

• End to end data movement & conversion with no staging. The product

does not require multiple steps to make z/OS data available for analytics

(vs. traditional ETL or manual processes). No Cobol, no SQL and no associated cost to maintain those file transfer jobs

• Minimal z/OS CPU usage (MIPS) as all data conversions are happening on Linux processors

• Integration with existing infrastructure (schedulers,...)

Key Benefits


THANK YOU

what's new and exciting in tools for db2 for...

Documents