2012 ibm tivoli workload automation - batch modernization
DESCRIPTION
2012 IBM Tivoli Workload Automation - Batch ModernizationTRANSCRIPT
IBM Presentation Template Full VersionProduct Management
TWA Architectures
Ever asked yourself these questions?
What’s the impact on workload automation of adopting a full Java-based application park?
How to integrate my e-business applications into my workload automation tool?
What I need to do to incorporate into workload automation new applications when they emerge?
Then this presentation is for YOU!
TWA Architectures
Source: “Consider Scheduling Tools for Batch Application Integration”, Gartner.
Source: IDC: What % server resource is used for “bulk” and “batch” computing by platform
Batch is and will remain a fundamental asset in large enterprise computing
Remains the best IT metaphor for bulk business processes
Batch volume is increasing year on year (IDC)
Addresses economies of scale and efficiencies necessary to process large quantities of data
TWA Architectures
The batch still represents a business critical asset in most organizations, deserving an increasing percentage of server resources, as indicated in this IDC report, where you can see that
The best example to describe this strategy is in distribution: You can buy a book from a book store in one transaction, but books are shipped in bulk, not one at a time, so the bulk process, the batch process is needed to address economies of scale and make the business model more profitable. Batch is still the best paradigm to process that data, to perform repetitive tasks, periodic calculations, multi-steps processes, it has been used for a variety of workload types, including data manipulation or conversion, printing/reporting, database updating, cyclic processing (end of day, end of week, end of month), file backups or to process large volumes of calculated business data (financial, payroll, inventory, etc.).
But the book store does not ship books from the warehouse one book at a time -- books are shipped in bulk. That (bulk) feature of the process lends efficiencies through economies of scale to the business model -- making the business model less costly and therefore more profitable. Considering the role of bulk activities within the business process is the starting point. Once you have identified the bulk activities within you business processes, and the business outcomes that produces, you can then begin to map the automation of those bulk activities to Batch services and programs in your information system. That allows you to recognize and rationalize the role of those batch programs within your information -- including what should remain as a Batch program, what should be introduced as a Batch program, what should be converted to a Batch program, and what should not.
Chart1
Mainframe
Mainframe
Infrastructure services
All layers above this interacts with or uses the services from the core OS
Batch application development and delivery
Environment for creating and migrating bulk applications
System management and operations
Analytics
Partner services
Scheduler services
Invocation services
Ad hoc
Analytics
Scheduler services
TWA Architectures
Around the batch, a layered architecture has been created that represents the logical artifacts, the services, components and data flows needed to implement batch business processes,
- Batch needs an application development and delivery environment, and environment to manage the batch lifecycle.
It needs an execution environment, that comprises: invocation and scheduling services – batch needs to be initiated or invoked, and this can be done on an ad-hoc or planned basis – data access management – typicall batch processes high data volumes which must be stored in data arehouses, and then the infrastructural services, that represent the base of the operational environment – hardware, network, data storage. The analytics layer applies to any other layer and system management, which includes scheduling, monitoring and brokering the batch jobs.
Tivoli Workload Automation enables service-driven management and control of heterogeneous batch tasks under a single point of governance, and brokers batch tasks over resources according to service level agreements defined on the batch.
© 2011 IBM Corporation
Today’s batch
Batch workload may be an issue in most IT transformation projects – inability to reuse and integrate assets
Cost and complexity of maintaining and operating existing batch applications continues to grow – creating demand for improved runtimes and tools
Gaps in technology and skills
Variants of batch processing are emerging
that run on new platforms, infrastructure
and middleware
The context
TWA Architectures
Today the business organizations relies for a 70%-80% - according to the analysts - on a ground of legacy applications – both online and batch – tipically developed in C, C++, Assembler and COBOL. Additionally, applications have been deeply customized along the years, according to a strategic vertical approach, for which the more the applications were customized, specialized, the more they produced a competitive advantage. This approach, that has produced the expected business results, is posing some challenges/issues today.
Operation challenges – Those deeply customized applications are rigid, and can be hardly integrated with the modern applications entering the business, which is more evident today, since the web applications, as well as other forms of batch processing that run on new platforms, infrastructures, and middleware, are emerging.
Business challenges – Even for the only maintenace of those applications, organization must develop special skills and cannot see a fast return of investment. Last, but not least, I want to mention the cost that the ground of applications brings to the organizations, since they load the mainframe CPU.
Many of these are proprietary implementations, built several
years ago, to cater to specific requirements.
© 2011 IBM Corporation
Elastic batch, flexible, able to change at the speed of the business – faster turnaround to implement newer or modified business processes
Reduce maintenance and skill costs
Satisfy new functional requirements
model
The needs
TWA Architectures
Again, a fast return of investment for implementing new business processes or modifying existing business processing, therefore flexibility, abiltiy to rapidly change at the speed of the business.
There are also new functional requirements emerging... For example:
Creating PDFs... Many banks send the statement accounts to their customers in form of PDFs
Sending e-mails – still to remain with the Bank example, banks need to send e-mail confirmation for notification that a payment is processed
Creating .xls or .doc, to be exchanged with third parties or other companies
Generating graphics
Examples of batch modernization
Packaged solutions and technologies
Re-write all batch applications into modern languages (ex. COBOL to Java)
Global modernization
Integrate: add some Java, reuse COBOL
Incremental modernization
Adopt fit-for-purpose workload placement
Some customers are replacing customer-developed code for batch with commercial off-the-shelf packages.
Other customers are rewriting all COBOL-based batch applications into Java-based applications. Notice that this is not a recommended approach by IBM, IBM does not recommend customers to modernize all their COBOL batch inventory, COBOL is performing very well, provides real business value and rewriting it totally is a very risky process. But some customers are doing it anyway or at least have the project to do it step-by-step.
But the batch modernization is not a kind of “all or nothing” approach. Rather, batch modernization encompasses a wide range of projects going from:
Implement new business logic in Java and integrate it with existing logic
Share existing COBOL modules across Java and COBOL domains
Convert a portion of COBOL applications to Java or gradually convert COBOL to Java.
A key word in the batch modernization is the “reuse”. Implementing the batch modernization through the reuse is very important, because it helps saving money, energy, resources, guarantees stability, leverages the product lifecycle.
SwissRe is in the process of a three-phase migration project that will eliminate a previously developed, homegrown batch Java infrastructure, leverage and transition its existing batch COBOL portfolio, and transition to an implementation of common business services that will be used across its online and batch processing environments.
Phase 1 implement new business logic in Java with WAS XD CG
Phase 2 share existing COBOL modeuls acros Java and COBOL domains
Phase 3 incrementally migrate COBOL modules to Java, running in XD CG.
© 2011 IBM Corporation
Batch Modernization increases flexibility for business and IT
Customers are modernizing batch infrastructure to make it more flexible, and more responsive to new functional and business requirements
Re-using existing assets with modern interfaces, integrating traditional and cloud workloads, moving workloads and operational point.
Transform: batch applications using modern languages (ex. COBOL to Java)
Re-use: existing applications with business oriented Web Services
Integrate: legacy applications with new applications
Examples of batch modernization projects
TIME TO BUILD NEW FEATURES
BATCH MODERNIZATION
TWA Architectures
Tivoli Workload Scheduler helps supporting transformation, re-using and integration projects
TWS enables invoking scheduling services as Java API, and embrace scheduling of Java classes, in a closed loop.
Enable wrapping existing scheduling services with web services
Edit and submit jobstreams with variable substitution
Link TWS with e-business applications
Extensible framework through application plug ins, to extend the automation to potentially any new job types
Tivoli Workload Scheduler supports batch modernization projects
Business benefits
Re-use of existing processes running rather than encouraging a re-write
Reduce costs offloading MIPS to zAAP
Enable easy remote access to scheduling services
Proofpoints – Customer quotes
Transform: batch applications using modern languages (ex. COBOL to Java)
Re-use: existing applications with business oriented Web Services
Integrate: legacy applications with new applications
TWA Architectures
© 2011 IBM Corporation
Application Extensions allow business users to take advantage of processes in a managed approach
Customers are shifting from traditional backend transaction focused systems to modern systems running new and heterogeneous applications
Workload Automation role is to expand the automation to additional processes, keeping governance and control
TWS allows building application plug ins for any new applications, creating an ecosystem of partners willing to explore this capability
Business benefits
Share infrastructure among applications
Reduces labor costs, enabling to automate new workloads with the same staff of people
No request for new skill: re-using of workload automation processes and procedures already in place
Proofpoints – Customer quotes
Extensible framework through application plug ins
SAP
Oracle
PeopleSoft
© 2011 IBM Corporation
What we offer to our partners (Clients, BP, System Integrators) for new Application Plug-ins implementation
Extensible framework application Plug-ins : details
Tivoli Workload Scheduler Integration Workbench
Takes you through the creation of your plug-ins
Two main phases:
Application Plug-in
(jar file)
After deployment of new Application Plug-in, you will be able to manage the new job type in the same fashion as all other TWA job typesd creation (JSDL)
TWA Architectures
Panels for definition of the parameters which are needed to be specified for the type of job you are creating.
For example, if you are creating a job that launches a request over the HTTP, you probably have to define parameters.........
When defining your parameters, you have to specify captions to every object in your interface panel, you can create radio button groups, combobox
While creating the executor, you can create the method for validating your definitions
© 2011 IBM Corporation
Batch Modernization is not a kind of all or nothing approach: it may be worth to leave COBOL on System z and offload only applications that are better served or more economically served on distributed machines
zEnterprise is ideal to do so, in that it allows to place workloads where they best fit, while keeping a cohesive – and therefore predictable – environment
Tivoli Workload Automation and zEnterprise
Tivoli Workload Automation and zEnterprise together deliver exceptional workload management capabilities, relying on a predictable and consolidated environment. A closely federated set of resources and a central management for end-to-end workloads support cohesion of heterogeneous resources and enhance the ability to respond to demanding business changes.
c
Blades and Accelerators
System z Host
Private Management Network
Future option to exploit Unified Resource Management interfaces would provide unprecedented workload moving and optimization capabilities
Fit for purpose workload deployment
Reduce costs with fit-for-purpose platform, and implement a virtualized and green data center
Realize data-proximity processing with high bandwidth for distributed applications
Business benefits
TWS for z/OS
TWA Architectures
Ever asked yourself these questions?
What’s the impact on workload automation of adopting a full Java-based application park?
How to integrate my e-business applications into my workload automation tool?
What I need to do to incorporate into workload automation new applications when they emerge?
Then this presentation is for YOU!
TWA Architectures
Source: “Consider Scheduling Tools for Batch Application Integration”, Gartner.
Source: IDC: What % server resource is used for “bulk” and “batch” computing by platform
Batch is and will remain a fundamental asset in large enterprise computing
Remains the best IT metaphor for bulk business processes
Batch volume is increasing year on year (IDC)
Addresses economies of scale and efficiencies necessary to process large quantities of data
TWA Architectures
The batch still represents a business critical asset in most organizations, deserving an increasing percentage of server resources, as indicated in this IDC report, where you can see that
The best example to describe this strategy is in distribution: You can buy a book from a book store in one transaction, but books are shipped in bulk, not one at a time, so the bulk process, the batch process is needed to address economies of scale and make the business model more profitable. Batch is still the best paradigm to process that data, to perform repetitive tasks, periodic calculations, multi-steps processes, it has been used for a variety of workload types, including data manipulation or conversion, printing/reporting, database updating, cyclic processing (end of day, end of week, end of month), file backups or to process large volumes of calculated business data (financial, payroll, inventory, etc.).
But the book store does not ship books from the warehouse one book at a time -- books are shipped in bulk. That (bulk) feature of the process lends efficiencies through economies of scale to the business model -- making the business model less costly and therefore more profitable. Considering the role of bulk activities within the business process is the starting point. Once you have identified the bulk activities within you business processes, and the business outcomes that produces, you can then begin to map the automation of those bulk activities to Batch services and programs in your information system. That allows you to recognize and rationalize the role of those batch programs within your information -- including what should remain as a Batch program, what should be introduced as a Batch program, what should be converted to a Batch program, and what should not.
Chart1
Mainframe
Mainframe
Infrastructure services
All layers above this interacts with or uses the services from the core OS
Batch application development and delivery
Environment for creating and migrating bulk applications
System management and operations
Analytics
Partner services
Scheduler services
Invocation services
Ad hoc
Analytics
Scheduler services
TWA Architectures
Around the batch, a layered architecture has been created that represents the logical artifacts, the services, components and data flows needed to implement batch business processes,
- Batch needs an application development and delivery environment, and environment to manage the batch lifecycle.
It needs an execution environment, that comprises: invocation and scheduling services – batch needs to be initiated or invoked, and this can be done on an ad-hoc or planned basis – data access management – typicall batch processes high data volumes which must be stored in data arehouses, and then the infrastructural services, that represent the base of the operational environment – hardware, network, data storage. The analytics layer applies to any other layer and system management, which includes scheduling, monitoring and brokering the batch jobs.
Tivoli Workload Automation enables service-driven management and control of heterogeneous batch tasks under a single point of governance, and brokers batch tasks over resources according to service level agreements defined on the batch.
© 2011 IBM Corporation
Today’s batch
Batch workload may be an issue in most IT transformation projects – inability to reuse and integrate assets
Cost and complexity of maintaining and operating existing batch applications continues to grow – creating demand for improved runtimes and tools
Gaps in technology and skills
Variants of batch processing are emerging
that run on new platforms, infrastructure
and middleware
The context
TWA Architectures
Today the business organizations relies for a 70%-80% - according to the analysts - on a ground of legacy applications – both online and batch – tipically developed in C, C++, Assembler and COBOL. Additionally, applications have been deeply customized along the years, according to a strategic vertical approach, for which the more the applications were customized, specialized, the more they produced a competitive advantage. This approach, that has produced the expected business results, is posing some challenges/issues today.
Operation challenges – Those deeply customized applications are rigid, and can be hardly integrated with the modern applications entering the business, which is more evident today, since the web applications, as well as other forms of batch processing that run on new platforms, infrastructures, and middleware, are emerging.
Business challenges – Even for the only maintenace of those applications, organization must develop special skills and cannot see a fast return of investment. Last, but not least, I want to mention the cost that the ground of applications brings to the organizations, since they load the mainframe CPU.
Many of these are proprietary implementations, built several
years ago, to cater to specific requirements.
© 2011 IBM Corporation
Elastic batch, flexible, able to change at the speed of the business – faster turnaround to implement newer or modified business processes
Reduce maintenance and skill costs
Satisfy new functional requirements
model
The needs
TWA Architectures
Again, a fast return of investment for implementing new business processes or modifying existing business processing, therefore flexibility, abiltiy to rapidly change at the speed of the business.
There are also new functional requirements emerging... For example:
Creating PDFs... Many banks send the statement accounts to their customers in form of PDFs
Sending e-mails – still to remain with the Bank example, banks need to send e-mail confirmation for notification that a payment is processed
Creating .xls or .doc, to be exchanged with third parties or other companies
Generating graphics
Examples of batch modernization
Packaged solutions and technologies
Re-write all batch applications into modern languages (ex. COBOL to Java)
Global modernization
Integrate: add some Java, reuse COBOL
Incremental modernization
Adopt fit-for-purpose workload placement
Some customers are replacing customer-developed code for batch with commercial off-the-shelf packages.
Other customers are rewriting all COBOL-based batch applications into Java-based applications. Notice that this is not a recommended approach by IBM, IBM does not recommend customers to modernize all their COBOL batch inventory, COBOL is performing very well, provides real business value and rewriting it totally is a very risky process. But some customers are doing it anyway or at least have the project to do it step-by-step.
But the batch modernization is not a kind of “all or nothing” approach. Rather, batch modernization encompasses a wide range of projects going from:
Implement new business logic in Java and integrate it with existing logic
Share existing COBOL modules across Java and COBOL domains
Convert a portion of COBOL applications to Java or gradually convert COBOL to Java.
A key word in the batch modernization is the “reuse”. Implementing the batch modernization through the reuse is very important, because it helps saving money, energy, resources, guarantees stability, leverages the product lifecycle.
SwissRe is in the process of a three-phase migration project that will eliminate a previously developed, homegrown batch Java infrastructure, leverage and transition its existing batch COBOL portfolio, and transition to an implementation of common business services that will be used across its online and batch processing environments.
Phase 1 implement new business logic in Java with WAS XD CG
Phase 2 share existing COBOL modeuls acros Java and COBOL domains
Phase 3 incrementally migrate COBOL modules to Java, running in XD CG.
© 2011 IBM Corporation
Batch Modernization increases flexibility for business and IT
Customers are modernizing batch infrastructure to make it more flexible, and more responsive to new functional and business requirements
Re-using existing assets with modern interfaces, integrating traditional and cloud workloads, moving workloads and operational point.
Transform: batch applications using modern languages (ex. COBOL to Java)
Re-use: existing applications with business oriented Web Services
Integrate: legacy applications with new applications
Examples of batch modernization projects
TIME TO BUILD NEW FEATURES
BATCH MODERNIZATION
TWA Architectures
Tivoli Workload Scheduler helps supporting transformation, re-using and integration projects
TWS enables invoking scheduling services as Java API, and embrace scheduling of Java classes, in a closed loop.
Enable wrapping existing scheduling services with web services
Edit and submit jobstreams with variable substitution
Link TWS with e-business applications
Extensible framework through application plug ins, to extend the automation to potentially any new job types
Tivoli Workload Scheduler supports batch modernization projects
Business benefits
Re-use of existing processes running rather than encouraging a re-write
Reduce costs offloading MIPS to zAAP
Enable easy remote access to scheduling services
Proofpoints – Customer quotes
Transform: batch applications using modern languages (ex. COBOL to Java)
Re-use: existing applications with business oriented Web Services
Integrate: legacy applications with new applications
TWA Architectures
© 2011 IBM Corporation
Application Extensions allow business users to take advantage of processes in a managed approach
Customers are shifting from traditional backend transaction focused systems to modern systems running new and heterogeneous applications
Workload Automation role is to expand the automation to additional processes, keeping governance and control
TWS allows building application plug ins for any new applications, creating an ecosystem of partners willing to explore this capability
Business benefits
Share infrastructure among applications
Reduces labor costs, enabling to automate new workloads with the same staff of people
No request for new skill: re-using of workload automation processes and procedures already in place
Proofpoints – Customer quotes
Extensible framework through application plug ins
SAP
Oracle
PeopleSoft
© 2011 IBM Corporation
What we offer to our partners (Clients, BP, System Integrators) for new Application Plug-ins implementation
Extensible framework application Plug-ins : details
Tivoli Workload Scheduler Integration Workbench
Takes you through the creation of your plug-ins
Two main phases:
Application Plug-in
(jar file)
After deployment of new Application Plug-in, you will be able to manage the new job type in the same fashion as all other TWA job typesd creation (JSDL)
TWA Architectures
Panels for definition of the parameters which are needed to be specified for the type of job you are creating.
For example, if you are creating a job that launches a request over the HTTP, you probably have to define parameters.........
When defining your parameters, you have to specify captions to every object in your interface panel, you can create radio button groups, combobox
While creating the executor, you can create the method for validating your definitions
© 2011 IBM Corporation
Batch Modernization is not a kind of all or nothing approach: it may be worth to leave COBOL on System z and offload only applications that are better served or more economically served on distributed machines
zEnterprise is ideal to do so, in that it allows to place workloads where they best fit, while keeping a cohesive – and therefore predictable – environment
Tivoli Workload Automation and zEnterprise
Tivoli Workload Automation and zEnterprise together deliver exceptional workload management capabilities, relying on a predictable and consolidated environment. A closely federated set of resources and a central management for end-to-end workloads support cohesion of heterogeneous resources and enhance the ability to respond to demanding business changes.
c
Blades and Accelerators
System z Host
Private Management Network
Future option to exploit Unified Resource Management interfaces would provide unprecedented workload moving and optimization capabilities
Fit for purpose workload deployment
Reduce costs with fit-for-purpose platform, and implement a virtualized and green data center
Realize data-proximity processing with high bandwidth for distributed applications
Business benefits
TWS for z/OS