complex bmc itsm data migrations - software tool house€¦ · complex bmc itsm data migrations ......

25
www.softwaretoolhouse.com Complex BMC ITSM Data Migrations Software Tool House Inc. Ben Chernys Moderator/Presenter Challenges of ITSM Data Migrations, ITSM Data, Migration Projects, Tools, Using Meta-Update, Live Demo, Customer Case Studies BMC RUG Dresden Software Tool House is a BMC Partner 2012-Oct-29 Rüdiger Tams Co-Moderator NTT DATA Germany

Upload: dangxuyen

Post on 25-Jul-2018

222 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Complex BMC ITSM Data Migrations - Software Tool House€¦ · Complex BMC ITSM Data Migrations ... methods described, and any mapping LookUp spreadsheets ... of the volume and performance

www.softwaretoolhouse.com

Complex BMC ITSM

Data Migrations

Software Tool House Inc. Ben Chernys Moderator/Presenter

Challenges of ITSM Data Migrations, ITSM Data, Migration Projects, Tools,

Using Meta-Update, Live Demo, Customer Case Studies

BMC RUG Dresden

Software Tool House

is a BMC Partner

2012-Oct-29

Rüdiger Tams Co-Moderator

NTT DATA Germany

Page 2: Complex BMC ITSM Data Migrations - Software Tool House€¦ · Complex BMC ITSM Data Migrations ... methods described, and any mapping LookUp spreadsheets ... of the volume and performance

www.softwaretoolhouse.com

Objectives

Agenda

Challenges of ITSM Data Migrations

An Introduction into ITSM Forms and Fields

Migration Projects: Typical Milestones and Challenges

Migration Tools & Methods Options

Using Meta-Update for Data Migrations

Live Demo: Real World Complexity CSV CMDB Import

Customer Case Studies

Services of Software Tool House and NTT DATA

Page 3: Complex BMC ITSM Data Migrations - Software Tool House€¦ · Complex BMC ITSM Data Migrations ... methods described, and any mapping LookUp spreadsheets ... of the volume and performance

www.softwaretoolhouse.com

Challenges of ITSM Data Migrations

Complex

Internal Structures Not Documented

Hierarchical trees of records in many

forms

Hidden Redundant References

Migrations

Migrating data across ITSM releases are significant

undertakings

Migrating legacy data or data from other systems is

almost impossible

Onboarding

Can be a complex, time consuming, expensive ARS or ITSM development task using staging forms and merge workflow and a host of tools.

Validations

Validating, Archiving, Merging,

Changing, Reporting, Extracting

hierarchical trees of data

– as a unit – is difficult.

ITSM Data Pain Points

Page 4: Complex BMC ITSM Data Migrations - Software Tool House€¦ · Complex BMC ITSM Data Migrations ... methods described, and any mapping LookUp spreadsheets ... of the volume and performance

www.softwaretoolhouse.com

Clarify to ITSM Challenges

Complex

Internal Structures Not Documented

Hierarchical trees of records in many tables

Hidden Redundant References

Select subsets of companies

Transforms

Login Ids Company Names

Site Names Relationship

Keys Status Fields Clarify Notes Task Types

Support Org, Groups, Staff

Transforms are often complex:

two or more Look Ups

Hierarchical

Clarify data, like ITSM data is a set of (more normalised)

hierarchical data –with different

keys and tables– which must be translated into

ITSM trees.

Ticket State

States must be set appropriately which means all requirements of ITSM workflow

must be met including

generating extra records in the

tree.

Often this means two updates.

ITSM Data Pain Points

Page 5: Complex BMC ITSM Data Migrations - Software Tool House€¦ · Complex BMC ITSM Data Migrations ... methods described, and any mapping LookUp spreadsheets ... of the volume and performance

www.softwaretoolhouse.com

ITSM Data Pain Points

Page 6: Complex BMC ITSM Data Migrations - Software Tool House€¦ · Complex BMC ITSM Data Migrations ... methods described, and any mapping LookUp spreadsheets ... of the volume and performance

www.softwaretoolhouse.com

ITSM Data Pain Points

Page 7: Complex BMC ITSM Data Migrations - Software Tool House€¦ · Complex BMC ITSM Data Migrations ... methods described, and any mapping LookUp spreadsheets ... of the volume and performance

www.softwaretoolhouse.com

Complex, Changing, Undocumented

7.6.3

2,314 Forms

194,790 Fields

7.6.04

2,794 Forms

235,196 Fields

8.0.0

3,188 Forms

249,109 Fields

ITSM Forms and Fields

Restrained, linear progression: New functionality, enhancements, maintenance.

Sign of maturity.

Page 8: Complex BMC ITSM Data Migrations - Software Tool House€¦ · Complex BMC ITSM Data Migrations ... methods described, and any mapping LookUp spreadsheets ... of the volume and performance

www.softwaretoolhouse.com

Fixed Price ARS Data Migrations

Tests

Scripts Analysis

Data Migration Projects

Tasks

Milestones

Typical

Page 9: Complex BMC ITSM Data Migrations - Software Tool House€¦ · Complex BMC ITSM Data Migrations ... methods described, and any mapping LookUp spreadsheets ... of the volume and performance

www.softwaretoolhouse.com

Data Migration Projects Tasks Milestones Description

Data

Analysis

Running of scripts and SQL queries to collect reports on productions data. Q & A on data

relationships, migration needs.

Data

Summary

Inventory of production data. Spreadsheets and documents listing forms, records counts, sets of

different data statistics (CMDB by Asset Type, PCT, etc.)

Data Work

Shops

Q & A on data relationships, migration needs with data users and managers.

Migration

Summary

Documents the data that to be migrated and to be left. Built from the Data Summary.

Transform

Work

Shops

Required data transforms are identified, methods described, and any mapping LookUp spreadsheets

identified.

Mapping

Sheets

Documents transforms, methods, and LookUp sheets. Sheets are populated with source / Traget data

and instructions.

Mapping

Sheets

Developed

Customer responsibility with guidance. Continuously tested as scripts developed.

Script

Developed

Scripts are modified and developed to implement all Mapping transforms. Small subsets of data are

often migrated and updated.

Scripts

Delivered

Scripts are ready and a Usage Guide is published listing the script running order and dependencies.

Mapping

Sheets

Completed

Customer completes all transform sheets.

Migration Tests are scheduled.

Full

Migration

Tests

Full data is migrated according to the Usage Guide to measure correctness, completeness, volume,

performance.

Migration

Plan

A step by step plan is documented for the production migration weekend or longer if required because

of the volume and performance parameters.

Migration

Weekend

The Migration plan is carried out step by step. When complete, customer people validate the data and

make a Go / No Go decision.

Post

Migration

Support

Immediate support for any data issues on new system. Missing records investigated, LookUps or

Source corrected, and migrated.

Migration

Results

Summary of data migrated, immediate post production support. Guide to corrective LookUp changes

for remaining missing records.

Migration

Support

Assistance with individual runs and LookUp corrections.

Page 10: Complex BMC ITSM Data Migrations - Software Tool House€¦ · Complex BMC ITSM Data Migrations ... methods described, and any mapping LookUp spreadsheets ... of the volume and performance

www.softwaretoolhouse.com

ITSM Forms and Fields

ARS Import Export

Manual exports to

CSVs, Excel to change values, manual imports.

Automated with staging forms and

import workflow: a

time-consuming, expensive Workflow Project.

Web Services

Real time, retries,

queuing, logging is

implementation (staging

form) dependent

Automated with staging forms and

import workflow

similar to ARS Import / Export

Migrator DSO

Automated with complex

Merge workflow.

May require changes to application.

Workflow Project can

approach API costs.

AIE AI

Limited mapping

capabilities.

Fixed targets: multiple

Exchanges / Outputs.

Point & Click interface

cumbersome to set

qualifications, conditions.

Needs Recon jobs.

Hard to debug.

API Perl

Java c

Sure way to automate.

Expensive and slow to

develop.

Usually needs outside, non-Remedy staff.

SQL

Dangerous to the integrity of

the ARS system data.

Expensive and slow to

develop.

Usually needs outside, non-Remedy staff.

Page 11: Complex BMC ITSM Data Migrations - Software Tool House€¦ · Complex BMC ITSM Data Migrations ... methods described, and any mapping LookUp spreadsheets ... of the volume and performance

www.softwaretoolhouse.com

Meta-Update In a Nutshell

ETL – Extract Transform Load – Scripting tool

• Meta-Update allows any BMC Remedy administrator / developer to automate any ETL data operation, using the API, with no need for API or

programming skills at all!

with no staging forms,

workflow, customizations, or

server changes at all!

• With Meta-Update, one develops proven, repeatable scripts that can be used over and over again,

Page 12: Complex BMC ITSM Data Migrations - Software Tool House€¦ · Complex BMC ITSM Data Migrations ... methods described, and any mapping LookUp spreadsheets ... of the volume and performance

www.softwaretoolhouse.com

Easy, Quick, Robust — Savings for any Remedy shop

Meta-Update In a Nutshell

Simple

One single “ini” file

No Staging forms and

Merge workflow

Scripts can be done in minutes

Time To Market

Order of magnitude

reductions in time and efforts

Clean

Uses BMC Remedy API

No ARS / ITSM server

changes

No -ITSM- Customization

Auditable

Logging with full control

Configurable Record Audits

Remedy Client Logging

Page 13: Complex BMC ITSM Data Migrations - Software Tool House€¦ · Complex BMC ITSM Data Migrations ... methods described, and any mapping LookUp spreadsheets ... of the volume and performance

www.softwaretoolhouse.com

Meta-Update In a Nutshell

Meta-Update — extends the concept of a field reference and adds value transform features.

• ARS: $ Field $ • Meta-Update: $ Tag, Field $

With a few simple words, queries and files are iterated, records looked up, references are made and used, and with all these Tags in memory at the same time, updates are made and files are output.

Page 14: Complex BMC ITSM Data Migrations - Software Tool House€¦ · Complex BMC ITSM Data Migrations ... methods described, and any mapping LookUp spreadsheets ... of the volume and performance

www.softwaretoolhouse.com

Meta-Update Jobs

How Meta-Update Helps

Migrations

ITSM releases, Legacy apps

Imports

On-boarding, foundation data, CMDB data

Corrections

Change, merge, adjust data at will

Extracts

Save and create files of any kind

Integrations .

Validations

Page 15: Complex BMC ITSM Data Migrations - Software Tool House€¦ · Complex BMC ITSM Data Migrations ... methods described, and any mapping LookUp spreadsheets ... of the volume and performance

www.softwaretoolhouse.com

Meta-Update script functions gives you better control over your data manipulation tasks.

Meta-Update is a script based solution. It is easier to learn than a "pure" development language like Java or C#. People who are used to this should be able to do scripting after some days.

Meta-Update "hides" necessary function calls you normally need when using a development language.

You can combine the scripts with e.g. ARS escalations to run them on a time triggered level.

Re-use of scripts possible and with this a fast adaptability to new requirements.

With a Meta-Update license different additional helper-tools are delivered to make your work easier.

Experiences and Impressions with Meta-Update

in a multi-national company project

Page 16: Complex BMC ITSM Data Migrations - Software Tool House€¦ · Complex BMC ITSM Data Migrations ... methods described, and any mapping LookUp spreadsheets ... of the volume and performance

www.softwaretoolhouse.com

Some points to keep in mind:

If not structured accordingly you could loose overview in big scripts – so stick to the recommended structure.

When developing different scripts in a team, you should use a Source Control System like SVN or MS Visual SourceSafe

Performance of data transfer: C API gives you a limit – but Meta-Update already caches data structures to make the best out of it.

“Meta-Update can give you much more power and control over your BMC Remedy ITSM installation and in combination with existing tools, like BMC Remedy AI, it can make your life much easier, e.g.

when you need to handle many data manipulations.”

Rüdiger Tams, NTT DATA Germany

Experiences and Impressions with Meta-Update

in a multi-national company project

Page 17: Complex BMC ITSM Data Migrations - Software Tool House€¦ · Complex BMC ITSM Data Migrations ... methods described, and any mapping LookUp spreadsheets ... of the volume and performance

www.softwaretoolhouse.com

A UsedBy and OwnedBy column contain lists

and need to generate appropriate associations.

Needs 2nd CSV to

get Class, CTI.

Lists of Login IDs

need to build CI

Associations.

Class dependent

attributes.

BMC_Disk “Size”

attribute is a decimal.

Needs normalization.

Live Demo

Time and effort savings

Sample CSV CI Import

Page 18: Complex BMC ITSM Data Migrations - Software Tool House€¦ · Complex BMC ITSM Data Migrations ... methods described, and any mapping LookUp spreadsheets ... of the volume and performance

www.softwaretoolhouse.com

Case Study: CSV CI Import

Needs 2nd CSV to

get Class, CTI.

Time and effort savings

Sample CSV CI Import

Page 19: Complex BMC ITSM Data Migrations - Software Tool House€¦ · Complex BMC ITSM Data Migrations ... methods described, and any mapping LookUp spreadsheets ... of the volume and performance

www.softwaretoolhouse.com

Live Demo

Live Demo

Page 20: Complex BMC ITSM Data Migrations - Software Tool House€¦ · Complex BMC ITSM Data Migrations ... methods described, and any mapping LookUp spreadsheets ... of the volume and performance

www.softwaretoolhouse.com

Reference: Logica UK

Case Studies

Meta-Update migration scripts were entirely customer written by their off-shore team directed by the UK Transition Manager after attending a one-hour WebEx meeting with two colleagues.

“Heavily customised” ITSM 5 → ITSM 7 data migration comprising Foundation and Transaction (Help Desk, Changes, Assets) for 400+ customers including 1500+ SLAs.

"We had a mammoth task of migrating 400+ companies from ITSM 5.5 to ITSM 7.1. Initially this task looked very complicated as the 5.5 system was highly customized to suit individual customer's requirements and the migration into mismatched schemas looked impossible. 1500 SLAs for these customers added to the complications.

“But when we started evaluating Meta-Update, we saw some light. We can now migrate SLAs, implement complicated transformation rules, stamp mapping values by looking up external configuration files, read from multiple servers, etc. Best part is that once Meta-Update scripts are done and tested, you can repeat it for any number of times.

“I recommend Meta-Update for all those who want the flexibility of a scripting language to handle bulk ARS data."

Transition manager:

"We used Meta Update to migrate Foundation, Asset, SLM and Ticket data from ITSM 5.5 to ITSM 7. Meta Update enabled us to perform this complex migration for many customers in record time. Without Meta Update. we would not have been able to achieve this task.

Project manager:

Page 21: Complex BMC ITSM Data Migrations - Software Tool House€¦ · Complex BMC ITSM Data Migrations ... methods described, and any mapping LookUp spreadsheets ... of the volume and performance

www.softwaretoolhouse.com

Reference: Dataport

Case Studies

• 35K Changes

• 65K CIs

• 120K Help Desk Tickets

• 120K Associations

• 80K Custom Records

ITSM 6 → 7 “Transaction Data” Migration

• Milestone based payments

Fixed price Monday morning, all open tickets and changes were ready for work with attachments and diary fields in work logs, all associations in place, assigned to the correct people and groups.

Migration across a weekend. Significant foundation data changes: Support Groups, Sites, Organisation data; substantial data cleansing; some missing requesters and support staff – all automated.

Page 22: Complex BMC ITSM Data Migrations - Software Tool House€¦ · Complex BMC ITSM Data Migrations ... methods described, and any mapping LookUp spreadsheets ... of the volume and performance

www.softwaretoolhouse.com

Reference: Met Office

Case Studies

Problem that had been in place for one year solved in a week with Meta-Update.

A single Meta-Update script read a control CSV that gave source CTI and new target classes. Script fixed Product Catalogue, replicated CIs, Work Logs, audit records, associations and relationships. Then ran with a –del to delete original CIs.

“Bad ITSM migration” caused 12,000 non-computer systems in CMDB Class Computer System. Worked a year: associated with tickets, changes, contracts, people, related to other CIs.

Page 23: Complex BMC ITSM Data Migrations - Software Tool House€¦ · Complex BMC ITSM Data Migrations ... methods described, and any mapping LookUp spreadsheets ... of the volume and performance

www.softwaretoolhouse.com

Reference: Multi-national in Germany

Case Studies

Developed in 1 man * 2 months. Clarify shut down on Friday. All tickets being worked on in ITSM on Monday. All historical data in place.

Migration of a running Clarify Help Desk into a production ITSM 7.6 Incident, Problem, Task.

Meta-Update loaded all foundation data including People, Support Staff, Company Organisation, Sites, Operational Catalogue, Product Catalogue, CMDB CIs & relationships.

New ITSM 7.5 full suite implementation. Migration from Customer Developed CMDB & Identity Management Database application with over 200 tables, as well as Support Staff, rights, groups, structure, Change & Task Templates, Approval Groups and members from 4 independent ITSM 6 servers.

Clarify Help Desk ITSM 6 & Bespoke CMDB & Identity Mgmt.

Page 25: Complex BMC ITSM Data Migrations - Software Tool House€¦ · Complex BMC ITSM Data Migrations ... methods described, and any mapping LookUp spreadsheets ... of the volume and performance

www.softwaretoolhouse.com

Next Steps www.SoftwareToolHouse.com

Check out the web site

Pick up a trial license with our Live Install and

Scripting 101 presentation

Download the Scripting 101 presentation and

the Meta-Update User’s Guide

Look at the samples in the Script Library

Contact Us at www.softwaretoolhouse.com

Ben Chernys

+49 171 380 2329

Rüdiger Tams

+49 151 1680 2589