db2 migration -_tips,_tricks,_and_pitfalls

22
1 DB2 Migration – Tips, Tricks, and Pitfalls 1.1 Overview The following sections include tips and tricks for migrating from DB2 environments to Teradata. The concepts put forth here were derived (and in some cases updated) from two web casts that are available on Teradata University. They are: Migrating Your DB2 Database to Your Teradata Database Webcast (about 74 minutes long) Migrating your DB2 Applications to your Teradata Data Warehouse Webcast (about 61 minutes long) If you wish to view these web casts in their entirety, log on to Teradata University and search for “DB2”. These courses will appear in the search results. You can then choose to enroll and take the courses. 1.2 Key Information The following is a list of information that is important to gather in advance of planning your DB2 to Teradata Migration. Production Job Schedule – It is important to understand what each production job you need to convert is. You should understand the types of jobs (autoloader, optiload, cube build, etc), from what platform they run, and what tools are employed if any. Concurrency/Dependency of Load Jobs (How many at once and to what tables in what tablespaces). Load / Batch Windows, SLAs, and procedures if SLAs are missed. What Scheduler and Version Control software is being used and on what platform? What kind of connectivity is available for Teradata to the Source (e.g. Channel Connect – ESCON, etc.) Is there any use of native CLI?

Upload: sam2sung2

Post on 20-May-2015

1.565 views

Category:

Technology


2 download

TRANSCRIPT

Page 1: Db2 migration -_tips,_tricks,_and_pitfalls

1 DB2 Migration – Tips, Tricks, and Pitfalls

1.1 OverviewThe following sections include tips and tricks for migrating from DB2 environments to Teradata. The concepts put forth here were derived (and in some cases updated) from two web casts that are available on Teradata University. They are:

Migrating Your DB2 Database to Your Teradata Database Webcast (about 74 minutes long)

Migrating your DB2 Applications to your Teradata Data Warehouse Webcast (about 61 minutes long)

If you wish to view these web casts in their entirety, log on to Teradata University and search for “DB2”. These courses will appear in the search results. You can then choose to enroll and take the courses.

1.2 Key Information The following is a list of information that is important to gather in advance of planning

your DB2 to Teradata Migration.

Production Job Schedule – It is important to understand what each production job you need to convert is. You should understand the types of jobs (autoloader, optiload, cube build, etc), from what platform they run, and what tools are employed if any.

Concurrency/Dependency of Load Jobs (How many at once and to what tables in what tablespaces).

Load / Batch Windows, SLAs, and procedures if SLAs are missed.

What Scheduler and Version Control software is being used and on what platform?

What kind of connectivity is available for Teradata to the Source (e.g. Channel Connect – ESCON, etc.)

Is there any use of native CLI?

Reverse engineered Erwin Models from all warehouses

Are there any application / Non-DBMS (or coexisting processes on same node) node analysis, End User Access

What processes are being performed on these nodes, ETL, reporting?

Representative sample of applications and queries from user community.

What user tools / applications are running against which systems?

What access methods (ODBC, JDBC, SQLJ, SPUFI, QMF, QMF Windows, mainframe, etc…) are used?

What is the level of ad-hoc usage vs. canned queries, What are these queries/reports written in? (COBOL, Java, Power Builder, Visual Basic, Cognos, etc.)

Is there and Data Replication (DPROP/DPROPR)

Which tables, when, what sort of process?

ETL Phase and Methodology

Page 2: Db2 migration -_tips,_tricks,_and_pitfalls

o Processed on the mainframe/what flavor of Unix/Windows/ETL Tool/combination?

o Are SQL Processes used in ETL?o Is there INSERT, UPDATE, DELETE, UPSERT logic?o Are Temp Tables?o Does old data roll off, when or how often, how much?

What is the Refresh Methodology currently

o Monthly?

o Daily?

o Continual?

o Triggers, MQSeries?

o RI?

Across all of history or just insert new data? Relative Table sizing in rows and gigabytes Partition key analysis for DB2 – What tables belong to what tablespaces which existing

on which node groups?

1.3 Migration OverviewA DB2 Migration consists of three distinct functions:

Establishing the Teradata Data Warehouse Infrastructure Migrating the existing DB2 Environment to the new Teradata Environment Establishing a production environment

1.3.1 Establishing the Teradata Data Warehouse InfrastructureThis phase establishes the hardware and software environment that will support the new warehouse environment. This includes Teradata and non-Teradata environments for development, test, QA, and production. Major activities include:

Prepare data center Establish Teradata environment Establish BAR environment Establish network connectivity Establish ETL server(s) and cloning of old environment Establish BI server(s) and cloning of old environment

1.3.2 Migrating the existing DB2 Environment to the new Teradata Environment

This involves migrating all the data, processes and applications associated with the DB2 Environment to Teradata. Major areas of focus include:

Physical Database – Design and Build of the physical database with modifications made to take into consideration the differences between DB2 and Teradata

Database Security – Design and Build of the security views. Possibly implementation of LDAP

Page 3: Db2 migration -_tips,_tricks,_and_pitfalls

ETL – Migration of the ETL that will maintain the data. Additionally, all supporting process and scheduling will migrate

Post Load Processes – Migration of processes that are performed after the load is complete. For example: post load reports

Historical Data Migration – The migration and validation of the historical data that is in the current DB2 system

Reinstatement of Archive data – It is normal process to archive data to tape. This activity sets up the process to allow retrieval of the DB2 archive data on to the new EDW.

User Table Migration – Migrate the tables and possibly the data created by users

BI Migration – Migrate the BI applications to Teradata

Regression / Benchmark Test Suite – Establish a set of scripts that will be utilized to determine the impact of upgrades or changes in the new environment

Continuity of Business (COB) – Design and build the COB solution for the new warehouse

Backup, Archive and Recovery (BAR) – Design and build the BAR solution for the new warehouse

Testing – Define the test and execute the different levels of testing:

o Unit Test

o Integration Test

o System Test

o User Validation Test

o User Parallel Testing

Production Install – Establishing the new EDW into the production environment

Decommission – Decommission the existing DB2 system and infrastructure

1.3.3 Establishing a production environment This involves establishing the processes and procedures that will be used to maintain the new Teradata environment and assisting the end users in the migration. Major areas of focus include:

Establish operational processes and procedures

User Change Management – Defines and Builds the infrastructure that will be used to communicate to the end user of the EDW. This activity may also include establishing a help desk

1.4 DB2 Database Migration Considerations

1.4.1 Physical Data Base Conversion

1.4.1.1 Table ConversionThe first thing to do when converting the physical database is to collect an inventory of DB2 DDL and Indexes. There are three basic methods of converting DB2 DDL to Teradata DDL.

Page 4: Db2 migration -_tips,_tricks,_and_pitfalls

1.4.1.2 Using a Script to Convert DB2 DDLThis is the best approach if there are a large number of tables to convert. There is a set of Perl scripts that have been developed as part of the DB2 to Teradata Migration tool set that are discussed later is this offer guide. These scripts will read the DB2 catalog and create DDL. Please refer to that section for more detail on this option. Note that in some cases these scripts may need to be modified slightly to work for your particular client. If this is the case, and if you do not have anyone on your team who is comfortable modifying Perl scripts you may want to choose one of the other options listed below for migrating tables.

1.4.1.3 Table Conversion Using ErwinTo use this method the steps are as follows:

Reverse Engineer the DB2 database(s) into an Erwin Model. Then in Erwin:

Make the target database Teradata. This will change the data types for you Go to the Tasks menu and forward engineer to generate the schema. Do a Preview or Report, outline all, and Ctl-C (copy) to copy the text Open up a text editor, Paste Remove objects that are not to be ported Remove\change primary index names If there are many tables, consider doing the above using a

scripting language Do object renames as necessary Insert a logon statement before CREATE TABLE statement Using the text file and Bteq, create the objects

When using this method you may want to use a scripting language for certain modification tasks, especially when dealing with a large number of tables. Below is an example of a Perl script that removes index names (assuming that is what you wanted to do) from the DDL.

#!/usr/bin/perl

$FINPUT = "./before.txt" ; $FOUTPUT = "./after.txt" ;

$FINPUT = $ARGV[0] if ($ARGV[0]) ; $FOUTPUT = $ARGV[1] if ($ARGV[1]) ;

open(FILE, $FINPUT) || do { warn "Can't open $FINPUT: $!\n"; return; };

open(FOUT, ">$FOUTPUT") || do { warn "Can't create $FOUT: $!\n"; return; };

while (<FILE>) {

@Fld = split( /\s+/, $_, 100);

if (($Fld[1] eq "PRIMARY" ) && ($Fld[2] eq "INDEX" ))

{ print FOUT "$Fld[1] $Fld[2] Fld[4]\n"; }

elsif (($Fld[1] eq "UNIQUE" ) && ($Fld[2] eq "PRIMARY" ))

{print FOUT "$Fld[1] $Fld[2] $Fld[3] $Fld[5]\n";}

elsif (($Fld[0] eq "CREATE" ) && ($Fld[1] eq "INDEX" ))

{print FOUT "$Fld[0] $Fld[1] $Fld[3]\n";}

Page 5: Db2 migration -_tips,_tricks,_and_pitfalls

CREATE TABLE accts ( CUST_ID INTEGER NOT NULL FORMAT '-(10)9', ACCT_TYPE CHAR(2) NOT NULL, ACCT_NBR CHAR(16) NOT NULL, ACCT_START_DATE DATE NOT NULL FORMAT 'YYYYMMDD', ACCT_END_DATE DATE FORMAT 'YYYYMMDD') PRIMARY INDEX XIE1accts ( CUST_ID, ACCT_TYPE );

CREATE TABLE accts ( CUST_ID INTEGER NOT NULL FORMAT '-(10)9', ACCT_TYPE CHAR(2) NOT NULL, ACCT_NBR CHAR(16) NOT NULL, ACCT_START_DATE DATE NOT NULL FORMAT 'YYYYMMDD', ACCT_END_DATE DATE FORMAT 'YYYYMMDD') PRIMARY INDEX ( CUST_ID, ACCT_TYPE );

Before After

elsif (($Fld[0] eq "CREATE" ) && ($Fld[1] eq "UNIQUE" ))

{print FOUT "$Fld[0] $Fld[1] $Fld[2] $Fld[4]\n";}

elsif ($Fld[1] eq "INDEX" ) {print FOUT "$Fld[1] $Fld[3]\n";}

else {print FOUT $_; }

}

close FILE; close FOUT;

Here is the before and after picture of the DDL that was modified by this scripts

1.4.1.4 DB2LOOK CommandThe third method that can be employed is using the DB2LOOK command. This command is similar to the SHOWTABLE command in Teradata. IBM uses this command in order to move objects from a production database to a test database but it can be used to extract DDL as well. One caveat is that it is not always installed in all environments so you should check with the DB2 administrator to determine if it has been on your system. If not it is a relatively simple process to get it installed (similar to running a dip script). Once available you can run this command and loop through getting DDL for each table and placing it in a text file where it can be modified. Of course you can either do this manually or you can use the scripting language of your choice to make the modifications, as mentioned above.

1.4.1.5 Index ConversionFor the indexes, you should focus on the primary index selection first. Secondary indexes in DB2 are not necessarily comparable to Teradata Secondary Indexes so it is usually best to select a primary index and the select secondary indexes based solely on Teradata criteria.

Page 6: Db2 migration -_tips,_tricks,_and_pitfalls

1.4.1.6 View ConversionErwin can be used to reverse engineer the views out of db2 as long as they are fairly simple and straight forward. It does not, however, excel at converting more complex views. It does not do well with the following:

Converting views that contain multiple joins Converting all of the DB2 functions to proper Teradata functions Converting view that contain derived tables

For these more complex views it would be best to build a conversion tool of your own using your favorite scripting language or to do them manually. Even when using a scripting language you may need to make some manual adjustments to some views in order to get them to perform adequately.

1.4.1.7 UDFs and UDTsThese are being encountered more and more in conversions. They will require special attention because while Teradata supports UDFs the languages and features are different. Converting may not always be the answer. Each of these should be taken on a case by case basis. Consider the following alternatives.

Port UDFs to Macros or Stored Procedures Port UDTs to data containers in Stored Procedures Consider if UDF/UDT logic can be replaced by a Teradata function thereby eliminating

the need for it.

1.4.1.8 Stored ProceduresStored procedures have been out on IBM for a few years now. On IBM platforms they can be written in Java, C, or any other platform supported language, as well as in SPL (Stored Procedure Language). Terdata internal stored procedures must be written in ANSI Standard SPL. External Stored Procedures may be written in C.

This is one of those conversion categories that you must grind out manually. One tip is to look for alternative ways of accomplishing the same task. For example many Stored Procedures make use of sequential looping. There may be a way to accomplish the same thing in a different way using a parallel process in Teradata.

1.5 Historic Data Migration

1.5.1 Historic Data Migration ApproachesThere are three basic approaches to migrating history. They are outlined below.

1.5.1.1 Unload DB2 / Load Teradata via ScriptThere are a set of Perl scripts that are a part of the DB2 to Teradata Migration toolkit that will read the DB2 catalog and produce DB2 Unloads and corresponding FASTLOAD or MULTILOAD scripts. These scripts run on UNIX and at this time will operate only against

Page 7: Db2 migration -_tips,_tricks,_and_pitfalls

DB2 EEE or other UNIX flavor of DB2. There is also a tool that generates MULTILOAD scripts for the OS390 version of DB2. For more information on this option, refer to the section on tools with in this guide.

1.5.1.2 Use an Existing ETL Tool.If there is an ETL tool such as Informatica, Ab Initio, etc. available, this may provide a viable method for moving history from DB2 to Teradata.

1.5.1.3 Manual Unload / LoadThis is the least desirable option as it is the most labor intensive. It may, however be necessary because the options above are not suitable or because the data must be physically moved from one site to another.

If this is the method that is chosen, the first step is to create the exports. There are a number of methods to do this. You should use the one that is most familiar to you and keep the following in mind:

Sorting by any column is not desirable Export numeric data types as displayable, sign preceding Export dates as “YYYYMMDD” Try to export fixed length rows, adding blanks to fill out char strings (OS390) If VARCHAR exports are necessary, you need a 2 byte binary length preceding the

VARCHAR All data must be in the host format of the EXPORT requestor, if OS390 then EBCDIC

(except for 2 byte lengths, which must be gig endian, little endian)

When creating ASCII flat files from a DB2 EEE environment the records can be fixed or delimited either can be handled by Teradata. Here are some guidelines.

If Delimited

Choose a delimiter – Standard Pip “|” or other Do not use a cancellation delimiter as they are different in DB2 than in Teradata. A delimiter followed by a delimiter (e.g. ||) indicates a null value There should be no delimiter at the end of a line

If Fixed

Determine the null indicator that you will use

For Both

Date - format ‘YYYY-mm-dd’ Time w/timezone – format HH:MI:SS.S(F)Z Timestamp w/timezone – format YYYY-MM-DDBHH:MI:SS.S(F)Z

Once you have created the extracts you must create FASTLOAD scripts. The following Perl script will take text version of the DDL from Erwin and create a Fastload script.

#!/usr/bin/perl

$FINPUT = "./before.txt" ; $FOUTPUT = "./after.txt" ;

Page 8: Db2 migration -_tips,_tricks,_and_pitfalls

$FINPUT = $ARGV[0] if ($ARGV[0]) ; $FOUTPUT = $ARGV[1] if ($ARGV[1]) ;

open(FILE, $FINPUT) || do { warn "Can't open $FINPUT: $!\n"; return; };

open(FOUT, ">$FOUTPUT") || do { warn "Can't create $FOUT: $!\n"; return; };

while (<FILE>) {

@Fld = split( /\s+/, $_, 100);

if (($Fld[0] eq "CREATE" ) && ($Fld[1] eq "TABLE" )) {

print FOUT "sessions 2;\nerrlimit 25;\nlogon tdpid/username,password;\n";

print FOUT "set record unformatted;\ndefine\n";

$TNAME = $Fld[2]; @NAMES = (); $I=0;

while (<FILE>) {

@Fld = split( /\s+/, $_, 100);

if ($Fld[0] eq ")") {last;}

$NAMES[$I] = $Fld[1];

if ($I++ > 0) {print FOUT ",";}

if ($Fld[2] eq "INTEGER") {print FOUT "$Fld[1] ($Fld[2])\n";}

elsif ($Fld[2] eq "SMALLINT") {print FOUT "$Fld[1] ($Fld[2])\n";}

elsif ($Fld[2] eq "BYTEINT") {print FOUT "$Fld[1] ($Fld[2])\n";}

elsif ($Fld[2] eq "DATE") {print FOUT "$Fld[1] ($Fld[2] FORMAT 'YYYYMMDD')\n";}

elsif (substr($Fld[2],0,4) eq "CHAR") {print FOUT "$Fld[1] ($Fld[2])\n";}

# and then other datatypes

}

print FOUT "file=insert.input;\n";

print FOUT "begin loading $TNAME errorfiles $TNAME.err1, $TNAME.err2;\n";

print FOUT "INSERT INTO $TNAME (\n";

for ($j=0; $j<$I; $j++) {

.................................................................................................if ($j > 0) {print FOUT ",";}

print FOUT ":$NAMES[$j]\n";

}

print FOUT ");\nend loading;\nlogoff;\n\n\n\n\n";

}

}

close FILE; close FOUT;

Below is a figure that shows the text file input of the create table statement on the left and the resulting FASTLOAD script on the right.

Page 9: Db2 migration -_tips,_tricks,_and_pitfalls

CREATE TABLE accts ( CUST_ID INTEGER NOT NULL FORMAT '-(10)9', ACCT_TYPE CHAR(2) NOT NULL, ACCT_NBR CHAR(16) NOT NULL, ACCT_STRT_DATE DATE NOT NULL FORMAT 'YYYYMMDD', ACCT_END_DATE DATE FORMAT 'YYYYMMDD') PRIMARY INDEX XIE1accts ( CUST_ID, ACCT_TYPE );

sessions 2;errlimit 25;logon tdpid/username,password;set record unformatted;defineCUST_ID (INTEGER),ACCT_TYPE (CHAR(2)),ACCT_NBR (CHAR(16)),ACCT_STRT_DATE (DATE FORMAT 'YYYYMMDD'),ACCT_END_DATE (DATE FORMAT 'YYYYMMDD')file=insert.input;begin loading accts errorfiles accts.err1, accts.err2;INSERT INTO accts (:CUST_ID,:ACCT_TYPE,:ACCT_NBR,:ACCT_START_DATE,:ACCT_END_DATE);end loading;logoff;

Input Output

1.5.2 Historic Data Migration SynchronizationThere are two kinds of synchronization that you will encounter.

Batch load synchronization This type of synchronization assumes that the load process adds rows to a table on

each cycle and that the rows of data are stamped with a date or id, the migration will move the tables from a point in time.

Synchronization of updated tables These are tables that are not loaded by adding rows, but rather by updating current

rows or by doing a full refresh. With this type of table, the synchronization can approached in two different ways. The best approach is to move the tables at the very end of the migration process, after the last run on the legacy system and before the first run of the new system. In cases where this is not possible it might be necessary to build a custom solution to achieve synchronization. A custom solution might be to move the tables over and then on the legacy system perform a delta process to send a synchronization file.

Synchronization fits into the general project timeline right before you are ready to begin the production version of the incremental loads.

Basically your synchronization strategy is attempting to take any kind of gap there may be between the DB2 system and the Teradata system and fill it. For example you may have loaded all historical data as of the end of last month (for all tables). Three weeks have passed

Page 10: Db2 migration -_tips,_tricks,_and_pitfalls

and you need to load the three weeks of data that have been added/updated in the DB2 System to the Teradata system.

The basic steps for synchronization are as follows:

1.5.2.1 Step 1Record the period id(s) for each table that you are loading. This is anything that can identify a block of data that you are loading (e.g. timestamp, batch id, etc.)

1.5.2.2 Step 2Migrate the data. This is the initial history migration. You may choose to do this in batches of periods (e.g. 1 month of data at a time), to make each load smaller and more manageable.

1.5.2.3 Step 3From recorded period(s) on, load additional periods into temporary tables. It may be important to do this on a regular basis if the time from the initial history load to the time of production turnover is protracted for some reason.

1.5.2.4 Step 4Validate the data migrated to the temporary tables.

1.5.2.5 Step 5Insert data from temporary tables into base production tables and delete temporaries. This should be completed right before cutover to production.

It should be noted that for tables that require updates or delta loads you will have to develop your own custom solution to ensure that updates are applied properly in the synchronization process.

For tables that are totally refreshed each time they are loaded, you should plan to just do those at the end of the migration process using a flat file as input.

1.5.3 Process Control and TrackingFor large migrations with hundreds or thousands of tables keeping track of which ones have been loaded and with which periods is a complex task. As part of the DB2 To Teradata Migration Tools package a tracking database and a set of macros for both updating the database and reporting on the progress have been developed. The reporting macros can be executed using a tool such as an excel spreadsheet. The maintenance macros can be executed either manually or within a script depending on process that you are using for loading and verifying data. This automated approach to tracking is detailed later in the section of this document that is dedicated to conversion tools.

1.5.4 TestingTesting will differ from site to site based on requirements however here are a few guidelines for testing a history migration that can be considered as components of your overall test plan.

Page 11: Db2 migration -_tips,_tricks,_and_pitfalls

A field by field compare of all of the history may be cumbersome but could be performed on random samples of rows from each table to ensure that the data appears to have loaded successfully

Perform a check of the catalog to ensure that all physical structures have been created Accessibility check – ensure that the roles, profiles, permissions, rights, etc allow for the

appropriate update/access of the data. A field by field compare of incremental loads may be appropriate depending on the size

of the loads. Again if they are large you could reduce the compare to a sample of the rows.

1.5.5 User TablesUser tables typically fall outside of normal maintenance activities. In most cases you will have to do some digging to find out when they are loaded, who loads, them how they get loaded, etc. These tables are typically small in size but can be great in number so you will need an approach for migrating them. There are two basic ways to deal with them.

Use the self service approach. In other words give the users some sort of GUI tool (preferably freeware or trial version software of some kind that allows them to migrate their tables on their own schedule (of course with an appropriate mandatory cut over date).An advantage of this option is that users may decide that they don’t need certain tables thereby reducing the total number of tables migrated. Note that this option is probably only good for smaller tables

Move all the tables using one of the methods described earlier for migrating tables. Note that this will require some level of coordination with the general user population as to when it will be performed and that after that given date that changes made on the DB2 system will not be moved forward. This most likely will have to be a general rather than a specific communication as it in most cases will be difficult to discover who owns which tables.

1.5.6 Archived Data MigrationOne area that is often overlooked in migrations and in scoping migrations is the effort required to migrate archived data from the DB2 environment. Archived data is typically required for accounting, regulatory, legal, etc. reasons. These reasons mandate that the data may need to be restored at some point. You will need to understand the both the extent of the archives that need to be migrated as well as the intended service level that is expected for a restore in order to make a decision as to how to handle this type of data. It should be noted that DB2 Archives are stored in a proprietary format so they cannot be directly restored to Teradata.

There are three general options from which to choose when dealing with archived data. They are as follows:

Restore Extract Load Archive on Teradata Clean up The steps for this option are to restore the archives on DB2, Extract the data From

DB2, load the data to Teradata, archive the data on Teradata, and, finally delete the

Page 12: Db2 migration -_tips,_tricks,_and_pitfalls

data from Teradata. This is the most involved in terms of effort to get the data moved to Teradata and then archive the data in that format. This approach will allow for the quickest turnaround time for a restore process.

Restore Extract The steps for this option are to restore the archives on DB2, then unload them and the

associated DDL to text files. When a restore is needed the files would need to be loaded table by table (most likely using FASTLOAD) The number of tables needed to restore will directly affect the amount of time needed.

Do nothing now If for some reason there will continue to be a DB2 system on site (e.g. for

applications that are not being migrated), you may chose to not migrate the history. The restore system could simply be the remaining DB2 system. This of course would have to be acceptable to the client and you would have to carefully explore the steps required to restore and retrieve the data. At a minimum these would need to be documented as part of your engagement. This would also have to meet any SLAs for restore that the customer had in place.

1.6 DB2 Application Migration Considerations

1.6.1 Data Type Considerations

1.6.1.1 Decimal 19 and AbovePrior to Teradata 12, Decimal 19 and above is not supported by Teradata but is by DB2. If dealing with a Teradata system that has not been upgraded to Teradata 12, the following may apply. When considering conversion options one should determine the purpose for which the field is used. In this example we will use currency. Decimal (19,4) will store a value that represents over 999 trillion dollars. In most scenarios, there should be no need to store a value greater than 99 Trillion dollars which can be accomplished in Teradata using a Decimal (18,4). You should determine the max value on each column within the DB2 system and see if it will fit into a Decimal 18 on Teradata. It would also be a good idea to check with subject matter experts at the client site to determine if there is a reason to believe that a value of greater than 100 million would ever need to be stored in one of these columns. If not than the preferred method would be to convert this columns to a smaller decimal field. If the evidence shows that values greater than 100 million may be required than you must use either a Float or Char (as required) data type to store the values.

1.6.1.2 VARCHARVARCHARs are used heavily in DB2, especially in DB2 EEE. This is probably a result of DB2 not having column level compression until release 8. In Teradata, there is a 2 byte overhead that is used for VARCHARs. Depending on the situation it may be more advantageous to use CHAR data types with Multi-Value Compression (which cannot be used on a VARCHAR in Teradata) as this may save more space than straight conversion from to VARCHARs.

Page 13: Db2 migration -_tips,_tricks,_and_pitfalls

1.6.1.3 SHORT FLOATThis is a data type that was used in DB2 because it was believed that it would save space. These should be converted to the FLOAT data type within Teradata

1.6.1.4 BOOLEANThis data type is not that common but could be used by procedural logic in downstream processing. When converting this data type, use the BYTEINT data type in Teradata.

1.6.1.5 The Dumb Tilde (character delimiter)DB2 exports and imports will use this in their processing. The default is a quote character. In most cases you will need to strip those out when you find them. There are a few methods that are can be used.

Use the SED command from Unix Use the Substring functionality in SQL Use an ETL Tool to strip them out

1.6.1.6 BIGINTFor Releases of Teradata that do not support 64 bit integer data types (those prior to TD12), use the DECIMAL data type to convert DB2 BIGINT data types. In some cases you may find that 3rd party tools have not yet incorporated support for Teradata’s 64 bit integer data type. In that case you would also use a Decimal field in its place.

1.6.2 Industry Standard Interfaces / ProtocolsODBC, OLEDB, ADO, RDO, and JDBC are all supported by Teradata. You may run into something called SQLJ (which is for Java pre-processing), this is not supported by Teradata unless the C++ pre-processor can be used.

It is interesting to note that the Native DB2 CLI can be mapped for the most part directly to ODBC 1 for 1 as the two are very similar. This makes ODBC a good choice when converting DB2 CLI rather than converting to native Teradata CLI

1.6.3 GUI ToolsGUI tools can be broken down into few different categories. The first category is tools that are procedural in nature. This category includes tools like PowerBuilder, MS Access, Visual Basic, Visual C, Excel, etc. These tools typically use ODBC JDBC interfaces. When doing your initial assessment these should be considered as being moderately complex.

SQL that is generated by these tools shouldn’t require conversion as it should be generating ANSI compliant SQL. Note that an SQL that part of one of these applications and is not directly generated by the tool may contain DB2 specific extensions and would require conversion in order work with Teradata.

The second category of applications is SQL Submitters. An example of this type of tool in the Teradata world is SQL Assistant (Queryman). In the DB2 world you may run into QMF or some home grown query tool (e.g. John’s Query Tool).

If you want or need to run QMF against Teradata there is TSAPI which can be used to run QMF against Teradata as DB2 but because the SQL may contain DB2 specific, syntax you will probably end up converting the SQL anyway. Therefore it is probably best to convert all

Page 14: Db2 migration -_tips,_tricks,_and_pitfalls

SQL from any DB2 SQL Submitter and migrate it to SQL Assistant. Then you should perform testing and parallel testing to assure the same results are being generated.

The third category of GUI tools is GUI Application Writers. This category includes tools such as Cognos, Microstrategy, Hummingbird, Crystal Reports, etc. These tools build some kind of metadata layer or pseudo dictionary within the tool. They use these to understand the data types which were in DB2, how to access tables, and what join criteria to use. These tools typically have both ODBC support and CLI support for different database vendors. Because most implementations make use of the ODBC interface these are amount the easiest applications to convert. Typically you rebuild the metadata layer to point to Teradata structures and all of the queries should work. The proof is in the testing and environments with lots of standard query’s / reports and lots of users will require diligent management of the testing process.

1.6.4 GUI Three/N Tier ArchitecturesGUI Three/N Tier Architectures include tools like Cognos build cubes on a middle tier server and then records are accessed from these middle tier servers. These servers can be either small windows servers or mid range to large Unix servers. The migration of these cubes to Teradata is relatively easy except if they are using some sort of “pass through” functionality. With this type of functionality these tools send requests to the database to drill into the underlying data that was used to build the cube. This SQL may need to be converted for two reasons. The first is if it uses any DB2 specific extensions (as previously mentioned in this document). The second is to ensure that it is optimized for Teradata. In some, DB2 implementations the SQL may be optimized to take advantage of DB2 indexing. Because the indexing scheme will be different on Teradata, this SQL may need to be re-optimized for Teradata.

1.6.5 Web Enabled ApplicationsWeb enabled applications are those that allow for access to the application through a browser. The idea behind these applications was to relieve the desktop of the need to have client software resident on it. These applications require multiple tiers and web services to talk with the DBMS and usually use Java technology and JDBC. One thing to watch out for when migrating these applications is performance. In some cases large buffer pools may be used in conjunction with a succession of small result sets. This may not work well in a Teradata environment and may need to be re-engineered to make better use of the database engine. Other than that these types of applications make use of standard interfaces and are relatively easy to migrate. Again rigorous testing is the key to success.

1.6.6 Custom ApplicationsCustom Applications are by far the most difficult to migrate to Teradata. In the DB2 world you will find many applications that are written in Cobol, C, PL/1, etc. Converting these programs to work with Teradata may require and entire re-write. Often this is difficult because these applications were written long ago and the documentation is poor or does not exist. In some cases you will have to convert these applications which will involve re-writing (in total or in part) them and/or at least qualifying and recompiling them. Before re-writing 1000s of lines of code consider the following alternatives.

Page 15: Db2 migration -_tips,_tricks,_and_pitfalls

Is the functionality required in the new warehouse environment or can the application be retired?

Can the functionality be replicated using a standard piece of software or some standard functionality within a tool? For example you may encounter a complex scheduling application that can be replaced with an off the shelf solution.

Can the program functionality be separated from the database access by creating a separate database interface (still requires changes to the program but may avoid a total rewrite which is helpful in poor documentation scenarios)?

.

1.6.7 Downstream Data Feeds and ExportsIt is very common for databases to feed downstream, applications, data marts, ODSs, etc. These are performed in a couple of different ways. First they may be built in a Cobol, C, C++, etc. Interface. Second they may be exported from DB2 using a command line processor. Most of these should be converted to FAST EXPORT. And most will be relatively easy to convert.

Some things to consider when converting command line exports are as follows:

When using UNIX you will need a disk landing area or pipe as you cannot go directly to tape.

You will need to pay attention to character, column, and file delimiters used in DB2. As of TTU 8.1 there is no delimiter function so if delimiters are required in the output you will have add them in yourself. This can be done with an OUTMOD if you so desire.

You will need to understand whether the character set is ASCII or EBCIDIC