data conversions - scgmisdata conversions the good, bad and ugly. ... tools for data conversion...
TRANSCRIPT
DATA
CONVERSIONS
The Good, Bad
and UglyCarolyn Yon, PMI-ACP
Richland County Government
DATA
CONVERSIONS
The Good, Bad
and Ugly
Lazlo – Real Genius (1985)
Practices for
Successful Data
Conversions
Complex
and
Difficult
Difficult
to
Estimate • Time
• Budget
Expect
Perfection
90-95%
Other Considerations
• Delivery Methods
• XP, Lean
• Scrumban
• Team Composition
• IT
• Users/Stakeholders
• Issue Tracking
• Information Radiation
Data to Convert
Reference Tables System/Setup/Configuration
Entity / Child Data to LoadNeeds little/no pre-processing before loading
Entity / Child Data to ConvertNeeds re-processing/validation before loading
Data Mapping Workbook
Target Table
Target Field
Source Table
Source Field
Notes, Formulas
Tools for Data Conversion
Reference Tables SQL Server Integration Services (SSIS)
Entity / Child Data to Load SQL Server Integration Services (SSIS)
Entity / Child Data to ConvertMicrosoft Visual Studio .Net
UniSys(Legacy)
Interim SQL Server
Conversion DB
Assess SQL
InterimConversion
SQL
New CAMASQL
Flat Files(Weekly)
SSIS(Weekly)
Stored Procedures
(On Demand)
Internet Apps
Conversion or SSIS
Tools for Issue Tracking
Tools for Issue Tracking
Status
• Central Body of Knowledge• Fast Import wherever possible• Enforced Data Integrity• Increased Application Integrity• Facilitate product growth
Data Documentation & Tools
• Manual documentation• Time to learn, architect SSIS
Data Documentation & Tools
• Time Consuming Administrivia• Manual SSIS Runs
Data Documentation & Tools
SQL Server Agent
SSIS Packages
Entity Pkgs
Parcel DEV Job
SQL Server Agent
Delivery Time
Environment
Scripts
SQL Server Agent
Delivery Time
Delivery Cycles
Delivery Cycles
Conversion
Effort
Demo to
Users
Evaluate
Feedback
Entity & Issue
Analysis
Retro &
Planning
• Involved user’s in selecting data subset;
• Early detection on accuracy of data;
• Early connection with users;
• Early considerations of business process changes;
• Identification of desired future enhancements;
• Early scheduling helped to ensure user’s presence;
• The further along we got, users began to see some of the benefits of the new system.
Delivering
• Inconsistent business process knowledge;
• Progress of user acceptance testing;
• Live legacy system is a moving target;
• User’s lack of understanding of the new system;
• User’s lack of experience with conversions;
• Dependencies among the entities.
Delivering
• Inconsistent data integrity;
• Users reluctant buy-in to new system;
• Users reluctant to consider business process changes;
• Changes in conversion inadvertently broke something else;
• Integration testing laborious;
• Features not present/ not identified back during development.
Delivering
SUMMARY
Data Conversion
is only going to be as easy
as the quality of the data.
SUMMARY
People Collaboration
can be just as complex and
difficult.
