data migration in a live gxp environment - labware migration in a... · data migration in a live...
TRANSCRIPT
Data Migration in a live GXP genvironmentK Sti kKaren StickneyCSDM Information System AnalystLabWare CEC 2012
19 December 201319 December 2013
Legacy System g y y
A validated, custom software application for tracking and managing GLP and GCP samples and associated data.
System comprised of approximately 142 studies including both active and closed studies. Each study behaves as a self contained database in its own right.
High variability of data between studies Examples: Documented Visit formatp
> Visit and Visit time in one field (visit: Day 1 Predose, visit time: blank)
> Visit and Visit time in separate fields (visit: DAY 1, visit time: Predose)
3
Predose)
Migration Methodology Overviewg gy
Initial mapping and migration tool development
Data migration process
Move imported data into the live Production systemp y
4
Flow Diagramg
Data ImportingData Importing
Data Verification and Validation
Initial Mapping and Migration Tool Development
Data ManipulationValidation
Data Importing
Data Verification and Validation
5
Initial Field Mapping and Migration Tool Developmentp
The initial mapping of all fields between the Legacy t d th l lid t d R h LIMS (RLIMS)system and the newly validated Research LIMS (RLIMS)
system.
Example of mapping for Clinical ProtocolsLW Column Type SDMS Mapping Comments
LOGIN_DATE Date/Time SAMPLE.CreatedStamp parsed out
Example of mapping for Clinical Protocols
LOGIN_BY str 20 SAMPLE.CreatedStamp parsed out
SAMPLED DATE Date/Time
SDMS.SAMPLE_DOCUMENT.CollectionDate + CollectionTimeSAMPLED_DATE Date/Time CollectionTime
SAMPLE_TYPE str 20 SAMPLE_DOC.KitType MappedPROJECT str 20
6
CONDITION str 20 SDMS.SAMPLE.Condition mapped
Initial Mapping and Migration Tool Development (cont)p ( )
This process took several months and was re-visited multiple times throughout the migration process. This was performed using a representative sampling across all studies. Risk
t l t d f h fi ldassessment completed for each field.
Custom migration tool configured and tested
Data Export/Mappingp pp g
Data Cleaning performed on the studies in the Legacy g p g ysystem for how data was reported within a study and across all studies. Duplicate Sample ID’s samples with more than one location Duplicate Sample ID s, samples with more than one location,
blank fields (Sample ID, Condition)
Remove commas from any text field (such as comment and discrepancy fields)discrepancy fields)
The data was exported out of the Legacy system utilizing the systems data export tool. Templates created tothe systems data export tool. Templates created to ensure file format consistency. File are then saved as a .CSV file
8
Data Export/Mapping (cont) p pp g ( )
Study specific mapping files were also created for sample type, assay, visit and assay status.
E l f i it i fil
Sample_doc.visit DAY 0 PRE DOSE 1DAY 28-35 POST DOSE 2 DAY 28-42 PRE DOSE 2 UNSCHEDULED 1 UNSCHEDULED 2
Example of visit mapping file
Orders.Clin_trl_visit DAY 0 DAY 28-35 DAY 28-42 UNSCHEDULED 1 UNSCHEDULED 2
Orders.Clin_trl_event PRE DOSE 1 POST DOSE 2 PRE DOSE 2 N/A N/A
Flow Diagramg
Data ImportingData Importing
Data Verification and Validation
Initial Mapping and Migration Tool Development
Data ManipulationValidation
Data Importing
Data Verification and Validation
10
Data Migration Toolg
This is a set of custom written Perl scripts to process export files and create files for upload to the RLIMS database, and custom written SQL scripts to verify the d t tl l d d i t th RLIMS d t b ftdata was correctly loaded into the RLIMS database after each migration.
A Custom Data Migration Tool was required to perform A Custom Data Migration Tool was required to perform processing and conversion because of significant differences between the two systems.
11
Data Manipulationp
Within the PERL scripts (migration tool) Data Manipulation is required to meet the convention of the
RLIMS database. E.g. changing date format 20070503 to 5/3/07g g g
By evaluation of the Mapping files Example: Example: Patient Gender
> In Legacy system this field could have a M, F, Female, Male, FEMALE, MALE
> In Research LIMS system this data was mapped to either Male, Female, M – Male, F - Female
12
Flow Diagramg
Data ImportingData Importing
Data Verification and Validation
Initial Mapping and Migration Tool Development
Data ManipulationValidation
Data Importing
Data Verification and Validation
13
Data import to Staging Environmentp g g
This was done to minimize risk to the live Production SystemSystem.
Allows for removal and re-importation if needed without impacting daily business operationsimpacting daily business operations. Increments table – needed to set the increments in the
Production staging system so that when data moved to the actual Production system there would not be a conflictactual Production system, there would not be a conflict. > Example: Sample Number was set to 900,000,000
After data verification (validation) data moved into the After data verification (validation), data moved into the live Production system.
14
Validation Methodologygy
Data Migration Validation Black Box Approach for Custom Migration Tool
> Significant Time & Cost Savings> Significant Time & Cost Savings> Validation based on expected outcome of “known data sets”> Combination of Simulated & Real Data Sets
Oracle’s External Tables used for Post Migration Verification Oracle s External Tables used for Post Migration Verification> Original Data Source lacked Native Database> External Tables allowed comparison of “apples to apples”> SQL “Minus” Queries compared External Tables to LIMS Tables> SQL Minus Queries compared External Tables to LIMS Tables> Automated> 100% Source-to-Destination
15