optimizing data extracts from oracle clinical sas views
DESCRIPTION
This presentation delivered at OCUG Annual Conference 2009 at New Orleans is on a new way developed to optimize SAS dataset extracts in Batch Data Load (BDL) format from Oracle Clinical SAS views.TRANSCRIPT
© 2009 Confidential and proprietary property of RadPharm, all rights reserved.
OCUG Annual Conference – October 2009
Optimizing Data Extracts from OC SAS
Views
Prashant Rajopadhye, PMPRadPharm, Inc.
OncologyTrials
To Date
Outline
• About RadPharm, Inc.• Problem Overview
– BDL File Formats– Old Extraction Method
• Solution– Implementation Details– Process Flow
• Key Success Factor• Contact
© 2009 Confidential and proprietary property of RadPharm, all rights reserved.
• Leading full service imaging core lab• Managing all facets of the imaging segment of
global clinical trials, Phases I-IV.– Computed Tomography (CT)– Magnetic Resonance Imaging (MRI)– Positron Emission Tomography (PET) and PET/CT– Single Photon Emission Computed Tomography (SPECT)– Ultrasound– X-ray
About RadPharm, Inc.
© 2009 Confidential and proprietary property of RadPharm, all rights reserved.
• 15 years experience in– oncology– cardiovascular– musculoskeletal– diagnostic contrast imaging agents– Medical device studies
About RadPharm, Inc. (cont.)
© 2009 Confidential and proprietary property of RadPharm, all rights reserved.
• 300 Clinical Trials• Across 60 countries
About RadPharm, Inc. (cont.)
© 2009 Confidential and proprietary property of RadPharm, all rights reserved.
Problem
• Phase 3 Study data extraction– 15 Files
• 5 BDL Files of ‘A’ type• 5 BDL Files of ‘B’ type• 5 ASCII Preview Files for the Clinical Data Coordinator
– Total Extraction Time about 12 hours– One dataset alone took over 9 hours– Would cause database performance issues– Needed to stop PSUB during extraction
© 2009 Confidential and proprietary property of RadPharm, all rights reserved.
BDL A File Format
P A T I E N T _ I D | V I S I T | S U B E V E N T | V I S I T _ D A T E | D C I | D C M | D C M _ S U B S E T | Q U E S T I O N _ G R O U P |
Q U E S T I O N | Q U E S T I O N _ O C C U R R E N C E _ S E Q U E N C E _ N U M B E R | R E P E A T _ S E Q U E N C E _ N U M B E R | V A L U E | C O M M E N T
BDL B File Format
I N V E S T I G A T O R | S I T E | P A T I E N T _ I D | V I S I T | S U B E V E N T
| V I S I T _ D A T E | D C I | D C M | D C M _ S U B S E T | Q U E S T I O N _ G R O U P | Q U E S T I O N | Q U E S T I O N _ O C C U R R E N C E _ S E Q U E N C E _ N U M B E R | R E P E A T _ S E Q U E N C E _ N U M B E R | V A L U E |
C O M M E N T
Preview File Format
P A T I E N T _ I D | V I S I T | S U B E V E N T | V I S I T _ D A TE | D C I | D C M | D C M _ S U B S E T | Q U E S T I O N _ G R O U P | Q U E S T I O N _ O C C U R R E N C E _ S E Q U E N C E _ N U M B E R | R E P E A T _ S E Q U E N C E _ N U M B E R |
R E A D E R _ I D | R E A D _ T Y P E _ S P | S I T E | R E S P O N S E _ N Y | E XA M _ S P | A S S E S S M E N T _ D T | N E W _ N Y | S A M P L E _ N Y | G R A D E _ C D | P E R F O R M E D _ N Y | CR I T E R I A _ M E T _ N Y | C O M M E N T S _ 1 | C O M M E N T S _ 2 |C O M M E N T S _ 3 | C O M M E N T S _ 4 | C O M M E N T S _ 5 | L E S I O N _ I D |
S T U DY | I N V S I T E | P T | V I S I T | S U B E V E | C P E V E N T | Q U A L I F Y V
© 2009 Confidential and proprietary property of RadPharm, all rights reserved.
Old way to extract FRACSP dataset
• Run the BDL A file, BDL B file and Preview file SQL extraction scripts directly against the OC SAS views one at a time
• Resulting datasets spooled to files
© 2009 Confidential and proprietary property of RadPharm, all rights reserved.
Solution
• SELECT data from OC SAS views• INSERT into a set a three indexed tables
– SRE_EXTRACT_TPHD– SRE_EXTRACT_COMMENTS– SRE_EXTRACT_DATA_WORK
• UPDATE table SRE_EXTRACT_DATA_WORK with time point header and reader comments data
• WRITE files to OS using following mapping tables and final data – SRE_BDL_QUESTION_MAP– SRE_A_B_FILE_MAP– SRE_EXTRACT_DATA_WORK
New way: Package PKG_SRE_FRACSP
CREATE OR REPLACE PACKAGE PKG_SRE_FRACSP as
PROCEDURE fracsp_fill (p_study_name varchar2 ,p_view_type varchar2 DEFAULT 'STABLE',p_schema varchar2 DEFAULT 'RADPHARM',p_a_b_file_map_table varchar2 DEFAULT ‘SRE_A_B_FILE_MAP');
PROCEDURE fracsp_write_file(p_study_name varchar2 ,p_view_type varchar2 DEFAULT 'STABLE',p_file_type varchar2 DEFAULT 'all',p_schema varchar2 DEFAULT 'RADPHARM',p_map_table varchar2 DEFAULT 'SRE_BDL_QUESTION_MAP');
END PKG_SRE_FRACSP;
TABLESRE_EXTRACT_TPHD
VARCHAR2 (200 Byte)YDTP_READ_TYPE_SP13
VARCHAR2 (200 Byte)YDTP_READER_ID12
VARCHAR2 (200 Byte)YDTP_VISIT_DATE11
VARCHAR2 (200 Byte)YDTP_SUBEVENT10
VARCHAR2 (100 Byte)YDTP_VISIT9
VARCHAR2 (200 Byte)YDTP_PATIENT_ID8
VARCHAR2 (70 Byte)NQUALIFYV7
NUMBER (2)NSUBEVE6
NUMBER (5)NVISIT5
VARCHAR2 (20 Byte)NCPEVENT4
VARCHAR2 (10 Byte)NINVSITE3
VARCHAR2 (10 Byte)NPT2
VARCHAR2 (10 Byte)NSTUDY1
D a t a T y p eN u ll?C o l u m n N a m eI D
TABLESRE_EXTRACT_COMMENTS
VARCHAR2 (200 Byte)YDTP_COMMENT511
VARCHAR2 (200 Byte)YDTP_COMMENT410
VARCHAR2 (200 Byte)YDTP_COMMENT39
VARCHAR2 (200 Byte)YDTP_COMMENT28
VARCHAR2 (200 Byte)YDTP_COMMENT17
VARCHAR2 (70 Byte)NQUALIFYV6
NUMBER (2)NSUBEVE5
NUMBER (5)NVISIT4
VARCHAR2 (10 Byte)NPT3
VARCHAR2 (10 Byte)NINVSITE2
VARCHAR2 (10 Byte)NSTUDY1
D a t a T y p eN u ll?C o l u m n N a m eI D
TABLESRE_EXTRACT_DATA_WORK
VARCHAR2 (50 Byte)YDTP_READER_ID13
NUMBERYDTP_REPEAT_SEQUENCE_NUMBER12
NUMBERYDTP_QUESTION_OCCURRENCE_SEQ11
VARCHAR2 (50 Byte)YDTP_QUESTION_GROUP10
VARCHAR2 (50 Byte)YDTP_DCM_SUBSET9
VARCHAR2 (50 Byte)YDTP_DCM8
VARCHAR2 (50 Byte)YDTP_DCI7
VARCHAR2 (50 Byte)YDTP_VISIT_DATE6
NUMBERYDTP_SUBEVENT5
VARCHAR2 (50 Byte)YDTP_VISIT4
VARCHAR2 (50 Byte)YDTP_PATIENT_ID3
VARCHAR2 (50 Byte)YDTP_RAD_SITE2
VARCHAR2 (50 Byte)YDTP_INVESTIGATOR1
D a t a T y p eN u ll?C o l u m n N a m eI D
TABLESRE_EXTRACT_DATA_WORK (Cont.)
VARCHAR2 (250 Byte)YDTP_COMMENT527
VARCHAR2 (250 Byte)YDTP_COMMENT426
VARCHAR2 (250 Byte)YDTP_COMMENT325
VARCHAR2 (250 Byte)YDTP_COMMENT224
VARCHAR2 (250 Byte)YDTP_COMMENT123
VARCHAR2 (50 Byte)YDTP_PERFORMED_NY22
VARCHAR2 (50 Byte)YDTP_GRADE_CD21
VARCHAR2 (50 Byte)YDTP_SAMPLE_NY20
VARCHAR2 (50 Byte)YDTP_NEW_NY19
VARCHAR2 (50 Byte)YDTP_ASSESSMENT_DT18
VARCHAR2 (50 Byte)YDTP_EXAM_SP17
VARCHAR2 (50 Byte)YDTP_RESPONSE_NY16
VARCHAR2 (50 Byte)YDTP_SITE15
VARCHAR2 (50 Byte)YDTP_READ_TYPE_SP14
TABLESRE_EXTRACT_DATA_WORK (Cont.)
VARCHAR2 (50 Byte)YLESION_ID37
NUMBERYREPEATSN36
VARCHAR2 (70 Byte)NQUALIFYV35
NUMBER (2)NSUBEVE34
NUMBER (5)NVISIT33
VARCHAR2 (50 Byte)NCPEVENT32
VARCHAR2 (10 Byte)NINVSITE31
VARCHAR2 (10 Byte)NPT30
VARCHAR2 (10 Byte)NSTUDY29
DATEYCREATED_DATE28
TABLESRE_BDL_QUESTION_MAP
VARCHAR2 (50 Byte)NQUESTION_NAME3
VARCHAR2 (50 Byte)NDATASET2
VARCHAR2 (10 Byte)NSTUDY1
D a t a T y p eN u ll?C o l u m n N a m eI D
SRE_BDL_QUESTION_MAP Data
CRITERIA_MET_NYFRACSPProtocol###
PERFORMED_NYFRACSPProtocol###
GRADE_CDFRACSPProtocol###
SAMPLE_NYFRACSPProtocol###
NEW_NYFRACSPProtocol###
ASSESSMENT_DTFRACSPProtocol###
EXAM_SPFRACSPProtocol###
RESPONSE_NYFRACSPProtocol###
SITEFRACSPProtocol###
READ_TYPE_SPFRACSPProtocol###
READER_IDFRACSPProtocol###
COMMENT5FRACSPProtocol###
COMMENT4FRACSPProtocol###
COMMENT3FRACSPProtocol###
COMMENT2FRACSPProtocol###
COMMENT1FRACSPProtocol###
Q U E S T I O N _ N A M ED A T A S E TS T U D Y
TABLESRE_A_B_FILE_MAP
VARCHAR2 (200 Byte)NPROTOCOL11
VARCHAR2 (50 Byte)NINVESTIGATOR10
VARCHAR2 (50 Byte)NRAD_PATIENT9
VARCHAR2 (50 Byte)NRAD_SITE8
VARCHAR2 (50 Byte)NPT7
VARCHAR2 (50 Byte)NINVSITE6
VARCHAR2 (50 Byte)NSTUDY5
CHAR (1 Byte)NDESTINATION_FILE4
VARCHAR2 (8 Byte)YTERMINATION_DATE3
VARCHAR2 (8 Byte)YSTART_DATE2
CHAR (1 Byte)NREC_TYPE1
D a t a T y p eN u ll?C o l u m n N a m eI D
Execution
• exec pkg_sre_fracsp.fracsp_fill(‘PROTOCOL###', 'STABLE');
• exec pkg_sre_fracsp.fracsp_write_file(‘PROTOCOL###', 'STABLE');
Process Flow
VERTFRAC, OVETFRAC, VTF2FRAC
VIEWS
Timepoint Header View Reader Comments View
OC SAS Views
INSE
RT
INSE
RT
INSE
RT
Procedure FRACSP_FILL
Procedure FRACSP_
WRITE_FILE
BDL A FileBDL B File
Preview File
© 2009 Confidential and proprietary property of RadPharm, all rights reserved.
Key Success Factor
• FRACSP Dataset BDL A file, BDL B file, Preview file– Old Method: 9 hours– New Method: 1 hour 25 minutes
• Full set of files– Old Method: 12 hours– New Method: 3 hours
• Overall improvement– 400%
Contact
Prashant Rajopadhye, PMPOracle Database AdministratorRadPharm, Inc.Phone: +1 609 936 2600 x 2526E-mail: [email protected]