otm data management using python
TRANSCRIPT
![Page 1: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/1.jpg)
<Insert Picture Here>
OTM Data Management Using Python Eric Rosenbloom ([email protected]) Solutions Director, Consulting August 2012
![Page 2: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/2.jpg)
OTM Data Management Using Python
• We show you how to: • Use ClientUtil.py to export/import CSV • Write a Python script that uses ClientUtil.py to automate an
OTM to OTM data replication process using CSV approach • Use ClientUtil.py to export/import db.xml • Use ClientUtil.py to push a GLogXML document to OTM • Time permitting, touch briefly upon using Python to perform
XML data transformations
![Page 3: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/3.jpg)
Target Audience
• This talk is meant primarily for technically inclined individuals responsible for OTM data management
![Page 4: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/4.jpg)
Background on Python
• Python is an interpreted scripting language that comes pre-installed on your OTM application server along with OTM specific utilities like ClientUtil.py
• If you have no prior experience with Python, suggest doing the python tutorial at http://docs.python.org/tutorial/
• Examples presented utilize Python that is pre-installed on an OTM application server. • However, they will work as well when installed on a laptop or
desktop
![Page 5: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/5.jpg)
Diagram: ClientUtil.py in the middle
![Page 6: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/6.jpg)
CSV, db.xml, and GLogXML
• CSV = “Comma Separated Variable”. Standard file format whereby fields are delimited by commas. Supported by Microsoft Excel
• Db.xml is a “database-centric” xml format where there is a close mapping between the xml and the underlying table and columns
• GLogXML is described by the GLogXML.xsd schema. Specifies XML representation for OTM business objects like Item, Location, Release, PlannedShipment, etc
![Page 7: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/7.jpg)
PYTHONPATH environment variable
• On your OTM application server, set PYTHONPATH environment variable to include directory where OTM Python utilities are installed: export PYTHONPATH=$GLOG_HOME/utils/integration/python
![Page 8: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/8.jpg)
Contents of utils/integration/python
![Page 9: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/9.jpg)
Running ClientUtil.py for help • Run ClientUtil.py with no arguments to get help text:
![Page 10: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/10.jpg)
ClientUtil.py to CSV export python ClientUtil.py
-command csvExport -hostname consapp07 -username GUEST.ADMIN -password CHANGEME -tableName ACTIVITY -whereClause "rownum < 2" -localDir . -localFileName myfile.csv -excludePublic N
![Page 11: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/11.jpg)
Myfile.csv contents ACTIVITY ACTIVITY_GID,ACTIVITY_XID,ACTIVITY_NAME,DOMAIN_NAME,INSERT_USER,INSERT_D
ATE,UPDATE_USER,UPDATE_DATE EXEC SQL ALTER SESSION SET NLS_DATE_FORMAT = 'YYYYMMDDHH24MISS' "RECEIVE","RECEIVE","RECEIVING FREIGHT","PUBLIC","DBA.ADMIN","20110130185513",,
![Page 12: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/12.jpg)
ClientUtil.py to CSV export: Ex #2 python ClientUtil.py -command csvExport -hostname consapp07 -username GUEST.ADMIN -password CHANGEME -tableName LOCATION -whereClause "1=1" -localDir . -localFileName location.csv -excludePublic Y
![Page 13: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/13.jpg)
Export a subset of columns • The csvExport command exports all columns in the
table • If you want a subset of the columns in the table, use
the csvQuery command instead of the csvExport command
![Page 14: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/14.jpg)
CSV query example python ClientUtil.py -command csvQuery -hostname consapp07 -username GUEST.ADMIN -password CHANGEME -sqlQuery "SELECT location_gid FROM location" -localDir . -localFileName myfile.csv
![Page 15: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/15.jpg)
Myfile.csv contents SELECT location_gid FROM location LOCATION_GID "PHILLY" "NYC" ,
![Page 16: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/16.jpg)
CSV Import Example: data content
LOCATION LOCATION_GID,LOCATION_XID,COUNTRY_CODE3
_GID,DOMAIN_NAME GUEST.SIG2012,SIG2012,USA,GUEST ,
![Page 17: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/17.jpg)
CSV import example: command-line python ClientUtil.py -command csvImport -hostname consapp07 -username GUEST.ADMIN -password CHANGEME -localDir . -localFileName new_location.csv -xvalidate Y -csvUtilCommand ii
![Page 18: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/18.jpg)
CSV import example: xml output <?xml version="1.0" encoding="UTF-8"?> <CSVUtilServletHelper> <CSVUtil> <Command>ii</Command> <DataDir>/home/oracle/otm62/temp/upload/</DataDir> <DataFileName>Transmission1338186382454.csv</DataFileName> <ExcludePublic>true</ExcludePublic> <ProcessCSV> <xvalidate>true</xvalidate> <DatabaseGlobalName>CONSAPP06.US.ORACLE.COM</DatabaseGlobalName> <TableName>LOCATION</TableName> <ColumnList>LOCATION_GID,LOCATION_XID,COUNTRY_CODE3_GID,DOMAIN_NAME</ColumnList> <sqlString>insert into LOCATION (
LOCATION_GID,LOCATION_XID,COUNTRY_CODE3_GID,DOMAIN_NAME) values (?,?,?,?)</sqlString> <ProcessCount>1</ProcessCount> <ErrorCount>0</ErrorCount> <SkipCount>0</SkipCount> </ProcessCSV> </CSVUtil> </CSVUtilServletHelper>
![Page 19: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/19.jpg)
Data Replication Script
• Now let’s write our own “copytabs.py” Python script to use ClientUtil to facilitate replicating data from OTM instance A to OTM instance B
![Page 20: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/20.jpg)
copytabs.py: command line
![Page 21: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/21.jpg)
copytabs.py: includes
![Page 22: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/22.jpg)
copytabs.py: main
![Page 23: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/23.jpg)
copytabs.py: Main class
![Page 24: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/24.jpg)
copytabs method (all in list)
![Page 25: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/25.jpg)
copytab method (copy one table)
![Page 26: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/26.jpg)
Rate Data Replication Script
• Now let’s write a shell script that uses copytabs.py to copy rate data from one OTM instance to another
![Page 27: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/27.jpg)
copyrates.sh
![Page 28: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/28.jpg)
ClientUtil.py to single table db.xml export
python ClientUtil.py -command xmlQuery -hostname consapp07 -username GUEST.ADMIN -password CHANGEME -sqlQuery "select * from activity" -rootName ACTIVITY -localDir .
-localFileName activity.db.xml
![Page 29: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/29.jpg)
single table db.xml export
![Page 30: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/30.jpg)
ClientUtil.py to multi-table db.xml export
python ClientUtil.py -command xmlQuery -hostname consapp07 -username GUEST.ADMIN -password CHANGEME -sqlQuery "select location.*, (cursor (select location_role_profile.* from location_role_profile where location_role_profile.location_gid = location.location_gid)) as location_role_profile from location where domain_name = ‘GUEST’ " -rootName LOCATION -localDir . -localFileName location.db.xml
![Page 31: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/31.jpg)
multi-table db.xml export
![Page 32: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/32.jpg)
ClientUtil.py db.xml import
python ClientUtil.py -command xmlImport -hostname consapp07 -username GUEST.ADMIN -password CHANGEME -transactionCode IU -localDir . -localFileName location.db.xml
![Page 33: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/33.jpg)
db.xml import
![Page 34: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/34.jpg)
Sample “Location” GLogXML document
![Page 35: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/35.jpg)
ClientUtil.py: Push GLogXML document
python ClientUtil.py -command sendXMLFileViaHttpPost -url http://consapp07/GC3/glog.integration.servlet.WMServlet -fileName Location.xml
![Page 36: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/36.jpg)
Sample Command Line and Response
![Page 37: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/37.jpg)
Login to OTM
![Page 38: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/38.jpg)
Query Location Manager
![Page 39: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/39.jpg)
Python for XML Data Transformations
• Performing XML data transformations in Python is relatively straight-forward
• Helper methods in commonTools.py • Sample script SampleLocationTransform.py
![Page 40: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/40.jpg)
SampleLocationFlatFile.loc
![Page 41: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/41.jpg)
SampleLocationTransform.py
![Page 42: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/42.jpg)
Summary
• You have learned • How to get started with Python • About CSV, db.xml, and GLogXML • About the various Python tools that come installed with OTM • How to use ClientUtil.py to export/import CSV and db.xml • Intro to writing your own Python scripts • Use ClientUtil.py to
• export/import db.xml • Push GLogXML to OTM
• Introduction to using Python for XML data transformations
![Page 43: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/43.jpg)
If you want to learn more…
• Reference the OTM Data Management Guide:
• http://download.oracle.com/docs/cd/E20111_01/otm/acrobat/datamanagement.pdf
![Page 44: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/44.jpg)
If you want to learn more…
![Page 45: OTM Data Management Using Python](https://reader031.vdocument.in/reader031/viewer/2022012312/58a5eb141a28aba5728b4db5/html5/thumbnails/45.jpg)