ODI11g: Creating an ODI Project: Developing an ODI XML to Database Transformation Using Interface with ODI Constraint

 

Purpose

This OBE tutorial describes and shows you how to create an Oracle Data Integrator (ODI) interface that will import an XML file into a database table CLIENT. This process also involves adding an ODI constraint to the CLIENT model. Any records that do not pass the constraint will be loaded to the E_CLIENT table.

Time to Complete

Approximately 40 minutes

Overview

Oracle Data Integrator provides the ability to validate data during the load to the target using ODI constraints or database constraints. When this data is checked against a constraint using the Flow Control, any errors that are found are not loaded to the target but are loaded to an errors table that is created and managed by ODI. Each row of this table represents a record that did not pass a constraint, the row also has a message column that explains why the records were rejected. This table can be edited within ODI Designer or any other tool that can edit relational tables. However, this is not always a convenient way for the end user or business user edit the data. This example uses an ODI interface, alternatively you can build an ODI procedure that will load an XML file with a constraint.

Scenario

Linda works as a database administrator for Global Enterprise. In Global Enterprise, Linda is responsible for performing database management and integration tasks on various resources within the organization. In particular, Linda is responsible for data loading, transformation, and validation. To begin working on her projects, Linda created the new Master repository and Work repository. Now Linda needs to create a project and an interface to export data from a source XML file, perform transformations, and load this data into RDMBS table using ODI constraints.

Software and Hardware Requirements

The following is a list of software requirements:

The system should include the following installed products:o Oracle Database 11go Oracle Data Integrator 11gR1

If not done before, start the services and components for Oracle Database 11g

Prerequisites

Before you start the tasks, make sure that your system environment meets the following requirements:

1 . Have installed Oracle Database 11g. If not done before, start the services and components for Oracle Database 11g2 . Have installed Oracle Data Integrator 11gR13 . Before attempting this OBE, you should have successfully completed the following OBEs:

ODI11g: Creating and Connecting to ODI Master and Work Repositories.

ODI11g: Creating an ODI Project and Interface : Exporting a Flat File to a Flat File

To access these OBEs, click HERE.

 

Creating a New Project with Oracle Data Integrator

To create a new project within Oracle Data Integrator, perform the following steps:

1. Start ODI Designer: Start > Programs > Oracle > Oracle Data Integrator > ODI Studio . Click Connect to Repository.

 

2. Select WORKREP1 from the Login Name drop-down list if not already selected. Enter SUPERVISOR in the User field and SUNOPSIS in the Password field. Click OK to login. In the Designer tab, click the Projects tab, click New

Project icon  , and then select New Project.

3. On the screen that appears, set the Name of the project to ODI_exp_XML-RT in the Name field. The Code field is

filled automatically. Click Save icon  . The newly created ODI_exp_XML-RT project now appears in the Projects tree view. You have now successfully added a new ODI project.

 

4 . You export an XML file directly to a relational table target. The knowledge modules required for this are LKM File to SQL and IKM SQL Incremental Update. To import the KMs, expand the Project tab in the left panel, right-click the Knowledge Modules folder and select Import Knowledge Modules. 

Note: In this example, the generic SQL KMs are used. However, specific KMs for the RDBMS technology can be used as well.

5 . On the screen that follows,in the File Name, navigate to xml-reference directory as shown in the following screenshot. Click Open.The files to import should appear in the Import Knowledge Modules window. Press and hold the CTRL key to select CKM SQL, IKM SQL Control Append, and LKM SQL to SQL. Click OK.

Parameter Value

Loading LKM SQL to SQL

Integration IKM SQL Control Append

Check CKM SQL

 

6. On Import Report window, click Close. Expand the Loading (LKM), Check, and Integration (IKM) folders, and view each imported KM in the tree view as shown below.

 

Creating New Data Server and Physical Schema for the XML Source Model

In this section you use the sample XML file GEO_DIM.xml provided with this OBE. Download file DEMO.zip from HERE, and extract it in your <ODI-HOME> directory. For example: C:\oracle\product\11.1.1\Oracle_ODI_1\oracledi\demo\

To create new Data server and Physical schema for a new XML model, perform the following steps:

1 . Click Topology tab. In Topology navigator, click the Physical Architecture tab, select Technologies > XML. Right-click XML and select New Data Server. Name this data server XML File. Click JDBC tab. 

 

2 . In JDBC tab, select driver: com.sunopsis.jdbc.driver.xml.SnpsXmlDriver. Enter the following JDBC URL, click

Save  . To verify connection, click Test Connection. Click Test and then click OK.

<ODI_HOME>/demo/xml/GEO_DIM.xml&re=GEOGRAPHY_DIM&ro=false&case_sens=true

 

 3 . Right-click XML File Data server, and select New Physical Schema. For the Directory (Schema) and Directory (Work

Schema) fields, select GEO_D. Click Save  . In Information window, click OK.

4 . .Open Logical Architecture, navigate to Technologies > XML, right-click XML and select New Logical Schema. Enter the Logical Schema name: GEO_DIM, select the your nely created physical schema in Global context as shown

below. Click Save  and close the editing windows.

 

Creating RDBMS Schema/User for ODI Target Datastore

You need RDBMS schema/users to host the ODI target datastore.

Note: If you completed the OBE " Creating an ODI Project and Interface: Exporting a Flat File to a RDBMS Table" earlier, you should already have ODI_STAGE schema created. In this case, you can skip this section.

To create a new RDBMS schema for the ODI target datastore, perform the following steps:

1 . Start SQL Developer. You will create the new schema/user by executing the following SQL commands: 

create user ODI_STAGE identified by ODI_STAGE

default tablespace users temporary tablespace temp;

grant connect, resource, create trigger, create view to ODI_STAGE;

Creating a New ODI Target Dataserver and the Physical Schema

Note: Skip this section, if you completed the OBE " Creating an ODI Project and Interface: Exporting RDBMS Table to a Flat File" earlier.

After you created the new database target datastore, you need to create a new ODI target data server and the Physical schema. To create the ODI target data server and physical schema, perform the following steps:

1. In ODI, Open ODI Topology Navigator and then select the Physical Architecture tab. Expand Technologies, right-click Oracle and select New Data Server. 

 2. In the Data Server: New window, enter the values provided in the table below. Click the JDBC tab.

ParameterValue

Name ODI_STAGEInstance/dblink (Data Server)ORCLUser ODI_STAGEPassword ODI_STAGE

3 .Click   for JDBC Driver. Select Oracle JDBC Driver. Click OK. Click   for JDBC Url, select jdbc:oracle:thin:@<host>:<port>:<sid>, and then click OK. Edit the Url to read: jdbc:oracle:thin:@localhost:1521:ORCL for Oracle Database. Note: Do not copy and paste in the JDBC Url field. This may cause problems with entering a valid URL string.

4 . Click Test Connection. In the window that opens, click Yes to save your data. In the Information window, click OK. Click Test to verify successful connection. Click OK.

 

5 . Click Expand Oracle technology node, right-click ODI_STAGE dataserver, and then select New Physical Schema.

6 . In Schema (Schema) and Schema (Work Schema) field enter your ODI_STAGE schema. Click Save button. Close ODI_STAGE.ODI_STAGE physical schema window. In the Information window, click OK.

7 . Open Logical Architecture tab, expand Technologies > Oracle. Right-click Oracle technology and then select New Logical Schema.

8 . Name logical schema ODI_STAGE. In the Global context, connect this logical schema to ODI_STAGE physical schema as shown below. Click Save button and then close the tabs.

Creating a New ODI Source Model

Create a new ODI source XML model that will be used within your ODI Interface. To create a new ODI source model, perform the following steps:

1 . Open ODI Designer. Click Models tab and select New Model. On the screen that appears, enter the values provided in the following table. Click the Reverse Engineer tab.

Parameter Value

Name SRC_GEO_DIM_XML

Technology XML

Logical Schema GEO_DIM

 

 2 .

On the Reverse Engineer tab, set the Context to Global . Click Save button  and then close the tab.

3 . In the Models tab, right-click the SCR_GEO_DIM_XML model. Select Reverse Engineer. To verify that the model is successfully reversed, expand the model as shown below.

 

Creating a New ODI Target Model

Create a new ODI target model that will be used within your ODI Interface. To create a new ODI target model, perform the following steps:

1 . Open ODI Designer. Click Models tab and select New Model. On the screen that appears, enter the values provided

in the following table. Click the Reverse Engineer tab.

Parameter Value

Name RDBMS_TARGET

Technology Oracle

Logical Schema ODI_STAGE

 

 2 . On the Reverse Engineer tab, set the Context to Global . Click Save button and then close the tab.

3 . In the Models pane, click "+" to expand the RDBMS_TARGET and SRC_GEO_DIM_XML. UnderSRC_GEO_DIM_XML, right-click the CLIENT datastore and select Duplicate selection . Click Yes to confirm.

4 . Drag and drop the new duplicate Copy of CLIENT from the SRC_GEO_DIM_XML model to the RDBMS_TARGET model. Next, right-click the Copy of CLIENT data store within the RDBMS_Target and click Open. Incase you get an

Object Locking dialog box, click No. Change the name of the datastore to CLIENT and click Save button  . Close the tab.

Note: No physical table has been created on the RDBMS_TARGET. When the interface is run, it will create the physical table CLIENT.

 

 

Setting up ODI Constraint on CLIENT Datastore

During the duplication of the CLIENT data store the constraint for the XML file was also copied. You have to delete this constraint and set another constraint in CLIENT datastore. To set up the ODI constraint , perform the following steps:

1 . In the Models tab, expand RDBMS_TARGET > CLIENT > Constraints > CITY Copy of CITY_CLIENT. Right-

clickCITY_CLIENT CITY and click Delete. Click Yes to confirm.

 

 2 . Under RDBMS_TARGET > CLIENT > Constraints right-click and select New Condition.

3 .In the New object window, provide the following values and then click Save  .

Parameter Value

Name Age_Constraint

Type Oracle Data Integrator Condition

Where cast(Client.age as numeric) >=21

Message Client must be at least 21 years old

Creating a New ODI Interface to Perform XML File to RDBMS Table Transformation

To create a new ODI Interface to perform XML file to RDBMS table transformation, pefrom the following steps:

1 . In ODI Designer, click the Projects tab. Expand your project ODI_exp_XML-RT, and then expand First Folder. Right-click Interfaces and select New Interface. In Optimization Context field, select Global.

2 . On the screen that follows, enter the interface name as INT-LOAD_TBL_CLIENT . Set Optimization context to Global. Click Mapping tab. 

3 . Click the Models tab to drag the source and target to the diagram. Drag the CLIENT data store from the SRC_GEO_DIM_XML into the Source container and Drag the CLIENTdata store from the RDBMS_TARGET into the Target container. When Designer asks “Do you want to perform an Automatic Mapping?” click Yes. 

4 . In the target datastore, select CLIENT_ID column. Scroll down to select the Key option, as shown below. Click the Flow tab. 

 

5 . Click Definition tab again, and make sure that the settings match those provided iin the table below. Click the flow tab.

Parameter Value

Name INT_LOAD_TBL_CLIENT

Staging Area Different From Target Select this option

In-Memory Engine Sunopsis_Memory_Engine

 

6 . Click the Flow tab. Click the Satging Area, and then in LKM Selector, select: LKM SQL to SQL. Click the Target datastore . In the Target properties, select the following options, and then click Control tab. In the Control tab, verify

settings and save your interface  .

Note: Incase you get an Object Locking dialog box, click No.

Option Value

RECYCLE_ERRORS true

CREATE_TARG_TABLE true

 

 

7 .To test your interface, click Execute button  . The following screen appears. Retain the defaults and click OK. On the next screen, click OK. 

 

 

8 . Open ODIOperator and verify that your interface was executed successfully. In Operator, click Session List tab,

select All Executions, and the then click refresh button  . View execution results for interface ODI_exp_XML-RT, as shown below.

 

9 . Double-click Step 16 and click the Execution tab. View the number of rows inserted into the error table E_CLIENT in ODI_STAGE schema.

Note: Optionally, you can open SQL Developer, navigate to Administrator > Other Users > ODI_STAGE > Tables > E_CLIENT, and view error records populated in E_CLIENT error table. 

10 . Open the Interface tab. Select Mapping tab, right-click Target Datastore - CLIENT, and then select Data. View Data inserted in the target table. Close Data Editor. Close the tabs.

 

Summary

In this tutorial, you have learned how to:

Verify the Prerequisites Create a New Project with Oracle Data Integrator Create new Data server and Physical schema for a new XML model Create new RDBMS schema for the ODI target datastore Create ODI target data server and physical schema Create a new ODI XML source model Create a new ODI target model that will be used within ODI Interface Set up the ODI constraint on CLIENT target datastore Create a New Interface with Oracle Data Integrator for XML file to RDBMS table data transformation