data profiling in oracle warehouse builder - iware · pdf fileoracle applications and...

iWare Logic Technologies Pvt. Ltd Oracle Applications & Technologies | Consulting | Training | Implementation
Email: [email protected] http://www.iwarelogic.com/
Data Profiling in Oracle Warehouse Builder
Data profiling is the first step for any organization to improve information quality and provide better decisions, because of its integration with the ETL features in Warehouse Builder and other data quality features, such as data rules and built-in cleansing algorithms. From this article beginners will get an idea about how profiling works. I hope you have an idea about all the ETL objects and how they work. This following scenario will help you understand Data Profiling in a better way.
Scenario: I have a data source that contains two External tables: Expense_Categories and Expense_Data. The Expense_Data table contains Expenses based on different Expense Categories. With the help of data profiling the managers can analyze the data on which the cost cuttings should be implemented. Following are the steps which will help you to implement Data Profiling. Step 1) Create a Data Profile: I have chosen the External Table (named as EXPENSE_DATA) in Source Module EXPENSE_WH. Now to create Data Profile just right click on Data Profile in project explorer of design center and select new. On next Window I should provide the name for Data Profile. I have given the name as DF. On next window you have to choose objects over here which you want to profile. You can see the chosen objects in selected pane. Here I have selected EXPENSE_DATA External file from Expense_wh Module.

To finalize your Data profile click on finish. After that the created Profile is shown in Data Profile Editor Window with following notification will be shown.

Step 2) Profile the Data Now to profile the object, click on Profile in Profile menu. As the process gets ready to start, alert message comes.
As I click on ok the actual profiling process gets started and will be shown in Monitor Panel,
After Completion of Data Profiling Process, it will ask for retrieving the results. If I click yes then retrieving gets started. After Completion of retrieving results the process completion will be shown in Monitor Panel. The profiling results will be shown in Data Profile Editor.

The profile results Canvas provides you different tabs facilitating Attribute Analysis, Referential Analysis, checking Functional Dependency. Data Drill Panel provides you drilled information of selected attribute in profile results canvas,

Step 3) Derive Data Rules Now after completion of profiling I can create my own Data rules and apply it to make data more specific. I can create Data Rules in two ways,
1) To create data rule go to the design center, right click on Data Rules in project Explorer window, create one Data Rule Folder (I have created Derived_Data_Rules).
2) Directly choose any columns minimum or Maximum from profile results canvas and click on Derive Data Rule.
For creating Data Rule I need to right click on Data rule in Design Center and select New. On next window provide the name you want to give for the Data Rule (I have given here EXPENSE_TYPE) On next window define the Data Rule,
Choose the Type of rule you want to create. From provided data rules like, Custom, Domain List, Domain Pattern List, Domain Range, Common, Format, No nulls, Functional Dependency, Referential, Name/ Address, Unique Key. Depending on Data Rule type chosen you should provide parameters

Here I have chosen Data Rule Type as Domain List and specified the domain values. After all specifications go to Next Window and click on Finish.
Now I Click Finish to Complete Data Rule creation. For applying Created Data Rules I will click on Apply Rule in Data Rule Panel.

Here I Choose the Data Rule EXPENSE_TYPE which we created earlier. As I click on drop down list in Binding Column. It shows list of columns in profiled table. I have selected the column EXPENSE_TYPE on which I want to apply the rule. Now go to next page and click finish.

Now the applied Data Rule EXPENSE_TYPE will be shown in Data Rule Panel. For getting the updated profile results I need to profile the object again. So I click on Profile in Profile Menu. To see the updated results with respect to applied rule,
choose the Data Rule tab select value below %compliant table
Now you can see in Data Drill Panel it shows me,
By default all Distinct values of the column Values which are complaint to applied Data Rule will be shown with green
symbol. Values which are Non complaint to applied Data Rule will be shown with red
symbol. If I choose any value from distinct values list then at right side it shows me all the rows in table with all columns where selected value got repeated.

If I select complaint option from drop down button and I can see list of complaint values only. I have applied the rule of type Domain List with list items as, Air Direct Billed, Breakfast, Car Rental, Dinner, Entertainment, Lunch and parking. So it shows me only these records which are included in my domain list.

Now I select Non complaint option from drop down button and you can see list of Non complaint values.

Step 4) Generating Corrections-: After I have derived data rules from the profiling results, I can create the schema and mapping corrections. The schema correction creates scripts that can be used to create a corrected set of source data objects with the derived data rules applied. For creating Correction I select Create Correction Option from Profile Menu. I can only create Correction pointing to target Modules. Here it provides me list of available Target Modules or I can create my own Target Module. I can Check the checkbox below if I want to remove my previous correction objects.
Now I Select the Object which I want to correct. Here I selected EXPENSE_DATA External File from EXPENSE_WH Module.

By selecting the Data Rules tab you can select the Data rules that are already
defined.(in this case there is only one rule)

I can Select Data Type tab and Check or uncheck columns to generate new data
types.

Here I verify the applied Data Rule and Data Types for corrected object.

On next Window I have to specify the cleanse strategy for Non complaint records. Here I selected Remove.
As I click on next, Mapping and staging tables are created in Target Module for
the correction purpose.
As you can see in Target Module T_EXPENSE, Mapping =>M_EXPENSE_DATA, Tables => EXPENSE_DATA, TMP__EXPENSE_DATA is created.

iWare Logic Technologies

data profiling in oracle warehouse builder - iware · pdf fileoracle applications and...

Documents