The Oracle Data Integrator 11g Blog

Sunday, 5 February 2012

Creating Models

 Hi Gurus,

Let us see how to create models using designer in ODI Studio.
To practice this exercise you should finish the following exercise
Setting the topology

Run the ODI Studio
Click on Designer
Click on New model as shown in the below window

 Now the Model Creation wizard opens
Give some name to the model: SCOTT
Select technology as "ORACLE"
Select the logical schema : SRC_SCOTT_LS
Click on Selective Reverse Engineering


 Check All
now you are able to see the tables to import
Click on top left Green button to reverse engineer
Click ok to save

 Repeat the above steps to create another model for TRG_SCOTT as shown below


After creating Models you can create an Integration Project



Thanks & Regards
 WWW.ODIGURUS.COM

Setting The Topology

 Hi Gurus,



Today Let me show you how to set the topology for oracle schemas.

Assuming that you have the following schemas defined in your oracle database.
1. SRC_SCOTT

2. TRG_SCOTT

Run the ODI Studio


Click  on connect to repository.
Select the appropriate ODI Login
Click OK
Now ODI is connected to Repository and you would able to see the ODI tools Desiner,Topology, Operator and Security manager


Click on topology
Expand the technolgy tree
Select oracle technology


Right Click on Oracle technology
Click on New Data server
Now the Data server creation wizard opens

Enter same name for your data server: ORACLE_DS
Give the user name and password of System user of your database: system/sysdba
Then click on JDBC
Click on search button to select the appropriate JDBC Driver
Enter the JDBC url to connect to your database
My database is running on my local system (localhost) on port number 1521 and SID is "odigurus"


Click on Test connection
Click OK to save
Click on Test
If the test connection is successful then you need to create physical schemas
Right click on the Created Data Server
Click on New physical schema



Now the Physical schema creation wizard opens
Select "SRC_SCOTT" as shown below
Click on Context
 Click "+" symbol

Give some logical schema name: SRC_SCOTT_LS
Click on Save

 Repeat the Above steps to create a physical schema for TRG_SCOTT



Now you have perfectly defined the topology for the Oracle schemas SRC_SCOTT and TRG_SCOTT
You can go to designer and start creating models for these two physical schemas



Thanks & Regards

WWW.ODIGURUS.COM

Tuesday, 3 January 2012

SCD Type 3



Hi Gurus, 
      Here we are trying to implement the “Slowly Changing Dimension Type 3”  functionality which is not available in ODI 11g.   
     To implement this functionality we have to customize the IKM Oracle Incremental Update which is detailed here.  

      In the following steps we try to capture the updates/inserts in the SCOTT.EMP table SAL Column and store it in TRG_EMP4 table.

1.     Create a target table for this exercise from SCOTT Model  as TRG_EMP4 (or as done here from another table TRG_EMP3 by ‘Duplicate Selection’).





 2.     In Columns tab create another column PREV_SAL and move it near SAL Column and rename the SAL as CURR_SAL.


  3.     Create an interface INF_SCOTT_EMP4 (or anything you like) and in mapping drop the EMP and TRG_EMP4 tables. 

4.     a. Make EMPNO as the Update KEY and map the SAL column to CURR_SAL and PREV_SAL in Target (TRG_EMP4) which will define the EMPNO as UK (unique Key).

b. Select the PREV_SAL and CURR_SAL as UD1 and UD2 respectively and unselect the insert and update boxes for PREV_SAL column.

 
 5.       Import the IKM Oracle Increment Update and set it to the Target Mapping.
6.     Set the Flow control to FALSE and CREATE_Targ_Table = true ,TRUNCATE=false.
7.     To catch the changes in target table change one of the curr_sal  value.
8.     Write a query in SQL editor to track the changes in EMP.SAL Column  and execute it to verify that it updates the TRG_EMP4 with those changes.



  9.       Run the interface to verify that it updates the column. The columns are not updated in the TRG_EMP table! (we have not yet implemented it).
  10.       To catch this change we have to include this logic in IKM.
  11.       Set the UD1 and UD2 as True for PREV_SAL and CURR_SAL and  remove the checks for the insert and update for PREV_SAL.


 12.    Create another IKM by duplicating the IKM Oracle Incremental Update  as IKM Oracle Incremental Update SCD3.




13.   Create another step before the Update existing rows step indicated above. Observe how the ODIref variables used in this step.

  14.   Add the SQL developed before by adding the ODI variables by analyzing the code in the above update step.   

a.     The Target table is changed with “TARG_NAME” and “A” for automatic mapping of the schema.


 b.     Change the PREV_SAL column with UD2 as 


 c.     Change the target CURR_SAL column as         



 d.     T.EMPNO will be replaced with UK as below AND for the SCOTT.EMP retrieve the Integration parameter value INT_NAME. (we need to get it from the interface I$ table).

UPDATE  <%=odiRef.getTable("L", "TARG_NAME", "A")%> T
   SET <%=odiRef.getColList("", "T.[COL_NAME]", ",\n\t", "", "(UD2)")%> =<%=odiRef.getColList("", "T.[COL_NAME]", ",\n\t", "", "(UD1)")%>
WHERE    (<%=odiRef.getColList("","T.[COL_NAME]", ", ", "", "UK")%>)
   IN    (
SELECT  <%=odiRef.getColList("","T.[COL_NAME]", ",\n\t\t\t", "", "UK")%>
FROM  <%=odiRef.getTable("L", "TARG_NAME", "A")%> T ,
<%=odiRef.getTable("L", "INT_NAME", "A")%> S
WHERE  <%=odiRef.getColList("","T.[COL_NAME]", ", ", "", "UK")%> =<%=odiRef.getColList("","S.[COL_NAME]", ", ", "", "UK")%>
AND  <%=odiRef.getColList("", "T.[COL_NAME]", ",\n\t", "", "(UD1)")%>!=
                       <%=odiRef.getColList("", "S.[COL_NAME]", ",\n\t", "", "(UD1)")%>
                        )

15.   Set the Flow_Control =false, TRUNCATE=false, CREATE_TARG_TABLE= true, DELETE_TEMPORARY_OBJECTS=false.

16 .   Make some changes in EMP table and commit.
17.      Execute the interface and verify that the changes are updated in SCD.

18.   Check the I$ table for update flag.


 19.   Verify the no. of rows updated or inserted by the Interface and the query generated.


  20.  The update in SAL column in EMP is stored in the TRG_EMP4.Prev_SAL in the same row with the new value (Slowly Changing Dimension Type 3 behavior).



Thanks

Mohan Ramaswamy
Sr Analyst (Contractor)
Snap-On Business Solutions
Richfield, OHIO, US 44286

 

Tuesday, 27 December 2011

Oracle Data Integrator Components

Oracle Data Integrator Components
Oracle Data Integrator (ODI) is a software application based on ETL (extract-transform-load) structure used for data transformation and merging processes. In today's business the meaning of well organized data has increased dramatically and that is why having specialized software is so important.

Furthermore, these applications must be prepared to work in mixed systems and platforms environment and be sure that none of the quality will deteriorate. Oracle Data Integrator provides decalrative rules apart from implementation details and also its E-LT (extract-load, transform) structure eliminates need of having separate ETL server which results in optimized performance, efficiency and scalability.

Main architecture component in Oracle Data Integrator is the repository that is accessed by client-server mode or thin client mode. Repository is absolutely open, comprehensive, shared metadata database which is accessible for the applications responsible for transformation, project development, integration and information redistribution. Other architecture components are the Topology Manager, Designer, Security Manager, Operator, Agent, Scheduler and Metadata Navigator. Graphical interface module that manages all the information about system's physical and logical structure is called the Topology Manager. Descriptions created with the Topology Manager of server data, machine and site will allow to run same interfaces on different systems. A tool designed to develop transformations and to check data integrity is called Designer. Among many fuctions its most significant are visualization of data flow, development and maintenance data integration and transformation, reverse-engineering of applications or repository's data. Oracle Data Integrator permissions and users management tool is simply called Security Manager. Operator is the module responsible for monitoring and production management.


Agent executes application interfaces which Designer designed, next the Scheduler schedules them. Metadata Navigator browses through contents of Oracle Data Integrator Repository in search of wanted content. It is a thin client-oriented for search and analysis but also to manage all production processes, it can also trace and cross-reference data lineage. Apart from this standard modules comes a bunch of „out-of-the-box” modules which can be fully extensible, e.g. Log Miner, SQL Server Triggers, Oracle DBLink, Check MS Excel, Oracle Merge, Check Sybase – all as a part of Pluggable Knowledge Architecture are written for certain systems and environments and in fact are the very core of the Oracle Data Integrator, making the whole process accessible, modular and as simple as it can be. In case where our target is an Oracle database, natural choice is to go with Oracle Warehouse Builder, but where target is not an Oracle based environment, like SQL Server or XML, Teradata we can use Oracle Data Integrator to support OWB. When the source is XML or Web Service, SOAP then ODI supplements as well.

Clearly Oracle Data Integrator can be used in many different cases – e.g. when some dirty data has to be filtered out before loading, or data integration is very complicated or/and processed in real-time.

Oracle Data Integrator works as well in the staging and transforming area as the support for other Oracle software. It is „hot pluggable” which means connecting to any heterogeneous environment and still provide with number of required operations.

Thanks & Regards
 WWW.ODIGURUS.COM

Tuesday, 22 November 2011

Router Transformation

Hi Gurus,

 The Following videos explain how to do Router Transformation using ODI.


 Hope you will learn several ODI features by watching these videos











 Thanks & Regards
 WWW.ODIGURUS.COM

Thursday, 3 November 2011

ODI JOIN

Hi Gurus,
Today i am sharing the video which explains how to do join between two tables using ODI.

In this video i used to perform join between EMP and DEPT table using the following condition

EMP.DEPTNO=DEPT.DEPTNO

And the result is stored in a table called EMP_DEPT.

For this i had created a data store with the name EMP_DEPT with all columns from EMP data store and two columns LOC and DNAME from DEPT.

The interface KM option "Create Target Table" is set to true to create target table at run time.

Hope this will useful to understand some good features of ODI 





Thanks ,

ODI FILTER

Hi Gurus,


Today let us see the video how the filter transmission is implemented using ODI.

In this example i took EMP table as my source where i applied the filter condition.


I hope you enjoy the vedio.



Thanks Gurus,

Ram kumar Lanke