p.3 pts odi11g odi architecture

8/12/2019 P.3 PTS ODI11g ODI Architecture

1/36

Copyright 2010, Oracle. All rights reserved.


2/36

Oracle Data Integrator

Architecture

PTS Workshop: ODI 11g


3/36


Insert Picture Here>

Agenda

ODI Architecture Overview

ODI Components

Agents

Console

Repositories Data Flow

Q&A


4/36


ODI Architecture


5/36


ODI Architecture

Desktop

Repositories

ODI Studio

Operator

Designer

Topology

Security

ODI MasterRepository ODI Work

Repository

Sources and Targets

Legacy Applications

ERP/CRM/PLM/SCM

Files / XML DBMS DW / BI / EPM

JVM

JavaApplication

ODI SDK

WebLogic11g/ ApplicationServer

Data Sources Connection Pool

Web Service Container

Public WS DataServices

FMW ConsoleODI Plug-in

Servlet Container

ODI Console

JavaEE

Application

ODI SDK

Runtime WS

Java EEAgent

JVMRuntime WS

StandaloneAgent

Open Web Services fordecoupled invocation of

any ODI jobs

Open WebServices fordecoupled

invocation of anyODI jobs

SDK APIs for Javageneration of any ODI

system metadata

Metadata is an openrelational modelwith built-in Flex

Fields

OpenTools for adding OSfunctionality to any ETL

job


6/36


ODI Components


7/36


Graphical Modules

JavaMost Platforms

DB - Most ISO-92

Security Manager

Manage user

privileges

Operator

Operate production

Monitor sessions

Topology Manager

Define the infrastructure

of the IS

Designer

Reverse-Engineer

Develop Projects

Release Scenarios

Repository

ODI Studio


8/36


SOA / Java EE Run-Time Components

Any ISO-92 RDBMS

Execute Jobs

Java EE Agent

Handles schedules

Orchestrate sessions

Java - Any Platform

Operator

Operate productionMonitor sessions

Designer

Reverse-EngineerDevelop Projects

Release Scenarios

WebLogic Java EE server

Information System

Return Code

Submit Jobs

Read sessionsWrite reports

Monitor sessionsView Reports

Repository

Public Web Services

Manage sessions

Access Data

ODI Studio

Any Client App

Consume DataControl EL-T Operations

Invoke and managesessions

Data Services

Access Data and

Changed Data

Access Data


9/36


Standalone Run-Time Components

Any ISO-92 RDBMS

Execute Jobs

Standalone Agent

Handles schedules


Java - Any Platform

Operator

Operate production

Monitor sessions

Designer

Reverse-Engineer

Develop ProjectsRelease Scenarios

Java - Any Platform

Information System

Return Code

Submit Jobs

Read sessionsWrite reports

Monitor sessionsView Reports

Lightweight

Distributed Architecture

Repository


10/36


Enterprise Manager

and ODI ConsoleAny Web Browser

Control and monitor ODI

production

Browse metadata lineage

Any ISO-92 RDBMS

SubmitExecutions

Execute Jobs

Java EE Agent

Handles schedules


Information System

Return Code

Repository

WebLogic Java EE server

ODI Console

Web access to the

repository

Enterprise ManagerFusion Middleware Console

Other components

SOA Suite, WLS

Servers, Web apps.


11/36


ODI ConsoleWeb access to the

repository

Enterprise Manager FMC

Other

componentsSOA Suite, WLS

Servers, Web apps.

Security

ManagerManage user privileges

OperatorOperate production

Monitor sessions

Topology

ManagerDefine the IS

infrastructure

DesignerReverse-Engineer

Develop Projects

Release Scenarios

ODI Studio

Components: a Global View

Any ISO-92 RDBMS

Java EE Agent

Handles schedules


Java - Any Platform

Java - Any Platform

Any Web BrowserBrowse metadata

lineage

Operate production

Information System

Repository Access

HTTP Connection

Execution Query

Repository

Any Client AppConsume DataControl EL-T Operations

Public Web Services

Manage sessions

Access Data

Invoke andmanage sessions

Data Services

Access Data and

Changed Data

Access Data


12/36


Designer Navigator

In ODI Designer, youcan create, configure,and execute various

ODI objects.

Workspace to defineODI objects and designEL-T transformations

Toolbar

Objecttree

OtherDesignereditors

DesignerProjects

editor


13/36


Operator Navigator

In OperatorNavigator, you canmonitor execution

of ODI objects.Toolbar

Workspace todefine ODI objects

and design EL-Ttransformations

The monitoringsession steps

enable you to

perform debugging.


14/36


Topology Navigator

In ODI Topology Manager,

you set the architecture ofyour information system, thetechnologies, servers,

schemas, and repositories.

Technologies

tree

Toolbar

Workspace to

define ODITopology objects


15/36


Security Navigator

In Security Navigator,you manage objects,profiles, users, and

hosts.Objecttree

Workspace to

define objects,profiles, users, and

hosts


16/36


Agents


17/36


What Is an Agent?

An agent is a run-time component of ODI that orchestrates theintegration process.

At design time, developers generate scenarios from the business

rules that they have designed. The code of these scenarios is

then retrieved from the repository by the agent at run-time.

This agent then connects to the data servers and orchestratesthe code execution on these servers.


18/36


ODI Agents

Agents are lightweight Java processesthat orchestrate the execution of objects

at run time.

Agents can:

Execute objects on demand, or

Execute according to predefinedschedules


19/36


Two Types of AgentsJEE and Standalone

JDeveloper

ODI Studio

Designer Operator

Security Topology

WebLogic 11g(or other JEE)

Data Sources Connection Pool

Repository Data Sources Sources Data Sources Target Data SourcesRepository Data Sources Sources Data Sources Target Data Sources

Repository Data Sources Sources Data Sources Target Data Sources

Servlet Container

ODI Agent

ODI Console

Web Service Container

ODI Public

Web Service

ODI DataServices

Any Application Container

Any Application

ODI SDK APIs

MBeans Server Registry

Any MBeans App

ODI MBeansfor ODI Agent

Sources and Targets

ERP

---

------

---

---

---

CRM

- - -

Legacy

---

------

---

---

------

------

---

---

---

Best-of-Breed ApplicationsPLM

RDBMS ODI Repositories

ODI MasterRepository

ODI WorkRepositoryODI Work

RepositoryODI Work

Repository

Standalone Java

Any Java App

ODI SDK APIs

Any Web App

KnowledgeModules

ODI Agent

Standaloneagent

JEEagent


20/36


21/36


Example of Standalone Agent

Lightweight

Distributed Architecture

Any ISO-92 RDBMS

ExecuteJobs

Standalone Agent

Orchestratessessions

Java - Any Platform

Operator

Operateproduction

Monitorsessions

Java - Any Platform Return Code

SubmitJobs

Readsessions

Write reports

Monitor sessions

View Reports

Designer

Reverseengineer

Developprojects

Releasescenarios

Information System

Repository


22/36


Console


23/36


ODI ConsoleDevelopment

ODIConsole

ODIConsole

Agent


24/36


Enterprise Manager Console

Development

Enterprise ManagerConsoleODI Plug-infor EM

Console

Agent


25/36


ODI Repositories


26/36


ODI Repositories

There are two types of repositories in ODI: Master repository

Work repository

Development repository

Execution repository

Work repositories are always attached to a Master repository.


27/36


Master Repository

Master and Work Repositories

Models

Projects

Execution

Work Repository(Development)

Execution

Execution Repository(Production)

Security

Topology

Versioning

Two type of Repositories: Master and Work

Work Repositories are always attached to a Master Repository


28/36


Example of a Repository Set-Up

Models

Projects

Execution

Work Repository

(Development)

Models

Projects

Execution

Work Repository

(Test & QA)

Execution

Execution Repository(Production)

Security

Topology

Versioning

Master

Repository

Create and archive

versions of models,

projects and

scenarios Import released

versions of models,

projects andscenarios for testing

Import released and

tested versions of

scenarios for

production

DevelopmentTest Production Cycle


29/36


Models

Projects

Execution

Work Repository

(Development)

Models

Projects

Execution

Work Repository

(Test & QA)

Execution

Execution Repository

(Production)

Security

Topology

Versioning

Master

Repository

Create and archive

versions of models,

projects and

scenarios (from the

GUI)Import released

versions of models,

projects andscenarios for testing

(From the GUI)

XML export / Import

(manual)

DevelopmentTest Production Cycle

Security

Topology

Master

Repository

Repository Set-Up: Separate Networks

XML export / Import

(manual)

Firewall


30/36


ODI Data Flow


31/36


Oracle Data IntegratorArchitecture

Repositories

User Interfaces

Developer

Developer

DeveloperExecution

Log

Code

Metadata

Rules

Agent

Data FlowConductor

Servers and Applications

CRM

Legacy

ER

P

Data

Warehouse

Files /

XMLESB

Return

Codes

Execution

No Server: only an orchestrator (The Agent) that will send theappropriate SQL statements to the different databases.

Developers each have the Java GUI on their machineand share a central repository to store the metadataand the transformation processes (no local storage).


32/36



Repositories

User Interfaces

Developer

Developer

Developer

Execution

Log

Code

Metadata

Rules

Agent

Data FlowConductor


CRM

Legacy

ER

P

Data

Warehouse

Files /XML

ESB

Return

Codes

Execution

Processes are orchestrated (and possibly scheduled)by an ODI Java agent (no graphical component to

allow for the largest possible set of platform support).


33/36



Repositories

User Interfaces

Developer

Developer

Developer

Execution

Log

Code

Metadata

Rules

Agent

Data FlowConductor


CRM

Legacy

ER

P

Data

Warehouse

Files /XML

ESB

Return

Codes

Execution

Multiple agents can co-exist in any given environment.Typical locations for the agents are on the targetsystem and/or on the system that hosts the ODI

repository.

O i 6 t t P d ti


34/36


ODI Design-Time Environment ODI Runtime Environment

User Interfaces

Overview: 6 steps to Production

1. Retrieve/Enrich metadata

2. Design transformations

3. Orchestrate data flows

4. Generate/Deploy data flows

5. Monitor executions

6. Analyze impact / data lineage

Development

Development Servers and Applications

Agent

Data FlowConductor

CRM

Legacy

ERP

DataWarehouse

Files / XML

User Interfaces

Administrators

Designers

ESB

Design-timeRepositoriesDesign-time

Repositories

Production

Production Servers and Applications

Agent

Data FlowConductor

CRM

Legacy

ERP

DataWarehouse

Files / XML

Operator

ODI Console

ESB

RuntimeRepository


35/36


p.3 pts odi11g odi architecture

Documents