oracle data integrator 12c - getting started
TRANSCRIPT
OracleDataIntegrator12c:GettingStarted
MichaelRainey|UTOUGTrainingDays2017
• MichaelRainey-TechnicalAdvisor• Spreadingthegoodwordaboutgluentproductswiththeworld
• OracleDataIntegrationexpertise
• OracleACEDirector
• mRainey.co
• gluent.com
2
Introduction
we liberate enterprise data
AboutGluent
We are long term Oracle Database & Data Warehousing guys – long history of performance & scaling
About 20 people in distributed teams in US & UK
The world is changing – we help customers to get the best out of both worlds!
we liberate enterprise data
Gluent Data Platform enables all applications to access all enterprise data, at anytime, on demand, and without any code migration or rewrites
• GluenttobefeaturedintheStartupShowcaseatStrata+HadoopWorld(Tomorrow!)
• HadoopforDatabaseProfessionals one-daytrainingcourse(March21st)
4
Quickannouncements
gluent.com/event/hadoop-for-database-professionals-2
Why“GettingStarted”
6
Manyconferencepresentationsgettootechnical,toofast
WhatisODI?
• E-LT(Extract,Load,Transform)vsETL• Heterogeneoussourcesandtargets• Builttointegratealldata
• Flexibility• Reusablecodetemplates(KnowledgeModules)
• OpenAPI
• ODIwilladapttoyourdatawarehouse-andnottheotherwayaround
• Flowbasedmappings• LogicalvsPhysical
8
WhatisODI?
9
WheredidODIcomefrom?
Early2000s:Sunopsisformed
2006:OracleacquiresSunopsis.
2010:OracleDataIntegrator11greleased.Oracle’sfirstrealupdatetoODI.
Late2013:OracleDataIntegrator12creleased,combiningODIandOWBfeatures.
2012:ODIforBigDatafirstreleased
10
ODI12cArchitectureandComponents
*imagesource:https://docs.oracle.com/middleware/1221/odi/concepts/img/architecture.png
11
TypicalODIrepositorysetup-thebasics
Source:ateam-oracle.com/how-many-odi-master-repositories-should-we-have
12
TypicalODIrepositorysetup-full-blownenvironment
Source:ateam-oracle.com/how-many-odi-master-repositories-should-we-have
InstallationandConfiguration
• Knowthedifferenttypesofinstalls• Standard-usedforStandaloneAgentoronclientmachine(ODIStudioonly)
• Enterprise-JEEAgentorColocatedAgent,ODIConsole,FMCplugin
14
Chooseyourinstallationtypewisely
15
Configurationishalfthebattle
CONFIGURATION
• ODI_HOME/odi/studio/odi.sh|exe• Connecttoexistingrepository• Ifnoneexists,runRCUfirst
• Boom,you’rein!
16
ODIStudioConfiguration
• ODI_HOME/oracle_common/common/bin/config.sh|exe• RuntheWebLogicServerconfigurationwizard
• StandaloneAgent• CreatedomainoutsideofODIOracleHome
• JEEAgent• Addcredentialsmapping
• Create/configureadminserver,nodemanager,topology,etc.
17
StandaloneorJEEAgentConfiguration
18
ODIAgent-howitworks
ODIAgent
ODIrepository
Sourcedatabase
Targetdatabase
1.Executionrequestviahttp(s)
2.AgentcommunicateswithODIrepositorytogeneratecodeandgathersource/targetconnection
3.Agentpassescodetosource/targetdatabasesandupdatesrepositoryexecutionlogs
GettingStarted
• DataServer• Connectiontothedatasource
• PhysicalSchema• Databaseschema
• Workschema
• LogicalSchema• Abstractsthephysicalconnection/schemafromthelogical
• DevelopmentoccursagainsttheLogicalSchema
• Model• MetadataobjecttiedtoLogicalSchema
• Representsdatabaseschema,filedirectory,XMLdocument,etc
• Datastore• Logicalrepresentationoftable,file,XMLelement,etc.
20
Topology,Contexts,Models,andDatastores-Ohmy!
Alloftheseobjectsmustbecreatedandsetuppriortoanymappingdevelopmentcanoccur.
• There’sanotherwaytogetstarted-quickly!• NewModelandTopologyObjectswizard-ODI12.1.3• InODI11g,theODIQuickStartwasa10stepchecklist,whereeachstepledtoanothersectioninthedocumentation
21
Developer’sQuickstart
22
Topology,Contexts,Models,andDatastores
23
LogicalConstraintsonthemetadata
23
LogicalConstraintsonthemetadata
23
LogicalConstraintsonthemetadata
23
LogicalConstraintsonthemetadata
MappingsandKnowledgeModules
25
Mappings-LogicalDatastorecomponent
Joincomponent
Splitoutputtomultipletargettables
MappingsarebuiltfromdatastoreandcomponentstologicallyformtheETL
26
Mappings-Physical
Physicalimplementationsofthemappingsarerepresentedbasedonlocationofthedatastore.KnowledgeModulesareselectedtogeneratethecode.
26
Mappings-Physical
Physicalimplementationsofthemappingsarerepresentedbasedonlocationofthedatastore.KnowledgeModulesareselectedtogeneratethecode.
ODIwilladapttoyourdatawarehouse-andnottheotherwayaround
27
KnowledgeModules-thesecretsauce
*imagesource:FusionMiddlewareDevelopingKnowledgeModuleswithOracleDataIntegrator
27
KnowledgeModules-thesecretsauce
*imagesource:FusionMiddlewareDevelopingKnowledgeModuleswithOracleDataIntegrator
27
KnowledgeModules-thesecretsauce
*imagesource:FusionMiddlewareDevelopingKnowledgeModuleswithOracleDataIntegrator
OrchestrationPackagesandLoadPlans
• Grouplikeobjectstogether• Performspecializedtasks(looping,etc)
29
Packages
• ExecutesScenarios(executionobjectsthatareessentiallycompiledcode)• Stepscanberuninparallelorserially• If-then-elsestepcanevaluateavariabletodeterminepath• Exceptionstepscanbecustomizedandexecutedonfailure
30
LoadPlans-thetoplevelorchestrationobject
MoreflexibilityProcedures
• Reusablecomponentsforadmintasks• DDL,filearchivescripts,exceptionhandling,etc
• Movingortransformingdata,itprobablyshouldbeamapping• Creatingacomplex,recurringtaskfordatamanipulation,probablyaKnowledgeModule
32
Proceduresformaintenancetasksandone-offfunctions
Imagessource:http://ritt.md/9E
• IntegrationwithGoldenGate(changedatacapture)andEnterpriseDataQuality(for…dataquality)• ODIforBigData• ODICloudService
33
Othernotablefeatures
• OracleDataIntegratoris…• anE-LT/dataintegrationtool
• builttointegratealldata
• flexible.ODIwilladapttoyourdatawarehouse-andnottheotherwayaround
• Chooseyourinstallationwiselyandknowconfigurationismoreinvolved• ManyobjectsmustbecreatedpriortoMappingdevelopment
• UsetheDeveloper’sQuickstart!
• Flowbasedmappingsseparatebusinesslogicfromphysicalimplementation• KnowledgeModulesarethe“secretsauce”ofODI• Moreinfo: mRainey.co/odi12c-getting-started ateam-oracle.com/?cat=64 blogs.oracle.com/dataintegration
34
Recap
35
we liberate enterprise data
thank you!