unlocking big data silos in the enterprise or the cloud (con7877)

34
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Oracle Data Integration: CON7877 Unlocking Big Data Silos in the Enterprise or the Cloud Jeff Pollock Vice President, Oracle Data Integration 1 Oracle OpenWorld 2014

Upload: jeffrey-t-pollock

Post on 28-Nov-2014

141 views

Category:

Software


1 download

DESCRIPTION

Oracle OpenWorld Presentation: Unlocking Big Data Silos in the Enterprise or the Cloud (Con7877)

TRANSCRIPT

  • 1. Copyright 2014 Oracle and/or its affiliates. All rights reserved. | Oracle Data Integration:CON7877 Unlocking Big Data Silos in the Enterprise or the Cloud Jeff Pollock Vice President, Oracle Data Integration 1 Oracle OpenWorld 2014
  • 2. Copyright 2014 Oracle and/or its affiliates. All rights reserved. | Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracles products remains at the sole discretion of Oracle. Oracle OpenWorld 2014 2
  • 3. Copyright 2014 Oracle and/or its affiliates. All rights reserved. | Todays Agenda 3 Oracle Data Integration Oracle Data Integration for Cloud Modern data integration infrastructure, proven for the cloud Oracle Data Integration for Big Data Reservoir Exciting new data architectures designed for big data Putting it All Together Getting the best of all worlds 1 2 3 4 Oracle OpenWorld 2014
  • 4. Copyright 2014 Oracle and/or its affiliates. All rights reserved. | Oracle Data Integration Solutions and Proven Benefits Oracle OpenWorld 2014 4 Improve Agility Deploy Projects Faster Reliable Real-Time Reduce Risk Popular, Proven Tools Open, Not Proprietary Reduce Costs Better Productivity Eliminate ETL Servers Analytic Data Integration Big Data Integration & Governance Data Warehouse Integration Business Intelligence Applications Enterprise Data Integration and Governance Enterprise Data Quality and Profiling Comprehensive, Heterogeneous Data Integration Business Glossary and Metadata Management Business Continuity Active-Active for Maximum Availability Zero Downtime Migrations Data Consolidation / Application Modernization 24 x 7 x 365
  • 5. Copyright 2014 Oracle and/or its affiliates. All rights reserved. | Comprehensive Data Integration & Governance Capabilities Oracle OpenWorld 2014 5 Real-Time Data Movement Low impact capture, stage in Hadoop Continuous data availability Data Transformation Bulk data movement Pushdown data processing Data Federation Virtualized Data Services Data Quality & Verification Fix quality at the source Verify data consistency Metadata Management Lineage and Impact Analysis Business Glossary Semantics Data GovernanceFoundationOracle Data Integrator(Transformation) Enterprise Data Quality(Profile, Cleanse, Match and De-duplicate) FastLoadOracle GoldenGate(Movement) Enterprise Metadata Management & Business Glossary(Business Glossary, Data Lineage, Impact Analysis and Data Provenance) Data Service Integrator(Federation) GoldenGateVeridata(Online Data Verification) ELT Processingon Hadoop or SQL Continuous Availability
  • 6. Copyright 2014 Oracle and/or its affiliates. All rights reserved. | Data GovernanceFoundation Differentiated Technical Approach Oracle OpenWorld 2014 6 Dynamic Data Movement Real-time CDC is by default, not ETL Least invasive on sources Proven best performance Integrated Oracle capture/apply No ETL Engines Take the processing to the data; dont move the data to the process Leverage your data engines for the workloads (Hadoop or SQL) Most Heterogeneous Leverage open source Hadoop, not proprietary distributions Hadoop is the Hub, not ETL tools Open metadata standardsOracle Data Integrator(Transformation) Enterprise Data Quality(Profile, Cleanse, Match and De-duplicate) FastLoadOracle GoldenGate(Movement) Enterprise Metadata Management & Business Glossary(Business Glossary, Data Lineage, Impact Analysis and Data Provenance) Data Service Integrator(Federation) GoldenGateVeridata(Online Data Verification) ELT Processingon Hadoop or SQL Continuous Availability
  • 7. Copyright 2014 Oracle and/or its affiliates. All rights reserved. | Most Heterogeneous Solution Oracle OpenWorld 2014 7 Hadoop HBase Hadoop Hive/Flume HP Enscribe HP NonStop HP Neoview Hypersonic SQL IBM DB2 iSeries IBM DB2 UDB IBM DB2 z Series IBM Informix IBM Netezza JMS / MQ Microsoft Access Microsoft SQLServer MySQL Pivotal Greenplum PostgreSQL Salesforce.com SAP BW / BI SAP ERP / ECC SAS SQL/MP SQL/MX Sybase ASE Sybase IQ Teradata Adaptive Altova Apache Hcatalog Apache Hive/HQL Borland CA ERwin ClouderaImpala COBOL Copybook DataStax Embarcadero EMC ProActivity GentleWare Google BigQuery Grandite HadaptHive HortonworksHive IBM Cognos IBM DB2 IBM DataStage IBM Discovery IBM Federation Server IBM Lotus Notes IBM Netezza IBM Rational Rose IBM Rational Architect InformaticaMetadata Mgr. InformaticaPowerCenter CoSORT ISO SQL Standard (DDL) MapRHadoop Hive MicroFocus Microsoft Access Microsoft Office Excel Microsoft Visio Microsoft SQL Server Microsoft SSIS Microsoft Visual Studio Microstrategy Magic Draw OMG CWM Standard OMG UML Standard Oracle BI Answers Oracle BI Enterprise Edition Oracle BI Server Oracle DAC Oracle Data Integrator Oracle Data Modeler Oracle Database Oracle Designer Oracle Hyperion Applications Oracle Hyperion Essbase Oracle Warehouse Builder Pivotal Greenplum PostgreSQL QlikView SAP BO Crystal Reports SAP BO Designer SAP BO Desktop Intelligence SAP BO Repository SAP BO Data Integrator SAP BO Data Steward SAP Master Data Management SAP Sybase PowerDesigner SAP Sybase ASE Database SAS Data Integration Studio SAS BI Server SAS Information Map SAS Metadata Management SAS OLAP Server Select SparxArchitect Syncsort Tableau Talend Teradata Tigris Visible W3C DTD & XSD Schema Operational Integration (Movement / Transformation) Metadata Harvesting (Glossary, Lineage & Impact Analysis) Oracle Database Oracle Exadata Oracle Big Data Appliance Oracle TimesTen Oracle OLAP Oracle Business Intelligence Oracle BI Applications Oracle E-Business Suite Oracle JD Edwards Enterprise One Oracle JD EdwardsWorld Oracle Fusion Applications Oracle Governance Risk and Compliance Oracle Fusion AIA OracleRetail Applications Oracle Agile BI/ DW OracleAgile PLM for Process OracleiFlexFlexCUBE Oracle iFlexMantas Oracle HyperionApplications Oracle PeopleSoft Oracle Siebel CRM / OnDemand Oracle Communications Oracle WebLogic Server Oracle Coherence Data Grid Oracle SOA Suite Oracle Enterprise Service Bus + open APIs and standards based meta-model
  • 8. Copyright 2014 Oracle and/or its affiliates. All rights reserved. | Oracle Middleware Cloud Platform Cloud On-Premise DIGITAL ENGAGEMENT APPLICATION & DATA INTEGRATION IDENTITYMANAGEMENT SYSTEMS MANAGEMENT APPLICATION INFRASTRUCTURE & TOOLS BUSINESS PROCESS MANAGEMENT BUSINESS ANALYTICS CONTENT & COLLABORATION Web Mobile Social Internet of Things
  • 9. Copyright 2014 Oracle and/or its affiliates. All rights reserved. | PREVIEW: Oracle Integration Cloud Service Oracle OpenWorld 2014 9 Key Features Portable to on-premiseand Cloud Simple, Web integration experience Recommendations driven mappings On-premise / Cloud lifecycle tools Secure development and operations Highly available service levels Monitoring & error management Managed back-ups, patches, upgrades Benefits High value and accelerated solutions Pre-integrated applications Rapid and fully automated provisioning Connectivity for on-premise
  • 10. Copyright 2014 Oracle and/or its affiliates. All rights reserved. | Oracle Analytics Cloud Services Metrics available across HCM, CX & ERP out of the box expertise Cross-source Connectors and Analytics bridging HCM, CX and Finance Proven results capitalizing on 4000+BI Apps deployments worldwide Daily users of reports and dashboard across the Oracle Cloud 5000+ 2M+ 10
  • 11. Copyright 2014 Oracle and/or its affiliates. All rights reserved. | Todays Agenda 11 1 2 3 4 Oracle Data Integration Oracle Data Integration for Cloud Modern data integration infrastructure, proven for the cloud Oracle Data Integration for Big Data Reservoir Exciting new data architectures designed for big data Putting it All Together Getting the best of all worlds Oracle OpenWorld 2014
  • 12. Copyright 2014 Oracle and/or its affiliates. All rights reserved. | Oracle Data Integration Pragmatic Solutions for Cloud 12 Cloud BI / Analytics Oracle Business Intelligence Cloud uses Oracle Data Integration Oracle Data Integration also supports non-Oracle BI/Analytics Cloud SaaS to Mart/EDW Bring SaaS Application data into on-premdata warehouses Synchronize reference data or master data with SaaS Apps Cloud Database Sync On premise DBs to managed or private cloud data centers Sync local databases with Database as a Service (DBaaS) Oracle OpenWorld 2014
  • 13. Copyright 2014 Oracle and/or its affiliates. All rights reserved. | On Premise Amazon S3 Bucket Amazon Redshift FTP On Premise Apps to Heterogeneous Cloud BI/AnalyticsOGGODIEDQODIOBIEE Key Opportunity Provide high volume data movement and data synchronization capabilities between on-premises and Cloud- based resources Perform E-LT/ETL and Data Quality transformations natively on Cloud BI/Analytics platforms Avoid using different Data Integration solutions for Cloud and on-premises deployments Oracle OpenWorld 2014 13
  • 14. Copyright 2014 Oracle and/or its affiliates. All rights reserved. | 14 UK ConservatoryOracle Data Integration with Amazon Redshift for moving transforming data Cloud Analytics Oracle OpenWorld 2014
  • 15. Copyright 2014 Oracle and/or its affiliates. All rights reserved. | On Premise SaaS Application Data into On Premise BI/AnalyticsODIEDQOGGODI Key Opportunity Integrate natively with on-premises resources and Cloud-based Applications such as Salesforce.com, Sales Cloud, Service Cloud or Eloqua Offload reporting to eliminate impact on production systems Provide high volume data movement and data synchronization capabilities for Cloud and on- premises Apps Oracle OpenWorld 2014 15
  • 16. Copyright 2014 Oracle and/or its affiliates. All rights reserved. | 16 Premier Roaster and Retailer of Specialty CoffeeOracle Data Integration to sync customer records to between Cloud and on premise Cloud Integrated Oracle OpenWorld 2014
  • 17. Copyright 2014 Oracle and/or its affiliates. All rights reserved. | 17 Australian WineryOracle Data Integration for Integrating Salesforce.com to On PremDatabases SaaS to On Prem Oracle OpenWorld 2014
  • 18. Copyright 2014 Oracle and/or its affiliates. All rights reserved. | On Premise Database to Database Replication in the CloudOGGOGG PrivateCloud / ManagedCloudODI Key Opportunity Synchronize data efficiently between on-premises databases and Oracle DBaaS Consolidate numerous databases into a Private or Public Cloud database infrastructure Implement an highly-available infrastructure for both on- premises and Cloud database deployments Oracle OpenWorld 2014 18
  • 19. Copyright 2014 Oracle and/or its affiliates. All rights reserved. | 19 World Wide Shipping and Logistics ProviderOracle Data Integration to Migrate Hundreds of Databases to a Consolidated Private Cloud 480 DB to Cloud Oracle OpenWorld 2014
  • 20. Copyright 2014 Oracle and/or its affiliates. All rights reserved. | Todays Agenda 20 1 2 3 4 Oracle Data Integration Oracle Data Integration for Cloud Modern data integration infrastructure, proven for the cloud Oracle Data Integration for Big Data Reservoir Exciting new data architectures designed for big data Putting it All Together Getting the best of all worlds Oracle OpenWorld 2014
  • 21. Copyright 2014 Oracle and/or its affiliates. All rights reserved. | Oracle Data Integration Powerful Big Data Solutions 21 Commodity Data Reservoir Leverage Oracle Data Integration with a wide array of databases or data warehouse appliances Support Hadoop distributions on commodity hardware Oracle Engineered Systems Deeply integrated with Oracle Big Data Appliance and Exadata Take advantage of Infinibandperformance, Oracle Big Data SQL, Columnar Compression, and all integrated Loader technologies Streaming Big Data Integrate realtimetransactional databases with streaming analytics Filter, join and transform data while it is in motion, make business decisions while data is in memory Oracle OpenWorld 2014
  • 22. Copyright 2014 Oracle and/or its affiliates. All rights reserved. | Heterogeneous Reservoir with Oracle Data Integration 22 Flume Hive on MR, Tez, Spark Logs OLTP DB SQOOP OGG Pig on MR, Tez, SparkODI SQOOP Any DWOGG Spark OozieOEDQOEMM Data Validation & Cleansing Metadata Mgmt & Lineage API/File Hive/HCat, HDFS,HBase Hive/HCat, HDFS,HBase NoSQL Flume Oracle OpenWorld 2014
  • 23. Copyright 2014 Oracle and/or its affiliates. All rights reserved. | 23 Top US AutomakerOracle Data Integration for RealtimeData Delivery to Hadoop Petabyte Scale Oracle OpenWorld 2014
  • 24. Copyright 2014 Oracle and/or its affiliates. All rights reserved. | 24 US Digital TV ProviderOracle Data Integration with Hadoop & Kafka 100m Tx/Hr Oracle OpenWorld 2014
  • 25. Copyright 2014 Oracle and/or its affiliates. All rights reserved. | Load to Oracle OLH/OSCH Red Stack Reservoir with Oracle Data Integration 25 TransformHiveODI Hive/HDFS FederateHive/HDFS to Oracle Big Data SQL Oracle DB OLTP Load from OracleCopyToBDA Hive/HDFS OGGOGG Hive/HDFS Oracle OpenWorld 2014
  • 26. Copyright 2014 Oracle and/or its affiliates. All rights reserved. | 26 Top European BankOracle Data Integration MapReduceData Transformations in Big Data Appliance Massively Parallel Oracle OpenWorld 2014
  • 27. Copyright 2014 Oracle and/or its affiliates. All rights reserved. | Oracle Does Big Data Integration Better 27 Dynamic Data Movement CDC is by default, not an add-on Least invasive on sources Proven best performance Native Oracle capture/apply NoETLEngine Take the processing to the data; dont move the data to the process Leverage your data engines for the workloads (Hadoop or SQL) Most Heterogeneous Leverage open source Hadoop, not proprietary distributions Hadoop is the Hub, not ETL tools Open metadata standards vs. Batch Data Movement Typical ETL vendors all default to batch data movement in their reference architectures Some can talk the talk but their CDC tech cant touch Oracle GoldenGatescale/performance ETL Engine Must Scale Alongside Hadoop Carefully watch how ETL engines scale out; parallelism runs via the Engine more H/W to buy Map out the physical deployment architecture, compare to ODI, the TCO difference will be clear Proprietary Vendor Lock-in One popular ETL vendor puts their engines at the center of the architecture, not Hadoop The mainframe of ETL vendors is has proprietary features that mainly run in their own distro A fake free ETL vendor sells proprietary add-ons vs. vs. Oracle OpenWorld 2014
  • 28. Copyright 2014 Oracle and/or its affiliates. All rights reserved. | Todays Agenda 28 Oracle Data Integration Oracle Data Integration for Cloud Modern data integration infrastructure, proven for the cloud Oracle Data Integration for Big Data Reservoir Exciting new data architectures designed for big data Putting it All Together Getting the best of all worlds 1 2 3 4 Oracle OpenWorld 2014
  • 29. Copyright 2014 Oracle and/or its affiliates. All rights reserved. | Data GovernanceFoundation Putting it All Together Oracle OpenWorld 2014 29 Dynamic Data Movement Real-time CDC is by default, not ETL Least invasive on sources Proven best performance Integrated Oracle capture/apply No ETL Engines Take the processing to the data; dont move the data to the process Leverage your data engines for the workloads (Hadoop or SQL) Most Heterogeneous Leverage open source Hadoop, not proprietary distributions Hadoop is the Hub, not ETL tools Open metadata standardsOracle Data Integrator(Transformation) Enterprise Data Quality(Profile, Cleanse, Match and De-duplicate) FastLoadOracle GoldenGate(Movement) Enterprise Metadata Management & Business Glossary(Business Glossary, Data Lineage, Impact Analysis and Data Provenance) Data Service Integrator(Federation) GoldenGateVeridata(Online Data Verification) ELT Processingon Hadoop or SQL Continuous Availability
  • 30. Copyright 2014 Oracle and/or its affiliates. All rights reserved. | 2014 2014 Oracle Excellence Award Ceremony for Fusion Middleware Innovation ORACLE FUSION MIDDLEWARE: CELEBRATE THIS YEAR'S MOST INNOVATIVE CUSTOMER SOLUTIONS Tuesday, September 30, 2014 5:00-5:45pm YBCA Theater (next to MosconeNorth) Session ID: CON7029
  • 31. Copyright 2014,Oracle and/or its affiliates. All rights reserved. | Join the Community #OOW14 #ODI12c #GoldenGate12c #EDQ12c Oracle Data Integration blogblogs.oracle.com/dataintegration Connect with Oracle on Social Media OR connect via the web Oracle Data Integration Home Pageoracle.com/goto/dataintegration
  • 32. Copyright 2014 Oracle and/or its affiliates. All rights reserved. | Questions and Answers Oracle OpenWorld 2014 32
  • 33. Copyright 2014 Oracle and/or its affiliates. All rights reserved. | 34 Oracle OpenWorld 2014