agileanalytics: agile real-time bi with oracle business intelligence, oracle data integrator and...
DESCRIPTION
A successful real-time BI implementation involves integrating the logical and physical model, source and target databases, ETL subsystems, and front-end dashboards and reports. Agile methodologies often struggle to deliver real-time implementations due to the complexity of the intermingled moving parts. Using the Oracle Information Management Reference Architecture as a backdrop, attendees will see how to construct a complete real-time BI system using Oracle Business Intelligence, Oracle Data Integrator and Oracle GoldenGate, driven by an Agile approach focusing on user stories and time-boxed iterations.TRANSCRIPT
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
AgileAnalytics: Real-Time BI with Oracle Business Intelligence, Oracle
Data Integrator and Oracle GoldenGate
2
Stewart Bryson
medium.com/@stewartbryson @stewartbryson linkedin.com/in/stewartbryson
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
About Red Pill Analytics
Capacity Analytics Agile Cloud Development for Oracle BI & Analytics
!
Checkmate DevOps for Oracle BI (Cloud-hosted or On-premises)
!
Single Doses Training + Strategy + Solutioning + Infrastructure + Support
3
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Real Time BI with Kevin & Stewart
• 19 episodes over the last 2 years
• Featured guests from the Oracle BI & EPM industry
• Published on iTunes and YouTube
• Join us at the Thursday Deep-Dive session from 9-11am for a LIVE show that will be simultaneously broadcast online!
4
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
AgileAnalytics 1.0: Focus on Performance
5
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
AgileAnalytics 1.0: Focus on Performance
6
Exadata: Extreme Performance
OBIEE: Extreme Metadata
Agile: Extreme Methodology
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
AgileAnalytics 1.0: Focus on Performance
6
Exadata: Extreme Performance
OBIEE: Extreme Metadata
Agile: Extreme Methodology
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
AgileAnalytics 2.0: Focus on Integration
7
GoldenGate: Extreme Realtime
ODI: Extreme Transformation
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Oracle Information Management Reference Architecture
11
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Oracle Information Management Reference Architecture
1212
Virtu
aliz
atio
n &
Q
uery
Fed
erat
ion
Enterprise Performance Management
Pre-built & Ad-hoc BI Assets
Information Services
Data Ingestion
Information Interpretation
Access & Performance Layer
Foundation Data Layer
Raw Data Reservoir
Data Science
Data Engines & Poly-structured sources
Content
Docs Web & Social Media
SMS
Structured Data Sources
•Operational Data •COTS Data •Master & Ref. Data •Streaming & BAM
Immutable raw data reservoir Raw data at rest is not interpreted
Immutable modeled data. Business Process Neutral form. Abstracted from business process changes
Past, current and future interpretation of enterprise data. Structured to support agile access & navigation
Discovery Lab Sandboxes Rapid Development Sandboxes
Project based data stores to support specific discovery objectives
Project based data stored to facilitate rapid content / presentation delivery
Data Sources
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Oracle Information Management Reference Architecture
13
Abstracts rate of reception from rate of
distribution
Facilitates “right-time” flow of data
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Oracle Information Management Reference Architecture
14
Abstracts atomic data from the business process.
Process-neutral; Business logic agnostic
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Oracle Information Management Reference Architecture
15
Process-specific
Imbued with business logic
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
SugarCRM (Our Sample Dataset)
16
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Typical Waterfall Approach
• Interview users • Construct requirements • Create logical data model • SQL prototyping of source model
• Document source-to-target mapping
• ETL development • Front-end development • Performance tuning
18
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Alternative: Iterative Waterfall?• Design iterations around smaller chunks
• Iteration 1: Interviews and user requirements • Iteration 2: Logical modeling • Iteration 3: ETL Development • Iteration 4: Front-end development
• Requires 4 iterations before we get any usable content
19
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics 20
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Manifesto for Agile Software Development
“We are uncovering better ways of developing software by doing it and helping others do it. !
That is, while there is value in the items on the right, we value the items on the left more.” http://agilemanifesto.org/
22
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Manifesto Applied to AgileAnalytics• User Stories instead of requirements documents • User asks for content or functionality through a narrative
• Typically includes current version of the report
• Time-boxed iterations • Iteration has a standard length • Choose one or more user stories to fit in that iteration
• Rework is part of the game • There are no “missed requirements”... only those that haven’t been delivered yet.
23
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Rapid Development Sandbox
27
Oracle GoldenGate can
bypass the Staging layer to trickle-feed the
Foundation layer
Staging layers exist to support batch
load scenarios
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Rapid Development Sandbox
28
We use our Foundation layer
combined with any Access & Performance layer content to close stories immediately
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Rapid Development Sandbox
29
Developers can use independent development
sandboxes to work on their issues in isolation
Are development sandboxes radical?
RPD Development requires a BI Server
!
RPD Development requires a query tool
!
RPD Development requires data visualization
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Oracle GoldenGate Architecture
32
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Loading Foundation: Standard ETL Approach
33
Source System
Staging Layer
Foundation Layer
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Loading Foundation: AgileAnalytics Approach
34
Source System
Staging Layer
Foundation Layer
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
GoldenGate for AgileAnalytics: Loading the Foundation Layer
35
map SUGARCRM.ACCOUNTS, target SUGARFND.ACCOUNTS, KEYCOLS(ID, FND_SCN), INSERTALLRECORDS, colmap( USEDEFAULTS, FND_SCN=@GETENV("TRANSACTION" , "CSN"), FND_ROW_RANK=@TOKEN("TK_ROW_RANK"), FND_COMMIT_DATE=@GETENV("GGHEADER" , "COMMITTIMESTAMP"), FND_DML_TYPE=@GETENV("GGHEADER" , "OPTYPE") );
Map SOURCE and TARGET
KEYCOLS defines matching criteria
USEDEFAULTS defines column for column mapping Built-in attributes from the
“token” and the “header”
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
GoldenGate for AgileAnalytics: Loading the Foundation Layer
36
map SUGARCRM.ACCOUNTS, target SUGARFND.ACCOUNTS, KEYCOLS(ID, FND_SCN), INSERTALLRECORDS, colmap( USEDEFAULTS, FND_SCN=@GETENV("TRANSACTION" , "CSN"), FND_ROW_RANK=@TOKEN("TK_ROW_RANK"), FND_COMMIT_DATE=@GETENV("GGHEADER" , "COMMITTIMESTAMP"), FND_DML_TYPE=@GETENV("GGHEADER" , "OPTYPE") );
Map SOURCE and TARGET
KEYCOLS defines matching criteria
USEDEFAULTS defines column for column mapping Built-in attributes from the
“token” and the “header”
INSERTALLRECORDS makes this our Foundation Layer
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
GoldenGate INSERTALLRECORDS
37
NAME TICKER_SYMBOL
OWNERSHIP FND_SCN FND_COMMIT_DATE FND_DML_TYPERed Pill Analytics 2992758 08/20/2012 12:00 AM INSERT
NAME TICKER_SYMBOL OWNERSHIP
Red Pill Analytics
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
GoldenGate INSERTALLRECORDS
38
NAME TICKER_SYMBOL
OWNERSHIP FND_SCN FND_COMMIT_DATE FND_DML_TYPERed Pill Analytics 2992758 08/20/2012 12:00 AM INSERT
Red Pill Analytics, LLC Private 2992760 03/13/2013 12:00 AM UPDATE
NAME TICKER_SYMBOL OWNERSHIP
Red Pill Analytics, LLC Private
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
GoldenGate INSERTALLRECORDS
39
NAME TICKER_SYMBOL
OWNERSHIP FND_SCN FND_COMMIT_DATE FND_DML_TYPERed Pill Analytics 2992758 08/20/2012 12:00 AM INSERT
Red Pill Analytics, LLC Private 2992760 03/13/2013 12:00 AM UPDATE
Red Pill Analytics, Inc. RPAI Public 2992762 05/02/2013 12:00 AM UPDATE
NAME TICKER_SYMBOL OWNERSHIP
Red Pill Analytics, Inc. RPAI Public
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
GoldenGate INSERTALLRECORDS
40
NVL (!LEAD ( fnd_commit_date ) over ! ! ! ( PARTITION BY id ! !! ORDER BY fnd_commit_date ),!to_date( '12/31/9999','mm/dd/yyyy' )!)
NAME FND_SCN FND_COMMIT_DATE FND_EXPIRE_DATE FND_DML_TYPE FND_CURRENTRed Pill Analytics 2992758 08/20/2012 12:00 AM 03/13/2013 12:00 AM INSERT N
Red Pill Analytics, LLC 2992760 03/13/2013 12:00 AM 05/02/2013 12:00 AM UPDATE N
Red Pill Analytics, Inc. 2992762 05/02/2013 12:00 AM 12/31/1999 12:00 AM UPDATE Y
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
GoldenGate INSERTALLRECORDS
41
CASE!WHEN ( RANK() over (PARTITION BY id ! ! ! ORDER BY fnd_commit_date desc )) = 1!THEN 'Y'!ELSE 'N'!END
NAME FND_SCN FND_COMMIT_DATE FND_EXPIRE_DATE FND_DML_TYPE FND_CURRENTRed Pill Analytics 2992758 08/20/2012 12:00 AM 03/13/2013 12:00 AM INSERT N
Red Pill Analytics, LLC 2992760 03/13/2013 12:00 AM 05/02/2013 12:00 AM UPDATE N
Red Pill Analytics, Inc. 2992762 05/02/2013 12:00 AM 12/31/1999 12:00 AM UPDATE Y
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Model-Driven Development•Start with thin requirements • Sample reports • SQL whenever possible •Annotated discrepancies
•Development against the Foundation Layer • Top-down approach • Build empty BMM containing logical “facts”, “dimensions” and “hierarchies” •Map physical sources to the BMM
•Develop permanent, high-impact dashboards •No work in this iteration is throw-away
43
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Model-Driven Development
44
•Start with thin requirements • Sample reports • SQL whenever possible •Annotated discrepancies
•Development against the Foundation Layer • Top-down approach • Build empty BMM containing logical “facts”, “dimensions” and “hierarchies” •Map physical sources to the BMM
•Develop permanent, high-impact dashboards •No work in this iteration is throw-away
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Model-Driven Development
45
Simplification of the Data ModelIntegration of Disparate Data Sources
Addition of Business Logic and Calculations
OBIEE is for star schemas
OBIEE is for star schemas False
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics 48
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics 49
Logical Table Sources (LTS) provide the
functionality for mapping transactional schemas as
dimensional models. !
Siebel initially acquired nQuire and the
technology that would become OBIEE to report
against Siebel CRM.
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Model-Driven: Mapping Physical to Logical
50
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Model-Driven: Mapping Physical to Logical
50
Dragging and dropping tables from the Physical Layer is the only way to create logical tables in the Business
Model
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Model-Driven: Dimension Tables
55
1
23
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Model-Driven: Dimension Tables
56
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Model-Driven: Dimension Tables
57
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Model-Driven: Dimension Tables
58
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Model-Driven: Dimension Tables
59
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Model-Driven: Fact Tables
60
1
23
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Model-Driven: Fact Tables
61
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Model-Driven: Fact Tables
62
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Model-Driven: Fact Tables
63
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Model-Driven: Fact Tables
64
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Model-Driven: EDW Content (Date Dimension)
65
1
2
3
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Model-Driven: EDW Content (Date Dimension)
66
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Model-Driven: EDW Content (Date Dimension)
67
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Model-Driven: EDW Content (Date Dimension)
68
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Model-Driven: EDW Content (Date Dimension)
69
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Model-Driven: Initial Dashboard
70
Dimensional Prompts
Performance Tiles
Operational MetricsTrending Information
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
ETL Development•We us GoldenGate to populate our Staging layer • Standard batch •Micro batch
•Use Oracle Data Integrator (ODI) to transform our data to meet business requirements •Need to decide what flavor of GoldenGate to use • Extend our current implementation •Use ODI journalization framework
72
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Loading Access and Performance: Standard Approach
73
Staging Layer
Foundation Layer
Access & Performance
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Loading Access and Performance: AgileAnalytics Approach
74
Staging Layer
Foundation Layer
Source System Access & Performance
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Loading Access and Performance: AgileAnalytics Approach
74
GoldenGate handles Staging and Foundation
layers
Staging Layer
Foundation Layer
Source System Access & Performance
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Loading Access and Performance: AgileAnalytics Approach
74
ODI from Staging for transactional table loading
GoldenGate handles Staging and Foundation
layers
Staging Layer
Foundation Layer
Source System Access & Performance
ODI from Foundation for complex calculations
and integration
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Oracle Data Integrator (ODI)
•Data Integration tool within Oracle Fusion Middleware 12c •Robust sources and targets • Relational databases • Essbase • Flat file and XML •Web services • Big Data
•Declarative design • Logical implementation • Physical implementation
•ETL or E-LT •Database as ETL engine • Java Agents as ETL engine
75
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Where Have We Seen This Before?
76
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Where Have We Seen This Before?
78
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Where Have We Seen This Before?
79
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Where Have We Seen This Before?
•Abstraction of physical sources using logical entities •Logic built into a model instead of a process •Metadata-based paradigm selection •“Do the work once”
80
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Where Have We Seen This Before?
81
Abstraction of physical sourcesLogic built into Models not Processes Metadata-based paradigm-selection
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
ETL Development: Joins with Declaration
82
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
ETL Development: Joins with Declaration
83
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
ETL Development: Joins with Declaration
83
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
ETL Development: Collateral from Model-Driven Development
84
1
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
ETL Development: Collateral from Model-Driven Development
84
1
2
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
ETL Development: Collateral from Model-Driven Development
84
1
2
3
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
ETL Development: Collateral from Model-Driven Development
84
1
2
3
WITH SAWITH0 AS (select count(T2196.ID) as c1, sum(T2196.AMOUNT) as c2, concat(concat(T3853.FIRST_NAME, ' '), T3853.LAST_NAME) as c3, T2196.LEAD_SOURCE as c4, T2196.SALES_STAGE as c5 from SUGARFND.ACCOUNTS T2041 /* Accounts (Standard) */ , SUGARFND.USERS T3853 /* Users (Account Owner Role) */ , SUGARFND.OPPORTUNITIES T2196 /* Opportunties (Standard) */ , SUGARFND.ACCOUNTS_OPPORTUNITIES T2221 /* Accounts Opportunities (Standard) */ where ( T2041.ASSIGNED_USER_ID = T3853.ID and T2041.ID = T2221.ACCOUNT_ID and T2196.ID = T2221.OPPORTUNITY_ID ) group by T2196.LEAD_SOURCE, T2196.SALES_STAGE, concat(concat(T3853.FIRST_NAME, ' '), T3853.LAST_NAME)), SAWITH1 AS (select distinct 0 as c1, D1.c3 as c2, D1.c4 as c3, D1.c5 as c4, D1.c2 as c5, D1.c1 as c6 from SAWITH0 D1) select D1.c1 as c1, D1.c2 as c2, D1.c3 as c3, D1.c4 as c4, D1.c5 as c5, D1.c6 as c6, D1.c7 as c7, D1.c8 as c8 from ( select D1.c1 as c1, D1.c2 as c2, D1.c3 as c3, D1.c4 as c4, D1.c5 as c5, D1.c6 as c6, sum(D1.c5) over (partition by D1.c3) as c7, sum(D1.c6) over (partition by D1.c3) as c8 from SAWITH1 D1 order by c1, c7 desc, c3 ) D1 where rownum <= 65001 4
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
ETL Development: Finished Product
85
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
ETL Development: Finished Product
86
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
ETL Development: Finished Product
87
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
ETL Development: Finished Product
88
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
ETL Development: Finished Product
89
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics 90
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics 91
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Build Now, Refactor Later
92
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Build Now, Refactor Later
93
Front-end developers
aggressively close stories any
way they can
Complex calculations,
direct SQL, BI Publisher, Custom
JavaScript: Nothing is off
limits
A new “Refactor” Story is opened for the Metadata
Developer to consider this content for
inclusion in the RPD
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Build Now, Refactor Later
94
Metadata developer decides if the complex
front-end content is viable
If viable, the developer aggressively
solves the Refactor Story using complex
calculations or opaque views
The original Refactor Story is reassigned to
the front-end developer to have the analysis point instead to the new metadata
A new Refactor Story is opened for the ETL
developer for any complex
transformations that may belong in the ETL
layer
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Build Now, Refactor Later
95
ETL Developer decides if the
transformations are viable, and if so, adds new transformations
The Refactor Story is then
reassigned to the Metadata
developer to rework the RPD
using new database attributes
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Build Now, Refactor Later
96
Where did our Real-Time go?
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Realtime Integration
99
ETL added latency Pure batch: significant Micro batch: medium
We lost our Realtime capability when we added our Access & Performance
layer
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Realtime Integration
101
ETL added latency Pure batch: significant Micro batch: medium
We need to a way to get that back utilizing the
Foundation layer
OBIEE Fragmentation
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Realtime: Foundational Federation
102
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Realtime: Foundational Federation
103
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Realtime: Foundational Federation
104
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Realtime: Foundational Federation
105
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Realtime: Foundational Federation
106
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Realtime: Foundational Federation
107
WITH SAWITH0 AS (select T8974.OPPORTUNITY_COUNT as c1, T8974.AMOUNT as c2, T8939.INDUSTRY as c3, T8981.LEAD_SOURCE as c4, T8981.NAME as c5, T8981.OPPORTUNITY_ID as c6 from EDW.OPPORTUNITY_DIM T8981 /* Opportunity Dim (Standard) */ , EDW.ACCOUNT_DIM T8939 /* Account Dim (Standard) */ , EDW.OPPORTUNITIES_FACT T8974 /* Opportunities Fact (Standard) */ where ( T8939.ACCOUNT_KEY = T8974.ACCOUNT_KEY and T8939.INDUSTRY = 'Technology' and T8974.OPPORTUNITY_KEY = T8981.OPPORTUNITY_KEY and T8981.LEAD_SOURCE = 'Conference' and 'EDW' = 'Foundation' ) ), SAWITH1 AS (select cast(1 as INTEGER ) as c1, T2196.AMOUNT as c2, T2041.INDUSTRY as c3, T2196.ID as c4, T2196.LEAD_SOURCE as c5, T2196.NAME as c6 from SUGARFND.ACCOUNTS T2041 /* Accounts (Standard) */ , SUGARFND.OPPORTUNITIES T2196 /* Opportunties (Standard) */ , SUGARFND.ACCOUNTS_OPPORTUNITIES T2221 /* Accounts Opportunities (Standard) */ where ( T2041.ID = T2221.ACCOUNT_ID and T2196.ID = T2221.OPPORTUNITY_ID and T2196.LEAD_SOURCE = 'Conference' and case T2041.INDUSTRY when ' ' then 'Not Applicable' else T2041.INDUSTRY end = 'Technology' and TO_DATE('2013-08-15 00:00:00' , 'YYYY-MM-DD HH24:MI:SS') < T2196.DATE_ENTERED ) ), !(continued...)
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Realtime: Foundational Federation
108
WITH SAWITH0 AS (select T8974.OPPORTUNITY_COUNT as c1, T8974.AMOUNT as c2, T8939.INDUSTRY as c3, T8981.LEAD_SOURCE as c4, T8981.NAME as c5, T8981.OPPORTUNITY_ID as c6 from EDW.OPPORTUNITY_DIM T8981 /* Opportunity Dim (Standard) */ , EDW.ACCOUNT_DIM T8939 /* Account Dim (Standard) */ , EDW.OPPORTUNITIES_FACT T8974 /* Opportunities Fact (Standard) */ where ( T8939.ACCOUNT_KEY = T8974.ACCOUNT_KEY and T8939.INDUSTRY = 'Technology' and T8974.OPPORTUNITY_KEY = T8981.OPPORTUNITY_KEY and T8981.LEAD_SOURCE = 'Conference' and 'EDW' = 'Foundation' ) ), SAWITH1 AS (select cast(1 as INTEGER ) as c1, T2196.AMOUNT as c2, T2041.INDUSTRY as c3, T2196.ID as c4, T2196.LEAD_SOURCE as c5, T2196.NAME as c6 from SUGARFND.ACCOUNTS T2041 /* Accounts (Standard) */ , SUGARFND.OPPORTUNITIES T2196 /* Opportunties (Standard) */ , SUGARFND.ACCOUNTS_OPPORTUNITIES T2221 /* Accounts Opportunities (Standard) */ where ( T2041.ID = T2221.ACCOUNT_ID and T2196.ID = T2221.OPPORTUNITY_ID and T2196.LEAD_SOURCE = 'Conference' and case T2041.INDUSTRY when ' ' then 'Not Applicable' else T2041.INDUSTRY end = 'Technology' and TO_DATE('2013-08-15 00:00:00' , 'YYYY-MM-DD HH24:MI:SS') < T2196.DATE_ENTERED ) ), !(continued...)
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Realtime: Foundational Federation
108
WITH SAWITH0 AS (select T8974.OPPORTUNITY_COUNT as c1, T8974.AMOUNT as c2, T8939.INDUSTRY as c3, T8981.LEAD_SOURCE as c4, T8981.NAME as c5, T8981.OPPORTUNITY_ID as c6 from EDW.OPPORTUNITY_DIM T8981 /* Opportunity Dim (Standard) */ , EDW.ACCOUNT_DIM T8939 /* Account Dim (Standard) */ , EDW.OPPORTUNITIES_FACT T8974 /* Opportunities Fact (Standard) */ where ( T8939.ACCOUNT_KEY = T8974.ACCOUNT_KEY and T8939.INDUSTRY = 'Technology' and T8974.OPPORTUNITY_KEY = T8981.OPPORTUNITY_KEY and T8981.LEAD_SOURCE = 'Conference' and 'EDW' = 'Foundation' ) ), SAWITH1 AS (select cast(1 as INTEGER ) as c1, T2196.AMOUNT as c2, T2041.INDUSTRY as c3, T2196.ID as c4, T2196.LEAD_SOURCE as c5, T2196.NAME as c6 from SUGARFND.ACCOUNTS T2041 /* Accounts (Standard) */ , SUGARFND.OPPORTUNITIES T2196 /* Opportunties (Standard) */ , SUGARFND.ACCOUNTS_OPPORTUNITIES T2221 /* Accounts Opportunities (Standard) */ where ( T2041.ID = T2221.ACCOUNT_ID and T2196.ID = T2221.OPPORTUNITY_ID and T2196.LEAD_SOURCE = 'Conference' and case T2041.INDUSTRY when ' ' then 'Not Applicable' else T2041.INDUSTRY end = 'Technology' and TO_DATE('2013-08-15 00:00:00' , 'YYYY-MM-DD HH24:MI:SS') < T2196.DATE_ENTERED ) ), !(continued...)
SAWITH2 AS ((select D0.c3 as c3, D0.c4 as c4, D0.c5 as c5, 'EDW' as c6, D0.c6 as c7, D0.c1 as c8, D0.c2 as c9 from SAWITH0 D0 union all select case D0.c3 when ' ' then 'Not Applicable' else D0.c3 end as c3, D0.c5 as c4, D0.c6 as c5, 'Foundation' as c6, D0.c4 as c7, D0.c1 as c8, D0.c2 as c9 from SAWITH1 D0)), SAWITH3 AS (select sum(D3.c8) as c1, sum(D3.c9) as c2, D3.c3 as c3, D3.c4 as c4, D3.c5 as c5, D3.c6 as c6, D3.c7 as c7 from SAWITH2 D3 !(continued...)
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
Realtime: Foundational Federation
109
WITH SAWITH0 AS (select T8974.OPPORTUNITY_COUNT as c1, T8974.AMOUNT as c2, T8939.INDUSTRY as c3, T8981.LEAD_SOURCE as c4, T8981.NAME as c5, T8981.OPPORTUNITY_ID as c6 from EDW.OPPORTUNITY_DIM T8981 /* Opportunity Dim (Standard) */ , EDW.ACCOUNT_DIM T8939 /* Account Dim (Standard) */ , EDW.OPPORTUNITIES_FACT T8974 /* Opportunities Fact (Standard) */ where ( T8939.ACCOUNT_KEY = T8974.ACCOUNT_KEY and T8939.INDUSTRY = 'Technology' and T8974.OPPORTUNITY_KEY = T8981.OPPORTUNITY_KEY and T8981.LEAD_SOURCE = 'Conference' and 'EDW' = 'Foundation' ) ), SAWITH1 AS (select cast(1 as INTEGER ) as c1, T2196.AMOUNT as c2, T2041.INDUSTRY as c3, T2196.ID as c4, T2196.LEAD_SOURCE as c5, T2196.NAME as c6 from SUGARFND.ACCOUNTS T2041 /* Accounts (Standard) */ , SUGARFND.OPPORTUNITIES T2196 /* Opportunties (Standard) */ , SUGARFND.ACCOUNTS_OPPORTUNITIES T2221 /* Accounts Opportunities (Standard) */ where ( T2041.ID = T2221.ACCOUNT_ID and T2196.ID = T2221.OPPORTUNITY_ID and T2196.LEAD_SOURCE = 'Conference' and case T2041.INDUSTRY when ' ' then 'Not Applicable' else T2041.INDUSTRY end = 'Technology' and TO_DATE('2013-08-15 00:00:00' , 'YYYY-MM-DD HH24:MI:SS') < T2196.DATE_ENTERED ) ), !(continued...)
SAWITH2 AS ((select D0.c3 as c3, D0.c4 as c4, D0.c5 as c5, 'EDW' as c6, D0.c6 as c7, D0.c1 as c8, D0.c2 as c9 from SAWITH0 D0 union all select case D0.c3 when ' ' then 'Not Applicable' else D0.c3 end as c3, D0.c5 as c4, D0.c6 as c5, 'Foundation' as c6, D0.c4 as c7, D0.c1 as c8, D0.c2 as c9 from SAWITH1 D0)), SAWITH3 AS (select sum(D3.c8) as c1, sum(D3.c9) as c2, D3.c3 as c3, D3.c4 as c4, D3.c5 as c5, D3.c6 as c6, D3.c7 as c7 from SAWITH2 D3 !(continued...)
BI Server does a union between the
Foundation and Performance layers
Union is eliminated when the
“fragmentation column” is used as a
filter
www.redpillanalytics.com [email protected] @RedPillA © 2014 RED PILL Analytics
AgileAnalytics: Real-Time BI with Oracle Business Intelligence, Oracle
Data Integrator and Oracle GoldenGate
110
Stewart Bryson
medium.com/@stewartbryson @stewartbryson linkedin.com/in/stewartbryson