agile business intelligence
DESCRIPTION
TRANSCRIPT
www.iccohio.com
Agile Methods – A New Model
Less Time, Less Cost,Way
Better Results!
Donald JacksonVice President
www.iccohio.com
Agile BI Agenda
A Business Imperative Big Data Leverage Mobile platforms Deliver Analytics faster
Methodology Maturity Model
Agile Methodology
Information Factory Onshore resources at Offshore price; better results
www.iccohio.com
Business Intelligence
ICC applies Agile principles to Business
Intelligence development to cost-
effectively deliver BI solutions that drive
business strategies.
• Leader in Agile Business Intelligence• Leader in Information Factory Services• Leader in Mobile Business Intelligence
ICC proprietary and confidential 3
www.iccohio.com 4
DataArchitecture &
Governance Practice
• Requirements Gathering• Proposal Writing• Metadata• Data Modeling• BPM & CRM
• Requirements Gathering• Proposal Writing• Metadata• Data Quality
• Data Modeling & Architecture• Metadata• Governance• Master Data Mgmt• BPM• Change Mgmt• Database Development• Data Quality
Data Warehouse Architect
Metadata Specialist
Data Architect
Data Modeler
Roles
DataIntegration
Practice
• Process Modeling• ETL / EAI Methods• Data Integration• Data Acquisition• Balancing Controls• SQL Programming• Electronic Data Interchange• Object Oriented Programming• Quality Engineering• XML & Script Programming
Integration Architect
EAI Specialist
ETL Specialist
InformationDeliveryPractice
• Requirements Definition• Business Modeling• Prototyping• EII Methods• OLAP / Reporting• JAD Facilitation• Project Management• Data Mining• BI Tool Implementation
Information Delivery Architect
ID Specialist
Business Analysts
Skills
Project Manager
ICC Business Intelligence Group
www.iccohio.com
Agile BI – Data Tsunami Big Data
Increased user expectations for more detailed analyses for longer periods
Data redundancy
A 'just in case' mentality, fueled by concerns over corporate compliance
Unstructured data from the Internet
7ICC proprietary and confidential
www.iccohio.com
Agile BI Mobile
Provide your data in context, anytime, anywhere
www.iccohio.com
Agile BI ROI
By delivering smaller and more frequent iterations, reusable code is used and the ROI can be seen quickly.
www.iccohio.com
Agile BI ROI
Sprint 2 Sprint 3 Sprint 40.00
0.50
1.00
1.50
2.00
2.50
3.00
3.50
4.00
2.31
3.00
3.82
ETL Jobs per Day
Jobs per Day
Productivity increased by 66% from Sprint 2 to Sprint 4
Sprint 2 Sprint 3 Sprint 40%
10%
20%
30%
40%
50%
60%
70%
0%
30%
66%
Efficiency
Efficiency
www.iccohio.com
Agile BI Discussion
Which came first:
the Chicken or the Egg?
The answer: they are interrelated and codependent
A parallel question in business would be:
Operational Systems
Operational Data
www.iccohio.com
Agile BI Discussion
Which comes first:- the business vision or data architecture?
Business Vision
Sales Analysis Customer Service Analysis Dimensional P&L Marketing Segmentation Single View of Customer
Data Architecture
Data Strategy Data Governance Data Inventory Data Quality Data Warehouse
The truth: they are interrelated and codependent
www.iccohio.com
Agile BI As-Is Process
The business is busy
We have got to start somewhere…
So let’s look at the transactional systems, the source data and our ETL processes
Operational Systems
Operational Data
ETL
www.iccohio.com
Agile BI Standard Models
Complex and focus primarily on building the data warehouse
www.iccohio.com
Status QuoAgile BI
Manual, Paper Based, Slow, Error Prone
Automated, Online, Reusable, Accessible, Governed, Self-documented delivered in weeks!
Focus on Purpose• Who – is the intended audience for using
the dashboards and reports?
• How – will the information be used?
• What – are the expected benefits?
• When – is the information available for use?
• Where – will the decisions be made from?
• Why – will the users want to use a new interface?
www.iccohio.com
MethodAgile BI
www.iccohio.com
FeaturesAgile BI
Business-Driven Approach Adapt to Changing Market Conditions Align with Flexible Business Initiatives Ability to Minimize Risk
– Prototype then build– Limit Scope– Deliver in 10 – 12 Week Cycles– Employ Iterative and Incremental
Methods
www.iccohio.com
Agile BI Model
Monitor
Analyze
Drill to Detail
Dashboards
OLAP
Detail Reports and Queries
Su
mm
ari
zati
on
www.iccohio.com
Agile BI Business Value
www.iccohio.com
Information Delivery Model
Data Repository Information Delivery
Dat
a T
rans
form
atio
nSource Systems
DataWarehouse
Dimensional Models /
Data Marts /
OLAP Cubes
Business Information Portals
Sales +GP +++OI --+- AAAA 00.25
CCCC 005
Plane Hrs Until15AA 25 .2515BB 35 .4222X2 100 .4522X 0 0
Details:15AA – 255 – 35 XX22X – 255 – 25 RLS
Requirements Driven from the BusinessQuality Content built from Source Systems
Consistent Data Meaning Throughout
www.iccohio.com
Agile BI As-Is Process
Inefficient, labor intensive process without automation
Analysis & Reporting
Data Models
Data Movement & Quality
SourceData
DeployTestBuildDesignAnalysisRequirements
Wh
at is b
ein
g b
uilt?
How is it being built?
Entity RelationshipDiagram
in ERWinby Data Modeler
ETL Modules
in PowerCenterby ETL Developer
Use Case
in Excel or Wordby Business Analyst
Manual rekeying?
Manual rekeying?
Business Object Definition
in Excel or Wordby Solution Architect
Manual rekeying?
Cube
by OLAP Developer
Manual rekeying?Tables
in SQLby DBA
ETL Test Scripts
in Excel orQuality Center
Manual rekeying?
Manual rekeying?
Reports
by Report Developer
In SQLBy DBA
User Documentation
by Business Analyst
ETL DocumentationIn Word or Visio
Operations Documentation in Word
Manual rekeying?
Manual rekeying?
www.iccohio.com
Agile BI Accelerator
Analysis & Reporting
Data Models
Data Movement & Quality
SourceData
DeployTestBuildDesignAnalysisRequirements
Wh
at is b
ein
g b
uilt?
How is it being built?
Entity RelationshipDiagram
in ERWinby Data Modeler
ETL Modules
in PowerCenterby ETL Developer
Use Case
in Excel or Wordby Business Analyst
Manual rekeying?
Manual rekeying?
Business Object Definition
in Excel or Wordby Solution Architect
Manual rekeying?
Cube
by OLAP Developer
Manual rekeying?Tables
in SQLby DBA
ETL Test Scripts
in Excel orQuality Center
Manual rekeying?
Manual rekeying?
Reports
by Report Developer
In SQLBy DBA
User Documentation
by Business Analyst
ETL DocumentationIn Word or Visio
Operations Documentation in Word
Manual rekeying?
Manual rekeying?
ICC
Business ObjectsCognos
MicroSoftMicroStrategy
OBIEE
DataStageInformatica
SSIS
Auto-generated
Auto-generated
www.iccohio.com
Agile BI Path Forward
Build a prototypeTalk to the customer Document their business Develop a menu of services
Cook to order Leverage model-driven architecture Maximize reusable components Deliver new stuff every 12 weeks
How would you like your eggs?
Over easy, Scrambled or Poached?
www.iccohio.com
Agile BI Enterprise
Finance
• Income Statement• Drill-down Variance• Operational Plan Variance • Cash Flow/Working Capital• Balance Sheet• CapEx/Strategic Investments• Treasury
Marketing
• Market Opportunities• Competitive Positioning• Product Life Cycle Management• Pricing• Demand Generation
Sales
• Sales Results• Customer/Product Profitability• Sales Tactics• Sales Pipeline• Sales Plan Variance
Customer Service
• On-Time Delivery• Information, Complaint, and Claims• Service Benchmarks• Service Value
Product Development
• Product & Portfolio Innovation• Product Development Milestones• Market & Customer Feedback
Operations
• Purchasing and Procurement • Production and Capacity• Inventory Management• Distribution and Logistics• Cost & Quality Management• Process Efficiency
Human Resources
• Organization and Staffing• Compensation• Talent and Succession• Training and Development• Benefits
Information Technology
• Business Value Map• IT Portfolio Management• Project/SDLC Management• IT Vendor Management• IT Compliance Management
Executive Management
• Financial Performance • Risk Management • Compliance Management
Shared Data – Conformed Dimensions
www.iccohio.com
Agile BI Method
We have a receivables problem approaching $50M. - CEO
Do we know if it is isolated to an operating company? … or a service area?- COO
Is it getting worse over time? What are the reasons customers are not paying?- VP Customer Service
www.iccohio.com
Agile BI Value Chain
www.iccohio.com
Agile BI Method
Interview Notes
BusinessQuestions
BusinessTerms
Dimensions andMeasures
OLAP Model(s)
SMEs
BISolution
www.iccohio.com
Agile BI Method
We have a
receivables problem approaching $50M. - CEO
Do we know if it is isolated to
an operating company? … or a
service area?- COO
Is it getting worse over
time? What are the
reasons customers are not paying?- VP Customer Service
www.iccohio.com
Agile BI Maturity Model
www.iccohio.com
Agile BI 3 Perspectives
ProjectDevelopment
Lifecycle
Budget
Control
Manage Resources
Monitor Progress
Create Deliverables
Implement Solutions
Release Backlog(Themes/Goals)
Product Backlog(Stories/
Questions)
Sprint Backlog(Story Features/
Deliverables &WPs)
Perspective
Focus
Agile/Scrum
1 to 3 days Sprint 02 to 4 weeks
Sprints 1 - n10 to 12 week releases
(5 to 6 two-week Sprints)
Duration
ToolsBI / Reporting
Consensus
DBMS
Data Profiling
Story Generation Utility
ETL
BI / Reporting
Data Modeling
Data Quality
DBMS
Testing
Story Management Utility
Business DefinitionsPrototype
Business TermsBusiness Model
Data ProfilesStory Cards
Sprint Roadmap
Databases / Cubes
ETL
Dashboards & Reports
High Level Estimate
Release Roadmap
Estimating Model
Facts Qualifier Matrix (FQM)
FQM to Source Matrix
Business Goals / Objectives
Preliminary DQ Assessment
Executive
Outcomes11
www.iccohio.com
Agile BI Maturity Model
1Business
Case Assessment
2Enterprise
Infrastructure
Evaluation
3Project
Planning
Justification Initiate
Solution ScopingDesign
Development
Implement
15Implementati
on
16Project
Evaluation
Sprint Release
Proceed Gate
Final AcceptGate
Close Gate
Begin Gate
Define
Commit Accept
Mini-Gates within each Sprint
Iteration 1- n
Product Backlog
Time to Value: 12 – 16 weeks
www.iccohio.com
Agile BI Maturity Model
Maturity Levels
Key Process Areas
Goals
The Maturity Model involves the following aspects:
Maturity Levels: A 5-Level process maturity continuum - where the uppermost (5th) level is a notional ideal state where processes would be systematically managed by a combination of process optimization and continuous process improvement.
Key Process Areas: A Key Process Area (KPA) identifies a cluster of related activities that, when performed collectively, achieve a set of goals considered important.
Goals: The goals of a key process area summarize the states that must exist for that key process area to have been implemented in an effective and lasting way. The extent to which the goals have been accomplished is an indicator of how much capability the organization has established at that maturity level. The goals signify the scope, boundaries, and intent of each key process area.
www.iccohio.com
Agile BI Maturity Model
1
Defined
2
Architected
3
Implemented
4
Integrated
5Optimized
(Continuous)
ABI Maturity Levels
People
Process
Platform
RolesResponsibilitiesCompetencies
AlignmentDynamicsMeasures
FormationExecutionResults
Cross FunctionalCohesiveAccountable
GrowthLearningImprovement
GoalsDeliverablesMeasurements
WorkflowsArtifactsSignoffs
ReleaseSprintsAcceptance
StreamlinedProductionSupport
LeanCost EffectiveValuable
ToolTechniqueFramework
PatternsTemplatesStandards
SharedConnectedConformed
ReusableConfigurableMaximized
InstalledMaintainedSupported
www.iccohio.com
Agile BI Productivity
Quality
Speed
People
Platform
Process
0Current State
1Defined
2Architected
3Implemented
4Integrated
5Optimized
Combined
Velocity
www.iccohio.com
Questions?
35
www.iccohio.com
Information Factory Features
Data Integration
EAI
ERP
ESB
ETL
MDM
Conversions
Migrations
Job creation in the USAo Located in the Midwesto Increased visibility into the project statuso Ability to adapt to changing project priorities
High Performance Teams ensures qualityo Senior industry veterans guiding the processo Deliverable focusedo Consistent project standards and documentationo 0% defect guarantee
Cost effectiveo Outcome based pricingo On-time delivery
www.iccohio.com
Information Factory References
Ohio's Largest IT Service Firm Hits Gold with Agile Data Warehousing(TM) Method
Information Control Corporation (ICC), Ohio's largest privately-owned IT services company, recently doubled its development speed and nearly eliminated programming defects for its business intelligence services division by converting its data warehousing operations to Scrum-based methodology, as outlined in Ceregenic's book “Agile Data Warehousing.”
Ceregenics', Ralph Hughes, MA, PMP, CSM can be contacted at [email protected] or via www.ceregenics.com.
3737
“Within 2 distinct projects, ICC consistently has displayed technical creativity, willingness to go the extra mile, and a true commitment to ensuring that CAH reaches the project
goals successfully, on time and within budget. ICC has matured in their knowledge of CAH EIT and shown an ever-improving ability to navigate the organization and provide
effective communication. “
Liddy Heath Director, Core R/3 Applications, Enterprise IT
“I highly recommend the ICC Information Factory Process for projects requiring quick ramp up and additional staffing.”
Michael Tripp – Sr. Manager IT Abercrombie & Fitch
“ICC has been a very valuable partner in helping us define industry trends and future needs. Additionally, they have provided opportunities for our faculty to learn about new and immerging areas by welcoming us to their facility and by encouraging our involvement with industry groups involved in areas like Business Intelligence in the greater Columbus area.”
Thom Luce – Professor & Chair MIS, Ohio University
“The Information Factory development approach that ICC utilizes is one of the best I have ever come across. They are the only consulting firm that I have used that have truly been able to put in practice a systemic, Agile project approach for Business Intelligence and ETL development.”
Jerry Bisaha – Director BI Limited Brands
www.iccohio.com
Information Factory Success
Company Project Technologies UsedIDP1 IDP2 DAP1 DAP2 DIP1 DIP2
Abbott Nutrition Customer Profitability Data mart
Consensus Cognos Informatica
Abbott Nutrition Oracle to Cognos Report Conversion
Consensus Cognos
AEP SDLC Review and Emissions POC
SSRS SQL Server SSAS SSIS
Affinion Group Shopping Data Mart - Actuate Replacement
Cognos 8 Oracle 9 Informatica
Grange Insurance MicroStrategy vs. Microsoft Assessment
Microsoft BI Microsoft BI
LimitedBrands Financial Data Warehouse MicroStrategy Teradata Informatica
Limited Stores MicroStrategy BI Assessment
MicroStrategy
Mettler Tolodo Field Services Analysis Prototype
Microsoft BI Consensus SQL Server SSAS
Nationwide Childrens Hospital
Grant Summary Prototype Microsoft BI Consensus SQL Server
Nationwide Insurance
GSRM Galaxy Data Warehouse Dashboards
MicroStrategy Teradata
NetJets Business Objects Migration and Data Warehouse
Consensus Business Objects
Oracle Erwin BODI
OSF Financial Reports Cognos
OPERS Government Relations Data Mart
Consenus Performance Point
SQL Server SSAS SSIS
PMI Business Objects
SSRS SQL Server
A&F DataStage / QualityStage development with ICC Information Factory
DataStage QualityStage
Affinion Group Membership Analytical Cubes
Cognos 8 PowerPlay
Victoria's Secret Direct
Financial Data Warehouse MicroStrategy Teradata
www.iccohio.com
Agile BI Productivity
“The secret of getting ahead is getting started. The secret of getting started is breaking your complex overwhelming tasks into small manageable tasks, and then starting on the first one.”
…Mark
Twain
www.iccohio.com
Information Factory Scrum
Scrum is based on a "Sprint," which is a 30-day period focused on a specific deliverable.
1.Product backlog is determined
2.Backlog is prioritized
3.10 day sprint is commenced
4.Daily standup meetings are directed by the Scrum Master
5.Product is delivered
www.iccohio.com
Information Factory Team
Product Owner: responsible for the features of the product to be worked on, as well as the release date. They are responsible for prioritizing the product work backlog.
Scrum Master: acts as the project manager, and is responsible for shielding the team from outside interferences. They also lead task assignment and daily stand up meetings with the team.
Scrum Team: consistent group of people with core BI experience in DataStage, Informatica and other tools. The team is self organizing and tasks are assigned based on availability and capability.
Product Owner(Client)
Scrum Master
Developers Sr. QA AnalystArchitect
The IF Scrum Team
www.iccohio.com
Agile BI Productivity
www.iccohio.com
Information Factory Offshore Math
www.iccohio.com
Information Factory Math Problem
Rate ≠ PricePrice = Rate x Efficiency + Overhead
Rate Calculate total team avg. hourly cost including
expenses
Efficiency = Velocity * QualityScrum/Agile Methodology Auto-generated codeLibrary of reusable components
Quality Test first development Focus on reducing the number of defects or re-
work
Overhead On-site Project Coordinator, Business Analyst
www.iccohio.com
Information Factory What IF?
“A collective ofintelligent mindsfunctioning as onecohesive unit”
“Information Factory; professional end-to-end delivery of exceptional data integration through nimble, adaptive practices at extremely competitive rates”
“Robust and flexible processesimplemented in a cost efficient andhighly optimized factory like setting.”
“IF differentiates itself from legacy development
processes and offshore solutions due to our
accountability, quality and delivery times.
The fact that IF is located onshore means face to
face checkpoints, quality levels that have never been
seen before, we have the results to prove it, and
delivery times that match or exceed any other
proposed solution.”
www.iccohio.com
Questions?
46