agile and bi a perfect matchbi-podium.nl/mediafiles/upload/agile_bi/marco... · agile . the devils...
TRANSCRIPT
Agile and BI – a perfect match Agile BI congres | Marco Coopmans | 13 december 2012
?
What is Agile?
Why start working differently?
A software development project …
• Takes too long
• Is too expensive
• Does not deliver what is asked
• Does not deliver what is needed
The Agile Manifesto
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
[Agile Manifesto, Utah, 2001]
Values
The main goal of agile is:
Delivering as much value as possible for the given time and money
How?
• Only make realy needed functionality, the highest value first
• Eliminate waste (lean) – only added value
• Mitigate risks – show working, production feasible software
The goal of agile
The characteristics of agile vs. waterfall
The user knows what he needs
The developer knows how to build it
During the project nothing changes
Waterfall
The user discovers during the project what he needs
The developer discovers during the project how to build it
There is a lot of dynamics and changes during the project
Agile
The devils triangle - Waterfall vs. Agile
Waterfall Agile
Time
Money
Quality
Functionality
Functionality
Time
Money
Quality
Fixe
d
Var
iab
le
Fixe
d
Var
iab
le
Determ
ines
Determ
ines
Agile and architecture
See the curve coming, but don’t overdrive
your headlights
• Keep it simple – As simple as possible
• Eliminate waste – Don’t carry any waste as long as it is
not necessary
– Don’t invest (too much) in the future
• Postpone choices as long as possible – Do things only when they are really
needed
Some architectural agile principles
Agile is delivering working software as quickly as
possible (business value)
Be a minimalist
• Architecture is not determined at the beginning,
but “emerges” (form follows function)
• Determine a start architecture: a (minimal) set
of guide lines
• That gives a lean-and-mean
framework that
grows during development
Emerging architecture
Refactoring is a fact!
Agile is not short term solutions
To avoid a dead end and minimalize the impact of
refactoring you have to look ahead,…
but not too far!
An experienced
SCRUM team will not
let this happen!
SCRUM is the most applied form of agile
The SCRUM process
Grooming/Poker sessions
Sprint Backlog
Demo
Sprint (2 to 4 weeks)
Production feasible Product/software
Feedback, new input
Product Backlog
Priority Sprint planning
•Product Vision •Solution outline
(architecture, tooling) •Business Case / ROI
•Release planning (1st release = MVP)
Development
Daily
stand up
Re
leas
e 1
R
ele
ase
2
Re
leas
e x
R
ele
ase
..
SCRUM Roles and Organisation
Product Owner
Scrum Master
Stake Holders Scrum Team
Tune Tune
Supports
Project Team
Roles and Organization – Product Owner
Is a delegate of the business who has the mandate to determine what is on the product backlog and the prioritization of it
Holder of the product vision
Owner of the product backlog and user stories (requirements)
Represents all stake holders (business, IT, external, …)
Accepts or rejects the product of a sprint
Decides release date and content
Product Owner
Product Owner
Product Owner sits
in the driver seat
Roles and Organization – Scrum Team
A multidisciplinary team (designers, developers, testers, architects, …) – all expertise necessary to deliver a production feasible product
Design, develop and test the user stories
Is self-organizing, without hierarchy
Determines, together with the Product Owner, the content off a sprint and commits itself to it!
Scrum Team
Scrum Team
Scrum Team is self-
organizing
Roles and Organization – Scrum Master
Facilitates the SCRUM process of the team
Ensures optimal conditions for the productivity of the team
Solves impediments (blocking issues)
Is part of the team, … but is supporting (has no authority)
Scrum Master
Scrum Master
Scrum Master is a
facilitator
Roles and Organization – Stake Holders
Everybody who has an interest in the product (endusers, managers, security officers, architects, system administrators, controllers, supervisors, …)
Are being represented by the Product Owner
Stake Holders don’t have direct access to the Scrum Team! – takes place via the Product Owner
Stake Holders
Stake Holders
Agile and BI
Why start working differently?
A software development project …
• Takes too long
• Is too expensive
• Does not deliver what is asked
• Does not deliver what is needed
Does this also apply for BI projects?
According to Gartner research estimation is that
70 to 80% of BI projects fail. And the number
one reason cited for this disturbing rate of
debacle? Poor communication. (Patrick Meehan, President and Research Director at Gartner, 2011)
Agile BI is about tools?
Agile BI = Delivering functionality as soon as
possible
Agile BI = Self-service
Agile BI = Tools
It’s not only
about
delivering
data fast, it’s
also about the
right data and
quality
Agile BI is only suited for the front-end?
Back-end
Front-end
Waterfall
Agile
Only 20/30% would be agile!
70/80%
Agile BI is layer driven?
Sprint 1
Sprint 2
Sprint 3
Sprint 4
Sprint 5 First visible result in sprint 5
Fro
m s
ou
rce
to
info
rmat
ion
business value
Agile is delivering business value (each sprint)!
But how are you
gonna do this every
sprint, from the
beginning?
The minimal architecture
Choose a minimal set of components to achieve the requested functionality. This depends on tools, source systems, transformation, integration, …
2 ways - #1 Report driven (slicing)
A report is growing with
each sprint
This will need a few sprints to get a sustainable report to get efficient feedback
2 ways - #2 Data / information driven (refining)
Reports are refining /
maturing with each
sprint
The ‘Mona Lisa model’ – from raw data to information
Production Base
Dimensional model (Facts and Dimensions)
Initiate & Model Storming
Making accessible
Refine Raw
ETL
Universe(s)
Standard reports
Experienced Domainexperts
User documentation
System- & Operatingdocumentation
Trained key-users
Sprint … Sprint …
Sprint … Sprint …
Datastorage (DWH/DM/…)
Unlock as much as possible (raw)
data
Refine content: - Refine Dimensional
model - Refine Universe - Make basic
report(s) - Add business rules,
transformations, integrations, …
- Add history - ….
Refine to end users:
- Make final rapports - Add security/
authorization - Finalize metadata - Finalize
documentation - ….
Bring to Production
Set op a base version
(dimensional) information-
model
Confronting users with data as soon as
possible
Organisation – BICC and SCRUM team
IV IT-supplier Business
Business Intelligence Competence Center
Legacy Designer/Developer
Technical Consultant (dba, network, infra, etc)
ETL Designer/Developer
BI/Warehouse Consultant
BI Tool Specialist
Power User
Key-User
Proces/Data owner BICC manager
BI Business Modeler (BI Tool Specialist)
Enterprise WH Architect
BI Domain Expert
Management Reporting Authority
End-User
BI/ETL Designer/Developer
Production Guard
Service Coördinator
Project Team
Product owner
Scrum master
SCRUM Team
BI/ETL Designer/Developer
Enterprise WH Architect BI Domain Expert BI Business Modeler Developer/ Specialist Power User
Business IT
BI issues Agile
Why agile is so well suited for BI
Unclear information need Agile assumes that requirements are not clear and/or change, that is part of the approach. By a constant feedback requirements become clear.
Management / business involvement
Business is in the lead (Product Owner). Business is part of the team (domain experts and power users). Stake holders are being involved through demo’s, grooming sessions, sprint plannings.
Quality of data Quality of data is being tackled at the moment the issue arrises. The issue is made clear from a concrete information need and is better negotiable than.
Definitions Definitions are being established during development (by the business), when it is necessary, based upon real data/visible results.
Reliability of information Reliability is being guaranteed by the participation of domain experts and power users in the SCRUM team (not by IT). They also give the demo!
Organisation A SCRUM team fits realy good with a BICC. The business (power users, etc.) will grow into their role during the project.
Agile and BI are a perfect match!
Preconditions and Pitfalls
Preconditions - Culture
• Give confidence
• Learning organisation - making mistakes is allowed
• Transparancy - no judgement
• Entrepenearship and
creativitity
• Belief
Agile means a change of culture
Preconditions – Process en organisation
• Committed Product Owner, with
mandate
• Sufficient business (customer)
participation – Preferably Power Users
• IT Professionals (craftsmanship)
• (SCRUM) process is being
followed – meetings, stand-ups etc. are not for
discussion
The business sits in the driver seat
Preconditions – Technique
• Enough complexity
• Emergent architecture
• Continuous build and
deploy
• Automated testing
• Production(like) data available
• The SCRUM team determines the solution – no
one else
Pitfalls - Sprint
• No ‘closed’ sprint – New features, requirements added during
the sprint
• No real commitment for a sprint
• Laying the bar too high – Don’t do too much at the same time
• Laying the bar too low – Don’t be satisfied too easily
• Too much production-oriented – Communicating is production! It’s not
about the amount of code
– 10 to 30% preserved for grooming etc.
• No Definition of Done (DoD)
Pitfalls - Solution
• Holding on to standard
solutions, architectures – New techniques provide many
opportunities, try them
• Change to generic / “flexible”
solutions too quickly – Generic datamodels, Datavault, …
• Too little focus on quality – short
term solutions
Pitfalls - Leadership
• Agile is not about working late
(on the contrary)
• Fall back on classical
management – The Team is self-organizing, even at “bad
weather”
It’s not about working harder
but working smarter
Pitfalls – Client/Business
• No clear business objective, lack
of product vision – Why are we doing this? What are we trying
to achieve?
• Insufficient coordination with
stakeholders (by the Product
Owner) – Lack of support
Conclusion
BI and Agile are a perfect match,
but ….
It’s a difficult game to play
Thank you
Marco Coopmans [email protected] +31 6 11916413