big data lab for the canadian government
TRANSCRIPT
magnara.ca
Big Data Laboratory
Creating a space for continuous improvement
START POINT GOAL
START POINT
GOAL
START POINT
GOAL
START POINT
GOAL
START POINT
GOAL
START POINT
GOAL
START POINT
GOAL
???
START POINT
GOAL
But wait…
GOAL
GOAL
GOAL
GOAL
NOBODY DOES.We don’t know.
Dev or Lab?
Development
• Known solutions• Clear deliverables• Fixed timelines• Standard management• Minimized failures• Known skills• All requirements met
Laboratory
• Unknown solutions• Lots of mistakes• Unpredictable completion• Managed risk• Controlled failures• New skills• Options eliminated
Big Data
• What is it… exactly?• Is it the answer?• Is it only part of the answer?• Is it a magic bullet?• Is it too hyped and immature?• Is it a pipe dream?• Is it a bubble ready to pop?• Will it meet the needs of the Canadian government?• When and how will we be ready?
Cloud + Hadoop
MapReduce
MapReduce
Hadoop
IT + Business
…and the two shall become one flesh.
Blueprint 2020
• “…envisions a capable and high-performing Public Service that embraces innovation, transformation and continuous renewal.”
• “An agile, efficient and effective Public Service is essential to the well-being of Canadians.”
• “…the world continues to change rapidly, putting significant stress on the existing way of doing things.”
GOAL
What to do about…
• Tableau Server• SAS VA
NEXT STEPS?If we need to really innovate, what are the…
Possible directions…
Traditional BI with a twist?
Something brand new?
We need to stop…
• Making promises• Buying tools• Hiring consultants• Developing reports• Answering questions• Repeating the same mistakes• Aiming only for success
We need to start…
• Re-asking old questions• Brainstorming new ideas• Experimenting in safe ways• Breaking the goal down differently• Defining lab boundaries• Leveraging lessons from others• Deliberating failure for maximum learning
Buyer Beware
“Organizations are scrambling to implement new software and hardware to increase the amount of data they collect and store. However, in doing so they are unwittingly making it harder to find the needles of useful information in the rapidly growing mounds of hay. If you don't know how to differentiate signals from noise, adding more noise only makes things worse.”
Signal: Understanding What Matters in a World of Noise
RESOURCES
7 TED Talks
• How to learn from mistakes– On being wrong– How to learn? From mistakes– There are no mistakes on the bandstand– The dangers of “willful blindness”– Doctors make mistakes. Can we talk about that?– Trial, error and the God complex– Embrace the near win
The Lean Startup
• Eric Ries talks at Google• TheLeanStartup.com• Build. Measure. Learn.• Running Lean
