seeking the agile path through database design
DESCRIPTION
Being first to market or meeting rapidly changing customer demands compels development teams to build systems while requirements are still being discovered. Developing a relational database design ahead of its requirements can paint you into a corner—with a product that suffers from legacy-like limitations. Jonathan Wiggs shares ideas to solve this problem in an agile way that provides both support for the present and flexibility for the unknown future. When a database design is trying to meet too many requirements, you can tie its architecture to product concepts and get answers to questions such as one database or many? Learn to balance design flexibility with operational requirements including understanding the role of disk storage and physical implementation across various technologies from the desktop to the cloud. Jonathan covers specifics including the proper use of minor entity tables, denormalization, and elastic scaling mechanisms. Join Jonathan to see how your data architecture can lead innovation rather than drag behind as an obstacle to success.TRANSCRIPT
AW3 Session 6/5/2013 10:15 AM
"Seeking the Agile Path Through Database Design"
Presented by:
Jonathan Wiggs Netmotion Wireless, Inc.
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073 888‐268‐8770 ∙ 904‐278‐0524 ∙ [email protected] ∙ www.sqe.com
Jonathan Wiggs Netmotion Wireless, Inc.
The data architect for Netmotion Wireless, Inc., Jonathan Wiggs is an accomplished software architect with significant experience in the fields of big data, Bayesian analytics, enterprise architecture, and cloud computing. Jonathan has helped launch startup companies including Jott Networks & RGB Labs, and has led engineering and research groups at companies such as Microsoft and Nuance. He enjoys writing, speaking, sharing his experiences with his peers, and giving back to the industry he has loved for more than twenty years. Jonathan lives in the country and spends his free time with his wife and several admirable dogs. Contact Jonathan at [email protected].
SEEKING THE AGILE PATH THROUGH DATABASE DESIGNDATABASE DESIGN
Jonathan WiggsNetmotion Wireless [email protected]
Agile Development Conference WestJune 2013
Not shipping a new product with a legacy database.
The Goals
Preventing being locked into a rigid data design that acts as an obstacle for nimble and Agile change.Establishing an architect role for a new way of d i g thi gdoing things.Avoiding siloing of data that prevents innovation and bringing new products to market.
Culture of the Cog.Assumption of rough skill set parity.
Agile & XP (Cogs and Bogs)
p g p yThe occasional fallacy of the single team product.The architect as technical owner and engineering peer. Mixing the need to bring design consistency to a product with tactical execution. Leading design by example.Cross team technical coordination.
How does the architect fit into the Agile process / Scrum?
The Role of Technical Ownership
If the Scrum Master removes process roadblocks the architect should help remove technical roadblocks.The architect as the technical owner, design
g li t d t d d hevangelist, and standards coach.A good idea isn’t always the right idea for the team or the product.
The dangers of product incrementalism.
Sprinting Your Way to a Legacy Product
Bite sized changes require both product vision as well as design vision.As design is ignored with tactical changes, code becomes more complex and difficult to change.Changes and bug fixes become more and more Changes and bug fixes become more and more expensive as the project progresses.
Data analytics and data aggregation.
The State of the World & Its Data
Products and infrastructure are becoming commodities while large scale data ownership is now vital to product differentiation.Target Corporation, data, Agile, and an angry fatherfather.
Can Agile Drive Data Security?
Why is cross team technical and architectural coordination important?coordination important?Preventing the siloing of data and missed opportunities. This is especially true when bringing new products to market.The lessons of the Netflix Prize.
(Private lawsuits, FTC investigation, all from zip code, age, and gender lead to disambiguation. This plus birthdate identifies 87% of Americans)
Is the process or the design behind the wheel?
Does the Process Drive the Design
Keeping the eye on the ball, process should improve execution in all of its various meanings.Avoiding philosophical arguments about the process & keeping focus on the end goalprocess & keeping focus on the end goal.
Nimble practices in relational design.
How Does Database Design Fit?
Avoiding having databases serve more than one master.Continuous integration and testing carrying through to the data layer.Carry the testing of the database and data layer Carry the testing of the database and data layer through to the unit test framework.
Isolation of the data access layer.
Is the Database Really Important?
Do not encode business rules in the underlying data system.How well will your product scale?Considering multi-tenancy and cloud based productproduct.Amazon RDS & Azure SQL Server.
Enumerated data types of the database world.
The Use of Minor Entity Tables
Flexibility & performance!
Time zones and standardizing on GMT.
Ensuring the Capability for Localization
Use of the proper data types.Character sets and their implementation varies greatly from database system to system.Always use proper ISO codes when mixing to ensure support for regional and countryensure support for regional and country.
Normalized Forms.Is Normalization and De Normalization a myth?
Normalization versus Over‐normalization
Is Normalization and De-Normalization a myth?Functional based design as opposed to strict adherence to generic rules.Storage & performance impacts of the approaches.What purpose does the database serve and how can we stay Agile?
Why be so firm about this point?
Rigidly Abstracted Data Access Layer
Ever changing database technology.Business driven support for differing technology.Isolate performance modifications.L t f t hit t i t f Large part of system architecture in support of ACID.
The big pro to many teams is that object oriented database are instantly familiar to
We Can Just Use an Object Oriented DB
oriented database are instantly familiar to software engineers.Many solid open source offerings.Scale is often an issue, large scale implementations such as PostgreSQL at Yahoo
se c stom b ild implementationsuse custom build implementations.Object relational impedance mismatch.Not constructed for the job at hand.
Open source and horizontal scale.
Is a NoSQL System the Answer?
MapReduce is not magic.Easy to use without the need for schema design, which means no ACID.Little to no index support.D t hid i th d d i t i Data can hide in the edges and consistency is nearly impossible to manage.High memory consumption and rough around the edges.
The implementation of both process and technology should be based on the stage of life
Conclusions
technology should be based on the stage of life of both the company and the product.The role of the architect is by no means a dead role in the Agile world, but the goals have changed.Finall architect re & design sho ld and does Finally, architecture & design should and does make a huge impact on how Agile a product can be.
Questions & Answers