agile database techniques tim dawson international decision systems © 2006 international decision...
Post on 06-Jan-2018
217 Views
Preview:
DESCRIPTION
TRANSCRIPT
Agile Database Techniques
Tim DawsonInternational Decision Systems
© 2006 International Decision Systems, Inc.
Speaker Background Chief Architect, International Decision Systems
object oriented development with relational databases for 15 years custom software development & consulting – 9 years independent software vendors – 6 years
IDS is an enterprise software vendor, not a tools vendor I’m not here to sell you anything! These items are tools and tricks I have developed over the years
Agenda Why do we need Agile Database Techniques? Common Best Practices Example Problem Approaches & Tools That Can Help
Why Do We Need Agile? “Shouldn’t we invest the analysis and design time to create the
proper data model upfront and get it right the first time?” Do your best, but don’t expect to get it perfect from the start – even
if you do, business needs change. It’s inevitable, so embrace it.
“But change is hard – I can live with that old table/column name.” You can for now, but just like bad code, eventually this stuff
catches up with you in increased maintenance costs. It used to be hard with Java too… Its not as hard as it looks - even on multi-application databases.
Typical Problems Developers need to make data model changes without
impacting other team members. Race Condition at Check-in Time
Need to be able to make changes to databases involving multiple applications. Scott Ambler’s book has some great ideas about this
DBAs have to maintain both “Upgrade” and “Create” scripts Does ERWin help? Which versions are you upgrading from and to?
Test Data is a major challenge Differing test data between Dev & QA, sometimes even between
QA, and definitely between versions of the application.
Common “Best Practices” Development Sandboxes
separate DB Schemas for each Developer (maybe even QA?) allows independent execution of data-altering tests allows one developer to work against a modified schema while
developing code that works with the new structure – preventing impact to the entire team
(not sufficient in itself - increases DBA workload unless automated; also have to worry about race conditions)
Common “test data” Allows Developers & QA to easily reproduce defects (can be difficult to keep track of across multiple versions)
Is that the best we can do? Any others?
Suggested Practices Put control where it needs to be
DBAs define structure Developers control timing
Create a flexible system of upgrade scripts sensitive to multiple release maintenance and branch development
Revision control your test databases ideally import / export to CSV revision alongside source code along with install scripts
Example Problem Movie Database
Start Simple – Implement Basic Functionality
Minor Requirement Changes
Major Requirement Change
Using ADDAM for Test Data, Lifecycle, & Multiple Releases
Movie Database (Initial Design)
Simple movie database, needs some work though, as requirements change.
Let’s start by creating the schema...
create table Movie ( movieID integer primary key, movieName varchar2(255), yearMade number(4,0));
create table Actor ( actorID integer primary key, firstName varchar2(255), lastName varchar2(255));
create table MovieActor( movieID integer references Movie, actorID integer references Actor);
Movie Database – Test Data Aliens,
starring Sigourney Weaver, Bill Paxton, Paul Reiser
Director James Cameron
Click starring Adam Sandler,
Christopher Walken, David Hasselhoff, Henry Winkler
Directed by Frank Coraci
Small Change (during Release 1.0)
NOT NULLs missinginsert into Actor (4,null,null);
Need to apply a change... How do we roll out to development, QA, production? What about branching?
alter table Actor modify (firstname not null);alter table Actor modify (lastname not null);
alter table Movie modify (moviename not null);
Bug Fix (scheduled for Release 1.1)
Some actors don’t have a last name, e.g. Madonna. Simple Change…
alter table Actor modify (lastname null);
Medium Change (Release 1.1)
The Director wants credit too. Easy enough to add…
create table Director ( directorID integer primary key, firstName varchar2(255) not null, lastName varchar2(255) not null);
create table MovieDirector( movieID integer references Movie, directorID integer references Director);
Larger Change What about other roles, such as Key Grip and Best Boy?
Larger Change
Merge MovieActor and MovieDirector into Credit Merge Actor and Director into Person Create Role to identify usage
Role is a “coded” table
Larger Change
Create Actor and Director views until reports are upgraded in next release.
Release Procedures Upgrade Scripts ready Need to generate updated Install Scripts (and with it Test Data)
This is particularly important for Product Development Also as a periodic practice per iteration in your project – keeps
“upgrade” times to a minimum when refreshing
How Does ADDAM Work? Install and Upgrade are very similar
install.drv, upgrade.drv specify directories to traverse master.drv inside directories specify scripts to execute VERSION and SCRIPT_LOG table
DDL Generator Generic or Vendor-Specific DDL Generation Generic uses JDBC DatabaseMetaData Oracle implementation uses dbms_metadata.get_ddl()
Test Data Import / Export Simple CSV codec; first line is column names “delete *” or “truncate” followed by batch insert statements
Current Best Practices Fall Short On medium-to-large teams, managing individual developer and
QA schemas is a huge time sink for development DBAs. Development DBAs are distracted from other key activities
Reviewing data model proposals Enforcing standards Performance tuning
More importantly, developers have to wait for their changes Saving binary snapshots of test data is also time consuming and
space-wasting compared to revision control of individual CSV files.
Lifecycle Automation Solves Problems By automating upgrades, control is in the right hands
DBAs control upgrade scripts Developers control timing of when scripts are applied No “race condition” to apply to all developer schemas after a
checkin that requires a DB change Simply uncomment needed scripts, “ant upgrade”, and check-in
when ready Huge benefits to QA – easy to build QA environments with test
data from any build label Package via ServletContextListener for auto-upgrade
Can also be used in production rollout Easy to package as a command line, or auto-upgrade
Other Useful References Scott Ambler’s guide to database refactoring
http://www.ambysoft.com/books/refactoringDatabases.html
top related