december 6, 20041 release management – ship it! phyllis kaiden release infrastructure manager the...
TRANSCRIPT
December 6, 2004 1
Release Management – Ship It!
Phyllis Kaiden
Release Infrastructure Manager
The Cobalt Group, Inc.
December 6, 2004 2
Outline
What is Release Management? Why does it matter? How does it work? Would you ship it? What did we learn?
December 6, 2004 3
What is Release Management?
Release Management Definition
Release Management Responsibilities
What is Configuration Management?
Product and Project Management
December 6, 2004 4
Release Management Definition
The process by which a product is made ready for distribution to customers. Release management is the coordination of all the activities leading to and including product release to customers.
The purpose of release management is to ensure products are ready when promised.
Focus on software release management but basics apply to all product releases.
December 6, 2004 5
It’s Like Baking
Recipe – the set of activities in order
Ingredients – quality components
Variations – Icing? Nuts?
Mixing – executing the recipe steps
Baking time – varies based on desired outcome
Is it done? – knowing when it’s just right
Final product – presentation, consumption
December 6, 2004 6
Baking & Releasing
Recipe – the set of activities in order (the plan)Ingredients – quality components (content, schedule,
resources)Variations – Icing? Nuts? (special considerations)Mixing – executing the recipe steps (do the plan)Baking time – varies based on desired outcome
(priority of time to market, features, quality)Is it done? – knowing when it’s just right (assess
readiness against measurable criteria)Final product – presentation, consumption (packaging
and distribution)
December 6, 2004 7
What is a release?
Delivery of a “system” from a supplier to a customer.
The “system” consists of a set of authorized and integrated components.
The supplier is usually the development organization.
The customer is the internal or external recipient of the system.
December 6, 2004 8
Key Practices
• Release Planning: Manage and publish release schedule
• Define deliverables included in a release• Build and Version the integrated release package• Manage dependencies across components• Coordinate activities across company • Control changes that would impact the release
schedule• Establish release criteria• Assess release readiness
December 6, 2004 9
Key Principle: Define “Done”
Deliverables Dependencies Avoid scope creep but allow change Release criteria
• Objective
• Quantifiable
• Cross-functional
December 6, 2004 10
Components of a release
Golden Build (software): CD-ROM, files Documentation and Help Training materials Marketing collateral Internal documents Customer Notification Release Notes
December 6, 2004 11
Types of Products Consumer: shrink-wrapped, commercial off-the-shelf
(COTS), e.g. MS Word Open-source: download as needed, e.g. MySQL Hosted: Application Service Provider (ASP), e.g. Yahoo
mail Corporate: Enterprise Resource Planning (ERP), e.g.
Oracle Financials Corporate desktop: enterprise distribution of desktop
applications, e.g. MS Office Operating system: core tools for IT, e.g. Sun Solaris
December 6, 2004 12
Attributes by Product Type
Consumer Open-Source
Hosted Corporate Corporate Desktop
Operating System
Installed by Self Self or IT
Vendor Vendor IT IT
Distribution Pull Pull Push Pull Push Push
Cost $ $$ $$$ $ $
Training No No Maybe Yes Maybe Maybe
Support Vendor No Vendor Vendor IT IT
Multiple supported versions
Yes Maybe No Yes Yes Yes
December 6, 2004 13
What is Configuration Management?
The process of building the product, tracking its components, and managing change.
The purpose of configuration management is to guarantee reproducibility of the product and traceability of all changes.
Release Management and Configuration Management are often the same group.
December 6, 2004 14
Common Responsibilities of CM
• Source Code and Version Control
• Reproducible – components of the release
• Traceable – what changed in this release
• Definitive Software Library: Baseline + delta
Version 1.0
Version 2.0
1.1 1.2delta deltabaseline
baseline branch merge
December 6, 2004 15
Product and Project Management
“ABCs of Release Management” by Mario Moreira. CM Crossroads, Aug 17 2004. http://www.cmcrossroads.com/ubbthreads/showflat.php?Cat=&Number=31323&Main=31323#Post31323
December 6, 2004 16
Why does Release Management matter?
Common Release Problems
Benefits of Release Management
Who Cares?
December 6, 2004 17
A Story – Timeliness, Quality
Three companies decide independently to develop appointment scheduling software. They all have the knowledge and funding to create a good product.
Company A releases on time with 100 known issues and is a success.
Company B releases on time with 50 known issues and fails.
Company C releases late with 10 known issues but fails because Company A had already captured the market.
December 6, 2004 18
Common Release Problems
Take too long – impacts revenue Unpredictable – impacts dependent plans Poor quality – impacts support Not useful – impacts revenue Contents unplanned – impacts integrity Contents uncontrolled – impacts reliability Overlooked distribution/installation – impacts
support
Cowham, Robert. “Release Management – Making it Lean and Agile” CM Crossroads, Aug 16, 2004 http://www.cmcrossroads.com/ubbthreads/showflat.php?Cat=&Number=31243
December 6, 2004 19
The Challenge
Time to Market vs Features vs Quality• Only one top priority• Balance – don’t ignore the other two• Release criteria reflect priorities• Product development methodology supports priorities:
concurrent engineering, test-first development, prototyping
Time to Market
Features Quality
December 6, 2004 20
Benefits of Release Management More successful releases Consistent release process Predictability Integration Completeness Quality Communication
December 6, 2004 21
Sales &Marketing
Support
IT
Pro*Management
QualityAssurance
Developers
Customers
Executives
Who Cares?
Worth installing?InstallableUsableSupportedBudgetTiming
Ready to sell?Demo updatedTrained sales forcePricingOrder FulfillmentContracts ready
Documentation readyTraining materialsTrained support staffRelease NotesKnown Problems FAQ, Knowledge BaseImplementation impact
Dependable InstallationResources to supportPredictable scheduleConfiguration OptionsTechnical DocumentationStability, Scalability, Reliability
Well-defined scopeResources to supportPredictable scheduleSufficient test timeMeets requirements
Product ManagementMarket positionProduct RoadmapProject ManagementProject delivery on-timeProgram ManagementProgram meets expectations
Dependencies managedMeets requirementsMinimize mergingNo “death march”Next project
Commitments toCustomers, Stockholders, EmployeesLegal ConsiderationsReturn on Investment
December 6, 2004 22
How does it work?
Types of Releases
Software Development Life Cycle
Release Process
December 6, 2004 23
Types of Releases
• Major = significant new features, product launch, planned, pull, annual
• Minor = small new features, backwards compatible, planned, pull, semi-annual
• Patch = collection of bug fixes, planned, pull, quarterly
• Emergency = Urgent bug fix, harmful if not fixed, unplanned, push, as needed
December 6, 2004 24
Attributes by Release Type
Features/Functions
Bugs Cost Urgent Scheduled Pull Backwards Compatible
Major Y (new)
Y Y Y Y
Minor Y (upgrade)
Y Y Y Y Y
Patch Y Y Y Y
Fix Y Y Y
December 6, 2004 25
Release Management in the Software Development Life Cycle
December 6, 2004 26
Activity DistributionBy Effort
10%
6%
8%
16%
40%
20%
Release
Reqts
Architecture
Detail Design
Construction
System Test
McConnell, Steve. Software Project Survival Guide. Microsoft Press, 1998.
Release activities account for
10% of effort and 15% of schedule.
December 6, 2004 27
Emergency?
BASELINE
RELEASE CYCLE
BASELINEDEVELOPMENT
LIFE CYCLE
UPGRADERELEASE
MAINTENANCERELEASE
PATCH RELEASE
ResolveWith
Initiator
FIX LATER OR
NEW FEATURE
SCHEDULED
MAINTENANCE
SCHEDULED
NEW FEATURES
BECOMES NEW
BASELINE
ROLL UP
ROLL UP
APPROVE
REJECT
DEVRM
RM
RM
RM
OWNERSCCB Change Control BoardDEV DevelopmentIS Information ServicesMRK MarketingPM Product ManagementQA Quality AssuranceRM Release ManagementSAL SalesSUP Support
3.1.2
3.1.3
3.1.1
3.1.4
3.2.0
3.1.0
3.2.0
DEFER
MOREINFORMATION
DEV
CHANGECONTROL
BOARD (CCB)PM
FIX NOW
UNSCHEDULED
MAINTENANCE
SUP
NOTIFY CCB
NEW CUSTOMERRELEASE
RM
TRACKINCIDENTS
SUP
TRACKCHANGES &
DEFECTSRM
Sources of IncidentsCustomersSystems Integrators
Sources of Changes &DefectsProduct ManagementMarketingSalesProduct SupportDevelopmentQAUser GroupStaffetc.
December 6, 2004 28
Release Process
1. Define Content2. Assign Version 3. Plan Release4. Build 5. Package6. Deploy7. Test8. Control Change9. Assess Readiness10. Release
Iterate
December 6, 2004 29
1. Define Content
Golden Build (software) Documentation Training materials Marketing collateral Internal documents Customer Notification Release Notes
December 6, 2004 30
1. Define Content: Must and NiceDisp ID Severity Pri Description
Nice 41928enhanceme
nt P3 Performance - Publish: loadSiteLite() to replace loadSite()
Nice 59439 trivial P4Misalignment of Editable site and Proof site sections on Manage site screen once the
actor checks Lock-site checkbox.
Nice 59929 minor P5Error message:Get A Quote form:The listed items in error message may need some
order; by logical or by alphabetic
Nice 57954 major P2 Only Ford Make is displayed in 'Quick Quote Form' : GM Design
Nice 59368 minor P2 Masthead descenders are being chopped off in Klamath_B_Dst design
Nice 57019 minor P3 Can't tell mouse over selection from current page selection in nav
Nice 59726 minor P3Design Celandine: Titles (Header) for static pages are not bold as those for dynamic
pages
Nice 60108 major P3 Misalignment of footer in Hours and directions page-Celandine designs
Must 64793enhanceme
nt P3 create new Saab luxury design and homepage
Must 64796enhanceme
nt P3 create new Land Rover luxury design and homepage
December 6, 2004 31
2. Assign version
Version is a unique identifier for the product state as of a certain point in time. Not all versions are released. Versions are sequential.
Versioning Schemes relate to release types, product management, and configuration management:
Major.Minor.Patch.Fix, e.g. 1.0, 1.1, 1.1.2, 1.1.2.1 Major.Feature.Patch, e.g. 1.0, 1.1, 1.1.2 Major.Feature.Patch.Build, e.g. 1.0.0.301
December 6, 2004 32
3. Plan Release
Dependencies Customer commitments Revenue recognition Resource availability: people, environments Introducing too much change at once Stabilizing between releases Organizational capability
December 6, 2004 33
3. Plan Release: MilestonesScope Freeze 9/7 COMPLETE
Functional Freeze 9/17 COMPLETE
Functional Test Complete 9/30 COMPLETE
Code Freeze 10/1 COMPLETE
Regression Test Complete 10/13 COMPLETE
Golden Build 10/8 COMPLETE
Performance Test Complete 10/13 COMPLETE
Mock Deployment 10/12 FAILED
Mock Deployment 10/14 COMPLETE
Deployment Plan Review 10/15 COMPLETE
Go/No-Go 10/15 COMPLETE – GO
Executive Approval 10/15 COMPLETE-APPROVED
RELEASE 10/15 COMPLETE
December 6, 2004 34
4. BuildBuild (What) Visibility (Who) Frequency (When) Purpose (Why)
Private System Build
Individual Developer
One or more times per change task
Provide feedback to the developer of the changes just implemented in the private workspace.
Integration Build
Whole Development Team
After each change task and/or one or more times per day (e.g. Nightly)
Integrates the latest changes in the repository and runs automated tests against the release providing feedback to the development team.
Release Build
Independent Test (QA or V&V) Team and Customers
At the end of each iteration/release
Packages release for distribution to the development team’s customer.
Appleton, Brad. “Agile Build Promotion: Navigating the Ocean of Promotion Notions”, CM Crossroads, September 2004. http://www.cmcrossroads.com/ubbthreads/showflat.php?Cat=&Number=32900
December 6, 2004 35
5. Package
Create the release package containing all release deliverables
File: tar, .zip, .exe, .ear Medium: CD-ROM, tape, file server Bill of Materials lists what’s in the
package Final release package is “golden build”
December 6, 2004 36
6. Deploy
Deployment Plan Install Team Deployment Plan Review Mock Deploy Mock Rollback Contact List
December 6, 2004 37
6. Deploy: Sample Steps1 Nitra "Version 2.3.3" Deployment
2 WLS 8.1 on SMTP servers Thomas
4 Verify email router updates in env file Thomas
5 Verify UPSShip updates in Env File Thomas
9Validate receipt of Nitra, LMCC Tar Balls and
Mapping Loader script Claus
10 Add Scion Altercast fonts (Bug 61506) Thomas
11 Prep-Outage Tasks 1 hr Steven
12 Hot Backup Nitra DB 55 min 9:00 PM 9:55 PM Claus
13Check 3rd Party Services (Chrome, AIC,
Vicinity) 10 mins 9:30 PM 9:45 PM Jeff M
14 Team Check-In 5 mins 9:45 PM 9:50 PM Team
15 Check CDC design flag to true 5 mins 9:45 PM 9:50 PM Thomas
16 Outage
17 NO DEALER ACCESS 5 mins 9:50 AM 9:55 AM Thomas
18 NO CONSUMER ACCESS 5 mins 9:55 AM 10:00 PM Thomas
22 Shutdown Servers 40 mins
23 Disable Nitra Monitoring 3 mins 10:05 PM 10:08 PM Thomas
24 Disable Incoming Leads (no consumer access) 2 mins 10:08 PM 10:10 PM Thomas
25 Allow Existing leads to pass (Big IP) 2 mins 10:10 PM 10:12 PM Thomas
26 Bring down WLS (Nitra and LM) 30 mins 10:12 PM 10:42 PM Thomas
27 Snapshot net-app (back up file system) 2 mins 10:42 PM 10:44 PM Thomas
28 Deployment time noted (for rollback purposes) 1 mins 10:44 PM 10:45 PM Thomas
29 Database Modifications 5 mins
31 LMCC DB Script 10 min 10:55 PM 11:05 PM Claus
32 Run script for Mapping loader 10 min 11:05 PM 11:15 PM Claus
33 Code Deployment (LM and Nitra Clusters) 40 mins
December 6, 2004 38
7. Test
Plan ahead for test environments Execute Test Plan
• Functional test: does it function as required?
• Regression test: did old bugs reappear?
• Performance test: does it perform within acceptable limits?
• Reliability test: does it function consistently over time?
• Stress test: does it function consistently under volume?
December 6, 2004 39
8. Control Change
• Baseline scope, requirements, design• Accommodate controlled change
• Avoid scope creep• Allow change, it’s reality
• Change requests• Change request decision (approve, reject, defer)
• Change Control Board (CCB)• Cross-functional representation• Impact analysis• Business drivers
December 6, 2004 40
9. Assess Readiness
• Release Criteria define “done”• Defined Deliverables
• Features: Must Have vs Nice to Have
• Quality: test results, metrics
• Schedule: time-to-market criteria
• Objective, quantifiable criteria preferred
• Organizational Readiness• Support, Training, Marketing, Sales
• Acceptable Risk
December 6, 2004 41
9. Assess Readiness: Release CriteriaQuality Testing Complete (functional, regression, performance, reliability, stress)
Quality Successful Mock Deployment
Quality Successful Mock Rollback
Quality "Golden" Build in Hand
Features All “must have” features complete
Features Documentation and Help updated
Features Release Notes Accepted by Customer Support
Schedule Schedule Clearance (no conflict)
Schedule Outage Notification Processed
Schedule Release Package Delivered to IT
Schedule Acceptable Deployment Night Coverage
Schedule Acceptable Post-Deployment Coverage
Quality Go/No Go Meeting yields Go decision. Punch List completed.
December 6, 2004 42
Assess Readiness: MetricsCommon metrics: fix rate, find rate, defect
density, rate of changeExample: find rate vs fix rate
Stability reached at week 35 of testing, when defects fixed exceeds defects opened.
December 6, 2004 43
9. Assess Readiness: Go/No-Go
Release Readiness Assessment• Are all release criteria equal?
• Can any criteria be waived?
Go/No-Go• Formal sign-off by stakeholders
• Meeting, email vote
• Recommendation by Release Manager
• Go with Punch List
Executive Approval
December 6, 2004 44
10. Release
Formal release by Configuration Mgmt Deploy to Production or Distribute to
Customer
December 6, 2004 45
10. Release Programs
Alpha = test by in-house department Beta = test by small group of customers
• Pilot
• Early Adopters
General Availability (GA) = all customers
December 6, 2004 46
Would you ship it?
Situations
Is it ready?
Would you ship it?
December 6, 2004 47
Situation #1: Class registration Criteria
• Zero priority-one defects
• Ready in time for class registration on April 1 Data
• One priority-one defect affecting 100 music majors out of 3,000 students.
• Fix estimate is 3 days.
• Today is March 15. Ship it? More data?
December 6, 2004 48
Situation #1: Other Factors Cost: Penalty for late delivery? Confidence: accurate estimate? Complexity? Quality: Danger of breaking something else? How
much testing is needed? Schedule: Holidays? Vacation? Customer
availability? Hard deadline? Relationship: Unhappy customer? Reference
site? Workaround: Manual? Send staff on site?
December 6, 2004 49
Situation #2: Java web search tool Criteria
• Acceptable performance: + 10% current
• Backwards compatibility Data
• Performance on page load is 5% better
• Performance on search is 20% worse
• Fix will require customer to download new Java Virtual Machine (JVM)
Ship it? More data?
December 6, 2004 50
Situation #2: Other Factors Quantify: How much loss? < second? Accuracy: performance test environment? Alternatives: Optional download?
Automatic download? Support: Ease of download? Increased
support load? Competition: Market leader? Slipping? Usability: lose customers if slow?
December 6, 2004 51
What did we learn?
Release Management Definition
Key Practices
Key Principle
Release Process
Features, Quality, Time to Market
December 6, 2004 52
Release Management Definition
The process by which a product is made ready for distribution to customers. Release management is the coordination of all the activities leading to and including product release to customers.
The purpose of release management is to ensure products are ready when promised.
December 6, 2004 53
Key Practices
• Release Planning: Manage and publish release schedule
• Define deliverables included in a release• Manage dependencies across components
and activities in the release • Coordinate activities across company • Control changes that would impact the
release schedule• Establish release criteria• Assess release readiness
December 6, 2004 54
Key Principle: Define “Done”
Deliverables Dependencies Avoid scope creep but allow change Release criteria
• Objective
• Quantifiable
• Cross-functional
December 6, 2004 55
Release Process
1. Define Content2. Assign Version 3. Plan Release4. Build 5. Package6. Deploy7. Test8. Control Change9. Assess Readiness10. Release
Iterate
December 6, 2004 56
The Challenge
Time to Market vs Features vs Quality• Only one top priority• Balance – don’t ignore the other two• Release criteria reflect priorities• Product development methodology supports priorities:
concurrent engineering, test-first development, prototyping
Time to Market
Features Quality
December 6, 2004 57
Q & A
THANK YOU!