best practices for advanced source control beyond checkout and checkin steven wilssens compuware.net...

Best Practices for Advanced Best Practices for Advanced Source ControlSource Control

Beyond CheckOut and CheckInBeyond CheckOut and CheckIn

Steven WilssensSteven WilssensCompuware .NET ConsultantCompuware .NET Consultant

http://http://Steven.WilssensSteven.Wilssens.net.net

http://http://www.visug.bewww.visug.be

Visual Studio User Group Visual Studio User Group BelgiumBelgium

Community Meeting Point

Evening MeetingsEach Month, starting March 2006

3 parts:Round Table

Formal practical covering

Socializing part

http://http://www.visug.bewww.visug.be

Have you ever…Have you ever…

Had a small part of the development team finishing a release, while the others were waiting

Needed to support released versions

Needed to develop multiple versions

Have you ever…Have you ever…

Needed to prepare interim releases for testing

Suffered from integration nightmares

Suffered from source control contention

GoalsGoals

After this session you will be able to:Identify different branching strategies

Choose the appropriate branching strategyIncrease overall product quality

Increase overall process efficiency

Apply the appropriate strategies with Team Foundation Source Control

Start the implementation of continuous integration

ContentContent

Branching strategiesBranching terminologyReasons for branchingDifferent branching strategiesGuidelines on how to adopt the right branching strategy

ContentContent

Team Foundation Source ControlFeaturesHow to implement a branching strategyContinuous integrationSNAP

Branching StrategiesBranching Strategies

Contains:Entire source codeDocumentation files (optional)Anything else necessary to build and release a complete product (optional)

It is sequential and it only grows

Branching TerminologyBranching TerminologyMainline/TrunkMainline/Trunk

Change 1 Change 2

Change 5

Change 4

Change 6

Change 3

Copy of another branch (or trunk)Taken at a certain timeContains new developmentsGrows independently of its parentTrunk is a special type of branch

Branching TerminologyBranching TerminologyBranchBranch

Change 1 Change 2

Change 5

Change 4

Branch 1

Change 6

Change 3

MergingProcess of taking source files from one branch and applying them to another

Branching TerminologyBranching TerminologyMergingMerging

Change 1

Change 2

Change 5

Change 4

Branch 1

TrunkMerging

Change 6

Change 3

Moving changes from a child branch to the parent branch

Branching TerminologyBranching TerminologyReverse IntegrationReverse Integration

Change 1 Change 2 Change 4

Branch 1

TrunkReverse Integration

Change 5

Change 3

Integration Time!

Possible merge conflicts need to be solved on the Trunk

Branch 1

Change 5

Change 3

Integration Time!

Build and unit/component/… test problems are discovered on the trunk!

Change 2 Change 4

Branch 1

Change 5

Change 3

Integration Time!

Moving changes from a parent branch to a child branch

Branching TerminologyBranching TerminologyForward IntegrationForward Integration

Branch 1

TrunkFoward Integration

Change 3

Integration Time!

Change 5

Branching TerminologyBranching TerminologyEnd to End ScenarioEnd to End Scenario

Branch 1

Change 3

Integration Time!

RI: Reverse Integration

FI: Forward Integration

Branch 1

TrunkFI

Change 3

Integration Time!

Change 5

Branch 1

TrunkFI

Change 3

Integration Time!

Change 5

Branch 1

TrunkFI

Change 3

Integration Time!

Change 5

Change 6

Reasons for branchingReasons for branching

Primary reason for branching:

If concurrent or parallel development must occur!

Need to develop bugfixes/patches for a previous release(s)

Need to support multiple released versions

Need to develop multiple versions at the same time

Need to prepare interim releases for testing

Need to reduce source control contention

Want your team to be productive during the testing phases of the SDLC

Branch by Release Branch by Purpose

Virtual Build Lab

Quality GatesBridge

Branching Strategies

Continuous Integration

Everything should be made as simple as possible, but not one bit simpler.Albert Einstein

Virtual Buid Lab

Quality GatesBridge

Branching StrategiesBranching Strategies Branch by ReleaseBranch by Release

Release 1.1 Development

Release to QA Release to Production

Development

Release to QA

ProductionRelease 1.2

Development

QA ProductionRelease 1.3

Release to Production

Production

Approved bug fix

Development

Release to QA

Development

Production

Approved bug fix

Development

Release to QA

Development

Production

Merge bug fix to Release 1.2

Branching StrategiesBranching StrategiesBranch by ReleaseBranch by Release

Approved bug fix

Development

Release to QA

Development

Production

Branching StrategiesBranching Strategies Branch-by-ReleaseBranch-by-Release

ProvidesSeries of successive baselines

Common base for developers to use in making further changes to code

Simple to use when only supporting 1 version

DrawbacksDoes not support long term parallel development (code chill phase)

Added complexity and overhead to support of released versions

Build-by-bug-number syndrome

Branching StrategiesBranching Strategies Branch-by-ReleaseBranch-by-Release

Build-by-bug-number syndromeOccurs when code gets checked in a branch after the release

:-(Who?

Release Engineer

Assignment? Hand pick only the pinpointed fixes and all bits of code needed

When? Usually at the worst time -> an emergency release

Virtual Build Lab

Quality GatesBridge

Branching StrategiesBranching Strategies Branch-by-PurposeBranch-by-Purpose

Decision to branch is based on the need to satisfy a specific purpose:

Release of software and associated items

Typically mark significant milestones:

Release for Alpha testing

Release for Beta testing

Release to QA for system testing

Release to production

Service Release

Trunk Development

Feature Freeze

Alpha 1

Testing

Testing and approved changes only

Trunk Development

Feature Freeze

Alpha 1

Testing

Alpha n

Testing

Approved Bug Fix n

Trunk Development

Feature Freeze

Alpha 1

Testing

Alpha n

Testing

Beta 1

Testing

Beta n

Testing

Approved Bug Fix n

Trunk Development

Feature Freeze

Code Freeze(Final Release

to QA)

Alpha 1

Testing

Release Candidate

Alpha n

Testing

Beta 1

Testing

Beta n

Testing

Development

Approved Bug Fix n

Trunk Development

Feature Freeze

to QA)

Alpha 1

Testing

Approved Bug Fix

Release Candidate

Alpha n

Testing

Beta 1

Testing

Beta n

Testing

Development

Approved Bug Fix n

Trunk Development

Feature Freeze

to QA)

Alpha 1

Testing

Approved Bug Fix

Release Candidate

Alpha n

Testing

Beta 1

Testing

Beta n

Testing

Development

Merge approved and tested bug

Approved Bug Fix n

Trunk Development

Feature Freeze

to QA)

Alpha 1

Testing

Approved Release toProduction

Approved Bug Fix

Release Candidate

Alpha n

Testing

Beta 1

Testing

Beta n

Testing

RTM 1.0

Development

Approved Bug Fix n

Drawbacks:More complex

Code chill phase

Advantages:Supports regular releases by design

Supports emergency releases as required

Avoids problems caused by build-by-bug-number syndrome

Lets the developer work from the main branch

Virtual Build Lab

Quality GatesBridge

Branching StrategiesBranching Strategies BridgeBridge

Trunk Development

Feature Freeze

to QA)

Alpha 1

Testing

Approved Bug Fix

Release Candidate

Alpha n

Testing

Beta 1

Testing

Beta n

Testing

Development

Approved Bug Fix n

Code Chill Phase

Proceed with enhancements not intended for the release currently in progress

Solution:1.Creation of bridge line

2.Apply the enhancements

3.Merge the bridge line back down into the main development line after the release candidate releases

Trunk Development

Feature Freeze

to QA)

Alpha 1

Testing

Approved Bug Fix

Release Candidate

Alpha n

Testing

Beta 1

Testing

Beta n

Testing

Development

Approved Bug Fix n

Bridge

Code Chill Phase

Trunk Development

Feature Freeze

to QA)

Alpha 1

Testing

Approved Bug Fix

Release Candidate

Alpha n

Testing

Beta 1

Testing

Beta n

Testing

Development

Approved Bug Fix n

Bridge

Code Chill Phase

Trunk Development

Feature Freeze

to QA)

Alpha 1

Testing

Approved Bug Fix

Release Candidate

Alpha n

Testing

Beta 1

Testing

Beta n

Testing

Development

Approved Bug Fix n

Bridge

Code Chill Phase

Lets multiple teams work on multiple subprojects in parallel

Allows developers to modify code without impacting other developments

Virtual Build Lab

Quality GatesBridge

Branching StrategiesBranching StrategiesVirtual Build LabVirtual Build Lab

Specific for a task branchOwned by a specific component or project teamThe owner is responsible for integrating his code into the mainlineEach VBL performs full build and installable releases from code in its source trees and the mainline

Virtual Build Lab

Quality GatesBridge

Branching StrategiesBranching StrategiesContinuous IntegrationContinuous Integration

Source Control Client

Build Store

Continous Integration Service

Continuous Integration Server

Source Control Service

Repository

Check In1

Build Service

Build Scripts and

Targets

Build Drop Site

Source Control

Build Server

Build Store

Repository

Check In1

Store changeset2

Build Service

Build Scripts and

Targets

Build Drop Site

Source Control

Build Server

Build Store

Repository

Check In1

Store changeset2

Build Service

Build Scripts and

Targets

Build Drop Site

Source Control

Build Server

Poll for changes

Build Store

Repository

Check In1

Store changeset2

Build Service

Build Scripts and

Targets

Build Drop Site

Source Control

Build Server

Retrieve latest version

Poll for changes

Build Store

Repository

Check In1

Store changeset2

Build Service

Build Scripts and

Targets

Start Build

Build Drop Site

Source Control

Build Server

Poll for changes

Build Store

Repository

Check In1

Store changeset2

Build Service

Build Scripts and

Targets

Start Build

Build Drop Site

Execute Build

Source Control

Build Server

Poll for changes

Build Store

Repository

Check In1

Store changeset2

Build Service

Build Scripts and

Targets

Start Build

Build Drop Site

Execute Build

Copy Binaries

Source Control

Build Server

Poll for changes

Build Store

Repository

Check In1

Store changeset2

Build Service

Build Scripts and

Targets

Start Build

Build Drop Site

Execute Build

Build Finished

Copy Binaries

Source Control

Build Server

Poll for changes

Build Store

Repository

Check In1

Store changeset2

Build Service

Build Scripts and

Targets

Start Build

Build Drop Site

Execute Build

Build Finished

Store Build Data

Copy Binaries

Source Control

Build Server

Poll for changes

Advantages:Integration problems are detected and fixed continuouslyEarly warning of broken/incompatible codeImmediate unit testing of all changesConstant availability of a "current" build for testing, demo, or release purposesThe immediate impact of checking-in incomplete or broken code acts as an incentive

Disadvantages:Maintenance overheadNeed for dedicated buildserver(s)Immediate impact of checking-in incomplete or broken code acts as disincentive to provide frequent (backup) check-insIncomplete or broken code is only detected after the code has been added to the repository!

Virtual Build Lab

Quality GatesBridge

Branching StrategiesBranching Strategies Quality GatesQuality Gates

CriteriaGeneral stability and completeness of the product

Percentage of test coverage on the codebase

Level of documentation provided

Results of a security assessment

Performance and scalability metrics

How to ensure that only changes that pass the quality gate can be reverse integrated into the parent branch?

Security settings and separate rolesFeature Team, Integration Team, Build Team

Change 1 Change 2

Change 3

Change 4

Feature A

Integration TeamFeature Team Build Team

FI: Forward Integration RI: Reverse Integration

Change 1 Change 2

Change 3 Change 5

Change 4

Feature A

Change 1 Change 2

Change 3 Change 5

Change 4

Feature A

FI Build

Change 1 Change 2

Change 3 Change 5

Change 4

Feature A

Change 1 Change 2

Change 3 Change 5

Change 4

Feature A

Change 6

Change 1 Change 2

Change 3 Change 5

Change 4

Feature A

Change 6

Build Build

Branching StrategiesBranching StrategiesQuality GatesQuality Gates

Track all check-ins to a secured, locked-down, golden source tree

Reject check-ins that do not comply

Assign integration sponsors for the mainline branch

Virtual Build Lab

Quality GatesBridge

Branching StrategiesBranching StrategiesSNAPSNAP

Shiny New Automated ProcessUsed to protect the Golden Tree (Quality Gates)First developed by the InfoPath team at MicrosoftUsed by many teams at MicrosoftPlans to release a SNAP system in combination with Team System

Change 1 Change 2

Change 3 Change 5

Change 4

Feature A

Feature Team SNAP

RI: Reverse IntegrationFI: Forward Integration

Change 1 Change 2

Change 3 Change 5

Change 4

Feature A

Feature Team SNAP

Change 1 Change 2

Change 3 Change 5

Change 4

Feature A

Feature Team SNAP

Submit Changes to SNAP

Change 1 Change 2

Change 3 Change 5

Change 4

Feature A

Feature Team SNAP

Change 1 Change 2

Change 3 Change 5

Change 4

Feature A

Feature Team SNAP

Change 1 Change 2

Change 3 Change 5

Change 4

Feature A

Feature Team SNAP

Build4

Change 1 Change 2

Change 3 Change 5

Change 4

Feature A

Feature Team SNAP

Build4

5Change 6

Branching StrategiesBranching StrategiesVersioningVersioning

What should versioning allow you to do?Track your product binaries to the original source code

Re-create a past build by having meaningful labels

Branching StrategiesBranching StrategiesVersioningVersioning

What should be kept under source control?

Code or documents that are under development – YESYES

Third party or external binaries– IT IT DEPENDSDEPENDS

You can keep third party or external binaries, development tools on a file server

Storing them in a repository will allow you to recreate any build at any time

Guidelines on how to adopt Guidelines on how to adopt the right branching strategy the right branching strategy

RecommendationsUse consistent reverse and forward integration practices across branches

Use atomic check-in when Reverse Integrating

Consider adopting the Branch-by-Purpose model

Guidelines on how to adopt Guidelines on how to adopt the right branching strategy the right branching strategy

RecommendationsThe mainline should always be buildable and consumable

A developer check-in should normally only be made to the private branch, not the golden branch

SNAP/Gauntlet service should protect the golden branch

Use Continuous Integration on all branches except the golden branch

Team Foundation Source ControlTeam Foundation Source Control

TFSC Terminology TFSC Terminology

RepositoryData store that contains all files and folders in TFSC

database

MappingAssociation of a repository path with a local working

folder on a local computer

Working FolderDirectory on client computer that contains local copy of

some subset of files in a repository

WorkspaceContains a reference to the repository and a series of

mappings that associate a repository path with a working folder on a users computer

Change setA set of modifications to one or more files/folders that

is atomically applied to a repository at check-in

A Picture of a Check-inVisual Studio

Checkin

ValidatePolicies

Checkin PolicyEngine

CommitChanges

V1V1V1V1V5V5V5V5V4V4V4V4V5V5V5V5

V2V2V2V2V3V3V3V3

Change Set 727012Change Set 727012Change Set 727012Change Set 727012Change Set 727012Change Set 727012Change Set 727012Change Set 727012

Change Set 727011Change Set 727011Change Set 727011Change Set 727011Change Set 727012Change Set 727012

WorkItem 33231WorkItem 33231

UpdateWork Items

Events Raised for3rd party sync purposes

ArtifactLink

WorkspaceWorkspace

Integration Infrastructure

Check E-mailNotification

Version Control

Work Item Tracking

TFSC Shelving TFSC Shelving

Scenario1. Working on a new feature

2. A critical bug is found that needs immediate attention

3. Shelve current change set

4. Make changes to address the bug and check-in

5. Unshelve previous change set

6. Sync workspace to merge any modifications that are in the workspace

7. Continue work on the new feature

8. Check in changes

Shelve1.Archiving of all modifications in current

change set

2.Replacing those modified files with original files

UnShelve1.Merge current modifications with the

changes in the repository

ContinuousContinuous Integration IntegrationSource Control Client

Eventing and Notification Service

Check In ChangeSet1

Build Service

Build Scripts and

Targets

Build Drop Site

Team Foundation Integration Serivces

Build Server

Team Foundation Object Model (Managed API)

Linking Service Registration Service

Work Item Services Source Code Service Build Data Services

Team Foundation Data Serivces

Methodology

WI SCCBuild Data Warehouse

Service

Continuous Integration Service

Check In ChangeSet1

Build Service

Build Scripts and

Targets

Build Drop Site

Build Server

Methodology

Http/SOAP message2

Service

Check In ChangeSet1

Build Service

Build Scripts and

Targets

Build Drop Site

Build Server

Methodology

Http/SOAP message2

Store ChangeSet3

Service

Check In ChangeSet1

Build Service

Build Scripts and

Targets

Build Drop Site

Build Server

Methodology

Http/SOAP message2

Store ChangeSet3

Service

Check In ChangeSet1

Build Service

Build Scripts and

Targets

Build Drop Site

Build Server

Methodology

Http/SOAP message2

Store ChangeSet3

Service

Receive Notification

Check In ChangeSet1

Build Service

Build Scripts and

Targets

Build Drop Site

Build Server

Methodology

Http/SOAP message2

Store ChangeSet3

Service

Start Build

Check In ChangeSet1

Build Service

Build Scripts and

Targets

Build Drop Site

Execute Build

Build Server

Methodology

Http/SOAP message2

Store ChangeSet3

Service

Start Build

Check In ChangeSet1

Build Service

Build Scripts and

Targets

Build Drop Site

Execute Build

Copy Binaries

Build Server

Methodology

Http/SOAP message2

Store ChangeSet3

Service

Start Build

Check In ChangeSet1

Build Service

Build Scripts and

Targets

Build Drop Site

Execute Build

Copy Binaries

Build Server

Methodology

Http/SOAP message2

Store ChangeSet3

Service

Start Build

Update Build Store

SNAPSNAP SNAP Client

Team Build Store

TF Warehouse

Source Control

Team Build

TFS Data Tier

SNAP SNAP Store

SNAP Request- Project Name

- Source Branch

- Target Branch- Version Specs

1 Team Build Service

SNAP Build Scripts and

Targets

Build Drop Site

Build Server

Team Build Store

TF Warehouse

Source Control

Team Build

TFS Data Tier

SNAP SNAP Store

- Source Branch

Enqueue SNAP Request2

Team Build Service

Targets

Build Drop Site

Build Server

Team Build Store

TF Warehouse

Source Control

Team Build

TFS Data Tier

SNAP SNAP Store

- Source Branch

Team Build Service

Targets

Build Drop Site

Build Server

Team Build Store

TF Warehouse

Source Control

Team Build

TFS Data Tier

SNAP SNAP Store

- Source Branch

Team Build Service

Targets

Build Drop Site

Build Server

Retrieve Golden Branch

Team Build Store

TF Warehouse

Source Control

Team Build

TFS Data Tier

SNAP SNAP Store

- Source Branch

Team Build Service

Targets

Build Drop Site

Build Server

Merge Changes from Feature Branch to Golden Branch

Team Build Store

TF Warehouse

Source Control

Team Build

TFS Data Tier

SNAP SNAP Store

- Source Branch

Team Build Service

Targets

Build Drop Site

Build Server

Start Build

Team Build Store

TF Warehouse

Source Control

Team Build

TFS Data Tier

SNAP SNAP Store

- Source Branch

Team Build Service

Targets

Start Build

Build Drop Site

Build Server

Start Build

Team Build Store

TF Warehouse

Source Control

Team Build

TFS Data Tier

SNAP SNAP Store

- Source Branch

Team Build Service

Targets

Start Build

Build Drop Site

Execute Build

Build Server

Start Build

Team Build Store

TF Warehouse

Source Control

Team Build

TFS Data Tier

SNAP SNAP Store

- Source Branch

Team Build Service

Targets

Start Build

Build Drop Site

Execute Build

Copy Binaries

Build Server

Start Build

Team Build Store

TF Warehouse

Source Control

Team Build

TFS Data Tier

SNAP SNAP Store

- Source Branch

Team Build Service

Targets

Start Build

Build Drop Site

Execute Build

Build Finished

Copy Binaries

Build Server

Start Build

Team Build Store

TF Warehouse

Source Control

Team Build

TFS Data Tier

SNAP SNAP Store

- Source Branch

Team Build Service

Targets

Start Build

Build Drop Site

Execute Build

Build Finished

Store Build Data

Copy Binaries

Build Server

Start Build

Take AwayTake Away

Use consistent reverse and forward integration practices across branches

Use atomic check-in when Reverse Integrating

Consider adopting the Branch-by-Purpose model

Reduce code chill phase by using applying the bridge model in combination with Branch-by-Release model

Quality Gates protect the golden branch

Take AwayTake Away

Team System Source Control is pretty advanced

Shelving is a handy feature

Team System Source Control is available to VS 2005, VS 2003, VB 6,... developers

How to implement a Continuous Integration process

How to implement a SNAP System

ResourcesResources

Continuous Integration with Team SystemMSDN Magazine

http://msdn.microsoft.com/msdnmag/issues/06/03/TeamSystem/default.aspx

Branching Models and Team SystemBrian Harry

http://blogs.msdn.com/bharry/

Mitch Dennyhttp://notgartner.com/

Steven Wilssenshttp://Steven.Wilssens/

The Build Master [Vincent Maraia]ISBN: 0-321-33205-9

ResourcesResources

Team SystemRob Caron

http://blogs.msdn.com/robcaron/default.aspx

Professional Visual Studio 2005 Team System ISBN: 0764584367

Working with Microsoft Visual Studio 2005 Team SystemISBN: 0735621853

Thank You!Thank You!

Thanks for your attention

I’ll be happy to answer all your questions:Right after this session

Compuware booth

Visual Studio User Group BoothCommunity Booth: Ask-The-Experts

best practices for advanced source control beyond checkout and checkin steven wilssens compuware.net...

source files

reverse integrationfi

integration nightmaressuffered

reverse integrationbranch

trunkreverse integrationchange

trunkfoward integrationchange

appropriate strategies

development team

Documents

checkin off the wish lists

checkin 2.0

children’s checkin kiosks...updated: 8/24/15 page: 1...

checkin enhancements that help you find & qualify event...

chicken or checkin'? rational learning in repeated chess...

local de embarque: aeroporto de brasÍlia horÁrio :...

front desk best practices · front desk best practices day...

teaching schedule ug courses... · iufd history checkin...

announcements checkin

new teamcenter enterprise - fermilab · 2006. 4. 30. ·...

touchscreen checkin, usage and setup. checkin demonstrate...

checkout our

aéropostale - checkout

experiment checkout during postmanufacturing checkout of

mobiquity®wallet checkout - quick, seamless and secure...

ncr selfserv checkout - acme retail systemsncr selfserv...

how to use the fort jackson isportsman site to checkin

ynpn chapter congress checkin - 6.12.12

are you ready for the facts about self-checkout software?...

checkin procedure