© 2008 IBM Corporation

Erich GammaIBM distinguished engineerIBM rational zurich research lab

how (8 years of) eclipse changed my views on software development

SECR 08

Outline

Closed Development Open Development

2003200320002000 20012001 20022002

JuneMarch JuneJune

20042004

June

20052005 20062006

NovemberFall June

3.12.1 3.02.0 3.21.0ProjectStarts

TechPreview

OpenOpenSourceSourceOpenOpen

SourceSource

20072007

June

3.3

Built to LastExtensibility

API

CommunityTransparency

Development ProcessAgility

TeamTools

20082008

June

3.4

© 2005 IBM Corporation

how buildings last

Site

• Stewart Brand: how buildings learn– what happens after they're built

• stuff: furniture

• services: electrical, plumbing (7-15y)

• structure: foundation, load bearing walls (30-300y)

• site: geographical setting (forever)

• layers:

• evolve at different rates during the life of a building

• shear against each other as they change at different rates

• an adaptive building must allow slippage a building that lasts is adaptive and can change over time lasts for generations without total rebuilding

© 2005 IBM Corporation

structure foundation

the eclipse plug-in architecture everything is a plug-in

simple and consistent

© 2005 IBM Corporation

eclipse plug-in architecture

plug-in == component set of contributions

smallest unit of Eclipse function

details spelled out in plug-in manifest

extension point – named entity for collecting contributions

extension – a contribution Example: a specific spam filter tool

runtime – controls and manages contributions

plug-in

platform

plug-in

ExtensionExtension point

runtime

© 2005 IBM Corporation

services plumbing: APIs

Plug-in dependencies through APIs define APIs for stability

binary compatibility is highest priority

© 2005 IBM Corporation

APIs first

APIs don’t just happen; we need to design them

specifications with precisely defined behavior what you can assume (and what you cannot)

it works ≠ API compliant

documented classes ≠ API

must have at least one client involved, preferably more

© 2005 IBM Corporation

practical extensibility

extensible in ways that are known useful needed by us, requested by others

we don't provide hypothetical generality - we want to be extensible in ways that matter

don't over generalize

© 2005 IBM Corporation

stuff, furniture - UI

eclipse extension architecture is contribution based extensions contribute to the workbench

the workbench manages and presents the contributions

enables UI evolution 3.0 new look

© 2008 IBM Corporation10

Key Lessons Modularity matters

Everything is a plug-in “no exceptions”

APIs are a huge commitment we would rather provide less API than desired (and

augment) than provide the wrong (or unnecessary) API and need to support it indefinitely

the tyranny of stable APIs API layers…

the challenge of product developers which API level does our product require and support

n–1, n-2

© 2008 IBM Corporation11

Eclipse 3.4 API Tools

Support to define an API baseline e.g. Eclipse 3.3 when working

on 3.4 Check access restrictions

API javadoc tags:@noimplement, @noinstantiate, @noextend

Detect binary compatibility violations

Detect version problems @since

Problems are reported during builds

© 2008 IBM Corporation12

stable API evolution patterns

compatibility layer I*2 extensions interfaces extension interface: IAdaptable restart in a new package

© 2008 IBM Corporation13

extension interfaces: IAdaptable

supports adding interfaces to existing types Interface negotiation

IPersistable p= (IPersistable ) o.getAdapter(IPersistable.class);p.saveState(…);

<extension point="org.eclipse.core.runtime.adapters"><factory class=“someAdapterFactory" adaptableType=“someType"> <adapter type="IPersistableElement"/>

© 2008 IBM Corporation14

Outline

Closed Development Open Development

2003200320002000 20012001 20022002

JuneMarch JuneJune

20042004

June

20052005 20062006

NovemberFall June

3.12.1 3.02.0 3.21.0ProjectStarts

TechPreview

OpenOpenSourceSourceOpenOpen

SourceSource

20072007

June

3.3

Built to LastExtensibility

API

CommunityTransparency

Development ProcessAgility

TeamTools

20082008

June

3.4

How we Started: Closed developmentHow we Started: Closed development

The Swiss Bank approach to software development If it hasn't shipped

it doesn’t exist

Strong firewall between developers and customers

The Swiss Bank approach to software development If it hasn't shipped

it doesn’t exist

Strong firewall between developers and customers

Shipping Matters…Shipping Matters…

our objective is to ship software anything that contributes to this goal is good anything that does not is bad

team member recognition is based on the ability to ship quality software on time

the culture: "If you ship, then you may speak."

the question: "Did they ever ship anything?"

the insult: "They never ship anything!"

our objective is to ship software anything that contributes to this goal is good anything that does not is bad

team member recognition is based on the ability to ship quality software on time

the culture: "If you ship, then you may speak."

the question: "Did they ever ship anything?"

the insult: "They never ship anything!"

Eclipse 1.0

Nov 2001

Eclipse 2.0

June 2002

Eclipse 2.0.1

Sept 2002

Eclipse 2.0.2

Nov 2002

Eclipse 2.0.3

Mar 2003

Eclipse 2.1

Mar 2003

Eclipse 2.1.1

June 2003

Eclipse 2.1.2

Oct 2003

Eclipse 2.1.3

Feb 2004

Eclipse 3.0

June 2004

Eclipse 3.0.1

Sept 2004

Eclipse 3.0.2

Mar 2005

Eclipse 3.0.3 Jul 2005

Eclipse 3.1

June 2005

Eclipse 3.1.1

Sept 2005

Eclipse 3.1.2

Feb 2006

Eclipse 3.2

June 2006

Eclipse 3.2.1

Sept 2006

Eclipse 3.2.2

Jan 2007

Eclipse 3.3

June 2007

Eclipse 3.3.1

Sept 2007

Eclipse 3.3.2

Feb 2008

Eclipse 3.4.0

June 2008

Eclipse 3.4.1

Sept 2008

November 2001: “Open Source”Reaction from the development teamNovember 2001: “Open Source”Reaction from the development team

You want usto do what?

Code in public?

Have technical discussions in public?

Answer all those dumb questions?

Why are we doing this again?

Key Lessons Key Lessons Transparency helps existing

development Better understanding of current

status Responding to feedback takes

time, but pays off

Use same communication channels inside as outside

Not limited to Open Source projects “Open Commercial

Development”

Transparency helps existing development Better understanding of current

status Responding to feedback takes

time, but pays off

Use same communication channels inside as outside

Not limited to Open Source projects “Open Commercial

Development”

CommunityCommunityCommunityCommunity

Transparency

Feedbackand Support

DevelopersDevelopers

Open Commercial DevelopmentOpen Commercial Development Open Commercial Development is more than publishing the

source code Open, transparent process, from feature requests and

planning through delivery

What can the community members do: Download milestones, try them, and provide feedback on

betas and incubators, including source code Access, Create, and update defects Access milestone and component iteration plans Access the development wiki Participate in discussions on the development

community newsgroups

Example: www.jazz.net

Open Commercial Development is more than publishing the source code Open, transparent process, from feature requests and

planning through delivery

What can the community members do: Download milestones, try them, and provide feedback on

betas and incubators, including source code Access, Create, and update defects Access milestone and component iteration plans Access the development wiki Participate in discussions on the development

community newsgroups

Example: www.jazz.net

OutlineOutline

Closed Development Open Development

2003200320002000 20012001 20022002

JuneMarch JuneJune

20042004

June

20052005 20062006

NovemberFall June

3.12.1 3.02.0 3.21.0ProjectStarts

TechPreview

OpenOpenSourceSourceOpenOpen

SourceSource

20072007

June

3.3

Built to LastExtensibility

API

CommunityTransparency

Development ProcessAgility

TeamTools

20082008

June

3.4

SECR 08

The Eclipse Way Practices

milestonesfirst

APIfirst

endgame

retrospectives

always havea client

continuousintegration

community involvement

new & noteworthy

adaptiveplanning

continuous testing

consume yourown output

componentcentric

drive with open eyes

validate

reduce stress

learn

enable

attract to latest

transparency

validateupdate

dynamic teams

show progress

enable

explore

validate

livebetas

feedback

signoff

common agile practices

common Open Source practices

scaling-up practices

SECR 08

What is behind the Eclipse Way

Practices underpinned with values ship quality on time

Used, developed and improved over time A mix of practices that worked for us

Another mix of practices works for others

Practices are from all kinds of sources XP, Scrum, Crystal Clear, RUP, …

Patterns - Organizational Patterns of Agile Software Development – Coplien

It is not low ceremony Approvals, verifications, reviews

It is agile: incremental, iterative, collaborative, transparent, customizable And it scales up

SECR 08

In the Past…

Time

Effort/

Pain

Level

Look at calendar.

All the time in the world

Heads down

Say goodbye to your loved ones

Exhaustion

SECR 08

Iterative – No hanging rope

fitness

no “hanging rope” stress reduction

3.1 t3.2In the past

SECR 08

Iterative – Time-boxed

endgame

release 3.2

fitness

M1

pla

n

de

velo

p

sta

bili

ze

6 weeks

warm-up

retr

os

pe

cti

ve

init

ial r

ele

ase

pla

n

de

com

pre

ss

ion

3.1

M2

pla

n

de

velo

p

sta

bili

ze

…

pla

n

de

velo

p

sta

bili

ze

sign-offsign-off sign-off

6 weeks 6 weeks

fix

- s

pit

& p

olis

h

test fix

test

SECR 08

Continuous…

It is all about being continuous Continuous iterative and adaptive planning

Continuous design/refactoring

Continuous integration/testing

Continuous delivering/demos

Continuous feedback

Continuous learning

Many effective teams work like this

SECR 08

Key Lessons

Agility can scale up to globally distributed teams

… but it would be nice to have better tool support

SECR 08

Outline

Closed Development Open Development

2003200320002000 20012001 20022002

JuneMarch JuneJune

20042004

June

20052005 20062006

NovemberFall June

3.12.1 3.02.0 3.21.0ProjectStarts

TechPreview

OpenOpenSourceSourceOpenOpen

SourceSource

20072007

June

3.3

Built to LastExtensibility

API

CommunityTransparency

Development ProcessAgility

TeamTools

20082008

June

3.4

SECR 08

But… there are Pain Points…

joining a team get my environment configured to be productive what is happening in my team collecting progress status following the team’s process ad hoc collaboration/sharing of changes starting an ad hoc team

is the fix in the build? what will be in the next build? tracking a broken build Avoid breaking a build/personal build why is this change in the build? reconstructing a context for a bug/build failure

creating, tracking iteration plans interrupting development due to a high priority bug fix working on multiple releases concurrently tracking the code review of a fix referencing team artifacts in discussions how healthy is a component? collecting project data/metrics?

Boring and painful

Collaboration

Development

ProjectManagement

SECR 08

Why are we doing Jazz?

Late 90’s: Focus on Point ToolsLate 90’s: Focus on Point ToolsWho can build the best Java IDE, the best C IDE, the best Web Tool,…

When we built Eclipse: Focus on One DeveloperWhen we built Eclipse: Focus on One DeveloperSeamless integration across a set of tools to improve the productivity of one developer

Today, we must focus on the Team and its CollaborationToday, we must focus on the Team and its Collaboration Geographically Distributed

Accelerated Delivery Demands

Agility with Predictability

Innovation and Repeatability

Increased Need for Transparency

Seamless integration across all the Phases of the Software Lifecycleto improvethe Productivity of the Entire Team.

Seamless integration across all the Phases of the Software Lifecycleto improvethe Productivity of the Entire Team.

SECR 08

Jazz and Team Concert

SECR 08

Team ConcertCollaboration in Context

Team First

Process

Transparency

SECR 08

Collaboration in Context

SECR 08

Collaboration with Context

SECR 08

Scoped builds Personal builds

Continuous team builds

Scheduled integration builds

Collaborate in context on build failures

Collaboration: Builds

SECR 08

Rules/Process

All collaborations have underlying context specific rules

Each project follows a process

Team’s own their process

SECR 08

Process manifested through: artifact types and their states

role-specific preconditions and actions on operations manipulating artifacts

role-specific and permissionsEclipse Way

Scrum

OpenUP

Process

SECR 08

Transparency/Visibility: Dashboards

Viewlets At a glance view on team artifacts

Three dashboard types: project

team

individual user

SECR 08

A Personal Dashboard

SECR 08

A Team Dashboard

SECR 08

Team of Teams Dashboard

SECR 08

Team of Teams Dashboard

SECR 08

Jazz Project Self-Hosting since Oct 2006

Beaverton

•Build•Process

Ottawa

•Source Control•Reporting•Community Site

Zurich

•Work Items•Agile Planning•Code Coverage

Bangalore

•VS Client

Lexington

•Source Control•Requirements•Interop

Raleigh

•Repository•Web UI

Toronto

JazzDevelopment

Server

SECR 08

Summary: Our JourneyEclipse Way

SECR 08

Workshop: 1400-1600

Scaling-up Agility to Globally Distributed Teams – How we build Jazz using Jazz

SECR 08

Try it yourself on www.jazz.net