how to (not) open source - javazone, oslo 2014

Blueflood: A case study in turning a large piece of

infrastructure software into an open source project

Javazone 2014 • Rackspace • @gdusbabek

Blueflood:�How I learned the hardest problems are not technical

Javazone 2014 • Rackspace • @gdusbabek

About Me

Rackspace

About Me

Cassandra

About Me

Monitoring

About Me

Metrics

About Me

Open Source Veteran

About Blueflood

Distributed

About Blueflood

Ingest

About Blueflood

Rollup

About Blueflood

Query

About Blueflood

Started 2012

About Blueflood

Open source 2013

About You

Contemplating releasing a project

About You

Having trouble with open sores

About You

Having trouble with open source

YOU ARE IN THE RIGHT

PLACE

Objectives

Devote resources?

Objectives

Maybe not?

Objectives

Measure success

Objectives

Measure failure

Objectives

History 101 Business 101

BORING!

We made this thing

We did some things right

We did some things wrong

We Learned

Something

We Learned

Some things

OK. What things?

HISTORY 101

Software is relatively new

In the beginning it was ALL open

source

In 1953 the A-2

shipped with all

of its source code

Patches plz

Changed by late 1960s

$$$

1. More Complex

2. Growing Market (Fewer engineers)

Natural Scarcity + More Work To Do ______________________________

Expensive

Protect intellectual property

This trend continues

Not much open source

activity

1991 �

Linux kernel

Modern open source advocacy

Attacked in 1990s

Continued growth well into

2000s

Current State

Open source work is appealing

Good for publicity

Trendy

Hacker News Cycle

1. ▲ Release Something

2. ▲ Post it on HN

3. ▼ See interest in the project

4. ▲ Secondary interest in your business

Y  

Hacker News Cycle

Not conducive to a healthy ecosystem

Y  

So Many Ways to Collaborate

I �♥ �

PULL REQUESTS

I �♥ �

PULL REQUESTS

Stimulating Professional

Conversations 140 Characters

at a time

Mailing Lists IRC

Promote Communities

Releasing a project is easy

Making it successful is very hard

What is “successful”

anyway?

“We reviewed and merged 10

pull requests in our OS project

this week. It was a lot of work. It

was beautiful and awesome!”

“You didn't merge any

features we needed to run our

business this week. I’m going

to eat your family!”

Success is in the eyes of the

beholder

Hipster Open Source

Different ways of releasing a project

Current State

Code Dump (Sea Turtle Approach)

Lay a bunch of eggs, then run off.

Code Dump (Sea Turtle Approach) What you see: •  code •  Probably no wiki

•  Little to no-code activity

•  "Throw it over the wall"

Code Dump (Sea Turtle Approach) What you get: •  Very little community •  Infrequent support

•  No real ownership

worst kind of open source

only adds noise

Primary Sponsorship�

Let someone else sit on our eggs.

One entity does most of the work •  Free for everybody to use •  Good stewards •  Could be a good community and

support •  Not welcoming to outside contributors

Primary Sponsorship (Cuckoo approach)

Incubation

(Elephant Approach)

Incubation

Recruit committers

Build community

Real Work

Incubation

Most difficult Long lasting

Provides for lifecycle Meritocracy

Good News:

None of these are wrong

Bad News:

None of these are wrong

Why is project X open source?

BUSINESS 101

Type 1 It makes money • Profit center • AdWords, Github hosting

Type 2 It doesn't make money • Cost center • Your database • Your queueing software

Reduce the expense of cost centers Increase profitability of profit centers

Nobody will release source for profit centers.

Everybody wants to release the source of their cost centers.

Wrong Mentality

Wrong Mentality

Free doesn’t make it less expensive.

Ok. That was harsh.

(End of business lesson)

What kind of future?

Think about what you really want.

Software should be frictionless.

Easy

CAP Theorem?

My Theorem 1. Easy to use 2. Community 3. Quality

CEQ? QEC?

It probably will not catch on.

10 minute test Too many

projects fail this

Proposition We can improve the

experience of using open source software by

improving the process of creating it.

Our Humble Goal We can improve the

experience of using open source software by

improving the process of creating it.

DO THESE THINGS (7)

1. Understand your motivations

Why are you releasing this

code?

For the recognition?

Don't do it!

You want to help others?

Why?

I dunno.

Don’t do it!

Because it’s good for business

Ok. Do it!

You wan to commoditize something?

That’s interesting.

Very interesting.

STOP!

Offload Others will do the work

Commoditize Reduce the cost for everybody

Yes, do it, definitely!

2. Then focus on execution

Code dump == Bad

Primary sponsorship: Works, but still expensive

for you.

Incubate: Works, but so much

initial effort!

Clearly this is where things get Hard™

3. Define how you will measure success

Can be anything But make it explicit

And write it down Publicly

WARNING: The following contains

controversial opinions. Viewer discretion is advised.

4. Split Your Team

Team Foo: Production Team Bar: Open Source

Case α: Bug in Production

All hands on deck Open source work grinds to a halt

Case β: Big OS Feature

Focus, Focus, Focus Product work becomes a distraction

5. Invest in documentation

More than a day

Ok. At least a week.

Maybe more

Landing Website

Wiki

10 minute guide

And practice the 10 minute guide!

6. Set up communication channels

Mailing lists

IRC

Twitter

RESPOND TO EVERY BIT OF

COMMUNICATION

Black Hole == Bad

Nobody is using it

Reflection of quality

7. Participate & Advocate

Tell your story

Like minds will find you

Difficult for some Just keep practicing

You can get good at it even if

you don’t like doing it.

Wait, what were all those things? 1.  Understand your motivations 2.  Focus on execution 3.  Measure Measure Measure 4.  Split your team 5.  Document all the things 6.  Communicate with all the things 7.  Participate & Advocate

How will we know when we’re there?

Our Humble Goal Change the experience

of using software by improving the process of

creating it.

Evidence of Success What will our success

look like? How shall we measure it?

Evidence of Success

When open source no longer hinders us

When you and others are able to leverage open source projects to run your business with: – More speed – Less resources – Greater focus

Evidence of Success

Evidence of Success: A Casualty

Fewer open source projects.

Evidence of Success: A Casualty

What about the bazaar?

Healthier Ecosystems

Evidence of Success

Easily identified lifecycle

Young: growing & changing

Mature: less risk

Evidence of Success

You know how to engage the community and support channels

Evidence of Success

Evidence of Success Your split team does not feel compromised or stretched They aren't riding two horses Your leadership is comfortable with this.

This all sounds really good, but…

Return on investment Staffing engineers who work on

100% on open source

Perception: expensive cost centers

One Approach: Change that perception

Dubious proposition

The Problem: This will probably not last

long.

Not a good approach.

Another Approach: Skunkworks

Split your team, but don’t tell

anybody.

Sneaky, but that doesn’t prove ROI.

Measure Everything

Prove that that things are better Use science!

Support tickets

Monitoring alerts

Days spent fixing bugs

Prepare for unexpected results Measurements may indicate that

you’re wasting time or other resources.

Be honest with yourself

Measure, Measure, Measure.

History 101 + Business 101 _______________ Undeniable Insight

• Things are changing like they never have • Commoditization of software + hardware • Accelerating pace

• Become adept • Less friction

• More time

• Time is money

• Solid return on investment

Observed Trend! The Open Source Startup Model

Venture Backed

Great supporters of open source

ROI easily justified

Good thing right?

It’s the least bad thing.

Product of prevailing attitudes in non VC companies

1.  Contributing to open source is a drain on

resources 2. Companies are being short sighted

The Choice Is Ours

Small Steps

Consider carefully your choice to release

a project.

Hold yourself accountable

THANKS!

@gdusbabek

Some images required attribution

boring https://www.flickr.com/photos/phoenixdailyphoto/1467681879 professor https://www.flickr.com/photos/judy-van-der-velden/4468986943 tank https://www.flickr.com/photos/defenceimages/14465175042 tools https://www.flickr.com/photos/75905404@N00/7126147125 businessmen https://www.flickr.com/photos/sgis/6532363 rainbow https://www.flickr.com/photos/liesforaliar/6057583336 trash https://www.flickr.com/photos/en321/14611573210 skunk https://www.flickr.com/photos/i_am_just_a_cloud/1079804050 inspire https://www.flickr.com/photos/mikeyphillips/14654963066