Open Source By The Numbers

Rich SandsDirector of Developer Communities

Black Duck Software, Inc.

How Big is FOSS?

• GitHub: 4,751,000 repositories• SourceForge: 324,000 projects• Ohloh: 550,000 projects

BIG

No, REALLY, How Big is FOSS?

• It depends on how you count.• Lots of projects, but– How many are active, how many

abandoned?– How many have a team?

A better question to ask:How much FOSS is actually being

worked on now?

How Many Projects are Active?

• 550,000+ projects on Ohloh.• 271,372 with a code analysis.• 96,824 with a commit in the past 2 years.• 46,883 with a commit in the past year.• 29,303 with a commit in the past 6 months.• 21,251 with a commit in the past 3 months.• 12,870 with a commit in the past month.• 5,629 with a commit in the past week.• 1,224 with a commit in the past day (3/30-

3/31, a weekend)

How Many Projects Are Active?D

ays

Sin

ce L

ast

Com

mit

1000

2000

3000

4000

5000

6000

100 90 80 70 60 50 40 30 20 10

% of Analyzed Projects With a Commitin the last Y Days

1 Yr

17.3%

But Do All These Projects Have a Team?

100 90 80 70 60 50 40 30 20 10

% of Active Projects WithAt Least Y Committers

Nu

mb

er

of

Com

mit

ters

10

20

30

40

50

2827

2

2 or more49.3% 8.5% of all analyzed projects

What is a “Live” Project, Anyway?

• Lets invent a new metric – “Liveness”:– At least one commit in the last year, and at least

2 committers for liveness to be non-zero.– Time-weighted roll-up of activity, where older

activity counts less than more recent activity.– For this presentation, activity is committer count.– Exponential time-weighting decay such that the

most recent month’s activity counts fully, and 11 months back activity counts nearly zero.

– Normalized; liveness of the Linux Kernel = 1000.

Sniff Test – What Are theTop 50 Live Projects?

1000.00 Linux Kernel711.40 Chromium (Google Chrome)516.68 KDE491.68 Mozilla Firefox491.37 Mozilla Core473.17 Boot To Gecko396.51 GNOME322.54 Homebrew319.47 Gentoo Linux300.32 WebKit273.38 Qt 5226.36 FreeBSD Ports194.87 OpenStack163.50 docrails163.19 Ruby on Rails159.54 Android155.82 LibreOffice154.54 MediaWiki146.83 FreeBSD145.55 GNU Compiler Collection129.94 FFmpeg124.62 OpenERP123.33 SBo-git123.33 SlackBuilds.org118.32 OpenStack Nova

118.32 openstack's nova117.52 The LLVM Compiler Infrastructure115.64 llvm-mirror115.20 NetBeans IDE114.01 JBoss Application Server113.96 NetBSD112.73 JBossAS7112.73 JBoss Application Server 7109.69 Jenkins109.26 U-Boot108.83 Go programming language107.60 tav's go105.34 QEMU103.62 pkgsrc: The NetBSD Packages Collection101.50 platform_frameworks_base101.39 Trinity Core100.86 LLVM C/Objective-C/C++ frontend (old)100.86 LLVM/Clang C family frontend100.16 Symfony95.76 WSO2 Business Process Server94.23 Intellij Community90.95 Wine89.46 Qt 489.01 XBMC Media Center88.39 Chromium Tools (Google Chrome)

Note – there are a few duplicates and mirrors in this list

How Big Are Live Projects?

0 1000500 750250

10M

20M

30M

40M

50M

60M

Lin

es

of

Cod

e

Liveness (0-1000 scale)

For most projects,bigger means

less activeLinux Kernel

Git

Chromium

KDE

FirefoxGNOME

Android

LibreOffice

AndroidPlatform Frameworks Base

Aptosid (Debian distro)No really big,really active

projects

“Famous”projects

“Distros”

FreeBSD

GCC

WebKitQt

Ruby on Rails

MySQL

Top 5000 live projects

How Does Size Relate to Committer Count?

10M

20M

30M

40M

50M

60M

Lin

es

of

Cod

e

1-Year Committer Count

0 300020001000

Linux Kernel

GNOMEChromium

KDE

Firefox

Ruby on Rails

Android

LibreOffice

QtGCC

Similar effect – largermeans fewer committers

Top 5000 live projects

Languages of Live Projects

Other

C

C#

C++

Java

JavaScript

Perl

Python

PHP

Ruby

Top 5000 live projects

Average Liveness By Language

C

C++

PHP

Ruby

Java

JavaScript

Perl

Python

C#

0 2 4 6 8 10 12 14 16 18

Top 5000 live projectsLiveness

Average Project Size By Language

C

C++

PHP

Ruby

Java

JavaScript

Perl

Python

C#

0 1 2 3 4 5 6 7 8

Millions of lines of codeTop 5000 live projects

Language vs. Number of Committers

C C++ PHP RubyJava JavaScript PerlPython C#1000

10000

100000

All-time Committers 1 Year Committers 30 Day Committers

Top 5000 live projects

Tota

l C

om

mit

ters

Languages of New Projects – Then and Now

CC++ PHP RubyJava JavaScript PerlPython C#Other0%

5%

10%

15%

20%

25%

30%

Started 5 Years Ago Started in the Past Year

% N

ew

Pro

ject

s Pri

mary

Langu

age

The 8 Most Live New Projectsin the Past Year

Project Description Why So Active?

oVirt-engine KVM Management System(Liveness: 50.5, 457K LOC, Java)

Open governance, backed by Cisco, Red Hat, IBM, Canonical, Intel, ... and aims at a burning problem.

WebRTC Implements W3C RFC for streaming media JavaScript API(Liveness: 47.3, 407K LOC, C++)

Supported by Google (Chrome), Mozilla, and Opera, a core HTML5 streaming media std.

Khan Academy Exercises

Crowdsourced exercises for a self-service educational platform.(Liveness: 44.3, 90K LOC, JavaScript)

Very easy to contribute, taps altruistic impulses of educators worldwide.

Twitter Bootstrap CSS, HTML, JavaScript toolkit for rapid webapp development.(Liveness: 40.0, 41K LOC, JavaScript)

Heavily promoted by Twitter, high-quality, aims at a burning problem.

Wikimedia Puppet Wikimedia’s Puppet configuration.(Liveness: 33.8, 37K LOC, Puppet)

Exemplary Puppet implementation by for a very heavily trafficked site.

Katello RHEL server system lifecycle mgmt.(Liveness: 31.6, 137K LOC, Ruby)

Announced at Red Hat 2011 Summit, follow-on to Satellite project.

Cloud Foundry VMWare’s PaaS platform.(Liveness: 30.7, 29K LOC, Ruby)

Substantial industry support, marketing. Aims at a burning problem.

Composer Package manager for PHP.(Liveness: 30.3, 14K LOC, PHP)

Aims at a burning problem for PHP.

Open Source by the Numbers

• Only a small fraction of all the projects ever started gain long-term traction.

• Less than 5% of all projects on Ohloh are “live”: a commit in the past year, and more than 1 committer, ever.

• The larger the code base, the less contributors and activity.

• “Famous” projects are mostly Java and C-family, and these older, established languages retain their dominant mindshare.

• New live projects trending towards Python, PHP, JavaScript and away from C-family languages.

• The “most likely to succeed” new projects:– Have big backers and marketing behind them.– Are still small enough for people joining them to have an impact.

Questions?

Ohloh.net Your guide to open source

Join the Ohloh community and gain critical insights into the world of open source projects