crunching the numbers: open source community metrics
DESCRIPTION
Every community manager knows that community metrics are important, but how do you come up with a plan and figure out what you want to measure? Most community managers have their own set of hacky scripts for extracting data from various sources after they decide what metrics to track. There is no standardised Community Software Dashboard you can use to generate near-real-time stats on your community growth.Like most open source projects, we have diverse community infrastructure for MeeGo, including Mailman, Drupal, Mediawiki, IRC, git, OpenSuse Build Service, Transifex and vBulletin. We wanted to unify these sources together, extract meaningful statistics from the data we had available to us, and present it to the user in a way that made it easy to see if the community was developing nicely or not.Building on the work of Pentaho, Talend, MLStats, gitdm and a host of others, we built a generic and open source community dashboard for the MeeGo project, and integrated it into the website. The project was run in the open at http://wiki.meego.com/Metrics/Dashboard and all products of the project are available for reuse.This presentation will cover the various metrics we wanted to measure, how we extracted the data from a diverse set of services to do it, and more importantly, how you can do it too.TRANSCRIPT
![Page 1: Crunching the numbers: Open Source Community Metrics](https://reader034.vdocument.in/reader034/viewer/2022042614/5551458db4c905bd1c8b4f72/html5/thumbnails/1.jpg)
1
Crunching the numbers:Open Source Community Metrics
Photo CC, by aussiegal on Flickr
Dave Neary <[email protected]>Neary Consulting
Dawn Foster <[email protected]>MeeGo Community Manager at Intel
![Page 2: Crunching the numbers: Open Source Community Metrics](https://reader034.vdocument.in/reader034/viewer/2022042614/5551458db4c905bd1c8b4f72/html5/thumbnails/2.jpg)
2
Why are metrics important?
● Measure progress in your community over time● Who contributes● Where are people contributing● Spot trends● Gauge interest● Learn more about key contributors● Recognize contributions
![Page 3: Crunching the numbers: Open Source Community Metrics](https://reader034.vdocument.in/reader034/viewer/2022042614/5551458db4c905bd1c8b4f72/html5/thumbnails/3.jpg)
3
Example: Components of an Open Source Community
http://www.flickr.com/photos/korou/2586472234
![Page 4: Crunching the numbers: Open Source Community Metrics](https://reader034.vdocument.in/reader034/viewer/2022042614/5551458db4c905bd1c8b4f72/html5/thumbnails/4.jpg)
4
Example: June MeeGo Community Metrics Summary
• 439,963 unique people visited this month (430,070 last month).– 1,796,288 pageviews (1,857,907 last month); 599,573 total visits (597,658 last month)
• 25,801 people are members of MeeGo.com (was 24,019 last month)– Dev ML subscribers = 4772; Community = 3736; iL10N = 2607; SDK = 3197
• Mailing Lists: 5890 posts this month; 200 people posted 2+ msgs
• Forums: 2255 posts. 242 people posted 2+ messages
• New Bugs Created: 1938; Bugs Resolved: 2871
• Downloads: 52,627 1.2 Netbook, 626 1.2 IVI, 1505 1.1 N900, 3072 Tablet
• Active Users: Estimated at 800 – 1000 people.– Mailing Lists: 302 people with unique email addresses posted (308 last month)
– Forums: 412 people posted at least one item (262 last month)
– Bugzilla: 773 people performed some action (624 last month)
– Wiki: 276 edited the wiki (205 last month)
– IRC: 410-520 people logged into #meego simultaneously most days
http://wiki.meego.com/Metrics
![Page 5: Crunching the numbers: Open Source Community Metrics](https://reader034.vdocument.in/reader034/viewer/2022042614/5551458db4c905bd1c8b4f72/html5/thumbnails/5.jpg)
5
What are the Right Metrics for YOUR Project
● Goals● What are your overall goals for the project?● How can you measure progress toward those goals?● What is important to you and your progress?
● Trends● What should you measure to recognize trends?● How do you recognize when something is going wrong?● Do you notice big improvements?
Note: I measure way too muchhttp://www.flickr.com/photos/bandfan/5548675317/
![Page 6: Crunching the numbers: Open Source Community Metrics](https://reader034.vdocument.in/reader034/viewer/2022042614/5551458db4c905bd1c8b4f72/html5/thumbnails/6.jpg)
6
Mailing Lists: mlstats
Mailing List Stats is a command line tool used to analyze mailing list archives. It downloads the archives, places
them in a directory and stores all the information contained in each mailing list post into a database
http://libresoft.es/tools/mlstats
![Page 7: Crunching the numbers: Open Source Community Metrics](https://reader034.vdocument.in/reader034/viewer/2022042614/5551458db4c905bd1c8b4f72/html5/thumbnails/7.jpg)
7
Code contributions: cvsanaly
CVSAnaly parses commit logs of various source control systems, and stores the data into a database
http://libresoft.es/tools/cvsanaly
![Page 8: Crunching the numbers: Open Source Community Metrics](https://reader034.vdocument.in/reader034/viewer/2022042614/5551458db4c905bd1c8b4f72/html5/thumbnails/8.jpg)
8
Code contribution: gitdm
Gitdm parses git logs and extracts committer and employer information
git://git.lwn.net/gitdm.git
![Page 9: Crunching the numbers: Open Source Community Metrics](https://reader034.vdocument.in/reader034/viewer/2022042614/5551458db4c905bd1c8b4f72/html5/thumbnails/9.jpg)
9
IRC: irssistats & superseriousstats
Generates IRC stats for active people, by hour of the day, by day, most used words, quotes and more.
http://royale.zerezo.com/irssistats
http://code.google.com/p/superseriousstats/
![Page 10: Crunching the numbers: Open Source Community Metrics](https://reader034.vdocument.in/reader034/viewer/2022042614/5551458db4c905bd1c8b4f72/html5/thumbnails/10.jpg)
10
Media Wiki
![Page 11: Crunching the numbers: Open Source Community Metrics](https://reader034.vdocument.in/reader034/viewer/2022042614/5551458db4c905bd1c8b4f72/html5/thumbnails/11.jpg)
11
Bugs
● New bugs vs. resolved bugs● Can't just look at monthly trends● Need to take release cycle into account● Before release: more resolved bugs● After release: more new bugs
● Participants● People who file new bugs● Participate in bugs (comment, etc.)● Careful with people who resolve bugs (usually QA)
● Most bug trackers have reporting / queries for metrics
Image: http://www.thegeekstuff.com/2010/05/install-bugzilla-on-linux/
![Page 12: Crunching the numbers: Open Source Community Metrics](https://reader034.vdocument.in/reader034/viewer/2022042614/5551458db4c905bd1c8b4f72/html5/thumbnails/12.jpg)
12
Website: Google Analytics
Nokia Announcement
Roadmaps Mobile World Congress
![Page 13: Crunching the numbers: Open Source Community Metrics](https://reader034.vdocument.in/reader034/viewer/2022042614/5551458db4c905bd1c8b4f72/html5/thumbnails/13.jpg)
13
Automate
● My less than elegant method● Giant bash script● Uses wget, awk, mysql queries, etc.● Dumps a bunch of csv files on my hard drive
● A better dashboard approach● Open source metrics dashboard ● http://wiki.meego.com/Metrics/Dashboard
![Page 14: Crunching the numbers: Open Source Community Metrics](https://reader034.vdocument.in/reader034/viewer/2022042614/5551458db4c905bd1c8b4f72/html5/thumbnails/14.jpg)
14
Why build a dashboard?
● Provide up to date information● Automate repetitive menial work● Provide actionable data for qualitative analysis
![Page 15: Crunching the numbers: Open Source Community Metrics](https://reader034.vdocument.in/reader034/viewer/2022042614/5551458db4c905bd1c8b4f72/html5/thumbnails/15.jpg)
15
● Tracks total contributions, code review, individual contributions
● Easy to spot important information in visualisation
● Code only
Mozilla contributor dashboard
Credits: David Eaves
![Page 16: Crunching the numbers: Open Source Community Metrics](https://reader034.vdocument.in/reader034/viewer/2022042614/5551458db4c905bd1c8b4f72/html5/thumbnails/16.jpg)
16
WikiMedia stats
Tracks editor activity, content pages, access by language
http://stats.wikimedia.org
![Page 17: Crunching the numbers: Open Source Community Metrics](https://reader034.vdocument.in/reader034/viewer/2022042614/5551458db4c905bd1c8b4f72/html5/thumbnails/17.jpg)
17
● Company contributions● Total activity● Mostly code
Eclipse project view
![Page 18: Crunching the numbers: Open Source Community Metrics](https://reader034.vdocument.in/reader034/viewer/2022042614/5551458db4c905bd1c8b4f72/html5/thumbnails/18.jpg)
18
Ohloh
http://www.ohloh.net/p/gimp/contributors
![Page 19: Crunching the numbers: Open Source Community Metrics](https://reader034.vdocument.in/reader034/viewer/2022042614/5551458db4c905bd1c8b4f72/html5/thumbnails/19.jpg)
19
Paul Adams: Visualising contributions
http://blogs.fsfe.org/padams/?p=18
![Page 20: Crunching the numbers: Open Source Community Metrics](https://reader034.vdocument.in/reader034/viewer/2022042614/5551458db4c905bd1c8b4f72/html5/thumbnails/20.jpg)
20
● Pentaho BI suite● JasperSoft
● BI suites, including ETL, reporting, metadata● BIRT
● Reporting tool from Actuate● Associated with Eclipse
● Talend● Petals
● ETL tools, useful in combination with other tools
Open Source business intelligence
![Page 21: Crunching the numbers: Open Source Community Metrics](https://reader034.vdocument.in/reader034/viewer/2022042614/5551458db4c905bd1c8b4f72/html5/thumbnails/21.jpg)
21
● Open Source BI suite● Includes reporting tool, ETL, dashboard framework,
scheduling engine, metadata layer, all in Community Edition
● Used by other open source projects (Mozilla)
MeeGo Dashboard & Reports
![Page 22: Crunching the numbers: Open Source Community Metrics](https://reader034.vdocument.in/reader034/viewer/2022042614/5551458db4c905bd1c8b4f72/html5/thumbnails/22.jpg)
22
Pentaho Report Designer
![Page 23: Crunching the numbers: Open Source Community Metrics](https://reader034.vdocument.in/reader034/viewer/2022042614/5551458db4c905bd1c8b4f72/html5/thumbnails/23.jpg)
23
Mailing list and forum statistics
![Page 24: Crunching the numbers: Open Source Community Metrics](https://reader034.vdocument.in/reader034/viewer/2022042614/5551458db4c905bd1c8b4f72/html5/thumbnails/24.jpg)
24
Mailing list and forum activity
![Page 25: Crunching the numbers: Open Source Community Metrics](https://reader034.vdocument.in/reader034/viewer/2022042614/5551458db4c905bd1c8b4f72/html5/thumbnails/25.jpg)
25
Wiki activity
![Page 26: Crunching the numbers: Open Source Community Metrics](https://reader034.vdocument.in/reader034/viewer/2022042614/5551458db4c905bd1c8b4f72/html5/thumbnails/26.jpg)
26
IRC stats
![Page 27: Crunching the numbers: Open Source Community Metrics](https://reader034.vdocument.in/reader034/viewer/2022042614/5551458db4c905bd1c8b4f72/html5/thumbnails/27.jpg)
27
Now What?
● Report Regularly● Pick a period that makes sense
for your project● Share with the community
● http://wiki.meego.com/Metrics● Recognize your top contributors● Use data to take action
http://www.flickr.com/photos/play4smee/2439494411/
![Page 28: Crunching the numbers: Open Source Community Metrics](https://reader034.vdocument.in/reader034/viewer/2022042614/5551458db4c905bd1c8b4f72/html5/thumbnails/28.jpg)
Dawn FosterMeeGo Community Manager for Intel
@geekygirldawn
Dave NearyNeary Consulting
@nearyd
![Page 29: Crunching the numbers: Open Source Community Metrics](https://reader034.vdocument.in/reader034/viewer/2022042614/5551458db4c905bd1c8b4f72/html5/thumbnails/29.jpg)
29
Credits
Thank you to the many people who have contributed to the metrics● Dave Neary for many helpful tips & for providing a lot of
help with mailing list stats (mlstats)● Reggie Suplido for automating forum stats:
http://forum.meego.com/stats/● Carsten Munk for the IRC stats● Stephen Gadsby for the bug jars:
http://www.octofish.net/meegobugjar/● Mike Shaver for a variety of help● Arjan Van De Ven for some Perl magic● Adam Gretzinger for providing download data