Investigating the Dynamics ofFLOSS Development Teams Investigating the Dynamics ofFLOSS Development Teams

Peo

ple

Peo

ple not shown:

Andrea

Wiggins

Keisuke Inoue

Sarah Harwell

Steven Rowe

Nancy

McCracken

What are the dynamics through which self-organizing, distributed teams develop and work?Research partially funded by NSF grant 05–27457, with prior support from 03–41475 & 04–14468

Na LiQing Li

Kangning Wei

Robert Heckman

U. Yeliz Eseryel

Elizabeth Liddy

James Howison

Kevin Crowston

Eileen Allen

Michael Scialdone

Key

Find

ings

To D

ate

Key

Find

ings

To D

ate

Cita

tions

Cita

tions

Part

ners

Part

ners

• Barbara Scozzi, Politecnico di Bari, Italy• Megan Conklin, Elon University, North Carolina

Scozzi, B., Crowston, K., Eseryel, U. Y., & Li, Q. (Accepted). Shared mental models among open source software developers. Hawai'i International Conference on System Science, Big Island, Hawai'i, Jan 2008.

Crowston, K., Wei, K., Li, Q., Eseryel, U. Y., & Howison, J. (2007) Self-organization of teams in free/libre open source software development. Information and Software Technology Journal, Special issue on "Understanding the Social Side of Software Engineering, Qualitative Software Engineering Research", vol. 49, p. 564. Published.

Heckman, R., Crowston, K., Eseryel, Y., Allen, E., & Li, Q. (2007) Emergent Decision-making Practices in Free/Libre Open Source Software (FLOSS) Development Teams. Third International Conference on Open Source Systems, IFIP 2.13 Working Conference, Limerick, Ireland. Published.

Heckman, R., Crowston, K., Misiolek, N. & Eseryel, Y. (2007) Emergent leadership in self-organizing virtual teams. Proceedings of the International Conference on Information Systems (ICIS 2007), Montreal, Quebec, Canada, Dec 2007. Accepted with revisions.

Crowston, K., Conklin, M. & Howison, J. (2007) A Proposed Data and Analysis Archive for Research on Free and Open Source Software and Its Development. Third International Conference on e-Social Science. Accepted as poster.

Crowston, K. (2007) The bug fixing process in proprietary and free/libre open source software: A coordination theory analysis, bibl. M. E. Sharpe, Inc, Armonk, NY. Book Published of Collection: Grover, V. and Markus, M. L., Business Process Transformation.

Plus 6 publications from 2006

Plan

sPl

ans

• Shared Mental Models: Transcripts of 27 interviews with members of the open-source community are being analyzed through cognitive mapping, and the integration of Natural Language Processing is being considered. Results to date indicate a degree of sharing of mental models, and we are looking for casual links within the community.

• Leadership: A detailed coding scheme has been developed that considers the concepts of first and second order leadership, including respective indicators. This code book is currently in the process of validation and the subsequent content analysis is planned for inclusion in the doctoral dissertation of one of our graduate assistants.

• Group Maintenance: A detailed coding scheme drawing on politeness theory, organizational citizenship behavior, and social presence theory literature has been developed. The code book is being validated and refined through a pilot analysis on episodic data from two open-source teams. Once finalized, our content analysis will continue on additional episodic data.

• We have developed National Language Processing (NLP) text extraction rules to reliably identify 63-72% of decision triggers, and 71-85% of decision announcements.

• There is a positive relationship between the level of participation and group effectiveness, with effective projects exhibiting high involvement from peripheral members.

• Some open-source projects that appear decentralized are actually centralized, but with individuals at the center changing their roles over time.

• In studying shared-mental models, norms and rules are highly shared within groups. However, it seems that tenure in projects can cause those to change within individuals, indicating dynamic processes over time.

• Second-level leadership in open-source teams is enabled by first-level leadership. First-level includes task coordination and contribution, group maintenance, and boundary spanning; while second-level is behavior that influences changes in structure that guides group action.

Multiple incompatibleoverlapping repositories

Projectselection

Datacollection

Datacleaning Data storage

Analysis

FLOSSMole

Publication

Prod

ucts

Prod

ucts

• floss.syr.edu website with copies of papers and analyses• FLOSSmole, a collaborative project to gather, share and store comparable

data and analyses of free and open source software development for academic research.

2Natural language processing for content analysis

After preprocessing, manual processes were automated by using manual coding as the Gold Standard• IM projects data were analyzed for decision announcements and triggers with the sentence as a unit of analysis • Manual coding was validated by 2 or 3 coders, and used as training data for rule-writing and evaluation of results• Rules utilize NLP features such as morphology, word form, syntax, semantics, discourse, and pragmatics

Automating manual coding processes

Automatic Coding - Decison Episodes

Fire

aMSN : doc E01.NT5126frametype = entity trigger =   Today I received aproposal for a new logo for the website.

aMSN

GAIM: docNT217frametype = entityDecision-Announcement = I disagree ( which is why Icommitted the change after KingAnt suggested it ) .

Decision Triggers

DecisionAnnouncements

1-19 14:51:41"

o-Realname="Steven Garrity"

Tue, 2004-08-17 at 17:15, Steven Garrity wrote:">

f Retail on Ink & Toner - Free Shipping and Free Gift.

--

Sectionsidentified

Paragraphsseparated

Metadataidentified

<?xml version="1.0"?><DOC><TEXT><SECTION type="email-message" List-Id="24" Msg-Id="9259245" Post-Id="3744"

Thread-Id="1285" Child-Of="3741" Depth="3" Post-Date="2004-08-18 08:30:00" Posters-Realname="Matthew Keller" Date-Collected="2005-1 Datasource-Id="26" Replying-T Message-Subject="Re: Gaim Sounds - First Draft"> <SECTION type="from">Matthew Keller</SECTION> <SECTION type="replying-to">Steven Garrity</SECTION> <SECTION type="subject">Re: Gaim Sounds - First Draft</SECTION> <SECTION type="date">2004-08-18 08:30:00</SECTION>

I think it goes without saying that your sounds are better than the defaults. I doubt anyone who's checked them out would disagree.

Then again, I think that llama's bleating would be better on the ears than the default AOLish sounds, but that's not to diminish the sounds you and Brad have produced.

<SECTION type="quote" level="1" attribution="On <SECTION type="quote" level="2" attribution="Tyler Lane wrote:"> I totally love the sounds. Great work. However, after a couple days of using them, I find that it is occasionally hard to distinguish the sounds with music playing in the background. Perhaps something with a higher pitch would make it easier to hear the sounds with music playing. </SECTION>

Glad to hear you like them Tyler. They are someone low in volume. We could probably boost the levels for final versions.

Other opinions? I though we'd been swamped! :-)

Steven

<SECTION type="sourceforge-signature"> -------------------------------------------------------

SF.Net email is sponsored by Shop4tech.com-Lowest price on Blank Media 100pk Sonic DVD-R 4x for only $29 -100pk Sonic DVD+R for only $33 Save 50% of http://www.shop4tech.com/z/Inkjet_Cartridges/9_108_r285 ----------------------------------------------- Gaim-devel mailing list Gaim-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/gaim-devel </SECTION> </SECTION> <SECTION type="tagline">

Matthew Keller signat-url: http://mattwork.potsdam.edu/signat-url/ "No one ever says, "I can't read that ASCII E-mail you sent me."" </SECTION></SECTION></TEXT></DOC>

Preprocessed message in xml format

• Identification ofmajor sections ofan email message,eg header,signature, tagline

• removal of “>”symbols thatinterfere withsentence detection

• identification ofquote levelsembedded inmessages andauthor of thequoted snippets.

All - analysis by each marked sentenceGroup - analysis by each decision group of sentencesSM - software modification decision episodesSt - strategic decision episodesDA - Decision Announcements

Key Metrics

Precision - % marked that agree with Gold StandardRecall - % of Gold Standard that are markedUtility - % marked that assist researcher in understanding Decision Episodes

Results on IM Projects

All

Gro

ups

SM St

0%20%40%60%

80%100%

Precision - DA

All

Gro

ups

SM S t

0%20%40%60%80%

100%

Precision - Triggers

All

Gro

ups

SM St

0%20%40%60%80%

100%

Recall - DA

All

Gro

ups

SM S t

0%20%40%60%80%

100%

Recall - Triggers

GaimaMSNFire

All

Gro

ups

SM St

0%20%40%60%

80%100%

Utility - DA

All

Gro

ups

SM S t

0%20%40%60%80%

100%

Utility - Triggers

Findings: The more successfulprojects exhibited higherperipheral involvement in

their decision making processesthan the less successful projects.

Decision making processes in FLOSS teamsWe analyzed the email listservs and fora for 6 FLOSS project teams using Decision Episodes as the unit of analysis.The analysis data consist of 360 Decision Episodes from the beginning, middle, and end periods of email interactions.1We analyz

analys

Linear

Loop-back2.3%

NormativeNormative

28.7%

22.5%

30 %

25 %

20 %

15 %

10 %

5 %

0 %

No decision

Short-cut

No development

No selection

All phases

webERPOFbizCompiereIndividualEnterprise Resource PlanningSystem (ERP)

FireaMSNGaimCompanyInstant Messaging Client (IM)

More Successful--->Less Successful

Team SuccessIntended UsersProduct ComplexityCase Selection

Coding Scheme IDecision Episode

TT DD DD DD DADA

Descriptive StatisticsDescriptive Statistics•Length of Episode (N of messages)• Duration of Episodes (N of days)•N of Participants (Admin, Developer, User)

DecisionEpisode

•Trigger Type•(Bug, Feature, Problem…)•Triggering Person•(Admin, Developer, User)

•Decision Announcement Type•(Code change needed, Code change committed)•Decision Announcer•(Core [Admin, Developer], Periphery [User])

•Decision Episode Type•(Software-modification*;Strategic )

I-IdentificationI-Identification II-DevelopmentII-Development III-SelectionIII-Selection IV-Announcement IV-Announcement

1a:decisionrecognition1b:diagonosis

2a:solution analysis2b:solution search2c:solution design

3a-1: solution evaluation-opinion 3a-2:solution evaluation-action3b: solution confirmation

4:decisionannouncement

Unit of Coding:Functional Move

No decision 9.3%

Short-cut 28.3%

No Development 0.8%

No Selection 30.6%

All phases 31%

Identification Development Selection Announcement

Identification Development Selection AnnouncementLinear (41.9%)

Loop-back (59.1%) Identification Development Selection Announcement

Decision Styles

Coding Scheme II

Normative

Decision Making

Process

0

1020

30

4050

60

Autocratic/Autonomous

Independent/Autonomous

Deliberate/Autonomous

Democratic/Autonomous

TrulyCollaborative/Consensus

Gaim

aMSN

Fire

010

203040

5060

Autocratic/Autonomous

Independent/Autonomous

Deliberate/Autonomous

Democratic/Autonomous

TrulyCollaborative/Consensus

Compiere

OFbiz

WebERP

Level ofsuccess

IM

ERP

Level ofsuccess

Findings: Generally, the decision making processes

in the open source projects are not normative.

We are reviewing the research on FLOSS development and use in an effort to assess the state of the literature. We have reviewed 586 papers in the first round, which were collected from Web of Science, ABI/Inform, journal special issues, AOM and AIS conferences, IntOSS (IFIP) conferences, ICSE workshops and opensource.mit.edu.

We are currently organizing the review around an input-mediator-output-input (IMOI) model. The 295 papers that were found both empirical and relevant are being categorized into issues pertaining to

- inputs (member characteristics, technology use and project characteristics)- processes (social processes, software development processes and IT-related processes)- emergent states (social states, task related states and IT-related states)- output (team performance and FLOSS implementation)1reviewing

ewes, AO

ntly und bmemes (sont sta

IMOI model of FLOSS literature review

Socialization: What processes to members go through in becoming a part of an open-source team, and how are different levels of membership defined?

Decision Making: How are decisions made in matrix environment such as open-source teams where leadership is not always clearly defined?

Leadership Building: How is leadership defined in open-source teams, and how do members rise into such positions?

Coordination/Collaboration: What are the dynamics of interaction within open-source teams where roles are not always clearly defined?

Team Maintenance: How do open-source teams form, and what keeps these teams together?Knowledge Management: Where do open-source teams store their group knowledge, and how is it

accessed by members?Trust: How is trust formed within open-source teams?Conflict: What types of conflicts arise within open-source teams, and how are such conflicts resolved?Level of Commitment: Considering the often voluntary nature of open-source teams, what amounts of

commitment do members invest?

Process

OutputTeam PerformanceMeasure

Softwareimplementation

Input

Member characteristics

Demographics,company, discontinuities,motivation, skills

Technology use

Types of technology used, technology features

Project characteristics

Software type, license types

Software development process

Planning, requirements, coding, release, maintenance, project management

IT-related process

Technology collaboration

Emergent States

Social statesTrust, cohesion, conflict,commitment level, interaction structure

IT-related states

Shared norms of IT use

Task-related states

Member roles, individual contributions, development status, shared understanding, evolution of the software

Social processesSocialization, decision making, leadership building, coordination/collaboration, team maintenance, knowledge management

Engagement with FLOSS community advisory boardRese

arch

Pla

nRe

sear

ch P

lan

2006 2007 2008

Manual examination of transcripts to locate

evidence of concepts

Refining applicationof NLP techniques

Utilizing NLP , SNAand Code metrics on large sample

Specialization of NLPtechniques for novel text

types (e.g., chat logs)

Exploration of dynamic Social Network Analysis

techniques

Manual refiningof identified

features according to conceptual framework

Extraction of code and SNA metrics from

growing sample

Clustering of projects through optimal pattern

matching of timelines

Comparison of dynamics across

projects to provide

generalizableknowledge

1

2

Demographicdata

Source Code

Interactionlogs

Plans and procedures

Manual and NLP-based

contentanalysis

SNA

Code metrics

Cognitivemaps

Roles

Norms and rules

Contribution and coordination

Processmaps

Event and developerdynamics

Raw data

Stage 1

analysis

Stage 2

analysis

Stage 3

analysis

Teamoutcomes

Ana

lysi

s Pl

anA

naly

sis

Plan

Data IntegrationData Integration

Raw DataRaw Data

<?php/** * abook_local_file.php * * Copyright (c) 1999-2005 The SquirrelMail Project Team * Licensed under the GNU GPL. For full terms see the file COPYING. * * @version $Id: abook_local_file.php,v 1.35 2005/03/20 10:06:45 tokul $ * @package squirrelmail * @subpackage addressbook */

/** * Backend for address book as a pipe separated file * Stores the address book in a local file * * An array with the following elements must be passed to * the class constructor (elements marked ? are optional): *<pre> * filename => path to addressbook file * ? create => if true: file is created if it does not exist. * ? umask => umask set before opening file. * ? name => name of address book. *</pre> * @package squirrelmail */class abook_local_file extends addressbook_backend {…<?php/** * abook_local_file.php * * Copyright (c) 1999-2005 The SquirrelMail Project Team * Licensed under the GNU GPL. For full terms see the file COPYING. * * @version $Id: abook_local_file.php,v 1.35 2005/03/20 10:06:45 tokul $ * @package squirrelmail * @subpackage addressbook */