2010 esem - an empirical study on the relationship between the use of agile practices and the...
DESCRIPTION
An Empirical Study on the Relationship between the Use of Agile Practices and the Success of Scrum Projects A. César C. França, Fabio Q. B. da Silva, Leila M. R. de Sousa Mariz Abstract-In this article, factors considered critical for the success of projects managed using Scrum are correlated to the results of software projects in industry. Using a set of 25 factors compiled in by other researchers, a cross section survey was conducted to evaluate the presence or application of these factors in 11 software projects that used Scrum in 9 different software companies located in Recife-PE, Brazil. The questionnaire was applied to 65 developers and Scrum Masters, representing 75% (65/86) of the professionals that have participated in the projects. The result was correlated with the level of success achieved by the projects, measured by the subjective perception of the project participant, using Spearman's rank correlation coefficient. The main finding is that only 32% (8/25) of the factors correlated positively with project success, raising the question of whether the factors hypothesized in the literature as being critical to the success of agile software projects indeed have an effect on project success.Given the limitations regarding the generalization of this result, other forms of empirical results, in particular case-studies, are needed to test this question. Paper presented at Empirical Software Engineering and Measurement Conference, Bolzano, 2010. http://www.haseresearch.comTRANSCRIPT
S
O
P
E
I
Infl
uen
ces
Is influenced
An Empirical Study on the Relationship between the Use of
Agile Practices and the Success of Scrum Projects
A. César C. França; Fabio Q. B. da Silva; Leila Mariz<[email protected]; [email protected];
S
O
P
E
I
Infl
uen
ces
Is influenced
Research Project
2
(empirical) research project on theinfluence of human factors on softwareengineering
Long Term Objective: to understandthe influence of personality, behavior,and cognition on individuals, teams,projects, processes, and organizationsin software engineering.
S
O
P
E
I
Infl
uen
ces
Is influenced
Research Project
3
(empirical) research project on theinfluence of human factors on softwareengineering
Centro de Informática – UFPE
Since 2003 ...
1 professor/researcher
4 PhD students
18 master students
S
O
P
E
I
Infl
uen
ces
Is influenced
Context
4
Critical SuccessFactors
Agile Attibutes
Delivery strategy
A01 Regular delivery of softwareA02 Delivering most important features first
Agile software engineering techniques
A03 Well-defined coding Standards up frontA04 Pursuing simple designA05 Rigorous refactoring activitiesA06 Right amount of documentationA07 Correct integration testing
Team capability A08 Team members with high competence and expertiseA09 Team members with great motivationA10 Managers knowledgeable in agileA11 Managers who have adaptative management styleA12 Appropriate technical training to team
Project management process
A13 Following agile-oriented requirement management processA14 Following agile-oriented project management processA15 Following agile-oriented configuration management
processA16 Good progress tracking mechanismA17 Strong communication focus with daily face-to-face
meetingsA18 Honoring regular working Schedule
Team environment
A19 Collocation of the whole teamA20 Coherent, self-organizing team-workA21 Projects with small teamA22 Projects with no multiple independent teams
Customer involvement
A23 Good customer relationshipA24 Strong customer commitment and presenceA25 Customer having full authority
T. Chow, e D. Cao, "A Survey Study of Critical Success Factors in Agile Software Projects.", The Journal of Systems and Software, n. 81, 2007, pp. 961–971
S
O
P
E
I
Infl
uen
ces
Is influenced
Context and Goals
5
Critical SuccessFactors
Agile Attibutes
Delivery strategy A01 Regular delivery of softwareA02 Delivering most important features first
Agile software engineering techniques
A03 Well-defined coding Standards up frontA04 Pursuing simple designA05 Rigorous refactoring activitiesA06 Right amount of documentationA07 Correct integration testing
Team capability A08 Team members with high competence and expertise
A09 Team members with great motivationA10 Managers knowledgeable in agileA11 Managers who have adaptative management
styleA12 Appropriate technical training to team
Project management process
A13 Following agile-oriented requirement management process
A14 Following agile-oriented project management process
A15 Following agile-oriented configuration management process
A16 Good progress tracking mechanismA17 Strong communication focus with daily face-to-
face meetingsA18 Honoring regular working Schedule
Team environment A19 Collocation of the whole teamA20 Coherent, self-organizing team-workA21 Projects with small teamA22 Projects with no multiple independent teams
Customer involvement
A23 Good customer relationshipA24 Strong customer commitment and presenceA25 Customer having full authority
Is the adoption of the
25 agile attributes
related to the
success of software
development projects
managed using
Scrum?
S
O
P
E
I
Infl
uen
ces
Is influenced
Study Design
6
Method
Cross sectional survey
S
O
P
E
I
Infl
uen
ces
Is influenced
Study Design
7
Context
Small software firms
Located in Recife.br
Finished Scrum Project
Method
Cross sectional survey
S
O
P
E
I
Infl
uen
ces
Is influenced
Study Design
8
Method
Cross sectional survey
Context
Small software firms
Located in Recife.br
Finished Scrum Project
Perspective/Population
Scrum Team, Scrum
Master, Product Owner
S
O
P
E
I
Infl
uen
ces
Is influenced
Study Design
9
Method
Cross sectional survey
Context
Small software firms
Located in Recife.br
Finished Scrum Project
Perspective/Population
Scrum Team, Scrum
Master, Product Owner
Questionnaire
Closed questions
Perception of use of agile attribute
Perception of project success
S
O
P
E
I
Infl
uen
ces
Is influenced
Study Design
10
Method
Cross sectional survey
Context
Small software firms
Located in Recife.br
Finished Scrum Project
Perspective/Population
Scrum Team, Scrum
Master, Product Owner
Questionnaire
Closed questions
Perception use agile attribute
Perception of project success
Variables
A01 ... A25: level of use of Agile Attribute
S: project success
Scale
Five point Likert type
S
O
P
E
I
Infl
uen
ces
Is influenced
Development
11
Method
Cross sectional survey
Context
Small software firms
Located in Recife.br
Finished Scrum Project
Perspective/Population
Scrum Team, Scrum
Master, Product Owner
Questionnaire
Closed questions
Perception use agile attribute
Perception of project success
Variables
A01 ... A25 and S
Study Design
Time
July-August, 2009
S
O
P
E
I
Infl
uen
ces
Is influenced
Development
12
Method
Cross sectional survey
Context
Small software firms
Located in Recife.br
Finished Scrum Project
Perspective/Population
Scrum Team, Scrum
Master, Product Owner
Questionnaire
Closed questions
Perception use agile attribute
Perception of project success
Variables
A01 ... A25 and S
Study Design
Time
July-August, 2009Context
9 firms
11 projects
Recife Scrum Group
S
O
P
E
I
Infl
uen
ces
Is influenced
Development
13
Method
Cross sectional survey
Context
Small software firms
Located in Recife.br
Finished Scrum Project
Perspective/Population
Scrum Team, Scrum
Master, Product Owner
Questionnaire
Closed questions
Perception use agile attribute
Perception of project success
Variables
A01 ... A25 and S
Study Design
Time
July-August, 2009
Context
9 firms
11 projects
Recife Scrum Group
Participants (62)
48 Scrum Team
10 Scrum Master
4 Product Owner
S
O
P
E
I
Infl
uen
ces
Is influenced
Development
14
Method
Cross sectional survey
Context
Small software firms
Located in Recife.br
Finished Scrum Project
Perspective/Population
Scrum Team, Scrum
Master, Product Owner
Questionnaire
Closed questions
Perception use agile attribute
Perception of project success
Variables
A01 ... A25 and S
Study Design
Time
July-August, 2009
Context
9 firms
11 projects
Recife Scrum Group
Participants (62)
48 Scrum Team
10 Scrum Master
4 Product Owner
Questionnaire
65 questionnaire sent
62 answered
95% response rate
S
O
P
E
I
Infl
uen
ces
Is influenced
Statistical Analysis
15
Method
Cross sectional survey
Context
Small software firms
Located in Recife.br
Finished Scrum Project
Perspective/Population
Scrum Team, Scrum
Master, Product Owner
Questionnaire
Closed questions
Perception use agile attribute
Perception of project success
Variables
A01 ... A25 and S
Study Design
Time
July-August, 2009
Context
9 firms
11 projects
Recife Scrum Group
Participants (62)
48 Scrum Team
10 Scrum Master
4 Product Owner
Questionnaire
65 questionnaire sent
62 answered
95% response rate
Correlation
Spearman's rank correlation coefficient
Factor Analysis
PCA
Development
S
O
P
E
I
Infl
uen
ces
Is influenced
Results: Correlation
16
Critical SuccessFactors
Agile Attibutes
Delivery strategy
A01 Regular delivery of softwareA02 Delivering most important features first
Agile software engineering techniques
A03 Well-defined coding Standards up frontA04 Pursuing simple designA05 Rigorous refactoring activitiesA06 Right amount of documentationA07 Correct integration testing
Team capability A08 Team members with high competence and expertiseA09 Team members with great motivationA10 Managers knowledgeable in agileA11 Managers who have adaptative management styleA12 Appropriate technical training to team
Project management process
A13 Following agile-oriented requirement management processA14 Following agile-oriented project management processA15 Following agile-oriented configuration management
processA16 Good progress tracking mechanismA17 Strong communication focus with daily face-to-face
meetingsA18 Honoring regular working Schedule
Team environment
A19 Collocation of the whole teamA20 Coherent, self-organizing team-workA21 Projects with small teamA22 Projects with no multiple independent teams
Customer involvement
A23 Good customer relationshipA24 Strong customer commitment and presenceA25 Customer having full authority
S
O
P
E
I
Infl
uen
ces
Is influenced
Results: PCA
17
C1Management style
A17 Strong communication focus with daily face-to-face meetingsA10 Managers knowledgeable in agileA18 Honoring regular working ScheduleA11 Managers who have adaptative management styleA20 Coherent, self-organizing team-work
C2Software process
A23Good customer relationship
A13Following agile-oriented requirement management process
A15 Following agile-oriented configuration management process
A14Following agile-oriented project management process
A02Delivering most important features first
C3Team Structure
A22Projects with no multiple independent teams
A21Projects with small teamC4
TechnologyA03 Well-defined coding Standards up front A07 Correct integration testingA12 Appropriate technical training to teamA04 Pursuing simple design
C5Team capability
A09Team members with great motivationA08Team members with high competence and expertise
C6Customer commitment
A24Strong customer commitment and presenceA06Right amount of documentation
C7Delivery strategy
A01Regular delivery of software
C8Team Location
A19Collocation of the whole teamA05 Rigorous refactoring activities
C9Customer awareness
A25Customer having full authorityA16Good progress tracking mechanism
S
O
P
E
I
Infl
uen
ces
Is influenced
Discussion
18
S
O
P
E
I
Infl
uen
ces
Is influenced
“Human Factors”
19
Critical SuccessFactors
Agile Attibutes
Delivery strategy
A01 Regular delivery of softwareA02 Delivering most important features first
Agile software engineering techniques
A03 Well-defined coding Standards up frontA04 Pursuing simple designA05 Rigorous refactoring activitiesA06 Right amount of documentationA07 Correct integration testing
Team capability A08 Team members with high competence and expertiseA09 Team members with great motivationA10 Managers knowledgeable in agileA11 Managers who have adaptative management styleA12 Appropriate technical training to team
Project management process
A13 Following agile-oriented requirement management processA14 Following agile-oriented project management processA15 Following agile-oriented configuration management
processA16 Good progress tracking mechanismA17 Strong communication focus with daily face-to-face
meetingsA18 Honoring regular working Schedule
Team environment
A19 Collocation of the whole teamA20 Coherent, self-organizing team-workA21 Projects with small teamA22 Projects with no multiple independent teams
Customer involvement
A23 Good customer relationshipA24 Strong customer commitment and presenceA25 Customer having full authority
S
O
P
E
I
Infl
uen
ces
Is influenced
Correlated “Human Factors”
20
Critical SuccessFactors
Agile Attibutes
Delivery strategy
A01 Regular delivery of softwareA02 Delivering most important features first
Agile software engineering techniques
A03 Well-defined coding Standards up frontA04 Pursuing simple designA05 Rigorous refactoring activitiesA06 Right amount of documentationA07 Correct integration testing
Team capability A08 Team members with high competence and expertiseA09 Team members with great motivationA10 Managers knowledgeable in agileA11 Managers who have adaptative management styleA12 Appropriate technical training to team
Project management process
A13 Following agile-oriented requirement management processA14 Following agile-oriented project management processA15 Following agile-oriented configuration management
processA16 Good progress tracking mechanismA17 Strong communication focus with daily face-to-face
meetingsA18 Honoring regular working Schedule
Team environment
A19 Collocation of the whole teamA20 Coherent, self-organizing team-workA21 Projects with small teamA22 Projects with no multiple independent teams
Customer involvement
A23 Good customer relationshipA24 Strong customer commitment and presenceA25 Customer having full authority
S
O
P
E
I
Infl
uen
ces
Is influenced
Correlated “Human Factors”
21
Critical SuccessFactors
Agile Attibutes
Delivery strategy
A01 Regular delivery of softwareA02 Delivering most important features first
Agile software engineering techniques
A03 Well-defined coding Standards up frontA04 Pursuing simple designA05 Rigorous refactoring activitiesA06 Right amount of documentationA07 Correct integration testing
Team capability A08 Team members with high competence and expertiseA09 Team members with great motivationA10 Managers knowledgeable in agileA11 Managers who have adaptative management styleA12 Appropriate technical training to team
Project management process
A13 Following agile-oriented requirement management processA14 Following agile-oriented project management processA15 Following agile-oriented configuration management
processA16 Good progress tracking mechanismA17 Strong communication focus with daily face-to-face
meetingsA18 Honoring regular working Schedule
Team environment
A19 Collocation of the whole teamA20 Coherent, self-organizing team-workA21 Projects with small teamA22 Projects with no multiple independent teams
Customer involvement
A23 Good customer relationshipA24 Strong customer commitment and presenceA25 Customer having full authority
Not a surprise. Happens with non Agile
methodologies.
S
O
P
E
I
Infl
uen
ces
Is influenced
Correlated “Human Factors”
22
Critical SuccessFactors
Agile Attibutes
Delivery strategy
A01 Regular delivery of softwareA02 Delivering most important features first
Agile software engineering techniques
A03 Well-defined coding Standards up frontA04 Pursuing simple designA05 Rigorous refactoring activitiesA06 Right amount of documentationA07 Correct integration testing
Team capability A08 Team members with high competence and expertiseA09 Team members with great motivationA10 Managers knowledgeable in agileA11 Managers who have adaptative management styleA12 Appropriate technical training to team
Project management process
A13 Following agile-oriented requirement management processA14 Following agile-oriented project management processA15 Following agile-oriented configuration management
processA16 Good progress tracking mechanismA17 Strong communication focus with daily face-to-face
meetingsA18 Honoring regular working Schedule
Team environment
A19 Collocation of the whole teamA20 Coherent, self-organizing team-workA21 Projects with small teamA22 Projects with no multiple independent teams
Customer involvement
A23 Good customer relationshipA24 Strong customer commitment and presenceA25 Customer having full authority
Agrees with research on team work!
Not ease to achieve in SE!
S
O
P
E
I
Infl
uen
ces
Is influenced
“Surprises”
23
Critical SuccessFactors
Agile Attibutes
Delivery strategy
A01 Regular delivery of softwareA02 Delivering most important features first
Agile software engineering techniques
A03 Well-defined coding Standards up frontA04 Pursuing simple designA05 Rigorous refactoring activitiesA06 Right amount of documentationA07 Correct integration testing
Team capability A08 Team members with high competence and expertiseA09 Team members with great motivationA10 Managers knowledgeable in agileA11 Managers who have adaptative management styleA12 Appropriate technical training to team
Project management process
A13 Following agile-oriented requirement management processA14 Following agile-oriented project management processA15 Following agile-oriented configuration management
processA16 Good progress tracking mechanismA17 Strong communication focus with daily face-to-face
meetingsA18 Honoring regular working Schedule
Team environment
A19 Collocation of the whole teamA20 Coherent, self-organizing team-workA21 Projects with small teamA22 Projects with no multiple independent teams
Customer involvement
A23 Good customer relationshipA24 Strong customer commitment and presenceA25 Customer having full authority
S
O
P
E
I
Infl
uen
ces
Is influenced
Four Areas of Further Research
24
Self-organizing Teams
What are they in SE?
How to build one?
How to keep them cohesive?
S
O
P
E
I
Infl
uen
ces
Is influenced
Four Areas of Further Research
25
Motivation
How is motivation related to
performance?
How is motivation related to
success?
Self-organizing
Teams
What are they in SE?
How to build one?
How to keep them
cohesive?
S
O
P
E
I
Infl
uen
ces
Is influenced
Four Areas of Further Research
26
Motivation
How is motivation
related to
performance?
How is motivation
related to success?
Leadership Style
How the style of
leadership and
management affects
success?
Self-organizing
Teams
What are they in SE?
How to build one?
How to keep them
cohesive?
S
O
P
E
I
Infl
uen
ces
Is influenced
Four Areas of Further Research
27
Leadership Style
How the style of
leadership and
management affects
success?
Personality and
Behavior
Does personality matter?
How different team work
behavior affects team
work effectiveness? Motivation
How is motivation
related to
performance?
How is motivation
related to success?
Self-organizing
Teams
What are they in SE?
How to build one?
How to keep them
cohesive?
S
O
P
E
I
Infl
uen
ces
Is influenced
Conclusions
• Limitations and Threats to Validity:
– Restricted context
– Small number of participants, projects, and firms
• Contributions:
– New research questions
– More detailed factor structure
28
S
O
P
E
I
Infl
uen
ces
Is influenced
An Empirical Study on the Relationship between the Use of
Agile Practices and the Success of Scrum Projects
A. César C. França; Fabio Q. B. da Silva; Leila Mariz<[email protected]; [email protected];
Thank you!
S
O
P
E
I
Infl
uen
ces
Is influenced
An Empirical Study on the Relationship between the Use of Agile Practices and the Success of Scrum Projects by A. César C. França, Fabio Q. B. da Silva, Leila M. R. de Sousa Mariz is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 UnportedLicense.
Based on a work at Empirical Software Engineering and Measurement Conference, Bolzano, 2010.
Permissions beyond the scope of this license may be available at http://www.haseresearch.com