fables fantasies and facts

40
FABLES, FANTASIES & FACTS Finding the signal in the noise Working with Agility

Upload: kelsey-van-haaster

Post on 19-Jul-2015

263 views

Category:

Software


0 download

TRANSCRIPT

Page 1: Fables fantasies and facts

FABLES, FANTASIES & FACTS

Finding the signal in the noise

W o r k i n g w i t h A g i l i t y

Page 2: Fables fantasies and facts

WELCOME

Yet another Agile myths session?

Well, Yes!

But hopefully, this talk is a bit different from the

ones you may have seen before.

2

Page 3: Fables fantasies and facts

FIRSTLY, SOME CONTEXT

A Google search for "Agile myths" returns 443,000 results.

You can increase that number by modifying your search term

include "software", "development", " teams" or any number

of other combinations.

These are just the English results.

3

Page 4: Fables fantasies and facts

THERE IS SO MUCH STUFF OUT THERE!

Start clicking on some of these results and you will find some

fantastic, well written evidence based materials.

You will also find a cacophony of opinion that is;

• Often contradictory

• Context specific

• Written by vendors of various products in support of those products

• Of highly variable quality

It's all very confusing

4

Page 5: Fables fantasies and facts

BUT IT PROVES ME RIGHT SO THAT’S OK!

There's a very good chance that you will find that much of it

supports your own perspective and experience.

This feels very reassuring, we like it, because we are human and

like to be validated.

This is called confirmation bias!

The tendency to search for, interpret and recall information in a

way that confirms your beliefs or hypothesis

http://www.oxforddictionaries.com/definition/en

5

Page 6: Fables fantasies and facts

I KNEW THAT!

Confirmation bias is something we all unknowingly suffer from

nearly all of the time.

Thucydides (c 460BC - 395BC), Dante (1265-1321) Bacon (1561-

1662) and Tolstoy (1829-1910) all noticed and wrote about the

effects of confirmation bias before it had a name

An English psychologist, named Peter Wason first used the term

confirmation bias in 1960 and lots of further research has looked

into why we do this and what its consequences are. (Jones &

Sugden, 2001)

6

Page 7: Fables fantasies and facts

BUT IT DOESN’T HELP ME FIGURE OUT WHAT TO BELIEVE

This is of course all very interesting;

but a more useful question is, how do we go about

acknowledging that this happens and how do we

try to avoid it? (Darmstadter, 2013)

7

Page 8: Fables fantasies and facts

THIS IS WHAT SCIENTIFIC RESEARCH TRIES TO DO!

Scientific research tries to avoid confirmation bias by the following:

• Providing sufficient information about how a conclusion was reached, such

that an experiment or study can replicated.

• Identifying (calling out) the limitations of any research, e.g. we only found

these results in aardvarks, it may not apply to ruby developers.

• Identifying other research or theory on which the study is based, as well as

explaining how it was interpreted.

• Making their work available for scrutiny and review by their peers.

• Publishing their failures and negative results *

8

Page 9: Fables fantasies and facts

THIS IS NOT PERFECT!

Dodgy research does get published, but not often.

Research which is subsequently discredited, can

do long term damage whilst it's out there being

read and used.

Doing bad research can end a research career.

(Gregor, 2006; Guba, 1981; Weber, 2012)

9

Page 10: Fables fantasies and facts

10

So, with this in mind, let's take a quick look at what

the research says about a number of common

fables and fantasies around Lean and Agile

thinking and Agile approaches to software

development!

Page 11: Fables fantasies and facts

1. WHAT IS THIS AGILE THINGY ANYWAY?

The terms Agile and Agility refer to [insert (Method, philosophy,

approach, mindset etc.)here]!

• There are many perspectives on this. The term is interpreted in

many different ways including by researchers.

• There are more than 47 different definitions of Agile, in the

research literature published over the last 5 years.

• There are countless definitions in the technical literature.

• You probably have your own unique view *

11

Page 12: Fables fantasies and facts

THE RESEARCH SAYS

There is no agreed definition amongst researchers, or practitioners of what

Agile is, or is not.

• Some scientists and philosophers, argue that definitions are not needed because they constrain

ideas.

• Other scientists and philosophers think that they are important because without them we cannot

know that we are talking about the same thing.

While we are on the subject of definitions

• Definitions are tricky, they can and do change over time as we learn more about something

• To be useful a definition does not need to be objectively correct; it just needs to be widely

agreed upon (Gupta, 2008).

12

Page 13: Fables fantasies and facts

AND ANOTHER PROBLEM IS

There is no single grand theory of Agile, or

what makes Agile work

• This means that we do not have access to a shared

understanding through which we can explain, predict and

inform developments in the domain and which allows us to

position new and existing research within the body of Agile

knowledge (McDonald, 2012).

13

Page 14: Fables fantasies and facts

WHICH IS WORRYING BECAUSE

Given the increasingly pervasive nature of all things Agile into

almost every aspect of the way we do business, this lack of

a shared understanding is a matter for concern

Amongst the research community and amongst practitioners

and has led to calls for the selection and use of theory to

support research in the field to be more rigorously evaluated

(Weber, 2012).

14

Page 15: Fables fantasies and facts

WHAT DO KNOW IS THAT WE REALLY DON’T KNOW FOR SURE!

15

This lack of an agreed theoretically sound basis for

understanding Agility is considered to be one of the most

urgent questions amongst Information Systems researchers

at present.

(Conboy, 2009)(Dingsøyr, Dybå, & Moe, 2010; Torgeir &

Nils, 2013)

Page 16: Fables fantasies and facts

2: PERHAPS WE SHOULD ALL BE CERTIFIED

Do you or don't you need a certification?

• Almost everyone selling, teaching or issuing

some kind of certification, usually with a

requisite course or consultancy engagement

thinks you do.

• Equally, everyone who isn't says certification is

unnecessary.

16

Page 17: Fables fantasies and facts

THE RESEARCH SAYS

Not very much about the value of specific certifications such as Scrum Master, ITIL etc. But it does say

• Vendors use certifications to reduce product support costs.

• Brand matters.

• Certifications are seen (by HR) to reduce uncertainty and make hiring easier.

• Certifications for a specific technology are seen as more valuable than those that are

method based or technology neutral (Eg Scrum Master/SaFE).

• Whether someone maintains a certification is most strongly impacted by the cost and who

is paying.

• Experience is king - certification can be complementary to experience, but cannot replace

it and experience has far greater impact on an individuals success than any certifications

(Hunsinger, Smith, & Winter, 2011; McGill & Dixon, 2013; Robin, 2011)

17

Page 18: Fables fantasies and facts

3. YOU CAN’T BE A LITTLE BIT PREGNANT (OR AGILE)

Can you combine Agile approaches and traditional approaches?

• Most of the traditional project management associations, PMBOK, Prince2,

are now arguing that Agile approaches work well within these kinds of

frameworks.

• Supporters of both CMMI and 6 Sigma also argue that both Agile

approaches and process improvement models have the same goals

• Equally many others, including those considered to be thought leaders in

the field argue that such models are counter intuitive to the principles and

values expressed by the Agile Manifesto

• Or that they are based on old-fashioned views of software development as

a process which must be managed

18

Page 19: Fables fantasies and facts

THE RESEARCH SAYS

• A large number of studies have evaluated Agile

principles and practices against the

requirements of CMMI and 6 sigma and have

found that they are highly compatible, in

particular, Scrum is fully compliant with the

requirements of CMMI level 5 certification.

19

Page 20: Fables fantasies and facts

THE RESEARCH SAYS

• Whilst there are some challenges integrating

traditional and Agile models, these can be

overcome and the two can work together in

harmony

20

Page 21: Fables fantasies and facts

IT’S NOT A SIMPLE PROCESS

The key challenges arise from:

• Increased complexity in the IT landscape, due to the

different approaches being followed by different

development streams

• Difficulties in ensuring sufficient business involvement

21

Page 22: Fables fantasies and facts

BUT THERE ARE THINGS YOU CAN DO

Even these challenges can be mitigated by;

• Working to change the mindset of business stakeholders

• Focussing on managing alignment at the program level

• Facilitating the role of the project manager

• Channelling business knowledge through the product

owner/manager role

• Aligning knowledge and requirements at the business level

22

Page 23: Fables fantasies and facts

AND IT DOESN’T APPLY IN EVERY CASE

In most of the organisations studied, Agile methods were introduced

in a bottom up fashion, that is they were led by the software

development function, not by the organisational governance body.

Where Agile methods are introduced in a top-down fashion, a

different set of challenges emerge – but that’s a whole other talk in

itself.

(Boehm & Turner, 2005), (Bustard, Wilkie, & Greer, 2013; Fontana, Fontana, da

Rosa Garbuio, Reinehr, & Malucelli, 2014; van Waardenburg & Hans, 2013)

23

Page 24: Fables fantasies and facts

4. THE PARADOX OF THE AGILE BUSINESS ANALYST

The BA role on an Agile project is fundamentally different to that of the BA

role on a traditional, plan driven project. Therefore you must hire an Agile

BA

• You can go on Agile BA training courses (refer earlier comments on

certifications)

• The BA role is not explicitly identified by the Scrum guide, or in XP

• There’s a perception that Agile projects, don’t have much/any documentation so

BA’s do something else

• There’s a perception that because Agile projects may have an onsite customer,

there is no need for traditional BA stakeholder management skills.

24

Page 25: Fables fantasies and facts

WHAT THE RESEARCH SAYS

• The biggest reason for project failure is still not

building the right thing, (bad requirements)

regardless of methodology; Software

development is a really hard thing to do, and

Agile projects can and do still build the wrong

thing. (Adamczyk & Hafiz, 2010)

25

Page 26: Fables fantasies and facts

WHAT THE RESEARCH SAYS

• Requirements management for Agile projects is just as

important as it is for traditional projects, it may take different

forms and needs to always add value, but, BA’s still produce

and manage much of it. (Lan & Ramesh, 2008)

• Whilst some Agile approaches assume that the whole team

can and will share responsibility for project requirements and

other documentation, this is not always easy or possible.

• In these circumstances, the role of a BA as a facilitator,

communicator and liaison is key (Cao, Mohan, Xu, & Ramesh,

2009; Hochmüller, 2011; Hoda, Noble, & Marshall, 2010).

26

Page 27: Fables fantasies and facts

WHAT THE RESEARCH SAYS

• Several studies show that insufficient requirements documentation

presents a barrier to integrating new developers into a team,

transferring a completed system to a maintenance team and where

stringent compliance requirements exist (O’Connor, 2010; Stettina &

Heijstek, 2011).

• Lack of documentation or insufficient requirements documentation in

Agile projects is also identified as contributing to poor traceability,

inadequate architectures, and an inability of products to meet basic

non-functional or Quality of Service requirements (Kelly, 2010).

27

Page 28: Fables fantasies and facts

HOORAY FOR THE BA

The findings described here show that the role of the BA

is critical to any project using any approach and it’s

more than just requirements, at a minimum a BA can;

• Take responsibility for the creation and communication of a single vision

for the project which takes into account the needs of diverse stakeholder

groups, including other members of the software development team;

working as the facilitator of a single voice representing the perspectives

of divergent stakeholders; acting in lieu of or providing support to a

product owner or customer representative (IIBA, 2011).

28

Page 29: Fables fantasies and facts

5. THAT’S NOT REAL AGILE

If you’re not doing [insert your favourite

methodology/framework/practice here] by the

book, you’re not really doing Agile

Mostly said by people who are doing [the

methodology/framework/practice] or people who

are making their living through the promotion or

delivery of [the methodology/framework/practice]

29

Page 30: Fables fantasies and facts

THERE’S ALSO THE OPPOSING VIEW;

That you should always adapt your

[methodology/framework/practice] to suit your

organisation

Mostly said by people who have adapted or

modified a [methodology/framework/practice]

30

Page 31: Fables fantasies and facts

WHAT THE RESEARCH SAYS

That very few projects/organisations adhere to the

practices specified by formal methods or

frameworks, those that do exist primarily in the

context of something called the Agile sweet spot.

31

Page 32: Fables fantasies and facts

THE AGILE SWEET SPOT

“a small, co-located team; an on-site or available

customer representative; an emphasis on coding

and testing early; and frequent feedback into

updated requirements. “ (Hoda, Kruchten, Noble,

& Marshall, 2010)

32

Page 33: Fables fantasies and facts

BUT WHAT ABOUT EVERYONE ELSE

Most organisations modify practices for a range of

different reasons and there are a number of

studies which show that this does not have an

impact on their success rates (Franca, Silva, &

Mariz, 2010).

33

Page 34: Fables fantasies and facts

BUT WE DO KNOW ABOUT A FEW THINGS THAT ARE;

One very interesting study has shown there is a

positive relationship between clusters of Agile

practices which improve the likelihood of

success, but that these do not necessarily have

to belong to a single methodology or framework,

for example

34

Page 35: Fables fantasies and facts

FACTORS WHICH POSITIVELY CORRELATE WITH SUCCESS

• Agile quality assurance practices,

• continuous code integration,

• test driven development,

• code refactoring,

• developer tests,

• flexible architecture,

• evolutionary design,

• simple design and

• collective ownership.

• Iterative and incremental development,

• incremental delivery,

• small releases,

• iterative development, sustainable pace,

• active stakeholder participation and

• working demoable software.

35

Page 36: Fables fantasies and facts

FACTORS WHICH NEGATIVELY CORRELATE WITH SUCCESS

Interestingly and in (some cases counter intuitively) some

practices had a negative correlation with success

• The use of traditional analysis practices, including case tool modeling,

architecture specification and detailed requirements specifications.

• Coding standards practices including two practices the use of coding

standards and data naming conventions.

• Whiteboard practices including whiteboard sketches and whiteboard

modeling

(Abbas, Gravell, & Wills, 2010; McLeod & MacDonell, 2011)

36

Page 37: Fables fantasies and facts

SO WHAT DOES ALL THIS MEAN?

• If you were hoping for the final word on Agility, you

wont find it here today.

• We are still working in a nascent field

• Most of this research has been conducted in the

context of small scale Agile

• We know very little about what really happens

when you scale Agile, or take a top down approach

37

Page 38: Fables fantasies and facts

REFERENCES

If you would like a full list of the references I used

to put this talk together you can contact me at

[email protected] or @kelseyvh on

Twitter and I will be happy to provide it.

In the meantime: here they all are in tiny

unreadable font.

38

Page 39: Fables fantasies and facts

Abbas, N., Gravell, A. M., & Wills, G. B. (2010). Using Factor Analysis to Generate Clusters of Agile Practices (A Guide for Agile Process Improvement). In Agile Conference (AGILE), 2010(pp. 11–20). doi:10.1109/agile.2010.15

Adamczyk, P., & Hafiz, M. (2010). The Tower of Babel did not fail. Proceedings of the ACM International Conference on Object Oriented Programming Systems Languages and Applications. Reno/Tahoe, Nevada, USA: ACM. doi:10.1145/1869459.1869537

Boehm, B., & Turner, R. (2005). Management challenges to implementing agile processes in traditional development organizations. Software, IEEE, 22(5), 30–39. doi:10.1109/ms.2005.129Bustard, D., Wilkie, G., & Greer, D. (2013). The Maturation of Agile Software Development Principles and Practice: Observations on Successive Industrial Studies in 2010 and 2012. In

Engineering of Computer Based Systems (ECBS), 2013 20th IEEE International Conference and Workshops on the (pp. 139–146). doi:10.1109/ECBS.2013.11Cao, L., Mohan, K., Xu, P., & Ramesh, B. (2009). A framework for adapting agile development methodologies. European Journal of Information Systems, 18(4), 332–343.

doi:http://dx.doi.org/10.1057/ejis.2009.26Conboy, K. (2009). Agility from First Principles: Reconstructing the Concept of Agility in Information Systems Development. Information Systems Research, 20(3), 329–354,478. Retrieved

from http://search.proquest.com/docview/208160739?accountid=10344Darmstadter, H. (2013). Why do humans reason? A pragmatist supplement to an argumentative theory. Thinking & Reasoning, 19(3-4), 472–487. doi:10.1080/13546783.2013.802256Dingsøyr, T., Dybå, T., & Moe, N. B. (2010). Agile Software Development: Current Research and Future Directions. doi:10.1007/978-3-642-12575-1Fontana, R. M., Fontana, I. M., da Rosa Garbuio, P. A., Reinehr, S., & Malucelli, A. (2014). Processes versus people: How should agile software development maturity be defined? Journal

of Systems and Software, 97(0), 140–155. doi:http://dx.doi.org/10.1016/j.jss.2014.07.030Franca, A. C. C., Silva, F. Q. B. da, & Mariz, L. M. R. de S. (2010). An empirical study on the relationship between the use of agile practices and the success of Scrum projects. Proceedings

of the 2010 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement. Bolzano-Bozen, Italy: ACM. doi:10.1145/1852786.1852835Gregor, S. (2006). The Nature of Theory in Information Systems. MIS Quarterly, 30(3), 611–642. Retrieved from http://www.jstor.org/stable/25148742Guba, E. G. (1981). Criteria for assessing the trustworthiness of naturalistic inquiries. Educational Communication & Technology, 29(2), 75–91.Gupta, A. (2008, April 10). Definitions. Retrieved March 14, 2015, from http://plato.stanford.edu/entries/definitions/Hochmüller, E. (2011). The requirements engineer as a liaison officer in agile software development. Proceedings of the 1st Workshop on Agile Requirements Engineering. Lancaster,

United Kingdom: ACM. doi:10.1145/2068783.2068785Hoda, R., Kruchten, P., Noble, J., & Marshall, S. (2010). Agility in context. Proceedings of the ACM International Conference on Object Oriented Programming Systems Languages and

Applications. Reno/Tahoe, Nevada, USA: ACM. doi:10.1145/1869459.1869467Hoda, R., Noble, J., & Marshall, S. (2010). How much is just enough?: some documentation patterns on Agile projects. Proceedings of the 15th European Conference on Pattern Languages

of Programs. Irsee, Germany: ACM. doi:10.1145/2328909.2328926Hunsinger, D. S., Smith, M. A., & Winter, S. J. (2011). A Framework of the Use of Certifications by Hiring Personnel in It Hiring Decisions. SIGMIS Database, 42(1), 9–28.

doi:10.1145/1952712.1952714IIBA. (2011). The Agile Extension to the BABOK Guide. Toronto: International Institute of Business Analysis.Jones, M., & Sugden, R. (2001). Positive confirmation bias in the acquisition of information. Theory and Decision, 50(1), 59–99. doi:10.1023/A:1005296023424Kelly, S. (2010). Towards an evolutionary framework for agile requirements elicitation. Proceedings of the 2nd ACM SIGCHI Symposium on Engineering Interactive Computing Systems.

Berlin, Germany: ACM. doi:10.1145/1822018.1822078Lan, C., & Ramesh, B. (2008). Agile Requirements Engineering Practices An Empirical Study. Software, IEEE, 25(1), 60–67.McDonald, C. (2012). Theory: An informatics perspective. In Information Systems Foundations (“Theory Building in Information Systems”) (pp. 253 –262). Canberra, Australia: ANU E Press.McGill, T., & Dixon, M. (2013). An Investigation of the Impact of Recertification Requirements on Recertification Decisions. In Proceedings of the 2013 Annual Conference on Computers

and People Research (pp. 79–86). New York, NY, USA: ACM. doi:10.1145/2487294.2487310McLeod, L., & MacDonell, S. G. (2011). Factors that affect software systems development project outcomes: A survey of research. ACM Comput. Surv., 43(4), 1–56.

doi:10.1145/1978802.1978803O’Connor, C. P. (2010). Letters from the edge of an agile transition. Proceedings of the ACM International Conference Companion on Object Oriented Programming Systems Languages

and Applications Companion. Reno/Tahoe, Nevada, USA: ACM. doi:10.1145/1869542.1869557Robin, G. J. (2011). Do Companies Look for Education, Certifications or Experience: A Quantitative Analysis. In Proceedings of the 49th SIGMIS Annual Conference on Computer

Personnel Research (pp. 1–5). New York, NY, USA: ACM. doi:10.1145/1982143.1982145Stettina, C. J., & Heijstek, W. (2011). Necessary and neglected an empirical study of internal documentation in agile software development teams. Proceedings of the 29th ACM

International Conference on Design of Communication. Pisa, Italy: ACM. doi:10.1145/2038476.2038509Torgeir, D., & Nils, B. M. (2013). Research challenges in large-scale agile software development. SIGSOFT Softw. Eng. Notes, 38(5), 38–39. doi:10.1145/2507288.2507322Van Waardenburg, G., & Hans, van V. (2013). When agile meets the enterprise. Information and Software Technology, 55(12), 2154–2171. doi:http://dx.doi.org/10.1016/j.infsof.2013.07.012Weber, R. (2012). Theory Building in the Information Systems Discipline: Some critical reflections. In S. Hart, Dennis. Gregor (Ed.), Information Systems Foundations (“Theory Building in

Information Systems”) (pp. 1–21). Canberra, Australia: ANU E Press. Retrieved from http://press.anu.edu.au?p=191431

39

Page 40: Fables fantasies and facts

THANK YOU

Any Questions