Agile Meets CMMI… Again

Notes from Tes Trek conference -Keynote presentation by

James McHaleNovember 8th 2012

“Notes from Tes Trek conference” by Maira Bay de Souza is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License

Before we begin ...Items in this font are the notes I took of what Jim said

Items in this font are my own comments

Introduction

Process-oriented people have always tried to be more Agile, while Agile defenders are constantly trying to be more process-oriented!

Interesting comment. I had never thought of it that way, but I think he is right.Good process professionals will build a process that works for the people, instead of having people work for the process. That means no useless paperwork or bureaucracy. Only do what is essential. And isn't that what Agile is about?

About SEI and CMMISEI works on the state of the practice, not the state of the art

CMMI will soon move out of SEI and into a new organization called the CMMI Institute. There will be little or no change on how you work with them and on the contracts themselves.

State of the art = latest research on process improvementState of the practice = what companies are actually doingI have studied about process and then worked in organizations of all sizes. I agree that there is a difference between the best thing that companies should be doing (according to research) and what they are actually doing.From what Jim said, it seems like this new organization will be more “for profit” than SEI. There isn't much info on the web yet about this transition. The best place I could find was here: http://sei-pab-chair.blogspot.com/

CMMI and Agile - similarities

CMMI and Agile have roots in common: Agile was developed after CMMI because CMMI was too heavyweight for some organizations.

CMMI and Agile - differencesCMMI was developed based on ideas of authors that came from

a manufacturing background (see recommended readings 1, 2 and 3)

The people who developed Agile were more in line with the thoughts of Peter Drucker. He was more focused on the knowledge industry.

Very interesting remark. I have heard a few people in the software community say that we shouldn't take a manufacturing approach to software development. We are after all building a product based on information. We are not manufacturing tangible goods.

I hope to learn more about the connection between Peter Drucker and Agile, and then share it with you!

Other SDLCsThe original author of the Waterfall model (see recommended

reading 5) intended it to be iterative. But the DOD ended up implementing it as a one-time iteration.

Agile is very similar to TSP, making it closer to CMMI than we would think.

Interesting comment about the waterfall model. It relates to what he said before about state of the art and state of the practice: what the author intended was not what ended up being done.

Organizations

Most organizations adapt themselves to look like their products and not the other way around (unfortunately)

I agree with him. It is unfortunate but true. I've seen many organizations that work around their systems instead of changing or modernizing their systems to make them more efficient and customer-focused.

BDUFBDUF = Big Design Up-FrontThis type of software life-cycle was not

created because of CMMI. It was created because of the DOD's competition rules. Design had to be ready before the competition started.

Ah, that explains a lot!It's amazing how much the DOD has shaped

the world of software development.

New trend5 years ago the DOD noticed that CMMI L3 to L5 companies had

no difference in the performance between them.

Interesting. If I were to make a guess I would say that L5 companies aren't following their processes as strictly as they should, hence performing as L3. But that's just a guess. I wonder if they found a reason for this, and what the reason was.

ContradictionSome people in the Agile world are very strict on their

processes, which is funny, because that was the whole point of Agile in the first place ( = to not follow any process!)

A company called Systematic is following Agile, but has lots of data (just like a CMMI L4 company)

Well, what can I say... humans are contradictory!By the way, I don't have anything against the Agile people.

I believe that whatever works for an organization is what they should use - as long as it will still keep on working in the short to medium term future.

Kelvin's quote“When you can measure what you are speaking about, and

express it in numbers, you know something about it, when you cannot express it in numbers, your knowledge is of a meager and unsatisfactory kind; it may be the beginning of knowledge, but you have scarely, in your thoughts advanced to the stage of science.”

Measurement is the foundation for improvementMeasurement is the act of reducing uncertaintyLook at recommended reading 7

Metrics and QAQA has most of the data about an SDLCI go to organizations and the first place I look for data is the QA

departmentSometimes it's the first time anyone has ever asked to see the

data they have been collecting for so long

I'm surprised and disappointed by that information. The data collected by QA should be used in improving the process. If the organization is not using the data, then they are wasting the QA team's time and consequently their own money.

As I love to say it: the process should work for the people, and not the people work for the process. The case Jim describes is a classical case of people working for the process. That means huge waste of resources, time and money.

QA and defectsIn a project where testing is only done at the end (i.e., using

waterfall SDLC), the # of defects found/hour varies a lot. Example: QA spent 5 hours of test execution and didn't find any defects. Then, in the next 30 min they found 14 defects.

In a project with early testing (using V-Model SDLC), the # of defects found/hour is more constant. Example: QA found 1-2 defects every 30 min.

Interesting observation. That tells me that QA will be perceived as more efficient with early testing.

Scenario CoverageDid you know? When Microsoft tests Windows, they only cover

0.1% to 2% of the possible scenarios!That is because the PC has too many possible configurations

(think of all the different brands of motherboard, video card, mouse, etc that exist).

Apple computers, on the other hand, have much less variability. That's why they are able to test a much bigger % of the configuration possibilities.

Micheal Fagan (see recommended reading 9) started doing inspection in the 60s!

A-ha! That explains a lot!

Code inspectionThe latest trend in testing is having the testers fix the code

themselves, during code inspection.Less than 50% of organizations do inspection. Of those, less

than 50% do it effectively.Even CMMI doesn't value inspection that much. It only allows

you to inspect 10% of the code.Inspection saves more time than any other testing method. It's

better to do only 100% inspection than to do other methods of testing.

Lots of food for thought on inspection here.I think what he is saying makes sense. After all, the earlier the

testing, the better. And if we find all the defects in inspection, then it's true: we don't need other types of testing.

That is quite a revolutionary idea, but definitely something to think about (and maybe experiment in your organization!).

Defect-free cycleHow long should a full defect-free test cycle

last (including test design)?The answer to that question will be how

much the organization is saving by doing inspection

What a smart way to measure the value of inspection!

Call to Arms!For all QA professionals:- Bring discipline to development organizations (not

punishment!)- Measurement is essential- Encourage disciplined process changes (whatever your

process is) based on feedback given by the data/measurement

- If you have many testing cycles, use one of them to inspect the code

Great presentation! Great tips!I learned a lot from Jim. I hope those of you who were

unable to attend the conference have learned a lot from my notes too!

Recommended Readings[1] Quality is Free - Philip B. Crosby[2] works of Watts Humphrey[3] The Knowledge-Creating Company - Ikujiro Nonaka and Hirotaka Takeuchi[4] Software Engineering Best Practices - Capers Jones[5] works of Winston Royce[6] Jeff Sutherland's articles on MSDN:http://msdn.microsoft.com/en-us/library/dd997578(v=vs.100).aspxhttp://msdn.microsoft.com/en-us/library/hh350860(v=vs.100).aspx[7] How to Measure Anything - Douglas W. Hubbard[8] The Checklist Manifesto - Atul Gawande[9] works of Michael Fagan

DisclaimerThe notes presented here are what I understood from what Jim

communicated. They might not be 100% accurate, as I was taking notes and listening to the presentation at the same time.

All the information I am quoting from Jim is his intellectual property. I am reproducing it here under the fair use policy, for quoting purposes only.