keys to software success efficient software processes

Keys toSoftware Success

Efficient Software Processes

2010 October 25 Keys to Software Success 2

• Universal Laws

• History of Software

• 9 Key Practices

• Discussion

Agenda


Gods of Software

Deming

Beck PoppendiecksBrooks


There are a lot of “good” laws

E=mc2

Do onto others…

If anything can happen, it will.


But they aren’t Universal Laws

E≠MC2

Do onto others….?

Thou shall not commit adultery?

h/2π ?


Universal Law #1

The Universe is infinitely complex At least, reality >> human brain capacity

• At the particle level, we even know what causes gravity.

• At the astronomical scale, we don’t even know if the universe is open or closed.

• At the human scale, we can predict the interaction of two billiard balls, but not three.

• All the computers in the world can not predict what a single person will choose for lunch.


Dealing with Complexity

• Humans make simplified models of complex systems. We can understand the models, and we hope they are close to reality, but don’t pretend they are real.

• We decompose complex systems into pieces and those pieces into pieces until we get to pieces simple enough to model.


Universal Law #2All Complex systems need feedback to remain stable.

• In biological systems, half of the components are there to balance the other half.

• In economic systems companies fail with the wrong business model, and grow with the right model.

• In political systems your career ends if you can’t get people to re-elect you.

There aren’t any big systems without significant feedback loops!


Dealing with Dynamics• Humans can construct complex

systems, but they can not hope to micro-manage them.

• Like the stable systems in nature, you have to introduce feedback loops.

• If you become unstable, add feedback.

• If you become immobile, decrease feedback.


Universal Law #3Entropy causes information to rot

• That sweet application you wrote last year does not look familiar to you any more.

• The detailed spec they wrote two months ago needs dozens of changes.

• The feature that was most critical when the project started is now priority number eight.

• Writing it down stops if from rotting in your head, but then it rots on paper.


Dealing with Entropy • The less time it takes to finish a task, the

less information will rot.

• The fewer tasks you have at once, the less information you will have to hold in your head.

• If you write it down, you will have to maintain it. Information rots, wherever it is recorded, but paper may rot more slowly that what is in your head.


6 Sigma

History of Programming

Junk(First Software)

StructuredProgramming

XPScrum

RUP

LeanKanban

The NavyWay

OOP

Rapidprototyping

CMMCMMI

PMBOK

Ha

ckin

g???

TSP/PSP

Time

Pro

duct

ivity


9 Keys to Success1. Continuous Improvement

2. Eliminate Waste

3. BDUF / Small Bites / Agility / Change

4. Flow / Work In Progress

5. Team Agility

6. Continuous Integration

7. Configuration Management

8. Build Tools

9. Test first / Automate Testing


1 Continuous improvement

Move forward or die!


Continuous Improvement

• Change is painful

• You can’t get better if you don’t change.

• Try to learn something from everything you do.

• Constantly try things that would have prevented past problems.

• Find a way to measure if changes make things better or worse.



The Deming cycle is a very powerful example of feedback



Locking in a standard process that can not be changed means there can be no improvement.

There are no standards, no mythologies, no theories that you can not improve upon.



The Deming cycle tells you to measure results, and then optimize them. Be careful of what you measure!

• If you measure busyness, you will get busyness not work.

• If you measure documents written, you will get documents, not products.

• Measure the things customers will pay for, and the time it take to make them.


2 Waste

The stuff you make, but don’t use is killing your schedule and budget.


Waste

• Features that no one uses

• Documents that no one reads

• Leaving tasks half done, (to rot) while you do other things

• Inefficient communication

• Debugging


Waste

• Find the sweet spot.

• Use science not philosophy to find it.

No overhead

Few tools

Little QA

Lots of planning

Formal methods

Many documents

Planning costsRework costs

Sweet SpotHacking BDUF


Waste

Scrum and some other Agile methodologies require detailed estimation to make every sprint the same size.

This is waste.

Measuring features delivered over time tells you everything you need to know.


3 Big Design Up Front

The idea is that you can save time and money later by thinking smarter now.


Big Design Up Front

But humans are not smart enough to do it.


Big Design Up Front• Nobody knows what they really want.

• Nobody is smart enough to precisely specify anything complex.

• Few people can imagine the details of a system they can’t try.

• Needs change and keep changing, so if you did ever get everything right, it is wrong now.


Big Design Up Front


Big Design Up Front

One excuse for a big design is to support maintenance. The Big Spec. you wrote to start the project can not describe how you built it. Don’t try to morph it into a maintenance document. Start a new maintenance document after you know what you build.


Big Design Analogy: Cannon Fire

• You have an old cannon, a barrel of cannon balls and a barrel of gun powder. Hit the target with a cannon ball.



• On the other hand, if you just shoot the cannon a few times, you are likely to get it by trial and error in a few tries. (Ready, Fire, Aim)


BDUF - Small Bites

“Economies of scale” is a dangerous concept. Making things bigger can make them useless.


BDUF alternative - Small Bites

Small bites flow. Big bites hurt, and they might kill you.


BDUF alternative - Small Bites

• Build one feature at a time.– Design it, Code it, Integrate it, Test it, Release it.

• Break big features into little ones.– This is where the art comes in.

• It isn’t done until it is integrated and tested, and maybe released.

• Add new features to systems that work.


BDUF alternative - Agility

• The old rule was to get the customer to sign the plan, and then to deliver what they signed for, whether it is what they need or not.

• Now, you want the customer to tell you what is wrong with the plan. You want to deliver what the customer needs.


BDUF alternative - Agility

• You can’t afford to finish the whole project and then have the customer say, “No, that’s not right. Start over.” So, you have to deliver in small increments.

• If they like most of what you give them, you can calculate how long it will take to give them what they need.

• If they don’t like what you are delivering, you can “cut bait” early or have an executive conference while there are resources left.


BDUF - Change Tolerant Software

• Chances are, you will be forced to make many changes to your module before the test team and the customer like it.

Don’t insist on making it right (you don’t even know what right is yet.) Make it flexible.

Preserve ease of use, security, and simplicity, but sacrifice efficiency to get flexibility.



• Low dependence between modules - They used to call it information hiding, but it is really all about protecting your module from someone else's changes.



• Interfaces -Be rigorous in your specification, communication and preservation of interfaces.



• Object Orientation -Each device, database, user interface, or external software needs to have a isolated interface, so that anything can be changed without effecting more than one object.



Object oriented languages support encapsulation which helps to simplify your software by separating into chunks.

But Inheritance and Polymorphism can sometimes add complexity. Use them carefully.


Big Design Up Front

• Do just enough design up front

• Implement the project in small steps

• Learn from the current step, before you plan the next step.

• Each step should deliver value.

• The steps reduce the surprises and risk.


4 Flow

“Economies of scale” is a dangerous concept.

More and faster can lead to chaos.


Flow

Tight Tight feedback feedback between between train cars train cars allows allows hundreds hundreds of cars to of cars to travel travel togethertogether


Flow

Figure where the “tracks” run.

Keep things moving, but don’t allow pile-ups.

Try to eliminate waiting time.

Faster is better.

Faster delivery means faster feedback.


Work In Progress

The work that youare scheduled to docan get in the wayof the work youare supposed tobe doing now.


Work In Progress

Juggling takes a lot of energy and concentration. You get less useful work done when you are juggling, and some things get dropped (rot).


Work In Progress

• Small bites (iterations, features…) give you less to keep in your head and for less time.

• Try to finish one task before starting another. (Juggling is hard, but you can run with just one or two balls.)

• Break for another task, only if you are stuck. (You are now starting to juggle. Two balls is okay, three is possible. Don’t do four. This is not a circus; you get no points for juggling.)

• Reorganize features, schedules, team structure….anything to minimize the people who can’t finish the task they started.


Work In Progress

Modern scheduling systems try to keep people busy all the time by giving them lots of task to juggle.

It keeps people busy, but the lack of focus cause lots of errors that lead to a long debug cycle on every project.


5 Team Agility

What you thought your specialty was is not as important as finding new ways to get everyone to the goal


Team Agility

• Team members need broad skills

• For each job, there are other people who can fill in if necessary.

• Since nobody gets credit for a feature until everyone is finished, everyone helps the other team members.

• Everyone looks for better processes.


Team Agility

Testing holds up most projects, because testing is last, and testers don’t build tools. In Agile teams, programmers who are idle, because their features are not tested yet, build tools to make testing go faster. This alone is enough to explain the increased productivity of Agile.


6 Continuous Integration

Assemblies integrate into bigger assemblies that integrate into cars.Parts don’t stack up anywhere.


Continuous IntegrationThe longer you wait to integrate your work, the more likely that someone else changed something you need.

The longer you wait to integrate, the more likely that you will have forgotten some of the changes you made.

The longer you wait to integrate, the more rework you will have to do when you find a bug.

Make it, integrate it, test it.


Continuous Integration

• Modern CI tools use hierarchies or other models to allow changes to flow up to the final product.

• Conflicts at lower levels can be resolved with less pain to the whole team.

• Small changes are easier to synchronize, resolve or back out.


7 Configuration Management

• If you could just go back to where you were a few minutes ago……


Configuration Management

• Personal backups for your work

• Backups for each part of the build

• Backup to previous releases

• Backup historical tools

• Backup historical data

• When you get in trouble, go back.


8 Build Tools

• This guy had about the same brain as your programmers, but his tools sucked.


Build Tools

Do you suppose these guys use a dip stick to check the oil each week?


Build Tools

Let the robots do any repeated tasks.

Give humans creative tasks.


Build Tools• Programming requires huge amount of

cleverness that can not (yet) be automated.

• But there are also many repetitious tasks.

• Automate everything you can to get the most productivity out of the humans.

• Early investments pay off big.


Build Tools

Sometimes a tool builder creates a tool from scratch. Other times, it is less work to buy a tool, learn it, and adapt it. Some clever scripting around a tool is often enough to make it do what you need.


9 Test First

How long should you wait to test your blood sugar after taking insulin? Why wait.


Entropy Example 1

• Who did you talk to on the phone?

• Today? I remember

• This week? Check the call log

• 5 weeks ago Find the phone bills, then find last month (You might have to check on line. Do you remember the URL and password?) Then scan through for the week…..


Entropy Example 2

You dropped your keys:

• 5 seconds ago Look down

• 15 minutes ago Back track

• Three days ago Make a list of the places you have been. Call public places that have lost-and-founds. Post signs in popular places. Think about changing your locks.


Test First

• The longer you wait to test, the less efficient it will be to find the bugs.

• The more stuff you integrate before you test, the less efficient it will be to find the bugs.

• The longer you take to write the test the more the spec will have rotted.


Test First

• Specification and testing are the most dangerous part of a project.

• If you use tests as a specification, the tests are ready before the code is done.

• Debugging becomes a part of development, not a project phase.

• Customers know what to expect.

• Testers become part of the design team


Automate Testing

Flow requires nearly instant feedback.

Automated testing is the only way to do that, but even automated tests take time.

PCs are cheap. Implement massively parallel testing.


Testable SoftwareIt used to be bad style to leave any test software in the production code. Now, it is the only way to get the testing done.

• Implement interfaces to give test software access to critical data.

• Pass test data up from low level routines.

• Build command level interfaces used by both the UI and test software.


Test First

UI’s are constantly in flux.

Do not put your tests over the UI.

Design-in test interfaces.

Use frameworks like FIT.


Questions?

Tomo Lennox:

Cell - 612-385-4326

Email - [email protected]

Blog - tomoLennox.com

mailto:[email protected]

keys to software success efficient software processes

Documents

software success12

software success7

software success5

software success4

software success9

software success11

stable systems

big systems