Unsustainable Software Development and its Causes

Lecture 1CIS 6101 – Software Processes and Metrics

Outline

• 1. Background of Unsustainable Development • 2. Causes of Unsustainable Development• 3. Project Controls• 4. Summary

1. Background of Unsustainable Development

Definition• Unsustainable development is characterizd by

constantly increasing the cost of change to the software.– Evidence of a high cost of change is a constantly increasing

number of defects.– How does this happen? <Discuss>

• Each change – adds complexity and – uncovers or causes of defects that require more changes.

• This complexity leads to a declining ability to respond to customer requests.

Fixes versus New Features• Unsustainable development is all too-common today in

software industry.• We place too much emphasis on

– short term feature development (new) and – fixing defects (existing) and – NOT paying attention to the health of the underlying software.

• Result: Cost of change rising with new changes coupled with the risk where each change can destabilize the product.

• <Discuss: What does it mean to ‘destabilize’ the product?>

Fixes and Stability of Software• Here, teams spend more and more time fixing defects and

in stabilizing software.• Features – Do get developed, but – Less time available due to time for stabilizing the software.

• Development teams want to freeze requirements • But this reduces ability to respond to client requests.– Newer methodologies embrace customer involvement!!

• So, much time spent on stabilization and not on new work.

Threshold Reached – When?• When team spends much time on defect backlog and not enough time

developing new features.• When: occur within a few weeks, months, years.• Finger pointing and blame game.

• Consequences: – Morale down; – job layoffs; – outsourcing; – poor customer satisfaction.

• Most teams, however, do not live with unsustainable software but get uncomfortably close.

• <Discuss: CMDS; the methodology (process); reactions!>

Technical Debt• Underlying cause of inability to develop new

features due to defect burden is attributable to technical debt.

• Cause of Technical Debt: caused by build up of poor decisions over time

• Cause of Technical Debt: decisions that seem right but often made for short-term fixes.

• <Discuss: How does this happen?>– Rarely a single decision is the cause.

The Perils of Jumping in Place• Evidence of tech debt is clear.

– When the number a products must be completely rebuilt to keep up with competitive products or

– When products need to be rebuilt to respond to major technology changes.

• Rewrites require – massive investments in time and effort and – often results in loss of market share while the replacement product is

being built.

• Often impossible to recover market share where competitors get stronger and new competition enters marketplace.

2. The Causes of Unsustainable Development

The Causes of Unsustainable Development

• Why is unsustainable development so common?• A number of reasons called Product Stresses– There are things we cannot control (project stresses)

and, – There are a number of things we can control (project

controls).• Different for every project; stresses change– <Discuss Stresses without looking ahead…>

• Let’s consider Project Stresses.

Causes of Unsustainable Development – External Dependencies

• Dependencies on versions of the OS, compiler, IDEs, web browser components, etc.– Impossible to dictate at the end of a development

environment what the dependencies might be.

• Must often develop to support both old and new interfaces, such as two different versions of IE, netBeans, etc…

Causes of Unsustainable Development - Competition

• Very easy to set up a software development company nowadays. – Need few bright people, a few computers, and Internet

• Many do not have – a workable business plan and – a clear idea of their true market worth due to overestimating – do not know their market size.

• Almost always suffer from inability to adequately sell or market their work.– Usually get initial funding and make some initial sales, – but struggle to stay in business.

Causes of Unsustainable Development - Disruptive Technologies

• Constant advances in state of the technology can be disruptive.

• Products need to meet users’ demands over the long term.

• Cannot restrict thinking for ONLY what the customer wants, because customers may initially be interested in immediate needs.

• But customers may be offered more features on newer technology for less price

• The Internet is a disruptive technology – more and more support can easily be offered over the Internet.

• Companies must learn how to deal with the Internet and its threats too (illicit software, hackers, a forum for malicious people to exploit security flaws, and more.)

Causes of Unsustainable Development - Disruptive Business Models

• Consider Dell: They have a disruptive business model.• Their model allows them to produce computers on demand by

– clustering their suppliers together and – taking all orders over the Internet.

• Other companies (HP, Compaq, IBM) have not been able to keep pace or duplicate Dell’s business.

• Open Source software. A disruptive role!– Open source is ‘free,’ and usually usable by highly technical

people. – No question that open source will play an increasingly

disruptive role in the software industry

Causes of Unsustainable Development - Disruptive Business Models

• Microsoft, while having a monopoly on desktop market, cannot compete against free software.

• Linux: can its business model be sustained by keeping itself going almost exclusively through support and services as claimed.

Causes of Unsustainable Development - Cost Management

• Software Projects are exceedingly expensive!

• Difficult to control costs through greater efficiency due to mindset of the organization, its processes, training, deadlines, etc.

• Best way: employ less expensive people.– This explains the increasing use of outsourcing.

• But efficiency is at the heart of sustainable development.

3. Project ControlsThere are things we can do and we can control.

Project Controls – Collaboration

• Must work with other people within the organization– May involve others • with similar development projects or • With a history of similar projects

• Must work with your customers!!– May involve end-user representatives on staff

Project Controls – Methodology• Development methodology – the principles (mindset) and

practices of planning, design, testing, coding, etc.) has a dramatic impact on the success of a project.

• Methodology directly influences – efficiency, – change tolerance, – responsiveness, and – software quality.

• <Discuss these bullets>

Methodology – Plan Driven Development - 1

• Some stick to a plan-driven methodology (which works well sometimes) because this is what is taught in school especially to engineering.

• If this doesn’t work, they may use a variant such as code-then-fix approach.

– Here, teams spend lots of time writing software and very little time on planning and analysis.

– Software is brittle and will resist change.– Lack of sound practices leads to a great deal of dead code and

thrashing.

Methodology – Plan Driven Development - 2

• Plan-driven development, derived from engineering, is best typified by the ISO 9000 set of standards. <Good research topic>

• Downside of plan-driven approach– some software developers feel all the complexities of the

software can be understood before any code is written.

• This is rarely true and definitely false over the long term.– <Discuss: When might this be true?>

Plan Driven Development - 3• While documentation of requirements, design, test plans, etc.

have value as a means of communication, their real value comes not from the documentation but rather from the process one must go through to produce documentation.

• Working though a design with your peers is a– valuable exercise; – fosters collaboration, understanding, and creativity; – The document is of less value and must be maintained.

• <Discuss: is the document used?>• <Discuss: “must be maintained?”>

Plan Driven Development – 4Negatives on Documentation

• Work gets done; but result is too often time spent producing and maintaining the documents detracts from amount of time spent on analysis.– On the other hand, if ‘maintained’ these represent a living, real set of specs,

design, maintenance manuals…

• Plan-driven software does not work for software development because it is impossible to put together a plan when the project starts to take into account all the changes in requirements and in the environment that will occur over the course of the project.

<Discuss: Is this really true??>

Plan Driven Development - 5• Plan-driven approach stifles agility, because– They emphasize ceremony and – Assert documentation over having a working product in the

erroneous belief that somehow having a pile of documentation describing requirements and design is proof that • That the project cannot fail, • that a discipline is in use, and by extension, • That a project without the depth of documentation does not have

discipline.

• What is really needed is/are working projects in user’s hands such that feedback is produced, since this is the only real way to know if the desired system meets user needs.

Project Controls – Expertise

• This will be a strong indicator of success.• Will be able to understand what is really

important and what is not.

Project Controls – Decision Making

• The ability to make decisions in – a complex and in constantly changing environment – often with incomplete information is crucial to

success.

• An important component of this is prioritization, where you chose what must be worked on and when.

Project Controls – Leadership

• Helps to set up and communicate a vision, strategy, and tactics.

• Keep projects on track.

Project Controls – Culture• Organizational culture shapes the attitude and practices

of the people who work on a project.

• Culture defines the requirements • for leaders, and • for how decisions will be made and by whom, and the

methodology.

• If new leaders or a decision-making style is required, then the culture must change.

Project Controls – Simplicity and Reliability - 1

• Most software today is – overly complex, – hard to use and learn, and – has too many features for most of its users.

• Think about word processors: – have features that most users will never use.

• Interesting: Word2007 had 265 functions; – only 12 were used by more than 75% of its users; – 42 were not used at all!

• Ill-effects of unnecessary complexity on users? Hard to say.• Complexity is behind at least part of the outsourcing trend

because one way to manage complexity is to get someone else to do it for you.

Project Controls – Simplicity and Reliability – 2

• Huge disparity between what consumers want and what companies would like to sell them.

• Too many companies still compete on features and price alone. – Often encouraged • by the most technical users who are the early adopters

of any technology and • by the media, who still publish reviews that contain

feature comparison charts.

Project Controls – Simplicity and Reliability – 3

• Related to simplicity is the need for reliability.– Too many software organizations emphasize features over reliability, as many users

refuse to buy the first version of any software, and– Users are frustrated by version upgrades that break or remove features relied upon in

previous version.

• There’s not enough emphasis placed on simplicity and reliability. – Consider the iPod or the Palm PDA. Successful because they do few things very well.

– There was a great deal of restraint in their development.

– Easy to add features, but difficult to show restraint to keep them out.

4. Summary - 1• Unsustainable development is a development pace that is

typified by stress, frustration, and a sense of not being in control.

• Evidenced by continually increasing the cost of change and defect rate and corresponding decreasing ability to respond to changing conditions.

• Unsustainability may not be apparent to the people on the project due to the varying pace of decline. But it is there! <Discuss: It is really there. No time to see it>

• Some unsustainable development comes from continual change and stresses that are out of control of the organization.

4. Summary - 2

• Let’s look at the principles and practices and outline some key principles that leads to sustainability.

• We have the principles and these are used to frame the practices that reinforce them.