ral14114usen

A Forrester Consulting

Thought Leadership Paper

Commissioned By IBM

October 2014

The New Software

Imperative: Fast Delivery

With Quality Eight DevOps Practices Are The Key To Success

Table Of Contents

Executive Summary ........................................................................................... 1

The Race Is On To Deliver In Under Three Weeks With Quality .................. 2

The Challenge: Achieve Consistently Fast Cycle Times .............................. 4

To Deliver Fast With Quality, Adopt Eight Key DevOps Practices .............. 5

Working Smarter Achieves Rapid Delivery Cadences .................................. 8

DevOps Eliminates The Tradeoff Between Cycle Time And Risk ............... 8

Key Recommendations ................................................................................... 10

Appendix A: Methodology .............................................................................. 12

Appendix B: Demographics/Data ................................................................... 13

ABOUT FORRESTER CONSULTING

Forrester Consulting provides independent and objective research-based

consulting to help leaders succeed in their organizations. Ranging in scope from a

short strategy session to custom projects, Forresters Consulting services connect

you directly with research analysts who apply expert insight to your specific

business challenges. For more information, visit forrester.com/consulting.

2014, Forrester Research, Inc. All rights reserved. Unauthorized reproduction is strictly prohibited. Information is based on best available resources. Opinions reflect judgment at the time and are subject to

change. Forrester, Technographics, Forrester Wave, RoleView, TechRadar, and Total Economic Impact are trademarks of Forrester Research, Inc. All other trademarks are the property of their respective companies. For additional information, go to www.forrester.com. [1-Q4RGU3]

1

Executive Summary

The pressure to quickly develop and deliver quality software has entered a phase of

new intensity. Most shops report being pressured to cut their cycle times.1 And its no

wonder: Demands for new customer-oriented software, new systems to counter digital

competitors, and new software-driven business innovations make application

development crucial to business success.

IBM commissioned Forrester Consulting to conduct research on development

practices and cycle times within businesses to help evolve its DevOps maturity model.

Through a survey of 600 IT professionals with application development responsibilities in the US, Canada, France, Germany,

and the UK, Forrester found that about a third of teams can consistently deliver at cadences of one to three weeks. The

fastest teams generated far better business satisfaction than slower teams, an indication they dont sacrifice quality to deliver

fast. Our study reveals how these teams were able to break through to achieve consistently fast software delivery with high

business satisfaction: by adopting eight DevOps practices to replace waterfall methods and surpass Agile methods.

DevOps practices will be the new normal as software development teams respond to business needs for more software

delivered faster at an efficient cost and with high quality. Without adopting DevOps practices, teams will fail to deliver quality

software consistently on fast business schedules.

KEY FINDINGS

Forresters study yielded five key findings:

Development teams that consistently deliver at the fastest cycle times enjoy the highest business satisfaction. We found a high correlation between fast cycle times and business satisfaction in our survey results. In this study, high

business satisfaction is an indicator of software quality. The reverse is also true teams delivering at the slowest cycle

times self-reported the lowest business satisfaction with their projects.

Incremental improvements to waterfall methods run out of steam at one- to two-month delivery cycles. Our findings strongly suggest that sustained cycle times of two months or less are impossible to achieve with incremental

improvements to waterfall methods. The overhead of those regimes even when supplemented by Agile methods

eats up too much of a multiweek delivery window to succeed. To consistently deliver quality software at the fastest cycle

times, teams adopt new methods and software designs and leave waterfall behind.

Eight DevOps/continuous delivery practices are the key. DevOps/continuous delivery relies on eight practices: 1) deliver small increments of functionality; 2) use dedicated, cross-functional teams; 3) use loose architectural coupling; 4)

automate environment provisioning; 5) continuously integrate code; 6) continuously test; 7) continuously fund; and 8)

provide real-time transparency.

DevOps practices address the top reasons for project disappointment. DevOps practices directly address the reported causes of business dissatisfaction with software projects. We found the practices used by the fastest teams to

achieve high business satisfaction were a mirror image of the practices the slowest teams reported caused low business

satisfaction.

DevOps practices reduce cycle time and the risk of failure at the same time. Conventional wisdom and intuition say that faster release must come at the expense of higher risk of project failure. DevOps techniques turn this logic on its head.

DevOps practices enable faster and safer releases because they are smaller, less complex, and more thoroughly tested.

Automating provisioning and deployment further reduces configuration errors and speeds up delivery.

Development teams

that delivered software

fastest enjoyed far

better business

satisfaction than

slower teams.

2

The Race Is On To Deliver In Under Three Weeks With Quality

Software development and delivery has always been a difficult race against time. Now, that race has entered an even more

challenging phase. Software delivery schedules measured as cycle time from project start to delivery of working product

have followed a downward curve since the 1960s, dropping further with the advent of each major era in our industry. The

latest era focuses on software for and about customers, including consumer apps, connected products, B2B sites and apps,

and all manner of systems to enable and support citizens, students, soldiers, parents, and so on.

FIGURE 1

The Age Of The Customer Demands Faster Software Delivery

Years

1960s

Data Processing Batch automation of accounting, back office

IT Database, online systems-of-record,and PCs automate front office

The Internet eBusiness brings external(Web) access to internal business processes

The Customer Mobile and socialempower customers; systems of engagementwin, serve, and retain them

1980s

1990s

2010

RequiredCycleTime

Year

Days

Source: Forrester Research, Inc. 2014

3

Why? To start, customer-facing software must keep pace with changing tastes, needs, and competition. And theres a lot of

demand for customer-facing software. Consumers are as fickle as ever, but now have more choices for products and

services. Your hot offering last week will shortly be old news and youll have to build new software or modify existing

software to create a new advantage.

But theres more (see Figure 2):

Digital competitors firms that use software to disrupt established markets are everywhere, even in staid industries like power generation/distribution, travel, and banking. Digital businesses move faster than hardware- or

people-based businesses because software can evolve faster than physical things. The result: escalating demands for

new digital services.

Software success is increasingly indistinguishable from business success. All business innovation requires new software, changes to software, or both. And business innovations cant wait for long software cycles to finish. The result:

demands to test business ideas in software within days or hours at a low risk and low cost.

No shop is immune from these business pressures. As a software delivery community, we need to raise our game to deliver

faster. How much faster? Our research suggests that one- to three-week cycle times are the gold standard. Software

delivery will utterly change to hit that mark.

FIGURE 2

Whats Driving Fast Delivery Requirements And How Well Respond

New App Delivery Models

As a service APIs

Mobile apps

Apps composed of services

DevOps/continuous delivery

New App Delivery Platforms

Cloud platforms

SaaS

Continuous Integration DevOps platforms

New Business Models

Open source

Subscriptions

On demand

Deliver fast to keep pace with fads, market shifts, opportunities

Deliver more digital business products, services, channels

Deliver quickly to prove out business ideas and innovations

Empowered,demandingcustomers

Increasingdigital

competition

Increasingexpectationsof software

Source: Forrester Research, Inc. 2014

4

The Challenge: Achieve Consistently Fast Cycle Times

Some teams in our study are producing software at the fastest cycle times today, which in our study we defined as one to

three weeks. We can learn from their experiences. As Figure 3 illustrates, about a third of our sample delivers all or some of

its work within one- to three-week cycles a group we call Delivering Fast. This group was also the most highly motivated

to reduce cycle times, and that is no coincidence. A second sizable group can sometimes deliver within three weeks, but not

consistently.

FIGURE 3

We Found Four Development Team Segments

High

Motivationto cut cycle

times

Ability to maintain fast cycle timesLow High

Delivering Fast32%

Improving Speed37%

Struggling forSpeed19%

DeliveringSlowly13%

Base: 600 IT professionals with application development responsibilities from the US, Canada, UK, France, and Germany

Source: A commissioned study conducted by Forrester Consulting on behalf of IBM, May 2014

5

As well see, the delivering fast group succeeds because its development and delivery practices are tuned to delivering

quality software quickly. Anecdotally, introducing these new practices comes with the pain of any big change in the way

people work. Our studys results indicate that the pain is justified by the results.

Among the four groups of development teams in our study, when asked how successful they were at meeting the needs of

their individual, or companywide, business leaders, the delivering fast group enjoyed the highest rates of business

satisfaction with its work and the lowest negative ratings at the two fastest cycle times by far. The contrast between success

and failure is highest at the fastest cycle times, highlighting the need to change practices to consistently achieve those cycle

times (see Figure 4).

Delivery success is difficult to measure. Our study relies on self-reporting of how cycle times and dozens of software

development practices correlated with the project meeting the demands of the business leaders associated with the

respondents team.

FIGURE 4

The Fastest Teams Achieve High Business Satisfaction At Fast Cycle Times; The Others Do Not

Does the work you deliver at these cycle-times tend to be more

successful or less successful at meeting the demands of your business leaders?

Less successful at meetingbusiness demands

More successful at meetingbusiness demands

Delivering Fast

Delivering Slowly

Struggling for Speed

Improving Speed

1-3 weeks

1-3 weeks

1-2 months

1-3 weeks

1-3 weeks

1-2 months

1-2 months

1-2 months

25%

42%

30%

40%

21%

22%

6%

14%

15%

11%

27%

8%

26%

27%

53%

48%



6

To Deliver Fast With Quality, Adopt Eight Key DevOps Practices

How do the teams that consistently deliver in two months or less do so? Their software practices show the way to fast

delivery with quality. Moreover, their experience suggests that to deliver consistently in two months or less, software teams

must adopt eight key practices:

1. Deliver in small increments of functionality. Fast delivery means building on a minimum viable product by delivering

value in the smallest increments possible. Small releases provide the fastest path to customer value realization. Small

releases mean fast feedback, which focuses effort on building the right things. Small releases also minimize risk by

reducing complex release dependencies.

2. Form dedicated, cross-functional teams. Sharing resources between teams wastes time, complicates scheduling,

weakens collaboration, reduces team throughput, and dilutes business expertise. Overspecialization and role silos kill

productivity and reduce delivery effectiveness.

3. Use loose architectural coupling within and between applications. Loose coupling allows straightforward change of

whole applications or components within applications. It can greatly reduce complexity and is one of the key enablers of

delivery in small increments.

4. Automate environment provisioning. Lack of consistent environment configurations causes hard-to-diagnose

production outages resulting from the it works fine on my machine syndrome. Long delays in getting suitably configured

development and test environments cause delay and increases cost. Automating provisioning dramatically reduces cost,

improves the effectiveness of development and testing, and reduces the number and severity of production incidents.

5. Continuously integrate code. Continuous Integration (CI) the building and testing of code every time a change is

delivered promotes faster user feedback. Early and continuous integration the building and testing of change sets

together exposes the most uncertain issues first and enables feedback on the most valuable usage models earlier.

6. Continuously test. Continuous testing results from extending CI with API-driven test automation that executes every

time code is changed. When enabled by loosely coupled architectures, continuous testing can automate the execution of

all functional and nonfunctional testing, leaving only UAT and exploratory testing for manual testers. Continuous testing

dramatically improves delivery speed, quality, and testing accuracy, while dramatically reducing cost.

7. Continuously fund. Organizations that deliver at rapid release cycles create a stable basis for funding a continuous

stream of innovation. They use customer feedback and business priorities to continuously prioritize the capabilities that will

result in the greatest business value.

8. Provide real-time transparency. The overhead of periodic and manual status reports and status meetings absorbs

valuable time and distracts focus from delivery value. Increasing transparency through instrumentation of the evolving

code and test base improves visibility into progress, and risk without decreasing productivity.

The survey results show a close correspondence between the practices of the delivering fast group and the barriers to

business success reported by the slowest delivering group (see Figure 5). The delivering slowly groups two biggest

barriers team members working on too many projects at the same time and too much status reporting and too many

meetings are the mutually reinforcing consequences of waterfall processes. No amount of incremental process

improvement will eliminate those barriers.

7

FIGURE 5

Delivering Fasts Strengths Are Delivering Slowlys Weaknesses

What are the characteristics of the projects that are most successfulat meeting the expectations of your business leaders?

(Delivering Fast only)

What are the characteristics of the projects that are least successfulat meeting the expectations of your business leaders?

(Delivering Slowly only)

53% 53% 52% 51% 51%48% 46%

20%

RequirementsTeam-memberscope

DesignUnits ofwork

Specialization& handoffs

Projectvisibility

Managementoverhead

Testing

26%

50%

23%28% 28% 28%

50%

33%

RequirementsDone In

Increments

TeamMembersAssignedTo OneStream

LooselyCoupledDesigns

SmallBatch

Releases

Cross-FunctionalTeams,Few

Handoffs

HighVisibility

intoProgress

FewStatusReports

&Meetings

AutomatedTesting

AllRequirementsDone Upfront

TeamMembersAssigned

ToManyProjects

TightlyCoupledDesigns

LargeBatch

Releases

SiloedSpecialists,

ManyHandoffs

LowVisibility

IntoProgress

ManyStatusReports

&Meetings

ManualTesting



8

Working Smarter Achieves Rapid Delivery Cadences

Working more frantically to meet release dates will improve cycle times only incrementally. To achieve fast and predictable

releases consistently requires simplifying the work and automating everything possible. Hybrids of Agile methods and

traditional approaches (often called water-SCRUM-fall) can produce one- to two-month releases sometimes, but achieving

three-week cycle times consistently, inexpensively, and with high quality requires consistent use of DevOps practices and

supporting tools (see Figure 6). Traditional methods simply cant meet the mark.

FIGURE 6

DevOps Essential To Leap The Fast-Delivery Chasm

Delivery Cycle Times

Degree ofRisk

12+months

6-11months

3-5months

1-2months

1-3weeks

Days

Waterfall andisolated Agilepractices cantcross the chasm

DevOps practices,automation, and toolingare essential to fastdelivery with low

failure risk

Source: Forrester Research, Inc.

9

DevOps Eliminates The Tradeoff Between Cycle Time And Risk

Conventional wisdom and intuition say that a faster release must come at the expense of higher risk. Organizations steeped

in conventional delivery approaches assume that the only way to deliver faster is to cut corners, and the corner they most

often cut is testing. The result? Increased production incidents and lower quality. DevOps techniques turn this on its head;

faster releases, enabled by DevOps practices, are safer because (see Figure 7):

Faster releases demand smaller and less complex increments. Smaller, less complex releases are easier to deploy and easier to roll back when needed. They also have fewer dependencies because they use loose coupling.

Faster releases demand more test automation, which results in more thorough and continuous testing. DevOps practices like continuous integration and continuous testing produce higher quality because they find defects early, when

they are easier to fix, and they are more thoroughly tested. Continuous testing and service virtualizations enable

comprehensive testing from the start of the delivery cycle, shattering the conventional risk-speed tradeoff.

Automated environment provisioning can prevent and minimize configuration errors. Preventing configuration errors prevents production incidents that testing cant catch. Eliminating the root cause for these errors not only reduces

risk and improves stability, but it also eliminates costly incident response.

FIGURE 7

DevOps Practices Have A Counterintuitive Effect On Risk

HighIn waterfall practices,the risk of failurequickly rises asdelivery speedincreasesDevOps

practices enablefast delivery andfalling risk offailure at thesame time

Risk ofFailedRelease

Delivery Speed

Low

Years

This is a conceptual diagram.

Days

Source: Forrester Research, Inc.

10

Key Recommendations

DevOps/continuous delivery is a departure in practices for most teams and larger application development

organizations. We describe it as a revolution, but it is a revolution that unfolds over time. The danger is that application

delivery leaders will confuse incremental improvement of practices incapable of ever meeting the need for speedy

delivery of quality software with progress toward a new fast-delivery regime.

The start of your journey will depend on what kind of shop you are now. The delivering fast teams have already

arrived; they need to extend their success into the full measure of DevOps. The three other segments usually need to

start with the basics. If your teams are in the delivering slowly, struggling for speed, or improving speed categories,

here is how to start your journey (see Figure 8):

Start with consistent environments. Effective, automated on-demand environment provisioning provides the foundation for nearly all subsequent improvements. It reduces production incidents by preventing configuration

drift between development, test, and production environments. A closely related practice is the use of service

virtualization, which enables services and applications to be simulated without actually provisioning them. All

delivery cadences and all software life cycles, both waterfall and agile, benefit from these practices.

Reduce management overhead. Project status reporting overhead is often an empty ceremony that decreases the probability of success by diverting valuable resources to non-value added overhead activities. Providing

transparent visibility into progress and priority information, to anyone who needs it, eliminates pointless ceremony

while providing better access to more accurate information. Everyone knows the status at all times.

Stop multitasking. Focusing staff on one initiative at a time reduces task-switching time, eliminates schedule coordination, reduces unproductive wait time, and frees people to do productive work. Spreading resources

across projects reduces throughput due to the overhead of context switching and wait time.

Automate the integration and release process. Continuous integration and release automation provide the engine for the delivery process that eliminates manual errors, dramatically reduces manual effort, instruments the

product delivery pipeline and enables further lean transformation. Manual integration and release processes

simply cannot keep pace with the rapid flow of work.

Automate testing using APIs. Manual testing is expensive, inconsistent, hard to manage, and slow. Automated UI-based testing is expensive and usually happens too late in a release to achieve desired coverage rates. API-

based testing can start as soon as code is delivered in the CI process; it achieves high coverage rates and, once

developed, is virtually free. Shifting testing left is one of the key practices enabling faster delivery.

11

FIGURE 8

Where Should You Start?

DeliveringSlowly

1. Paperwork, management overhead

2. Too much manual testing

3. Too many concurrent projects

1. Project status visible to anyone, anytime

2. Automate most testing

3. Small batches4. 1 team on 1 project at a time

Strugglingfor Speed







IncreasingSpeed







Segment Top Pains DevOps Practices Priorities



12

Appendix A: Methodology

In this study, Forrester conducted a global online survey of 600 IT professionals with application development responsibilities

from organizations with an IT budget of $10 million or more from the US, Canada, UK, France, and Germany. Survey

participants included those in manager level or higher, with oversight of or direct involvement with application development

responsibilities. Respondents were offered a small incentive as a thank you for time spent on the survey. The survey began

in April 2014 and was completed in May 2014.

Its important to understand our use of terms in the survey and in this paper. In some cases, Forrester uses different

terminology to: 1) elicit information from respondents and 2) analyze and explain survey results. Why?

Our survey tested the hypothesis that certain key development and delivery practices contribute to a teams ability to deliver software at fast cycle times and other practices prevent fast cycle times. We phrased our

questions using generic and self-evident terms, avoiding either leading respondents toward foregone conclusions or

confusing respondents with industry jargon and unfamiliar terms.

We asked respondents for the same information about their practices in three different ways. First, we asked them directly about their practices across the project life cycle, starting with funding. Second, we asked them which practices

lead to project success or failure. Third, we asked them what theyd change if they could to improve business satisfaction

with their teams software project(s).

DevOps Practices Supporting Survey Questions (Examples)

Deliver in small increments of functionality How often do the following characteristics apply to the definition of your requirements? [Choice: Team

members initially define a minimal product concept and

seek to validate the concept as quickly as possible.]

Form dedicated, cross-functional teams What are some of the characteristics of the projects that are most successful at meeting the expectations of

your business leaders? [Choice: We dedicate team

members to one project at a time.]

Use loose architectural coupling within and between applications

How often do these [design/structural] characteristics apply to your applications? [Choice: The applications

are composed of loosely coupled components or

services.]

Automate environment provisioning Given total authority, what would you start doing /do more of to better meet the demands from your

business leaders? [Choice: We would automate more

of the production deployment process.]

Continuously integrate code What are some of the characteristics of the projects that are most successful at meeting the expectations of

your business leaders? [Choice: Each function is an

atomic unit of work that passes as quickly as possible

through the SDLC and into production.]

Continuously test How often do these characteristics apply to your testing processes? [Choice: Testing begins as soon as there is

some code to test and proceeds in parallel with

13

development.]

Continuously fund How often do the following budgeting/funding characteristics apply to the initiation of your software

projects? [Choice: Funding is based on an investment

stream for the product were building.]

Provide real-time transparency Given total authority, what would you start doing /do more of to better meet the demands from your

business leaders? [Choice: We would make the entire

development process visible to team members, other

teams, and all stakeholders/levels of management.]

Appendix B: Demographics/Data

We identified the four segments used in this paper using a cluster analysis of our survey responses. See Figure 9 for the

results of that analysis.

FIGURE 9

Cluster Analysis Groupings

Deliver most/ somework in 1-3 week cycle

Pressured to make deliverydates more predictable

Pressured to reducedelivery cycle time

Pressured to reducedelivery cost

DeliveringFast(32%)

100%

100%

78%

74%

Improvingspeed(37%)

54%

0%

56%

50%

Strugglingfor Speed(19%)

0%

100%

80%

54%

DeliveringSlowly(13%)

0%

45%

0%

100%


Source: A commissioned study conducted by Forrester Research on behalf of IBM, May 2014

14

FIGURE 10

Country Distribution

Canada

0%

15%

16%

55%

14% 5%

14%

12%

26%

42%Delivering

FastStrugglingFor Speed

DeliveringSlowly

ImprovingSpeed

1%

22%

12%

22%

44%

2%

20%

23%

23%

32%

France

Germany

United Kingdom

United States

Country Distribution Of Respondents



15

Appendix C: Endnotes

1 In our survey, conducted during May 2014, only 13% of respondents reported having no pressure to reduce cycle times.

FIGURE 11

Industry Distribution

Manufacturing, energy

DeliveringFast

DeliveringSlowly

ImprovingSpeed

Retail and wholesale

Utilities, telecoms

Business services, construction

Media, entertainment, leisure

Financial services (all)

Public sector, healthcare

Other

StrugglingFor Speed

19%

23%

7%13%

6%

17%

14%

2%

14%

7%

9%

11%

12%

30%

7%

10%12%

4%

12%

10%

4%33%

17%

9%

14%

11%

12%

14%

1%

23%

12%

13%

Industry Distribution Of Respondents



ral14114usen

Documents