© 2014 IBM Corporation

DevOps 101Concepts and OverviewWai Lee

Vice President

Digital Platform Engineering

Prudential Financial

© 2013 IBM Corporation

Sanjeev Sharma

IBM WorldWide Lead – DevOps Technical Sales

DevOps Blog: http://bit.ly/sdarchitect

Twitter: @sd_architect

Author: DevOps For Dummies –

http://ibm.co/devopsfordummies

Please Note

IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion.

Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision.

The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion.

Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user’s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.

Agenda

• What is DevOps?• Adopting DevOps

o Peopleo Processo Technology

• Implementing Delivery pipelines

Agenda

• What is DevOps?• Adopting DevOps

o Peopleo Processo Technology

• Implementing Delivery pipelines

A lack of continuous delivery impacts the entire business enterprise in the new reality of “Systems Of Interaction”

>45%of customers experience production

delays

>50%of outsourcedprojects fail to

meet objectives

>70%of budgets devoted

to maintenanceand operations

4-6 weeks

to deliver even minor application changes

to customers

Systems of Interaction

Continuousclient

experience

Partnervalue chain

Cloud-basedServices

Systems of Engagement Systems of Record

OperationsRapid app releases impact

system stability and compliance

SuppliersDelivery in the context of

agile

Development/TestSpeed mismatch between faster moving front office and slower moving back office systems, delaying time to obtain feedback

Line-of-businessTakes too long to introduce or make

changes to mobile apps and services

HR

DB ERP

MF iSeries

CRM

Plan

Do

Check

Adjust

William Deming – American statistician

Major influencer of Japanese manufacturing and business

Famous for Plan-Do-Check-Act cycle (Deming Cycle)

PDCA cycles found in DevOps

6

William Edwards Deming

Plan

Do

Check

Adjust

Deming Cycle

Quality

Plan

Do

Check

Adjust

DevOps approach: Apply Lean principles to software innovation and delivery to create a continuous feedback loop with customers

Line-of-business

Customer

1

3

2

1. Get ideas into production fast2. Get people to use it3. Get feedback

Adopt DevOps approach to continuously manage changes, obtain feedback and , deliver changes to users

Eliminate any activity that is not

necessary for learning what

customers want

8

The Big Sources of Wasted Efforts: Find the Hidden Factory

Overhead, rework, over-production

Value-added production work

80%

20%

LeanTransformation

Type of Waste Create Feature Deliver Feature

Unnecessary Overhead

Communicating ideas/knowledge Communicating between development and operations

Unnecessary Re-work

Tasks assigned back to developers from testing

Tasks assigned back to developers from production rollbacks

Over-production Unnecessary functionality produced Unnecessary hardware, data center, personnel

60%

40%

The Lean Adoption Framework Uncovers the Hidden Factory Opportunities

Fat efforts to minimize Waiting Training

Reporting TraceabilityLate rework

Duplicate effortsMetrics collection

Regression testingChange propagationDocument generationMeetings/CheckpointsSystem administrationResource accountingHuman inspections

Streamline or automate

More Valuable efforts to improveScoping LearningFeedback

RefactoringDesigningTeamingCodingTesting

PlanningEngineeringEmpoweringPredictionDecidingSteering

Facilitate or smarten

Priorities of Indian Global System Integrators (200 responses)

Fat efforts to minimize

Late rework Waiting

Regression testing Duplicate efforts

Reporting

Document generation Training

Metrics collection Change propagation

Traceability Human inspections

Meetings/Checkpoints System administration Resource accounting

Streamline or automate

More Valuable efforts to improve

Scoping Designing Planning Testing

Reusing Deciding Steering Feedback Coding

Prediction Engineering

Learning Teaming

Refactoring

Facilitate or smarten

Enterprise capability for continuous software delivery that enables clients to seize market opportunities and reduce time to customer feedback

Acceleratesoftware deliveryfaster time to value

Acceleratesoftware deliveryfaster time to value

Balance speed, cost, quality and risk

increased capacity to innovate

Balance speed, cost, quality and risk

increased capacity to innovate

Reduce time to customer feedback

improved customer experience

Reduce time to customer feedback

improved customer experience

IBM DevOps

Continuous Release and Deployment

ContinuousCustomer

Feedback and Optimization

Monitor& Optimize

Develop& Test

Release& Deploy

Plan& Measure

Continuousinnovation,

feedback and improvements

ContinuousMonitoring

Collaborative Development

ContinuousBusiness Planning

ContinuousTesting

Systems of Engagement (SoE) Apps

RapidReleases AppStore

Monitorand Optimize

Releaseand Deploy

Developand Test

Inte

gra

tio

n T

est

12

Monitorand Optimize

Developand Test

Web Apps

FrequentReleases

ProductionEnvironment

Databases

Systems of Record (SoR) AppsFewer

Releases Databases

The need: Integrate systems of engagement with systems of record

By bringing together the culture, processes, and tools across the entire software delivery lifecycle – spanning mobile to mainframe platforms

A Lean View of DevOps

Idea/Feature/Bug Fix/Enhancement

Production

Development Build QA SIT UAT Prod

PMO

Requirements/Analyst

Developer

CustomersLine of Business

BuildEngineer

QA Team Integration Tester User/Tester Operations

Artifact Repository

Deployment Engineer

Release Management

Code Repository

Deploy

Get Feedback

Infrastructure as Code/Cloud Patterns

Feedback

Customer or Customer Surrogate

Metrics - Reporting/Dashboarding

Tasks

Artifacts

Demands on IT

April 7, 202314

Limited amount of features and changes being deployed to keep pace with rapidly evolving business requirements

NEW demand from business and technology

BAU, Maintenance, Break Fix, Lights On

Deliver systems of interaction for a consistent user experience across the

enterprise

Cloud

Customer Experience

Quality

Agility

Simplicity

Speed

DevOps

April 7, 202315

Collaboration

Communications

Standards

AGILE

•Bridges not Silos•S.M.A.R.T.•Risk Management

•Change and Release Management Policies and Procedures

•Architecture

•Continuous Integration•Continuous Delivery

• Transform Legacy processes• Simplify workflows• Automate!• provide agility to keep pace• Audit and Compliance• Pre-approved Routine Changes

• Service Catalog – Rapid Deployment and provisioning

• Increase frequency of changes• Systems of Interaction/Innovation vs.

Systems Record• smaller set of changes vs. large scale

periodic releases

• Do the things right vs. Do the right things• Tolerance of Risk for Development and

Production• Engagement oriented teams• Specific Measurable Attainable Realistic

Timely projects

Agenda

• What is DevOps?• Adopting DevOps

o Peopleo Processo Technology

• Implementing Delivery pipelines

17

• Common Business Objectives

• Vision Statement

• Common measures of Success

• Building a DevOps Culture

• There is no Silver Bullet

• Right People are needed

• Everyone is responsible for Delivery

Product Owner

Team Member

Team Lead

Team Member

Team Member

Senior Executives

UsersDomain Experts

Auditors

Gold Owner

Support Staff

External System Team

OperationsStaff

Adopting DevOps in the Enterprise: People/Culture

• Organizational Change

‘Shift Left’ – Operational Concerns‘Shift Left’ – Operational Concerns

Adopting DevOps in the Enterprise: People/Culture

• Do more upstream • App and infrastructure configuration management database • Dynamic Provisioning and Automated deployments • Automated testing – Test earlier and get feedback earlier• User testing early in QA and Staging. Not just Production.• Create test plans and test cases in Requirements Phase

Agenda

• What is DevOps?• Adopting DevOps

o Peopleo Processo Technology

• Implementing Delivery pipelines

Orchestration

End-to-End Software Delivery Process

Provision Hardware

•Virtual server configuration (S,M,L)

•Operating System version

Provision Server

Instrumentation

•Metering software

•Monitoring software

Provision Security

•Groups

•Service accounts

•User accounts

Provision Software

•Provision Application Enabling

•Software (AES)

•Configure AES software

•Archive and Catalog pattern image for reuse

Produce App. Build

•Build application version

•Create install package

•Create application configuration scripts

Deploy App. Build

•Install package

•Run application configuration scripts

Provision App. Data

•Acquire application data and archive

•Deploy seeded data

•Run data configuration scripts (if any)

I&O Processes

I&O Processes w/ BU IT configuration/settingsBU IT Processes

21

• DevOps as a Business Process• A Process to get Capabilities from Ideation to Value

• Apply Lean Thinking to Processes

Adopting DevOps in the Enterprise: Process

DevOps Solution: Adoption paths and key capabilities

Deployment

Provisioning

Release / Deploy

Develop /Test

Monitor / Optimize

Monitoring

Customer Feedback

Code

Test

Portfolio Management

Requirements

Plan /Measure

Change & Configuration Management

Dashboards/Analytics

Business Owner

PlatformOperations

ServiceDeveloper

Collaboration

TargetCustomer

Jazz, OSLC and Open Standards Platform

Where do you start: DevOps Adoption Roadmap

What am I trying to achieve?

• Think through business-level drivers for improvement • Align vision and pains to common business drivers across

silos• Look across silos, not just within the team, for improvements

Where am I currently?

• What do you measure and currently achieve• What don’t you measure, but should to improve• What practices are well scaled vs. incubating• Refine objectives to particular practice areas

Where are my bottlenecks

and priorities?

• Business-level drivers expose practice gaps across silos• Focusing outside of the bottleneck limits overall

improvement• It’s not just about tools, its about People, Practices,

Technology, and information

Ste

p 1

Ste

p 2

Ste

p 3

Current PracticeAssessment

Objective & Prioritized Capabilities

Business Goal Determination

Determine Activities Objective

How should I plan my practice

improvement?

Ste

p 4

• Identify improvements to skills, processes, and tools to achieve desired outcome

• Roadmap activity to define actionable plan• Target improvements which get the best bang for the buck

Roadmap

February 26, 2014

23

Understanding Maturity Levels

PracticedSome teams exercise activities associated with the practice, inconsistently. No enterprise standards defined. Automation may be in place but without consistent usage models.

ConsistentEnterprise standards for practice are defined. Some teams exercise activities associated with the practice and follow the standards. No core team or COE to assist with practice adoption. Automation, if used, follows enterprise standards.

Reliable Mechanisms exist to assist adoption and ensure that standards are being followed. Core team of mentors available to assist in adoption.

ScaledInstitutionalized adoption across the enterprise. COE is a matured and integral part of continuous improvement and enablement. Practices are mainstreamed across the enterprise. Feedback process in place to improve the standards.

Specific maturity levels are defined by how well an organization can perform practices. The levels look at consistency, standardization, usage models, defined practices, mentor team or center of excellence, automation, continuous improvement and organizational or technical change management.

February 26, 2014

24

Plan / Measure Development / Test Release / Deploy Monitor / Optimize

Scaled

Reliable

Repeatable

Practiced

Practice Based Maturity Model

Define release with business objectives

Measure to customer value

Define release with business objectives

Measure to customer value

Optimize applicationsUse enterprise issue resolution procedures

Optimize applicationsUse enterprise issue resolution procedures

Standardize and automate cross-enterprise

Automate patterns-based provision and deploy

Standardize and automate cross-enterprise

Automate patterns-based provision and deploy

Manage data and virtualize services for test

Deliver and integrate continuously

Manage data and virtualize services for test

Deliver and integrate continuously

Link objectives to releasesCentralize Requirements

ManagementMeasure to project metrics

Link objectives to releasesCentralize Requirements

ManagementMeasure to project metrics

Link lifecycle information Deliver and build with testCentralize management

and automated test

Link lifecycle information Deliver and build with testCentralize management

and automated test

Plan departmental releases and automate status

Automated deployment with standard topologies

Plan departmental releases and automate status

Automated deployment with standard topologies

Document objectives locallyManage department

resources

Document objectives locallyManage department

resources

Manage Lifecycle artifactsSchedule SCM integrations

and automated builds Test following construction

Manage Lifecycle artifactsSchedule SCM integrations

and automated builds Test following construction

Plan and manage releases Standardize deployments

Plan and manage releases Standardize deployments

Monitor resources consistently

Collaborate Dev/Ops informally

Monitor resources consistently

Collaborate Dev/Ops informally

Plan and source strategically

Dashboard portfolio measures

Plan and source strategically

Dashboard portfolio measures

Monitor using business and end user context

Centralize event notification and incident resolution

Monitor using business and end user context

Centralize event notification and incident resolution

Automate problem isolation and issue resolution

Optimize to customer KPIs continuously

Automate problem isolation and issue resolution

Optimize to customer KPIs continuously

Improve continuously with development intelligence

Test Continuously

Improve continuously with development intelligence

Test Continuously

Manage environments through automation

Provide self-service build, provision and deploy

Manage environments through automation

Provide self-service build, provision and deploy

Area of primary focus

February 26, 2014

25

We are

Here

Key Capabilities

1. Collaborative Development & Continuous

Integration

2. Continuous Business Planning

3. Continuous Release and Deploy

4. Continuous Testing

5. Continuous Feedback

Adopting DevOps in the Enterprise: Process

Region Configuration – To Be Environment

27

Agenda

• What is DevOps?• Adopting DevOps

o Peopleo Processo Technology

• Implementing Delivery pipelines

1. Collaborative Development and Continuous Integration

http://bit.ly/PRQ4a7

Mobile App Development

Teams

Enterprise Services

Development Teams

Adopting DevOps in the Enterprise: Process

Deployment Automation

http://bit.ly/1cr40TB

What one deploys may be anything from simple

configuration changes; to incremental code changes

towards a new feature; to Database schema changes; to

changes to the environment; to the whole stack.

Not just the application• Application Components• Middleware Configurations• Database Changes• Environment Configurations• Orchestrating the deployment

process

Investing in Release Automation … Introducing UrbanCode

Drive down cost

Remove manual effort and wasted resource time with push button deployment processes

Speed time to market

Simple, graphical process designer, with built-in actions to quickly create deployment automation

Reduce risk

Robust configuration management, coordinated release processes, audits, and traceability

Enabling clients to more rapidly deliver mobile, cloud, big data and traditional applications with high quality and low risk

IBM UrbanCode Deploy automates the deployment of applications, databases and configurations into development, test and production environments, helping to drive down cost, speed time to market with

reduced risk.

IBM UrbanCode Release is an intelligent collaboration release management solution that replaces error-prone manual spreadsheets

and streamlines release activities for application and infrastructure changes.

https://www.ibmdw.net/urbancode/

32

Continuous Integration Architecture – Current State

Jenkins 1.5 server

Artifactory Server 3.0

Drop-Off

PVCS Server

Source Repository(SVN Server)

1 Devloper Checkin in SVN

Server

2 Pull code From SVN and Display Result

3 Build source , create artifact(WAR , EAR ,

Jar , etc)

5 . Check in file in PVCS

7. Copy file to Drop Off

ACME Server

Portal Server

33

Provisioning Manager – Process Orchestration

Network

VIRTUALIZATION

CPU MEMORY I/O & NETWORK

AES

OS

Application

VIRTUAL SERVER

AES

OS

VIRTUAL SERVER

2b. Deploy Pattern

1b. Capture Pattern

Provisioning Manager

ConfigurationDatabase

(Regions, Tiers, Applications)

PatternRepository

Deploy Application

Application Content (Data)

Reporsitory

Application Build packages

Repository

Build Master Process

Process to clone DB Datafiles

Application Buildn,n+1,n-1revisoins

SeededApplication

Data

AD Security Provisioning

AESOS

AESOS

1a. Capture & Catalog Pattern Request

2a. Deploy Pattern Request

3a. Provision Security

4a. Deploy application build

5a. Deploy application data Source CodeLibrary Management

ApplicationDatabases

Agenda

• What is DevOps?• Adopting DevOps

o Peopleo Processo Technology

• Implementing Delivery pipelines

Implementing a DevOps toolchain

SCM

Build / CI Server

Unit testingTest

AutomationTest Stubbing

Delivery Pipeline

Environment Configuration

Automated Monitoring

Asset Repository

April 7, 202336

Continuous Delivery

• Create a repeatable, reliable process for releasing software

• Automate almost everything• Keep everything in version control• If it hurts, do it more frequently, and

bring the pain forward• Instrument and Audit Everything• Test Everything• Everybody is responsible for the delivery

process• Continuous improvement• Dynamic Infrastructure supports

continuous delivery

36

Source: “Continuous Delivery: Reliable Software Releases Through Build, Test and Deployment Automation”

Application Artifacts

ApplicationCode

Instrumentationand

Configuration

Runtime Environment

DefinitionVersion

Automate

Test

Track and Plan

Instrument and Audit

DashboardEnvironment

ApplicationConfiguration

Continuous Delivery Pipeline

PVCSSVN

Artifactory

SplunkJenkins MavenAnt

Puppetchef

HP ALMPolicy TesterSelenium

Smart Cloud Control DeskPMGServiceNow

Microsoft Team Foundation ServerSerena Dimensions

Improve Change and Release Management4-8-12-16-20 hours…

Testing is 1/3 of our release time

Lack of automation

Too many hand offs

38

More environments Test earlier Automate Process Optimization Monitoring Continuous Service Improvement

Testers

Developers

MiddleWare AdminsDBA

Ops

Users

Release Manager

App Managers

Are you done yet?! Why is it taking so long?!

Where are we with the release?!When can I test?!

Let’s fix it in production!

New “DevOps” team

39

Application

EngineeringOperations & Release

Management

Infrastructure EngineeringMiddleware

Web, Security, Application Server

Architecture

Automation and Tooling

DEV OPS

BU Aligned

DevOps Tiger Teams

40

QA, TEST STRATEGY & PLANNING

DEVELOPMENT BUILD

PROCESS

RELEASE & CHANGE

GOVERNANCE & BUSINESS OUTCOMES

CULTURE

ELASTIC INFRASTRUCTUR

E

• Automated monitoring & dash-boarding inclusive of business drivers & events

• Implementation of application enabling software

• Process for determining Capacity Planning and Capacity/Event Monitoring

• Future scalability and how to measure

• Orchestration process • Virtualization strategy

across applications

• Automation Strategy and Scope • Data Management• Baseline data for each application• After image baseline for each application• Test case transaction for each application.

• Process controls & measures

• Define Phase Gate/SDLC intersection

• Changes/Implications to costing models based on dynamic virtualization

• Knowledge Management

• Define how to achieve the management and relationship between source, manifest, and executables

Questions?

We Value Your Feedback

Don’t forget to submit your Impact session and speaker feedback! Your feedback is very important to us – we use it to continually improve the conference.

Use the Conference Mobile App or the online Agenda Builder to quickly submit your survey

• Navigate to “Surveys” to see a view of surveys for sessions you’ve attended

42

Thank You

Legal Disclaimer

• © IBM Corporation and The Prudential Insurance Company of America 2014. AllRights Reserved

• The information contained in this publication is provided for informational purposes only. While efforts were made to verify the completeness and accuracy of the information contained in this publication, it is provided AS IS without warranty of any kind, express or implied. In addition, this information is based on IBM ’s current product plans and strategy, which are subject to change by IBM without notice. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this publication or any other materials. Nothing contained in this publication is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software.

• References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in this presentation may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other results.

• If the text contains performance statistics or references to benchmarks, insert the following language; otherwise delete:Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.

• If the text includes any customer examples, please confirm we have prior written approval from such customer and insert the following language; otherwise delete:All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer.

• Please review text for proper trademark attribution of IBM products. At first use, each product name must be the full name and include appropriate trademark symbols (e.g., IBM Lotus® Sametime® Unyte™). Subsequent references can drop “IBM” but should include the proper branding (e.g., Lotus Sametime Gateway, or WebSphere Application Server). Please refer to http://www.ibm.com/legal/copytrade.shtml for guidance on which trademarks require the ® or ™ symbol. Do not use abbreviations for IBM product names in your presentation. All product names must be used as adjectives rather than nouns. Please list all of the trademarks that you use in your presentation as follows; delete any not included in your presentation. IBM, the IBM logo, Lotus, Lotus Notes, Notes, Domino, Quickr, Sametime, WebSphere, UC2, PartnerWorld and Lotusphere are trademarks of International Business Machines Corporation in the United States, other countries, or both. Unyte is a trademark of WebDialogs, Inc., in the United States, other countries, or both.

• If you reference Adobe® in the text, please mark the first use and include the following; otherwise delete:Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries.

• If you reference Java™ in the text, please mark the first use and include the following; otherwise delete:Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.

• If you reference Microsoft® and/or Windows® in the text, please mark the first use and include the following, as applicable; otherwise delete:Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both.

• If you reference Intel® and/or any of the following Intel products in the text, please mark the first use and include those that you use as follows; otherwise delete:Intel, Intel Centrino, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries.

• If you reference UNIX® in the text, please mark the first use and include the following; otherwise delete:UNIX is a registered trademark of The Open Group in the United States and other countries.

• If you reference Linux® in your presentation, please mark the first use and include the following; otherwise delete:Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others.

• If the text/graphics include screenshots, no actual IBM employee names may be used (even your own), if your screenshots include fictitious company names (e.g., Renovations, Zeta Bank, Acme) please update and insert the following; otherwise delete: All references to [insert fictitious company name] refer to a fictitious company and are used for illustration purposes only.