softtest ireland - digital engineering evolution: devops

@jonathon_wright

#Softtest2015 www.hitachiconsulting.com/digitalenterprise

BetterTestOps.net

23rd September 2015

[email protected]

TestOps: Done in 60 secondsJonathon Wright, Director, Testing, Quality, Assurance

mailto:[email protected]

2

@jonathon_wright


Hitachi – Digital EvolutionSociety Changes, Hitachi Transforms It.

$93.4

Billion947Companies

320,000+Employees

$3.4

Billion

Hitachi ranks 78th on the 2014 FORTUNE Global 500®.

© Hitachi Europe Ltd. 2014. All rights reserved.

3

@jonathon_wright


Welcome to real world

The red pill and its opposite, the blue pill, are popular culture symbols representing the choice between embracing the sometimes painful truth of reality (red pill) and the blissful ignorance of illusion (blue pill).

@jonathon_wright

#Softtest2015

Last night: DevOps in the Pub

Culture

User Interface (UI)

Culture

Artificial Intelligence

Culture

Collaboration (email)

Culture

Methodologies (Agile)

@jonathon_wright

#Softtest2015

Last night: DevOps in the Pub

Behaviors

User Experience (UX)

Behaviors

Researchers

Behaviors

Sponsors (Mentors)

Behaviors

Methodologies (Tri-Modal)

6

@jonathon_wright


Welcome to the digital world

@jonathon_wright

#Softtest2015

What is Digital?

@jonathon_wright

#Softtest2015

8

@jonathon_wright


4

@jonathon_wright

#Softtest2015

What is Digital ?

The creation of new business designs that connect people, business and things (physical objects that are active players and contribute to business value) to drive revenue and efficiency.

9

@jonathon_wright


@jonathon_wright

#Softtest2015

• Reliable, compliant, secure

• Think price / performance

• Plan and approvals driven

• Long life-cycles

CoreIT• Agile and fluid• Innovation, brand,

profit measures• Minimum viable

experience / product • Think continuous• Think days, weeks

FluidIT

Digital Evolution

5

What is Tri-Modal IT?

1 2

3

10

@jonathon_wright


Digital is not just about “disrupters”

@jonathon_wright

#Softtest2015

6

11

@jonathon_wright

#Softtest2015 www.hitachiconsulting.com/digitalenterprise* Hitachi Consulting, 2015, ‘Becoming a Digital Enterprise’, www.hitachiconsulting.com/digitalenterprise

Digital Evolution

Disrupters

Becoming a Digital Enterprise

http://www.hitachiconsulting.com/digitalenterprise

What is Digital Engineering?

@jonathon_wright

#Softtest2015

13

@jonathon_wright


What is Digital Engineering?

Intel© XDK IoT in 2015

Does ‘Testing as a Function’ still have a future?

Smart Cities – Copenhagen (Big Data)

Hitachi Smart CitiesAward winning first-of-its-kind Big Data platform for Copenhagen on which data collection, integration and sharing is centralised for the entire city

Trains as a Service (IoT x 30,000)

Digital Engineering Evolution

@jonathon_wright

#Softtest2015

17

@jonathon_wright


What is the Digital Engineering Evolution?

Journey to Value Stream (Digitization Blueprints) mapping?

Predictive Improvement

Predictive Learning

Predictive Intelligence

Predictive Insight

Predictive Assessment

Predictive Quality

Predictive Innovation

Predictive Testing

Predictive Delivery

Predictive Support

Predictive Maintenance

Predictive Operations

DIGITAL AT THE HEART

Processes TechnologyBehaviours

DIGITAL PROCESSES LEANDIGITAL TECHNOLOGY DEVOPS

18

@jonathon_wright



• Behaviours - X as a Service (Tri-Modal Delivery)

• Process - Lean (Functional Testing)

• Technology - DevOps (Containerization)

Journey to Digital Engineering (Automation Enablement)

@jonathon_wright

#Softtest2015

DIGITAL BEHAVIOURS TESTING AS A SERVICE

Behaviours Processes Technology

19

@jonathon_wright


‘Testing as a Service’

A New Model for Testing (2010)

2011

2015

*Jonathon Wright, 2011, ‘Testing as a Service’ presented at BCS SIGIST, London, 5th December 2011

Testing as a Service - Models

20

@jonathon_wright

#Softtest2015 www.hitachiconsulting.com/digitalenterprise* Hitachi Consulting, 2015, Becoming a Digital Enterprise, www.hitachiconsulting.com/digitalenterprise

‘As a Service’ – Lean - Continuous Delivery

Tri-Modal - Digital Business Delivery Models

Becoming a Digital Enterprise

Business Solution Delivery Quality (cheap, good, fast)

Business Problem, Idea or Challenge (cost, benefit, time to market)

Modal 3 – Lean - Innovation Projects

• Experimenting / testing / piloting / experiencing to

define new ideas

• Rapid Experimentation & Prototyping

• Continuous Delivery / Deployment

Modal 2 - Fluid - Differentiation Projects

• Needs that can be defined now

• Agile Delivery / Short Release Cycles

Modal 1 – Core - Evolution of Core Systems

• Well defined needs essential for

organisational operation

• Iterative / Waterfall60%

30%

10%

21

@jonathon_wright

#Softtest2015 www.hitachiconsulting.com/digitalenterprise*Jonathon Wright, 2014, ‘Testing as Service’ Models’ discussion paper published, 26th October 2014

‘As a Service’ – Lean UX – Continuous Learning

Tri-Modal – Rapid Evolution (Fail Fast Experiments)

Testing as a Service – Models



Ideas & hypotheses

Experiment team assess the initial proposal and brainstorm

ideas on how to realise the vision. Each idea is defined by

hypotheses which can then be tested.

(Experimentation is prioritised based on the identification of a

minimum viable experience)

Design

An experiment is defined that will test one or more ideas

based on their hypotheses

Build

Assets required to perform the experiment are created

Measure

The experiment is performed against the criteria defined in

the hypothesis

Learn

The outcomes of the experiment are assessed and the

insights gained are used to iterate on the initial ideas and

hypotheses and improve the product/service. If required, a

decision is made on whether the proposal needs reworking

or the initial vision needs to be changed/rejected.

22

@jonathon_wright


‘As a Service’ – Lean IT – Teleportation (Shift X)

Shift up ‘minimum viable product’ to prove value to the ‘business’

Shift down ‘minimum viable experience’ for

target ‘customer’

Smart Predictive

Lean

Shift right from ‘ideas & hypotheses’ to ‘fail-fast

experiments’

Shift left from ‘support’ mapping to ‘stories&/or requirements’

*Huw Price & Jonathon Wright, 2015, ‘Testing Imperatives in the World of Agile Development & Continuous Delivery’, 28th July 2015



LeanDevOps

Digital Engineering – Testing Imperatives

Tri-Modal – FluidIT (Differentiation Projects)

23

@jonathon_wright




– Test Automation as a Service (Model-Based Design)




@jonathon_wright

#Softtest2015

DIGITAL BEHAVIOURS TEST AUTOMATION AS A SERVICE

* Dorothy Graham, ’Experiences of Test Automation’, http://www.dorothygraham.co.uk/automationExperiences/, ISBN-10: 0321754069, 9th January 2012

First Day Automation

Behaviours Processes Technology

http://www.dorothygraham.co.uk/automationExperiences/

24

@jonathon_wright


Automation – Memes

What if I told you Automation is not just testing?

25

@jonathon_wright


‘Test Automation as a Service’ (TAaaS.net)

A New Model for Testing (2010)

*Jonathon Wright, 2012, STARWest Track, ”Test Automation as a Service”, http://www.TAaaS.net, 2nd October 2012

Test Automation as a Service

2011

2015

http://www.taaas.net/

26

@jonathon_wright


Introduction to Model Based Design (MBD)

‘Test Automation as a Service’ - Models

Blueprint(s) Pattern(s) Recipe(s)

www.eurostarconferences.com* Test Automation Patterns, testautomationpatterns.wikispaces.com & testing-store.com

@jonathon_wright

#Softtest2015

http://testautomationpatterns.wikispaces.com/

http://www.testing-store.com/

27

@jonathon_wright


Model Based Design (MBD) – Blueprint(s)

‘As a Service’ - Models

We used drawing (visual 2D art)

to represent the foundation

of basic blueprints

GEN 1

GEN 1k

“Truly wonderful, the mind of a child is.”

This demonstrates that models created from

a single entity (transcendence) only represent

ontology of the domain of discourse and

exclude the quantitative intelligence of others.

“No matter their experience, masters can still

learn from their apprentices.“

“That’s No Moon!”

In Digital Enterprise

= Business on a Page

Enterprise Collaboration is the Evolution

(not revolution) of detailed expert Blueprints

28

@jonathon_wright


Model Based Design (MBD) – Pattern(s)


Death Star Interior / Inner Chasm & Corridors

Collaboration of individual applies unique

patterns that will refine, predict and

inform the model through exploration

(enquiring and challenging) each other

Quantum Multifractality

Applying constructive patterns

from two or more point sources

(interference) to render multi-

dimensional (3D) models

GEN 2k

GEN 1080Each iteration can increase the

representation of the model by

applying and interpreting data

29

@jonathon_wright


Model Based Design (MBD) – Pattern(s) – Real World Example

Test Infrastructure as a Service (TIaaS.net) – TestOps 1.0 (VM Role Dispenser)“The best aspects

of these proven

test approaches

demonstrate how

they have evolved

over the past

decade, and this

echoes some of

the changes

towards more lean

and agile business

methodologies.

They are in a

constant state of

evolution – just as

the underpinning

technology evolves

over time.”

“Over the last five

years we have been

developing a azure

cloud based solution to

provide true

Test Automaton as a

Service using the

hybrid approach

specifically designed

with the Hybrid Cloud

(Private/Public) in

mind to utilise the

instant scalability,

flexibility and

availability of the

Cloud.”

* Jonathon Wright, 2010, STARWest Track, ”Test Automation as a Service”, http://www.TAaaS.net, 2nd October 2012


30

@jonathon_wright


Model Based Design (MBD) – Recipe(s)


Early recipes date back to

1600BC, IFTTT recipes

(conditional statements) =

DestroyDeathStar1.0I. All wings report in

II. Lock S-foils in attack position

III. Switch deflectors ON

IV. Keep visual scanning

V. Start attack run

VI. Move into position

VII. Stay in attack formation

VIII. Stay on target

IX. Use the FORCE

GEN 4k

• Channels are the basic

building blocks

• Triggers are the "this" part

of a recipe

• Actions are the “that” part

of the recipe

• Recipes are the

“equations” made from

Triggers and Actions

• Ingredients are the basic

data model, node, quantum

materials (dot) particles

or even atom(s)

Enterprise Gamification

represents the success criteria

e.g.

“Great shot kid that

was one in a million.”

31

@jonathon_wright


Model Based Design (MBD) – Recipe(s) – Real World Example

Test Automation as a Service (TAaaS.net) – TestOps 1.0 (>5 Years)

Select the Business Process Tests Generated Recipe

“The best aspects

of these proven

test approaches

demonstrate how

they have evolved

over the past

decade, and this

echoes some of

the changes

towards more lean

and agile business

methodologies.

They are in a

constant state of

evolution – just as

the underpinning

technology evolves

over time.”

“Based on

business process

modelling to

create business

level keywords

written in natural

language

validated against

content sensitive

validation to

create reusable,

jigsaw pieces

representing the

Solution Under

Test (SUT).”

Test Automation as a Service

* Jonathon Wright, 2010, STARWest Track, ”Test Automation as a Service”, http://www.TAaaS.net, 2nd October 2012

@jonathon_wright

#Softtest2015


32

@jonathon_wright


Model Based Design (MBD) – Real World

Advanced Modelling Simulation Technologies

Watershed management

Pipe-network Management

Prediction of water pollution

Pipe-network analysis


Models can be analysed, visualized, and optimized

Models can provide assurance through, analysis, simulation and prediction

* http://www.computerweekly.com/news/2240233919/Societys-values-moving-from-Mono-to-Koto-says-Hitachi

http://www.computerweekly.com/news/2240233919/Societys-values-moving-from-Mono-to-Koto-says-Hitachi

33

@jonathon_wright







@jonathon_wright

#Softtest2015

DIGITAL PROCESSES LEAN

Behaviours Processes TechnologyProcesses

34

@jonathon_wright


Lean – Memes

Lean practices don’t make your core values change

35

@jonathon_wright


Lean - Pipeline

Devland, Testland and Prodland

@jonathon_wright

#Softtest2015

36

@jonathon_wright


Lean – Pipeline Hub

No release is a island?

@jonathon_wright

#Softtest2015

@jonathon_wright

#Softtest2015

37

@jonathon_wright


Lean = Open

Open (Digital) Innovation results in Open (Digital) Businesses

Open Digital Blueprints

Lean

Enterprise

Business Practices

PlatformsSmart

“Sharing intellectual property such as software code or research

rules, adhering to open standards rather than creating your own,

and giving customers the freedom to easily exit your platform”

*Jonathan Rosenberg & Eric Schmidt, 2014, ‘How Google Works’, ISBN: 9781455582341, 23rd September 2014

Open Digital Enterprises

38

@jonathon_wright


Open Innovation – Active Automation (Javelin)

SDLCi - Grid-Tools connected intelligence to PerfectoMobile

*Llyr Wyn Jones, ‘End-to-end vs. interface testing: the mathematics’, 31st March 2015

@jonathon_wright

#Softtest2015

39

@jonathon_wright


Open Innovation – Connected Intelligence (ALI)

40

@jonathon_wright




@jonathon_wright

#Softtest2015




Behaviours Processes TechnologyTechnology

41

@jonathon_wright


DevOps – Memes

Deliver business value faster with less risk?

42

@jonathon_wright


DevOps in the Wild

All media links:Domino’s Pizza orders Hitachi Consulting to advise on DevOps pushHitachi to boost Domino’s Pizza delivery with DevOps strategyDomino’s signs up to Hitachi Consulting’s DevOps expertiseHitachi Consulting and Domino’s Pizza collaborate for a slice of DevOps actionDomino’s continues to deliver with a DevOps strategy

Hitachi Consulting director of testing quality assurance Jonathon Wright said: "With DevOps, there is a strong focus on

continuous testing and continuous delivery. This means there is more emphasis on testing much earlier in the solution

delivery lifecycle. "Domino's Pizza has embraced continuous integration and testing to help the software delivery team

rapidly build, test, and deliver secure, high-quality applications."

Domino’s Pizza & Hitachi Consulting increase the agility as part of DevOps Strategy

http://www.computerweekly.com/news/4500251578/Dominos-Pizza-orders-Hitachi-Consulting-to-advise-on-DevOps-push

http://www.cbronline.com/news/enterprise-it/it-services/hitachi-to-boost-dominos-pizza-delivery-with-devops-strategy-130815-4646331

http://www.itpro.co.uk/strategy/25134/dominos-signs-up-to-hitachi-consultings-devops-expertise

http://www.economicvoice.com/hitachi-consulting-and-dominos-pizza-collaborate-for-a-slice-of-devops-action/

http://www.essentialretail.com/news/marketing-ecomm/article/55cc62eb8defe-dominos-continues-to-deliver-with-a-devops-strategy

43

@jonathon_wright


What is DevOps?

Digital Engineering?

Pipeline Latency

• Data Capturing

• Test Execution

• Reuse and Standardisation

• Repeatable Tasks

• Provisioning

• Release Management

• Integrate for better handshakes

• Code early requirements

• Test and data virtualization

• Simulate environment

• Reduce dependencies

• Reduce waiting time

Visibility

• User experience

• Measure everything

• Scope of work

• Version everything

• Pipeline visibility

• Shift left monitoring

“DevOps is a framework that allows development, testing, and operations to meet customer

needs. This integrated team approach facilitates communication, collaboration, and integration

to manage today’s rapidly changing business demands.”

44

@jonathon_wright


DevOps = Containerization

Container Revolution

@jonathon_wright

#Softtest2015 * Tony Bradley, “Container Revolution”, http://techbeacon.com/docker-just-first-killer-app-container-revolution, 22nd June 2015

http://techbeacon.com/docker-just-first-killer-app-container-revolution

45

@jonathon_wright


Containerization – Building Bridges in the Cloud

Infrastructure as Code - “If you build it, they will test” (IYBITWT)

*Jonathon Wright, 2014, ‘Ha(API)y testing in the hybrid-Cloud & beyond’ presented at Gartner ADM, London, May 19th 2014

Test Infrastructure as a Service (TIaaS) Model

NFV

46

@jonathon_wright


Containerization + Microservices

@jonathon_wright

#Softtest2015

Heterogeneous Complex Ecosystems

47

@jonathon_wright


Containerization – Where do you want to go today?*

Platform as Code - Desired State Configuration (DSC)

*Microsoft, 1994, http://www.nytimes.com/1994/11/11/business/media-business-advertising-microsoft-takes-user-friendly-approach-selling-its.html

Test Platform as a Service (TPaaS) Model

Release Management (InRelease) 2002 -2015 Q3

Release Management (Microsoft) 2015 Q4 (private preview)

http://www.nytimes.com/1994/11/11/business/media-business-advertising-microsoft-takes-user-friendly-approach-selling-its.html

48

@jonathon_wright


Containerization – Run everywhere?

Hybrid DevOps – Build Bridges in the (Public / Private) Clouds

@jonathon_wright

#Softtest2015

49

@jonathon_wright


Containerization – Deploy everywhere?

Hybrid DevOps – Virtual / Stateless / Small (Data) / Deployable?

@jonathon_wright

#Softtest2015

DIGITAL TECHNOLOGY

50

@jonathon_wright


Containerization – Data everywhere?

Big Data and Big (Test) Data

51

@jonathon_wright


Containerization – Data everywhere?

Big Data and Big (Test) Data

Tailor-made

UX for each

persona

Store and

model for

research

Capture data

about AUT

behavior

53

@jonathon_wright


Containerization – Test everywhere?

Black box is now Black containers?

54

@jonathon_wright


TestOps – Building Containers in the Cloud

Test First Delivery (TFD) First Day Delivery (FDDi)

TDD

* Jonathon Wright, 2014, Gartner, http://www.slideshare.net/mobile/Jonathon_Wright/addi-ha-apiytestinghybridcloud

Domain Driven Design (DDD) Domain Driven Delivery (DDDi)

Transcendence Driven Delivery (TDDi)

Test First Delivery (TFD)

http://www.slideshare.net/mobile/Jonathon_Wright/addi-ha-apiytestinghybridcloud

55

@jonathon_wright


TestOps – First Day Delivery (FDDi)

Model Driven Delivery (MDDi) integration

*The Advanced Research Projects Agency Network (ARPA) Network, 1969, 2 & 4 Nodes the precursor to the Internet

Business Process Modelling

Blueprint

Business Process Scenarios

Patterns

Business Process Components

RecipesNode Maps Node Learning Node Strengthening

• Model the proposed solution • Learning (Turing machine)

adapting to simulate logic

algorithms

• Validate all possible states

Smart Learning Machines

56

@jonathon_wright


TestOps – Domain Driven Delivery (DDDi)

Model Driven Delivery (MDDi) integration

Business Process Modelling

Blueprint

Business Process Scenarios

Patterns

Business Process Components

RecipesNode Discovery Node ValidationNode Systems

• Model the current solution • Validate all possible states• Apply:• Systems Dynamics

(epistemic & systemic

entropy)

• Systems Thinking

• Learn:• Ontology – formal naming,

definition of node types,

properties and

interrelationships

• Domain – universal set of

nodes (domain of discourse)

• Context – nouns/verb

discovery to enable context

sensitive validation.

* Huw Price, 2015, Grid-Tools, http://www.agile-designer.com/resources/model-based-testing-using-agile-designer/

@jonathon_wright

#Softtest2015

https://www.grid-tools.com/solutions/agile-designer/

57

@jonathon_wright


TestOps – Lean Functional Testing (LeanFT)

SDLCi – HP solution to continuous testing in a DevOps world

*HP Enterprise LeanFT Press Release, http://www8.hp.com/uk/en/hp-news/press-release.html?id=1998066, 2nd June 2015

@jonathon_wright

#Softtest2015

http://www8.hp.com/uk/en/hp-news/press-release.html?id=1998066

58

@jonathon_wright


QualityGate™ – Done in 60 seconds

Dynamic test asset cubes based on of fuzzy logic, meta mathematics & descriptive languages

QualityGate™ Map

Confidence Level 74%

40-50 secs – Predictive Testing

50-60 secs – Predictive Deploy

88% Quality Index Score (Assurance)

1-5 secs – Smart Build

5-10 secs – Smart Containers 30-40 secs – Smart Testing

10-20 secs – Smart Analysis (TAaaS.net)

* Vinnie Jones, 2000, Gone in 60 seconds, https://en.wikipedia.org/wiki/Gone_in_60_Seconds_(2000_film)

https://en.wikipedia.org/wiki/Gone_in_60_Seconds_(2000_film)

59

@jonathon_wright


TestOps – Release Management

Done in 60 minutes

60

@jonathon_wright


TestOps - Release Management (Private Beta)

Hybrid DevOps – Azure / Chef / Docker / Puppet / PowerShell v5 (DCS)

61

@jonathon_wright


TestOps – Analogy

Crashed in 60 milliseconds

a) DevOps can provide the capability to release faster. However, you can only go as fast as infrastructure (roads) and architecture (lanes) can support.

“The darkside, and the light”

b) Automation can drastically accelerate the release process. However, without delivery management (rules & regulations) then:

“If you automate chaos, all you get in faster chaos”

c) Technology can provide agility to scale out to the cloud. However, without release management (traffic) promoting containers between stages in complex ecosystems such as micro-services is limiting (restrictions).

“Evolution, over revolution"

b) Dorothy Graham, The CAST Report, Unicom Seminars, 1991 a/c) Jonathon Wright, TestOps: The Force Awakens, EuroSTAR, 2015

62

@jonathon_wright


DevOps - Summary

• Automate release times and ramp up velocity

Infrastructure as code (topology-base model(s) – Azure/HP Codar)

• Automate build to the hybrid cloud (containerization)

Platform as code (automation (build) model(s) – Docker/Chef/Puppet/PS DCS)

• Automate deploy to reduced failure rates, lead times between fixes and rates of recovery

Delivery Management (deployment model(s))

• Monitor releases in the wild

Operational Insight (release model(s))

• Measure releases in the wild

Quality index scores (reporting model(s))

• Delivery Insight (Sense, Analyze and Adapt)

Connected lifecycle intelligence (synchronisation model(s))

Unlock true delivery innovation

*Jonathon Wright, 2014, ‘Ha(API)y testing in the hybrid-Cloud & beyond’ presented at Gartner ADM, London, May 19th 2014

Test Infrastructure as a Service (TIaaS) Model

63

@jonathon_wright


Hitachi - ‘As a Service’ – Models

BusinessAssurance

BusinessQuality

BusinessTesting

Business Assurance provides the visibility and

transparency throughout the solution delivery lifecycle• Digital Enterprise

• Business on a Page

• Business Delivery Management

• Business Intelligence (Actionable Business Insight)

• Business Innovation (Enterprise Gamification)

Business Quality allows enforceable quality across the

entire solution delivery lifecycle owned by everyone• Business Lifecycle Management

• Business Solution Discovery / Delivery

• Business Enterprise Collaboration

• Business Acceptance Testing

Business Testing

• Testing as a Service - Models

• Solution Delivery Lifecycle integration (SLDCi)

• Test First Delivery / First Day Testing

• Continious, Integration, Build & Delivery

• Shift Everywhere (UP/DOWN/LEFT/RIGHT)

• All Channel Customer Experience

64

@jonathon_wright


Hitachi ConsultingHitachi’s Global Management Consulting & IT Services Business.

$740

MillionFY13 Revenue

6,500experienced &

dedicated

professionals

Better Approach. Better Results. Better Experience.

$10 Billion

results
