making the move to behavior driven...
TRANSCRIPT
| Create Better Software© 2018 QASymphony – Confidential & Proprietary
Making the Move to Behavior Driven Development RYAN YACKEL | DIRECTOR OF PRODUCT
© 2018 QASymphony – Confidential & Proprietary
Who’s this Guy?
Ryan YackelDirector of Product
• Testing Experience: CGI Group, Macy’s IT
• BDD Experience: PM for qTest Scenario, Cucumber Training
• Hobbies: Love Coffee & Comics
© 2018 QASymphony – Confidential & Proprietary
Company ProfileOur Vision Transform the Way Global Enterprises Create Quality Software
• Corporate Headquarters: Atlanta, Georgia
• Global Offices: London (Sales), Vietnam (Development)
• Customer Growth: June 2014: 150 customers | Jan 2018: 600+ customers
• Funding: Raised $50m equity
• Key Customers: USAA, Nordstrom, Office Depot, Verizon, Amazon, Samsung, Cisco, Visa, Lowe’s, Sony
• Technology Partners: Atlassian, CA, TestPlant , Neotys, GenRocket, Katalon, APIFortress
• Channel Partners: cPrime, Clearvision, Go2Group, Rightstar, Capgemini, MouSoft, Aservo, Column Technologies
© 2018 QASymphony – Confidential & Proprietary
The #1 Enterprise Testing Platform for Agile Teams
Enterprise BDD for Jira
Jira add-on that integrates with
qTest Pulse to scale BDD, and
ATDD in their DevOps pipelines. BI & Analytics Hub
Advanced reporting and BI
hub for all testing activities
across the enterprise.
Exploratory Testing
Intelligently capture
exploratory test sessions to
create robust documentation
and automated test scripts.
Centralized Test Automation
Centrally manage, schedule,
and report on test automation
executions across complex
environments.
Agile and DevOps Workflows
Event driven rules engine
designed for workflow
automation for qTest products.
Agile Test Management
Organize and manage all test
activities in one central location.
Real-time integration with Jira
and other Agile ALMs.
© 2018 QASymphony – Confidential & Proprietary
Agenda
Why BDD? History and Industry
Principles of BDD
Demo Workflow
Assessment of BDD
Q&A
© 2018 QASymphony – Confidential & Proprietary
The question is not “Do you have test
automation and development?”…
…rather it is “How are you approaching
your test automation with development?”
© 2018 QASymphony – Confidential & Proprietary
Enter Test Driven Development
Source: http://hanwax.github.io/assets/tdd_flow.png
Test Driven Development brings several major benefits to organization, most notably:
1. Move Testing Up Front – prevents having to rush testing at the end of the cycle
2. Bake in Automation from Day 1 – protects against getting behind with test and automation coverage
3. Build More Testable Software – requires developers to think about testability, and create more robust software
4. Push to Customers When Ready – allows you to push software to customers just in time, as it is developed
© 2018 QASymphony – Confidential & Proprietary
What’s the Difference?
Traditional Approach
Test-first Approach
© 2018 QASymphony – Confidential & Proprietary
What’s the Catch?
Test-First methodologies were coined “Test Driven Development”. Less technically focused versions called Acceptance Test Driven Development (ATDD) and Behavior
Driven Development (BDD) also emerged:
Test Driven Development
(TDD)
Behavior Driven Development BDD
Acceptance Test Driven
Development (ATDD)
Unit Test Driven Development
(“Technical TDD”)
© 2018 QASymphony – Confidential & Proprietary
Oh, no…
Freaking OutAcceptance
Tests
Integration
Tests
Unit TestsWe for sure have this covered.
Super Confident
I think we got this covered
Semi-Confident
QA Risk When Scaling Automation
RISK LEVEL
INSERT PHOTO ON TOP
© 2018 QASymphony – Confidential & Proprietary
Festering Problems
Code
Code
Code
Automation Scripts
Code
Code
Cod
e
Cod
e
What Are
We Testing?
Automation Confusion
Automation frameworks without feature visibility create
business confusion around application coverage.
• Automation becomes pure code, without clear understanding
of test coverage
• QA doesn't know if test coverage is sufficient
Communication Breakdowns
Features and requirements change during Agile and DevOps
sprints. If automation is disconnected from these features, we
might be testing a changed feature set without know it.
• Changes are discovered after automation is completed
• Automation is never updated to reflect adequate coverage
© 2018 QASymphony – Confidential & Proprietary
BDD is Growing in DevOps Environments
16%
BDD ADOPTION
39%
TDD ADOPTION
The Forrester Wave TM: Modern Application Functional Test Automation Tools, Q4 2016
© 2018 QASymphony – Confidential & Proprietary
The Move to BDD is Clear
1. Increased collaboration
2. Easier transition to test automation
3. Structured organization and guidance
4. Better alignment between businesses and customers
5. Alignment between Product / Dev / Test
© 2018 QASymphony – Confidential & Proprietary
Behavior
+
Driven
+
Development
“We collaborate, we record that
collaboration in some form of
specification and then we automate
that specification to drive out the
implementation.”
BDD in a Nutshell
Source: Intro to TDD and BDD - Seb Rose [ACCU 2017]
© 2018 QASymphony – Confidential & Proprietary
BDD to the Rescue
Collaborate
Owner, Dev, Test, design acceptance criteria
Record
Document common syntax for all
Automate
Execute test scenarios that validate behavior
Full Testability
Behavior Driven Development
© 2018 QASymphony – Confidential & Proprietary
Completing the Story
Almost Always,
Stories Have LITTLE Information
© 2018 QASymphony – Confidential & Proprietary
Collaborate – Through Examples
1. Write the story on a yellow card
2. Write each rule/acceptance criteria on a blue card
3. Write examples on green cards under each rule
4. Write any questions on red cards
5. Keep going until everyone understands the story
6. Stop after 30 mins
source: Matt Wynne - https://cucumber.io/blog/2015/12/08/example-mapping-introduction
© 2018 QASymphony – Confidential & Proprietary
Record - Specification by Example
Dev
Product
OwnerTester
Living
Documentation
Scenario: Buy Last Coffee
Given There are 1 Coffees left in the Machine
And I have deposited $5
And I previously have $100 in the bank
When I press the coffee button
Then I should be served a coffee
Feature: Serve Coffee
Coffee should not be served until paid for
Coffee should not be served until the button
has been pressed
If there is no coffee left, then the money
should be refunded
Three Amigos
in BDD
© 2018 QASymphony – Confidential & Proprietary
Automate - Specification by Example
PO / Customers
Developers
Testers
Briefing Estimate Prioritise
PrepareStoryTitleDescriptionPriorityValue PropositionMeasuresEstimateSketchNotes
Specification Workshop
ReadyA/CNFRsKey ExamplesDesign NotesWireframesAssumptions
Refine
Monitor Evaluate
ReviewValue returnedMetricsBugsNew StoriesReview
BuildExplore
Review & Demo
Done
ScenarioDefinition
Scenario Automation
Feature Implementation
Automate
Source: MagenTys, 2017
© 2018 QASymphony – Confidential & Proprietary
Traditional Testing
Tests Cases Test Scenarios
Requirements Features / Specifications
Separate Teams Embedded Teams
Functional Tests Specifications of Product
Behavior
Step 1, 2, 3 Given…When…Then
Testing in BDD
Static Behavior Living Documentation
© 2018 QASymphony – Confidential & Proprietary
Traditional Testing Testing in BDD
Step
Description
Expected
Result
Actual
Result
Walk up to machine Machine is
present
Select Coffee Buy Coffee selected Columbian Coffee
Selected
Enter money Machine takes
money
Machine only took
cash
Press Button Coffee Comes
out
Coffee didn’t
come out…maybe
I didn’t have
enough money?
Grab Coffee Coffee is Hot Couldn’t grab
coffee
Scenario: Buy Last Coffee
Given There are 1 Coffees left in the Machine
And I have deposited $5
And I previously have $100 in the bank
When I press the coffee button
Then I should be served a coffee
Feature: Serve Coffee
Coffee should not be served until paid for
Coffee should not be served until the button
has been pressed
If there is no coffee left, then the money
should be refunded
© 2018 QASymphony – Confidential & Proprietary
Ryan
Developer
Gina
Product Owner
Joseph
Test Engineer
BDD collaboration starts
Test scenarios versioned with code
Ryan
DeveloperJoseph
Test EngineerGina
Product Owner
Changes Synced in Real-time
? ?
Joseph
Test Engineer
Gina
Product Owner
Ryan
Developer
Fail feature for TDD
Source: http://hanwax.github.io/assets/tdd_flow.png
Joseph
Test Engineer
Gina
Product Owner
Ryan
Developer
Automate to Pass
Add Automated Steps& Implement
Source: http://hanwax.github.io/assets/tdd_flow.png
© 2018 QASymphony – Confidential & Proprietary
Scaling BDD
Management
Collaborate
Workflow
Report
Increased Customer Value
How do I enforce BDD
Structure?
How do store & manage
test scenarios?
How do I automate
BDD process?
How do I understand
business value?
© 2018 QASymphony – Confidential & Proprietary
Getting Started
NO
SOFTWARE
FOCUS ON
PROCESS
EXAMPLE MAPPING
ADOPT
CUCUMBER
STORE
FEATURES
AS CODE
CI
WORKFLOWS
SETUP REPORTING
SMALL
DECENTRALIZED TEAM
LEARN CORRECT
GHERKIN
© 2018 QASymphony – Confidential & Proprietary
Wrapping it up
1. BDD is an extension of TDD
2. BDD is about continuous conversation, not test automation
3. Organizational buy in is a must for success
4. Start lean, and prove it out before mass adoption
5. Alignment between Product / Dev / Test