Quality in an Agile worldAlon Weiss

About me

● 34, Married + 1

● Architect/Director of R&D at Sealights, 1st employee

○ (just announced a $11M funding round)

● Architect/Group Leader/Project Manager/Team Leader/Engineer at Varonis

● MBA, BA in CS from IDC

● I’ve been coding since I was 6

● I really love what I do, and I do it with passion

The Software IndustryWelcome :-)

A little history of software delivery for PCs

● Pre-packaged software, bundled with the hardware

○ MS-DOS, Macintosh

● Off-the-shelf, physically packaged software

○ MS Office, Games

● Downloadable software (with the Internet)

● Online services (“Software-as-a-service”)

○ Gmail.com, facebook.com

Pretty much the same with mobile and IoT

Imagine you need to create a note-taking app

Imagine you need to create a note-taking app

● Audience

● Use cases

● Platforms: Web-based? Mobile? Desktop?

● Features

● UI? UX

● Scale

● Roadmap

Waterfall

Source: wikipedia

Waterfall - Who will we need?

To provide requirements:

Market research

“Field work” inputs from sales/marketing

Visionaries / Strategists / Evangelists

Waterfall - Who will we need?

To create the software design:

Software Architects

Experts

Consultants

Additional design tasks:

● UX designers,

● Graphic designers

Waterfall - Who will we need?

To actually write the software:

Software Engineers

Waterfall - Who will we need?

To verify that what you wrote works:

Quality Assurance

Waterfall - Who will we need?

To maintain & support your software:

Support engineers

Some engineers to fix bugs

Some QA engineers to verify those bugs

Waterfall - Application Lifecycle

How do we ensure quality?

Software delivery - Release cycles

● Pre-packaged software, bundled with the hardware

○ Versions usually come out once, rarely modified afterwards

○ Extremely hard/impractical to recall a product or fix a bug after release

● Off-the-shelf, physically packaged software

○ Versions usually come out ~annually

○ Assembly lines, inventories

○ Recalls are costly

● Downloadable software

○ Versions are easily distributable, can be continuously updated

Scale

Cloud Computing - extremely short review

● We started with “Servers”:

○ Machines that are dedicated to run some software that others can use

○ E.g. Classic Web Server - listen for HTTP(s) traffic and delivers files (usually with some logic

running, like user managements, session management to keep track of orders, etc.)

○ Expensive

○ Scattered geographically at different data centers

○ Limited access

○ Hard to maintain (downtime? Load balancing?)

Cloud Computing - extremely short review

● We then had virtualization

○ Operating system-inside-an-Operating system

○ Software layer abstracts a lot of hardware concerns and allows tweaking resource usage:

■ CPU (dedicated cores / metered CPU cycles)

■ Memory (e.g. 2Gb RAM limit)

■ Storage (attached storage, like HDDs/SSDs, e.g. 80Gb)

○ Companies can provide these VMs as a service

■ EC2

■ Google Cloud

Cloud Computing - extremely short review

● “Get me 10 VMs running Centos 7, each having 4 CPU cores, 8Gb of ram,

40Gb of attached storage, 10Gb network interfaces”

● API-centric

○ Technology agnostic

○ Built for automation

● Additional low-level services

○ Cloud storage

○ Load balancers

○ Private networking

Cloud Computing - higher level services

● Platform as a service

○ “Run this Java application on 10 instances, and scale as required”

● Database-as-a-service

● Message queues

● Higher level automation of cloud services

● Identity & Access management

● Email/SMS

● Many more

Amazon Web Service (AWS)

Cloud computing - why does it matter to me

● No need to provision any hardware. pay-as-you-go

● Your software can reach millions in a few API calls

● Releases can be completely autonomous

● You can easily scale up and down to meet demand or reduce cost

● “Software is eating the world” (Marc Andreessen, 2011)

Modern software development

Continuous Integration

Automates a lot of processes that are required to create a “release”

● Automatic Builds

○ Desktop: for Windows 7, Windows 8, Windows 10, 32bit/64bit

○ Web: sometimes different builds for browsers/platforms

○ Mobile: Android, iOS, different form factors

● Source control

○ Where the source code files & assets are stored

● Quality

○ Run automatic tests on the software, to establish a minimum level of quality

Continuous Delivery

Automates a lot of processes that are required to take an approved “release” and

deliver it to customers

● If software is packaged, upload it to some public website/repository (e.g. App

Stores)

● If software is a service, gradually upgrade/replace servers with the updated

version

○ Continuously monitor the process, to detect quality issues as fast as possible

● Requires a lot automation efforts

Agile development

● “Agile software development refers to a group of software development methodologies based on iterative

development, where requirements and solutions evolve through collaboration between self-organizing cross-

functional teams. ... Read more about the Agile Manifesto. SCRUM. Scrum is a subset of Agile.”

● A buzz-word, that links to other buzz-words

● Iterative approach - work in short, defined cycles

○ No “Full requirements”

○ No “Grand Design”

○ No “All-or-nothing”

○ Short “QA” sessions

○ Features can be split into multiple cycles

○ Continuous improvements

Agile - but not only for developers

● Water-scrum-fall

○ The organization is mostly “waterfall” oriented, but R&D is agile

○ Completely missing the point of Agile development

○ Most companies are like that

● The Lean Startup

○ Completely agile organization

○ How to build a startup - Udacity

Software quality - why does it matter?

Lots of competition

In app stores, rating is the most important

One chance to make a first impression

Users may run away if they see too many bugs

The product/service need to work, otherwise it’s a waste of your user’s time

Unimaginable and uncontrollable number of devices, platforms, configurations,

technologies

(Unless you are Apple, in which this is kept under control)

Higher quality -> efficient R&D -> faster releases

We guarantee software quality by testing it

● Unit tests - test code in isolation

○ Your code needs to be testable

○ TDD, BDD

● Component tests - test whole components as a single

entity

● UI tests - test the UI component of the service,

individually or together with other products

● Integration tests - test multiple components working

together

● Regression tests - tests all features of your product

Continuous Testing

● Monitor quality across all components, test phases, and integrate with

○ Unified dashboard that shows all quality-related metrics and info

● Easily address day-to-day R&D issues

○ Does the release meet the criteria to deploy/ship it?

○ What change(s) caused bug X, who made those

○ What changes and new content have been introduced in this new build?

■ Were those changes properly tested? Where and how

Sealights

Thank you!