integration and functional testing using dockerized ......integration and functional testing using...

29
T11 Virtualization Testing Thursday, May 3rd, 2018 11:15 AM Integration and Functional Testing Using Dockerized Dependencies Presented by: Irene Dhong Bazaarvoice Brought to you by: 350 Corporate Way, Suite 400, Orange Park, FL 32073 888-- -268---8770 ·· 904- --278-- -0524 - [email protected] - http://www.stareast.techwell.com/

Upload: others

Post on 23-Jun-2020

24 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Integration and Functional Testing Using Dockerized ......Integration and Functional Testing Using Dockerized Dependencies Presented by: Irene Dhong Bazaarvoice Brought to you by:

T11 Virtualization Testing Thursday, May 3rd, 2018 11:15 AM

Integration and Functional Testing Using Dockerized Dependencies

Presented by:

Irene Dhong

Bazaarvoice

Brought to you by:

350 Corporate Way, Suite 400, Orange Park, FL 32073 888-- -268- - -8770 ·· 904- --278-- -0524 - [email protected] - http://www.stareast.techwell.com/

Page 2: Integration and Functional Testing Using Dockerized ......Integration and Functional Testing Using Dockerized Dependencies Presented by: Irene Dhong Bazaarvoice Brought to you by:

Irene Dhong Bazaarvoice Irene Dhong has been a QA Engineer for over 6 years. She has experience as a build engineer and a system administrator. She's passionate about building frameworks and tools that maintain a high level of quality in lean, rapid development environments. She enjoys bringing quality to new challenges like machine learning, natural language processing, and complex microservice ecosystems. Irene loves to encourage community among women in technology and runs a women's leadership guest speaker series at Bazaarvoice.

Page 3: Integration and Functional Testing Using Dockerized ......Integration and Functional Testing Using Dockerized Dependencies Presented by: Irene Dhong Bazaarvoice Brought to you by:

Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 0

Integration and Functional Testing Using Dockerized Dependencies

Irene Dhong

Page 4: Integration and Functional Testing Using Dockerized ......Integration and Functional Testing Using Dockerized Dependencies Presented by: Irene Dhong Bazaarvoice Brought to you by:

Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 1

Ice Cream Cone

Page 5: Integration and Functional Testing Using Dockerized ......Integration and Functional Testing Using Dockerized Dependencies Presented by: Irene Dhong Bazaarvoice Brought to you by:

Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 2

•High quality low defects

•Expensive

•Long running tests

•Extremely hard to add a new test

•Debugging difficult or near impossible

The Struggle

Page 6: Integration and Functional Testing Using Dockerized ......Integration and Functional Testing Using Dockerized Dependencies Presented by: Irene Dhong Bazaarvoice Brought to you by:

Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 3

•Amount of mocking out needed

•Interactions hard to simulate

•Manual Custom ops code

•Dozens of dependencies

•Fast pace of changing code

How did we get here?

Page 7: Integration and Functional Testing Using Dockerized ......Integration and Functional Testing Using Dockerized Dependencies Presented by: Irene Dhong Bazaarvoice Brought to you by:

Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 4

•Be able to write and run tests quickly

•Have all dependencies mocked out

•Try to catch quality issues before it even reaches ci

•Test on a very low level

•Write hermetic tests

•Have access to service/application/logs while developing

•Reuse code

My wish list

Page 8: Integration and Functional Testing Using Dockerized ......Integration and Functional Testing Using Dockerized Dependencies Presented by: Irene Dhong Bazaarvoice Brought to you by:

Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 5

•Docker is a computer program that performs operating-system-level virtualization also known as containerization. *wikipedia

•Docker is a platform for developers and sysadmins to develop, deploy, and run applications with containers. The use of Linux containers to deploy applications is called containerization. Containers are not new, but their use for easily deploying applications is. *Docker.com

Docker

Page 9: Integration and Functional Testing Using Dockerized ......Integration and Functional Testing Using Dockerized Dependencies Presented by: Irene Dhong Bazaarvoice Brought to you by:

Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 6

DockerizedWorld

•Elements of our system are run in containers

•Links used to expose elements to each other

•Use official images for infrastructure

Page 10: Integration and Functional Testing Using Dockerized ......Integration and Functional Testing Using Dockerized Dependencies Presented by: Irene Dhong Bazaarvoice Brought to you by:

Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 7

•Unit-like

•Local Integration

•Can be run locally or in ci

•Deployed integration tests in production or production-like environment

Supports different kinds of testing

Page 11: Integration and Functional Testing Using Dockerized ......Integration and Functional Testing Using Dockerized Dependencies Presented by: Irene Dhong Bazaarvoice Brought to you by:

Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 8 The image part with relationship ID rId22 was not found in the file.

Unit like testing environment

@BeforeSuite

• Start dep

@Test

• Test function

• Verify data

Page 12: Integration and Functional Testing Using Dockerized ......Integration and Functional Testing Using Dockerized Dependencies Presented by: Irene Dhong Bazaarvoice Brought to you by:

Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 9 The image part with relationship ID rId22 was not found in the file.

A Hermetic functional test environment

@BeforeSuite

• Start dep and local app

@Test

• Setup, Start operation, Verify data

@AfterTest

• Clean up data

Page 13: Integration and Functional Testing Using Dockerized ......Integration and Functional Testing Using Dockerized Dependencies Presented by: Irene Dhong Bazaarvoice Brought to you by:

Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 10 The image part with relationship ID rId22 was not found in the file.

Local Integration Test Environment

@BeforeSuite

• Start dep and local app

@Test

• Setup data, Trigger app, Verify data

@AfterTest

• Clean up

Page 14: Integration and Functional Testing Using Dockerized ......Integration and Functional Testing Using Dockerized Dependencies Presented by: Irene Dhong Bazaarvoice Brought to you by:

Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 11 The image part with relationship ID rId22 was not found in the file.

Local Integration Test Env or CI• Start dockerized

dependencies

• Start dockerizedapplication

• Run tests

Page 15: Integration and Functional Testing Using Dockerized ......Integration and Functional Testing Using Dockerized Dependencies Presented by: Irene Dhong Bazaarvoice Brought to you by:

Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 12 The image part with relationship ID rId22 was not found in the file.

Integration tests run in deployed env

• Reuse test framework using config.yaml

• Omit startup, setup, and cleanup steps

Page 16: Integration and Functional Testing Using Dockerized ......Integration and Functional Testing Using Dockerized Dependencies Presented by: Irene Dhong Bazaarvoice Brought to you by:

Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 13

•Simulate real life situation

•Reuse existing data loading capabilities

•Explore transitional states

•Insight to data

•Large datasets

•Clean databases

•Hermitic data

Issue: Hard to add data

Page 17: Integration and Functional Testing Using Dockerized ......Integration and Functional Testing Using Dockerized Dependencies Presented by: Irene Dhong Bazaarvoice Brought to you by:

Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 14

•One time cost

•Maintained through dockerfile

•Stubbed dockerized dependency only tests interactions that tests require

•As dependency changes, maintenance is little as can use POJO’s

Issue: Huge amounts of mocking out

Page 18: Integration and Functional Testing Using Dockerized ......Integration and Functional Testing Using Dockerized Dependencies Presented by: Irene Dhong Bazaarvoice Brought to you by:

Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 15 The image part with relationship ID rId22 was not found in the file.

Issue: Manual custom ops code• Database table

creation in dockerfile

• Devops/dev updates table files in one file

• Same image used for dev, devops, and testing

Page 19: Integration and Functional Testing Using Dockerized ......Integration and Functional Testing Using Dockerized Dependencies Presented by: Irene Dhong Bazaarvoice Brought to you by:

The image part with relationship ID rId2 was not found in the file.

Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 16

•Redeploy application pointing to dockerized dependency

•Hybridization

Issue: Disaster Recovery/Perf Test

16

Page 20: Integration and Functional Testing Using Dockerized ......Integration and Functional Testing Using Dockerized Dependencies Presented by: Irene Dhong Bazaarvoice Brought to you by:

Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 17

•Advantages over everything running locally

•Advantages over deployed environment

Issue: Debugging in Deployed Environment

Page 21: Integration and Functional Testing Using Dockerized ......Integration and Functional Testing Using Dockerized Dependencies Presented by: Irene Dhong Bazaarvoice Brought to you by:

Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 18 The image part with relationship ID rId22 was not found in the file.

Challenge: No official docker image

• S3, sqs

• FakeS3

• Redis

Page 22: Integration and Functional Testing Using Dockerized ......Integration and Functional Testing Using Dockerized Dependencies Presented by: Irene Dhong Bazaarvoice Brought to you by:

Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 19 The image part with relationship ID rId22 was not found in the file.

Challenge: Impossible to dockerize

• Stub it and dockerize it!

• Quick flask server

• Don’t hardcode, make stub interactive

• Risk analysis

Page 23: Integration and Functional Testing Using Dockerized ......Integration and Functional Testing Using Dockerized Dependencies Presented by: Irene Dhong Bazaarvoice Brought to you by:

Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 20

•Keys

•Mimic

Challenge: Security

Page 24: Integration and Functional Testing Using Dockerized ......Integration and Functional Testing Using Dockerized Dependencies Presented by: Irene Dhong Bazaarvoice Brought to you by:

Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 21

•Update shared pojos, classes, and tests

•Build latest image with latest table scripts or changes

•PR build which runs all local tests

Challenge: Development changes code

Page 25: Integration and Functional Testing Using Dockerized ......Integration and Functional Testing Using Dockerized Dependencies Presented by: Irene Dhong Bazaarvoice Brought to you by:

Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 22

•Mvn, gradle, sbt

•Python, Java, Javascript, Scala

Challenge: Multitude of technologies

Page 26: Integration and Functional Testing Using Dockerized ......Integration and Functional Testing Using Dockerized Dependencies Presented by: Irene Dhong Bazaarvoice Brought to you by:

The image part with relationship ID rId2 was not found in the file.

Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 23

Challenge: Multitude of technologies• Use common scripts

Page 27: Integration and Functional Testing Using Dockerized ......Integration and Functional Testing Using Dockerized Dependencies Presented by: Irene Dhong Bazaarvoice Brought to you by:

Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 24

•Investment of time

•Mocked/Stubbed components

•Isolated from the bigger picture/world

•Too close to the code

Testing specific cons

Page 28: Integration and Functional Testing Using Dockerized ......Integration and Functional Testing Using Dockerized Dependencies Presented by: Irene Dhong Bazaarvoice Brought to you by:

Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 25

•Offline development

•Containers are isolated so don’t have to pollute machine with various packages and system services

•Dev use same infrastructure for unit like dependency testing

•Devops use same image for deployment with different config

•Reuse of code in all aspects

Pros generally

Page 29: Integration and Functional Testing Using Dockerized ......Integration and Functional Testing Using Dockerized Dependencies Presented by: Irene Dhong Bazaarvoice Brought to you by:

Confidential and Proprietary. © 2015 Bazaarvoice, Inc. 26

Testing specific pros