continuous testing - what qa means for devops
Post on 13-Apr-2017
176 Views
Preview:
TRANSCRIPT
Continuous Testing What QA means for DevOps
Eran SherCo-founder & CEO of SeaLights
Nolio (Acquired by CA Technologies)Mercury Interactive (Acquired by HP)
Conduct (Acquired by Mercury)
The State of QA & DevOps
“managing and implementing quality and test measures represent a challenge for teams trying to increase productivity (or release velocity) with acceptable quality”WORLD QUALITY REPORT 2016-2017
IncreasingRelease speed
Decreasing Release quality
Continuous Delivery is becoming the de-facto standard
The Industry Reaction
The Industry Reaction
Organizational People Skills Tools Practices Communication
Now that the hysteria has passed
What QA Means for DevOps
Continuous IntegrationHow we build software
Continuous DeploymentHow we deploy software
Continuous TestingHow we test software
Continuous Delivery
Continuous Testing
Increase quality while increasing speed
Step 1
Understand what is causing quality to decrease when speed is increased
Understanding Test Quality
How do you measure test quality?
Quality Holes
How do you stop untested code changes from reaching production?
Centralized Quality Dashboard
How do you collect quality metrics across environments and tools?
Step 2
Find/ develop a solution for the identified root cause problems
Uncover Code Coverage
Unit Integration Functional UI API Exploratory
? ? ? ? ?
Identify Quality Holes
Scan every build and analyze code changes
Track every test and analyze footprints
Build Quality Dashboard
Quality report per build Collect statuses and coverage across tools, environments and silos
Track executed tests
Step 3
Integrate solutions into CI/CD
Case Study
Our Technical Stack:1. Node.js, Java, and Python
2. Microservices architecture, Mongo DB
3. CI: Jenkins
4. SCM: Github
5. Test automation frameworks: Mocha, TestNG, JUnit, and Py.test
6. Infrastructure: AWS
7. CD: homegrown
Case Study
Our CI/CD Workflow1. Pull request
2. Build + unit test
3. Verify that all tests passed, no decrease in coverage, and no quality holes
4. Integration build
5. Setup and deploy build to integration lab (AWS)
6. Run integration tests
7. Verify that all tests passed, no decrease in coverage, and no quality holes
8. Ready for production » Deploy
Copyright © 2016 Sealights. All rights reserved.
Q&A
Start testing with the lights on
Copyright © 2016 Sealights. All rights reserved.
top related