avocado and jenkins -...
TRANSCRIPT
AVOCADO AND JENKINSAVOCADO AND JENKINST E S T- A U TO M AT I O N A N D C O N T I N U O U ST E S T- A U TO M AT I O N A N D C O N T I N U O U SI N T E G R AT I O NI N T E G R AT I O N
YASH MANKAD LUKÁŠ DOKTORYASH MANKAD LUKÁŠ DOKTOR
Software Engineer, Red Hat Software Engineer, Red Hat
06-02-2016
DevConf 2016
OVERVIEWOVERVIEW( What are we going to learn ? )
Avocado and Jenkins: Test-automation and CI; Yash Mankad and Lukáš Doktor
Previous SetupProblem Statement
Patch handling
Development-Quality Engineeringhandoffs
Test-automation
OVERVIEWOVERVIEW( What are we going to learn ? )
Avocado and Jenkins: Test-automation and CI; Yash Mankad and Lukáš Doktor
Design GoalsWhere did we start ?
ApproachIntegrating test-automation
Challenges faced
Why Avocado ?Demo
Avocado and Jenkins: Test-automation and CI; Yash Mankad and Lukáš Doktor
PREVIOUS SETUPPREVIOUS SETUP( How did it work before ? )
Developer Patch Mailing List
Patch QueueBuildsQuality Engineering
Developer
Patch Mailing List Patch Queue Builds
Avocado and Jenkins: Test-automation and CI; Yash Mankad and Lukáš Doktor
Maintainer
PROBLEM STATEMENTPROBLEM STATEMENT( Why are we changing it ? )
ExtensiveGuidelines
Development
Avocado and Jenkins: Test-automation and CI; Yash Mankad and Lukáš Doktor
PROBLEM STATEMENTPROBLEM STATEMENT( Why are we changing it ? )
Quality Engineering
Development
Patch HandlingExtensive Patch guidelinesManual builds
Development-QE Handoffs
Different testing schedulesTime between patch submissionand testing is too long
PROBLEM STATEMENTPROBLEM STATEMENT( Why are we changing it ? )
Avocado and Jenkins: Test-automation and CI; Yash Mankad and Lukáš Doktor
Test-AutomationQuality Engineering teams used Autotestframework
Quality Engineering has to use lot of tools ontop of Autotest
Autotest is no longer maintained upstream
Avocado and Jenkins: Test-automation and CI; Yash Mankad and Lukáš Doktor
WHERE DID WE START?WHERE DID WE START?( Our initial efforts )
Developer Patch Mailing ListPatch Queue
Fail early, fail oftenCredit to Fam Zheng's Patchew(http://qemu.patchew.org/)Wrote a script that automates a part of the earlytesting process and runs functional sanity tests
Avocado and Jenkins: Test-automation and CI; Yash Mankad and Lukáš Doktor
WHERE DID WE START?WHERE DID WE START?( Making things run more efficiently )
Patch Queue
Had to focus on Development-Quality Engineeringhandoff
Script applies patches and triggers builds, whichtriggers the CI
Builds
Avocado and Jenkins: Test-automation and CI; Yash Mankad and Lukáš Doktor
INTEGRATING TEST-INTEGRATING TEST-AUTOMATIONAUTOMATION( How it is all coming together! )
Tests run based on keywords such as boot, migrate,network, virtio, etc.Different Jenkins slave based on architecture andcomponent
Development Quality Engineering
+Avocado
CHALLENGES FACEDCHALLENGES FACED( What we had to overcome )
Avocado and Jenkins: Test-automation and CI; Yash Mankad and Lukáš Doktor
Implement new features without affecting oldfeatures
libvirt CI supports upstream community anddownstream for RHEL
Jenkins jobs and configuration are automatedusing scripts
Currently, we are migrating from Autotest toAvocado
WHY AVOCADO ?WHY AVOCADO ?
Avocado and Jenkins: Test-automation and CI; Yash Mankad and Lukáš Doktor
Avocado and Jenkins: Test-automation and CI; Yash Mankad and Lukáš Doktor
The only possible alternative for virtualization:40017 qemu, 56745 libvirt tests (Also supportslibguestfs, lvsb, v2v, openvswitch, spice testproviders)3 (5) types of cpu architecture147 versions of operating systems864 hardware variants25412395680 theoretical combinations
Measured on 2016-02-03 using x86_64 + RHEL.7.devel to evaluate the number of tests and hw variants
WHY AVOCADO ?WHY AVOCADO ?
WHY AVOCADO ?WHY AVOCADO ?
Avocado and Jenkins: Test-automation and CI; Yash Mankad and Lukáš Doktor
Avocado and Jenkins: Test-automation and CI; Yash Mankad and Lukáš Doktor
Talks to JenkinsEven our developers are using it locallyYou can actually run “anything” with itGives you well defined results structure (withbenefits)Supports neat features
WHY AVOCADO ?WHY AVOCADO ?
Questions?