kvm-autotest

OutlineProblem Framing

Resolution strategy

KVM autotest: It is not just a QA tool anymore

Lucas Meneghel [email protected]

KVM Forum 2012November 5th, 2012

Lucas Meneghel Rodrigues [email protected] KVM autotest: It is not just a QA tool anymore

Resolution strategy

1 Problem Framing

2 Resolution strategy

Resolution strategy

The juggler dillema

It is hard to juggle

• Code/new featuredevelopment

• Bug handling

• Write tests for all thefeatures

Resolution strategy

The juggler dillema

• Bug handling

Resolution strategy

The juggler dillema

• Bug handling

Resolution strategy

How can we accomodate testing?

Test tools have to be easy

• To understand

• To hack

• To do useful things with

Resolution strategy

• To understand

• To hack

Resolution strategy

• To understand

• To hack

Resolution strategy

QA vs developer level testing

If the testing tools (written forQA) are hard to understand:

• “Ok, I’ll write my own tool”.

• It is fine. Then you startrepeating things over andover.

Resolution strategy

QA vs developer level testing

If the testing tools (written forQA) are hard to understand:

• “Ok, I’ll write my own tool”.

• It is fine. Then you startrepeating things over andover.

Resolution strategy

KVM autotest: The hard path

KVM autotest was written withfocus on QA level testing

• It has grown to cover libvirtand other virt backends

• It can do migration, virtioconsole, hotplug, amongothers

• But has a steep learningcurve

Resolution strategy

How to solve the problem

Layered approach

• Keep the code base workingfor the original cases

• Get rid of useless things forthe development use caseand expose the essentials

• “I don’t care about autotest,just give me a test suite”

Resolution strategy

Layered approach

Resolution strategy

Layered approach

Resolution strategy

Split test modules from autotest core

Going after the layered approach

• Autotest will provide astable test API

• Test modules are developedindependently

• Merged all virt types in asingle test module

Resolution strategy

Turn virt tests into a separate suite

Virt tests still have all the codeto work under autotest

• It works as a separatetestsuite, with autotest deps

• After a bootstrap stage, justexecute a simple runner

Resolution strategy

Turn virt tests into a separate suite

Virt tests still have all the codeto work under autotest

• It works as a separatetestsuite, with autotest deps

• After a bootstrap stage, justexecute a simple runner

Resolution strategy

Minimize deps: JeOS

Nowadays this is anotherbuzzword: “Just enough OS”

• In real life, people need asmall guest to run tests on

• Very small Fedora 17 x86 64sparse image

• Easy to maintain, fairlysmall compared to a fullDVD

Resolution strategy

Minimize deps: JeOS

Resolution strategy

Minimize deps: JeOS

Resolution strategy

Test runner

Present the tests in aminimalistic way

• Terse, unittest like output

• Can list available tests

• Provide a qemu path and atest list and you’re good

Resolution strategy

Test runner

Resolution strategy

Test runner

Resolution strategy

Changes in autotest (last year)

Lots of work under the hood

• Namespace fixes andcleanups

• Improved releasemanagement

• Fedora packaging work done

• Stand alone RPC client

Resolution strategy

Roadmap

What now?

• Allow to run tests written onany language

• Evolve core functionalityinto libraries

• Run tests out of tree (say,qemu tree)

• You all are welcome to help

Resolution strategy

Roadmap

What now?

Resolution strategy

Roadmap

What now?

Resolution strategy

Roadmap

What now?

Resolution strategy

Questions?

Resolution strategy

Contact

• [email protected]

• Virt test devel list ([email protected])

kvm-autotest

Documents