MicroTESK: An ADL-Based Reconfigurable Test Program Generator for Microprocessors

Alexander Kamkin, Andrey Tatarnikov{kamkin, andrewt}@ispras.ru

Institute for System Programming of the Russian Academy of Sciences (ISPRAS)http://hardware.ispras.ru

SYRCoSe 2012: Testing and Monitoring of Computer Systems– May 30, 2012 - Perm, Russia 2 of 14

Design and Verification of Microprocessors

Hardware Description Languages (HDL)VerilogVHDL

Architecture Description Languages (ADL) nMLSim-nML

3 of 14

Levels of Microprocessor Verification

Unit-Level (via Input and Output Signals)

Core-Level Verification (via Test Programs)

SYRCoSe 2012: Testing and Monitoring of Computer Systems– May 30, 2012 - Perm, Russia 4 of 14

Approaches to Test Program Construction

Manual Development Random Generation Template-Based Generation Advanced Model-Based Generation

Requirements

Design

lui s1, 0x2779ori s1, s1, 0x0lui s3, 0x4eeori s3, s3, 0xfadd v0, a0, a2sub t1, t3, t5add t7, s1, s3?

Test Programs

SYRCoSe 2012: Testing and Monitoring of Computer Systems– May 30, 2012 - Perm, Russia 5 of 14

Constraint-Based Test Program Generation

Features: Context-Independent Solver Engines (e.g. Z3) Flexible Language (SMT-LIB) Rich Set of Supported Theories

Benefits: Less Effort to Create Tests Better Coverage Less Testing Bugs

Input ValuesConstraints(Invariants)

Output Values(Test Data)

SYRCoSe 2012: Testing and Monitoring of Computer Systems– May 30, 2012 - Perm, Russia 6 of 14

Evolution of Test Program Generators

Generator for a Particular Microprocessor Model-Based Generator

Generation Core Microprocessor Model

Instruction Set Model Testing Knowledge

Reconfigurable Model-Based Generator Generation Core Microprocessor Description/Configuration

SYRCoSe 2012: Testing and Monitoring of Computer Systems– May 30, 2012 - Perm, Russia 7 of 14

Motivation of the Project MicroTESK

Reduce Effort Needed to Switch to New Designs Simplify Development of Test Cases

Main Requirements: It should be easy to change the instruction set:

add/remove microprocessor instructions modify instructions’ semantics

It should be easy to change microarchitectural properties: cache memory configuration address translation algorithm

It should be able to generate test programs automatically according to high-level parameters

SYRCoSe 2012: Testing and Monitoring of Computer Systems– May 30, 2012 - Perm, Russia 8 of 14

General Structure of MicroTESK

MicroTESK

Design Model(Instruction Set,

Resources)

TestingKnowledge

ModelEngine

ModelingEngineer

VerificationEngineer

Specifications (ADL, Configuration)

ADL Translator

Test programsTest programsTest Programs

Test templatesTest templatesUser–DefinedTest Templates

Test Template Generator

Test Program Generator

CSP Solver

CSP SolverAPI

Basic TK

Model API

Libraries

Test Templates

SYRCoSe 2012: Testing and Monitoring of Computer Systems– May 30, 2012 - Perm, Russia 9 of 14

Architectural Description (nML/Sim-nML)

op ADD(rd: GPR, rs: GPR, rt: GPR)action = { if(NotWordValue(rs) || NotWordValue(rt)) then UNPREDICTABLE(); endif; tmp_word = rs<31..31>::rs<31..0> + rs<31..31>::rt<31..0>; if(tmp_word<32..32> != tmp_word<31..31>) then SignalException("IntegerOverflow"); else rd = sign_extend(tmp_word<31..0>); endif;}

syntax = format("add %s, %s, %s", rd.syntax, rs.syntax, rt.syntax)

op ALU = ADD | SUB | ...

Precondition

Test Situations

Equivalence Classes

Test Template

ALU r(), r(), r();ADD R, r(), r();; default;SUB r(), R, r();; overflow;

SYRCoSe 2012: Testing and Monitoring of Computer Systems– May 30, 2012 - Perm, Russia 10 of 14

Memory ConfigurationBuffer Configuration

Buffer associativity Number of rows in a buffer Structure of data item (fields and their size) Index function that calculates the position of data by its address Predicate checking whether it is hit or miss Data displacement strategy

Test Template

LD r(), 0x0, r();; hit(TLB, loaded::PFN1);; miss(L1);; hit(L2);...SD r(), 0x0, r();; hit(TLB, loaded::PFN2);; equals(PFN1, PFN2);; hit(L1);

buffer L1 = { set = 4 length = 128 line = { tag:card(27), data:card(32) } index(addr:36) = { addr<8..2> } match(addr:36) = { addr<35..9> == tag<0..26> } policy = LRU}

SYRCoSe 2012: Testing and Monitoring of Computer Systems– May 30, 2012 - Perm, Russia 11 of 14

Constraints Preconditions Possible Value Ranges Value Dependencies Pseudorandom Values

(declare-const rs DWORD64)(declare-const rt DWORD64)

(assert (IsValidSignedInt rs))(assert (IsValidSignedInt rt))(assert (not (IsValidSignedInt (bvadd rs rt))))(assert (not (= rs rt)))

(check-sat)(get-value (rs rt))

(define-sort DWORD64 () (_ BitVec 64))

(define-fun INT_ZERO () DWORD64 (_ bv0 64))(define-fun INT_BASE_SIZE () DWORD64 (_ bv32 64))

(define-fun INT_SIGN_MASK () DWORD64 (bvshl (bvnot INT_ZERO) INT_BASE_SIZE))

(define-fun IsValidPos ((x!1 DWORD64)) Bool (ite (= (bvand x!1 INT_SIGN_MASK) INT_ZERO) true false))

(define-fun IsValidNeg ((x!1 DWORD64)) Bool (ite (= (bvand x!1 INT_SIGN_MASK) INT_SIGN_MASK) true false))

(define-fun IsValidSignedInt ((x!1 DWORD64)) Bool (ite (or (IsValidPos x!1) (IsValidNeg x!1)) true false))

SYRCoSe 2012: Testing and Monitoring of Computer Systems– May 30, 2012 - Perm, Russia 12 of 14

Test Templates Preconditions Whole Class of Values Custom Sequencing Combinatoricsclass MyTemplate < Template def test() data = [ [0xEF, 0xFF], [0x1EF, 0x1FF], [0xFEF, 0xFFF] ]; data.each { |d| xor r0, r0, r0; ori r(2), r0, d[0]; ori r(4), r0, d[1]; ld tmp1=r(1), 0x0, r(2);; hit([L1(), L2()], [25, 50, 75]); ld tmp2=r(3), 0x0, r(4);; hit([L1(), L2()], [25, 50, 75]); dadd r(5), tmp1, tmp2;; overflow; } endend # class MyTemplate

SYRCoSe 2012: Testing and Monitoring of Computer Systems– May 30, 2012 - Perm, Russia 13 of 14

Conclusion

Adapting the Model-Based Testing Approach for Verification Engineers

Domain-Oriented Languages

Handling Frequent Design Changes and Maintenance of the Models

Easy-to-Modify Specifications / Configurations Push-Button Test Generation Techniques

SYRCoSe 2012: Testing and Monitoring of Computer Systems– May 30, 2012 - Perm, Russia 14 of 14

Thank You!Questions?