Lab 5 Coverage-Driven Verification

Kun-Bin Lee 李坤儐

kblee@yankees.ee.nctu.edu.twDec 9, 2002

Department of Electronics EngineeringNational Chiao Tung University

IP Core Design 2002

1/21

Institute of Electronics,National C

hiao Tung U

niversityCoverage-D

riven Verification

SoC Development

High Level Algorithm ModelC/C++/COSSAP/VCC/MATLAB

Hardware/Software PartitionN2C/VCC

Communication RefinementN2C/Port-C/VCC

Front End

Back EndHar

dwar

eD

evel

opm

ent

Syst

em L

evel

Des

ign

Hardware/Software CoverificationN2C/Seamless/"Q/Bridge"

Specification

Chip

Software

Developm

ent

RTOSWinCE/VxWorks

Device DriverDriveway

API EmbeddedSoftware

2/21

Institute of Electronics,National C

hiao Tung U

niversityCoverage-D

riven Verification

Lab Organization (Current Status in NCTU)

Getting Start with ADSWorking with AXD Software Quality Measurement

ARM Integrator Environment

µC/OS-II

Virtual Prototyping

Profiling

Application

Digital IP Authoring RTOSµHAL Driver

Rapid Prototyping

Embedded SW Authoring

3/21

Institute of Electronics,National C

hiao Tung U

niversityCoverage-D

riven Verification

SoC Verification

• Problem: 50~70% of design process is spent inverification

• An effective verification methodology is highly desirable

code Verify Synth P&R

code Verify Synth P&R

30~40%

50~80%

1996300K Gates

20001M SOC

4/21

Institute of Electronics,National C

hiao Tung U

niversityCoverage-D

riven Verification

Subblock Design Flow

WRITE functional specification

WRITE technical specification

DEVELOP timing constraints WRITE RTLRUN Lint

DEVELOP testbench

SYNTHESISDesign Compiler

SIMULATEVerilog/VHDL

MEASURE testbench coverageVHDLCover/VeriSure/CoverMeter

PASSES - READY FOR INTEGRATION

PERFORM power analysisPowerCompiler/QuickPower

Coverage tool passesMeets timing, power, & area requirements

Source: Michael Keating and Pierrr Bricaud, Reuse Methodology Manual, 2nd ed. 1999.

5/21

Institute of Electronics,National C

hiao Tung U

niversityCoverage-D

riven Verification

Verification Technology Overview

• Simulation Technology– Event-based– Cycle-based– Transaction-based– Code coverage– HW/SW co-verification– Emulation– Rapid prototyping– Hardware accelerator

• Static Technology– Lint check– Static timing

• Formal Technology– Theorem proving– Formal model check– Formal equivalence check

6/21

Institute of Electronics,National C

hiao Tung U

niversityCoverage-D

riven Verification

Coverage-Driven Verification

• Quantitatively analyze the simulation completeness with well-defined coverage metrics– Although 100% coverage still cannot guarantee a

100% error-free design• Generate more patterns for the uncovered areas

using formal techniques or designers’ knowledge• Tests optimization by eliminating tests that do not

add new coverage• Prioritize tests for regression runs• Provide a more systematic way to manage the

verification process

7/21

Institute of Electronics,National C

hiao Tung U

niversityCoverage-D

riven Verification

Testbench

DUTInput(Stimulus)

Output(Response)

Testbench

DUTInput(Stimulus) Output

Self-checking Testbench ExpectedOutput

CompResults

8/21

Institute of Electronics,National C

hiao Tung U

niversityCoverage-D

riven Verification

Testbench Creation

Extract Specific Functions

Create Test for Functions

Define Success &Failure Conditions

All FunctionsCovered?

System Specification

Yes

No

Testbench Ready for Verification

Focus on:• Corner Cases• Boundary Conditions• Design Requirements• Error Conditions• Exception Handling

9/21

Institute of Electronics,National C

hiao Tung U

niversityCoverage-D

riven Verification

Lint

• Syntax and semantic checks that ensure compliance with VHDL (IEEE 1076-1993) and Verilog (IEEE 1364/OVI 1.0, 2.0) HDL language standards

• Coding checks that analyze the HDL for simulation,synthesizability, design-for-reuse, and test requirements

• Design practice checks that include hierarchy, combinatorial loops, reset/clocking styles and many more

• Style, documentation, and naming checks that ensure adherence to coding guidelines

• In this Lab: VN-Check from TransEDA Verification Navigator

10/21

Institute of Electronics,National C

hiao Tung U

niversityCoverage-D

riven Verification

Example of Violation in Coding12 always @ (addr_in)13 begin14 if ((addr_in >= lower) && (addr_in <= upper))15 begin16 enable <= 1;17 addr_out = addr_in - lower;18 end19 else20 begin21 enable = 0;22 addr_out = 0;23 end24 end

11/21

Institute of Electronics,National C

hiao Tung U

niversityCoverage-D

riven Verification

Coding Guidelines

• Check RMM for more guidelines !– Reuse Methodology Manual for System-on-a-Chip

Designs, 3ehttp://search.barnesandnoble.com/booksearch/isbnInquiry.asp?bt=y&isbn=1402071418

– Xilinx Design Reuse Field Guidehttp://www.xilinx.com/ipcenter/designreuse/xrfg.htm

– Briefly introduction to RTL Coding Guidelineshttp://twins.ee.nctu.edu.tw/courses/ip_core_01/lab_hw_pdf/RTL_coding_guidelines.pdf

12/21

Institute of Electronics,National C

hiao Tung U

niversityCoverage-D

riven Verification

Types of Coverage

• Code coverage– Statement coverage– Block coverage– Decision coverage– Path coverage– Expression coverage– Event coverage– Toggle coverage– Variable coverage

• FSM coverage– Conventional FSM

coverage– Semantic FSM (SFSM)

coverage• Functional Coverage

13/21

Institute of Electronics,National C

hiao Tung U

niversityCoverage-D

riven Verification

Statement Coverage

always @ ( in or reset ) begin

out = in;

if ( reset ) out = 0;

en = 1;

end

There are 4 independent statements.

14/21

Institute of Electronics,National C

hiao Tung U

niversityCoverage-D

riven Verification

Branch Coverage

Measure the coverage of each branch in the if and case statements

always @ (in or reset) begin

out = in;

if ( reset ) out = 0; else ?

en = 1;

end

Implied else is also measured.

out=in

reset

en=1

out=0

15/21

Institute of Electronics,National C

hiao Tung U

niversityCoverage-D

riven Verification

Differences between SC and BC

If (b==a)c=1;

d=c;

Design In simulation

b is forced to always equal aSC = 100%BC = 50%

SC view BC view

16/21

Institute of Electronics,National C

hiao Tung U

niversityCoverage-D

riven Verification

Typical Coverage Targets

100Toggle

> 50Path

60~100 *Condition

100Branch

100Statement

Coverage Test (%)Measurement

* Depending on coverage tool

source : “Verification Methodology Manual For Code Coverage In HDL Designs” by Dempster and Stuart

17/21

Institute of Electronics,National C

hiao Tung U

niversityCoverage-D

riven Verification

FSM Coverage

• State• Arc

– An arc is a transition between two 'adjacent' states.– The arc coverage metric reports on those arcs actually traversed

during simulation, expressing these as a proportion of all possible arcs defined in the HDL code.

• Path– Identifies all the fundamental cyclic paths from which it then

constructs one or more supercycles which represent the main functionality of the FSM. The smaller cycles are then a part of thesupercycles.

– To the extent to which supercycles represent the intended operation modes of the FSM, a measure of coverage can then be obtained by:

• The percentage of all supercycles that have been fully traversed• The number of times a particular subordinate cycle has been

traversed

18/21

Institute of Electronics,National C

hiao Tung U

niversityCoverage-D

riven Verification

Conventional FSM Coverage

• The measurement of state visitation and state transitions

module counter (clk, rst, load, in, count) ;input clk, rst, load ;input [7:0] in ;output [7:0] count ;reg [7:0] count ;

always @(posedge clk) beginif (rst) count = 0 ;else if (load) count = in ;else if (count == 255) count = 0 ;else count = count + 1 ;

endendmodule

count0

count1

count2

count255

count254

…...256 states 66047 transitions

19/21

Institute of Electronics,National C

hiao Tung U

niversityCoverage-D

riven Verification

Semantic FSM Coverage

• Merge the states with same behavior into one semantic state to reduce the complexity

3 states 11 transitions

count= 0

count= in

count =count+1

module counter (clk, rst, load, in, count) ;input clk, rst, load ;input [7:0] in ;output [7:0] count ;reg [7:0] count ;

always @(posedge clk) beginif (rst) count = 0 ;else if (load) count = in ;else if (count == 255) count = 0 ;else count = count + 1 ;

endendmodule

20/21

Institute of Electronics,National C

hiao Tung U

niversityCoverage-D

riven Verification

Example of FSM Path Coverage

S1: A1, A2, A3, A4

S2: B1, B2, B3, B4

C1: A1, A2, A3, A4

C2: B1, B2, B3, B4

C3: B2, A4

C4: B3, C1

L1: I, A1

L2: I, B1

21/21

Institute of Electronics,National C

hiao Tung U

niversityCoverage-D

riven Verification

More Verification Information

• Verification Methodology Manual, 3rd EditionTechniques for Verifying HDL Designs Author: David Dempster and Michael Stuart– http://www.dacafe.com/DACafe/EDATools/BOOKINF

O/TransEDA/index.html– Worked examples for TransEDA VN in Appendix