staffan berg - t&vs · itba verification results •reached coverage in a single itba sim vs...
TRANSCRIPT
Staffan Berg
European Applications Engineer
Graph-Based Verification in a UVM Environment
July 2012
© 2010 Mentor Graphics Corp. Company Confidential
www.mentor.com
Graph-Based Intelligent Testbench Automation (iTBA)
Welcome DVClub — Attendees — Organizers — Presenters
Verification Challenges — Time To Market — Quality Products
Step Function Gains — Higher Coverage Faster — Testbench Automation
at SoC Level — Testbench Re-Use — Verification Productivity
Questa - April 2011 2
© 2010 Mentor Graphics Corp. Company Confidential
www.mentor.com
AGENDA
Graphs, Rules & Intelligent Testbench Automation
Why Graph-Based Verification is important
Fundamentals of iTBA
The Role of Coverage
Advanced Concepts
Graph based stimulus in the Context of OVM/UVM/*VM
Applications and Results – what to expect
SB , Graph Based Verification June 2012 3
© 2010 Mentor Graphics Corp. Company Confidential
www.mentor.com
Intelligent Testbench Automation (iTBA) From a Graph-Based Stimulus Description Model
4
Gary Smith -
“ . . .The [automatic] generation of a testbench from a system-level design description. . . “
Start = init repeat ( wait_rdy Rw_opts Rw_size ack ) ;
Rw_opts = setup_rd | setup_wr ;
Rw_size = rw_1 | rw_2 | rw_4 ;
Stop
init
Rw_opts
Rw_size
setup_rd setup_wr
rw_4 rw_2 rw_1
ack
Start
wait_rdy
SB , Graph Based Verification June 2012
Mentor Graphics -
“ . . . testbench automation that is aware
of the valid test space, the engineer’s
verification targets within that space,
and the current state of the design - that
uses automation to efficiently achieve
the verification goals . . . “
iTBA Definitions
The graph defines the ―valid test space‖
© 2010 Mentor Graphics Corp. Company Confidential
www.mentor.com
Benefits of Graph-based Verification
Predictable Coverage Closure — Achieve coverage goals in a fraction of the time — Graph techology enables N vs N*ln(N) advantage
Facilitate Re-use — Stimulus model is independent of testbench environment
– Same model can be re-used for SystemC, RTL, Emulation… – Same model can be re-used between projects
Brings iTBA to SoC level — Automatically generate stimulus at RTL block level — Automatically generate stimulus at SoC level (s/w & h/w)
Productivity improvements — Generation of testbench code — Graphical analysis / debug — Visualizing test scenario correlation to design spec
SB , Graph Based Verification June 2012 5
© 2010 Mentor Graphics Corp. Company Confidential
www.mentor.com
Defining the valid test space - Rules
Rules have two sections: — Declarations — Grammar
Declarations include: — Graph nodes linked to tasks/functions — Variables linked to tasks/functions — Rule building blocks
Grammar defines abstract behavior — Replaces directed test procedural code — Replaces CRT constraint code — Often declarative decision tree of choices — May also use algebraic constraints
Rules are implementation-independent
iTBA Rule
User-Created Rule Text
1
6 SB , Graph Based Verification June 2012
© 2010 Mentor Graphics Corp. Company Confidential
www.mentor.com
Visualizing the test space - Graphs
compile 2
• Rules compiled into graphs
• Graphs visually depict stimulus
• Protocol behavior
• Packet construction options..
• Stimulus sizing
• Stimulus sizing useful to assess
• What to target for coverage
• What to generate randomly
• Simulation time needed to reach coverage goals
• For this example
• 864 total test combinations
• 108 tests in CovParams sub-graph
Rules
7 SB , Graph Based Verification June 2012
© 2010 Mentor Graphics Corp. Company Confidential
www.mentor.com
Defining the Verification Goals - Coverage
• Coverage Strategy defines Goals & Priorities
• Specify regions to cover graphically
• Generate stimulus coverage code
• Two types of stimulus coverage
• Path coverage – traversal paths / cross coverage
• Node coverage – like a system verilog coverpoint
• Path coverage atomic_tb_bl_combos covers
• 108 combinations of am, bt, bl
• Node coverage bsz[] covers
• 8 values of bsz
Path Coverage
Node Coverage
User Added Stimulus Coverage Directives 3
8 SB , Graph Based Verification June 2012
© 2010 Mentor Graphics Corp. Company Confidential
www.mentor.com
Graph-based iTBA in Simulation
tb.sv top.sv iTBA_comp*.sv
Compiled Graph(s)
compile
iTBA runtime
Simulator
Runtime Graph
Debugging
• Integration/compilation
• Plug-in to existing methodologies
• Runtime code
• Graphs loaded into simulator
• Algorithms manage graphs during simulation
• Runtime graph debug
• Debug TB or DUT problems
• Set graph breakpoints
• Single-step or run to breakpoint
9 SB , Graph Based Verification June 2012
© 2010 Mentor Graphics Corp. Company Confidential
www.mentor.com
The Role of Coverage
The rule graphs define the entire stimulus space
Without coverage goals, traversal will be purely random
Graph coverage strategy defines goals and priorities
iTBA algorithms prioritize generation — Can still generate random vectors
outside the coverage space
SB , Graph Based Verification June 2012 10
© 2010 Mentor Graphics Corp. Company Confidential
www.mentor.com
Graph-based Verification Advanced Concepts Reactive / adaptive graphs
Some fields in a stimulus item may be a property of the testbench or DUT state, not randomly selected — And may still be included in a cross cover goal
These fields become inputs (imports) to a stimulus graph — Enables opportunistic targeting of random-resistant scenarios
11 SB , Graph Based Verification June 2012
Graph behaviour adapts to TB or DUT state
DUT state can be used to change constraints
© 2010 Mentor Graphics Corp. Company Confidential
www.mentor.com
Integrating Graph Based Verification Generating Stimulus in an OVM/UVM Environment
Stimulus is generated by sequences
A sequence produces sequence items
Driver applies stimulus to DUT
12 SB , Graph Based Verification June 2012
© 2010 Mentor Graphics Corp. Company Confidential
www.mentor.com
Generating Graph-Based Stimulus
Graph integrates within a sequence — Graph execution produces sequence items
Maximizes reuse of infrastructure — No need to change existing drivers, monitors, etc
iTBA Sequence
13 SB , Graph Based Verification June 2012
© 2010 Mentor Graphics Corp. Company Confidential
www.mentor.com
iTBA OVM/UVM Integration Process
Identify the target sequence Item
Describe stimulus domain with a graph • Declare a graph variable for each sequence-item field • Define relationships between graph variables
Define stimulus-coverage goals • Often corresponds to existing functional coverage goals
Run the iTBA sequence via an OVM/UVM test
14 SB , Graph Based Verification June 2012
© 2010 Mentor Graphics Corp. Company Confidential
www.mentor.com
Sequence Item
Verifying a simple bus protocol — 32-bit address — 64-bit data bus — Supports burst lengths up to 16 beats
Existing UVM sequence Item — Contains fields that describe a transaction
15 SB , Graph Based Verification June 2012
© 2010 Mentor Graphics Corp. Company Confidential
www.mentor.com
Defining Graph Variables
Declare a graph variable for each item field
Declare the valid domain of each variable
Sequence Item Rules
16 SB , Graph Based Verification June 2012
© 2010 Mentor Graphics Corp. Company Confidential
www.mentor.com
Declare Variable Relationship
Simple bus protocol constraints — Address must be aligned to the transfer size — Burst-transfer beats may only be 32 or 64-bit width
Graph permits flexible description of relationships — Branches — Algebraic constraints
Reuse existing constraints — Can be imported automatically
17 SB , Graph Based Verification June 2012
© 2010 Mentor Graphics Corp. Company Confidential
www.mentor.com
Simple Bus Protocol Graph
Graphical view of the stimulus space — Automatically created from rule description — Intuitive way to visualize choice tree
Graph branches restrict burst_len/size — Enables visual approach to code review
18 SB , Graph Based Verification June 2012
© 2010 Mentor Graphics Corp. Company Confidential
www.mentor.com
Graph Integration into Sequence
Graph nodes link to tasks in the sequence class
Tasks set the value of sequence-item fields
Integration code is automatically-created from graph
19 SB , Graph Based Verification June 2012
© 2010 Mentor Graphics Corp. Company Confidential
www.mentor.com
Define Stimuli-Coverage Goals
Simple protocol coverage goals — Cover 64 address ranges — Cover all valid combinations of transaction parameters
Describe address value binning
Define graph-coverage strategy — Node Coverage for address — Path Coverage for transaction parameters
Pre-simulation coverage space analysis
20 SB , Graph Based Verification June 2012
© 2010 Mentor Graphics Corp. Company Confidential
www.mentor.com
Testbench Environment Integration
iTBA sequence is just like any other sequence — Use standard approaches to select and execute
Select the iTBA sequence via a type override
Explicitly create and run iTBA sequence
21 SB , Graph Based Verification June 2012
© 2010 Mentor Graphics Corp. Company Confidential
www.mentor.com
Example iTBA Applications
Architectural exploration — Performance characterization — Behavioral model verification
IP Block level verification — Bus (AXI, AHB, PCI*, …) — Memory (DDR,SDDR, nand flash,…) — Peripheral (i2c, dma,…)
SOC integration — Firmware — Subsystem interactions
Processor verification — L2 cache — Instruction set verification
22 SB , Graph Based Verification June 2012
© 2010 Mentor Graphics Corp. Company Confidential
www.mentor.com
iTBA Verification Results
• Reached coverage in a single iTBA sim vs multiple CRT sims/seeds
• Minimum 10x coverage closure advantage for simple CRT cases
• Often see >100x coverage closure advantage for common CRT cases
23 SB , Graph Based Verification June 2012
© 2010 Mentor Graphics Corp. Company Confidential
www.mentor.com
Graph-Based iTBA Gaining Acceptance Across The Globe
Office
Products
Printer Image
Processor
Questa
SystemVerilog
3
Days
36 hours on 6 CPUs 8 weeks on 6 CPUs
100% coverage 60% coverage
37 X faster
+ 40% coverage =
Wireless
Networking
Ethernet 802.11
Device
VCS
NTB 35
Hours
48 CPU hours 3175 CPU hours
97% coverage 95% coverage
66 X faster
+ 2 % coverage =
Consumer
Electronics
Error Checking and
Correcting Module
NC Sim
Specman e
1
Day
118 minutes >18 hours
100% coverage 100% coverage
9.5 X faster
Equal coverage =
Storage &
Networking AXI Bus Bridge
VCS
SystemVerilog
1
Week
170 X faster 196,000 tests 26,315,000 tests
100% coverage 79% coverage + 21% coverage =
Basestation
Telecom
Proprietary Interface
Module for a Router
NC Sim
Specman e
1
Week
75,000 vectors 825,000 vectors
100% coverage 100% coverage
10 X faster
Equal coverage =
Industry Design Verification Time Current Results Ultra Results Benefits
Switching
Subsystems
Multiple Master
AXI Bus Fabric
Questa
Directed Tests
2
Days
400,000 tests 10,000 tests 40 X more tests =
Wireless
Telecom Interrupt Controller
VCS
Vera and SV
6
Days
45 minutes 3 days
100% coverage 100% coverage
27 X faster
Equal coverage =
Processors Multi-Core Memory
Sub-system
Questa
SystemVerilog
1
Day
30 minutes 5 hours
100% coverage 100% coverage
10 X faster
Equal coverage =
170 X faster
37 X faster
9.5 X faster
10 X faster
66 X faster
+ 40% coverage
+ 21% coverage
40 X more tests
27 X faster
10 X faster
iTBA - Accelerating Time to Coverage Closure 24
© 2010 Mentor Graphics Corp. Company Confidential
www.mentor.com
Conclusions….
Graph algorithms enable fast, efficient — Coverage closure — Test scenario generation
Graph-based stimulus description emphasizes reuse — Vertically from block to SoC level
– And across multiple testbench environments / languages
— Horizontally from project to project
Easily integrate into existing environments, e.g. OVM/UVM
SB , Graph Based Verification June 2012 25
© 2010 Mentor Graphics Corp. Company Confidential
www.mentor.com SB , Graph Based Verification June 2012 26