better living through formal - test and verification ... · •add a formal element to your sanity...

1 © 2016 Broadcom. All Rights Reserved. "Broadcom" refers to Broadcom Limited and/or its subsidiaries. | © 2016 Broadcom. All Rights Reserved. "Broadcom" refers to Broadcom Limited and/or its subsidiaries.

Better Living Through Formal

16th June 2016

Alex Orr

2 © 2016 Broadcom. All Rights Reserved. "Broadcom" refers to Broadcom Limited and/or its subsidiaries. |

• D.A.V.E in various UK startups for ~25 years – Purely verification for last 5 years

– 3 years at Broadcom

• Exclusively dynamic simulation pre-Broadcom – Primarily constrained random methodology

– Migrating to UVM

– “Just show me the waves”

– Red Xs

• An EDA user – Future employment depends on pragmatic and timely application of methodologies and tools

Quick Biography

© 2016 Broadcom. All Rights Reserved. "Broadcom" refers to Broadcom Limited and/or its subsidiaries.

My First 100 Days in Formal Land


• First presented at FVC2015 (Reading)

• Main conclusions – Dynamic verification can be seen as attempting to prove a negative

– “I’ve never seen ...” caveated with “yet!”

– Formal verification is positive

– You get a mathematically correct proven status …

– … but can still be wrong (GIGO)

– Formal verification has some design sweet spots

– e.g Arbiters

– Can take billions of cycles to hit all corner cases but just a few assertions.

– Deliver these as separate, exhaustively proven modules along with proofs (Imagination)

– And so does dynamic verification

– Max and match approaches

– Don’t try to use just one methodology for all

My First 100 Days In Formal Land

FF1 RAM1 RAM2 IP

OP 1

2

3

4

Priority


– Don’t be afraid to use behavioral helper code

– There’s no need to capture everything as a single complex assertion

– Lowers the bar for adoption

– Formal verification has applications beyond pure RTL

– STA constraints capture design intent

– A primary input to the back end

– Often little to no verification beyond loading them into tool

– Formal verification run times can be much quicker than dynamic simulation

– A boon for design discovery and understanding

– “Time to bug” can be significantly faster then dynamic simulation

– Good for continuous integration/sanity tests



• Ultimately:

“Hard to imagine a modern flow describing itself as robust that does not have a Formal verification

element.”



One Year Later


• Continue to use Formal verification – An enthusiast rather than an expert

– Even on iterative/delta projects

– Making light of the more prosaic verification tasks

– Reverse connectivity for the win!

• Seeing it become a standard weapon in the arsenal of the verification engineer

• Speaker at JUG 2015 in San Jose – Saw some very advanced applications of Formal techniques

– Using tools to bug hunt not prove using specialised engines*

– Use cover statements to guide engine

– typically 10x wrt to properties

– Easy to believe you must strive to be at this level

– This advanced usage often requires a dedicated Formal verification engineer

– Significant gains to be made with more modest levels of expertise …

*I still assert that I shouldn’t care about engines outside of these specialised use cases

One Year Later


故善战者，求之于势，不责于人

“The expert in battle seeks his victory from strategic advantage and does not

demand it from his men”

Sun Tsu – The Art of War


“The expert in verification seeks zero bugs from best design practice and DfV and does not compel their team

verify harder”

Alex Orr – JUG 2015


“Put less bugs in the implementation for the verification team to find”

(paraphrasing even more)


• Shift left/start earlier/share the love

• Hand over better quality RTL – Improve the state of the RTL before it gets to the verification team

– Ideally the only bugs found will be diversions from the specification not implementation errors

• Find implementation bugs before 0ns – Formal doesn’t require a testbench

• Liberal use of assertions and covers – If you can make an assertion about a line of code

– capture it and prove it formally, before you simulate anything

– Put covers on your decodes and transitions

– Look for coverage holes before you fall into them!

• Use the mathematical rigor of the tool to your advantage – Let it do the de-morgans

– Let it highlight unexpected behaviours and side-effects

– Like having a omniscient minor deity at your command

Empower your RTL Engineers


• Use covers and assertions to explore your inherited design – It is the expert in the implementation

– Use assumptions/what-if analysis to understand implementation motivations

• It is easy to break the testbench *improving* it – You can add as many assertions as you like!

• Missing coverage on legacy blocks – Prove you can get there with covers

– Separate coverage item makes it easy to close ticket

• Use assertions to stimulate *features* found in silicon – Prove you’ve addressed them without re-writing the testbench/test suite if you are time limited

Building on legacy


• Use Formal to prove STA constraints – False and multicycle paths

– Often only reviewed by designer

– Are an assertion about the temporal design intent

– One of the reasons we still run SDF simulations

– Be part of the solution not the problem!

– Assertions will fire in RTL if modified

– detect and fix early

• Use Formal connectivity checks to prove DfT structures – Often don’t exist in RTL

– Quite often captured in csv format anyway!

– DfT engineers may not have skills to author UVM testcases

• Use assertions to prove test modes won’t wreak havoc on your IP

Silicon Backend


• Easy to report junit format for Jenkins – Speed of execution enables effective continuous integration

• Add a formal element to your sanity test – Likely to be first thing to fire

• Advanced flows such as UNR can massively help closure efforts

Automation


Conclusions


• I find myself using Formal verification more and more – Often for less glamorous tasks than full automated proofs …

– … just like my verification tasks!

– Formal makes it easier to address deltas and small changes

– Less destructive than extending the testbench

• Encourage the use of Formal earlier in the flow (shift left) – Improves implementation quality and therefore makes verification easier

– Can concentrate on debugging divergences from specification

– Less time spent debugging implementation mistakes

– Reduces the reliance on SDF sims

• Plenty of scope for use further down the flow – Prove you STA constraints

– Verify your test structures

Conclusions


• Ultimately the best thing I can do to reduce bugs and make my life better …

…is to help the RTL engineer put less of them in the design in the first place!

• To that extent Broadcom trains both the RTL and verification teams in Formal

Conclusions

19 © 2016 Broadcom. All Rights Reserved. "Broadcom" refers to Broadcom Limited and/or its subsidiaries. | © 2016 Broadcom. All Rights Reserved. "Broadcom" refers to Broadcom Limited and/or its subsidiaries.

Thank You

better living through formal - test and verification ... · •add a formal element to your sanity...

Documents