ibm verification seminar october 2003 - laurent fournier
TRANSCRIPT
![Page 1: IBM Verification Seminar October 2003 - Laurent Fournier](https://reader031.vdocument.in/reader031/viewer/2022012513/618c98b605398877bd66260e/html5/thumbnails/1.jpg)
IBM Labs in HaifaIBM Labs in Haifa
Deep Knowledge Test Generators & Functional Verification Methodology
IBM Verification SeminarOctober 2003 - Laurent Fournier
![Page 2: IBM Verification Seminar October 2003 - Laurent Fournier](https://reader031.vdocument.in/reader031/viewer/2022012513/618c98b605398877bd66260e/html5/thumbnails/2.jpg)
IBM Labs in Haifa
IBM Labs in Haifa
Fetch unit
Decode unit
Completion unit
Dispatch unit0 1 32
All Starting Stages in Holding Position
![Page 3: IBM Verification Seminar October 2003 - Laurent Fournier](https://reader031.vdocument.in/reader031/viewer/2022012513/618c98b605398877bd66260e/html5/thumbnails/3.jpg)
IBM Labs in Haifa
IBM Labs in Haifa
Extreme Case of Influence on Sticky Bit
1.00111011000111110001110000001110001000111
Rounder
1.00111011000111110010 I
1.cccccccccccccccccccccc000000000000000000001
![Page 4: IBM Verification Seminar October 2003 - Laurent Fournier](https://reader031.vdocument.in/reader031/viewer/2022012513/618c98b605398877bd66260e/html5/thumbnails/4.jpg)
IBM Labs in Haifa
IBM Labs in Haifa
Motivation for Deep Knowledge Test Generation
Control of test generator Interesting scenarios
+ Bug-ProneGap
![Page 5: IBM Verification Seminar October 2003 - Laurent Fournier](https://reader031.vdocument.in/reader031/viewer/2022012513/618c98b605398877bd66260e/html5/thumbnails/5.jpg)
IBM Labs in Haifa
IBM Labs in Haifa
Deep Knowledge Test Generator?
Definition� Test generation focused on specific verification areas, e.g. FPU,
Microarchitecture Flow, SCUProblem� Inefficiency of generic architecture tools in specific error-prone
verification areas (bugs not found or found late)Objectives� To provide greater control to reach non-covered areas� To enable a systematic and comprehensive verification approach to
speed up the rate of coverage
![Page 6: IBM Verification Seminar October 2003 - Laurent Fournier](https://reader031.vdocument.in/reader031/viewer/2022012513/618c98b605398877bd66260e/html5/thumbnails/6.jpg)
IBM Labs in Haifa
IBM Labs in Haifa
The Players
FPgen� Generic, quasi-optimal solution for a well-
defined, albeit complex, field� Complex mathematic algorithms
Piparazzi� An evolving solution to cope with
microarchitecture flow complexity� Classic constraint solving engine (CSP)
DeepTrans� More a library of services for now� Rounds off the model-based technology
![Page 7: IBM Verification Seminar October 2003 - Laurent Fournier](https://reader031.vdocument.in/reader031/viewer/2022012513/618c98b605398877bd66260e/html5/thumbnails/7.jpg)
IBM Labs in Haifa
IBM Labs in Haifa
General bug curve
time
number of bugs
time
FP bug curve
number of bugs
Motivation: Floating Point Bugs
![Page 8: IBM Verification Seminar October 2003 - Laurent Fournier](https://reader031.vdocument.in/reader031/viewer/2022012513/618c98b605398877bd66260e/html5/thumbnails/8.jpg)
IBM Labs in Haifa
IBM Labs in Haifa
Basic Functionality
DIV
OP1 Result
[FP1, PF2] 0X1X001X10Denormal
�
Max 4 bits set001000101010010011010101001110
OP2
100011000011010111001100100010
Random solution:Uniform distribution soughtMultiple random solution:Control of coverage density
![Page 9: IBM Verification Seminar October 2003 - Laurent Fournier](https://reader031.vdocument.in/reader031/viewer/2022012513/618c98b605398877bd66260e/html5/thumbnails/9.jpg)
IBM Labs in Haifa
IBM Labs in Haifa
Piparazzi – The Main Concept
Solver
A test program for Micro-processor
User’s request file
Micro-architectureModel
Fetch
Dispatch
Decode
iop[1].PIPELINE = LSU
iop[0].PIPELINE = FXU
Iop[0].stage[STAGE_2].STALL > 0
iop[1].DISPATCH = iop[0].DISPATCH
![Page 10: IBM Verification Seminar October 2003 - Laurent Fournier](https://reader031.vdocument.in/reader031/viewer/2022012513/618c98b605398877bd66260e/html5/thumbnails/10.jpg)
IBM Labs in Haifa
IBM Labs in Haifa
Input can be a Full Cross-Product
Example: All types model
+/- Infinity+/- Zero+/- Norm
+/- Denorm+/- Large number+/- Small number+/- Min Denorm+/- Max Denorm
+/- Min Norm+/- Max Norm
Operand1
+/- Infinity+/- Zero+/- Norm
+/- Denorm+/- Large number+/- Small number+/- Min Denorm+/- Max Denorm
+/- Min Norm+/- Max Norm
Operand2
+/- Infinity+/- Zero+/- Norm
+/- Denorm+/- Large number+/- Small number+/- Min Denorm+/- Max Denorm
+/- Min Norm+/- Max Norm
ResultMultiply =
![Page 11: IBM Verification Seminar October 2003 - Laurent Fournier](https://reader031.vdocument.in/reader031/viewer/2022012513/618c98b605398877bd66260e/html5/thumbnails/11.jpg)
IBM Labs in Haifa
IBM Labs in Haifa
Relationship between Generator Input Language and Test-plan
"Language shapes the way we think,and determines what we can think about" - B.L.Whorf
FPgen Test-Plan
![Page 12: IBM Verification Seminar October 2003 - Laurent Fournier](https://reader031.vdocument.in/reader031/viewer/2022012513/618c98b605398877bd66260e/html5/thumbnails/12.jpg)
IBM Labs in Haifa
IBM Labs in Haifa
Fetch unit
Decode unit
Completion unit
Dispatch unit0 1 32
Generalized PiparazziExample
![Page 13: IBM Verification Seminar October 2003 - Laurent Fournier](https://reader031.vdocument.in/reader031/viewer/2022012513/618c98b605398877bd66260e/html5/thumbnails/13.jpg)
IBM Labs in Haifa
IBM Labs in Haifa
Generalized FPgen Example
1.00111011000111110001110000001110001000111
Rounder
1.00111011000111110010 I
1.cccccccccccccccccccccc000000000000000000001
1.cccccccccccccccccccccc000000000000000000010
1.cccccccccccccccccccccc100000000000000000000
…
![Page 14: IBM Verification Seminar October 2003 - Laurent Fournier](https://reader031.vdocument.in/reader031/viewer/2022012513/618c98b605398877bd66260e/html5/thumbnails/14.jpg)
IBM Labs in Haifa
IBM Labs in Haifa
Generic Test Plans
� Methodology – Resource Dependent�Density�Crossing�Reduction�Huge models
Dsasjfdlhfkfjs dfajdlkfj lkDsafk l;kf;daslkf; aslkfa; Sadglk;flkgfdlkDsafkl kf;das lkf;aslkfa; Sadglk;flkgfdlkDsafkl;kf;d aslkf;aslkfa;Sadglk;flkgfdlk Dsasjfdlhfkfjs dfajdlk j lkDsafk l;kf;daslkf; aslkfa; Sadglk;flkgfdlkDsafkl;k f;daslkf;aslkfa;Sad glk; flkgfdlk fkl;kf;d aslkf;aslk fa;Sadglk; flkg fdlk
Dsasjfdlhfkfjs dfajdlkfj lkDsafk l;kf;daslkf; aslkfa; Sadglk;flkgfdlkDsafkl kf;das lkf;aslkfa; Sadglk;flkgfdlkDsafkl;kf;d aslkf;aslkfa;Sadglk;flkgfdlk Dsasjfdlhfkfjs dfajdlk j lkDsafk l;kf;daslkf; aslkfa; Sadglk;flkgfdlkDsafkl;k f;daslkf;aslkfa;Sad glk; flkgfdlk fkl;kf;d aslkf;aslk fa;Sadglk; flkg fdlk
![Page 15: IBM Verification Seminar October 2003 - Laurent Fournier](https://reader031.vdocument.in/reader031/viewer/2022012513/618c98b605398877bd66260e/html5/thumbnails/15.jpg)
IBM Labs in Haifa
IBM Labs in Haifa
Test-Plan
IEEE standardIEEE test suites Bug analysis
�������
��������
� � � ���
Papers
Test plans from users
Ingredients for Test-Plan
Generic &alternativeImplementations
![Page 16: IBM Verification Seminar October 2003 - Laurent Fournier](https://reader031.vdocument.in/reader031/viewer/2022012513/618c98b605398877bd66260e/html5/thumbnails/16.jpg)
IBM Labs in Haifa
IBM Labs in Haifa
The Evolution of Functional Verification Methodology
Small design: manual to reach all suspected cases
Slow and tedious
![Page 17: IBM Verification Seminar October 2003 - Laurent Fournier](https://reader031.vdocument.in/reader031/viewer/2022012513/618c98b605398877bd66260e/html5/thumbnails/17.jpg)
IBM Labs in Haifa
IBM Labs in Haifa
The Evolution of Functional Verification MethodologyIncreased complexity: Random generators
No uniformity in bug location probability
![Page 18: IBM Verification Seminar October 2003 - Laurent Fournier](https://reader031.vdocument.in/reader031/viewer/2022012513/618c98b605398877bd66260e/html5/thumbnails/18.jpg)
IBM Labs in Haifa
IBM Labs in Haifa
The Evolution of Functional Verification Methodology
Biased Random generator to control towards suspected areas
Faster and more effective
![Page 19: IBM Verification Seminar October 2003 - Laurent Fournier](https://reader031.vdocument.in/reader031/viewer/2022012513/618c98b605398877bd66260e/html5/thumbnails/19.jpg)
IBM Labs in Haifa
IBM Labs in Haifa
The Evolution of Functional Verification Methodology
Deep Knowledge Test Generators
Many fast accesses to areas with a high probability of bugs
![Page 20: IBM Verification Seminar October 2003 - Laurent Fournier](https://reader031.vdocument.in/reader031/viewer/2022012513/618c98b605398877bd66260e/html5/thumbnails/20.jpg)
IBM Labs in Haifa
IBM Labs in Haifa
Coverage by Generation
Type task
Type task
Type task
Type task
Type task
Jan Feb Mar Apr May Jun J ul Aug Sep Oct Nov Dec
Def-Files
Writes general Def-files
Writes specific Def-files to cover missing items
Coverage by feedback:
1. fourscore and
2. seven years ago
3. our fathers brought
4. forth upon this
5. continent a new
6. nation conceived in
7. liberty
Test Plan
Coverage by generation:
Writes Def-files directly asking for model
Comprehensive test generators(Gpro, X-Gen)
DKTGs(FPGen, Piparazzi)
Coverage tools (Meteor)
![Page 21: IBM Verification Seminar October 2003 - Laurent Fournier](https://reader031.vdocument.in/reader031/viewer/2022012513/618c98b605398877bd66260e/html5/thumbnails/21.jpg)
IBM Labs in Haifa
IBM Labs in Haifa
Control
Randomness
Broad approach TGGPro, AVPGen
DKTG
Manual testing
Speed DKTG�Performance
![Page 22: IBM Verification Seminar October 2003 - Laurent Fournier](https://reader031.vdocument.in/reader031/viewer/2022012513/618c98b605398877bd66260e/html5/thumbnails/22.jpg)
IBM Labs in Haifa
IBM Labs in Haifa
Coverage Graph
Generation method
Time
GPro
DKTG100%
# eventstested
![Page 23: IBM Verification Seminar October 2003 - Laurent Fournier](https://reader031.vdocument.in/reader031/viewer/2022012513/618c98b605398877bd66260e/html5/thumbnails/23.jpg)
IBM Labs in Haifa
IBM Labs in Haifa
Cross-Product Approach
� Bugs often lie in the interaction of several factors.
� This approach is more than a list of disparate tasks. It may include, often inadvertently, many “quasi corner cases”.
� All types model.
� Some cases are clearly corner cases, but others, while interesting, might have been overlooked.
![Page 24: IBM Verification Seminar October 2003 - Laurent Fournier](https://reader031.vdocument.in/reader031/viewer/2022012513/618c98b605398877bd66260e/html5/thumbnails/24.jpg)
IBM Labs in Haifa
IBM Labs in Haifa
Quasi Corner Cases
More interestingLess interesting
Corner casesStraightforward cases
Quasi corner cases
![Page 25: IBM Verification Seminar October 2003 - Laurent Fournier](https://reader031.vdocument.in/reader031/viewer/2022012513/618c98b605398877bd66260e/html5/thumbnails/25.jpg)
IBM Labs in Haifa
IBM Labs in Haifa
The Non-Uniform View
![Page 26: IBM Verification Seminar October 2003 - Laurent Fournier](https://reader031.vdocument.in/reader031/viewer/2022012513/618c98b605398877bd66260e/html5/thumbnails/26.jpg)
IBM Labs in Haifa
IBM Labs in Haifa
Pitfalls
� Quantity at the price of quality � Easy to create large, not meaningful event spaces
� More events than can be covered (waste of verification bandwidth)
� Leads people to be “thought-lazy” because easy to generate impressive test-plan (quantity-wise)
� Blindly rely on “nice” coverage numbers� Includes many events that require significant effort in
knowing whether or not they are reachable(double-edged sword)
![Page 27: IBM Verification Seminar October 2003 - Laurent Fournier](https://reader031.vdocument.in/reader031/viewer/2022012513/618c98b605398877bd66260e/html5/thumbnails/27.jpg)
IBM Labs in Haifa
IBM Labs in Haifa
Conclusion: The Test-Plan Feedback Loop
Test plan
DKTG
Generic test plan
Updated tool development
![Page 28: IBM Verification Seminar October 2003 - Laurent Fournier](https://reader031.vdocument.in/reader031/viewer/2022012513/618c98b605398877bd66260e/html5/thumbnails/28.jpg)
IBM Labs in Haifa
IBM Labs in Haifa
Knowledge & Control
Test Plan Scope
DKTG
Existing VerificationMeans
Conclusion: DKTG��Impact
![Page 29: IBM Verification Seminar October 2003 - Laurent Fournier](https://reader031.vdocument.in/reader031/viewer/2022012513/618c98b605398877bd66260e/html5/thumbnails/29.jpg)
IBM Labs in Haifa
IBM Labs in Haifa
Extended control:Randomness
vs.specificGeneric
Technology Perspective
RTPG GenesysMBTG
Genesys-ProGenie
DKTG
1994 200119981988
Manual testing
ProductivitySlow, tedious, small design
Core generation
CSP Solvers
Focused approach
Coverage-based generation
Bug prone area
Randomness Model based
Cross-Products