test and validation studies of mathematical software libraries
DESCRIPTION
Test and Validation Studies of Mathematical Software Libraries. A summary of my work as a technical student at CERN. LCG AA Meeting, 22. September 2004. Special Functions. Comparison of numerical results Performance GSL-NAG-C and GSL-TMath Bessel functions - PowerPoint PPT PresentationTRANSCRIPT
M. Hatlo / CERN 1
Test and Validation Studies of Mathematical Software Libraries
A summary of my work as a technical student at CERN
LCG AA Meeting, 22. September 2004
M. Hatlo / CERN 2
Special Functions
• Comparison of numerical results
• Performance
• GSL-NAG-C and GSL-TMath
• Bessel functions
• Gamma, Logarithm of Gamma, Error function and Complementary Error function
M. Hatlo / CERN 3
Results
• GSL performed very well compared to NAG-C– Difference usually less than the estimated
error
• Bigger differences between GSL and TMath
• Little difference in time for GSL
• NAG-C and TMath are faster
M. Hatlo / CERN 4
M. Hatlo / CERN 5
M. Hatlo / CERN 6
Timing of Special Functions
• 100 000 function calls for Bessel I0, I1, J0 and J1
• 50 000 function calls for the rest
M. Hatlo / CERN 7
Distributions
• Comparison of numerical results
• Performance in the evaluation
• Generation of random numbers according to distribution– Comparison and Kolmogorov-Smirnov test
• Normal distribution, Landau distribution, Gamma distribution, Poisson distribution and Chi-Square distribution
M. Hatlo / CERN 8
M. Hatlo / CERN 9
Timing Results for some Distributions
• 1 000 000 function calls
M. Hatlo / CERN 10
Random Numbers According to Distribution
M. Hatlo / CERN 11
Random Numbers
• Two tests– Frequency test– Point test
• Main generators from GSL– gsl_rng_mt19937– gsl_rng_cmrg– gsl_rng_mrg– gsl_rng_taus– gsl_rng_taus2– gsl_rng_gfsr4– gsl_rng_ranlux389– gsl_rng_ranlux– gsl_rng_ranlxd2
M. Hatlo / CERN 12
Frequency Test
Fill space in d dimensions with points formed from a sequence of random numbers.
Look in a small volume and the frequency as the number of bins which maximize |Nodd-Neven|.
gsl_rng_minstd
M. Hatlo / CERN 13
Frequency Test
• With this frequency, look other places in the space and compute Nodd.
• Nodd should be normal distributed
M. Hatlo / CERN 14
Results
• 10 results for Nodd
• Kolmogorov-Smirnov test• Taus, 8 dim and Ranlux389, 6 dim• New test for poor results• All passed
M. Hatlo / CERN 15
Point Test
• Arrange a sequence of random numbers into multidimensional points
• Define distance between two points as
• Find all points Pi that are closer to P1 than the mean-n*standard deviation (n=3,4,5)
• Calculate the distance between Pi+1 and P2
),...,( )()1( diii ppP
)()()()(
1
1
1,min
,
kj
ki
kj
ki
dkk
d
kkji
ppppdist
distPP
M. Hatlo / CERN 16
Point test
• For the distance should be normal distributed.• Use Kolmogorov-Smirnov test • All generators
pass
3d
M. Hatlo / CERN 17
Numerical Integration
• Wrapper for existing gsl algorithms
• Tested on a few number of integrals– Compare numerical results with analytical
results
• No difference larger than 10-7 (input tolerance), but need further testing
M. Hatlo / CERN 18
Integrals
1
02
02222
0
0
1
1
02
2/
1
0
)1(
11log
cossin
)1(
)sin(1(*)
271
)2
(2
1(*)
0.4log
(*)
2
pdxx
x
abxbxa
dx
xx
dx
pdxx
x
xx
dx
xerfdte
dxx
x
p
p
x
x
t
BackgroundeakLorenzianp
dxx
x
dxe
x
dxe
x
dxe
dxex
Cdxxx
x
x
x
x
(*)
2
sin
15
6
2
2
...5772157.01
1
log
1
0
0
4
1
3
0
2
1
0
0
1
0
2
M. Hatlo / CERN 19
Performance in Numerical Integration
• Quadrature routines• QAG – adaptive
integration• QAGUI – adaptive
integration from zero to infinity
• QAGS – adaptive integration with singularities
• QNG – non-adaptive Gauss-Kronrod integration
• NB! Different integrals are used, marked with (*) on last slide
M. Hatlo / CERN 20
Linear Algebra
• E. Myklebust summer student 2003– A Comparative study of Numerical Linear Algebra
Libraries• Particle Track Reconstruction, Kalman filter
update equations– Multiplication, addition, inversion and transpose
• Originally 2x2, 2x5, 5x2 and 5x5• Extended to bigger matrices
– 2x5, 4x10, 10x25 and 20x50• CLHEP, uBLAS, LAPACK, GSL and ROOT• Used timer from SEAL base
M. Hatlo / CERN 21
Results (Linux, P4 1.8 MHz )
• High RMS for GSL and LAPACK in 2x5• Error with ROOT for 20x50
M. Hatlo / CERN 22
Conclusions
• GSL performs reasonably good• Both tests of randomness were passed by all the
main generators from GSL• More testing is needed for the numerical
integration• All test programs are in the SEAL cvs repository • A test suite can be easily created and
automatically run for new SEAL releases • A written report of my work will be put on the
webpage when finished
M. Hatlo / CERN 23
Results
UBLAS 1 UBLAS 2 UBLAS 3 LAPACK CLHEP GSL ROOT
2x50.0000291446
(1.59)0.000036635
(2.00)0.000034575
5 (1.89)0.0000262477
(1.44)0.0000254471
1.39)0.0000315174
(1.72)0.000018277
(1.00)
4x100.000122262
(6.69)0.000128379
(7.02) 0.00004496
(2.46)0.0000558439
(3.06)0.0000591721
(3.24)0.0000387867
(2.12)
10x250.00162649
(89.0) 0.000313892
(17.2)0.000559981
(30.6)0.000401781
(22.0)0.000277101
(15.2)
20x500.0148939
(815) 0.00125005
(68.4)0.00302708
(166)0.0014349
(78.5)