nag presentation to eduserv maths and stats software dec2014

Experts in numerical algorithms and HPC services

John Holden

Louise Mitchell

Host: Howard Moody, eduserv

3rd December 2014

Maths and Stats Working Group

2Trusted experts in HPC & software innovation | professional services, training & impartial advice | Results Matter. Trust NAG.

“Howard, what’s the agenda?”

The content is reasonably technical, perhaps some new features, what’s coming in the development pipeline etc. I would also like to get some feedback from the members of the group as to any changes they would like when we renew the NAG Chest Agreement next summer.


AGENDA

Who are NAG?

Current agreement summary & highlights

Product highlights

“feedback from the members of the group”

4

NAG Profile

• Academia, Government Research

• Software companies (finance, analytics, research), Investment banks, aerospace, engineering

• Energy, manufacturing, consumer analytics

• HPC support - England, Wales, South Africa, US, Middle East

Customers

• US (Chicago, Houston)

• UK (Oxford, Manchester)

• Japan (Tokyo)

• Staff in Germany & France

Offices

• Started 1970 from five British universities & a national lab

• Commercial, Not-for-profit (no shareholders)

• ~60 staff, >50% technical, 20 PhDs in various disciplinesOrigins

5

What we do - today

• Mathematical, statistical, data analysis components for developers in libraries

• Bespoke (custom) methodsAlgorithms

• Scaling and performance tuning of user/ commercial applications

• Specialty performance libraries

• Specialty ports of libraries

Numerical Software

Engineering

• Procurement – planning, RFPs, benchmarks, acceptance

• Computational science & engineering for user applications

HPC

6

What we continue to do

•Mathematical & statistical components for

•C/C++, Fortran, MATLAB, Python, .NET, Java, etc.

•A new release annually with 100+ new methods

Numerical Libraries

•Tuned ports for hardware vendors of NAG Library

•Performance libraries for various chip architectures

Specialty Libraries

•Training courses and code tuning for various accelerator architectures (CUDA, OpenCL, etc)

•Scaling and performance tuning

•Support for various open source packages

Specialist Services

Whilst collaborating with a wide variety of academic partners


Current Product Portfolio

Numerical Libraries

NAG Fortran Library

NAG C Library

NAG Toolbox for MATLAB

NAG Library for .NET

NAG Library for Python (new)

NAG Library for Java (new)

NAG Data Mining Components

NAG HPC Library

NAG Library for SMP & Multi-core (Fortran)

NAG Library for Xeon Phi (Fortran) (new)

NAG Fortran Compiler inc. Fortran Builder - GUI based Compiler for Windows


NAG Library and Toolbox Contents

Root Finding

Summation of Series

Quadrature

Ordinary Differential Equations

Partial Differential Equations

Numerical Differentiation

Integral Equations

Mesh Generation

Interpolation

Curve and Surface Fitting

Optimization

Approximations of Special Functions

Dense Linear Algebra

Sparse Linear Algebra

Correlation & Regression Analysis

Multivariate Methods

Analysis of Variance

Random Number Generators

Univariate Estimation

Nonparametric Statistics

Smoothing in Statistics

Contingency Table Analysis

Survival Analysis

Time Series Analysis

Operations Research


NAG Libraries - Ease of integration

Supporting Wide Range of Operating systems…

Windows, Linux, Mac, …

…and a number of interfaces

C, C++,

Fortran,

VB, Excel & VBA,

C#, F#, VB.NET,

CUDA, OpenCL,

Java,

Python

…

Excel,

LabVIEW,

MATLAB,

Maple,

Mathematica

R, S-Plus,

Scilab, Octave

…


AGENDA

Who are NAG?


Product highlights



Current Agreement

Runs 1st August 2010 – 31st July 2015

Simplified licensing model Smaller number of product categories

Changed to Operating System based from Implementation based

Dept. or site licence

Generous and flexible terms Staff and students may install on personal machines as well as University

owned hardware such as HPC clusters, classroom work stations

Free product training


Current Agreement

Improved and more flexible licence management Changed from various mechanisms (FLEXlm and other) to KUSARI

Annual “Open” Keys are the most popular and flexible – provided on trust basis

Simplifies Image Roll out

Reduces headaches of server licences (sometimes caused by NAT routers, changing IP address)

Admin. burden lightened for site representatives* Software issued direct to users, technical support direct to users (staff

and students)

*subject to their agreement


Current Agreement – “user delight”

The “best” Toolbox for MATLAB and you get it from NAG……………

“The NAG Library range of implementations and interfaces such as VBA, MATLAB, Python allow fast transition between prototype on the desktop to production C or Fortran code on HPC facilities.”

“NAG’s SMP Libraries and NAG Toolbox

for MATLAB (also SMP enabled) allow me to do HPC on desktop before switching to the HPC machine for further speed up with. NAG’s SMP offering means sometimes MPI coding is not needed”

“A fraction of the price of other equivalent math site licences & delivering far more value & flexibility”


Current Agreement – “user delight”

NAG listen to academic user requests… “Closed Form” Options Pricing functions were added to the NAG Library to facilitate

better teaching of financial mathematics for a number of MSc programmes

Non-negative least squares solver added to the NAG Library for a research project

…

The NAG Fortran Compiler

Perfect for teaching as well as research Great licensing terms…. students can use on their own machines

Portable (Windows, Mac, Linux & other UNIX)

and

keeps pace with the latest Fortran Features

supports OpenMP

EXCELLENT (the world’s best) checking compiler

Windows version has a great GUI


Current Agreement – “partner delight”

Many active collaborations and contributions from UK academia

Nick Higham, UoM – NCM &

Matrix Functions

Cathyrn Powell, UoM – Gaussian

Random Field generators

Rebecca Killick, Lancs

– Change Point Analysis

William Shaw, UCL – Quantiles

Michal Kocvara, Birmingham – Optimisation

Oleg Davydov, Strathclyde – Splines & Wavelets Radial Basis functions


Current Agreement - frustration

With NAG being a not-for-profit organisation with its roots and collaboration relationships firmly established with the academic community we continue to provide our software on a “maximum usage” rather than “maximum revenue” basis

this is reflected in the current eduserv agreement Some Universities “cross charge” or enforce stricter terms on their users

preventing licences being used in the intended spirit of the agreement

We recognise there are only a few exceptions, some examples: Although the licence allows installation on personal machines the site rep has

blocked this arrangement

Site licence only funded by two or three depts. and an unrealistic cross charge scheme for individual users from other depts. meaning additional users are deterred

Possible solutions? Perhaps there isn’t one…


Current Agreement - observation

Most (not all) student and staff personal machines are Win or Mac o/s

Most HPC centres are Linux (or other UNIX) The Universities who have been most successful in creating “student & staff buy in” for

encouraging good Mathematical Programming and broader usage of HPC have adopted Linux and Windows and/or Mac o/s licence agreements

“If Universities are serious about encouraging wider adopt of HPC and programming they need to make software and tools available to user in an environment they are familiar with such as Windows and Mac o/s”

from leading scientist at Software Sustainability Institute

Universities who have taken this approach include (but not limited to)... Birmingham, Heriot-Watt, Manchester, Warwick,…


Current Agreement - training

Free to all Universities who hold a Dept or Site Licence

At your University or NAG Offices (Oxford or Manchester)

Example training courses

Using NAG Toolbox for MATLAB

Using the NAG Library with Excel/VBA

Using the NAG C Library with C & C++

Using the NAG Library for Python

OpenMP Demystified featuring the NAG SMP Library

How to write Portable, Bug-Free Fortran using the NAG Compiler

..

…


AGENDA

Who are NAG?


Product highlights



What’s new and coming soon Updates

NAG Libraries (inc. Toolbox for MATLAB) at Mark 24

NAG Library for .NET at release 2.0

NAG Fortran Compiler at release 6.0

New NAG Library for Java released

NAG Library for Python released

NAG Library for Xeon Phi released

Coming soon NAG C Library for SMP & Multi-core

Mark 25 – will be released in 2015

Implementations for ARM hardware

HPC Services


NAG Library, What’s new at Mark 24

Confluent Hypergeometric Function (1F1, 2F1)

Two-stage spline approximation to scattered data

Further additions to Nearest Correlation Matrix (Mark 24)

More Multi-start optimization

More Matrix Functions

Inhomogeneous time series

Gaussian mixture model

Partial Least Squares / Ridge Regression

Quantiles

Search routines

22Trusted experts in HPC & software innovation | professional services, training & impartial advice | Results Matter. Trust NAG.22

The NAG Library for Java

Offers easy access to NAG routines from Java.

History

It’s not too difficult to call the NAG Fortran Library from Java. But most Java developers don't have time to write the code to do this.

Now there is a full set of Java wrappers for NAG routines –at no cost to the users. (Users need to have a licence for the NAG Fortran Library)

Documentation for NAG Library and Users’ Note for Java wrappers

*Users need to licence and install the NAG Fortran Library

Skip

23

1 Java wrapper = 2 parts (2 files in one Zip):

1 Java class

1 C/JNI source file

User's Java AppNAG Java

Wrapper's class

NAG C/JNI

function

NAG Fortran

Library

User's callback

To use the NAG Library for Java

The design


The components

Need to install the NAG Fortran Library (MK 24)

Also download the Java wrappersSingle zip from NAG Includes 2 parts: Java class and C/JNI source file

The NAGLibraryFromJava.jar file contains all NAG Java classes

a shared library (a DLL file for windows / a .so file for Linux) the C/JNI interface to NAG Library

Usage A Java package under com.nag.routines is a single NAG chapter

A Java class is a single NAG routine

Other Notes: Arguments are Java basic types, except for complex numbers;

Complex numbers are represented by a Java interface to implement;

Arguments are attributes of the routine class;

Callback functions are Java interfaces to implement (force the function signature but leave the storage definition to the user)

25

import com.nag.routines.e04.E04GB;

[….]

public static main(String[ ] args) {

[…]

LSQFUN lsqfun = new LSQFUN();

[...]

E04GB e04gb = new E04GB(m, n, lsqlin, lsqfun, lsqmon, iprint, maxcal, eta, xtol, stepmx, x,

fsumq, fvec, fjac, ldfjac, s, v, ldv, niter, nf, iw, liw, w, lw, ifail);

[...]

e04gb.eval();

ifail = e04gb.getIFAIL();

[...] + Exception handling

}

private static class LSQLIN implements E04GBJ.E04GBJ_LSQLIN {

[...]

public void eval(int SELCT) {

E04HEV e04hev = new E04HEV(SELCT);

e04hev.eval();

this.SELCT = e04hev.getLSQLIN_SELECT();

}

}

For the Java developer:

The developer’s view

Import routine as a

Java ‘class’

Create an ‘object’

of the Eo4GB class

Arguments are Java

basic types.(in most cases)


Full set of bindings available for NAG C Library

Supported by white papers for calling NAG Fortran or C Library from Python

http://www.nag.co.uk/python.asp

Combines NAG’s high quality numerical routines with the ease of use of Python

Access to NAG routines from Python for quick prototyping

Same high quality NAG routines used in production system (C, Fortran, .NET, Java, …) as used under Python prototype

NAG Library for Python


nag4py

Built on top of NAG C Library + Documentation

1600 NAG functions easily accessible from python

25 examples programs to help users call NAG functions

from nag4py.c05 import c05ayc

from nag4py.util import NagError,Nag_Comm


NAG & Python - Implied Vol Timings example

Method Timing

fsolve + python

fsolve + NAG

nag4py

NAG C

~180 seconds

~15 seconds

~10 seconds

~.29 seconds


Examples of calling the wrapped

function normal_prob for

nag_prob_normal

Uses exception for error handling

Call the nag function nag_prob_normal

Hand written python function to wrap

nag_prod_normal using default arguments,

docstring and doctests

Imports from the nag4py module for

nag_prob_normal

from ctypes import c_double

from nag4py.util import NagError, Nag_LowerTail, Nag_UpperTail, INIT_FAIL

from nag4py.g01 import nag_prob_normal

def normal_prob(x, tail=Nag_LowerTail):

"“”The lower tail probability for the standard Normal distribution>>> normal_prob(1.0)

0.841344746069>>> normal_prob(1.0, tail=Nag_UpperTail)

0.158655253931>>> normal_prob(1.0, tail=1)Exception: Argument tail has an illegal value: 1""“x = c_double(x)fail = NagError()INIT_FAIL(fail)

res = nag_prob_normal(tail,x,fail)if (fail.code == 70):

raise Exception("Argument tail has an illegal value:" , tail)

return res

def main():

print normal_prob(1.0)

print normal_prob(1.0, tail=Nag_UpperTail)

print normal_prob(1.0, tail=1)

if __name__ == "__main__": main()

Python Example


Python is a great, easy to use, powerful language

NAG Library provides high quality numerical routines

We allow you to combine these strengths

Access to NAG routines from Python for quick prototyping

Same high quality NAG routines used in production system (C, Fortran, .NET, Java, …) as used under Python prototype

NAG and Python


NAG for Intel Xeon Phi

Numerical routines for Intel Many Integrated

Cores architecture

Library based on our SMP Library

Native mode: run entirely on MIC

Offload mode: heterogeneous execution on CPU/MIC Automatic and Programmer controlled offload modes supported

*NAG had the first MIC board in UK outside of Intel

32


[HPC] Training Courses

Users

• Introduction to HPC

• Using [Platform/Service] e.g. HECToR

• Transitioning to [Platform/Service]e.g. Cray XE6

• Multicore Awareness

• [Application specific]

Programmers

• Fortran, C, C++

• MPI, OpenMP, CAF, UPC

• Xeon Phi, CUDA, OpenCL

• Parallel I/O

• Debugging, Profilingand Optimisation

• Core Algorithms forHigh Performance Scientific Computing

• Best Practice in HPC Software Development

Management

• What an R&D manager needs to know about HPC

• Procurement

• Benchmarking

• Service Planning & Delivery

• …

Over the last 6 years NAG HPC services have trained >2000 course attendees


AGENDA

Who are NAG?


Product highlights


Experts in numerical algorithms and HPC services

Ways to contact us:

www.nag.co.uk

Technical Support and Help

[email protected]

Account Manager

[email protected]

NAGNews: http://www.nag.co.uk/NAGNews/Index.asp

Twitter:

www.twitter.com/NAGTalk

Blog:

http://blog.nag.com/

LinkedIn: http://www.linkedin.com/e/vgh/2707514/

http://www.nag.co.uk/

mailto:[email protected]

mailto:[email protected]

http://www.nag.co.uk/NAGNews/Index.asp

http://www.twitter.com/NAGTalk

http://blog.nag.com/

http://www.linkedin.com/e/vgh/2707514/


SPARES


NAG Library (24)

1F1(a;b;x) - Confluent Hypergeometric Function (Mark 24)

Also known as Kummer’sfunction M(a,b,x)

This has a wide variety of applications, including CIR processes and pricing Asian options.


NAG Library (24)

Two-stage spline approximation to scattered

data

Computes a spline approximation for curve and surface fitting.

Designed for large data sets: typical real-world input (from, say, geosciences, data mining, or medical imaging) can contain millions of points.

For sufficiently-dense input data consisting of n points, n-linear complexity and memory usage can be attained.


NAG Library (24)

Multi-start Optimization

Uses a local optimization routine with many different start points to address a global optimization problem .

There are 2 routines. E05UC is for an arbitrary smooth function subject to constraints.

E05US for constrained nonlinear least-squares problem.

The user can also get a set of solutions for other minima. Useful when there are other criteria desirable in an acceptable solution. For example a need for a very stable solution that doesn’t change

much if the independent variables are slightly different from their optimal values.


NAG Library (24)

Gaussian mixture model

A mixture model is a segmentation method where each subgroup is modelled by Gaussian (Normal) distribution.

It does not require that an observed data-set should identify the sub-population to which an individual belongs.

Examples of use: modelling buyer behaviour by identifying

different customer groups

the distribution of power loads over a network


NAG Library(24)

Matrix Functions (Mark 24)

Mark 24 contains new routines for estimating matrix function condition numbers.

condition numbers for the exponential, logarithm, sine,

cosine, or hyperbolic sine or cosine of general real and

complex matrices; and

condition numbers for matrix functions corresponding to

user-supplied subroutines, using either user supplied

derivatives or numerical differentiation.

Note: The condition number of a matrix function is a measure

of the sensitivity of the solution to small changes in the input.

Also Mark 24 has routines for computing the action of the

matrix exponential on a vector (also with a reverse

communication interface for use with sparse matrices)


NAG Library (24)

Inhomogeneous time series (Mark 24)

At Mark 24 we have introduced operators, that extract robust information directly from an inhomogeneous time series. The results are essentially independent of minor changes to the sampling mechanism used when collecting the data, for example, changing a number of time stamps or adding or removing a few observations.

Note: An inhomogeneous time series has unevenly spaced data

and standard time series analysis techniques cannot be used.


NAG Library (24)

Further additions to Nearest Correlation Matrix

The algorithm was described in a paper by Qi & Sun with a superior rate of convergence over previous methods. Further improvements, including different iterative solver and a means of preconditioning the linear equations.

Performance enhancements, by better exploiting the structure of the problem and added Normwise weighting on bounds on eigenvalues

Functionality has been enhanced to allow element-wise weighting.


How do you call the functions in Excel?

Excel function

wizard

Enter the

inputs from the

spreadsheet


NAG and Excel

Calling NAG DLLs using VBA NAG provide VB Declaration

Statements and Examples

NAG provide add-ins: Stats & Survival Analysis

And examples: Local volatility, Variance

Gamma, NCM, …

Calling NAG Library for .NET using VSTO

functions with Reverse Communication (useful for Solver replication for example)

Create NAG XLLs

Our libraries are easily accessible from Excel:

//192.156.217.231/windows/users/davidh/Demos/Excel

//192.156.217.231/windows/users/davidh/Demos/Excel

nag presentation to eduserv maths and stats software dec2014

Education

trust nag

matlab nag library

net nag library

python new nag library

nag chest agreement

new release

new features

windows8trusted experts