Computer Generation of Random Vectors from Continuous

Multivariate Distributions

Paola PalmitestaDepartment of Quantitative Methods

University of Siena

Corrado ProvasiDepartment of Statistical Sciences

University of Padova

Abstract

The evolution of statistical inference in the last years has been induced also by thedevelopment of new computational tools which have led both to the solution of classicalstatistical problems and to the implementation of new methods of analysis. In thisframework, a relevant computational tool is given by simulation which leads to dealwith methodological problems not solvable analytically. Simulation is frequently usedfor the generation of variates from known distributions with Monte Carlo methods forthe study of the behavior of statistics for which the sampling distribution is unknown.Moreover, Monte Carlo algorithms have been developed which can be also used to ap-proximate the stochastic integrals, as in bayesian statistical analysis and in optimizationproblems. The application of such algorithms needs the application of advanced com-putational tools and, particularly, computer algebra systems are surely one of the mostsuitable tools for the simulation of complex stochastic models. In this paper we presentthe methods used for the implementation of some procedures which use Monte Carloalgorithms to obtain random vectors from multivariate continuous distributions in acomputer algebra system, specifically Mathematica.Keywords: Random Vector Generators; Continuous Multivariate Distributions; MonteCarlo Algorithms; Algebraic Software.

1 Introduction

The evolution of statistical inference in the last years has been induced also by the develop-ment of new computational tools which have led both to the solution of classical statisticalproblems and to the implementation of new methods of analysis. In literature many exam-ples of the recent interrelation between numeric computation and statistics can be found, asin Thisted (1988), Fishman (1996), Tanner (1996) and Gentle (1998). In this framework, arelevant computational tool consists of simulation which leads to deal with methodologicalproblems not solvable analytically.

Simulation is frequently used for the generation of variates from known distributionswith Monte Carlo methods for the study of the behavior of statistics with unknown sam-pling distribution. Moreover, Monte Carlo algorithms have been developed which can bealso used to approximate the stochastic integrals, as in bayesian statistical analysis and inoptimization problems (for a review see Ripley, 1987).

The application of such algorithms needs the application of advanced computationaltools and, particularly, computer algebraic systems are surely one of the most suitable toolsfor the simulation of complex stochastic models. In statistics, computer algebraic systemshave been used in various contexts; in particular, among others, Heller (1991) shows how

1

computer algebraic systems can be used in a wide variety of statistical problems; Kendall(1988 and 1990) provides some procedures for the symbolic computation of expressionsused in the analysis of the diffusion of euclidean forms; Silverman and Young (1987) usesymbolic computation in order to study some bootstrap distributions and Venables (1985)in order to obtain maximum likelihood estimates in particular cases; Andrews and Stafford(1993) provide some procedures for the symbolic computation of asymptotic expansions instatistics; Provasi (1996) presents some symbolic procedures applicable to compute momentsof order statistics of some frequently used random variables (see also Varian, 1992 and 1996,for applications in econometrics).

In this paper we consider the application of the algebraic software Mathematica (Wol-fram, 1996) to obtain random vectors from continuous multivariate distributions with MonteCarlo methods. In general, the generation of multivariate distributions is not easily imple-mented, because the usual method based on the inverse of the cumulative distributionfunction used with univariate distributions can not be applied. This problem is stated andaddressed in Section 2. The functions built in Mathematica to generate random vectors fromspecific distributions are presented and discussed in Section 3, while in Section 4 we presentsome computational algorithms to generate random vectors from multivariate distributionswith given marginals and correlation matrix. Section 5 contains concluding remarks.

2 Methods for Generating Multivariate Distributions

Consider the random variable X with cumulative distribution function (cdf) FX and assumethat F−1

X (q) = infx : FX(x) ≥ q, 0 < q < 1. The traditional Monte Carlo method isbased on the well-known result that, for every random variable U uniformly distributed on(0, 1], we have that X and F−1

X (U) have the same cdf. As a consequence, a Monte Carlosimulation of a random variable X can be done at first drawing a uniform random numberu from U ∼Uniform(0, 1) and then inverting u by means of x = F−1

X (u).Likewise, a Monte Carlo simulation of p random variables (X1, . . . , Xp) with a depen-

dence structure can start with p uniform random variables (U1, . . . , Up) with the samedependence structure. However, in general the definition of a multivariate distribution withknown marginals and some dependence structure is very complex and sometimes ambigu-ous, therefore the construction of Monte Carlo algorithms for multivariate distributionssatisfying the usual assumptions could often be difficult.

In alternative, a method which converts the problem of the generation of a p-dimensionalrandom vector X = (X1, . . . , Xp)′ into the generation of p univariate random variables canbe applied when the probability density of X can be divided in factors as follows:

fX1,...,Xp(x1, . . . , xp) = fX1(x1)fX2|X1(x2|x1) · · · fXp|X1,...Xp−1

(xp|x1, . . . , xp−1).

Parrish (1990) used this method to generate random vectors from Pearson multivariatedistributions.

When the computation of the conditional distribution of X is difficult, a transformationof the vector can be considered. In this case, X is represented as a function of independentunivariate distributions. Notwithstanding the apparent easiness for applying this method,the search of a particular transformation to generate X could be difficult when X is specifiedby its probability density function fX1,...,Xp .

Other approaches for generating multivariate distributions are based either on the ex-tension to the multivariate case of the rejection method or on iterative methods. Among

2

them, we cite importance sampling, in which sampling points are concentrated in the mostrelevant regions for X rather then in the whole region, and methods based on Markovchains. For the latter methods, the idea is to simulate independent realizations formingan irreducible Markov chain, having as stationary distribution the distribution under study(for a review see Boswell et al., 1993).

Referring mainly on the transformation to independent forms, in the next two sectionswe present the methods we used in the construction of Mathematica functions, in order togenerate random vectors from many multivariate continuous distributions (some of thesemethods can be found in extended form in Johnson (1987)).

3 Generating Specific Multivariate Distributions

As disclosed below, in this section we show the methods we used for the construction ofthe Mathematica functions which lead to the generation of random vectors from specifiedmultivariate distributions. In general, these distributions are an extension to the multivari-ate case of known random variables which are often used in statistical applications withunidimensional samples.

The symbolic form of the Mathematica functions is

RandomArray[distribution[param1,param2, . . .],n]

where distribution is the name of one the distributions in Tab. 1 with the expected para-meters and n is the number of random vectors to be generated. These functions have beenorganized in the package MultiRand1.

Multiuniform Distribution. The density function of the random vector U(p) = (U1, . . . ,Up)′ uniformly distributed on the unit sphere in Rp is given by

fU1,...,Up(u1, . . . , up) =Γ(p/2)πp/2

(1−

p∑i=1

u2i

)−1

,

p∑i=1

u2i < 1, ui > 0, i = 1, . . . , p,

where Γ(·) is the gamma function. A technique for the generation of random vectors fromU(p) is based on the transformation (Muller, 1959)

Ui =Xi

(X1 + · · ·+Xp)1/2

, i = 1, . . . , p,

where X1, . . . , Xp are independent standard normal variates (see Tashiro, 1977, for someothers methods for the generation of random vectors from U(p)).

1The package MultiRand.m can be downloaded as a zipped file at the following URL:

http://turing.dmq.unisi.it/multirand.zip.

In MultiRand.m the generation of variates from univariate distributions is done using the package Continu-ousDistribution.m .

3

Table 1: Specific Multivariate Distributions in the Mathematica package MultiRand.m

MultiuniformDistribution[p] multivariate uniform distribution on the the unit p-sphere.

DiricheletDistribution[alpha] Dirichelet distribution with parameter vector α.MultinormalDistribution

[mu,Sigma]multivariate normal distribution with mean vector µand Σ.

MultivariateChiSquareDistribution[m,Sigma]

multivariate chi square distribution with m degrees offreedom and scale matrix Σ.

MultivariateFDistribution[m,k,Sigma]

multivariate F distribution with (m, k) degrees of free-dom and scale matrix Σ.

MultivariateTDistribution[m,mu,Sigma]

multivariate t distribution with m degrees of freedom,mean vector µ and scale matrix Σ.

MultivariateJohnson Distribu-tion[mu,sigma,a,b,R,option]

multivariate Johnson distribution with mean vector µ,standard deviation vector σ, skewness indices vectora, kurtosis indices vector b and with R that is eithera correlation matrix or a Spearman’s rank correlationmatrix or Kendall’s rank correlation matrix accordingto the value given to the logical variable option.

MultivariateBetaDistribution[alpha,beta]

multivariate beta distribution with parameter vectorsα and β.

MultivariateGammaDistribution[theta,lambda,gamma]

multivariate gamma distribution with shape parametervector θ, scale parameter vector λ and location para-meter vector γ.

MultivariateExtremeValueDistribution[mu,sigma,rho]

multivariate extreme values distribution with positionparameter vector µ, scale parameter vector σ and de-pendence parameter ρ.

MultivariateInverseGaussianDistribution[chi,psi,alpha]

multivariate inverse gaussian distribution with parame-ter vectors χ and α and scalar parameter φ.

MultivariateUniformDistribution[p,alpha]

p-dimensional uniform distribution with scalar parame-ter α.

MultivariateBurrDistribution[c,d,k]

multivariate Burr distribution with parameter vectorsc and d and scalar parameter k.

MultivariateParetoDistribution[theta,alpha]

multivariate Pareto distribution with location parame-ter vector θ and scalar parameter α.

MultivariateLogisticDistribution[p,alpha]

p-dimensional logistic distribution with scalar parame-ter α.

MultivariatePearsonTypeIIDistribution[v,mu,Sigma]

multivariate Pearson Type II distribution with shapeparameter v, mean vector µ and scale matrix Σ.

MultivariatePearsonTypeVIIDistribution[v,mu,Sigma]

multivariate Pearson Type VII distribution with shapeparameter v, mean vector µ and scale matrix Σ.

MultivariatePowerExponentialDistribution[beta,mu,Sigma]

multivariate power exponential distribution with shapeparameter β, mean vector µ and scale matrix Σ.

4

Dirichlet Distribution. Let Y1, . . . , Yp, Yp+1 be independent standard gamma randomvariables with shape parameters αi > 0 for i = 1, . . . , p, p+ 1, and let

Xi =Yi

Y1 + · · ·+ Yp + Yp+1, i = 1, . . . , p. (1)

Then, the random vector X = (X1, . . . , Xp)′ has a Dirichlet distribution with density func-tion given by (cf. Fang, Kotz and Ng, 1990, p. 17)

fX1,...;Xp(x1, . . . , xp) = (Bp(α)−1p+1∏i=1

xαi−1i ,

p+1∑i=1

xi = 1, xi > 0, i = 1, . . . , p, p+ 1,

where Bp(α) = (∏p+1i=1 Γ(αi))/Γ(α), α =

∑p+1i=1 αi and α = (α1, . . . , αp, αp+1)′. Hence, a

technique for the generation of random vectors from a Dirichlet distribution with parametervector α can be based on the transformation (1) (see Narayan, 1990, for some othersmethods for the generation of random vectors from a Dirichlet distribution).

Multinormal and Related Distributions. Let the distribution of the random vectorX = (X1, . . . , Xp)′ be a multivariate normal Np(µ,Σ) with mean vector µ = (µ1, . . . , µp)′

and covariance matrix Σ = (σij). Moreover, let the lower triangular matrix A be obtainedby the Cholesky decomposition AA′ = Σ. Then, given the random vector Z =(Z1, . . . , Zp)′

with independent standard normal components, the simulation of Np(µ,Σ) can be run usingthe transformation X = µ + AZ (Scheuer and Stoller, 1962).

A multivariate distribution with chi-squared marginals can be obtained as a func-tion of multinormal random vectors. In fact, let the independent random vectors Xi =(X1j , . . . , Xpj)′, j = 1, 2, . . . ,m, identically distributed as multivariate normal Np(0,Σ)and let Yi =

∑mj=1X

2ij/σii for i = 1, . . . , p. Then, the joint distribution of (Y1, . . . , Yp)

is characterized by the scale matrix Σ and has chi-squared marginals with m degrees offreedom (cf. Mardia, Kent and Bibby, 1979, p. 67). Moreover, let S2 be a chi-squaredrandom variable with k degrees of freedom independent from Yi and let Fi = k Yi/(m S2)for i = 1, . . . p. The joint distribution of (F1, . . . , Fp) is called multivariate F distributionwith (m, k) degrees of freedom and covariance matrix Σ of the accompanying multivariatenormal distribution (Schuurmann, Krishnaiah and Chattopadhyay, 1975). We can notethat the multivariate chi-squared distribution can be obtained by the main diagonal of theWishart matrix generated with the Bartlett algorithm (Bartlett, 1933).

Also a random vector distributed as a multivariate t distribution can be written as afunction of a multinormal random vector. As a matter of fact, if Z is a multivariate normalNp(0,Σ) and S2 is a chi-squared random variable with m degrees of freedom independentfrom Z, then the joint distribution of (T1, . . . , Tp), where Ti = (

√vZi/S) + µi for i =

1, . . . , p, has a multivariate t distribution with m degrees of freedom, mean vector µ =(µ1, . . . , µp)′ and scale matrix Σ. When Σ = I, where I is the identity matrix, and m = 1,the multivariate Student’s t distribution is equal to the multivariate Cauchy distribution(see Johnson and Kotz, 1972, Cap. 37, for details on the various types of multivariateStudent’s t distribution).

Multivariate Johnson Distribution. A continuous random variable X belongs to oneof the distribution families of the Johnson translation system (1949a) if the transformation

Z = γ + δ g[(X − ξ)/λ]

5

is true, where Z indicates the standard normal distribution, γ ∈ R and δ > 0 are shapeparameters, ξ ∈ R is a location parameter, λ > 0 is a scale parameter, and g(·) is one thefollowing transformations:

g(y) =

ln(y), for the SL (lognormal) family,sinh−1(y), for the SU (unbounded) family,ln[y/(1− y)], for the SB (bounded) family,y, for the SN (normal) family.

(2)

An important characteristic of the Johnson system, useful in Monte Carlo studies, is thatthe family to which X belongs can be unambiguously identified by skewness and kurtosis,as in Hill, Hill and Holder (1974). Moreover, we can obtain random variables from X atfirst by the generation of variates from Z and then applying the inverse translation

X = ξ + λ g−1[(Z − γ)/δ],

where

g−1(z) =

ez, for the SL (lognormal) family,sinh(z), for the SU (unbounded) family,1/(1 + e−z), for the SB (bounded) family,z, for the SN (normal) family.

(3)

Johnson (1949b) has proposed a bivariate distribution based on the univariate transla-tion system which can be easily extended to higher dimensions. Then, the random vectorX = (X1, . . . , Xp)′ with p components has a Johnson multivariate distribution if the trans-formation

Z = γ + δ g[λ−1(X− ξ)] ∼ Np(0,Σ), (4)

is true, where Σ = R = (ρij) is a correlation matrix, g[(y1, . . . , yp)′] = [g1(y1), . . . , gp(yp)]′

identifies the family to which the marginals of the translation belong, γ = (γ1, . . . , γp)′,δ =diag(δ1, . . . , δp), ξ = (ξ1, . . . , ξp)′ and λ =diag(λ1, . . . , λp) are, respectively, the shape,location and scale parameters of the components Xi for i = 1, . . . , p. Therefore, havingdetermined the families of the marginals of X on the basis of the first four moments,the generation of the random vectors is done generating Z from a multivariate normaldistribution Np(0,Σ) and then applying the inverse translation

X = ξ + λ g−1[δ−1(Z− γ)],

using the vectors of parameters previously determined and the vector of the inverse transla-tion functions g−1[(z1, . . . , zp)′] = [g−1

1 (z1), . . . , g−1p (zp)], where g−1

i (·) is defined by (3) fori = 1, . . . , p.

This method takes to the generation of random vectors with the same moments of themarginals, but not necessarily with the same correlation matrix, because the (4) is notlinear. This suggests to measure the association among the components of X by meansof the Spearman’s rank correlation matrix RS = (ρij) or the Kendall’s rank correlationmatrix Rτ = (τij) whose elements, as known, are not only invariant regarding to non lineartransformations but, in the normal case, are also connected to the correlation coefficient bythe relations

ρij = 2 sin[π

6ρij

]and

6

ρij = sin[π2τij

].

As a consequence, this method takes to the generation of random vectors from X withthe same moments of the marginals and given matrix RS or Rτ . In the next section weshow an extension of the multivariate Johnson distribution which leads to the generationof random vectors with given correlation matrix.

Multivariate Beta and Gamma Distributions. Mathai and Moschopoulos (1991 and1993) have introduced two multivariate distributions with beta and gamma marginals whichcan be easily indirectly simulated. Let B1, . . . , Bp be beta random variables with densityfunction

fBi(bi) =Γ(αi + βi)Γ(αi)Γ(βi)

bαi−1i (1− bi)βi−1,

with bi ∈ (0, 1) parameters αi > 0 and βi > 0 for i = 1, . . . , p, and let U be a uniform randomvariable with support on (0, 1]. Moreover, let U , B1, . . . , Bp be mutually independent andlet

Xi = Bi U1/(αi+βi), i = 1, . . . , p. (5)

Then, the random vector X = (X1, . . . , Xp)′ has components Beta(αi, βi + 1) for i =1, . . . , p and the following properties can be obtained directly from the definition :

E(Xi) =αi

(αi + βi + 1),

Var(Xi) =αi(βi + 1)

(αi + βi + 1)2((αi + βi + 2),

Cov(Xi, Xj) =αiαj

(αi + βi + 1)(αj + βj + 1)[(αi + βi + 1)(αj + βj + 1)− 1], i 6= j.

Now, consider the independent gamma random variables V0, V1, . . . , Vp with densityfunction

fVi(vi) =1

λθii Γ(θi)

(vi − γi)θi−1 exp [−(vi − γi)/λi] ,

with vi > γi and parameters θi > 0, λi > 0 and γi ∈ R for i = 0, 1, . . . , p, and let

Yi =λiλ0V0 + Vi, i = 1, . . . , p. (6)

Then, the random vector Y = (Y1, . . . , Yp)′ has components Gamma(θ0+θi, λi, (γ0/λ0)λi+γi) for i = 1, . . . , p, and, directly from the definition or from the moment generatingfunction, the following properties can be obtained:

E(Yi) = (θ0 + θi)λi + (γ0/λ0)λi + γi,

Var(Yi) = (θ0 + θi)λ2i ,

Cov(Yi, Yj) = θ0λiλj , i 6= j.

Therefore, the generation of random vectors from X or Y simply requires the generationof independent variates from the beta and uniform or gamma random variables and thenthe application of the transformations (5) or (6). Note that for both distributions thecorrelation structure is positive.

7

Multivariate Extreme Value Distribution. Suppose that the cdf of the random vectorX = (X1, . . . , Xp)′ follows the logistic model

FX1,...,Xp(x1, . . . , xp) = exp

−

[p∑i=1

exp(−xi − µi

ρ σi

)]ρ, (7)

where xi ∈ R and parameters µi ∈ R and σi > 0 for i = 1, . . . , p and ρ ∈ [0, 1] measures thedependence among the marginals. The extremes ρ→ 1 and ρ→ 0 correspond, respectively,to the independence and complete dependence. Then, the components of X have an extremevalue distribution (or Gumbel distribution) with cdf

FXi(xi) = exp[− exp

(−xi − µi

σi

)], i = 1, . . . , p.

Shi (1995) has computed the density function of X and he has proposed a transformationwhich allows its simulation. Let

yi = exp(−xi − µi

σi

), i = 1, ..., p,

z =

(p∑i=1

y1/ρi

)ρ.

Then, (7) can be written as

FX1,...,Xp(x1, . . . , xp) = exp

−

(p∑i=1

y1/ρi

)ρ= e−z,

from which we can derive the density function

fX1,...,Xp(x1, . . . , xp) =∂pFX1,...,Xp

∂x1 · · · ∂xp=

(p∏i=1

y1/ρi

σi

)z1−p/ρQp (z, ρ) e−z,

where

Qp(z, ρ) =(p− 1ρ

− 1 + z

)Qp−1(z, ρ)− z

∂Qp−1(z, ρ)∂z

, Q1(z, ρ) = 1. (8)

Let T1 = cos2 θ1,Ti =

(∏i−1i=1 sin2 θj

)cos2 θi, i = 2, . . . , p− 1,

Tp =∏p−1j=1 sin2 θj , θj ∈

[0, π2

], j = 1, . . . , p− 1.

and consider the transformation

yi = z T ρi , i = 1, . . . , p,

orxi = µi − σi(log z + ρ log Ti), i = 1, . . . , p. (9)

8

Then, the density function of the random vector (Z,Θ1,Θ2, . . . ,Θp−1) is given by (Shi,1995)

gZ,θ1,...θp−1(z, θ1, . . . , θp−1) = hz(z)p−1∏j=1

gθj(θj),

where

hZ(z) =ρp−1

(p− 1)!Qp(z, ρ) e−z, z > 0,

gθj(θj) = 2(p− j) cos θj(sin θj)2(p−j)−1, θj ∈

[0,π

2

], j = 1, . . . , p− 1,

therefore the random variables Z,Θ1, . . . ,Θp−1 are independent among them.Because Qp(z, ρ) is a p− 1-polynomial of z, if we write the density function of Z as

hZ(z) =p∑j=1

qp,jϕ(z, j),

whereϕ(z, k) =

1Γ(k)

zk−1e−z, z > 0,

we note that Z has a mixture gamma distribution with shape parameter k and weights qp,jdepending on Qp(z, α). It is easy to obtain the following recurrence relations from the (8):

qp,1 =Γ(p− ρ)

Γ(p)Γ(1− ρ),

(p− 1)qp,j = (p− 1− jρ)qp−1,j + ρ(j − 1)qp−1,j−1, j = 2, . . . , p− 1,qp,p = ρp−1.

Now, the random vectors can be easily generated from the distribution (7) with mar-ginals distributed as an extreme value distribution. As a matter of fact, applying the inversetransformation method, sin2 Θj , cos2 Θj can be obtained, respectively, from U1/(p−j) and1−U1/(p−j), where U is the uniform distribution on (0, 1], while the variates from the ran-dom variable Z with mixture gamma distribution can be generated using the compositionmethod. Therefore, using the (9), the simulated random vectors from X are obtained.

Multivariate Inverse Gaussian Distribution Barndorff-Nielsen, Blæsid and Seshadri(1992) have proposed an extension to the multivariate case of the inverse gaussian distri-bution using an additive random effects model which can be easily simulated. Let X be arandom variable InverseGaussian(χ, ψ) with density function

fX(x) =[ χ2π

]1/2e√χ ψx−3/2 exp

[−1

2

(χx

+ ψ x)], x > 0,

with parameters χ > 0 and ψ > 0. Now, let us suppose that the components of the randomvector X = (X1, . . . , Xp)′ are defined using the additive random effects model

Xi = αi T + Vi, i = 1, . . . , p, (10)

9

where T ∼InverseGaussian(χ, ψ) and V1, . . . , Vp, where Vi ∼InverseGaussian(χi, ψ/αi), areindependent among them and α1, . . . , αp are positive parameters. Then, X has InverseGaus-sian(

√χαi+

√χi)2, ψ/αi), i = 1, . . . , p, marginals and from the moment generating function

the following properties can be obtained:

E(Xi) =

√(√χαi +

√χi)2

ψ/αi,

Var(Xi) =

√(√χαi +

√χi)2

(ψ/αi)3,

Cov(Xi, Xj) =αiαjχ

2√(χψ)3

, i 6= j.

Therefore, the generation of random vectors from X simply requires at first the genera-tion of independent variates from inverse gaussian random variables and then the applicationof the transformation (10). In this case too, note that the model used to build the mul-tivariate distribution leads to a positive correlation structure, because it depends only onpositive parameters.

The Cook-Johnson Family of Multivariate Uniform Distributions. Let U =(U1, . . . , Up)′ be a p-dimensional uniform distribution with support on the hypercube (0, 1]p

and with cdf

FU1,...,Up(u1, . . . , up) =

p∑i=1

u−1/αi − p+ 1

−α, ui ∈ (0, 1], α > 0. (11)

Cook and Johnson (1981) have studied this family of distributions and have shown that

limα→0

FU1,...,Up(u1, . . . , up) = min [u1, . . . , up]

and

limα→∞

FU1,...,Up(u1, . . . , up) =p∏i=1

ui;

therefore the correlation among the components of U approaches to its maximum whenα→ 0 and approaches to zero when α→∞.

A simulation of U can be run using the following algorithm: let Y1, . . . , Yp be i.i.d.standard exponential distributions and let the distribution of V be a standard gamma withshape parameter α. Then, the cumulative distribution function of the random variables

Ui = [1 + Yi/V ]−α , i = 1, . . . , p,

have a cdf given by (11).For a set of arbitrary marginal distributions, FX1 , . . . , FXp, we can define the joint cdf

as

FX1,...,Xp(x1, . . . , xp) =

p∑i=1

FXi(xi)−1/α − p+ 1

−α. (12)

If we assume that the random vector X = (X1, . . . , Xp)′ has a multivariate distributionwith cdf given by the equation (12), then a simulation of it can be easily implementedinverting (U1, . . . , Up) with (F−1

X1, . . . , F−1

Xp). In this paper we have implemented the (11)also with the marginal distributions considered by Cook and Johnson (1981):

10

(i) Burr distribution:

FXi(xi) = 1− (1 + dixcii )−k,

with xi > 0 and parameters ci > 0, di > 0 for i = 1, . . . , p, and k > 0.

(ii) Pareto distribution:

FXi(xi) = 1−(θixi

)a,

with xi > θi and parameters θi > 0 for i = 1, . . . , p, and α > 0.

(iii) Logistic distribution:FXi(xi) = [1 + exp(−xi)]−α ,

with xi ∈ R and parameter α > 0.

Elliptically Contoured Distributions. An interesting class of multivariate distribu-tions including the gaussian distribution as a special case is given by the class of ellipticallycontoured distributions (Johnson, 1987, ch. 6; see also Fang, Kotz and Ng, 1990). A ran-dom vector X = (X1, . . . , Xp)′ is elliptically contoured with scale matrix Σ and locationvector µ if

X = µ +RAU(p), (13)

where A is a lower triangular matrix obtained by the Cholesky decomposition of Σ, U(p) isa random vector distributed as a uniform on the surface of the unit p-dimensional hyper-sphere as defined below and R is a random variable independent from U(p). The randomvariable R2 has the distribution of (X−µ)′Σ−1(X−µ) and in some cases of practical usagehas a recognizable distribution which can be easily generated; therefore we can simulate anobservation from X using its stochastic representation (13). These cases include the multi-variate Pearson Type II and Pearson Type VII distributions which, according to Johnson(1987), are suitable for simulation studies.

The random vector X = (X1, . . . , Xp)′ has a multivariate Pearson Type II distributionif the density function is given by

fX1,...,Xp(x1, . . . , xp) =Γ(p/2 + v + 1)Γ(v + 1)πp/2

|Σ|−1/2[1− (x− µ)′Σ−1(x− µ)

]m,

where x = (x1, . . . , xp)′ and v > −1, p is the dimension of X. The support of the distributionis restricted to the region (x− µ)′Σ−1(x− µ) ≤ 1.

The generation of random vectors from this distribution on the basis of the (13) is veryeasy, because the distribution of the quadratic form R2 = (X−µ)′Σ−1(X−µ) is beta withparameters p/2 and v + 1.

The random vector X = (X1, . . . , Xp)′ has a multivariate Pearson Type VII distributionif the density function

fX1,...,Xp(x1, . . . , xp) =Γ(v)

Γ(v − p/2)πp/2|Σ|−1/2

[1 + (x− µ)′Σ−1(x− µ)

]where x = (x1, . . . , xp)′, xi ∈ R for i = 1, . . . , p, and v > p/2, p is the dimension of X.

11

In this case the distribution of T = R2 has density function

fT (t) =Γ(v)

Γ(p/2)Γ(v − p/2)tp/2−1(1 + t)−v, t > 0.

This density is a Pearson Type VI distribution and can be generated as T = Y/(1 − Y ),where Y is beta with parameters p/2 e v − p/2 (Johnson and Kotz, 1972, Cap. 37, haveproposed a different parametrization of this distribution referring it to the multivariate tseen below).

Finally, Gomez, Gomez-Villegas and Marın (1998) have proposed a multivariate gener-alization of the exponential power distribution which is elliptically contoured. The randomvector X = (X1, . . . , Xp)′ follows a multivariate exponential power distribution if has densityfunction

fX1,...,Xp(x1, . . . , xp) =pΓ (p/2))

πp/2Γ(1 + p

2β

)21+ p

2β

|Σ|−1/2 exp−1

2[(x− µ)′Σ−1(x− µ)

]β,

where x = (x1, . . . , xp)′, xi ∈ R for i = 1, . . . , p, and β > 0 is a shape parameter, p is thedimension of X. Gomez, Gomez-Villegas e Marın (1998) have proved that the (13) is thestochastic representation of X when the distribution of R is given by

fR(r) =p

Γ(1 + p

2β

)2

p2β

rp−1 exp−1

2r2β, r > 0.

The generation of random vectors from this distribution on the basis of the (13) is veryeasy, because the distribution of the random variable T = R2β is a gamma with shapeparameter p/(2β) and scale parameter 2.

4 Generating Random Vectors with Given Marginal Distri-butions and Given Correlation Matrix

The dependence structure of non elliptical multivariate distributions showed in the lastsection is ruled in general by the values of the parameters of the distributions. As a con-sequence, it is not easy to run Monte Carlo experiments with these distributions when itis necessary to define a priori their dependence structure as it happens, for example, whenthe robustness of statistical methods is investigated. In this section we show the methodswe used to generate random vectors from non elliptical distributions with given marginalsand correlation matrix with Mathematica (about the definition of this kind of vectors see,for example, Tiit, 1984).

4.1 Infinitely divisible distributions

Suppose that X is a random variable with cdf FX and characteristic function ϕ(t). If[ϕ(t)]1/n is the characteristic function of a random variable for each positive entire numbern, then X is infinitely divisible, that is FX is the distribution of a sum of n independentrandom variables, where [ϕ(t)]1/n is the characteristic function of each of these distribu-tions. For example, the gaussian, Poisson and gamma distributions are infinitely divisiblerandom variables. Park and Shin (1998) have proposed an algorithm to generate random

12

vectors from multivariate distributions with marginals in the class of the random variablesclosed respect to the sum and given correlation matrix. Sim (1993) too has proposed analgorithm to generate Poisson and gamma random vectors with given marginals and covari-ance matrix. In this paper, on the contrary, we have followed the approach of Prekopa andSzantai (1978) to implement an algorithm using Mathematica which allows the generationof random vectors from a gamma distribution with given mean vector and covariance ma-trix. Differently from the method we followed, which always has a solution, the algorithmsby Park and Shin (1998) and Sim (1993) do not always converge.

Let X = (X1, . . . , Xp)′ be a multivariate distribution with gamma marginals with densityfunction

fXi(xi) =1

λθii Γ(θi)

xθi−1i e−xi/λi , xi > 0,

with parameters θi > 0 and λi > 0 for i = 1, . . . , p. The aim is to generate random vectorsfrom X when the mean vector µ = (µ1, . . . , µp)′ and the covariance matrix Σ = (σij)are given with positive elements. Obviously, in this case also the correlation matrix withpositive elements R = (ρij) is defined and the parameters of Xi can be derived fromthe corresponding mean and variance using the relations θi = µ2

i /σii and λi = σii/µi fori = 1, . . . , p.

Now, consider the standard multivariate Gamma distribution Y = (Y1, . . . , Yp)′ obtainedusing the transformation

Y = B−1X,

where B =diag(λ1, . . . , λp)′. Then, we have that

E(Yi) = ηi = θi,

Var(Yi) = δii = θi, i = 1, . . . , p,Cov(Yi, Yj) = δij = ρij

√θiθj , ρij ≥ 0,

i, j = 1, . . . , p, i 6= j.

At this point we introduce the independent gamma random variables again in standardform Z1, . . . , Zk with shape parameters, respectively, ξ1, . . . , ξk, k ≥ p, such that Y = TZ,where Z = (Z1, . . . , Zk)′ and T is an incidence matrix (0, 1) of dimension (p× k) and rankp. Then,

E(Y) = η = Tξ,

Cov(Y) = ∆ = TΩT′,

with η = (η1, . . . , ηp)′, ∆ = (δij), ξ = (ξ1, . . . , ξk)′ and Ω =diag(ξ1, . . . , ξk). Therefore,the generation of random vectors from Y to run stochastic simulations with Monte Carlomethods is very easy, because it is sufficient to generate independent Gamma random vectorswith the usual methods and, then, multiply them by T. Therefore, the generation of randomvectors from X can be obtained by means of the transformation

X = BY.

Since µ and Σ are given, and consequently also η and ∆, the values assigned to ξ1, . . . , ξkshould satisfy the following conditions:

Tξ = η,

Tξ = c,ξ ≥ 0,

13

where c = δ11, δ12, . . . , δ1p, δ22, . . . , δpp′ and T is a matrix of dimension (p(p + 1)/2) × kobtained multiplying the lines of T among them in the same order of the elements of c. Asa consequence of ηi = δii = θi for i = 1, . . . , p, these conditions are also satisfied solving thefollowing system:

Tξ = c,subject to ξ ≥ 0.

The following remarks can be done.

(i) With the aim of finding a parameter vector which satisfies the constraints for a widedomain, it is convenient to use an incidence matrix with a number of elements differentfrom zero at least equal to covariances. Prekopa and Szantai (1978) suggest to usean incidence matrix with a number of column vectors equal to the number of all thedifferent combinations which can be chosen from p different elements (removing thecolumn vector composed of all 0, the columns are 2p − 1 ). Using a pattern based ona recursive relation, Ronning (1977) has obtained three different incidence matricesof dimension p× (k(k − 1)/2).

(ii) The given conditions can determine θ1, . . . , θp unambiguously. If this is not the case,i.e. there are at least two vectors ξ satisfying the conditions, there are infinite solu-tions.

(iii) Since the solutions depend on T, it is not always possible to satisfy them. However,it is possible to face the problem as a linear programming problem minimizing theabsolute difference between the corresponding elements of Tξ and c. Therefore, it isnecessary to solve the following system (Prekopa and Szantai, 1978):

Minimize∑p(p+1)/2

i=1 ui +∑p(p+1)/2

i=1 vi,

subject to u− v + Tξ = c,u ≥ 0, v ≥ 0, ξ ≥ 0.

The function of MultiRand which allows the generation of n random vectors from amultivariate gamma with this method and with approximately mean vector mu, covariancematrix Sigma and incidence matrix of Prekopa and Szantai (1978) is

RandomArray[MultivariateGammaTypeDistribution[mu,Sigma],n]

4.2 Normal Copula

One of the most used methods to deal with multivariate distributions is the function copula,which is a cdf C on the unit hypercube (0, 1]p with uniform marginals such that, given thecumulative distribution functions FX1 , . . . , FXp and C (cf. Nelsen, 1999),

FX1,...Xp(x1, . . . , xp) = C(FX1(x1), . . . , FXp(xp)).

In general, the modelling of different correlated distributions is easier if a normal copulais used, which is not only very flexible in the choice of correlation parameters, but alsoeasier to be applied in Monte Carlo studies.

14

Assume that Z = (Z1, . . . , Zp)′ has a multivariate normal distribution with standardmarginals Zi ∼ N(0, 1) and correlation matrix R and indicate the joint cdf of Z withGZ1,...,Zp(z1, . . . , zp). Then, the cdf

C(u1, . . . , up) = GZ1,...,Zp

(Φ−1(z1), . . . ,Φ−1(zp)

),

where Φ(·) indicates the cdf of the standard normal, defines a multivariate cdf known asnormal copula. Therefore, for any set of cdf of given marginals FX1 , . . . , FXp , the randomvariables

X1 = F−1X1

(Φ(z1)), . . . , Xp = F−1Xp

(Φ(zp))), (14)

have joint cdf

FX1,...Xp(x1, . . . , xp) = GZ1,...,Zp

(Φ−1(FX1(z1)), . . . ,Φ

−1(FXp(zp)))

with cdf of the marginals given by FX1 , . . . , FXp .Also if the gaussian copula does not have a simple analytical expression, it is very easy

to build Monte Carlo algorithms using it. Suppose that a set of correlated distributions(X1, . . . , Xp) is given with cdf of the marginals given by FX1 , . . . , FXp . If we assume thatthe multivariate distribution (X1, . . . , Xp) can be described by the normal copula, then itis sufficient to draw a random vector from a multivariate normal with standard marginalsand correlation matrix R with the usual method and then to apply the (14) to obtain arandom vector from the desired distribution. However, the random vectors generated likethat not necessarily have the same given correlation matrix R, therefore it can be suitableto measure the association among the given marginals using either the Spearman’s rankcorrelation matrix RS = (ρij) or the Kendall’s rank correlation matrix Rτ = (τij), whoseelements, as seen before, are not only invariant regarding to non linear transformations but,in the normal case, are also connected to the correlation coefficient.

We have used the normal copula function to implement a procedure in MultiRand whichgenerate random vectors from a multivariate distribution with univariate marginals follow-ing a generalized extreme value distribution (Joe, 1994). For this, a usual parametrizationof the cdf is given by

FX(x) = exp

−(

1 + ξx− µ

σ

)−1/ξ, 1 + ξ

x− µ

σ≥ 0,

where µ ∈ R and σ > 0 are, respectively, position and scale parameters and ξ ∈ R is ashape parameter. The case ξ < 0 corresponds to the Weibull distribution, ξ = 0 to theGumbel and ξ > 0 to the Frechet distribution. Therefore, the function of MultiRand whichallows the generation of n random vectors from this distribution with parameter vectors ofthe marginals mu, sigma and xi is given by

RandomArray[MultivariateGeneralizedExtremeValueDistribution[mu,beta,sigma,R,option],n]

where R indicates either a correlation matrix or a Spearman’s rank correlation matrix orKendall’s rank correlation matrix according to the value given to the logical variable option.Note that an extreme value multivariate distribution defined like that allows to master, inMonte Carlo studies, the dependence structure among the marginals in a more flexible wayrespect to the extreme value multivariate distribution described in the last section.

15

4.3 Extension of the Multivariate Johnson Distribution

Stanfield et al. (1996) have obtained an extension of the multivariate Johnson distributionwhich allows the generation of random vectors with given correlation matrix and marginalswith the first four moments given. Suppose that the random vector X =(X1, . . . , Xp)′

has mean µX = (µX1 , . . . , µXp)′, standard deviations matrix σX =diag(σX1 , . . . , σXp) andcorrelation matrix RX. Therefore, let the lower triangular matrix LX = (lij) be obtainedby the Cholesky decomposition LXL′X = RX,

If the multivariate distribution Y = (Y1, . . . , Yp)′ has independent standard Johnsoncomponents, so that E(Yi) = 0 and Var(Yi) = 1 for i = 1, . . . , p, then

W = µX + σX LX Y (15)

has the same mean vector and covariance matrix of X. Assume that the parameters γi, δi,λi and ξi of Yi are arranged in a way that the i-th component Wi of the random vector Whas the same skewness and kurtosis of Xi for i = 1, . . . , p.

Let aX and bX be vectors whose i-th element is, respectively, the skewness and kurtosisof Xi and, in a similar way, let aY and bY be the vectors indicating the skewness andkurtosis of the components of Y. Moreover, define the k-fold Hadamard product of LX asL(k)

X = (lkij) for k = 3, 4, together with the auxiliary vector ΨX = (ψ1, . . . , ψp)′, where

ψi = 6p∑j=1

p∑k=j+1

l2ijl2ik, i = 1, . . . , p.

Now, if the random vector W is generated according to the transformation (15) it isto show that the vectors aW and bW of the skewness and kurtosis indices describing thecomponents of W have the following relation with the vectors aY and bY of the skewnessand kurtosis describing the components of Y:

aW = L(3)X aY,

bW = L(4)X bY + ΨX .

Therefore, the parameter vectors γ, δ, λ and ξ of the Johnson distribution which de-termine the distribution of Y can be modified to satisfy the conditions aY =

[L(3)

X

]−1aX,

bY =[L(4)

X

]−1(bX −ΨX).

Then, the transformed random vector W has in this case the same correlation matrix of Xand also skewness and kurtosis of Wi are same of those of Xi for i = 1, . . . , p.

The function of MultiRand which allows to obtain n random vectors from the extension ofa Johnson multivariate distribution with mean vector mu, standard deviation vector sigma,skewness vector a and kurtosis vector b and correlation matrix R is

RandomArray[ExtensionMultivariateJohnsonDistribution[mu,sigma,a,b,R],n]

16

5 Concluding remarks

As stressed in the introduction, the study of the behavior of sampling statistics is usuallydone by means of the simulation of known distributions. In this framework, it is necessarythat algorithms allowing the simulation of multivariate distribution are available and, tothis purpose, we have prepared a package in an algebraic software, specifically Mathematica,which allows the generation of various continuous multivariate distributions.

The distributions taken into consideration can be divided into two groups. A first grouprefers to specific distributions for which it is possible to write analytically the density func-tion or the moment generating function and, as a consequence, to derive the propertiesof the marginals. Such marginals, in general, are an extension to the multivariate case ofknown random variables, which are used very often in statistical applications with unidi-mensional samples. On the contrary, a second group refers to distributions with a givendependence structure of the components and are defined in indirect form. We can say thatthe high quantity of multivariate distributions, which can be generated using MultiRand,could be sufficient to run very complex Monte Carlo experiments.

In the future we plan to implement in Mathematica other multivariate distributions,too, for example, the multivariate Liouville distributions (cf. Fang, Kotz e Ng, 1990) andthe extension to the multivariate case of stable distributions (cf. Nolan, 1998).

6 References

Andrews, D.F. and Stafford, J.E. (1993). Tools for the symbolic computation ofasymptotic expansions. J. R. Statist. Soc. B., 55, 613-627.

Barndorff-Nielsen, O.E., Blæsild, P. and Seshadri, V. (1992). Multivariate distri-butions with generalized inverse Gaussian marginals, and associated Poisson mixtures,Canad. J. Statist., 20, 109-120.

Bartlett, M.S. (1933). On the theory of statistical regression, Proc. Royal Society ofEdinburg, 53, 260-283.

Boswell, M.T., Gore, S.D., Patil, G.P. and Taillie, C. (1993). The art of computergeneration of random variables, Handbook of Statistics 9, 661- 721.

Cook, R.D. and Johnson, M.E. (1981). A family of distributions for modelling non-elliptically symmetric multivariate data, J. R. Statist. Soc. B, 43, 210-218.

Fang, Kai-Tai, Kotz S. and Ng, Kai-Wang (1990). Symmetric Multivariate andRelated Distributions, Chapman & Hall, London.

Fishman, G.S. (1996). Monte Carlo: Concepts, Algorithms, and Applications Springer,New York.

Gentle, J.E. (1998). Random Number Generation and Monte Carlo Methods, Springer,New York.

Gomez, E., Gomez-Villegas, M.A. and Marın, J.M. (1998). A multivariate gener-alization of the power exponential family of distributions, Commun. Statist.-TheoryMeth., 27, 589-600.

17

Heller, B. (1991). Macsyma for Statistician, Wiley, New York.

Hill, I.D., Hill, R. and Holder, R.L. (1974). Fitting Johnson curves by moments,Appl. Statist., 25, 180-189.

Joe, H. (1994). Multivariate extreme-value distributions with applications to environ-mental data, Canad. J. Statist., 22, 47-64.

Johnson, M.E. (1987). Multivariate Statistical Simulation, Wiley, New York.

Johnson, N.L. (1949a). Systems of frequency curves generated by methods of translation,Biometrika, 36, 146-176.

Johnson, N.L. (1949b). Bivariate distributions based on simple translation system, Bio-metrika, 36, 207-304.

Johnson, N.L. and Kotz, S. (1972). Distributions in Statistics: Continuous MultivariateDistributions, Wiley, New York.

Kendall, W.S. (1988). Symbolic computation and the diffusion of shapes of triads, Adv.Appl. Probab., 20, 775-797.

Kendall, W.S. (1990). Computer algebra and stochastic calculus, Notices Am. Math.Soc., 37, 1254-1256.

Mardia, K.V., Kent, J.T. and Bibby, J.M. (1979). Multivariate Analysis, AcademicPress, London.

Mathai, A.M. and Moschopoulos, P.G. (1991). On a multivariate gamma, J. Multiv.Anal., 39, 135-163.

Mathai, A.M. and Moschopoulos, P.G. (1993). A multivariate beta model, Statistica,53, 231-241.

Muller, M.E. (1959). A note on a method for generating points uniformly on n-dimensional spheres, Comm. ACM, 2, 19-20.

Narayan, A. (1990). Computer generation of Dirichlet random vectors, J. Statist. Com-put. Simul., 36, 19-30.

Nelsen, R.B. (1999). An Introduction to Copulas, Springer-Verlag, New York.

Nolan, J.P. (1998). Multivariate stable distributions: Approximation, estimation, simu-lation and identification, in A Practical Guide to Heavy Tails (eds. R.J: Adler, R.E.Feldman and M.S. Taqqu), 509-525, Birkhauser.

Park Chul Gyu and Shin Dong Wan (1998). An algorithm for generating correlatedrandom variables in a class of infinitely divisible distributions, J. Statist. Comput.Simul., 61, 127-139.

Parrish, R.S. (1990). Generating random deviates from multivariate Pearson distribu-tions, Comp. Statist. Data Anal., 9, 283-295.

Prekopa, A. and Szantai, T. (1978). A new multivariate gamma distribution and itsfitting to empirical streamflow data, Water Resources Res., 14, 19-24

18

Provasi C. (1996). Using algebraic software to compute the moments of order statistics,Computational Economics, 9, 199-213.

Ripley, B.D. (1987). Stochastic Simulation, Wiley, New York.

Ronning, G. (1977). A simple scheme for generating multivariate gamma distributionswith non-negative covariance matrix, Technometrics, 19, 179-183.

Scheuer, E.M. and Stoller, D.S. (1962). On the generation of normal random vectors,Technometrics, 4, 278-281.

Schuurmann, F.J., Krishnaiah, P.R. and Chattopadhyay, A.K. (1975). Tables fora multivariate F distribution, Sankhya B, 37, 308-331.

Shi Daoji (1995). Multivariate extreme value distribution and its Fisher InformationMatrix, Acta Mathematica Applicata Sinica, 11, 421-428.

Silverman, B.W. and Young, A.E. (1987). The bootstrap: to smooth or not tosmooth?. Int. Statist. Rev., 74, 469-479.

Sim, C.H. (1993). Generation of Poisson and Gamma random vectors with given marginalsand covariance matrix, J. Statist. Comp. Simul., 47, 1-10.

Stanfield, P.M., Wilson, J.R., Mirka, G.A., Glasscock, N.F., Psihogios, J.P.and Davis, J.R. (1996). Multivariate input modeling with Johnson distributions,in Proceedings of the 1996 Winter Simulation Conference (eds. J.M. Charmes, D.J.Morrice, D.T. Brunner and J.J. Swain), 1457-1464.

Tanner, M.A. (1996). Tools for Statistical Inference: Methods for the Exploration ofPosterior Distributions and Likelihood Functions, Springer, New York.

Tashiro, Y. (1977). On methods for generating uniform points on the surface of a sphere,Ann. Inst. Stat. Math., 29, 295-300.

Thisted, R.A (1988). Elements of Statistical Computing, Chapman & Hall, New York.

Tiit, E.-M. (1984). Definitions of random vectors with given marginals distributions andgiven correlation matrix, Tartu Riikliku Ulikooli Toimetised, 685, 21-36.

Varian, H.R. (ed.) (1992). Economic and Financial Modeling with Mathematica, Springer,New York.

Varian, H.R (ed.) (1996). Computational Economics and Finance: Modeling and Analy-sis with Mathematica, Springer, New York.

Venables, W.N. (1985). The multiple correlation coefficient and Fisher’s A statistic,Austr. J. Statist., 27, 172-182.

Wolfram, S. (1996). The Mathematica Book (3rd ed.), Cambridge University Press,Cambridge (UK).

19