Research ArticleA New Ridge-Type Estimator for the Linear Regression ModelSimulations and Applications

B M Golam Kibria 1 and Adewale F Lukman 23

1Department of Mathematics and Statistics Florida International University Miami FL USA2Department of Physical Sciences Landmark University Omu-Aran Nigeria3Institut Henri Poincare Centre Emile Borel Paris France

Correspondence should be addressed to B M Golam Kibria kibriagfiuedu

Received 20 January 2020 Accepted 28 February 2020 Published 15 April 2020

Academic Editor Osman Kucuk

Copyright copy 2020 B M Golam Kibria and Adewale F Lukman +is is an open access article distributed under the CreativeCommons Attribution License which permits unrestricted use distribution and reproduction in any medium provided theoriginal work is properly cited

+e ridge regression-type (Hoerl and Kennard 1970) and Liu-type (Liu 1993) estimators are consistently attractive shrinkagemethods to reduce the effects of multicollinearity for both linear and nonlinear regression models +is paper proposes a newestimator to solve the multicollinearity problem for the linear regression model +eory and simulation results show that undersome conditions it performs better than both Liu and ridge regression estimators in the smaller MSE sense Two real-life(chemical and economic) data are analyzed to illustrate the findings of the paper

1 Introduction

To describe the problem we consider the following linearregression model

y Xβ + ε (1)

where y is an n times 1 vector of the response variable X is aknown n times p full rank matrix of predictor or explanatoryvariables β is an p times 1 vector of unknown regression pa-rameters ε is an n times 1 vector of errors such that E(ε) 0and V(ε) σ2In In is an n times n identity matrix +e ordinaryleast squares estimator (OLS) of β in (1) is defined as

1113954β (S)minus 1

Xprimey (2)

where S XprimeX is the design matrix+e OLS estimator dominates for a long time until it

was proven inefficient when there is multicollinearityamong the predictor variables Multicollinearity is theexistence of near-to-strong or strong-linear relationshipamong the predictor variables Different authors havedeveloped several estimators as an alternative to the OLSestimator +ese include Stein estimator [1] principal

component estimator [2] ridge regression estimator [3]contraction estimator [4] modified ridge regression(MRR) estimator [5] and Liu estimator [6] Also someauthors have developed two-parameter estimators tocombat the problem of multicollinearity +e authorsinclude Akdeniz and Kaccediliranlar [7] Ozkale and Kaccedilir-anlar [8] Sakallıoglu and Kaccedilıranlar [9] Yang and Chang[10] and very recently Roozbeh [11] Akdenizand Roozbeh [12] and Lukman et al [13 14] amongothers

+e objective of this paper is to propose a new one-parameter ridge-type estimator for the regression parameterwhen the predictor variables of the model are linear or near-to-linearly related Since we want to compare the perfor-mance of the proposed estimator with ridge regression andLiu estimator we will give a short description of each ofthem as follows

11 Ridge Regression Estimator Hoerl and Kennard [3]originally proposed the ridge regression estimator It is oneof the most popular methods to solve the multicollinearityproblem of the linear regression model +e ridge regression

HindawiScientificaVolume 2020 Article ID 9758378 16 pageshttpsdoiorg10115520209758378

estimator is obtained by minimizing the following objectivefunction

(y minus Xβ)prime(y minus Xβ) + k βprimeβ minus c( 1113857 (3)

with respect to β will yield the normal equations

XprimeX + kIp1113872 1113873β Xprimey (4)

where k is the nonnegative constant+e solution to (4) givesthe ridge estimator which is defined as

1113954β(k) S + kIp1113872 1113873minus 1

Xprimey W(k)1113954β (5)

where S XprimeXW(k) [Ip + kSminus 1]minus 1 and k is the biasingparameter Hoerl et al [15] defined the harmonic-meanversion of the biasing parameter for the ridge regressionestimator as follows

1113954kHM p1113954σ2

1113936pi1 α2i

(6)

where 1113954σ2 (YprimeY minus βprimeXprimeY)(n minus p) is the estimated meansquared error form OLS regression using equation (1)and αi is ith coefficient of α Qprimeβ and is defined underequation (17) +ere are a high number of techniquessuggested by various authors to estimate the biasingparameters To mention a few McDonald and Galarneau[16] Lawless and Wang [17] Wichern and Churchill[18] Kibria [19] Sakallıoglu and Kaccedilıranlar [9] Lukmanand Ayinde [20] and recently Saleh et al [21] amongothers

12 Liu Estimator +e Liu estimator of β is obtained byaugmenting d1113954β β + εprime to (1) and then applying the OLSestimator to estimate the parameter +e Liu estimator isobtained to be

1113954β(d) S + Ip1113872 1113873minus 1

Xprimey + d1113954β1113872 1113873 F(d)1113954β (7)

where F(d) [S + Ip]minus 1[S + dIp] +e biasing parameter dfor the Liu estimator is defined as follows

1113954dopt 1 minus 1113954σ21113936

pi1 1 λi λi + 1( 1113857( 1113857( 1113857

1113936pi1 α2i λi + 1( 1113857

21113872 1113873

⎡⎢⎢⎣ ⎤⎥⎥⎦ (8)

where λi is the ith eigenvalue of the XprimeX matrix and α Qprimeβwhich is defined under equation (17) If 1113954dopt is negativeOzkale and Kaccediliranlar [8] adopt the following alternativebiasing parameter

1113954dalt min1113954α2i

1113954σ2λi1113872 1113873 + 1113954α2i⎡⎢⎣ ⎤⎥⎦ (9)

where 1113954αi is the ith component of 1113954αi Qprime1113954βFor more on the Liu [6] estimator we refer our readers to

Akdeniz and Kaccediliranlar [7] Liu [22] Alheety and Kibria[23] Liu [24] Li and Yang [25] Kan et al [26] and veryrecently Farghali [27] among others

In this article we propose a new one-parameter esti-mator in the class of ridge and Liu estimators which will

carry most of the characteristics from both ridge and Liuestimators

13 0e New One-Parameter Estimator +e proposed es-timator is obtained by minimizing the following objectivefunction

(y minus Xβ)prime(y minus Xβ) + k (β + 1113954β)prime(β + 1113954β) minus c1113960 1113961 (10)

with respect to β will yield the normal equations

XprimeX + kIp1113872 1113873β Xprimey minus k1113954β (11)

where k is the nonnegative constant +e solution to (11)gives the new estimator as

1113954βKL S + kIp1113872 1113873minus 1

S minus kIp1113872 11138731113954β W(k)M(k)1113954β (12)

where S XprimeXW(k) [Ip + kSminus 1]minus 1 andM(k) [Ip minus kSminus 1] +e new proposed estimator will becalled the KibriandashLukman (KL) estimator and denoted by1113954βKL

131 Properties of the New Estimator

E 1113954βKL1113872 1113873 W(k)M(k)E(1113954β) W(k)M(k)β (13)

+eproposed estimator is a biased estimator unless k 0

B 1113954βKL1113872 1113873 W(k)M(k) minus Ip1113960 1113961β (14)

D 1113954βKL1113872 1113873 σ2W(k)M(k)Sminus 1

Mprime(k)Wprime(k) (15)

and the mean square error matrix (MSEM) is defined as

MSEM 1113954βKL1113872 1113873 σ2W(k)M(k)Sminus 1

Mprime(k)Wprime(k)

+ W(k)M(k) minus Ip1113960 1113961ββprime W(k)M(k) minus Ip1113960 1113961prime

(16)

To compare the performance of the four estimators(OLS RR Liu and KL) we rewrite (1) in the canonical formwhich gives

y Zα + ε (17)

where Z XQ and α Qprimeβ HereQ is an orthogonal matrixsuch that ZrsquoZQXrsquoXQΛ diag (λ1 λ2 λp) +e OLSestimator of α is

1113954α Λminus 1Zprimey (18)

MSEM(1113954α) σ2Λminus 1 (19)

+e ridge estimator (RE) of α is1113954α(k) W(k)1113954α (20)

where W(k) [Ip + kΛminus 1]minus 1 and k is the biasing parameter

MSEM(1113954α(k)) σ2W(k)Λminus 1W(k)

+ W(k) minus Ip1113872 1113873αα W(k) minus Ip1113872 1113873prime(21)

2 Scientifica

where (W(k) minus Ip) minus k(Λ + kIp)minus 1+e Liu estimator of α is

1113954α(d) Λ + Ip1113872 1113873minus 1

ZprimeY + d1113954α( 1113857 F(d)1113954α (22)

where F(d) [Λ + Ip]minus 1[Λ + dIp]

MSEM(1113954α(d)) σ2FdΛminus 1

Fd +(1 minus d)2(Λ + I)

minus 1

middot ααprime(Λ + I)minus 1

(23)

where Fd (Λ + I)minus 1(Λ + dI)+e proposed one-parameter estimator of α is

1113954αKL Λ + kIp1113872 1113873minus 1Λ minus kIp1113872 11138731113954α W(k)M(k)1113954α (24)

where W(k) [Ip + kΛminus 1]minus 1 and M(k) [Ip minus kΛminus 1]+e following notations and lemmas are needful to prove

the statistical property of 1113954αKL

Lemma 1 Let ntimes n matrices Mgt 0 and Ngt 0 (or Nge 0)then MgtN if and only if λ1 (NMminus 1)lt 1 where λ1 (NMminus 1) isthe largest eigenvalue of matrix NMminus 1 [28]

Lemma 2 Let M be an ntimes n positive definite matrix that isMgt 0 and α be some vector then M minus ααprime ge 0 if and only ifαprimeMminus 1αle 1 [29]

Lemma 3 Let 1113954αi Aiy i 1 2 be two linear estimators ofα Suppose that D Cov(1113954α1) minus Cov(1113954α2)gt 0 whereCov(1113954αi) i 1 2 denotes the covariance matrix of 1113954αi andbi Bias(1113954αi) (AiX minus I)α i 1 2 Consequently

Δ 1113954α1 minus 1113954α2( 1113857 MSEM 1113954α1( 1113857 minus MSEM 1113954α2( 1113857 σ2D + b1b2prime

minus b2b2prime gt 0(25)

if and only if b2prime[σ2D + b1b1prime]minus 1b2 lt 1 where MSEM(1113954αi)

Cov(1113954αi) + bibiprime [30]

+e other parts of this article are as follows +e theo-retical comparison among the estimators and estimation ofthe biasing parameters are given in Section 2 A simulationstudy has been constructed in Section 3 We conducted twonumerical examples in Section 4 +is paper ends up withconcluding remarks in Section 5

2 Comparison among the Estimators

21 Comparison between 1113954α and 1113954αKL +e difference betweenMSEM(1113954α) and MSEM(1113954αKL) is

MSEM[1113954α] minus MSEM 1113954αKL1113858 1113859 σ2Λminus 1minus σ2W(k)M(k)Λminus 1

Mprime(k)Wprime(k)

minus W(k)M(k) minus Ip1113960 1113961ααprime W(k)M(k) minus Ip1113960 1113961prime(26)

We have the following theorem Theorem 1 If kgt 0 estimator 1113954αKL is superior to estimator1113954α using the MSEM criterion that is MSEM[1113954α] minus

MSEM[1113954αKL]gt 0 if and only if

αprime W(k)M(k) minus Ip1113960 1113961prime σ2 Λminus 1minus W(k)M(k)Λminus 1

Mprime(k)W(k)k1113872 11138731113960 1113961 W(k)M(k) minus Ip1113960 1113961αlt 1 (27)

Proof +e difference between (15) and (19) is

D(1113954α) minus D 1113954αKL( 1113857 σ2 Λminus 1minus W(k)M(k)Λminus 1

Mprime(k)Wprime(k)1113872 1113873

σ2diag1λi

minusλi minus k( 1113857

2

λi λi + k( 11138572

⎧⎨

⎩

⎫⎬

⎭

p

i1

(28)

where Λminus 1 minus W(k)M(k)Λminus 1Mprime(k)Wprime(k) will be positivedefinite (pd) if and only if (λi + k)2 minus (λi minus k)2 gt 0 Weobserved that for kgt 0 (λi + k)2 minus (λi minus k)2 4λikgt 0

Consequently Λminus 1 minus W(k)M(k)Λminus 1Mprime(k)Wprime(k) is pd

22 Comparison between 1113954α(k) and 1113954αKL +e difference be-tween MSEM(1113954α(k)) and MSEM(1113954αKL) is

MSEM[1113954α(k)] minus MSEM 1113954αk1113858 1113859 σ2W(k)Λminus 1W(k) minus σ2W(k)M(k)Λminus 1

Mprime(k)W(k)

+ W(k) minus Ip1113872 1113873αα W(k) minus Ip1113872 1113873prime minus W(k)M(k) minus Ip1113960 1113961ααprime W(k)M(k) minus Ip1113960 1113961prime(29)

Scientifica 3

Theorem 2 When λmax(HGminus 1)lt 1 estimator 1113954αKL is supe-rior to 1113954α(k) in the MSEM sense if and only if

αprime W(k)M(k) minus Ip1113960 1113961prime V1 + W(k) minus Ip1113872 1113873ααprime W(k) minus Ip1113872 111387311138731113960 1113961

W(k)M(k) minus Ip1113960 1113961α

(30)

λmax HGminus 1

1113872 1113873lt 1 (31)

where

V1 σ2W(k)Λminus 1W(k) minus σ2W(k)M(k)Λminus 1

Mprime(k)W(k)

H 2W(k)

G kW(k)Λminus 1W(k)

(32)

Proof Using the dispersion matrix difference

V1 σ2W(k)Λminus 1W(k) minus σ2W(k)M(k)Λminus 1

Mprime(k)W(k)

σ2kΛminus 1 ΛW(k)Λminus 1W(k) + ΛW(k)Λminus 1

W(k)1113872

minus kW(k)Λminus 1W(k)1113873Λminus 1

σ2W(k)Λminus 1W(k) minus σ2W(k) Ip minus kΛminus 1

1113960 1113961

middot Λminus 1Ip minus kΛminus 1

1113960 1113961W(k)

σ2kΛminus 1(G minus H)Λminus 1

(33)

It is obvious that for kgt 0Ggt 0 andHgt 0 According toLemma 1 it is clear that G-Hgt 0 if and only if HGminus 1 lt 1where λmax(HGminus 1)lt 1 is the maximum eigenvalue of thematrix HGminus 1 Consequently V1 is pd

23 Comparison between 1113954α(d) and 1113954αKL +e difference be-tween MSEM(1113954α(d)) and MSEM(1113954αKL) is

MSEM[1113954α] minus MSEM 1113954αk1113858 1113859 σ2FdΛminus 1

Fd minus σ2W(k)M(k)Λminus 1Mprime(k)Wprime(k)

+(1 minus d)2(Λ + I)

minus 1ααprime(Λ + I)minus 1

minus W(k)M(k) minus Ip1113960 1113961ααprime W(k)M(k) minus Ip1113960 1113961prime(34)

We have the following theorem

Theorem 3 If kgt 0 and 0lt dlt 1 estimator 1113954αKL is superior toestimator 1113954α(d) using the MSEM criterion that isMSEM(1113954α(d)) minus MSEM(1113954αKL)gt 0 if and only if

αprime W(k)M(k) minus Ip1113960 1113961prime V2 +(1 minus d)2(Λ + I)

minus 1ααprime(Λ + I)minus 1

1113960 1113961

middot W(k)M(k) minus Ip1113960 1113961αlt 1

(35)

where V2 σ2FdΛminus 1Fd minus σ2W(k)M(k)Λminus 1Mprime(k)W(k)

Proof Using the difference between the dispersion matrix

V2 σ2FdΛminus 1

Fd minus σ2W(k)M(k)Λminus 1Mprime(k)W(k)

σ2 FdΛminus 1

Fd minus W(k)M(k)Λminus 1Mprime(k)W(k)1113872 1113873

σ2 Λ + Ip1113960 1113961minus 1 Λ + dIp1113960 1113961Λminus 1 Λ + Ip1113960 1113961

minus 1 Λ + dIp1113960 1113961

minus Λ(Λ + k)minus 1Λminus 1

middot (Λ minus k)Λminus 1Λminus 1(Λ minus k)Λ(Λ + k)

minus 1

(36)

where W(k) [Ip + kΛminus 1]minus 1 Λ(Λ + k)minus 1 andM(k) [Ip minus kΛminus 1] Λminus 1(Λ minus k)

σ2diagλi + d( 1113857

2

λi λi + 1( 11138572 minus

λi minus k( 11138572

λi λi + k( 11138572

⎧⎨

⎩

⎫⎬

⎭

p

i1

(37)

We observed that FdΛminus 1Fd minus W(k)M(k)Λminus 1Mprime(k)

W(k) is pd if and only if (λi + d)2(λi + k)2 minus

(λi minus k)2(λi + 1)2 gt 0 or (λi + d) (λi + k) minus (λi minus k)

(λi + 1)gt 0 Obviously for kgt 0 and 0lt dlt 1 (λi + d)(λi +

k) minus (λi minus k)(λi + 1) k(2λ + d + 1) + λ(d minus 1)gt 0 Conse-quently FdΛminus 1Fd minus W(k)M(k)Λminus 1Mprime(k)W(k) is pd

24 Determination of Parameter k +ere is a need to esti-mate the parameter of the new estimator for practical use+e ridge biasing parameter and the Liu shrinkage pa-rameter were determined by both Hoerl and Kennard [3]and Liu [6] respectively Different authors have developedother estimators of these ridge parameters To mention afew these include Hoerl et al [15] Kibria [19] Kibria andBanik [31] and Lukman and Ayinde [20] among others+eoptimal value of k is the one that minimizes

MSEM 1113954βKL1113872 1113873 σ2W(k)M(k)Sminus 1

Mprime(k)Wprime(k)

+ W(k)M(k) minus Ip1113960 1113961ββprime W(k)M(k) minus Ip1113960 1113961prime

p(k) MSEM 1113954αKL1113858 1113859 tr MSEM 1113954αKL( 11138571113858 1113859

p(k) σ2 1113944

p

i1

λi minus k( 11138572

λi λi + k( 11138572 + 4k

21113944

p

i1

α2iλi + k( 1113857

2

(38)

Differentiatingm(k d)with respect to k gives and setting(zp(k)zk) 0 we obtain

k σ2

2α2i + σ2λi( 1113857 (39)

4 Scientifica

+e optimal value of k in (39) depends on the unknownparameter σ2 and α2 +ese two estimators are replaced withtheir unbiased estimate Consequently we have

1113954k 1113954σ2

21113954α2i + 1113954σ2λi1113872 1113873 (40)

Following Hoerl et al [15] the harmonic-mean versionof (40) is defined as

1113954kHMN p1113954σ2

1113936pi1 21113954α2i + 1113954σ2λi1113872 11138731113960 1113961

(41)

According to Ozkale and Kaccediliranlar [8] the minimumversion of (41) is defined as

1113954kmin min1113954σ2

21113954α2i + 1113954σ2λi1113872 1113873⎡⎢⎣ ⎤⎥⎦ (42)

3 Simulation Study

Since theoretical comparisons among the estimators ridgeregression Liu and KL in Section 2 give the conditionaldominance among the estimators a simulation study hasbeen conducted using the R 341 programming languages tosee a better picture about the performance of the estimators

31 Simulation Technique +e design of the simulationstudy depends on factors that are expected to affect theproperties of the estimator under investigation and thecriteria being used to judge the results Since the degree ofcollinearity among the explanatory variable is of centralimportance following Gibbons [32] and Kibria [19] wegenerated the explanatory variables using the followingequation

xij 1 minus ρ21113872 111387312

zij + ρzip+1

i 1 2 n j 1 2 3 p(43)

where zij are independent standard normal pseudo-randomnumbers and ρ represents the correlation between any twoexplanatory variables We consider p 3 and 7 in thesimulation+ese variables are standardized so that XprimeX andXprimey are in correlation forms +e n observations for thedependent variable y are determined by the followingequation

yi β0 + β1xi1 + β2xi2 + β3xi3 + middot middot middot + βpxip + ei

i 1 2 n(44)

where ei are iidN (0 σ2) and without loss of any generalitywe will assume zero intercept for the model in (44) +evalues of β are chosen such that βprimeβ 1 [33] Since our main

Table 1 Estimated MSE when n 30 p 3 and ρ 070 and 080

n 30 07 08Sigma k d OLS Ridge Liu New est OLS Ridge Liu New est1 01 0362 0352 0291 0342 0547 0519 0375 0491

02 0342 0298 0323 0493 0391 044403 0333 0305 0307 0470 0407 040404 0325 0312 0293 0449 0425 037005 0317 0320 0280 0431 0443 034206 0309 0328 0268 0414 0462 031807 0302 0336 0258 0398 0482 029908 0296 0344 0249 0384 0503 028209 0290 0353 0242 0372 0525 026910 0284 0362 0235 0360 0547 0258

5 01 8021 7759 6137 7501 12967 12232 8364 1152202 7511 6331 7021 11567 8817 1026103 7277 6529 6577 10962 9284 915604 7056 6731 6165 10411 9766 818605 6846 6937 5784 9907 10263 733306 6647 7146 5430 9445 10775 658107 6459 7359 5102 9019 11301 591808 6280 7576 4797 8626 11842 533109 6109 7797 4513 8263 12397 481310 5947 8021 4250 7926 12967 4354

10 01 31993 30939 24421 29907 51819 48871 33333 4602202 29945 25203 27977 46201 35155 4095503 29005 26000 26189 43775 37034 3651404 28116 26812 24532 41561 38972 3261205 27274 27639 22995 39536 40968 2917606 26474 28480 21568 37677 43022 2614507 25715 29336 20241 35966 45134 2346608 24994 30207 19008 34387 47304 2109609 24307 31092 17860 32926 49532 1899610 23654 31993 16791 31570 51819 17134

Scientifica 5

Table 2 Estimated MSE when n 30 p 3 and ρ 090 and 099

n 30 09 099Sigma k d OLS Ridge Liu New est OLS Ridge Liu New est1 01 1154 1012 0532 0883 12774 4155 0857 1128

02 0899 0583 0691 2339 1388 194603 0809 0638 0555 1603 2117 327804 0736 0697 0459 1214 3045 441605 0675 0762 0392 0978 4170 534006 0625 0831 0346 0821 5495 609707 0582 0905 0317 0712 7017 672508 0545 0983 0299 0631 8738 725509 0514 1066 0291 0571 10656 77081 0487 1154 0289 0524 12774 8100

5 01 28461 24840 12067 21501 319335 102389 17451 2338302 21945 13492 16402 56008 31445 4036803 19588 15017 12625 36978 50327 7144704 17641 16640 9805 26816 74095 9832205 16010 18362 7690 20580 102751 12026906 14627 20184 6104 16415 136293 13826807 13442 22105 4917 13467 174723 15324008 12418 24124 4036 11293 218040 16588009 11526 26243 3393 9637 266244 1766951 10741 28461 2935 8343 319335 186058

10 01 113841 99331 48088 85947 1277429 409249 69149 9286802 87726 53814 65494 223571 125195 16055403 78277 59935 50326 147369 200793 28474904 70466 66450 38986 106666 295943 39218405 63919 73361 30469 81687 410644 47994006 58368 80667 24064 64998 544898 55191607 53612 88368 19262 53189 698703 61179408 49498 96464 15687 44476 872060 66235009 45910 104955 13064 37839 1064960 7056111 42758 113841 11182 32655 1277429 743065

Table 3 Estimated MSE when n 100 p 3 and ρ 070 and 080

n 100 09 099Sigma k d OLS Ridge Liu New est OLS Ridge Liu New est1 01 01124 01121 01105 01118 01492 01478 01396 01465

02 01118 01107 01114 01465 01404 0144103 01116 01108 01110 01453 01414 0142004 01114 01110 01106 01442 01423 0140105 01112 01112 01104 01432 01434 0138406 01110 01114 01101 01422 01444 0136907 01108 01116 01100 01412 01455 0135608 01106 01119 01099 01403 01467 0134509 01105 01121 01099 01395 01479 013361 01104 01124 01099 01387 01492 01328

5 01 20631 20452 19126 20274 32440 31954 28523 3147202 20276 19289 19924 31480 28942 3053803 20102 19454 19583 31019 29365 2963804 19932 19619 19249 30570 29793 2877105 19764 19785 18922 30133 30224 2793406 19599 19952 18603 29707 30659 2712807 19436 20121 18291 29291 31098 2635008 19276 20290 17986 28887 31542 2560009 19119 20460 17688 28492 31989 248761 18964 20631 17396 28108 32440 24178

10 01 81632 80901 75481 80174 129200 127234 113344 12528702 80182 76150 78747 125320 115045 12151103 79474 76822 77351 123456 116761 11786704 78777 77498 75984 121640 118493 114349

6 Scientifica

objective is to compare the performance of the proposedestimator with ridge regression and Liu estimators weconsider k d 01 02 1 We have restricted k between0 and 1 as Wichern and Churchill [18] have found that theridge regression estimator is better than the OLS when k isbetween 0 and 1 Kan et al [26] also suggested a smallervalue of k (less than 1) is better Simulation studies arerepeated 1000 times for the sample sizes n 30 and 100 andσ2 1 25 and 100 For each replicate we compute the meansquare error (MSE) of the estimators by using the followingequation

MSE αlowast( 1113857 1

10001113944

1000

i1αlowast minus α( 1113857prime αlowast minus α( 1113857 (45)

where αlowast would be any of the estimators (OLS ridge Liu or KL)Smaller MSE of the estimators will be considered the best one

+e simulated results for n 30 p 3 and ρ 070 080and ρ 090 099 are presented in Tables 1 and 2 respec-tively and for n 100 p 3 and ρ 07 080 and ρ 090099 are presented in Tables 3 and 4 respectively +ecorresponding simulated results for n 30 100 and p 7 arepresented in Tables 5ndash8 For a better visualization we haveplotted MSE vs d for n 30 σ 10 and ρ 070 090 and099 in Figures 1ndash3 respectively We also plotted MSE vs σfor n 30 d 50 and ρ 090 and 099 which is presentedin Figures 4 and 5 respectively Finally to see the effect ofsample size on MSE we plotted MSE vs sample size ford 05 and ρ 090 and presented in Figure 6

Table 3 Continued

n 100 09 099Sigma k d OLS Ridge Liu New est OLS Ridge Liu New est

05 78091 78178 74646 119870 120239 11095306 77415 78862 73336 118144 122001 10767407 76750 79549 72053 116462 123778 10450608 76096 80240 70797 114821 125570 10144709 75451 80934 69568 113220 127377 984901 74816 81632 68364 111658 129200 95634

Table 4 Estimated MSE when n 100 p 3 and ρ 090 and 099

n 30 09 099Sigma k d OLS Ridge Liu New est OLS Ridge Liu New est1 01 0287 0278 0230 0270 3072 2141 0688 1423

02 0270 0236 0255 1621 0836 076903 0263 0241 0242 1298 1013 052804 0256 0247 0231 1083 1219 047205 0250 0253 0221 0930 1455 050606 0244 0259 0213 0819 1720 058307 0239 0265 0206 0733 2014 068008 0234 0272 0200 0667 2337 078509 0230 0279 0195 0613 2690 08921 0226 0287 0191 0570 3072 0997

5 01 6958 6719 5256 6486 76772 53314 14746 3468902 6495 5431 6050 39905 18971 1666003 6283 5610 5649 31412 23862 883404 6083 5792 5278 25626 29420 580305 5893 5977 4935 21466 35645 517406 5714 6166 4617 18350 42537 579507 5544 6359 4324 15939 50096 707208 5383 6555 4052 14024 58321 868609 5230 6754 3799 12471 67213 104581 5085 6958 3566 11189 76772 12287

10 01 27809 26853 20970 25916 307086 213255 58717 13868502 25951 21675 24167 159582 75683 6635403 25100 22394 22551 125559 95308 3481504 24296 23126 21056 102365 117590 2246305 23535 23872 19672 85681 142529 1974306 22815 24632 18389 73175 170126 2204507 22131 25406 17200 63493 200380 2699508 21482 26193 16096 55802 233291 3330809 20865 26994 15071 49561 268860 402701 20279 27809 14120 44407 307086 47470

Scientifica 7

Table 5 Estimated MSE when n 30 p 7 and ρ 070 and 080

n 30 07 08Sigma k d OLS Ridge Liu New ridge OLS Ridge Liu New ridge1 01 0838 0811 0651 0785 1239 1179 0859 1121

02 0786 0670 0737 1124 0895 101803 0763 0689 0694 1074 0933 092804 0741 0709 0654 1029 0973 085005 0720 0729 0618 0987 1014 078106 0701 0750 0586 0949 1056 072107 0682 0771 0556 0914 1100 066908 0665 0793 0529 0881 1145 062309 0649 0815 0505 0851 1191 05831 0633 0838 0484 0823 1239 0549

5 01 20955 20275 16063 19608 30981 29455 21084 2797502 19633 16568 18362 28060 22071 2531403 19026 17083 17208 26780 23086 2295104 18452 17607 16139 25602 24130 2084505 17908 18141 15147 24513 25201 1896306 17391 18685 14226 23506 26301 1727907 16901 19238 13369 22570 27429 1576708 16435 19801 12572 21699 28585 1440809 15990 20373 11829 20885 29769 131851 15567 20955 11137 20125 30981 12081

10 01 83821 81095 64205 78423 123923 117811 84259 11188702 78523 66233 73429 112224 88219 10122503 76091 68299 68804 107097 92291 9174904 73789 70403 64513 102375 96475 8330105 71608 72545 60530 98014 100770 7575006 69537 74725 56827 93973 105177 6898307 67569 76942 53382 90220 109696 6290808 65698 79197 50173 86725 114327 5744109 63915 81490 47182 83463 119069 525151 62215 83821 44392 80411 123923 48069

Table 6 Estimated MSE when n 30 p 7 and ρ 09 and 099

N 30 09 099Sigma k d OLS Ridge Liu New ridge OLS Ridge Liu New ridge1 01 252 227 129 203 2868 1120 226 445

02 206 139 166 682 355 41603 188 151 137 478 525 57804 173 163 116 362 736 75805 161 176 099 288 989 92506 150 190 085 237 1283 107507 141 204 075 201 1617 120708 132 220 068 174 1993 132409 125 235 062 154 2410 14271 118 252 057 138 2868 1519

5 01 6303 5658 3123 5057 71709 27885 5083 1080002 5127 3411 4103 16823 8411 971703 4682 3715 3361 11638 12757 1345404 4303 4035 2778 8652 18123 1768905 3977 4372 2314 6736 24507 2164706 3694 4725 1942 5418 31910 2518707 3445 5095 1643 4467 40332 2831608 3225 5481 1401 3754 49772 3108009 3028 5884 1206 3206 60231 335291 2852 6303 1048 2773 71709 35710

10 01 25214 22630 12475 20223 286835 111506 20239 4314802 20503 13628 16403 67243 33562 38784

8 Scientifica

32 Simulation Results and Discussion From Tables 1ndash8and Figures 1ndash6 it appears that as the values of σ increasethe MSE values also increase (Figure 3) while the sample sizeincreases as the MSE values decrease (Figure 4) Ridge Liuand proposed KL estimators uniformly dominate the ordinaryleast squares (OLS) estimator In general from these tables anincrease in the levels of multicollinearity and the number ofexplanatory variables increase the estimated MSE values of theestimators +e figures consistently show that the OLS esti-mator performs worst when there is multicollinearity FromFigures 1ndash6 and simulation Tables 1ndash8 it clearly indicated thatfor ρ 090 or less the proposed estimator uniformly

dominates the ridge regression estimator while Liu performedmuch better than both proposed and ridge estimators for smalld say 03 or lessWhen ρ 099 the ridge regression performsthe best for higher k while the proposed estimator performsthe best for say k (say 03 or less) When d k 05 andρ 099 both ridge and KL estimators outperform the Liuestimator None of the estimators uniformly dominates eachother However it appears that our proposed estimator KLperforms better in the wider space of d k in the parameterspace If we review all Tables 1ndash8 we observed that theconclusions about the performance of all estimators remainthe same for both p 3 and p 7

Table 6 Continued

N 30 09 099Sigma k d OLS Ridge Liu New ridge OLS Ridge Liu New ridge

03 18721 14846 13432 46491 50959 5370704 17205 16130 11091 34538 72431 7062605 15899 17479 9229 26865 97978 8644306 14763 18895 7737 21588 127600 10058607 13766 20376 6534 17777 161296 11308908 12882 21923 5562 14924 199068 12413409 12095 23535 4775 12725 240914 1339211 11389 25214 4138 10992 286835 142634

Table 7 Estimated MSE when n 100 p 7 and ρ 070 and 080

n 100 07 08Sigma k d OLS Ridge Liu New ridge OLS Ridge Liu New ridge1 01 0174 0173 0163 0171 0263 0259 0235 0255

02 0171 0164 0169 0255 0238 024903 0170 0165 0166 0252 0241 024304 0169 0166 0164 0249 0244 023705 0167 0168 0161 0246 0247 023206 0166 0169 0159 0243 0250 022707 0165 0170 0157 0240 0253 022208 0164 0171 0155 0238 0256 021809 0163 0173 0154 0235 0259 02141 0162 0174 0152 0233 0263 0210

5 01 4356 4320 4055 4284 6563 6474 5852 638602 4285 4087 4214 6388 5928 621603 4250 4120 4146 6304 6005 605304 4216 4153 4079 6222 6082 589505 4182 4187 4013 6143 6160 574406 4149 4220 3949 6066 6239 559807 4116 4254 3887 5991 6319 545708 4084 4288 3826 5917 6399 532209 4053 4322 3767 5846 6481 51911 4022 4356 3708 5777 6563 5066

10 01 17425 17281 16219 17138 26250 25896 23408 2554502 17140 16350 16858 25551 23713 2486603 17001 16482 16584 25216 24020 2421204 16864 16614 16316 24891 24330 2358205 16729 16748 16054 24573 24643 2297506 16597 16882 15797 24265 24959 2238907 16467 17016 15547 23964 25277 2182508 16339 17152 15301 23671 25599 2128009 16213 17288 15062 23385 25923 207551 16089 17425 14827 23107 26250 20247

Scientifica 9

Table 8 Estimated MSE when n 100 p 7 and ρ 090 and 099

n 100 09 099Sigma k d OLS Ridge Liu New ridge OLS Ridge Liu New ridge1 01 0546 0529 0431 0512 6389 4391 1624 2949

02 0513 0442 0482 3407 1934 183603 0498 0454 0456 2819 2298 145304 0485 0466 0432 2423 2718 134705 0472 0478 0411 2135 3192 135906 0460 0491 0392 1914 3721 142607 0449 0504 0375 1738 4306 151908 0439 0517 0360 1593 4945 162509 0429 0531 0346 1472 5640 17371 0420 0546 0334 1370 6389 1851

5 01 13640 13216 10676 12802 159732 109722 38895 7328402 12820 10979 12037 84915 47018 4450603 12448 11289 11336 69971 56467 3386504 12099 11605 10693 59823 67242 3014605 11770 11928 10102 52370 79343 2941706 11460 12257 9558 46597 92769 3009007 11168 12593 9056 41953 107521 3145508 10891 12935 8593 38114 123599 3317109 10628 13284 8165 34875 141003 350631 10379 13640 7768 32097 159732 37036

10 01 54558 52866 42699 51212 638928 438910 155399 29312102 51282 43914 48150 339663 187945 17787403 49796 45155 45344 279860 225785 13515104 48399 46422 42768 239236 268921 12012005 47084 47714 40397 209391 317351 11705306 45843 49032 38214 186265 371077 11959907 44670 50375 36198 167659 430097 12492208 43560 51744 34336 152274 494412 13165409 42508 53138 32612 139287 564022 1390941 41509 54558 31014 128149 638928 146866

OLSRidge

LiuKL

025

030

035

040

MSE

02 04 06 08 1000d = k

(a)

OLSRidge

LiuKL

02 04 06 08 1000d = k

20

25

30

35

MSE

(b)

Figure 1 Continued

10 Scientifica

4 Numerical Examples

To illustrate our theoretical results we consider two datasets(i) famous Portland cement data originally adopted byWoods et al [34] and (ii) French economy data from

Chatterjee and Hadi [35] and they are analyzed in thefollowing sections respectively

41 Example 1 Portland Data +ese data are widely knownas the Portland cement dataset It was originally adopted by

OLSRidge

LiuKL

025

035

045

055

MSE

02 04 06 08 1000d = k

(c)

OLSRidge

LiuKL

02 04 06 08 1000d = k

20

30

40

50

MSE

(d)

Figure 1 Estimated MSEs for n 30 Sigma 1 10 rho 070 080 and different values of k d (a) n 30 p 3 sigma 1 and rho 070(b) n 30 p 3 sigma 10 and rho 070 (c) n 30 p 3 sigma 1 and rho 080 (d) n 30 p 3 sigma 10 and rho 080

OLSRidge

LiuKL

04

06

08

10

12

MSE

02 04 06 08 1000d = k

(a)

OLSRidge

LiuKL

20

40

60

80

100M

SE

02 04 06 08 1000d = k

(b)

OLSRidge

LiuKL

02468

101214

MSE

02 04 06 08 1000d = k

(c)

OLSRidge

LiuKL

0200

600

1000

1400

MSE

02 04 06 08 1000d = k

(d)

Figure 2 Estimated MSEs for n 30 sigma 1 10 rho 090 099 and different values of k d (a) n 30 p 3 sigma 1 and rho 090(b) n 30 p 3 sigma 10 and rho 090 (c) n 30 p 3 sigma 1 and rho 099 (d) n 30 p 3 sigma 10 and rho 099

Scientifica 11

OLSRidge

LiuKL

0

10

20

30

MSE

2 4 6 8 100Sigma

(a)

OLSRidge

LiuKL

0

10

30

50

MSE

2 4 6 8 100Sigma

(b)

OLSRidge

LiuKL

0

40

80

120

MSE

2 4 6 8 100Sigma

(c)

OLSRidge

LiuKL

2 4 6 8 100Sigma

0

400

800

1200

MSE

(d)

Figure 3 EstimatedMSEs for n 30 d 05 and different values of rho and sigma (a) n 30 p 3 d 05 and rho 070 (b) n 30 p 3d 05 and rho 080 (c) n 30 p 3 d 05 and rho 090 (d) n 30 p 3 d 05 and rho 099

OLSRidge

LiuKL

50 70 9030n

010

020

030

040

MSE

(a)

OLSRidge

LiuKL

50 70 9030n

02

04

06

MSE

(b)

Figure 4 Continued

12 Scientifica

Woods et al [34] It has also been analyzed by the followingauthors Kaciranlar et al [36] Li and Yang [25] and recentlyby Lukman et al [13] +e regression model for these data isdefined as

yi β0 + β1X1 + β2X2 + β3X3 + β4X4 + εi (46)

where yi heat evolved after 180 days of curing measured incalories per gram of cement X1 tricalcium aluminateX2 tricalcium silicate X3 tetracalcium aluminoferriteand X4 β-dicalcium silicate +e correlation matrix of thepredictor variables is given in Table 9

OLSRidge

LiuKL

50 70 9030n

02

06

10

MSE

(c)

OLSRidge

LiuKL

50 70 9030n

0

2

4

6

8

12

MSE

(d)

Figure 4 Estimated MSEs for sigma 1 p 3 and different values of rho and sample size (a)p 3 sigma 1 d 05 and rho 070(b)p 3 sigma 1 d 05 and rho 080 (c)p 3 sigma 1 d 05 and rho 090 (d)p 3 sigma 1 d 05 and rho 099

OLSRidge

LiuKL

20

30

40

MSE

4 5 6 7 83p

(a)

OLSRidge

LiuKL

3

4

5

6

7M

SE

4 5 6 7 83p

(b)

6

8

10

14

MSE

OLSRidge

LiuKL

4 5 6 7 83p

(c)

OLSRidge

LiuKL

4 5 6 7 83p

0

50

100

150

MSE

(d)

Figure 5 Estimated MSEs for n 100 d 05 sigma 5 and different values of rho and p (a) n 100 sigma 5 d 05 and rho 070 (b)n 100 sigma 5 d 05 and rho 080 (c) n 100 sigma 5 d 05 and rho 090 (d) n 100 sigma 5 d 05 and rho 099

Scientifica 13

+e variance inflation factors are VIF1 = 3850VIF2 = 25442 VIF3 = 4687 and VIF4 = 28251 Eigen-values of XprimeX are λ1 44676206 λ2 5965422

λ3 809952 and λ4 105419 and the condition numberof XprimeX is approximately 424 +e VIFs the eigenvalues

and the condition number all indicate the presence ofsevere multicollinearity +e estimated parameters andMSE are presented in Table 10 It appears from Table 11that the proposed estimator performed the best in thesense of smaller MSE

OLSRidge

LiuKL

0

100

200

300

MSE

075 085 095065Rho

(a)

OLSRidge

LiuKL

0

200

400

600

800

MSE

075 085 095065Rho

(b)

OLSRidge

LiuKL

0

20

40

60

80

MSE

075 085 095065Rho

(c)

OLSRidge

LiuKL

0

50

100

150

MSE

075 085 095065Rho

(d)

Figure 6 Estimated MSEs for n 100 p 3 7 d 05 sigma 5 and different values of rho (a) n 30 p 3 sigma 5 and d 05 (b)n 30 p 7 sigma 5 and d 05 (c) n 100 p 3 sigma 5 and d 05 (d) n 100 p 7 sigma 5 and d 05

Table 9 Correlation matrix

X1 X2 X3 X4

X1 1000 0229 minus 0824 minus 0245X2 0229 1000 minus 0139 minus 0973X3 minus 0824 minus 0139 1000 0030X4 minus 0245 minus 0973 0030 1000

Table 10 +e results of regression coefficients and the corresponding MSE values

Coef 1113954α 1113954α(k) 1113954α(d)1113954dalt 1113954αKL(kHNM) 1113954αKL(kmin)

α0 624054 85870 276490 minus 197876 276068α1 15511 21046 19010 23965 19090α2 05102 10648 08701 13573 08688α3 01019 06681 04621 09666 04680α4 minus 01441 03996 02082 06862 02074MSE 491209 298983 2170963 7255603 217096kd mdash 00077 044195 000235 000047

14 Scientifica

42 Example 2 French Economy Data +e French economydata in Chatterjee and Hadi [37] are considered in this ex-ample It has been analyzed by Malinvard [38] and Liu [6]among others+e variables are imports domestic productionstock formation and domestic consumption All are measuredin milliards of French francs for the years 1949 through 1966

+e regression model for these data is defined as

yi β0 + β1X1 + β2X2 + β3X3 + εi (47)

where yi IMPORT X1 domestic production X2 stockformation and X3 domestic consumption +e correlationmatrix of the predicted variable is given in Table 12

+e variance inflation factors areVIF1 469688VIF2 1047 and VIF3 469338 +e ei-genvalues of the XprimeX matrix are λ1 161779 λ2 158 andλ3 4961 and the condition number is 32612 If we reviewthe above correlation matrix VIFs and condition number itcan be said that there is presence of severe multicollinearityexisting in the predictor variables

+e biasing parameter for the new estimator is defined in(41) and (42) +e biasing parameter for the ridge and Liuestimator is provided in (6) (8) and (9) respectively

We analyzed the data using the biasing parameters foreach of the estimators and presented the results in Tables 10and 11 It can be seen from Tables 10 and 11 that theproposed estimator performed the best in the sense ofsmaller MSE

5 Summary and Concluding Remarks

In this paper we introduced a new biased estimator toovercome the multicollinearity problem for the multiplelinear regression model and provided the estimation tech-nique of the biasing parameter A simulation study has beenconducted to compare the performance of the proposedestimator and Liu [6] and ridge regression estimators [3]Simulation results evidently show that the proposed esti-mator performed better than both Liu and ridge under somecondition on the shrinkage parameter Two sets of real-lifedata are analyzed to illustrate the benefits of using the newestimator in the context of a linear regression model +eproposed estimator is recommended for researchers in this

area Its application can be extended to other regressionmodels for example logistic regression Poisson ZIP andrelated models and those possibilities are under currentinvestigation [37 39 40]

Data Availability

Data will be made available on request

Conflicts of Interest

+e authors declare that they have no conflicts of interest

Acknowledgments

We are dedicating this article to those who lost their livesbecause of COVID-19

References

[1] C Stein ldquoInadmissibility of the usual estimator for mean ofmultivariate normal distributionrdquo in Proceedings of the 0irdBerkley Symposium on Mathematical and Statistics Proba-bility J Neyman Ed vol 1 pp 197ndash206 Springer BerlinGermany 1956

[2] W F Massy ldquoPrincipal components regression in exploratorystatistical researchrdquo Journal of the American Statistical As-sociation vol 60 no 309 pp 234ndash256 1965

[3] A E Hoerl and R W Kennard ldquoRidge regression biasedestimation for nonorthogonal problemsrdquo Technometricsvol 12 no 1 pp 55ndash67 1970

[4] L S Mayer and T A Willke ldquoOn biased estimation in linearmodelsrdquo Technometrics vol 15 no 3 pp 497ndash508 1973

[5] B F Swindel ldquoGood ridge estimators based on prior infor-mationrdquo Communications in Statistics-0eory and Methodsvol 5 no 11 pp 1065ndash1075 1976

[6] K Liu ldquoA new class of biased estimate in linear regressionrdquoCommunication in Statistics- 0eory and Methods vol 22pp 393ndash402 1993

[7] F Akdeniz and S Kaccediliranlar ldquoOn the almost unbiasedgeneralized liu estimator and unbiased estimation of the biasand mserdquo Communications in Statistics-0eory and Methodsvol 24 no 7 pp 1789ndash1797 1995

[8] M R Ozkale and S Kaccediliranlar ldquo+e restricted and unre-stricted two-parameter estimatorsrdquo Communications in Sta-tistics-0eory and Methods vol 36 no 15 pp 2707ndash27252007

[9] S Sakallıoglu and S Kaccedilıranlar ldquoA new biased estimatorbased on ridge estimationrdquo Statistical Papers vol 49 no 4pp 669ndash689 2008

Table 11 +e results of regression coefficients and the corresponding MSE values

Coef 1113954α 1113954α(k) 1113954α(d)1113954dalt 1113954α(d)1113954dopt 1113954αKL(kHNM) 1113954αKL(kmin)

α0 minus 197127 minus 167613 minus 125762 minus 188410 minus 165855 minus 188782α1 00327 01419 02951 00648 01485 00636α2 04059 03576 02875 03914 03548 03922α3 02421 00709 minus 01696 01918 00606 01937MSE 173326 2130519 5828312 1660293 2211899 1660168kd mdash 00527 05282 09423 00258 00065

Table 12 Correlation matrix

X1 X2 X3

X1 1000 0210 0999X2 0210 1000 0208X3 0999 0208 1000

Scientifica 15

[10] H Yang and X Chang ldquoA new two-parameter estimator inlinear regressionrdquo Communications in Statistics-0eory andMethods vol 39 no 6 pp 923ndash934 2010

[11] M Roozbeh ldquoOptimal QR-based estimation in partially linearregression models with correlated errors using GCV crite-rionrdquo Computational Statistics amp Data Analysis vol 117pp 45ndash61 2018

[12] F Akdeniz and M Roozbeh ldquoGeneralized difference-basedweightedmixed almost unbiased ridge estimator in partially linearmodelsrdquo Statistical Papers vol 60 no 5 pp 1717ndash1739 2019

[13] A F Lukman K Ayinde S Binuomote and O A ClementldquoModified ridge-type estimator to combat multicollinearityapplication to chemical datardquo Journal of Chemometricsvol 33 no 5 p e3125 2019

[14] A F Lukman K Ayinde S K Sek and E Adewuyi ldquoAmodified new two-parameter estimator in a linear regressionmodelrdquo Modelling and Simulation in Engineering vol 2019Article ID 6342702 10 pages 2019

[15] A E Hoerl R W Kannard and K F Baldwin ldquoRidge re-gressionsome simulationsrdquo Communications in Statisticsvol 4 no 2 pp 105ndash123 1975

[16] G C McDonald and D I Galarneau ldquoA monte carlo eval-uation of some ridge-type estimatorsrdquo Journal of the Amer-ican Statistical Association vol 70 no 350 pp 407ndash416 1975

[17] J F Lawless and P Wang ldquoA simulation study of ridge andother regression estimatorsrdquo Communications in Statistics-0eory and Methods vol 5 no 4 pp 307ndash323 1976

[18] D W Wichern and G A Churchill ldquoA comparison of ridgeestimatorsrdquo Technometrics vol 20 no 3 pp 301ndash311 1978

[19] B M G Kibria ldquoPerformance of some new ridge regressionestimatorsrdquo Communications in Statistics-Simulation andComputation vol 32 no 1 pp 419ndash435 2003

[20] A F Lukman and K Ayinde ldquoReview and classifications ofthe ridge parameter estimation techniquesrdquoHacettepe Journalof Mathematics and Statistics vol 46 no 5 pp 953ndash967 2017

[21] A K M E Saleh M Arashi and B M G Kibria 0eory ofRidge Regression Estimation with Applications WileyHoboken NJ USA 2019

[22] K Liu ldquoUsing Liu-type estimator to combat collinearityrdquoCommunications in Statistics-0eory and Methods vol 32no 5 pp 1009ndash1020 2003

[23] K Alheety and B M G Kibria ldquoOn the Liu and almostunbiased Liu estimators in the presence of multicollinearitywith heteroscedastic or correlated errorsrdquo Surveys in Math-ematics and its Applications vol 4 pp 155ndash167 2009

[24] X-Q Liu ldquoImproved Liu estimator in a linear regressionmodelrdquo Journal of Statistical Planning and Inference vol 141no 1 pp 189ndash196 2011

[25] Y Li and H Yang ldquoA new Liu-type estimator in linear regressionmodelrdquo Statistical Papers vol 53 no 2 pp 427ndash437 2012

[26] B Kan O Alpu and B Yazıcı ldquoRobust ridge and robust Liuestimator for regression based on the LTS estimatorrdquo Journalof Applied Statistics vol 40 no 3 pp 644ndash655 2013

[27] R A Farghali ldquoGeneralized Liu-type estimator for linearregressionrdquo International Journal of Research and Reviews inApplied Sciences vol 38 no 1 pp 52ndash63 2019

[28] S G Wang M X Wu and Z Z Jia Matrix InequalitiesChinese Science Press Beijing China 2nd edition 2006

[29] R W Farebrother ldquoFurther results on the mean square errorof ridge regressionrdquo Journal of the Royal Statistical SocietySeries B (Methodological) vol 38 no 3 pp 248ndash250 1976

[30] G Trenkler and H Toutenburg ldquoMean squared error matrixcomparisons between biased estimators-an overview of recentresultsrdquo Statistical Papers vol 31 no 1 pp 165ndash179 1990

[31] B M G Kibria and S Banik ldquoSome ridge regression esti-mators and their performancesrdquo Journal of Modern AppliedStatistical Methods vol 15 no 1 pp 206ndash238 2016

[32] D G Gibbons ldquoA simulation study of some ridge estimatorsrdquoJournal of the American Statistical Association vol 76no 373 pp 131ndash139 1981

[33] J P Newhouse and S D Oman ldquoAn evaluation of ridgeestimators A report prepared for United States air forceproject RANDrdquo 1971

[34] H Woods H H Steinour and H R Starke ldquoEffect ofcomposition of Portland cement on heat evolved duringhardeningrdquo Industrial amp Engineering Chemistry vol 24no 11 pp 1207ndash1214 1932

[35] S Chatterjee and A S Hadi Regression Analysis by ExampleWiley Hoboken NJ USA 1977

[36] S Kaciranlar S Sakallioglu F Akdeniz G P H Styan andH J Werner ldquoA new biased estimator in linear regression anda detailed analysis of the widely-analysed dataset on portlandcementrdquo Sankhya 0e Indian Journal of Statistics Series Bvol 61 pp 443ndash459 1999

[37] S Chatterjee and A S Haadi Regression Analysis by ExampleWiley Hoboken NJ USA 2006

[38] E Malinvard Statistical Methods of Econometrics North-Holland Publishing Company Amsterdam Netherlands 3rdedition 1980

[39] D N Gujarati Basic Econometrics McGraw-Hill New YorkNY USA 1995

[40] A F Lukman K Ayinde and A S Ajiboye ldquoMonte Carlostudy of some classification-based ridge parameter estima-torsrdquo Journal of Modern Applied Statistical Methods vol 16no 1 pp 428ndash451 2017

16 Scientifica

where (W(k) minus Ip) minus k(Λ + kIp)minus 1+e Liu estimator of α is

1113954α(d) Λ + Ip1113872 1113873minus 1

ZprimeY + d1113954α( 1113857 F(d)1113954α (22)

where F(d) [Λ + Ip]minus 1[Λ + dIp]

MSEM(1113954α(d)) σ2FdΛminus 1

Fd +(1 minus d)2(Λ + I)

minus 1

middot ααprime(Λ + I)minus 1

(23)

where Fd (Λ + I)minus 1(Λ + dI)+e proposed one-parameter estimator of α is

1113954αKL Λ + kIp1113872 1113873minus 1Λ minus kIp1113872 11138731113954α W(k)M(k)1113954α (24)

where W(k) [Ip + kΛminus 1]minus 1 and M(k) [Ip minus kΛminus 1]+e following notations and lemmas are needful to prove

the statistical property of 1113954αKL

Lemma 1 Let ntimes n matrices Mgt 0 and Ngt 0 (or Nge 0)then MgtN if and only if λ1 (NMminus 1)lt 1 where λ1 (NMminus 1) isthe largest eigenvalue of matrix NMminus 1 [28]

Lemma 2 Let M be an ntimes n positive definite matrix that isMgt 0 and α be some vector then M minus ααprime ge 0 if and only ifαprimeMminus 1αle 1 [29]

Lemma 3 Let 1113954αi Aiy i 1 2 be two linear estimators ofα Suppose that D Cov(1113954α1) minus Cov(1113954α2)gt 0 whereCov(1113954αi) i 1 2 denotes the covariance matrix of 1113954αi andbi Bias(1113954αi) (AiX minus I)α i 1 2 Consequently

Δ 1113954α1 minus 1113954α2( 1113857 MSEM 1113954α1( 1113857 minus MSEM 1113954α2( 1113857 σ2D + b1b2prime

minus b2b2prime gt 0(25)

if and only if b2prime[σ2D + b1b1prime]minus 1b2 lt 1 where MSEM(1113954αi)

Cov(1113954αi) + bibiprime [30]

+e other parts of this article are as follows +e theo-retical comparison among the estimators and estimation ofthe biasing parameters are given in Section 2 A simulationstudy has been constructed in Section 3 We conducted twonumerical examples in Section 4 +is paper ends up withconcluding remarks in Section 5

2 Comparison among the Estimators

21 Comparison between 1113954α and 1113954αKL +e difference betweenMSEM(1113954α) and MSEM(1113954αKL) is

MSEM[1113954α] minus MSEM 1113954αKL1113858 1113859 σ2Λminus 1minus σ2W(k)M(k)Λminus 1

Mprime(k)Wprime(k)

minus W(k)M(k) minus Ip1113960 1113961ααprime W(k)M(k) minus Ip1113960 1113961prime(26)

We have the following theorem Theorem 1 If kgt 0 estimator 1113954αKL is superior to estimator1113954α using the MSEM criterion that is MSEM[1113954α] minus

MSEM[1113954αKL]gt 0 if and only if

αprime W(k)M(k) minus Ip1113960 1113961prime σ2 Λminus 1minus W(k)M(k)Λminus 1

Mprime(k)W(k)k1113872 11138731113960 1113961 W(k)M(k) minus Ip1113960 1113961αlt 1 (27)

Proof +e difference between (15) and (19) is

D(1113954α) minus D 1113954αKL( 1113857 σ2 Λminus 1minus W(k)M(k)Λminus 1

Mprime(k)Wprime(k)1113872 1113873

σ2diag1λi

minusλi minus k( 1113857

2

λi λi + k( 11138572

⎧⎨

⎩

⎫⎬

⎭

p

i1

(28)

where Λminus 1 minus W(k)M(k)Λminus 1Mprime(k)Wprime(k) will be positivedefinite (pd) if and only if (λi + k)2 minus (λi minus k)2 gt 0 Weobserved that for kgt 0 (λi + k)2 minus (λi minus k)2 4λikgt 0

Consequently Λminus 1 minus W(k)M(k)Λminus 1Mprime(k)Wprime(k) is pd

22 Comparison between 1113954α(k) and 1113954αKL +e difference be-tween MSEM(1113954α(k)) and MSEM(1113954αKL) is

MSEM[1113954α(k)] minus MSEM 1113954αk1113858 1113859 σ2W(k)Λminus 1W(k) minus σ2W(k)M(k)Λminus 1

Mprime(k)W(k)

+ W(k) minus Ip1113872 1113873αα W(k) minus Ip1113872 1113873prime minus W(k)M(k) minus Ip1113960 1113961ααprime W(k)M(k) minus Ip1113960 1113961prime(29)

Scientifica 3

Theorem 2 When λmax(HGminus 1)lt 1 estimator 1113954αKL is supe-rior to 1113954α(k) in the MSEM sense if and only if

αprime W(k)M(k) minus Ip1113960 1113961prime V1 + W(k) minus Ip1113872 1113873ααprime W(k) minus Ip1113872 111387311138731113960 1113961

W(k)M(k) minus Ip1113960 1113961α

(30)

λmax HGminus 1

1113872 1113873lt 1 (31)

where

V1 σ2W(k)Λminus 1W(k) minus σ2W(k)M(k)Λminus 1

Mprime(k)W(k)

H 2W(k)

G kW(k)Λminus 1W(k)

(32)

Proof Using the dispersion matrix difference

V1 σ2W(k)Λminus 1W(k) minus σ2W(k)M(k)Λminus 1

Mprime(k)W(k)

σ2kΛminus 1 ΛW(k)Λminus 1W(k) + ΛW(k)Λminus 1

W(k)1113872

minus kW(k)Λminus 1W(k)1113873Λminus 1

σ2W(k)Λminus 1W(k) minus σ2W(k) Ip minus kΛminus 1

1113960 1113961

middot Λminus 1Ip minus kΛminus 1

1113960 1113961W(k)

σ2kΛminus 1(G minus H)Λminus 1

(33)

It is obvious that for kgt 0Ggt 0 andHgt 0 According toLemma 1 it is clear that G-Hgt 0 if and only if HGminus 1 lt 1where λmax(HGminus 1)lt 1 is the maximum eigenvalue of thematrix HGminus 1 Consequently V1 is pd

23 Comparison between 1113954α(d) and 1113954αKL +e difference be-tween MSEM(1113954α(d)) and MSEM(1113954αKL) is

MSEM[1113954α] minus MSEM 1113954αk1113858 1113859 σ2FdΛminus 1

Fd minus σ2W(k)M(k)Λminus 1Mprime(k)Wprime(k)

+(1 minus d)2(Λ + I)

minus 1ααprime(Λ + I)minus 1

minus W(k)M(k) minus Ip1113960 1113961ααprime W(k)M(k) minus Ip1113960 1113961prime(34)

We have the following theorem

Theorem 3 If kgt 0 and 0lt dlt 1 estimator 1113954αKL is superior toestimator 1113954α(d) using the MSEM criterion that isMSEM(1113954α(d)) minus MSEM(1113954αKL)gt 0 if and only if

αprime W(k)M(k) minus Ip1113960 1113961prime V2 +(1 minus d)2(Λ + I)

minus 1ααprime(Λ + I)minus 1

1113960 1113961

middot W(k)M(k) minus Ip1113960 1113961αlt 1

(35)

where V2 σ2FdΛminus 1Fd minus σ2W(k)M(k)Λminus 1Mprime(k)W(k)

Proof Using the difference between the dispersion matrix

V2 σ2FdΛminus 1

Fd minus σ2W(k)M(k)Λminus 1Mprime(k)W(k)

σ2 FdΛminus 1

Fd minus W(k)M(k)Λminus 1Mprime(k)W(k)1113872 1113873

σ2 Λ + Ip1113960 1113961minus 1 Λ + dIp1113960 1113961Λminus 1 Λ + Ip1113960 1113961

minus 1 Λ + dIp1113960 1113961

minus Λ(Λ + k)minus 1Λminus 1

middot (Λ minus k)Λminus 1Λminus 1(Λ minus k)Λ(Λ + k)

minus 1

(36)

where W(k) [Ip + kΛminus 1]minus 1 Λ(Λ + k)minus 1 andM(k) [Ip minus kΛminus 1] Λminus 1(Λ minus k)

σ2diagλi + d( 1113857

2

λi λi + 1( 11138572 minus

λi minus k( 11138572

λi λi + k( 11138572

⎧⎨

⎩

⎫⎬

⎭

p

i1

(37)

We observed that FdΛminus 1Fd minus W(k)M(k)Λminus 1Mprime(k)

W(k) is pd if and only if (λi + d)2(λi + k)2 minus

(λi minus k)2(λi + 1)2 gt 0 or (λi + d) (λi + k) minus (λi minus k)

(λi + 1)gt 0 Obviously for kgt 0 and 0lt dlt 1 (λi + d)(λi +

k) minus (λi minus k)(λi + 1) k(2λ + d + 1) + λ(d minus 1)gt 0 Conse-quently FdΛminus 1Fd minus W(k)M(k)Λminus 1Mprime(k)W(k) is pd

24 Determination of Parameter k +ere is a need to esti-mate the parameter of the new estimator for practical use+e ridge biasing parameter and the Liu shrinkage pa-rameter were determined by both Hoerl and Kennard [3]and Liu [6] respectively Different authors have developedother estimators of these ridge parameters To mention afew these include Hoerl et al [15] Kibria [19] Kibria andBanik [31] and Lukman and Ayinde [20] among others+eoptimal value of k is the one that minimizes

MSEM 1113954βKL1113872 1113873 σ2W(k)M(k)Sminus 1

Mprime(k)Wprime(k)

+ W(k)M(k) minus Ip1113960 1113961ββprime W(k)M(k) minus Ip1113960 1113961prime

p(k) MSEM 1113954αKL1113858 1113859 tr MSEM 1113954αKL( 11138571113858 1113859

p(k) σ2 1113944

p

i1

λi minus k( 11138572

λi λi + k( 11138572 + 4k

21113944

p

i1

α2iλi + k( 1113857

2

(38)

Differentiatingm(k d)with respect to k gives and setting(zp(k)zk) 0 we obtain

k σ2

2α2i + σ2λi( 1113857 (39)

4 Scientifica

+e optimal value of k in (39) depends on the unknownparameter σ2 and α2 +ese two estimators are replaced withtheir unbiased estimate Consequently we have

1113954k 1113954σ2

21113954α2i + 1113954σ2λi1113872 1113873 (40)

Following Hoerl et al [15] the harmonic-mean versionof (40) is defined as

1113954kHMN p1113954σ2

1113936pi1 21113954α2i + 1113954σ2λi1113872 11138731113960 1113961

(41)

According to Ozkale and Kaccediliranlar [8] the minimumversion of (41) is defined as

1113954kmin min1113954σ2

21113954α2i + 1113954σ2λi1113872 1113873⎡⎢⎣ ⎤⎥⎦ (42)

3 Simulation Study

Since theoretical comparisons among the estimators ridgeregression Liu and KL in Section 2 give the conditionaldominance among the estimators a simulation study hasbeen conducted using the R 341 programming languages tosee a better picture about the performance of the estimators

31 Simulation Technique +e design of the simulationstudy depends on factors that are expected to affect theproperties of the estimator under investigation and thecriteria being used to judge the results Since the degree ofcollinearity among the explanatory variable is of centralimportance following Gibbons [32] and Kibria [19] wegenerated the explanatory variables using the followingequation

xij 1 minus ρ21113872 111387312

zij + ρzip+1

i 1 2 n j 1 2 3 p(43)

where zij are independent standard normal pseudo-randomnumbers and ρ represents the correlation between any twoexplanatory variables We consider p 3 and 7 in thesimulation+ese variables are standardized so that XprimeX andXprimey are in correlation forms +e n observations for thedependent variable y are determined by the followingequation

yi β0 + β1xi1 + β2xi2 + β3xi3 + middot middot middot + βpxip + ei

i 1 2 n(44)

where ei are iidN (0 σ2) and without loss of any generalitywe will assume zero intercept for the model in (44) +evalues of β are chosen such that βprimeβ 1 [33] Since our main

Table 1 Estimated MSE when n 30 p 3 and ρ 070 and 080

n 30 07 08Sigma k d OLS Ridge Liu New est OLS Ridge Liu New est1 01 0362 0352 0291 0342 0547 0519 0375 0491

02 0342 0298 0323 0493 0391 044403 0333 0305 0307 0470 0407 040404 0325 0312 0293 0449 0425 037005 0317 0320 0280 0431 0443 034206 0309 0328 0268 0414 0462 031807 0302 0336 0258 0398 0482 029908 0296 0344 0249 0384 0503 028209 0290 0353 0242 0372 0525 026910 0284 0362 0235 0360 0547 0258

5 01 8021 7759 6137 7501 12967 12232 8364 1152202 7511 6331 7021 11567 8817 1026103 7277 6529 6577 10962 9284 915604 7056 6731 6165 10411 9766 818605 6846 6937 5784 9907 10263 733306 6647 7146 5430 9445 10775 658107 6459 7359 5102 9019 11301 591808 6280 7576 4797 8626 11842 533109 6109 7797 4513 8263 12397 481310 5947 8021 4250 7926 12967 4354

10 01 31993 30939 24421 29907 51819 48871 33333 4602202 29945 25203 27977 46201 35155 4095503 29005 26000 26189 43775 37034 3651404 28116 26812 24532 41561 38972 3261205 27274 27639 22995 39536 40968 2917606 26474 28480 21568 37677 43022 2614507 25715 29336 20241 35966 45134 2346608 24994 30207 19008 34387 47304 2109609 24307 31092 17860 32926 49532 1899610 23654 31993 16791 31570 51819 17134

Scientifica 5

Table 2 Estimated MSE when n 30 p 3 and ρ 090 and 099

n 30 09 099Sigma k d OLS Ridge Liu New est OLS Ridge Liu New est1 01 1154 1012 0532 0883 12774 4155 0857 1128

02 0899 0583 0691 2339 1388 194603 0809 0638 0555 1603 2117 327804 0736 0697 0459 1214 3045 441605 0675 0762 0392 0978 4170 534006 0625 0831 0346 0821 5495 609707 0582 0905 0317 0712 7017 672508 0545 0983 0299 0631 8738 725509 0514 1066 0291 0571 10656 77081 0487 1154 0289 0524 12774 8100

5 01 28461 24840 12067 21501 319335 102389 17451 2338302 21945 13492 16402 56008 31445 4036803 19588 15017 12625 36978 50327 7144704 17641 16640 9805 26816 74095 9832205 16010 18362 7690 20580 102751 12026906 14627 20184 6104 16415 136293 13826807 13442 22105 4917 13467 174723 15324008 12418 24124 4036 11293 218040 16588009 11526 26243 3393 9637 266244 1766951 10741 28461 2935 8343 319335 186058

10 01 113841 99331 48088 85947 1277429 409249 69149 9286802 87726 53814 65494 223571 125195 16055403 78277 59935 50326 147369 200793 28474904 70466 66450 38986 106666 295943 39218405 63919 73361 30469 81687 410644 47994006 58368 80667 24064 64998 544898 55191607 53612 88368 19262 53189 698703 61179408 49498 96464 15687 44476 872060 66235009 45910 104955 13064 37839 1064960 7056111 42758 113841 11182 32655 1277429 743065

Table 3 Estimated MSE when n 100 p 3 and ρ 070 and 080

n 100 09 099Sigma k d OLS Ridge Liu New est OLS Ridge Liu New est1 01 01124 01121 01105 01118 01492 01478 01396 01465

02 01118 01107 01114 01465 01404 0144103 01116 01108 01110 01453 01414 0142004 01114 01110 01106 01442 01423 0140105 01112 01112 01104 01432 01434 0138406 01110 01114 01101 01422 01444 0136907 01108 01116 01100 01412 01455 0135608 01106 01119 01099 01403 01467 0134509 01105 01121 01099 01395 01479 013361 01104 01124 01099 01387 01492 01328

5 01 20631 20452 19126 20274 32440 31954 28523 3147202 20276 19289 19924 31480 28942 3053803 20102 19454 19583 31019 29365 2963804 19932 19619 19249 30570 29793 2877105 19764 19785 18922 30133 30224 2793406 19599 19952 18603 29707 30659 2712807 19436 20121 18291 29291 31098 2635008 19276 20290 17986 28887 31542 2560009 19119 20460 17688 28492 31989 248761 18964 20631 17396 28108 32440 24178

10 01 81632 80901 75481 80174 129200 127234 113344 12528702 80182 76150 78747 125320 115045 12151103 79474 76822 77351 123456 116761 11786704 78777 77498 75984 121640 118493 114349

6 Scientifica

objective is to compare the performance of the proposedestimator with ridge regression and Liu estimators weconsider k d 01 02 1 We have restricted k between0 and 1 as Wichern and Churchill [18] have found that theridge regression estimator is better than the OLS when k isbetween 0 and 1 Kan et al [26] also suggested a smallervalue of k (less than 1) is better Simulation studies arerepeated 1000 times for the sample sizes n 30 and 100 andσ2 1 25 and 100 For each replicate we compute the meansquare error (MSE) of the estimators by using the followingequation

MSE αlowast( 1113857 1

10001113944

1000

i1αlowast minus α( 1113857prime αlowast minus α( 1113857 (45)

where αlowast would be any of the estimators (OLS ridge Liu or KL)Smaller MSE of the estimators will be considered the best one

+e simulated results for n 30 p 3 and ρ 070 080and ρ 090 099 are presented in Tables 1 and 2 respec-tively and for n 100 p 3 and ρ 07 080 and ρ 090099 are presented in Tables 3 and 4 respectively +ecorresponding simulated results for n 30 100 and p 7 arepresented in Tables 5ndash8 For a better visualization we haveplotted MSE vs d for n 30 σ 10 and ρ 070 090 and099 in Figures 1ndash3 respectively We also plotted MSE vs σfor n 30 d 50 and ρ 090 and 099 which is presentedin Figures 4 and 5 respectively Finally to see the effect ofsample size on MSE we plotted MSE vs sample size ford 05 and ρ 090 and presented in Figure 6

Table 3 Continued

n 100 09 099Sigma k d OLS Ridge Liu New est OLS Ridge Liu New est

05 78091 78178 74646 119870 120239 11095306 77415 78862 73336 118144 122001 10767407 76750 79549 72053 116462 123778 10450608 76096 80240 70797 114821 125570 10144709 75451 80934 69568 113220 127377 984901 74816 81632 68364 111658 129200 95634

Table 4 Estimated MSE when n 100 p 3 and ρ 090 and 099

n 30 09 099Sigma k d OLS Ridge Liu New est OLS Ridge Liu New est1 01 0287 0278 0230 0270 3072 2141 0688 1423

02 0270 0236 0255 1621 0836 076903 0263 0241 0242 1298 1013 052804 0256 0247 0231 1083 1219 047205 0250 0253 0221 0930 1455 050606 0244 0259 0213 0819 1720 058307 0239 0265 0206 0733 2014 068008 0234 0272 0200 0667 2337 078509 0230 0279 0195 0613 2690 08921 0226 0287 0191 0570 3072 0997

5 01 6958 6719 5256 6486 76772 53314 14746 3468902 6495 5431 6050 39905 18971 1666003 6283 5610 5649 31412 23862 883404 6083 5792 5278 25626 29420 580305 5893 5977 4935 21466 35645 517406 5714 6166 4617 18350 42537 579507 5544 6359 4324 15939 50096 707208 5383 6555 4052 14024 58321 868609 5230 6754 3799 12471 67213 104581 5085 6958 3566 11189 76772 12287

10 01 27809 26853 20970 25916 307086 213255 58717 13868502 25951 21675 24167 159582 75683 6635403 25100 22394 22551 125559 95308 3481504 24296 23126 21056 102365 117590 2246305 23535 23872 19672 85681 142529 1974306 22815 24632 18389 73175 170126 2204507 22131 25406 17200 63493 200380 2699508 21482 26193 16096 55802 233291 3330809 20865 26994 15071 49561 268860 402701 20279 27809 14120 44407 307086 47470

Scientifica 7

Table 5 Estimated MSE when n 30 p 7 and ρ 070 and 080

n 30 07 08Sigma k d OLS Ridge Liu New ridge OLS Ridge Liu New ridge1 01 0838 0811 0651 0785 1239 1179 0859 1121

02 0786 0670 0737 1124 0895 101803 0763 0689 0694 1074 0933 092804 0741 0709 0654 1029 0973 085005 0720 0729 0618 0987 1014 078106 0701 0750 0586 0949 1056 072107 0682 0771 0556 0914 1100 066908 0665 0793 0529 0881 1145 062309 0649 0815 0505 0851 1191 05831 0633 0838 0484 0823 1239 0549

5 01 20955 20275 16063 19608 30981 29455 21084 2797502 19633 16568 18362 28060 22071 2531403 19026 17083 17208 26780 23086 2295104 18452 17607 16139 25602 24130 2084505 17908 18141 15147 24513 25201 1896306 17391 18685 14226 23506 26301 1727907 16901 19238 13369 22570 27429 1576708 16435 19801 12572 21699 28585 1440809 15990 20373 11829 20885 29769 131851 15567 20955 11137 20125 30981 12081

10 01 83821 81095 64205 78423 123923 117811 84259 11188702 78523 66233 73429 112224 88219 10122503 76091 68299 68804 107097 92291 9174904 73789 70403 64513 102375 96475 8330105 71608 72545 60530 98014 100770 7575006 69537 74725 56827 93973 105177 6898307 67569 76942 53382 90220 109696 6290808 65698 79197 50173 86725 114327 5744109 63915 81490 47182 83463 119069 525151 62215 83821 44392 80411 123923 48069

Table 6 Estimated MSE when n 30 p 7 and ρ 09 and 099

N 30 09 099Sigma k d OLS Ridge Liu New ridge OLS Ridge Liu New ridge1 01 252 227 129 203 2868 1120 226 445

02 206 139 166 682 355 41603 188 151 137 478 525 57804 173 163 116 362 736 75805 161 176 099 288 989 92506 150 190 085 237 1283 107507 141 204 075 201 1617 120708 132 220 068 174 1993 132409 125 235 062 154 2410 14271 118 252 057 138 2868 1519

5 01 6303 5658 3123 5057 71709 27885 5083 1080002 5127 3411 4103 16823 8411 971703 4682 3715 3361 11638 12757 1345404 4303 4035 2778 8652 18123 1768905 3977 4372 2314 6736 24507 2164706 3694 4725 1942 5418 31910 2518707 3445 5095 1643 4467 40332 2831608 3225 5481 1401 3754 49772 3108009 3028 5884 1206 3206 60231 335291 2852 6303 1048 2773 71709 35710

10 01 25214 22630 12475 20223 286835 111506 20239 4314802 20503 13628 16403 67243 33562 38784

8 Scientifica

32 Simulation Results and Discussion From Tables 1ndash8and Figures 1ndash6 it appears that as the values of σ increasethe MSE values also increase (Figure 3) while the sample sizeincreases as the MSE values decrease (Figure 4) Ridge Liuand proposed KL estimators uniformly dominate the ordinaryleast squares (OLS) estimator In general from these tables anincrease in the levels of multicollinearity and the number ofexplanatory variables increase the estimated MSE values of theestimators +e figures consistently show that the OLS esti-mator performs worst when there is multicollinearity FromFigures 1ndash6 and simulation Tables 1ndash8 it clearly indicated thatfor ρ 090 or less the proposed estimator uniformly

dominates the ridge regression estimator while Liu performedmuch better than both proposed and ridge estimators for smalld say 03 or lessWhen ρ 099 the ridge regression performsthe best for higher k while the proposed estimator performsthe best for say k (say 03 or less) When d k 05 andρ 099 both ridge and KL estimators outperform the Liuestimator None of the estimators uniformly dominates eachother However it appears that our proposed estimator KLperforms better in the wider space of d k in the parameterspace If we review all Tables 1ndash8 we observed that theconclusions about the performance of all estimators remainthe same for both p 3 and p 7

Table 6 Continued

N 30 09 099Sigma k d OLS Ridge Liu New ridge OLS Ridge Liu New ridge

03 18721 14846 13432 46491 50959 5370704 17205 16130 11091 34538 72431 7062605 15899 17479 9229 26865 97978 8644306 14763 18895 7737 21588 127600 10058607 13766 20376 6534 17777 161296 11308908 12882 21923 5562 14924 199068 12413409 12095 23535 4775 12725 240914 1339211 11389 25214 4138 10992 286835 142634

Table 7 Estimated MSE when n 100 p 7 and ρ 070 and 080

n 100 07 08Sigma k d OLS Ridge Liu New ridge OLS Ridge Liu New ridge1 01 0174 0173 0163 0171 0263 0259 0235 0255

02 0171 0164 0169 0255 0238 024903 0170 0165 0166 0252 0241 024304 0169 0166 0164 0249 0244 023705 0167 0168 0161 0246 0247 023206 0166 0169 0159 0243 0250 022707 0165 0170 0157 0240 0253 022208 0164 0171 0155 0238 0256 021809 0163 0173 0154 0235 0259 02141 0162 0174 0152 0233 0263 0210

5 01 4356 4320 4055 4284 6563 6474 5852 638602 4285 4087 4214 6388 5928 621603 4250 4120 4146 6304 6005 605304 4216 4153 4079 6222 6082 589505 4182 4187 4013 6143 6160 574406 4149 4220 3949 6066 6239 559807 4116 4254 3887 5991 6319 545708 4084 4288 3826 5917 6399 532209 4053 4322 3767 5846 6481 51911 4022 4356 3708 5777 6563 5066

10 01 17425 17281 16219 17138 26250 25896 23408 2554502 17140 16350 16858 25551 23713 2486603 17001 16482 16584 25216 24020 2421204 16864 16614 16316 24891 24330 2358205 16729 16748 16054 24573 24643 2297506 16597 16882 15797 24265 24959 2238907 16467 17016 15547 23964 25277 2182508 16339 17152 15301 23671 25599 2128009 16213 17288 15062 23385 25923 207551 16089 17425 14827 23107 26250 20247

Scientifica 9

Table 8 Estimated MSE when n 100 p 7 and ρ 090 and 099

n 100 09 099Sigma k d OLS Ridge Liu New ridge OLS Ridge Liu New ridge1 01 0546 0529 0431 0512 6389 4391 1624 2949

02 0513 0442 0482 3407 1934 183603 0498 0454 0456 2819 2298 145304 0485 0466 0432 2423 2718 134705 0472 0478 0411 2135 3192 135906 0460 0491 0392 1914 3721 142607 0449 0504 0375 1738 4306 151908 0439 0517 0360 1593 4945 162509 0429 0531 0346 1472 5640 17371 0420 0546 0334 1370 6389 1851

5 01 13640 13216 10676 12802 159732 109722 38895 7328402 12820 10979 12037 84915 47018 4450603 12448 11289 11336 69971 56467 3386504 12099 11605 10693 59823 67242 3014605 11770 11928 10102 52370 79343 2941706 11460 12257 9558 46597 92769 3009007 11168 12593 9056 41953 107521 3145508 10891 12935 8593 38114 123599 3317109 10628 13284 8165 34875 141003 350631 10379 13640 7768 32097 159732 37036

10 01 54558 52866 42699 51212 638928 438910 155399 29312102 51282 43914 48150 339663 187945 17787403 49796 45155 45344 279860 225785 13515104 48399 46422 42768 239236 268921 12012005 47084 47714 40397 209391 317351 11705306 45843 49032 38214 186265 371077 11959907 44670 50375 36198 167659 430097 12492208 43560 51744 34336 152274 494412 13165409 42508 53138 32612 139287 564022 1390941 41509 54558 31014 128149 638928 146866

OLSRidge

LiuKL

025

030

035

040

MSE

02 04 06 08 1000d = k

(a)

OLSRidge

LiuKL

02 04 06 08 1000d = k

20

25

30

35

MSE

(b)

Figure 1 Continued

10 Scientifica

4 Numerical Examples

To illustrate our theoretical results we consider two datasets(i) famous Portland cement data originally adopted byWoods et al [34] and (ii) French economy data from

Chatterjee and Hadi [35] and they are analyzed in thefollowing sections respectively

41 Example 1 Portland Data +ese data are widely knownas the Portland cement dataset It was originally adopted by

OLSRidge

LiuKL

025

035

045

055

MSE

02 04 06 08 1000d = k

(c)

OLSRidge

LiuKL

02 04 06 08 1000d = k

20

30

40

50

MSE

(d)

Figure 1 Estimated MSEs for n 30 Sigma 1 10 rho 070 080 and different values of k d (a) n 30 p 3 sigma 1 and rho 070(b) n 30 p 3 sigma 10 and rho 070 (c) n 30 p 3 sigma 1 and rho 080 (d) n 30 p 3 sigma 10 and rho 080

OLSRidge

LiuKL

04

06

08

10

12

MSE

02 04 06 08 1000d = k

(a)

OLSRidge

LiuKL

20

40

60

80

100M

SE

02 04 06 08 1000d = k

(b)

OLSRidge

LiuKL

02468

101214

MSE

02 04 06 08 1000d = k

(c)

OLSRidge

LiuKL

0200

600

1000

1400

MSE

02 04 06 08 1000d = k

(d)

Figure 2 Estimated MSEs for n 30 sigma 1 10 rho 090 099 and different values of k d (a) n 30 p 3 sigma 1 and rho 090(b) n 30 p 3 sigma 10 and rho 090 (c) n 30 p 3 sigma 1 and rho 099 (d) n 30 p 3 sigma 10 and rho 099

Scientifica 11

OLSRidge

LiuKL

0

10

20

30

MSE

2 4 6 8 100Sigma

(a)

OLSRidge

LiuKL

0

10

30

50

MSE

2 4 6 8 100Sigma

(b)

OLSRidge

LiuKL

0

40

80

120

MSE

2 4 6 8 100Sigma

(c)

OLSRidge

LiuKL

2 4 6 8 100Sigma

0

400

800

1200

MSE

(d)

Figure 3 EstimatedMSEs for n 30 d 05 and different values of rho and sigma (a) n 30 p 3 d 05 and rho 070 (b) n 30 p 3d 05 and rho 080 (c) n 30 p 3 d 05 and rho 090 (d) n 30 p 3 d 05 and rho 099

OLSRidge

LiuKL

50 70 9030n

010

020

030

040

MSE

(a)

OLSRidge

LiuKL

50 70 9030n

02

04

06

MSE

(b)

Figure 4 Continued

12 Scientifica

Woods et al [34] It has also been analyzed by the followingauthors Kaciranlar et al [36] Li and Yang [25] and recentlyby Lukman et al [13] +e regression model for these data isdefined as

yi β0 + β1X1 + β2X2 + β3X3 + β4X4 + εi (46)

where yi heat evolved after 180 days of curing measured incalories per gram of cement X1 tricalcium aluminateX2 tricalcium silicate X3 tetracalcium aluminoferriteand X4 β-dicalcium silicate +e correlation matrix of thepredictor variables is given in Table 9

OLSRidge

LiuKL

50 70 9030n

02

06

10

MSE

(c)

OLSRidge

LiuKL

50 70 9030n

0

2

4

6

8

12

MSE

(d)

Figure 4 Estimated MSEs for sigma 1 p 3 and different values of rho and sample size (a)p 3 sigma 1 d 05 and rho 070(b)p 3 sigma 1 d 05 and rho 080 (c)p 3 sigma 1 d 05 and rho 090 (d)p 3 sigma 1 d 05 and rho 099

OLSRidge

LiuKL

20

30

40

MSE

4 5 6 7 83p

(a)

OLSRidge

LiuKL

3

4

5

6

7M

SE

4 5 6 7 83p

(b)

6

8

10

14

MSE

OLSRidge

LiuKL

4 5 6 7 83p

(c)

OLSRidge

LiuKL

4 5 6 7 83p

0

50

100

150

MSE

(d)

Figure 5 Estimated MSEs for n 100 d 05 sigma 5 and different values of rho and p (a) n 100 sigma 5 d 05 and rho 070 (b)n 100 sigma 5 d 05 and rho 080 (c) n 100 sigma 5 d 05 and rho 090 (d) n 100 sigma 5 d 05 and rho 099

Scientifica 13

+e variance inflation factors are VIF1 = 3850VIF2 = 25442 VIF3 = 4687 and VIF4 = 28251 Eigen-values of XprimeX are λ1 44676206 λ2 5965422

λ3 809952 and λ4 105419 and the condition numberof XprimeX is approximately 424 +e VIFs the eigenvalues

and the condition number all indicate the presence ofsevere multicollinearity +e estimated parameters andMSE are presented in Table 10 It appears from Table 11that the proposed estimator performed the best in thesense of smaller MSE

OLSRidge

LiuKL

0

100

200

300

MSE

075 085 095065Rho

(a)

OLSRidge

LiuKL

0

200

400

600

800

MSE

075 085 095065Rho

(b)

OLSRidge

LiuKL

0

20

40

60

80

MSE

075 085 095065Rho

(c)

OLSRidge

LiuKL

0

50

100

150

MSE

075 085 095065Rho

(d)

Figure 6 Estimated MSEs for n 100 p 3 7 d 05 sigma 5 and different values of rho (a) n 30 p 3 sigma 5 and d 05 (b)n 30 p 7 sigma 5 and d 05 (c) n 100 p 3 sigma 5 and d 05 (d) n 100 p 7 sigma 5 and d 05

Table 9 Correlation matrix

X1 X2 X3 X4

X1 1000 0229 minus 0824 minus 0245X2 0229 1000 minus 0139 minus 0973X3 minus 0824 minus 0139 1000 0030X4 minus 0245 minus 0973 0030 1000

Table 10 +e results of regression coefficients and the corresponding MSE values

Coef 1113954α 1113954α(k) 1113954α(d)1113954dalt 1113954αKL(kHNM) 1113954αKL(kmin)

α0 624054 85870 276490 minus 197876 276068α1 15511 21046 19010 23965 19090α2 05102 10648 08701 13573 08688α3 01019 06681 04621 09666 04680α4 minus 01441 03996 02082 06862 02074MSE 491209 298983 2170963 7255603 217096kd mdash 00077 044195 000235 000047

14 Scientifica

42 Example 2 French Economy Data +e French economydata in Chatterjee and Hadi [37] are considered in this ex-ample It has been analyzed by Malinvard [38] and Liu [6]among others+e variables are imports domestic productionstock formation and domestic consumption All are measuredin milliards of French francs for the years 1949 through 1966

+e regression model for these data is defined as

yi β0 + β1X1 + β2X2 + β3X3 + εi (47)

where yi IMPORT X1 domestic production X2 stockformation and X3 domestic consumption +e correlationmatrix of the predicted variable is given in Table 12

+e variance inflation factors areVIF1 469688VIF2 1047 and VIF3 469338 +e ei-genvalues of the XprimeX matrix are λ1 161779 λ2 158 andλ3 4961 and the condition number is 32612 If we reviewthe above correlation matrix VIFs and condition number itcan be said that there is presence of severe multicollinearityexisting in the predictor variables

+e biasing parameter for the new estimator is defined in(41) and (42) +e biasing parameter for the ridge and Liuestimator is provided in (6) (8) and (9) respectively

We analyzed the data using the biasing parameters foreach of the estimators and presented the results in Tables 10and 11 It can be seen from Tables 10 and 11 that theproposed estimator performed the best in the sense ofsmaller MSE

5 Summary and Concluding Remarks

In this paper we introduced a new biased estimator toovercome the multicollinearity problem for the multiplelinear regression model and provided the estimation tech-nique of the biasing parameter A simulation study has beenconducted to compare the performance of the proposedestimator and Liu [6] and ridge regression estimators [3]Simulation results evidently show that the proposed esti-mator performed better than both Liu and ridge under somecondition on the shrinkage parameter Two sets of real-lifedata are analyzed to illustrate the benefits of using the newestimator in the context of a linear regression model +eproposed estimator is recommended for researchers in this

area Its application can be extended to other regressionmodels for example logistic regression Poisson ZIP andrelated models and those possibilities are under currentinvestigation [37 39 40]

Data Availability

Data will be made available on request

Conflicts of Interest

+e authors declare that they have no conflicts of interest

Acknowledgments

We are dedicating this article to those who lost their livesbecause of COVID-19

References

[1] C Stein ldquoInadmissibility of the usual estimator for mean ofmultivariate normal distributionrdquo in Proceedings of the 0irdBerkley Symposium on Mathematical and Statistics Proba-bility J Neyman Ed vol 1 pp 197ndash206 Springer BerlinGermany 1956

[2] W F Massy ldquoPrincipal components regression in exploratorystatistical researchrdquo Journal of the American Statistical As-sociation vol 60 no 309 pp 234ndash256 1965

[3] A E Hoerl and R W Kennard ldquoRidge regression biasedestimation for nonorthogonal problemsrdquo Technometricsvol 12 no 1 pp 55ndash67 1970

[4] L S Mayer and T A Willke ldquoOn biased estimation in linearmodelsrdquo Technometrics vol 15 no 3 pp 497ndash508 1973

[5] B F Swindel ldquoGood ridge estimators based on prior infor-mationrdquo Communications in Statistics-0eory and Methodsvol 5 no 11 pp 1065ndash1075 1976

[6] K Liu ldquoA new class of biased estimate in linear regressionrdquoCommunication in Statistics- 0eory and Methods vol 22pp 393ndash402 1993

[7] F Akdeniz and S Kaccediliranlar ldquoOn the almost unbiasedgeneralized liu estimator and unbiased estimation of the biasand mserdquo Communications in Statistics-0eory and Methodsvol 24 no 7 pp 1789ndash1797 1995

[8] M R Ozkale and S Kaccediliranlar ldquo+e restricted and unre-stricted two-parameter estimatorsrdquo Communications in Sta-tistics-0eory and Methods vol 36 no 15 pp 2707ndash27252007

[9] S Sakallıoglu and S Kaccedilıranlar ldquoA new biased estimatorbased on ridge estimationrdquo Statistical Papers vol 49 no 4pp 669ndash689 2008

Table 11 +e results of regression coefficients and the corresponding MSE values

Coef 1113954α 1113954α(k) 1113954α(d)1113954dalt 1113954α(d)1113954dopt 1113954αKL(kHNM) 1113954αKL(kmin)

α0 minus 197127 minus 167613 minus 125762 minus 188410 minus 165855 minus 188782α1 00327 01419 02951 00648 01485 00636α2 04059 03576 02875 03914 03548 03922α3 02421 00709 minus 01696 01918 00606 01937MSE 173326 2130519 5828312 1660293 2211899 1660168kd mdash 00527 05282 09423 00258 00065

Table 12 Correlation matrix

X1 X2 X3

X1 1000 0210 0999X2 0210 1000 0208X3 0999 0208 1000

Scientifica 15

[10] H Yang and X Chang ldquoA new two-parameter estimator inlinear regressionrdquo Communications in Statistics-0eory andMethods vol 39 no 6 pp 923ndash934 2010

[11] M Roozbeh ldquoOptimal QR-based estimation in partially linearregression models with correlated errors using GCV crite-rionrdquo Computational Statistics amp Data Analysis vol 117pp 45ndash61 2018

[12] F Akdeniz and M Roozbeh ldquoGeneralized difference-basedweightedmixed almost unbiased ridge estimator in partially linearmodelsrdquo Statistical Papers vol 60 no 5 pp 1717ndash1739 2019

[13] A F Lukman K Ayinde S Binuomote and O A ClementldquoModified ridge-type estimator to combat multicollinearityapplication to chemical datardquo Journal of Chemometricsvol 33 no 5 p e3125 2019

[14] A F Lukman K Ayinde S K Sek and E Adewuyi ldquoAmodified new two-parameter estimator in a linear regressionmodelrdquo Modelling and Simulation in Engineering vol 2019Article ID 6342702 10 pages 2019

[15] A E Hoerl R W Kannard and K F Baldwin ldquoRidge re-gressionsome simulationsrdquo Communications in Statisticsvol 4 no 2 pp 105ndash123 1975

[16] G C McDonald and D I Galarneau ldquoA monte carlo eval-uation of some ridge-type estimatorsrdquo Journal of the Amer-ican Statistical Association vol 70 no 350 pp 407ndash416 1975

[17] J F Lawless and P Wang ldquoA simulation study of ridge andother regression estimatorsrdquo Communications in Statistics-0eory and Methods vol 5 no 4 pp 307ndash323 1976

[18] D W Wichern and G A Churchill ldquoA comparison of ridgeestimatorsrdquo Technometrics vol 20 no 3 pp 301ndash311 1978

[19] B M G Kibria ldquoPerformance of some new ridge regressionestimatorsrdquo Communications in Statistics-Simulation andComputation vol 32 no 1 pp 419ndash435 2003

[20] A F Lukman and K Ayinde ldquoReview and classifications ofthe ridge parameter estimation techniquesrdquoHacettepe Journalof Mathematics and Statistics vol 46 no 5 pp 953ndash967 2017

[21] A K M E Saleh M Arashi and B M G Kibria 0eory ofRidge Regression Estimation with Applications WileyHoboken NJ USA 2019

[22] K Liu ldquoUsing Liu-type estimator to combat collinearityrdquoCommunications in Statistics-0eory and Methods vol 32no 5 pp 1009ndash1020 2003

[23] K Alheety and B M G Kibria ldquoOn the Liu and almostunbiased Liu estimators in the presence of multicollinearitywith heteroscedastic or correlated errorsrdquo Surveys in Math-ematics and its Applications vol 4 pp 155ndash167 2009

[24] X-Q Liu ldquoImproved Liu estimator in a linear regressionmodelrdquo Journal of Statistical Planning and Inference vol 141no 1 pp 189ndash196 2011

[25] Y Li and H Yang ldquoA new Liu-type estimator in linear regressionmodelrdquo Statistical Papers vol 53 no 2 pp 427ndash437 2012

[26] B Kan O Alpu and B Yazıcı ldquoRobust ridge and robust Liuestimator for regression based on the LTS estimatorrdquo Journalof Applied Statistics vol 40 no 3 pp 644ndash655 2013

[27] R A Farghali ldquoGeneralized Liu-type estimator for linearregressionrdquo International Journal of Research and Reviews inApplied Sciences vol 38 no 1 pp 52ndash63 2019

[28] S G Wang M X Wu and Z Z Jia Matrix InequalitiesChinese Science Press Beijing China 2nd edition 2006

[29] R W Farebrother ldquoFurther results on the mean square errorof ridge regressionrdquo Journal of the Royal Statistical SocietySeries B (Methodological) vol 38 no 3 pp 248ndash250 1976

[30] G Trenkler and H Toutenburg ldquoMean squared error matrixcomparisons between biased estimators-an overview of recentresultsrdquo Statistical Papers vol 31 no 1 pp 165ndash179 1990

[31] B M G Kibria and S Banik ldquoSome ridge regression esti-mators and their performancesrdquo Journal of Modern AppliedStatistical Methods vol 15 no 1 pp 206ndash238 2016

[32] D G Gibbons ldquoA simulation study of some ridge estimatorsrdquoJournal of the American Statistical Association vol 76no 373 pp 131ndash139 1981

[33] J P Newhouse and S D Oman ldquoAn evaluation of ridgeestimators A report prepared for United States air forceproject RANDrdquo 1971

[34] H Woods H H Steinour and H R Starke ldquoEffect ofcomposition of Portland cement on heat evolved duringhardeningrdquo Industrial amp Engineering Chemistry vol 24no 11 pp 1207ndash1214 1932

[35] S Chatterjee and A S Hadi Regression Analysis by ExampleWiley Hoboken NJ USA 1977

[36] S Kaciranlar S Sakallioglu F Akdeniz G P H Styan andH J Werner ldquoA new biased estimator in linear regression anda detailed analysis of the widely-analysed dataset on portlandcementrdquo Sankhya 0e Indian Journal of Statistics Series Bvol 61 pp 443ndash459 1999

[37] S Chatterjee and A S Haadi Regression Analysis by ExampleWiley Hoboken NJ USA 2006

[38] E Malinvard Statistical Methods of Econometrics North-Holland Publishing Company Amsterdam Netherlands 3rdedition 1980

[39] D N Gujarati Basic Econometrics McGraw-Hill New YorkNY USA 1995

[40] A F Lukman K Ayinde and A S Ajiboye ldquoMonte Carlostudy of some classification-based ridge parameter estima-torsrdquo Journal of Modern Applied Statistical Methods vol 16no 1 pp 428ndash451 2017

16 Scientifica

Theorem 2 When λmax(HGminus 1)lt 1 estimator 1113954αKL is supe-rior to 1113954α(k) in the MSEM sense if and only if

αprime W(k)M(k) minus Ip1113960 1113961prime V1 + W(k) minus Ip1113872 1113873ααprime W(k) minus Ip1113872 111387311138731113960 1113961

W(k)M(k) minus Ip1113960 1113961α

(30)

λmax HGminus 1

1113872 1113873lt 1 (31)

where

V1 σ2W(k)Λminus 1W(k) minus σ2W(k)M(k)Λminus 1

Mprime(k)W(k)

H 2W(k)

G kW(k)Λminus 1W(k)

(32)

Proof Using the dispersion matrix difference

V1 σ2W(k)Λminus 1W(k) minus σ2W(k)M(k)Λminus 1

Mprime(k)W(k)

σ2kΛminus 1 ΛW(k)Λminus 1W(k) + ΛW(k)Λminus 1

W(k)1113872

minus kW(k)Λminus 1W(k)1113873Λminus 1

σ2W(k)Λminus 1W(k) minus σ2W(k) Ip minus kΛminus 1

1113960 1113961

middot Λminus 1Ip minus kΛminus 1

1113960 1113961W(k)

σ2kΛminus 1(G minus H)Λminus 1

(33)

It is obvious that for kgt 0Ggt 0 andHgt 0 According toLemma 1 it is clear that G-Hgt 0 if and only if HGminus 1 lt 1where λmax(HGminus 1)lt 1 is the maximum eigenvalue of thematrix HGminus 1 Consequently V1 is pd

23 Comparison between 1113954α(d) and 1113954αKL +e difference be-tween MSEM(1113954α(d)) and MSEM(1113954αKL) is

MSEM[1113954α] minus MSEM 1113954αk1113858 1113859 σ2FdΛminus 1

Fd minus σ2W(k)M(k)Λminus 1Mprime(k)Wprime(k)

+(1 minus d)2(Λ + I)

minus 1ααprime(Λ + I)minus 1

minus W(k)M(k) minus Ip1113960 1113961ααprime W(k)M(k) minus Ip1113960 1113961prime(34)

We have the following theorem

Theorem 3 If kgt 0 and 0lt dlt 1 estimator 1113954αKL is superior toestimator 1113954α(d) using the MSEM criterion that isMSEM(1113954α(d)) minus MSEM(1113954αKL)gt 0 if and only if

αprime W(k)M(k) minus Ip1113960 1113961prime V2 +(1 minus d)2(Λ + I)

minus 1ααprime(Λ + I)minus 1

1113960 1113961

middot W(k)M(k) minus Ip1113960 1113961αlt 1

(35)

where V2 σ2FdΛminus 1Fd minus σ2W(k)M(k)Λminus 1Mprime(k)W(k)

Proof Using the difference between the dispersion matrix

V2 σ2FdΛminus 1

Fd minus σ2W(k)M(k)Λminus 1Mprime(k)W(k)

σ2 FdΛminus 1

Fd minus W(k)M(k)Λminus 1Mprime(k)W(k)1113872 1113873

σ2 Λ + Ip1113960 1113961minus 1 Λ + dIp1113960 1113961Λminus 1 Λ + Ip1113960 1113961

minus 1 Λ + dIp1113960 1113961

minus Λ(Λ + k)minus 1Λminus 1

middot (Λ minus k)Λminus 1Λminus 1(Λ minus k)Λ(Λ + k)

minus 1

(36)

where W(k) [Ip + kΛminus 1]minus 1 Λ(Λ + k)minus 1 andM(k) [Ip minus kΛminus 1] Λminus 1(Λ minus k)

σ2diagλi + d( 1113857

2

λi λi + 1( 11138572 minus

λi minus k( 11138572

λi λi + k( 11138572

⎧⎨

⎩

⎫⎬

⎭

p

i1

(37)

We observed that FdΛminus 1Fd minus W(k)M(k)Λminus 1Mprime(k)

W(k) is pd if and only if (λi + d)2(λi + k)2 minus

(λi minus k)2(λi + 1)2 gt 0 or (λi + d) (λi + k) minus (λi minus k)

(λi + 1)gt 0 Obviously for kgt 0 and 0lt dlt 1 (λi + d)(λi +

k) minus (λi minus k)(λi + 1) k(2λ + d + 1) + λ(d minus 1)gt 0 Conse-quently FdΛminus 1Fd minus W(k)M(k)Λminus 1Mprime(k)W(k) is pd

24 Determination of Parameter k +ere is a need to esti-mate the parameter of the new estimator for practical use+e ridge biasing parameter and the Liu shrinkage pa-rameter were determined by both Hoerl and Kennard [3]and Liu [6] respectively Different authors have developedother estimators of these ridge parameters To mention afew these include Hoerl et al [15] Kibria [19] Kibria andBanik [31] and Lukman and Ayinde [20] among others+eoptimal value of k is the one that minimizes

MSEM 1113954βKL1113872 1113873 σ2W(k)M(k)Sminus 1

Mprime(k)Wprime(k)

+ W(k)M(k) minus Ip1113960 1113961ββprime W(k)M(k) minus Ip1113960 1113961prime

p(k) MSEM 1113954αKL1113858 1113859 tr MSEM 1113954αKL( 11138571113858 1113859

p(k) σ2 1113944

p

i1

λi minus k( 11138572

λi λi + k( 11138572 + 4k

21113944

p

i1

α2iλi + k( 1113857

2

(38)

Differentiatingm(k d)with respect to k gives and setting(zp(k)zk) 0 we obtain

k σ2

2α2i + σ2λi( 1113857 (39)

4 Scientifica

+e optimal value of k in (39) depends on the unknownparameter σ2 and α2 +ese two estimators are replaced withtheir unbiased estimate Consequently we have

1113954k 1113954σ2

21113954α2i + 1113954σ2λi1113872 1113873 (40)

Following Hoerl et al [15] the harmonic-mean versionof (40) is defined as

1113954kHMN p1113954σ2

1113936pi1 21113954α2i + 1113954σ2λi1113872 11138731113960 1113961

(41)

According to Ozkale and Kaccediliranlar [8] the minimumversion of (41) is defined as

1113954kmin min1113954σ2

21113954α2i + 1113954σ2λi1113872 1113873⎡⎢⎣ ⎤⎥⎦ (42)

3 Simulation Study

Since theoretical comparisons among the estimators ridgeregression Liu and KL in Section 2 give the conditionaldominance among the estimators a simulation study hasbeen conducted using the R 341 programming languages tosee a better picture about the performance of the estimators

31 Simulation Technique +e design of the simulationstudy depends on factors that are expected to affect theproperties of the estimator under investigation and thecriteria being used to judge the results Since the degree ofcollinearity among the explanatory variable is of centralimportance following Gibbons [32] and Kibria [19] wegenerated the explanatory variables using the followingequation

xij 1 minus ρ21113872 111387312

zij + ρzip+1

i 1 2 n j 1 2 3 p(43)

where zij are independent standard normal pseudo-randomnumbers and ρ represents the correlation between any twoexplanatory variables We consider p 3 and 7 in thesimulation+ese variables are standardized so that XprimeX andXprimey are in correlation forms +e n observations for thedependent variable y are determined by the followingequation

yi β0 + β1xi1 + β2xi2 + β3xi3 + middot middot middot + βpxip + ei

i 1 2 n(44)

where ei are iidN (0 σ2) and without loss of any generalitywe will assume zero intercept for the model in (44) +evalues of β are chosen such that βprimeβ 1 [33] Since our main

Table 1 Estimated MSE when n 30 p 3 and ρ 070 and 080

n 30 07 08Sigma k d OLS Ridge Liu New est OLS Ridge Liu New est1 01 0362 0352 0291 0342 0547 0519 0375 0491

02 0342 0298 0323 0493 0391 044403 0333 0305 0307 0470 0407 040404 0325 0312 0293 0449 0425 037005 0317 0320 0280 0431 0443 034206 0309 0328 0268 0414 0462 031807 0302 0336 0258 0398 0482 029908 0296 0344 0249 0384 0503 028209 0290 0353 0242 0372 0525 026910 0284 0362 0235 0360 0547 0258

5 01 8021 7759 6137 7501 12967 12232 8364 1152202 7511 6331 7021 11567 8817 1026103 7277 6529 6577 10962 9284 915604 7056 6731 6165 10411 9766 818605 6846 6937 5784 9907 10263 733306 6647 7146 5430 9445 10775 658107 6459 7359 5102 9019 11301 591808 6280 7576 4797 8626 11842 533109 6109 7797 4513 8263 12397 481310 5947 8021 4250 7926 12967 4354

10 01 31993 30939 24421 29907 51819 48871 33333 4602202 29945 25203 27977 46201 35155 4095503 29005 26000 26189 43775 37034 3651404 28116 26812 24532 41561 38972 3261205 27274 27639 22995 39536 40968 2917606 26474 28480 21568 37677 43022 2614507 25715 29336 20241 35966 45134 2346608 24994 30207 19008 34387 47304 2109609 24307 31092 17860 32926 49532 1899610 23654 31993 16791 31570 51819 17134

Scientifica 5

Table 2 Estimated MSE when n 30 p 3 and ρ 090 and 099

n 30 09 099Sigma k d OLS Ridge Liu New est OLS Ridge Liu New est1 01 1154 1012 0532 0883 12774 4155 0857 1128

02 0899 0583 0691 2339 1388 194603 0809 0638 0555 1603 2117 327804 0736 0697 0459 1214 3045 441605 0675 0762 0392 0978 4170 534006 0625 0831 0346 0821 5495 609707 0582 0905 0317 0712 7017 672508 0545 0983 0299 0631 8738 725509 0514 1066 0291 0571 10656 77081 0487 1154 0289 0524 12774 8100

5 01 28461 24840 12067 21501 319335 102389 17451 2338302 21945 13492 16402 56008 31445 4036803 19588 15017 12625 36978 50327 7144704 17641 16640 9805 26816 74095 9832205 16010 18362 7690 20580 102751 12026906 14627 20184 6104 16415 136293 13826807 13442 22105 4917 13467 174723 15324008 12418 24124 4036 11293 218040 16588009 11526 26243 3393 9637 266244 1766951 10741 28461 2935 8343 319335 186058

10 01 113841 99331 48088 85947 1277429 409249 69149 9286802 87726 53814 65494 223571 125195 16055403 78277 59935 50326 147369 200793 28474904 70466 66450 38986 106666 295943 39218405 63919 73361 30469 81687 410644 47994006 58368 80667 24064 64998 544898 55191607 53612 88368 19262 53189 698703 61179408 49498 96464 15687 44476 872060 66235009 45910 104955 13064 37839 1064960 7056111 42758 113841 11182 32655 1277429 743065

Table 3 Estimated MSE when n 100 p 3 and ρ 070 and 080

n 100 09 099Sigma k d OLS Ridge Liu New est OLS Ridge Liu New est1 01 01124 01121 01105 01118 01492 01478 01396 01465

02 01118 01107 01114 01465 01404 0144103 01116 01108 01110 01453 01414 0142004 01114 01110 01106 01442 01423 0140105 01112 01112 01104 01432 01434 0138406 01110 01114 01101 01422 01444 0136907 01108 01116 01100 01412 01455 0135608 01106 01119 01099 01403 01467 0134509 01105 01121 01099 01395 01479 013361 01104 01124 01099 01387 01492 01328

5 01 20631 20452 19126 20274 32440 31954 28523 3147202 20276 19289 19924 31480 28942 3053803 20102 19454 19583 31019 29365 2963804 19932 19619 19249 30570 29793 2877105 19764 19785 18922 30133 30224 2793406 19599 19952 18603 29707 30659 2712807 19436 20121 18291 29291 31098 2635008 19276 20290 17986 28887 31542 2560009 19119 20460 17688 28492 31989 248761 18964 20631 17396 28108 32440 24178

10 01 81632 80901 75481 80174 129200 127234 113344 12528702 80182 76150 78747 125320 115045 12151103 79474 76822 77351 123456 116761 11786704 78777 77498 75984 121640 118493 114349

6 Scientifica

objective is to compare the performance of the proposedestimator with ridge regression and Liu estimators weconsider k d 01 02 1 We have restricted k between0 and 1 as Wichern and Churchill [18] have found that theridge regression estimator is better than the OLS when k isbetween 0 and 1 Kan et al [26] also suggested a smallervalue of k (less than 1) is better Simulation studies arerepeated 1000 times for the sample sizes n 30 and 100 andσ2 1 25 and 100 For each replicate we compute the meansquare error (MSE) of the estimators by using the followingequation

MSE αlowast( 1113857 1

10001113944

1000

i1αlowast minus α( 1113857prime αlowast minus α( 1113857 (45)

where αlowast would be any of the estimators (OLS ridge Liu or KL)Smaller MSE of the estimators will be considered the best one

+e simulated results for n 30 p 3 and ρ 070 080and ρ 090 099 are presented in Tables 1 and 2 respec-tively and for n 100 p 3 and ρ 07 080 and ρ 090099 are presented in Tables 3 and 4 respectively +ecorresponding simulated results for n 30 100 and p 7 arepresented in Tables 5ndash8 For a better visualization we haveplotted MSE vs d for n 30 σ 10 and ρ 070 090 and099 in Figures 1ndash3 respectively We also plotted MSE vs σfor n 30 d 50 and ρ 090 and 099 which is presentedin Figures 4 and 5 respectively Finally to see the effect ofsample size on MSE we plotted MSE vs sample size ford 05 and ρ 090 and presented in Figure 6

Table 3 Continued

n 100 09 099Sigma k d OLS Ridge Liu New est OLS Ridge Liu New est

05 78091 78178 74646 119870 120239 11095306 77415 78862 73336 118144 122001 10767407 76750 79549 72053 116462 123778 10450608 76096 80240 70797 114821 125570 10144709 75451 80934 69568 113220 127377 984901 74816 81632 68364 111658 129200 95634

Table 4 Estimated MSE when n 100 p 3 and ρ 090 and 099

n 30 09 099Sigma k d OLS Ridge Liu New est OLS Ridge Liu New est1 01 0287 0278 0230 0270 3072 2141 0688 1423

02 0270 0236 0255 1621 0836 076903 0263 0241 0242 1298 1013 052804 0256 0247 0231 1083 1219 047205 0250 0253 0221 0930 1455 050606 0244 0259 0213 0819 1720 058307 0239 0265 0206 0733 2014 068008 0234 0272 0200 0667 2337 078509 0230 0279 0195 0613 2690 08921 0226 0287 0191 0570 3072 0997

5 01 6958 6719 5256 6486 76772 53314 14746 3468902 6495 5431 6050 39905 18971 1666003 6283 5610 5649 31412 23862 883404 6083 5792 5278 25626 29420 580305 5893 5977 4935 21466 35645 517406 5714 6166 4617 18350 42537 579507 5544 6359 4324 15939 50096 707208 5383 6555 4052 14024 58321 868609 5230 6754 3799 12471 67213 104581 5085 6958 3566 11189 76772 12287

10 01 27809 26853 20970 25916 307086 213255 58717 13868502 25951 21675 24167 159582 75683 6635403 25100 22394 22551 125559 95308 3481504 24296 23126 21056 102365 117590 2246305 23535 23872 19672 85681 142529 1974306 22815 24632 18389 73175 170126 2204507 22131 25406 17200 63493 200380 2699508 21482 26193 16096 55802 233291 3330809 20865 26994 15071 49561 268860 402701 20279 27809 14120 44407 307086 47470

Scientifica 7

Table 5 Estimated MSE when n 30 p 7 and ρ 070 and 080

n 30 07 08Sigma k d OLS Ridge Liu New ridge OLS Ridge Liu New ridge1 01 0838 0811 0651 0785 1239 1179 0859 1121

02 0786 0670 0737 1124 0895 101803 0763 0689 0694 1074 0933 092804 0741 0709 0654 1029 0973 085005 0720 0729 0618 0987 1014 078106 0701 0750 0586 0949 1056 072107 0682 0771 0556 0914 1100 066908 0665 0793 0529 0881 1145 062309 0649 0815 0505 0851 1191 05831 0633 0838 0484 0823 1239 0549

5 01 20955 20275 16063 19608 30981 29455 21084 2797502 19633 16568 18362 28060 22071 2531403 19026 17083 17208 26780 23086 2295104 18452 17607 16139 25602 24130 2084505 17908 18141 15147 24513 25201 1896306 17391 18685 14226 23506 26301 1727907 16901 19238 13369 22570 27429 1576708 16435 19801 12572 21699 28585 1440809 15990 20373 11829 20885 29769 131851 15567 20955 11137 20125 30981 12081

10 01 83821 81095 64205 78423 123923 117811 84259 11188702 78523 66233 73429 112224 88219 10122503 76091 68299 68804 107097 92291 9174904 73789 70403 64513 102375 96475 8330105 71608 72545 60530 98014 100770 7575006 69537 74725 56827 93973 105177 6898307 67569 76942 53382 90220 109696 6290808 65698 79197 50173 86725 114327 5744109 63915 81490 47182 83463 119069 525151 62215 83821 44392 80411 123923 48069

Table 6 Estimated MSE when n 30 p 7 and ρ 09 and 099

N 30 09 099Sigma k d OLS Ridge Liu New ridge OLS Ridge Liu New ridge1 01 252 227 129 203 2868 1120 226 445

02 206 139 166 682 355 41603 188 151 137 478 525 57804 173 163 116 362 736 75805 161 176 099 288 989 92506 150 190 085 237 1283 107507 141 204 075 201 1617 120708 132 220 068 174 1993 132409 125 235 062 154 2410 14271 118 252 057 138 2868 1519

5 01 6303 5658 3123 5057 71709 27885 5083 1080002 5127 3411 4103 16823 8411 971703 4682 3715 3361 11638 12757 1345404 4303 4035 2778 8652 18123 1768905 3977 4372 2314 6736 24507 2164706 3694 4725 1942 5418 31910 2518707 3445 5095 1643 4467 40332 2831608 3225 5481 1401 3754 49772 3108009 3028 5884 1206 3206 60231 335291 2852 6303 1048 2773 71709 35710

10 01 25214 22630 12475 20223 286835 111506 20239 4314802 20503 13628 16403 67243 33562 38784

8 Scientifica

32 Simulation Results and Discussion From Tables 1ndash8and Figures 1ndash6 it appears that as the values of σ increasethe MSE values also increase (Figure 3) while the sample sizeincreases as the MSE values decrease (Figure 4) Ridge Liuand proposed KL estimators uniformly dominate the ordinaryleast squares (OLS) estimator In general from these tables anincrease in the levels of multicollinearity and the number ofexplanatory variables increase the estimated MSE values of theestimators +e figures consistently show that the OLS esti-mator performs worst when there is multicollinearity FromFigures 1ndash6 and simulation Tables 1ndash8 it clearly indicated thatfor ρ 090 or less the proposed estimator uniformly

dominates the ridge regression estimator while Liu performedmuch better than both proposed and ridge estimators for smalld say 03 or lessWhen ρ 099 the ridge regression performsthe best for higher k while the proposed estimator performsthe best for say k (say 03 or less) When d k 05 andρ 099 both ridge and KL estimators outperform the Liuestimator None of the estimators uniformly dominates eachother However it appears that our proposed estimator KLperforms better in the wider space of d k in the parameterspace If we review all Tables 1ndash8 we observed that theconclusions about the performance of all estimators remainthe same for both p 3 and p 7

Table 6 Continued

N 30 09 099Sigma k d OLS Ridge Liu New ridge OLS Ridge Liu New ridge

03 18721 14846 13432 46491 50959 5370704 17205 16130 11091 34538 72431 7062605 15899 17479 9229 26865 97978 8644306 14763 18895 7737 21588 127600 10058607 13766 20376 6534 17777 161296 11308908 12882 21923 5562 14924 199068 12413409 12095 23535 4775 12725 240914 1339211 11389 25214 4138 10992 286835 142634

Table 7 Estimated MSE when n 100 p 7 and ρ 070 and 080

n 100 07 08Sigma k d OLS Ridge Liu New ridge OLS Ridge Liu New ridge1 01 0174 0173 0163 0171 0263 0259 0235 0255

02 0171 0164 0169 0255 0238 024903 0170 0165 0166 0252 0241 024304 0169 0166 0164 0249 0244 023705 0167 0168 0161 0246 0247 023206 0166 0169 0159 0243 0250 022707 0165 0170 0157 0240 0253 022208 0164 0171 0155 0238 0256 021809 0163 0173 0154 0235 0259 02141 0162 0174 0152 0233 0263 0210

5 01 4356 4320 4055 4284 6563 6474 5852 638602 4285 4087 4214 6388 5928 621603 4250 4120 4146 6304 6005 605304 4216 4153 4079 6222 6082 589505 4182 4187 4013 6143 6160 574406 4149 4220 3949 6066 6239 559807 4116 4254 3887 5991 6319 545708 4084 4288 3826 5917 6399 532209 4053 4322 3767 5846 6481 51911 4022 4356 3708 5777 6563 5066

10 01 17425 17281 16219 17138 26250 25896 23408 2554502 17140 16350 16858 25551 23713 2486603 17001 16482 16584 25216 24020 2421204 16864 16614 16316 24891 24330 2358205 16729 16748 16054 24573 24643 2297506 16597 16882 15797 24265 24959 2238907 16467 17016 15547 23964 25277 2182508 16339 17152 15301 23671 25599 2128009 16213 17288 15062 23385 25923 207551 16089 17425 14827 23107 26250 20247

Scientifica 9

Table 8 Estimated MSE when n 100 p 7 and ρ 090 and 099

n 100 09 099Sigma k d OLS Ridge Liu New ridge OLS Ridge Liu New ridge1 01 0546 0529 0431 0512 6389 4391 1624 2949

02 0513 0442 0482 3407 1934 183603 0498 0454 0456 2819 2298 145304 0485 0466 0432 2423 2718 134705 0472 0478 0411 2135 3192 135906 0460 0491 0392 1914 3721 142607 0449 0504 0375 1738 4306 151908 0439 0517 0360 1593 4945 162509 0429 0531 0346 1472 5640 17371 0420 0546 0334 1370 6389 1851

5 01 13640 13216 10676 12802 159732 109722 38895 7328402 12820 10979 12037 84915 47018 4450603 12448 11289 11336 69971 56467 3386504 12099 11605 10693 59823 67242 3014605 11770 11928 10102 52370 79343 2941706 11460 12257 9558 46597 92769 3009007 11168 12593 9056 41953 107521 3145508 10891 12935 8593 38114 123599 3317109 10628 13284 8165 34875 141003 350631 10379 13640 7768 32097 159732 37036

10 01 54558 52866 42699 51212 638928 438910 155399 29312102 51282 43914 48150 339663 187945 17787403 49796 45155 45344 279860 225785 13515104 48399 46422 42768 239236 268921 12012005 47084 47714 40397 209391 317351 11705306 45843 49032 38214 186265 371077 11959907 44670 50375 36198 167659 430097 12492208 43560 51744 34336 152274 494412 13165409 42508 53138 32612 139287 564022 1390941 41509 54558 31014 128149 638928 146866

OLSRidge

LiuKL

025

030

035

040

MSE

02 04 06 08 1000d = k

(a)

OLSRidge

LiuKL

02 04 06 08 1000d = k

20

25

30

35

MSE

(b)

Figure 1 Continued

10 Scientifica

4 Numerical Examples

To illustrate our theoretical results we consider two datasets(i) famous Portland cement data originally adopted byWoods et al [34] and (ii) French economy data from

Chatterjee and Hadi [35] and they are analyzed in thefollowing sections respectively

41 Example 1 Portland Data +ese data are widely knownas the Portland cement dataset It was originally adopted by

OLSRidge

LiuKL

025

035

045

055

MSE

02 04 06 08 1000d = k

(c)

OLSRidge

LiuKL

02 04 06 08 1000d = k

20

30

40

50

MSE

(d)

Figure 1 Estimated MSEs for n 30 Sigma 1 10 rho 070 080 and different values of k d (a) n 30 p 3 sigma 1 and rho 070(b) n 30 p 3 sigma 10 and rho 070 (c) n 30 p 3 sigma 1 and rho 080 (d) n 30 p 3 sigma 10 and rho 080

OLSRidge

LiuKL

04

06

08

10

12

MSE

02 04 06 08 1000d = k

(a)

OLSRidge

LiuKL

20

40

60

80

100M

SE

02 04 06 08 1000d = k

(b)

OLSRidge

LiuKL

02468

101214

MSE

02 04 06 08 1000d = k

(c)

OLSRidge

LiuKL

0200

600

1000

1400

MSE

02 04 06 08 1000d = k

(d)

Figure 2 Estimated MSEs for n 30 sigma 1 10 rho 090 099 and different values of k d (a) n 30 p 3 sigma 1 and rho 090(b) n 30 p 3 sigma 10 and rho 090 (c) n 30 p 3 sigma 1 and rho 099 (d) n 30 p 3 sigma 10 and rho 099

Scientifica 11

OLSRidge

LiuKL

0

10

20

30

MSE

2 4 6 8 100Sigma

(a)

OLSRidge

LiuKL

0

10

30

50

MSE

2 4 6 8 100Sigma

(b)

OLSRidge

LiuKL

0

40

80

120

MSE

2 4 6 8 100Sigma

(c)

OLSRidge

LiuKL

2 4 6 8 100Sigma

0

400

800

1200

MSE

(d)

Figure 3 EstimatedMSEs for n 30 d 05 and different values of rho and sigma (a) n 30 p 3 d 05 and rho 070 (b) n 30 p 3d 05 and rho 080 (c) n 30 p 3 d 05 and rho 090 (d) n 30 p 3 d 05 and rho 099

OLSRidge

LiuKL

50 70 9030n

010

020

030

040

MSE

(a)

OLSRidge

LiuKL

50 70 9030n

02

04

06

MSE

(b)

Figure 4 Continued

12 Scientifica

Woods et al [34] It has also been analyzed by the followingauthors Kaciranlar et al [36] Li and Yang [25] and recentlyby Lukman et al [13] +e regression model for these data isdefined as

yi β0 + β1X1 + β2X2 + β3X3 + β4X4 + εi (46)

where yi heat evolved after 180 days of curing measured incalories per gram of cement X1 tricalcium aluminateX2 tricalcium silicate X3 tetracalcium aluminoferriteand X4 β-dicalcium silicate +e correlation matrix of thepredictor variables is given in Table 9

OLSRidge

LiuKL

50 70 9030n

02

06

10

MSE

(c)

OLSRidge

LiuKL

50 70 9030n

0

2

4

6

8

12

MSE

(d)

Figure 4 Estimated MSEs for sigma 1 p 3 and different values of rho and sample size (a)p 3 sigma 1 d 05 and rho 070(b)p 3 sigma 1 d 05 and rho 080 (c)p 3 sigma 1 d 05 and rho 090 (d)p 3 sigma 1 d 05 and rho 099

OLSRidge

LiuKL

20

30

40

MSE

4 5 6 7 83p

(a)

OLSRidge

LiuKL

3

4

5

6

7M

SE

4 5 6 7 83p

(b)

6

8

10

14

MSE

OLSRidge

LiuKL

4 5 6 7 83p

(c)

OLSRidge

LiuKL

4 5 6 7 83p

0

50

100

150

MSE

(d)

Figure 5 Estimated MSEs for n 100 d 05 sigma 5 and different values of rho and p (a) n 100 sigma 5 d 05 and rho 070 (b)n 100 sigma 5 d 05 and rho 080 (c) n 100 sigma 5 d 05 and rho 090 (d) n 100 sigma 5 d 05 and rho 099

Scientifica 13

+e variance inflation factors are VIF1 = 3850VIF2 = 25442 VIF3 = 4687 and VIF4 = 28251 Eigen-values of XprimeX are λ1 44676206 λ2 5965422

λ3 809952 and λ4 105419 and the condition numberof XprimeX is approximately 424 +e VIFs the eigenvalues

and the condition number all indicate the presence ofsevere multicollinearity +e estimated parameters andMSE are presented in Table 10 It appears from Table 11that the proposed estimator performed the best in thesense of smaller MSE

OLSRidge

LiuKL

0

100

200

300

MSE

075 085 095065Rho

(a)

OLSRidge

LiuKL

0

200

400

600

800

MSE

075 085 095065Rho

(b)

OLSRidge

LiuKL

0

20

40

60

80

MSE

075 085 095065Rho

(c)

OLSRidge

LiuKL

0

50

100

150

MSE

075 085 095065Rho

(d)

Figure 6 Estimated MSEs for n 100 p 3 7 d 05 sigma 5 and different values of rho (a) n 30 p 3 sigma 5 and d 05 (b)n 30 p 7 sigma 5 and d 05 (c) n 100 p 3 sigma 5 and d 05 (d) n 100 p 7 sigma 5 and d 05

Table 9 Correlation matrix

X1 X2 X3 X4

X1 1000 0229 minus 0824 minus 0245X2 0229 1000 minus 0139 minus 0973X3 minus 0824 minus 0139 1000 0030X4 minus 0245 minus 0973 0030 1000

Table 10 +e results of regression coefficients and the corresponding MSE values

Coef 1113954α 1113954α(k) 1113954α(d)1113954dalt 1113954αKL(kHNM) 1113954αKL(kmin)

α0 624054 85870 276490 minus 197876 276068α1 15511 21046 19010 23965 19090α2 05102 10648 08701 13573 08688α3 01019 06681 04621 09666 04680α4 minus 01441 03996 02082 06862 02074MSE 491209 298983 2170963 7255603 217096kd mdash 00077 044195 000235 000047

14 Scientifica

42 Example 2 French Economy Data +e French economydata in Chatterjee and Hadi [37] are considered in this ex-ample It has been analyzed by Malinvard [38] and Liu [6]among others+e variables are imports domestic productionstock formation and domestic consumption All are measuredin milliards of French francs for the years 1949 through 1966

+e regression model for these data is defined as

yi β0 + β1X1 + β2X2 + β3X3 + εi (47)

where yi IMPORT X1 domestic production X2 stockformation and X3 domestic consumption +e correlationmatrix of the predicted variable is given in Table 12

+e variance inflation factors areVIF1 469688VIF2 1047 and VIF3 469338 +e ei-genvalues of the XprimeX matrix are λ1 161779 λ2 158 andλ3 4961 and the condition number is 32612 If we reviewthe above correlation matrix VIFs and condition number itcan be said that there is presence of severe multicollinearityexisting in the predictor variables

+e biasing parameter for the new estimator is defined in(41) and (42) +e biasing parameter for the ridge and Liuestimator is provided in (6) (8) and (9) respectively

We analyzed the data using the biasing parameters foreach of the estimators and presented the results in Tables 10and 11 It can be seen from Tables 10 and 11 that theproposed estimator performed the best in the sense ofsmaller MSE

5 Summary and Concluding Remarks

In this paper we introduced a new biased estimator toovercome the multicollinearity problem for the multiplelinear regression model and provided the estimation tech-nique of the biasing parameter A simulation study has beenconducted to compare the performance of the proposedestimator and Liu [6] and ridge regression estimators [3]Simulation results evidently show that the proposed esti-mator performed better than both Liu and ridge under somecondition on the shrinkage parameter Two sets of real-lifedata are analyzed to illustrate the benefits of using the newestimator in the context of a linear regression model +eproposed estimator is recommended for researchers in this

area Its application can be extended to other regressionmodels for example logistic regression Poisson ZIP andrelated models and those possibilities are under currentinvestigation [37 39 40]

Data Availability

Data will be made available on request

Conflicts of Interest

+e authors declare that they have no conflicts of interest

Acknowledgments

We are dedicating this article to those who lost their livesbecause of COVID-19

References

[1] C Stein ldquoInadmissibility of the usual estimator for mean ofmultivariate normal distributionrdquo in Proceedings of the 0irdBerkley Symposium on Mathematical and Statistics Proba-bility J Neyman Ed vol 1 pp 197ndash206 Springer BerlinGermany 1956

[2] W F Massy ldquoPrincipal components regression in exploratorystatistical researchrdquo Journal of the American Statistical As-sociation vol 60 no 309 pp 234ndash256 1965

[3] A E Hoerl and R W Kennard ldquoRidge regression biasedestimation for nonorthogonal problemsrdquo Technometricsvol 12 no 1 pp 55ndash67 1970

[4] L S Mayer and T A Willke ldquoOn biased estimation in linearmodelsrdquo Technometrics vol 15 no 3 pp 497ndash508 1973

[5] B F Swindel ldquoGood ridge estimators based on prior infor-mationrdquo Communications in Statistics-0eory and Methodsvol 5 no 11 pp 1065ndash1075 1976

[6] K Liu ldquoA new class of biased estimate in linear regressionrdquoCommunication in Statistics- 0eory and Methods vol 22pp 393ndash402 1993

[7] F Akdeniz and S Kaccediliranlar ldquoOn the almost unbiasedgeneralized liu estimator and unbiased estimation of the biasand mserdquo Communications in Statistics-0eory and Methodsvol 24 no 7 pp 1789ndash1797 1995

[8] M R Ozkale and S Kaccediliranlar ldquo+e restricted and unre-stricted two-parameter estimatorsrdquo Communications in Sta-tistics-0eory and Methods vol 36 no 15 pp 2707ndash27252007

[9] S Sakallıoglu and S Kaccedilıranlar ldquoA new biased estimatorbased on ridge estimationrdquo Statistical Papers vol 49 no 4pp 669ndash689 2008

Table 11 +e results of regression coefficients and the corresponding MSE values

Coef 1113954α 1113954α(k) 1113954α(d)1113954dalt 1113954α(d)1113954dopt 1113954αKL(kHNM) 1113954αKL(kmin)

α0 minus 197127 minus 167613 minus 125762 minus 188410 minus 165855 minus 188782α1 00327 01419 02951 00648 01485 00636α2 04059 03576 02875 03914 03548 03922α3 02421 00709 minus 01696 01918 00606 01937MSE 173326 2130519 5828312 1660293 2211899 1660168kd mdash 00527 05282 09423 00258 00065

Table 12 Correlation matrix

X1 X2 X3

X1 1000 0210 0999X2 0210 1000 0208X3 0999 0208 1000

Scientifica 15

[10] H Yang and X Chang ldquoA new two-parameter estimator inlinear regressionrdquo Communications in Statistics-0eory andMethods vol 39 no 6 pp 923ndash934 2010

[11] M Roozbeh ldquoOptimal QR-based estimation in partially linearregression models with correlated errors using GCV crite-rionrdquo Computational Statistics amp Data Analysis vol 117pp 45ndash61 2018

[12] F Akdeniz and M Roozbeh ldquoGeneralized difference-basedweightedmixed almost unbiased ridge estimator in partially linearmodelsrdquo Statistical Papers vol 60 no 5 pp 1717ndash1739 2019

[13] A F Lukman K Ayinde S Binuomote and O A ClementldquoModified ridge-type estimator to combat multicollinearityapplication to chemical datardquo Journal of Chemometricsvol 33 no 5 p e3125 2019

[14] A F Lukman K Ayinde S K Sek and E Adewuyi ldquoAmodified new two-parameter estimator in a linear regressionmodelrdquo Modelling and Simulation in Engineering vol 2019Article ID 6342702 10 pages 2019

[15] A E Hoerl R W Kannard and K F Baldwin ldquoRidge re-gressionsome simulationsrdquo Communications in Statisticsvol 4 no 2 pp 105ndash123 1975

[16] G C McDonald and D I Galarneau ldquoA monte carlo eval-uation of some ridge-type estimatorsrdquo Journal of the Amer-ican Statistical Association vol 70 no 350 pp 407ndash416 1975

[17] J F Lawless and P Wang ldquoA simulation study of ridge andother regression estimatorsrdquo Communications in Statistics-0eory and Methods vol 5 no 4 pp 307ndash323 1976

[18] D W Wichern and G A Churchill ldquoA comparison of ridgeestimatorsrdquo Technometrics vol 20 no 3 pp 301ndash311 1978

[19] B M G Kibria ldquoPerformance of some new ridge regressionestimatorsrdquo Communications in Statistics-Simulation andComputation vol 32 no 1 pp 419ndash435 2003

[20] A F Lukman and K Ayinde ldquoReview and classifications ofthe ridge parameter estimation techniquesrdquoHacettepe Journalof Mathematics and Statistics vol 46 no 5 pp 953ndash967 2017

[21] A K M E Saleh M Arashi and B M G Kibria 0eory ofRidge Regression Estimation with Applications WileyHoboken NJ USA 2019

[22] K Liu ldquoUsing Liu-type estimator to combat collinearityrdquoCommunications in Statistics-0eory and Methods vol 32no 5 pp 1009ndash1020 2003

[23] K Alheety and B M G Kibria ldquoOn the Liu and almostunbiased Liu estimators in the presence of multicollinearitywith heteroscedastic or correlated errorsrdquo Surveys in Math-ematics and its Applications vol 4 pp 155ndash167 2009

[24] X-Q Liu ldquoImproved Liu estimator in a linear regressionmodelrdquo Journal of Statistical Planning and Inference vol 141no 1 pp 189ndash196 2011

[25] Y Li and H Yang ldquoA new Liu-type estimator in linear regressionmodelrdquo Statistical Papers vol 53 no 2 pp 427ndash437 2012

[26] B Kan O Alpu and B Yazıcı ldquoRobust ridge and robust Liuestimator for regression based on the LTS estimatorrdquo Journalof Applied Statistics vol 40 no 3 pp 644ndash655 2013

[27] R A Farghali ldquoGeneralized Liu-type estimator for linearregressionrdquo International Journal of Research and Reviews inApplied Sciences vol 38 no 1 pp 52ndash63 2019

[28] S G Wang M X Wu and Z Z Jia Matrix InequalitiesChinese Science Press Beijing China 2nd edition 2006

[29] R W Farebrother ldquoFurther results on the mean square errorof ridge regressionrdquo Journal of the Royal Statistical SocietySeries B (Methodological) vol 38 no 3 pp 248ndash250 1976

[30] G Trenkler and H Toutenburg ldquoMean squared error matrixcomparisons between biased estimators-an overview of recentresultsrdquo Statistical Papers vol 31 no 1 pp 165ndash179 1990

[31] B M G Kibria and S Banik ldquoSome ridge regression esti-mators and their performancesrdquo Journal of Modern AppliedStatistical Methods vol 15 no 1 pp 206ndash238 2016

[32] D G Gibbons ldquoA simulation study of some ridge estimatorsrdquoJournal of the American Statistical Association vol 76no 373 pp 131ndash139 1981

[33] J P Newhouse and S D Oman ldquoAn evaluation of ridgeestimators A report prepared for United States air forceproject RANDrdquo 1971

[34] H Woods H H Steinour and H R Starke ldquoEffect ofcomposition of Portland cement on heat evolved duringhardeningrdquo Industrial amp Engineering Chemistry vol 24no 11 pp 1207ndash1214 1932

[35] S Chatterjee and A S Hadi Regression Analysis by ExampleWiley Hoboken NJ USA 1977

[36] S Kaciranlar S Sakallioglu F Akdeniz G P H Styan andH J Werner ldquoA new biased estimator in linear regression anda detailed analysis of the widely-analysed dataset on portlandcementrdquo Sankhya 0e Indian Journal of Statistics Series Bvol 61 pp 443ndash459 1999

[37] S Chatterjee and A S Haadi Regression Analysis by ExampleWiley Hoboken NJ USA 2006

[38] E Malinvard Statistical Methods of Econometrics North-Holland Publishing Company Amsterdam Netherlands 3rdedition 1980

[39] D N Gujarati Basic Econometrics McGraw-Hill New YorkNY USA 1995

[40] A F Lukman K Ayinde and A S Ajiboye ldquoMonte Carlostudy of some classification-based ridge parameter estima-torsrdquo Journal of Modern Applied Statistical Methods vol 16no 1 pp 428ndash451 2017

16 Scientifica

+e optimal value of k in (39) depends on the unknownparameter σ2 and α2 +ese two estimators are replaced withtheir unbiased estimate Consequently we have

1113954k 1113954σ2

21113954α2i + 1113954σ2λi1113872 1113873 (40)

Following Hoerl et al [15] the harmonic-mean versionof (40) is defined as

1113954kHMN p1113954σ2

1113936pi1 21113954α2i + 1113954σ2λi1113872 11138731113960 1113961

(41)

According to Ozkale and Kaccediliranlar [8] the minimumversion of (41) is defined as

1113954kmin min1113954σ2

21113954α2i + 1113954σ2λi1113872 1113873⎡⎢⎣ ⎤⎥⎦ (42)

3 Simulation Study

Since theoretical comparisons among the estimators ridgeregression Liu and KL in Section 2 give the conditionaldominance among the estimators a simulation study hasbeen conducted using the R 341 programming languages tosee a better picture about the performance of the estimators

31 Simulation Technique +e design of the simulationstudy depends on factors that are expected to affect theproperties of the estimator under investigation and thecriteria being used to judge the results Since the degree ofcollinearity among the explanatory variable is of centralimportance following Gibbons [32] and Kibria [19] wegenerated the explanatory variables using the followingequation

xij 1 minus ρ21113872 111387312

zij + ρzip+1

i 1 2 n j 1 2 3 p(43)

where zij are independent standard normal pseudo-randomnumbers and ρ represents the correlation between any twoexplanatory variables We consider p 3 and 7 in thesimulation+ese variables are standardized so that XprimeX andXprimey are in correlation forms +e n observations for thedependent variable y are determined by the followingequation

yi β0 + β1xi1 + β2xi2 + β3xi3 + middot middot middot + βpxip + ei

i 1 2 n(44)

where ei are iidN (0 σ2) and without loss of any generalitywe will assume zero intercept for the model in (44) +evalues of β are chosen such that βprimeβ 1 [33] Since our main

Table 1 Estimated MSE when n 30 p 3 and ρ 070 and 080

n 30 07 08Sigma k d OLS Ridge Liu New est OLS Ridge Liu New est1 01 0362 0352 0291 0342 0547 0519 0375 0491

02 0342 0298 0323 0493 0391 044403 0333 0305 0307 0470 0407 040404 0325 0312 0293 0449 0425 037005 0317 0320 0280 0431 0443 034206 0309 0328 0268 0414 0462 031807 0302 0336 0258 0398 0482 029908 0296 0344 0249 0384 0503 028209 0290 0353 0242 0372 0525 026910 0284 0362 0235 0360 0547 0258

5 01 8021 7759 6137 7501 12967 12232 8364 1152202 7511 6331 7021 11567 8817 1026103 7277 6529 6577 10962 9284 915604 7056 6731 6165 10411 9766 818605 6846 6937 5784 9907 10263 733306 6647 7146 5430 9445 10775 658107 6459 7359 5102 9019 11301 591808 6280 7576 4797 8626 11842 533109 6109 7797 4513 8263 12397 481310 5947 8021 4250 7926 12967 4354

10 01 31993 30939 24421 29907 51819 48871 33333 4602202 29945 25203 27977 46201 35155 4095503 29005 26000 26189 43775 37034 3651404 28116 26812 24532 41561 38972 3261205 27274 27639 22995 39536 40968 2917606 26474 28480 21568 37677 43022 2614507 25715 29336 20241 35966 45134 2346608 24994 30207 19008 34387 47304 2109609 24307 31092 17860 32926 49532 1899610 23654 31993 16791 31570 51819 17134

Scientifica 5

Table 2 Estimated MSE when n 30 p 3 and ρ 090 and 099

n 30 09 099Sigma k d OLS Ridge Liu New est OLS Ridge Liu New est1 01 1154 1012 0532 0883 12774 4155 0857 1128

02 0899 0583 0691 2339 1388 194603 0809 0638 0555 1603 2117 327804 0736 0697 0459 1214 3045 441605 0675 0762 0392 0978 4170 534006 0625 0831 0346 0821 5495 609707 0582 0905 0317 0712 7017 672508 0545 0983 0299 0631 8738 725509 0514 1066 0291 0571 10656 77081 0487 1154 0289 0524 12774 8100

5 01 28461 24840 12067 21501 319335 102389 17451 2338302 21945 13492 16402 56008 31445 4036803 19588 15017 12625 36978 50327 7144704 17641 16640 9805 26816 74095 9832205 16010 18362 7690 20580 102751 12026906 14627 20184 6104 16415 136293 13826807 13442 22105 4917 13467 174723 15324008 12418 24124 4036 11293 218040 16588009 11526 26243 3393 9637 266244 1766951 10741 28461 2935 8343 319335 186058

10 01 113841 99331 48088 85947 1277429 409249 69149 9286802 87726 53814 65494 223571 125195 16055403 78277 59935 50326 147369 200793 28474904 70466 66450 38986 106666 295943 39218405 63919 73361 30469 81687 410644 47994006 58368 80667 24064 64998 544898 55191607 53612 88368 19262 53189 698703 61179408 49498 96464 15687 44476 872060 66235009 45910 104955 13064 37839 1064960 7056111 42758 113841 11182 32655 1277429 743065

Table 3 Estimated MSE when n 100 p 3 and ρ 070 and 080

n 100 09 099Sigma k d OLS Ridge Liu New est OLS Ridge Liu New est1 01 01124 01121 01105 01118 01492 01478 01396 01465

02 01118 01107 01114 01465 01404 0144103 01116 01108 01110 01453 01414 0142004 01114 01110 01106 01442 01423 0140105 01112 01112 01104 01432 01434 0138406 01110 01114 01101 01422 01444 0136907 01108 01116 01100 01412 01455 0135608 01106 01119 01099 01403 01467 0134509 01105 01121 01099 01395 01479 013361 01104 01124 01099 01387 01492 01328

5 01 20631 20452 19126 20274 32440 31954 28523 3147202 20276 19289 19924 31480 28942 3053803 20102 19454 19583 31019 29365 2963804 19932 19619 19249 30570 29793 2877105 19764 19785 18922 30133 30224 2793406 19599 19952 18603 29707 30659 2712807 19436 20121 18291 29291 31098 2635008 19276 20290 17986 28887 31542 2560009 19119 20460 17688 28492 31989 248761 18964 20631 17396 28108 32440 24178

10 01 81632 80901 75481 80174 129200 127234 113344 12528702 80182 76150 78747 125320 115045 12151103 79474 76822 77351 123456 116761 11786704 78777 77498 75984 121640 118493 114349

6 Scientifica

objective is to compare the performance of the proposedestimator with ridge regression and Liu estimators weconsider k d 01 02 1 We have restricted k between0 and 1 as Wichern and Churchill [18] have found that theridge regression estimator is better than the OLS when k isbetween 0 and 1 Kan et al [26] also suggested a smallervalue of k (less than 1) is better Simulation studies arerepeated 1000 times for the sample sizes n 30 and 100 andσ2 1 25 and 100 For each replicate we compute the meansquare error (MSE) of the estimators by using the followingequation

MSE αlowast( 1113857 1

10001113944

1000

i1αlowast minus α( 1113857prime αlowast minus α( 1113857 (45)

where αlowast would be any of the estimators (OLS ridge Liu or KL)Smaller MSE of the estimators will be considered the best one

+e simulated results for n 30 p 3 and ρ 070 080and ρ 090 099 are presented in Tables 1 and 2 respec-tively and for n 100 p 3 and ρ 07 080 and ρ 090099 are presented in Tables 3 and 4 respectively +ecorresponding simulated results for n 30 100 and p 7 arepresented in Tables 5ndash8 For a better visualization we haveplotted MSE vs d for n 30 σ 10 and ρ 070 090 and099 in Figures 1ndash3 respectively We also plotted MSE vs σfor n 30 d 50 and ρ 090 and 099 which is presentedin Figures 4 and 5 respectively Finally to see the effect ofsample size on MSE we plotted MSE vs sample size ford 05 and ρ 090 and presented in Figure 6

Table 3 Continued

n 100 09 099Sigma k d OLS Ridge Liu New est OLS Ridge Liu New est

05 78091 78178 74646 119870 120239 11095306 77415 78862 73336 118144 122001 10767407 76750 79549 72053 116462 123778 10450608 76096 80240 70797 114821 125570 10144709 75451 80934 69568 113220 127377 984901 74816 81632 68364 111658 129200 95634

Table 4 Estimated MSE when n 100 p 3 and ρ 090 and 099

n 30 09 099Sigma k d OLS Ridge Liu New est OLS Ridge Liu New est1 01 0287 0278 0230 0270 3072 2141 0688 1423

02 0270 0236 0255 1621 0836 076903 0263 0241 0242 1298 1013 052804 0256 0247 0231 1083 1219 047205 0250 0253 0221 0930 1455 050606 0244 0259 0213 0819 1720 058307 0239 0265 0206 0733 2014 068008 0234 0272 0200 0667 2337 078509 0230 0279 0195 0613 2690 08921 0226 0287 0191 0570 3072 0997

5 01 6958 6719 5256 6486 76772 53314 14746 3468902 6495 5431 6050 39905 18971 1666003 6283 5610 5649 31412 23862 883404 6083 5792 5278 25626 29420 580305 5893 5977 4935 21466 35645 517406 5714 6166 4617 18350 42537 579507 5544 6359 4324 15939 50096 707208 5383 6555 4052 14024 58321 868609 5230 6754 3799 12471 67213 104581 5085 6958 3566 11189 76772 12287

10 01 27809 26853 20970 25916 307086 213255 58717 13868502 25951 21675 24167 159582 75683 6635403 25100 22394 22551 125559 95308 3481504 24296 23126 21056 102365 117590 2246305 23535 23872 19672 85681 142529 1974306 22815 24632 18389 73175 170126 2204507 22131 25406 17200 63493 200380 2699508 21482 26193 16096 55802 233291 3330809 20865 26994 15071 49561 268860 402701 20279 27809 14120 44407 307086 47470

Scientifica 7

Table 5 Estimated MSE when n 30 p 7 and ρ 070 and 080

n 30 07 08Sigma k d OLS Ridge Liu New ridge OLS Ridge Liu New ridge1 01 0838 0811 0651 0785 1239 1179 0859 1121

02 0786 0670 0737 1124 0895 101803 0763 0689 0694 1074 0933 092804 0741 0709 0654 1029 0973 085005 0720 0729 0618 0987 1014 078106 0701 0750 0586 0949 1056 072107 0682 0771 0556 0914 1100 066908 0665 0793 0529 0881 1145 062309 0649 0815 0505 0851 1191 05831 0633 0838 0484 0823 1239 0549

5 01 20955 20275 16063 19608 30981 29455 21084 2797502 19633 16568 18362 28060 22071 2531403 19026 17083 17208 26780 23086 2295104 18452 17607 16139 25602 24130 2084505 17908 18141 15147 24513 25201 1896306 17391 18685 14226 23506 26301 1727907 16901 19238 13369 22570 27429 1576708 16435 19801 12572 21699 28585 1440809 15990 20373 11829 20885 29769 131851 15567 20955 11137 20125 30981 12081

10 01 83821 81095 64205 78423 123923 117811 84259 11188702 78523 66233 73429 112224 88219 10122503 76091 68299 68804 107097 92291 9174904 73789 70403 64513 102375 96475 8330105 71608 72545 60530 98014 100770 7575006 69537 74725 56827 93973 105177 6898307 67569 76942 53382 90220 109696 6290808 65698 79197 50173 86725 114327 5744109 63915 81490 47182 83463 119069 525151 62215 83821 44392 80411 123923 48069

Table 6 Estimated MSE when n 30 p 7 and ρ 09 and 099

N 30 09 099Sigma k d OLS Ridge Liu New ridge OLS Ridge Liu New ridge1 01 252 227 129 203 2868 1120 226 445

02 206 139 166 682 355 41603 188 151 137 478 525 57804 173 163 116 362 736 75805 161 176 099 288 989 92506 150 190 085 237 1283 107507 141 204 075 201 1617 120708 132 220 068 174 1993 132409 125 235 062 154 2410 14271 118 252 057 138 2868 1519

5 01 6303 5658 3123 5057 71709 27885 5083 1080002 5127 3411 4103 16823 8411 971703 4682 3715 3361 11638 12757 1345404 4303 4035 2778 8652 18123 1768905 3977 4372 2314 6736 24507 2164706 3694 4725 1942 5418 31910 2518707 3445 5095 1643 4467 40332 2831608 3225 5481 1401 3754 49772 3108009 3028 5884 1206 3206 60231 335291 2852 6303 1048 2773 71709 35710

10 01 25214 22630 12475 20223 286835 111506 20239 4314802 20503 13628 16403 67243 33562 38784

8 Scientifica

32 Simulation Results and Discussion From Tables 1ndash8and Figures 1ndash6 it appears that as the values of σ increasethe MSE values also increase (Figure 3) while the sample sizeincreases as the MSE values decrease (Figure 4) Ridge Liuand proposed KL estimators uniformly dominate the ordinaryleast squares (OLS) estimator In general from these tables anincrease in the levels of multicollinearity and the number ofexplanatory variables increase the estimated MSE values of theestimators +e figures consistently show that the OLS esti-mator performs worst when there is multicollinearity FromFigures 1ndash6 and simulation Tables 1ndash8 it clearly indicated thatfor ρ 090 or less the proposed estimator uniformly

dominates the ridge regression estimator while Liu performedmuch better than both proposed and ridge estimators for smalld say 03 or lessWhen ρ 099 the ridge regression performsthe best for higher k while the proposed estimator performsthe best for say k (say 03 or less) When d k 05 andρ 099 both ridge and KL estimators outperform the Liuestimator None of the estimators uniformly dominates eachother However it appears that our proposed estimator KLperforms better in the wider space of d k in the parameterspace If we review all Tables 1ndash8 we observed that theconclusions about the performance of all estimators remainthe same for both p 3 and p 7

Table 6 Continued

N 30 09 099Sigma k d OLS Ridge Liu New ridge OLS Ridge Liu New ridge

03 18721 14846 13432 46491 50959 5370704 17205 16130 11091 34538 72431 7062605 15899 17479 9229 26865 97978 8644306 14763 18895 7737 21588 127600 10058607 13766 20376 6534 17777 161296 11308908 12882 21923 5562 14924 199068 12413409 12095 23535 4775 12725 240914 1339211 11389 25214 4138 10992 286835 142634

Table 7 Estimated MSE when n 100 p 7 and ρ 070 and 080

n 100 07 08Sigma k d OLS Ridge Liu New ridge OLS Ridge Liu New ridge1 01 0174 0173 0163 0171 0263 0259 0235 0255

02 0171 0164 0169 0255 0238 024903 0170 0165 0166 0252 0241 024304 0169 0166 0164 0249 0244 023705 0167 0168 0161 0246 0247 023206 0166 0169 0159 0243 0250 022707 0165 0170 0157 0240 0253 022208 0164 0171 0155 0238 0256 021809 0163 0173 0154 0235 0259 02141 0162 0174 0152 0233 0263 0210

5 01 4356 4320 4055 4284 6563 6474 5852 638602 4285 4087 4214 6388 5928 621603 4250 4120 4146 6304 6005 605304 4216 4153 4079 6222 6082 589505 4182 4187 4013 6143 6160 574406 4149 4220 3949 6066 6239 559807 4116 4254 3887 5991 6319 545708 4084 4288 3826 5917 6399 532209 4053 4322 3767 5846 6481 51911 4022 4356 3708 5777 6563 5066

10 01 17425 17281 16219 17138 26250 25896 23408 2554502 17140 16350 16858 25551 23713 2486603 17001 16482 16584 25216 24020 2421204 16864 16614 16316 24891 24330 2358205 16729 16748 16054 24573 24643 2297506 16597 16882 15797 24265 24959 2238907 16467 17016 15547 23964 25277 2182508 16339 17152 15301 23671 25599 2128009 16213 17288 15062 23385 25923 207551 16089 17425 14827 23107 26250 20247

Scientifica 9

Table 8 Estimated MSE when n 100 p 7 and ρ 090 and 099

n 100 09 099Sigma k d OLS Ridge Liu New ridge OLS Ridge Liu New ridge1 01 0546 0529 0431 0512 6389 4391 1624 2949

02 0513 0442 0482 3407 1934 183603 0498 0454 0456 2819 2298 145304 0485 0466 0432 2423 2718 134705 0472 0478 0411 2135 3192 135906 0460 0491 0392 1914 3721 142607 0449 0504 0375 1738 4306 151908 0439 0517 0360 1593 4945 162509 0429 0531 0346 1472 5640 17371 0420 0546 0334 1370 6389 1851

5 01 13640 13216 10676 12802 159732 109722 38895 7328402 12820 10979 12037 84915 47018 4450603 12448 11289 11336 69971 56467 3386504 12099 11605 10693 59823 67242 3014605 11770 11928 10102 52370 79343 2941706 11460 12257 9558 46597 92769 3009007 11168 12593 9056 41953 107521 3145508 10891 12935 8593 38114 123599 3317109 10628 13284 8165 34875 141003 350631 10379 13640 7768 32097 159732 37036

10 01 54558 52866 42699 51212 638928 438910 155399 29312102 51282 43914 48150 339663 187945 17787403 49796 45155 45344 279860 225785 13515104 48399 46422 42768 239236 268921 12012005 47084 47714 40397 209391 317351 11705306 45843 49032 38214 186265 371077 11959907 44670 50375 36198 167659 430097 12492208 43560 51744 34336 152274 494412 13165409 42508 53138 32612 139287 564022 1390941 41509 54558 31014 128149 638928 146866

OLSRidge

LiuKL

025

030

035

040

MSE

02 04 06 08 1000d = k

(a)

OLSRidge

LiuKL

02 04 06 08 1000d = k

20

25

30

35

MSE

(b)

Figure 1 Continued

10 Scientifica

4 Numerical Examples

To illustrate our theoretical results we consider two datasets(i) famous Portland cement data originally adopted byWoods et al [34] and (ii) French economy data from

Chatterjee and Hadi [35] and they are analyzed in thefollowing sections respectively

41 Example 1 Portland Data +ese data are widely knownas the Portland cement dataset It was originally adopted by

OLSRidge

LiuKL

025

035

045

055

MSE

02 04 06 08 1000d = k

(c)

OLSRidge

LiuKL

02 04 06 08 1000d = k

20

30

40

50

MSE

(d)

Figure 1 Estimated MSEs for n 30 Sigma 1 10 rho 070 080 and different values of k d (a) n 30 p 3 sigma 1 and rho 070(b) n 30 p 3 sigma 10 and rho 070 (c) n 30 p 3 sigma 1 and rho 080 (d) n 30 p 3 sigma 10 and rho 080

OLSRidge

LiuKL

04

06

08

10

12

MSE

02 04 06 08 1000d = k

(a)

OLSRidge

LiuKL

20

40

60

80

100M

SE

02 04 06 08 1000d = k

(b)

OLSRidge

LiuKL

02468

101214

MSE

02 04 06 08 1000d = k

(c)

OLSRidge

LiuKL

0200

600

1000

1400

MSE

02 04 06 08 1000d = k

(d)

Figure 2 Estimated MSEs for n 30 sigma 1 10 rho 090 099 and different values of k d (a) n 30 p 3 sigma 1 and rho 090(b) n 30 p 3 sigma 10 and rho 090 (c) n 30 p 3 sigma 1 and rho 099 (d) n 30 p 3 sigma 10 and rho 099

Scientifica 11

OLSRidge

LiuKL

0

10

20

30

MSE

2 4 6 8 100Sigma

(a)

OLSRidge

LiuKL

0

10

30

50

MSE

2 4 6 8 100Sigma

(b)

OLSRidge

LiuKL

0

40

80

120

MSE

2 4 6 8 100Sigma

(c)

OLSRidge

LiuKL

2 4 6 8 100Sigma

0

400

800

1200

MSE

(d)

Figure 3 EstimatedMSEs for n 30 d 05 and different values of rho and sigma (a) n 30 p 3 d 05 and rho 070 (b) n 30 p 3d 05 and rho 080 (c) n 30 p 3 d 05 and rho 090 (d) n 30 p 3 d 05 and rho 099

OLSRidge

LiuKL

50 70 9030n

010

020

030

040

MSE

(a)

OLSRidge

LiuKL

50 70 9030n

02

04

06

MSE

(b)

Figure 4 Continued

12 Scientifica

Woods et al [34] It has also been analyzed by the followingauthors Kaciranlar et al [36] Li and Yang [25] and recentlyby Lukman et al [13] +e regression model for these data isdefined as

yi β0 + β1X1 + β2X2 + β3X3 + β4X4 + εi (46)

where yi heat evolved after 180 days of curing measured incalories per gram of cement X1 tricalcium aluminateX2 tricalcium silicate X3 tetracalcium aluminoferriteand X4 β-dicalcium silicate +e correlation matrix of thepredictor variables is given in Table 9

OLSRidge

LiuKL

50 70 9030n

02

06

10

MSE

(c)

OLSRidge

LiuKL

50 70 9030n

0

2

4

6

8

12

MSE

(d)

Figure 4 Estimated MSEs for sigma 1 p 3 and different values of rho and sample size (a)p 3 sigma 1 d 05 and rho 070(b)p 3 sigma 1 d 05 and rho 080 (c)p 3 sigma 1 d 05 and rho 090 (d)p 3 sigma 1 d 05 and rho 099

OLSRidge

LiuKL

20

30

40

MSE

4 5 6 7 83p

(a)

OLSRidge

LiuKL

3

4

5

6

7M

SE

4 5 6 7 83p

(b)

6

8

10

14

MSE

OLSRidge

LiuKL

4 5 6 7 83p

(c)

OLSRidge

LiuKL

4 5 6 7 83p

0

50

100

150

MSE

(d)

Figure 5 Estimated MSEs for n 100 d 05 sigma 5 and different values of rho and p (a) n 100 sigma 5 d 05 and rho 070 (b)n 100 sigma 5 d 05 and rho 080 (c) n 100 sigma 5 d 05 and rho 090 (d) n 100 sigma 5 d 05 and rho 099

Scientifica 13

+e variance inflation factors are VIF1 = 3850VIF2 = 25442 VIF3 = 4687 and VIF4 = 28251 Eigen-values of XprimeX are λ1 44676206 λ2 5965422

λ3 809952 and λ4 105419 and the condition numberof XprimeX is approximately 424 +e VIFs the eigenvalues

and the condition number all indicate the presence ofsevere multicollinearity +e estimated parameters andMSE are presented in Table 10 It appears from Table 11that the proposed estimator performed the best in thesense of smaller MSE

OLSRidge

LiuKL

0

100

200

300

MSE

075 085 095065Rho

(a)

OLSRidge

LiuKL

0

200

400

600

800

MSE

075 085 095065Rho

(b)

OLSRidge

LiuKL

0

20

40

60

80

MSE

075 085 095065Rho

(c)

OLSRidge

LiuKL

0

50

100

150

MSE

075 085 095065Rho

(d)

Figure 6 Estimated MSEs for n 100 p 3 7 d 05 sigma 5 and different values of rho (a) n 30 p 3 sigma 5 and d 05 (b)n 30 p 7 sigma 5 and d 05 (c) n 100 p 3 sigma 5 and d 05 (d) n 100 p 7 sigma 5 and d 05

Table 9 Correlation matrix

X1 X2 X3 X4

X1 1000 0229 minus 0824 minus 0245X2 0229 1000 minus 0139 minus 0973X3 minus 0824 minus 0139 1000 0030X4 minus 0245 minus 0973 0030 1000

Table 10 +e results of regression coefficients and the corresponding MSE values

Coef 1113954α 1113954α(k) 1113954α(d)1113954dalt 1113954αKL(kHNM) 1113954αKL(kmin)

α0 624054 85870 276490 minus 197876 276068α1 15511 21046 19010 23965 19090α2 05102 10648 08701 13573 08688α3 01019 06681 04621 09666 04680α4 minus 01441 03996 02082 06862 02074MSE 491209 298983 2170963 7255603 217096kd mdash 00077 044195 000235 000047

14 Scientifica

42 Example 2 French Economy Data +e French economydata in Chatterjee and Hadi [37] are considered in this ex-ample It has been analyzed by Malinvard [38] and Liu [6]among others+e variables are imports domestic productionstock formation and domestic consumption All are measuredin milliards of French francs for the years 1949 through 1966

+e regression model for these data is defined as

yi β0 + β1X1 + β2X2 + β3X3 + εi (47)

where yi IMPORT X1 domestic production X2 stockformation and X3 domestic consumption +e correlationmatrix of the predicted variable is given in Table 12

+e variance inflation factors areVIF1 469688VIF2 1047 and VIF3 469338 +e ei-genvalues of the XprimeX matrix are λ1 161779 λ2 158 andλ3 4961 and the condition number is 32612 If we reviewthe above correlation matrix VIFs and condition number itcan be said that there is presence of severe multicollinearityexisting in the predictor variables

+e biasing parameter for the new estimator is defined in(41) and (42) +e biasing parameter for the ridge and Liuestimator is provided in (6) (8) and (9) respectively

We analyzed the data using the biasing parameters foreach of the estimators and presented the results in Tables 10and 11 It can be seen from Tables 10 and 11 that theproposed estimator performed the best in the sense ofsmaller MSE

5 Summary and Concluding Remarks

In this paper we introduced a new biased estimator toovercome the multicollinearity problem for the multiplelinear regression model and provided the estimation tech-nique of the biasing parameter A simulation study has beenconducted to compare the performance of the proposedestimator and Liu [6] and ridge regression estimators [3]Simulation results evidently show that the proposed esti-mator performed better than both Liu and ridge under somecondition on the shrinkage parameter Two sets of real-lifedata are analyzed to illustrate the benefits of using the newestimator in the context of a linear regression model +eproposed estimator is recommended for researchers in this

area Its application can be extended to other regressionmodels for example logistic regression Poisson ZIP andrelated models and those possibilities are under currentinvestigation [37 39 40]

Data Availability

Data will be made available on request

Conflicts of Interest

+e authors declare that they have no conflicts of interest

Acknowledgments

We are dedicating this article to those who lost their livesbecause of COVID-19

References

[1] C Stein ldquoInadmissibility of the usual estimator for mean ofmultivariate normal distributionrdquo in Proceedings of the 0irdBerkley Symposium on Mathematical and Statistics Proba-bility J Neyman Ed vol 1 pp 197ndash206 Springer BerlinGermany 1956

[2] W F Massy ldquoPrincipal components regression in exploratorystatistical researchrdquo Journal of the American Statistical As-sociation vol 60 no 309 pp 234ndash256 1965

[3] A E Hoerl and R W Kennard ldquoRidge regression biasedestimation for nonorthogonal problemsrdquo Technometricsvol 12 no 1 pp 55ndash67 1970

[4] L S Mayer and T A Willke ldquoOn biased estimation in linearmodelsrdquo Technometrics vol 15 no 3 pp 497ndash508 1973

[5] B F Swindel ldquoGood ridge estimators based on prior infor-mationrdquo Communications in Statistics-0eory and Methodsvol 5 no 11 pp 1065ndash1075 1976

[6] K Liu ldquoA new class of biased estimate in linear regressionrdquoCommunication in Statistics- 0eory and Methods vol 22pp 393ndash402 1993

[7] F Akdeniz and S Kaccediliranlar ldquoOn the almost unbiasedgeneralized liu estimator and unbiased estimation of the biasand mserdquo Communications in Statistics-0eory and Methodsvol 24 no 7 pp 1789ndash1797 1995

[8] M R Ozkale and S Kaccediliranlar ldquo+e restricted and unre-stricted two-parameter estimatorsrdquo Communications in Sta-tistics-0eory and Methods vol 36 no 15 pp 2707ndash27252007

[9] S Sakallıoglu and S Kaccedilıranlar ldquoA new biased estimatorbased on ridge estimationrdquo Statistical Papers vol 49 no 4pp 669ndash689 2008

Table 11 +e results of regression coefficients and the corresponding MSE values

Coef 1113954α 1113954α(k) 1113954α(d)1113954dalt 1113954α(d)1113954dopt 1113954αKL(kHNM) 1113954αKL(kmin)

α0 minus 197127 minus 167613 minus 125762 minus 188410 minus 165855 minus 188782α1 00327 01419 02951 00648 01485 00636α2 04059 03576 02875 03914 03548 03922α3 02421 00709 minus 01696 01918 00606 01937MSE 173326 2130519 5828312 1660293 2211899 1660168kd mdash 00527 05282 09423 00258 00065

Table 12 Correlation matrix

X1 X2 X3

X1 1000 0210 0999X2 0210 1000 0208X3 0999 0208 1000

Scientifica 15

[10] H Yang and X Chang ldquoA new two-parameter estimator inlinear regressionrdquo Communications in Statistics-0eory andMethods vol 39 no 6 pp 923ndash934 2010

[11] M Roozbeh ldquoOptimal QR-based estimation in partially linearregression models with correlated errors using GCV crite-rionrdquo Computational Statistics amp Data Analysis vol 117pp 45ndash61 2018

[12] F Akdeniz and M Roozbeh ldquoGeneralized difference-basedweightedmixed almost unbiased ridge estimator in partially linearmodelsrdquo Statistical Papers vol 60 no 5 pp 1717ndash1739 2019

[13] A F Lukman K Ayinde S Binuomote and O A ClementldquoModified ridge-type estimator to combat multicollinearityapplication to chemical datardquo Journal of Chemometricsvol 33 no 5 p e3125 2019

[14] A F Lukman K Ayinde S K Sek and E Adewuyi ldquoAmodified new two-parameter estimator in a linear regressionmodelrdquo Modelling and Simulation in Engineering vol 2019Article ID 6342702 10 pages 2019

[15] A E Hoerl R W Kannard and K F Baldwin ldquoRidge re-gressionsome simulationsrdquo Communications in Statisticsvol 4 no 2 pp 105ndash123 1975

[16] G C McDonald and D I Galarneau ldquoA monte carlo eval-uation of some ridge-type estimatorsrdquo Journal of the Amer-ican Statistical Association vol 70 no 350 pp 407ndash416 1975

[17] J F Lawless and P Wang ldquoA simulation study of ridge andother regression estimatorsrdquo Communications in Statistics-0eory and Methods vol 5 no 4 pp 307ndash323 1976

[18] D W Wichern and G A Churchill ldquoA comparison of ridgeestimatorsrdquo Technometrics vol 20 no 3 pp 301ndash311 1978

[19] B M G Kibria ldquoPerformance of some new ridge regressionestimatorsrdquo Communications in Statistics-Simulation andComputation vol 32 no 1 pp 419ndash435 2003

[20] A F Lukman and K Ayinde ldquoReview and classifications ofthe ridge parameter estimation techniquesrdquoHacettepe Journalof Mathematics and Statistics vol 46 no 5 pp 953ndash967 2017

[21] A K M E Saleh M Arashi and B M G Kibria 0eory ofRidge Regression Estimation with Applications WileyHoboken NJ USA 2019

[22] K Liu ldquoUsing Liu-type estimator to combat collinearityrdquoCommunications in Statistics-0eory and Methods vol 32no 5 pp 1009ndash1020 2003

[23] K Alheety and B M G Kibria ldquoOn the Liu and almostunbiased Liu estimators in the presence of multicollinearitywith heteroscedastic or correlated errorsrdquo Surveys in Math-ematics and its Applications vol 4 pp 155ndash167 2009

[24] X-Q Liu ldquoImproved Liu estimator in a linear regressionmodelrdquo Journal of Statistical Planning and Inference vol 141no 1 pp 189ndash196 2011

[25] Y Li and H Yang ldquoA new Liu-type estimator in linear regressionmodelrdquo Statistical Papers vol 53 no 2 pp 427ndash437 2012

[26] B Kan O Alpu and B Yazıcı ldquoRobust ridge and robust Liuestimator for regression based on the LTS estimatorrdquo Journalof Applied Statistics vol 40 no 3 pp 644ndash655 2013

[27] R A Farghali ldquoGeneralized Liu-type estimator for linearregressionrdquo International Journal of Research and Reviews inApplied Sciences vol 38 no 1 pp 52ndash63 2019

[28] S G Wang M X Wu and Z Z Jia Matrix InequalitiesChinese Science Press Beijing China 2nd edition 2006

[29] R W Farebrother ldquoFurther results on the mean square errorof ridge regressionrdquo Journal of the Royal Statistical SocietySeries B (Methodological) vol 38 no 3 pp 248ndash250 1976

[30] G Trenkler and H Toutenburg ldquoMean squared error matrixcomparisons between biased estimators-an overview of recentresultsrdquo Statistical Papers vol 31 no 1 pp 165ndash179 1990

[31] B M G Kibria and S Banik ldquoSome ridge regression esti-mators and their performancesrdquo Journal of Modern AppliedStatistical Methods vol 15 no 1 pp 206ndash238 2016

[32] D G Gibbons ldquoA simulation study of some ridge estimatorsrdquoJournal of the American Statistical Association vol 76no 373 pp 131ndash139 1981

[33] J P Newhouse and S D Oman ldquoAn evaluation of ridgeestimators A report prepared for United States air forceproject RANDrdquo 1971

[34] H Woods H H Steinour and H R Starke ldquoEffect ofcomposition of Portland cement on heat evolved duringhardeningrdquo Industrial amp Engineering Chemistry vol 24no 11 pp 1207ndash1214 1932

[35] S Chatterjee and A S Hadi Regression Analysis by ExampleWiley Hoboken NJ USA 1977

[36] S Kaciranlar S Sakallioglu F Akdeniz G P H Styan andH J Werner ldquoA new biased estimator in linear regression anda detailed analysis of the widely-analysed dataset on portlandcementrdquo Sankhya 0e Indian Journal of Statistics Series Bvol 61 pp 443ndash459 1999

[37] S Chatterjee and A S Haadi Regression Analysis by ExampleWiley Hoboken NJ USA 2006

[38] E Malinvard Statistical Methods of Econometrics North-Holland Publishing Company Amsterdam Netherlands 3rdedition 1980

[39] D N Gujarati Basic Econometrics McGraw-Hill New YorkNY USA 1995

[40] A F Lukman K Ayinde and A S Ajiboye ldquoMonte Carlostudy of some classification-based ridge parameter estima-torsrdquo Journal of Modern Applied Statistical Methods vol 16no 1 pp 428ndash451 2017

16 Scientifica

Table 2 Estimated MSE when n 30 p 3 and ρ 090 and 099

n 30 09 099Sigma k d OLS Ridge Liu New est OLS Ridge Liu New est1 01 1154 1012 0532 0883 12774 4155 0857 1128

02 0899 0583 0691 2339 1388 194603 0809 0638 0555 1603 2117 327804 0736 0697 0459 1214 3045 441605 0675 0762 0392 0978 4170 534006 0625 0831 0346 0821 5495 609707 0582 0905 0317 0712 7017 672508 0545 0983 0299 0631 8738 725509 0514 1066 0291 0571 10656 77081 0487 1154 0289 0524 12774 8100

5 01 28461 24840 12067 21501 319335 102389 17451 2338302 21945 13492 16402 56008 31445 4036803 19588 15017 12625 36978 50327 7144704 17641 16640 9805 26816 74095 9832205 16010 18362 7690 20580 102751 12026906 14627 20184 6104 16415 136293 13826807 13442 22105 4917 13467 174723 15324008 12418 24124 4036 11293 218040 16588009 11526 26243 3393 9637 266244 1766951 10741 28461 2935 8343 319335 186058

10 01 113841 99331 48088 85947 1277429 409249 69149 9286802 87726 53814 65494 223571 125195 16055403 78277 59935 50326 147369 200793 28474904 70466 66450 38986 106666 295943 39218405 63919 73361 30469 81687 410644 47994006 58368 80667 24064 64998 544898 55191607 53612 88368 19262 53189 698703 61179408 49498 96464 15687 44476 872060 66235009 45910 104955 13064 37839 1064960 7056111 42758 113841 11182 32655 1277429 743065

Table 3 Estimated MSE when n 100 p 3 and ρ 070 and 080

n 100 09 099Sigma k d OLS Ridge Liu New est OLS Ridge Liu New est1 01 01124 01121 01105 01118 01492 01478 01396 01465

02 01118 01107 01114 01465 01404 0144103 01116 01108 01110 01453 01414 0142004 01114 01110 01106 01442 01423 0140105 01112 01112 01104 01432 01434 0138406 01110 01114 01101 01422 01444 0136907 01108 01116 01100 01412 01455 0135608 01106 01119 01099 01403 01467 0134509 01105 01121 01099 01395 01479 013361 01104 01124 01099 01387 01492 01328

5 01 20631 20452 19126 20274 32440 31954 28523 3147202 20276 19289 19924 31480 28942 3053803 20102 19454 19583 31019 29365 2963804 19932 19619 19249 30570 29793 2877105 19764 19785 18922 30133 30224 2793406 19599 19952 18603 29707 30659 2712807 19436 20121 18291 29291 31098 2635008 19276 20290 17986 28887 31542 2560009 19119 20460 17688 28492 31989 248761 18964 20631 17396 28108 32440 24178

10 01 81632 80901 75481 80174 129200 127234 113344 12528702 80182 76150 78747 125320 115045 12151103 79474 76822 77351 123456 116761 11786704 78777 77498 75984 121640 118493 114349

6 Scientifica

objective is to compare the performance of the proposedestimator with ridge regression and Liu estimators weconsider k d 01 02 1 We have restricted k between0 and 1 as Wichern and Churchill [18] have found that theridge regression estimator is better than the OLS when k isbetween 0 and 1 Kan et al [26] also suggested a smallervalue of k (less than 1) is better Simulation studies arerepeated 1000 times for the sample sizes n 30 and 100 andσ2 1 25 and 100 For each replicate we compute the meansquare error (MSE) of the estimators by using the followingequation

MSE αlowast( 1113857 1

10001113944

1000

i1αlowast minus α( 1113857prime αlowast minus α( 1113857 (45)

where αlowast would be any of the estimators (OLS ridge Liu or KL)Smaller MSE of the estimators will be considered the best one

+e simulated results for n 30 p 3 and ρ 070 080and ρ 090 099 are presented in Tables 1 and 2 respec-tively and for n 100 p 3 and ρ 07 080 and ρ 090099 are presented in Tables 3 and 4 respectively +ecorresponding simulated results for n 30 100 and p 7 arepresented in Tables 5ndash8 For a better visualization we haveplotted MSE vs d for n 30 σ 10 and ρ 070 090 and099 in Figures 1ndash3 respectively We also plotted MSE vs σfor n 30 d 50 and ρ 090 and 099 which is presentedin Figures 4 and 5 respectively Finally to see the effect ofsample size on MSE we plotted MSE vs sample size ford 05 and ρ 090 and presented in Figure 6

Table 3 Continued

n 100 09 099Sigma k d OLS Ridge Liu New est OLS Ridge Liu New est

05 78091 78178 74646 119870 120239 11095306 77415 78862 73336 118144 122001 10767407 76750 79549 72053 116462 123778 10450608 76096 80240 70797 114821 125570 10144709 75451 80934 69568 113220 127377 984901 74816 81632 68364 111658 129200 95634

Table 4 Estimated MSE when n 100 p 3 and ρ 090 and 099

n 30 09 099Sigma k d OLS Ridge Liu New est OLS Ridge Liu New est1 01 0287 0278 0230 0270 3072 2141 0688 1423

02 0270 0236 0255 1621 0836 076903 0263 0241 0242 1298 1013 052804 0256 0247 0231 1083 1219 047205 0250 0253 0221 0930 1455 050606 0244 0259 0213 0819 1720 058307 0239 0265 0206 0733 2014 068008 0234 0272 0200 0667 2337 078509 0230 0279 0195 0613 2690 08921 0226 0287 0191 0570 3072 0997

5 01 6958 6719 5256 6486 76772 53314 14746 3468902 6495 5431 6050 39905 18971 1666003 6283 5610 5649 31412 23862 883404 6083 5792 5278 25626 29420 580305 5893 5977 4935 21466 35645 517406 5714 6166 4617 18350 42537 579507 5544 6359 4324 15939 50096 707208 5383 6555 4052 14024 58321 868609 5230 6754 3799 12471 67213 104581 5085 6958 3566 11189 76772 12287

10 01 27809 26853 20970 25916 307086 213255 58717 13868502 25951 21675 24167 159582 75683 6635403 25100 22394 22551 125559 95308 3481504 24296 23126 21056 102365 117590 2246305 23535 23872 19672 85681 142529 1974306 22815 24632 18389 73175 170126 2204507 22131 25406 17200 63493 200380 2699508 21482 26193 16096 55802 233291 3330809 20865 26994 15071 49561 268860 402701 20279 27809 14120 44407 307086 47470

Scientifica 7

Table 5 Estimated MSE when n 30 p 7 and ρ 070 and 080

n 30 07 08Sigma k d OLS Ridge Liu New ridge OLS Ridge Liu New ridge1 01 0838 0811 0651 0785 1239 1179 0859 1121

02 0786 0670 0737 1124 0895 101803 0763 0689 0694 1074 0933 092804 0741 0709 0654 1029 0973 085005 0720 0729 0618 0987 1014 078106 0701 0750 0586 0949 1056 072107 0682 0771 0556 0914 1100 066908 0665 0793 0529 0881 1145 062309 0649 0815 0505 0851 1191 05831 0633 0838 0484 0823 1239 0549

5 01 20955 20275 16063 19608 30981 29455 21084 2797502 19633 16568 18362 28060 22071 2531403 19026 17083 17208 26780 23086 2295104 18452 17607 16139 25602 24130 2084505 17908 18141 15147 24513 25201 1896306 17391 18685 14226 23506 26301 1727907 16901 19238 13369 22570 27429 1576708 16435 19801 12572 21699 28585 1440809 15990 20373 11829 20885 29769 131851 15567 20955 11137 20125 30981 12081

10 01 83821 81095 64205 78423 123923 117811 84259 11188702 78523 66233 73429 112224 88219 10122503 76091 68299 68804 107097 92291 9174904 73789 70403 64513 102375 96475 8330105 71608 72545 60530 98014 100770 7575006 69537 74725 56827 93973 105177 6898307 67569 76942 53382 90220 109696 6290808 65698 79197 50173 86725 114327 5744109 63915 81490 47182 83463 119069 525151 62215 83821 44392 80411 123923 48069

Table 6 Estimated MSE when n 30 p 7 and ρ 09 and 099

N 30 09 099Sigma k d OLS Ridge Liu New ridge OLS Ridge Liu New ridge1 01 252 227 129 203 2868 1120 226 445

02 206 139 166 682 355 41603 188 151 137 478 525 57804 173 163 116 362 736 75805 161 176 099 288 989 92506 150 190 085 237 1283 107507 141 204 075 201 1617 120708 132 220 068 174 1993 132409 125 235 062 154 2410 14271 118 252 057 138 2868 1519

5 01 6303 5658 3123 5057 71709 27885 5083 1080002 5127 3411 4103 16823 8411 971703 4682 3715 3361 11638 12757 1345404 4303 4035 2778 8652 18123 1768905 3977 4372 2314 6736 24507 2164706 3694 4725 1942 5418 31910 2518707 3445 5095 1643 4467 40332 2831608 3225 5481 1401 3754 49772 3108009 3028 5884 1206 3206 60231 335291 2852 6303 1048 2773 71709 35710

10 01 25214 22630 12475 20223 286835 111506 20239 4314802 20503 13628 16403 67243 33562 38784

8 Scientifica

32 Simulation Results and Discussion From Tables 1ndash8and Figures 1ndash6 it appears that as the values of σ increasethe MSE values also increase (Figure 3) while the sample sizeincreases as the MSE values decrease (Figure 4) Ridge Liuand proposed KL estimators uniformly dominate the ordinaryleast squares (OLS) estimator In general from these tables anincrease in the levels of multicollinearity and the number ofexplanatory variables increase the estimated MSE values of theestimators +e figures consistently show that the OLS esti-mator performs worst when there is multicollinearity FromFigures 1ndash6 and simulation Tables 1ndash8 it clearly indicated thatfor ρ 090 or less the proposed estimator uniformly

dominates the ridge regression estimator while Liu performedmuch better than both proposed and ridge estimators for smalld say 03 or lessWhen ρ 099 the ridge regression performsthe best for higher k while the proposed estimator performsthe best for say k (say 03 or less) When d k 05 andρ 099 both ridge and KL estimators outperform the Liuestimator None of the estimators uniformly dominates eachother However it appears that our proposed estimator KLperforms better in the wider space of d k in the parameterspace If we review all Tables 1ndash8 we observed that theconclusions about the performance of all estimators remainthe same for both p 3 and p 7

Table 6 Continued

N 30 09 099Sigma k d OLS Ridge Liu New ridge OLS Ridge Liu New ridge

03 18721 14846 13432 46491 50959 5370704 17205 16130 11091 34538 72431 7062605 15899 17479 9229 26865 97978 8644306 14763 18895 7737 21588 127600 10058607 13766 20376 6534 17777 161296 11308908 12882 21923 5562 14924 199068 12413409 12095 23535 4775 12725 240914 1339211 11389 25214 4138 10992 286835 142634

Table 7 Estimated MSE when n 100 p 7 and ρ 070 and 080

n 100 07 08Sigma k d OLS Ridge Liu New ridge OLS Ridge Liu New ridge1 01 0174 0173 0163 0171 0263 0259 0235 0255

02 0171 0164 0169 0255 0238 024903 0170 0165 0166 0252 0241 024304 0169 0166 0164 0249 0244 023705 0167 0168 0161 0246 0247 023206 0166 0169 0159 0243 0250 022707 0165 0170 0157 0240 0253 022208 0164 0171 0155 0238 0256 021809 0163 0173 0154 0235 0259 02141 0162 0174 0152 0233 0263 0210

5 01 4356 4320 4055 4284 6563 6474 5852 638602 4285 4087 4214 6388 5928 621603 4250 4120 4146 6304 6005 605304 4216 4153 4079 6222 6082 589505 4182 4187 4013 6143 6160 574406 4149 4220 3949 6066 6239 559807 4116 4254 3887 5991 6319 545708 4084 4288 3826 5917 6399 532209 4053 4322 3767 5846 6481 51911 4022 4356 3708 5777 6563 5066

10 01 17425 17281 16219 17138 26250 25896 23408 2554502 17140 16350 16858 25551 23713 2486603 17001 16482 16584 25216 24020 2421204 16864 16614 16316 24891 24330 2358205 16729 16748 16054 24573 24643 2297506 16597 16882 15797 24265 24959 2238907 16467 17016 15547 23964 25277 2182508 16339 17152 15301 23671 25599 2128009 16213 17288 15062 23385 25923 207551 16089 17425 14827 23107 26250 20247

Scientifica 9

Table 8 Estimated MSE when n 100 p 7 and ρ 090 and 099

n 100 09 099Sigma k d OLS Ridge Liu New ridge OLS Ridge Liu New ridge1 01 0546 0529 0431 0512 6389 4391 1624 2949

02 0513 0442 0482 3407 1934 183603 0498 0454 0456 2819 2298 145304 0485 0466 0432 2423 2718 134705 0472 0478 0411 2135 3192 135906 0460 0491 0392 1914 3721 142607 0449 0504 0375 1738 4306 151908 0439 0517 0360 1593 4945 162509 0429 0531 0346 1472 5640 17371 0420 0546 0334 1370 6389 1851

5 01 13640 13216 10676 12802 159732 109722 38895 7328402 12820 10979 12037 84915 47018 4450603 12448 11289 11336 69971 56467 3386504 12099 11605 10693 59823 67242 3014605 11770 11928 10102 52370 79343 2941706 11460 12257 9558 46597 92769 3009007 11168 12593 9056 41953 107521 3145508 10891 12935 8593 38114 123599 3317109 10628 13284 8165 34875 141003 350631 10379 13640 7768 32097 159732 37036

10 01 54558 52866 42699 51212 638928 438910 155399 29312102 51282 43914 48150 339663 187945 17787403 49796 45155 45344 279860 225785 13515104 48399 46422 42768 239236 268921 12012005 47084 47714 40397 209391 317351 11705306 45843 49032 38214 186265 371077 11959907 44670 50375 36198 167659 430097 12492208 43560 51744 34336 152274 494412 13165409 42508 53138 32612 139287 564022 1390941 41509 54558 31014 128149 638928 146866

OLSRidge

LiuKL

025

030

035

040

MSE

02 04 06 08 1000d = k

(a)

OLSRidge

LiuKL

02 04 06 08 1000d = k

20

25

30

35

MSE

(b)

Figure 1 Continued

10 Scientifica

4 Numerical Examples

To illustrate our theoretical results we consider two datasets(i) famous Portland cement data originally adopted byWoods et al [34] and (ii) French economy data from

Chatterjee and Hadi [35] and they are analyzed in thefollowing sections respectively

41 Example 1 Portland Data +ese data are widely knownas the Portland cement dataset It was originally adopted by

OLSRidge

LiuKL

025

035

045

055

MSE

02 04 06 08 1000d = k

(c)

OLSRidge

LiuKL

02 04 06 08 1000d = k

20

30

40

50

MSE

(d)

Figure 1 Estimated MSEs for n 30 Sigma 1 10 rho 070 080 and different values of k d (a) n 30 p 3 sigma 1 and rho 070(b) n 30 p 3 sigma 10 and rho 070 (c) n 30 p 3 sigma 1 and rho 080 (d) n 30 p 3 sigma 10 and rho 080

OLSRidge

LiuKL

04

06

08

10

12

MSE

02 04 06 08 1000d = k

(a)

OLSRidge

LiuKL

20

40

60

80

100M

SE

02 04 06 08 1000d = k

(b)

OLSRidge

LiuKL

02468

101214

MSE

02 04 06 08 1000d = k

(c)

OLSRidge

LiuKL

0200

600

1000

1400

MSE

02 04 06 08 1000d = k

(d)

Figure 2 Estimated MSEs for n 30 sigma 1 10 rho 090 099 and different values of k d (a) n 30 p 3 sigma 1 and rho 090(b) n 30 p 3 sigma 10 and rho 090 (c) n 30 p 3 sigma 1 and rho 099 (d) n 30 p 3 sigma 10 and rho 099

Scientifica 11

OLSRidge

LiuKL

0

10

20

30

MSE

2 4 6 8 100Sigma

(a)

OLSRidge

LiuKL

0

10

30

50

MSE

2 4 6 8 100Sigma

(b)

OLSRidge

LiuKL

0

40

80

120

MSE

2 4 6 8 100Sigma

(c)

OLSRidge

LiuKL

2 4 6 8 100Sigma

0

400

800

1200

MSE

(d)

Figure 3 EstimatedMSEs for n 30 d 05 and different values of rho and sigma (a) n 30 p 3 d 05 and rho 070 (b) n 30 p 3d 05 and rho 080 (c) n 30 p 3 d 05 and rho 090 (d) n 30 p 3 d 05 and rho 099

OLSRidge

LiuKL

50 70 9030n

010

020

030

040

MSE

(a)

OLSRidge

LiuKL

50 70 9030n

02

04

06

MSE

(b)

Figure 4 Continued

12 Scientifica

Woods et al [34] It has also been analyzed by the followingauthors Kaciranlar et al [36] Li and Yang [25] and recentlyby Lukman et al [13] +e regression model for these data isdefined as

yi β0 + β1X1 + β2X2 + β3X3 + β4X4 + εi (46)

where yi heat evolved after 180 days of curing measured incalories per gram of cement X1 tricalcium aluminateX2 tricalcium silicate X3 tetracalcium aluminoferriteand X4 β-dicalcium silicate +e correlation matrix of thepredictor variables is given in Table 9

OLSRidge

LiuKL

50 70 9030n

02

06

10

MSE

(c)

OLSRidge

LiuKL

50 70 9030n

0

2

4

6

8

12

MSE

(d)

Figure 4 Estimated MSEs for sigma 1 p 3 and different values of rho and sample size (a)p 3 sigma 1 d 05 and rho 070(b)p 3 sigma 1 d 05 and rho 080 (c)p 3 sigma 1 d 05 and rho 090 (d)p 3 sigma 1 d 05 and rho 099

OLSRidge

LiuKL

20

30

40

MSE

4 5 6 7 83p

(a)

OLSRidge

LiuKL

3

4

5

6

7M

SE

4 5 6 7 83p

(b)

6

8

10

14

MSE

OLSRidge

LiuKL

4 5 6 7 83p

(c)

OLSRidge

LiuKL

4 5 6 7 83p

0

50

100

150

MSE

(d)

Figure 5 Estimated MSEs for n 100 d 05 sigma 5 and different values of rho and p (a) n 100 sigma 5 d 05 and rho 070 (b)n 100 sigma 5 d 05 and rho 080 (c) n 100 sigma 5 d 05 and rho 090 (d) n 100 sigma 5 d 05 and rho 099

Scientifica 13

+e variance inflation factors are VIF1 = 3850VIF2 = 25442 VIF3 = 4687 and VIF4 = 28251 Eigen-values of XprimeX are λ1 44676206 λ2 5965422

λ3 809952 and λ4 105419 and the condition numberof XprimeX is approximately 424 +e VIFs the eigenvalues

and the condition number all indicate the presence ofsevere multicollinearity +e estimated parameters andMSE are presented in Table 10 It appears from Table 11that the proposed estimator performed the best in thesense of smaller MSE

OLSRidge

LiuKL

0

100

200

300

MSE

075 085 095065Rho

(a)

OLSRidge

LiuKL

0

200

400

600

800

MSE

075 085 095065Rho

(b)

OLSRidge

LiuKL

0

20

40

60

80

MSE

075 085 095065Rho

(c)

OLSRidge

LiuKL

0

50

100

150

MSE

075 085 095065Rho

(d)

Figure 6 Estimated MSEs for n 100 p 3 7 d 05 sigma 5 and different values of rho (a) n 30 p 3 sigma 5 and d 05 (b)n 30 p 7 sigma 5 and d 05 (c) n 100 p 3 sigma 5 and d 05 (d) n 100 p 7 sigma 5 and d 05

Table 9 Correlation matrix

X1 X2 X3 X4

X1 1000 0229 minus 0824 minus 0245X2 0229 1000 minus 0139 minus 0973X3 minus 0824 minus 0139 1000 0030X4 minus 0245 minus 0973 0030 1000

Table 10 +e results of regression coefficients and the corresponding MSE values

Coef 1113954α 1113954α(k) 1113954α(d)1113954dalt 1113954αKL(kHNM) 1113954αKL(kmin)

α0 624054 85870 276490 minus 197876 276068α1 15511 21046 19010 23965 19090α2 05102 10648 08701 13573 08688α3 01019 06681 04621 09666 04680α4 minus 01441 03996 02082 06862 02074MSE 491209 298983 2170963 7255603 217096kd mdash 00077 044195 000235 000047

14 Scientifica

42 Example 2 French Economy Data +e French economydata in Chatterjee and Hadi [37] are considered in this ex-ample It has been analyzed by Malinvard [38] and Liu [6]among others+e variables are imports domestic productionstock formation and domestic consumption All are measuredin milliards of French francs for the years 1949 through 1966

+e regression model for these data is defined as

yi β0 + β1X1 + β2X2 + β3X3 + εi (47)

where yi IMPORT X1 domestic production X2 stockformation and X3 domestic consumption +e correlationmatrix of the predicted variable is given in Table 12

+e variance inflation factors areVIF1 469688VIF2 1047 and VIF3 469338 +e ei-genvalues of the XprimeX matrix are λ1 161779 λ2 158 andλ3 4961 and the condition number is 32612 If we reviewthe above correlation matrix VIFs and condition number itcan be said that there is presence of severe multicollinearityexisting in the predictor variables

+e biasing parameter for the new estimator is defined in(41) and (42) +e biasing parameter for the ridge and Liuestimator is provided in (6) (8) and (9) respectively

We analyzed the data using the biasing parameters foreach of the estimators and presented the results in Tables 10and 11 It can be seen from Tables 10 and 11 that theproposed estimator performed the best in the sense ofsmaller MSE

5 Summary and Concluding Remarks

In this paper we introduced a new biased estimator toovercome the multicollinearity problem for the multiplelinear regression model and provided the estimation tech-nique of the biasing parameter A simulation study has beenconducted to compare the performance of the proposedestimator and Liu [6] and ridge regression estimators [3]Simulation results evidently show that the proposed esti-mator performed better than both Liu and ridge under somecondition on the shrinkage parameter Two sets of real-lifedata are analyzed to illustrate the benefits of using the newestimator in the context of a linear regression model +eproposed estimator is recommended for researchers in this

area Its application can be extended to other regressionmodels for example logistic regression Poisson ZIP andrelated models and those possibilities are under currentinvestigation [37 39 40]

Data Availability

Data will be made available on request

Conflicts of Interest

+e authors declare that they have no conflicts of interest

Acknowledgments

We are dedicating this article to those who lost their livesbecause of COVID-19

References

[1] C Stein ldquoInadmissibility of the usual estimator for mean ofmultivariate normal distributionrdquo in Proceedings of the 0irdBerkley Symposium on Mathematical and Statistics Proba-bility J Neyman Ed vol 1 pp 197ndash206 Springer BerlinGermany 1956

[2] W F Massy ldquoPrincipal components regression in exploratorystatistical researchrdquo Journal of the American Statistical As-sociation vol 60 no 309 pp 234ndash256 1965

[3] A E Hoerl and R W Kennard ldquoRidge regression biasedestimation for nonorthogonal problemsrdquo Technometricsvol 12 no 1 pp 55ndash67 1970

[4] L S Mayer and T A Willke ldquoOn biased estimation in linearmodelsrdquo Technometrics vol 15 no 3 pp 497ndash508 1973

[5] B F Swindel ldquoGood ridge estimators based on prior infor-mationrdquo Communications in Statistics-0eory and Methodsvol 5 no 11 pp 1065ndash1075 1976

[6] K Liu ldquoA new class of biased estimate in linear regressionrdquoCommunication in Statistics- 0eory and Methods vol 22pp 393ndash402 1993

[7] F Akdeniz and S Kaccediliranlar ldquoOn the almost unbiasedgeneralized liu estimator and unbiased estimation of the biasand mserdquo Communications in Statistics-0eory and Methodsvol 24 no 7 pp 1789ndash1797 1995

[8] M R Ozkale and S Kaccediliranlar ldquo+e restricted and unre-stricted two-parameter estimatorsrdquo Communications in Sta-tistics-0eory and Methods vol 36 no 15 pp 2707ndash27252007

[9] S Sakallıoglu and S Kaccedilıranlar ldquoA new biased estimatorbased on ridge estimationrdquo Statistical Papers vol 49 no 4pp 669ndash689 2008

Table 11 +e results of regression coefficients and the corresponding MSE values

Coef 1113954α 1113954α(k) 1113954α(d)1113954dalt 1113954α(d)1113954dopt 1113954αKL(kHNM) 1113954αKL(kmin)

α0 minus 197127 minus 167613 minus 125762 minus 188410 minus 165855 minus 188782α1 00327 01419 02951 00648 01485 00636α2 04059 03576 02875 03914 03548 03922α3 02421 00709 minus 01696 01918 00606 01937MSE 173326 2130519 5828312 1660293 2211899 1660168kd mdash 00527 05282 09423 00258 00065

Table 12 Correlation matrix

X1 X2 X3

X1 1000 0210 0999X2 0210 1000 0208X3 0999 0208 1000

Scientifica 15

[10] H Yang and X Chang ldquoA new two-parameter estimator inlinear regressionrdquo Communications in Statistics-0eory andMethods vol 39 no 6 pp 923ndash934 2010

[11] M Roozbeh ldquoOptimal QR-based estimation in partially linearregression models with correlated errors using GCV crite-rionrdquo Computational Statistics amp Data Analysis vol 117pp 45ndash61 2018

[12] F Akdeniz and M Roozbeh ldquoGeneralized difference-basedweightedmixed almost unbiased ridge estimator in partially linearmodelsrdquo Statistical Papers vol 60 no 5 pp 1717ndash1739 2019

[13] A F Lukman K Ayinde S Binuomote and O A ClementldquoModified ridge-type estimator to combat multicollinearityapplication to chemical datardquo Journal of Chemometricsvol 33 no 5 p e3125 2019

[14] A F Lukman K Ayinde S K Sek and E Adewuyi ldquoAmodified new two-parameter estimator in a linear regressionmodelrdquo Modelling and Simulation in Engineering vol 2019Article ID 6342702 10 pages 2019

[15] A E Hoerl R W Kannard and K F Baldwin ldquoRidge re-gressionsome simulationsrdquo Communications in Statisticsvol 4 no 2 pp 105ndash123 1975

[16] G C McDonald and D I Galarneau ldquoA monte carlo eval-uation of some ridge-type estimatorsrdquo Journal of the Amer-ican Statistical Association vol 70 no 350 pp 407ndash416 1975

[17] J F Lawless and P Wang ldquoA simulation study of ridge andother regression estimatorsrdquo Communications in Statistics-0eory and Methods vol 5 no 4 pp 307ndash323 1976

[18] D W Wichern and G A Churchill ldquoA comparison of ridgeestimatorsrdquo Technometrics vol 20 no 3 pp 301ndash311 1978

[19] B M G Kibria ldquoPerformance of some new ridge regressionestimatorsrdquo Communications in Statistics-Simulation andComputation vol 32 no 1 pp 419ndash435 2003

[20] A F Lukman and K Ayinde ldquoReview and classifications ofthe ridge parameter estimation techniquesrdquoHacettepe Journalof Mathematics and Statistics vol 46 no 5 pp 953ndash967 2017

[21] A K M E Saleh M Arashi and B M G Kibria 0eory ofRidge Regression Estimation with Applications WileyHoboken NJ USA 2019

[22] K Liu ldquoUsing Liu-type estimator to combat collinearityrdquoCommunications in Statistics-0eory and Methods vol 32no 5 pp 1009ndash1020 2003

[23] K Alheety and B M G Kibria ldquoOn the Liu and almostunbiased Liu estimators in the presence of multicollinearitywith heteroscedastic or correlated errorsrdquo Surveys in Math-ematics and its Applications vol 4 pp 155ndash167 2009

[24] X-Q Liu ldquoImproved Liu estimator in a linear regressionmodelrdquo Journal of Statistical Planning and Inference vol 141no 1 pp 189ndash196 2011

[25] Y Li and H Yang ldquoA new Liu-type estimator in linear regressionmodelrdquo Statistical Papers vol 53 no 2 pp 427ndash437 2012

[26] B Kan O Alpu and B Yazıcı ldquoRobust ridge and robust Liuestimator for regression based on the LTS estimatorrdquo Journalof Applied Statistics vol 40 no 3 pp 644ndash655 2013

[27] R A Farghali ldquoGeneralized Liu-type estimator for linearregressionrdquo International Journal of Research and Reviews inApplied Sciences vol 38 no 1 pp 52ndash63 2019

[28] S G Wang M X Wu and Z Z Jia Matrix InequalitiesChinese Science Press Beijing China 2nd edition 2006

[29] R W Farebrother ldquoFurther results on the mean square errorof ridge regressionrdquo Journal of the Royal Statistical SocietySeries B (Methodological) vol 38 no 3 pp 248ndash250 1976

[30] G Trenkler and H Toutenburg ldquoMean squared error matrixcomparisons between biased estimators-an overview of recentresultsrdquo Statistical Papers vol 31 no 1 pp 165ndash179 1990

[31] B M G Kibria and S Banik ldquoSome ridge regression esti-mators and their performancesrdquo Journal of Modern AppliedStatistical Methods vol 15 no 1 pp 206ndash238 2016

[32] D G Gibbons ldquoA simulation study of some ridge estimatorsrdquoJournal of the American Statistical Association vol 76no 373 pp 131ndash139 1981

[33] J P Newhouse and S D Oman ldquoAn evaluation of ridgeestimators A report prepared for United States air forceproject RANDrdquo 1971

[34] H Woods H H Steinour and H R Starke ldquoEffect ofcomposition of Portland cement on heat evolved duringhardeningrdquo Industrial amp Engineering Chemistry vol 24no 11 pp 1207ndash1214 1932

[35] S Chatterjee and A S Hadi Regression Analysis by ExampleWiley Hoboken NJ USA 1977

[36] S Kaciranlar S Sakallioglu F Akdeniz G P H Styan andH J Werner ldquoA new biased estimator in linear regression anda detailed analysis of the widely-analysed dataset on portlandcementrdquo Sankhya 0e Indian Journal of Statistics Series Bvol 61 pp 443ndash459 1999

[37] S Chatterjee and A S Haadi Regression Analysis by ExampleWiley Hoboken NJ USA 2006

[38] E Malinvard Statistical Methods of Econometrics North-Holland Publishing Company Amsterdam Netherlands 3rdedition 1980

[39] D N Gujarati Basic Econometrics McGraw-Hill New YorkNY USA 1995

[40] A F Lukman K Ayinde and A S Ajiboye ldquoMonte Carlostudy of some classification-based ridge parameter estima-torsrdquo Journal of Modern Applied Statistical Methods vol 16no 1 pp 428ndash451 2017

16 Scientifica

objective is to compare the performance of the proposedestimator with ridge regression and Liu estimators weconsider k d 01 02 1 We have restricted k between0 and 1 as Wichern and Churchill [18] have found that theridge regression estimator is better than the OLS when k isbetween 0 and 1 Kan et al [26] also suggested a smallervalue of k (less than 1) is better Simulation studies arerepeated 1000 times for the sample sizes n 30 and 100 andσ2 1 25 and 100 For each replicate we compute the meansquare error (MSE) of the estimators by using the followingequation

MSE αlowast( 1113857 1

10001113944

1000

i1αlowast minus α( 1113857prime αlowast minus α( 1113857 (45)

where αlowast would be any of the estimators (OLS ridge Liu or KL)Smaller MSE of the estimators will be considered the best one

+e simulated results for n 30 p 3 and ρ 070 080and ρ 090 099 are presented in Tables 1 and 2 respec-tively and for n 100 p 3 and ρ 07 080 and ρ 090099 are presented in Tables 3 and 4 respectively +ecorresponding simulated results for n 30 100 and p 7 arepresented in Tables 5ndash8 For a better visualization we haveplotted MSE vs d for n 30 σ 10 and ρ 070 090 and099 in Figures 1ndash3 respectively We also plotted MSE vs σfor n 30 d 50 and ρ 090 and 099 which is presentedin Figures 4 and 5 respectively Finally to see the effect ofsample size on MSE we plotted MSE vs sample size ford 05 and ρ 090 and presented in Figure 6

Table 3 Continued

n 100 09 099Sigma k d OLS Ridge Liu New est OLS Ridge Liu New est

05 78091 78178 74646 119870 120239 11095306 77415 78862 73336 118144 122001 10767407 76750 79549 72053 116462 123778 10450608 76096 80240 70797 114821 125570 10144709 75451 80934 69568 113220 127377 984901 74816 81632 68364 111658 129200 95634

Table 4 Estimated MSE when n 100 p 3 and ρ 090 and 099

n 30 09 099Sigma k d OLS Ridge Liu New est OLS Ridge Liu New est1 01 0287 0278 0230 0270 3072 2141 0688 1423

02 0270 0236 0255 1621 0836 076903 0263 0241 0242 1298 1013 052804 0256 0247 0231 1083 1219 047205 0250 0253 0221 0930 1455 050606 0244 0259 0213 0819 1720 058307 0239 0265 0206 0733 2014 068008 0234 0272 0200 0667 2337 078509 0230 0279 0195 0613 2690 08921 0226 0287 0191 0570 3072 0997

5 01 6958 6719 5256 6486 76772 53314 14746 3468902 6495 5431 6050 39905 18971 1666003 6283 5610 5649 31412 23862 883404 6083 5792 5278 25626 29420 580305 5893 5977 4935 21466 35645 517406 5714 6166 4617 18350 42537 579507 5544 6359 4324 15939 50096 707208 5383 6555 4052 14024 58321 868609 5230 6754 3799 12471 67213 104581 5085 6958 3566 11189 76772 12287

10 01 27809 26853 20970 25916 307086 213255 58717 13868502 25951 21675 24167 159582 75683 6635403 25100 22394 22551 125559 95308 3481504 24296 23126 21056 102365 117590 2246305 23535 23872 19672 85681 142529 1974306 22815 24632 18389 73175 170126 2204507 22131 25406 17200 63493 200380 2699508 21482 26193 16096 55802 233291 3330809 20865 26994 15071 49561 268860 402701 20279 27809 14120 44407 307086 47470

Scientifica 7

Table 5 Estimated MSE when n 30 p 7 and ρ 070 and 080

n 30 07 08Sigma k d OLS Ridge Liu New ridge OLS Ridge Liu New ridge1 01 0838 0811 0651 0785 1239 1179 0859 1121

02 0786 0670 0737 1124 0895 101803 0763 0689 0694 1074 0933 092804 0741 0709 0654 1029 0973 085005 0720 0729 0618 0987 1014 078106 0701 0750 0586 0949 1056 072107 0682 0771 0556 0914 1100 066908 0665 0793 0529 0881 1145 062309 0649 0815 0505 0851 1191 05831 0633 0838 0484 0823 1239 0549

5 01 20955 20275 16063 19608 30981 29455 21084 2797502 19633 16568 18362 28060 22071 2531403 19026 17083 17208 26780 23086 2295104 18452 17607 16139 25602 24130 2084505 17908 18141 15147 24513 25201 1896306 17391 18685 14226 23506 26301 1727907 16901 19238 13369 22570 27429 1576708 16435 19801 12572 21699 28585 1440809 15990 20373 11829 20885 29769 131851 15567 20955 11137 20125 30981 12081

10 01 83821 81095 64205 78423 123923 117811 84259 11188702 78523 66233 73429 112224 88219 10122503 76091 68299 68804 107097 92291 9174904 73789 70403 64513 102375 96475 8330105 71608 72545 60530 98014 100770 7575006 69537 74725 56827 93973 105177 6898307 67569 76942 53382 90220 109696 6290808 65698 79197 50173 86725 114327 5744109 63915 81490 47182 83463 119069 525151 62215 83821 44392 80411 123923 48069

Table 6 Estimated MSE when n 30 p 7 and ρ 09 and 099

N 30 09 099Sigma k d OLS Ridge Liu New ridge OLS Ridge Liu New ridge1 01 252 227 129 203 2868 1120 226 445

02 206 139 166 682 355 41603 188 151 137 478 525 57804 173 163 116 362 736 75805 161 176 099 288 989 92506 150 190 085 237 1283 107507 141 204 075 201 1617 120708 132 220 068 174 1993 132409 125 235 062 154 2410 14271 118 252 057 138 2868 1519

5 01 6303 5658 3123 5057 71709 27885 5083 1080002 5127 3411 4103 16823 8411 971703 4682 3715 3361 11638 12757 1345404 4303 4035 2778 8652 18123 1768905 3977 4372 2314 6736 24507 2164706 3694 4725 1942 5418 31910 2518707 3445 5095 1643 4467 40332 2831608 3225 5481 1401 3754 49772 3108009 3028 5884 1206 3206 60231 335291 2852 6303 1048 2773 71709 35710

10 01 25214 22630 12475 20223 286835 111506 20239 4314802 20503 13628 16403 67243 33562 38784

8 Scientifica

32 Simulation Results and Discussion From Tables 1ndash8and Figures 1ndash6 it appears that as the values of σ increasethe MSE values also increase (Figure 3) while the sample sizeincreases as the MSE values decrease (Figure 4) Ridge Liuand proposed KL estimators uniformly dominate the ordinaryleast squares (OLS) estimator In general from these tables anincrease in the levels of multicollinearity and the number ofexplanatory variables increase the estimated MSE values of theestimators +e figures consistently show that the OLS esti-mator performs worst when there is multicollinearity FromFigures 1ndash6 and simulation Tables 1ndash8 it clearly indicated thatfor ρ 090 or less the proposed estimator uniformly

dominates the ridge regression estimator while Liu performedmuch better than both proposed and ridge estimators for smalld say 03 or lessWhen ρ 099 the ridge regression performsthe best for higher k while the proposed estimator performsthe best for say k (say 03 or less) When d k 05 andρ 099 both ridge and KL estimators outperform the Liuestimator None of the estimators uniformly dominates eachother However it appears that our proposed estimator KLperforms better in the wider space of d k in the parameterspace If we review all Tables 1ndash8 we observed that theconclusions about the performance of all estimators remainthe same for both p 3 and p 7

Table 6 Continued

N 30 09 099Sigma k d OLS Ridge Liu New ridge OLS Ridge Liu New ridge

03 18721 14846 13432 46491 50959 5370704 17205 16130 11091 34538 72431 7062605 15899 17479 9229 26865 97978 8644306 14763 18895 7737 21588 127600 10058607 13766 20376 6534 17777 161296 11308908 12882 21923 5562 14924 199068 12413409 12095 23535 4775 12725 240914 1339211 11389 25214 4138 10992 286835 142634

Table 7 Estimated MSE when n 100 p 7 and ρ 070 and 080

n 100 07 08Sigma k d OLS Ridge Liu New ridge OLS Ridge Liu New ridge1 01 0174 0173 0163 0171 0263 0259 0235 0255

02 0171 0164 0169 0255 0238 024903 0170 0165 0166 0252 0241 024304 0169 0166 0164 0249 0244 023705 0167 0168 0161 0246 0247 023206 0166 0169 0159 0243 0250 022707 0165 0170 0157 0240 0253 022208 0164 0171 0155 0238 0256 021809 0163 0173 0154 0235 0259 02141 0162 0174 0152 0233 0263 0210

5 01 4356 4320 4055 4284 6563 6474 5852 638602 4285 4087 4214 6388 5928 621603 4250 4120 4146 6304 6005 605304 4216 4153 4079 6222 6082 589505 4182 4187 4013 6143 6160 574406 4149 4220 3949 6066 6239 559807 4116 4254 3887 5991 6319 545708 4084 4288 3826 5917 6399 532209 4053 4322 3767 5846 6481 51911 4022 4356 3708 5777 6563 5066

10 01 17425 17281 16219 17138 26250 25896 23408 2554502 17140 16350 16858 25551 23713 2486603 17001 16482 16584 25216 24020 2421204 16864 16614 16316 24891 24330 2358205 16729 16748 16054 24573 24643 2297506 16597 16882 15797 24265 24959 2238907 16467 17016 15547 23964 25277 2182508 16339 17152 15301 23671 25599 2128009 16213 17288 15062 23385 25923 207551 16089 17425 14827 23107 26250 20247

Scientifica 9

Table 8 Estimated MSE when n 100 p 7 and ρ 090 and 099

n 100 09 099Sigma k d OLS Ridge Liu New ridge OLS Ridge Liu New ridge1 01 0546 0529 0431 0512 6389 4391 1624 2949

02 0513 0442 0482 3407 1934 183603 0498 0454 0456 2819 2298 145304 0485 0466 0432 2423 2718 134705 0472 0478 0411 2135 3192 135906 0460 0491 0392 1914 3721 142607 0449 0504 0375 1738 4306 151908 0439 0517 0360 1593 4945 162509 0429 0531 0346 1472 5640 17371 0420 0546 0334 1370 6389 1851

5 01 13640 13216 10676 12802 159732 109722 38895 7328402 12820 10979 12037 84915 47018 4450603 12448 11289 11336 69971 56467 3386504 12099 11605 10693 59823 67242 3014605 11770 11928 10102 52370 79343 2941706 11460 12257 9558 46597 92769 3009007 11168 12593 9056 41953 107521 3145508 10891 12935 8593 38114 123599 3317109 10628 13284 8165 34875 141003 350631 10379 13640 7768 32097 159732 37036

10 01 54558 52866 42699 51212 638928 438910 155399 29312102 51282 43914 48150 339663 187945 17787403 49796 45155 45344 279860 225785 13515104 48399 46422 42768 239236 268921 12012005 47084 47714 40397 209391 317351 11705306 45843 49032 38214 186265 371077 11959907 44670 50375 36198 167659 430097 12492208 43560 51744 34336 152274 494412 13165409 42508 53138 32612 139287 564022 1390941 41509 54558 31014 128149 638928 146866

OLSRidge

LiuKL

025

030

035

040

MSE

02 04 06 08 1000d = k

(a)

OLSRidge

LiuKL

02 04 06 08 1000d = k

20

25

30

35

MSE

(b)

Figure 1 Continued

10 Scientifica

4 Numerical Examples

To illustrate our theoretical results we consider two datasets(i) famous Portland cement data originally adopted byWoods et al [34] and (ii) French economy data from

Chatterjee and Hadi [35] and they are analyzed in thefollowing sections respectively

41 Example 1 Portland Data +ese data are widely knownas the Portland cement dataset It was originally adopted by

OLSRidge

LiuKL

025

035

045

055

MSE

02 04 06 08 1000d = k

(c)

OLSRidge

LiuKL

02 04 06 08 1000d = k

20

30

40

50

MSE

(d)

Figure 1 Estimated MSEs for n 30 Sigma 1 10 rho 070 080 and different values of k d (a) n 30 p 3 sigma 1 and rho 070(b) n 30 p 3 sigma 10 and rho 070 (c) n 30 p 3 sigma 1 and rho 080 (d) n 30 p 3 sigma 10 and rho 080

OLSRidge

LiuKL

04

06

08

10

12

MSE

02 04 06 08 1000d = k

(a)

OLSRidge

LiuKL

20

40

60

80

100M

SE

02 04 06 08 1000d = k

(b)

OLSRidge

LiuKL

02468

101214

MSE

02 04 06 08 1000d = k

(c)

OLSRidge

LiuKL

0200

600

1000

1400

MSE

02 04 06 08 1000d = k

(d)

Figure 2 Estimated MSEs for n 30 sigma 1 10 rho 090 099 and different values of k d (a) n 30 p 3 sigma 1 and rho 090(b) n 30 p 3 sigma 10 and rho 090 (c) n 30 p 3 sigma 1 and rho 099 (d) n 30 p 3 sigma 10 and rho 099

Scientifica 11

OLSRidge

LiuKL

0

10

20

30

MSE

2 4 6 8 100Sigma

(a)

OLSRidge

LiuKL

0

10

30

50

MSE

2 4 6 8 100Sigma

(b)

OLSRidge

LiuKL

0

40

80

120

MSE

2 4 6 8 100Sigma

(c)

OLSRidge

LiuKL

2 4 6 8 100Sigma

0

400

800

1200

MSE

(d)

Figure 3 EstimatedMSEs for n 30 d 05 and different values of rho and sigma (a) n 30 p 3 d 05 and rho 070 (b) n 30 p 3d 05 and rho 080 (c) n 30 p 3 d 05 and rho 090 (d) n 30 p 3 d 05 and rho 099

OLSRidge

LiuKL

50 70 9030n

010

020

030

040

MSE

(a)

OLSRidge

LiuKL

50 70 9030n

02

04

06

MSE

(b)

Figure 4 Continued

12 Scientifica

Woods et al [34] It has also been analyzed by the followingauthors Kaciranlar et al [36] Li and Yang [25] and recentlyby Lukman et al [13] +e regression model for these data isdefined as

yi β0 + β1X1 + β2X2 + β3X3 + β4X4 + εi (46)

where yi heat evolved after 180 days of curing measured incalories per gram of cement X1 tricalcium aluminateX2 tricalcium silicate X3 tetracalcium aluminoferriteand X4 β-dicalcium silicate +e correlation matrix of thepredictor variables is given in Table 9

OLSRidge

LiuKL

50 70 9030n

02

06

10

MSE

(c)

OLSRidge

LiuKL

50 70 9030n

0

2

4

6

8

12

MSE

(d)

Figure 4 Estimated MSEs for sigma 1 p 3 and different values of rho and sample size (a)p 3 sigma 1 d 05 and rho 070(b)p 3 sigma 1 d 05 and rho 080 (c)p 3 sigma 1 d 05 and rho 090 (d)p 3 sigma 1 d 05 and rho 099

OLSRidge

LiuKL

20

30

40

MSE

4 5 6 7 83p

(a)

OLSRidge

LiuKL

3

4

5

6

7M

SE

4 5 6 7 83p

(b)

6

8

10

14

MSE

OLSRidge

LiuKL

4 5 6 7 83p

(c)

OLSRidge

LiuKL

4 5 6 7 83p

0

50

100

150

MSE

(d)

Figure 5 Estimated MSEs for n 100 d 05 sigma 5 and different values of rho and p (a) n 100 sigma 5 d 05 and rho 070 (b)n 100 sigma 5 d 05 and rho 080 (c) n 100 sigma 5 d 05 and rho 090 (d) n 100 sigma 5 d 05 and rho 099

Scientifica 13

+e variance inflation factors are VIF1 = 3850VIF2 = 25442 VIF3 = 4687 and VIF4 = 28251 Eigen-values of XprimeX are λ1 44676206 λ2 5965422

λ3 809952 and λ4 105419 and the condition numberof XprimeX is approximately 424 +e VIFs the eigenvalues

and the condition number all indicate the presence ofsevere multicollinearity +e estimated parameters andMSE are presented in Table 10 It appears from Table 11that the proposed estimator performed the best in thesense of smaller MSE

OLSRidge

LiuKL

0

100

200

300

MSE

075 085 095065Rho

(a)

OLSRidge

LiuKL

0

200

400

600

800

MSE

075 085 095065Rho

(b)

OLSRidge

LiuKL

0

20

40

60

80

MSE

075 085 095065Rho

(c)

OLSRidge

LiuKL

0

50

100

150

MSE

075 085 095065Rho

(d)

Figure 6 Estimated MSEs for n 100 p 3 7 d 05 sigma 5 and different values of rho (a) n 30 p 3 sigma 5 and d 05 (b)n 30 p 7 sigma 5 and d 05 (c) n 100 p 3 sigma 5 and d 05 (d) n 100 p 7 sigma 5 and d 05

Table 9 Correlation matrix

X1 X2 X3 X4

X1 1000 0229 minus 0824 minus 0245X2 0229 1000 minus 0139 minus 0973X3 minus 0824 minus 0139 1000 0030X4 minus 0245 minus 0973 0030 1000

Table 10 +e results of regression coefficients and the corresponding MSE values

Coef 1113954α 1113954α(k) 1113954α(d)1113954dalt 1113954αKL(kHNM) 1113954αKL(kmin)

α0 624054 85870 276490 minus 197876 276068α1 15511 21046 19010 23965 19090α2 05102 10648 08701 13573 08688α3 01019 06681 04621 09666 04680α4 minus 01441 03996 02082 06862 02074MSE 491209 298983 2170963 7255603 217096kd mdash 00077 044195 000235 000047

14 Scientifica

42 Example 2 French Economy Data +e French economydata in Chatterjee and Hadi [37] are considered in this ex-ample It has been analyzed by Malinvard [38] and Liu [6]among others+e variables are imports domestic productionstock formation and domestic consumption All are measuredin milliards of French francs for the years 1949 through 1966

+e regression model for these data is defined as

yi β0 + β1X1 + β2X2 + β3X3 + εi (47)

where yi IMPORT X1 domestic production X2 stockformation and X3 domestic consumption +e correlationmatrix of the predicted variable is given in Table 12

+e variance inflation factors areVIF1 469688VIF2 1047 and VIF3 469338 +e ei-genvalues of the XprimeX matrix are λ1 161779 λ2 158 andλ3 4961 and the condition number is 32612 If we reviewthe above correlation matrix VIFs and condition number itcan be said that there is presence of severe multicollinearityexisting in the predictor variables

+e biasing parameter for the new estimator is defined in(41) and (42) +e biasing parameter for the ridge and Liuestimator is provided in (6) (8) and (9) respectively

We analyzed the data using the biasing parameters foreach of the estimators and presented the results in Tables 10and 11 It can be seen from Tables 10 and 11 that theproposed estimator performed the best in the sense ofsmaller MSE

5 Summary and Concluding Remarks

In this paper we introduced a new biased estimator toovercome the multicollinearity problem for the multiplelinear regression model and provided the estimation tech-nique of the biasing parameter A simulation study has beenconducted to compare the performance of the proposedestimator and Liu [6] and ridge regression estimators [3]Simulation results evidently show that the proposed esti-mator performed better than both Liu and ridge under somecondition on the shrinkage parameter Two sets of real-lifedata are analyzed to illustrate the benefits of using the newestimator in the context of a linear regression model +eproposed estimator is recommended for researchers in this

area Its application can be extended to other regressionmodels for example logistic regression Poisson ZIP andrelated models and those possibilities are under currentinvestigation [37 39 40]

Data Availability

Data will be made available on request

Conflicts of Interest

+e authors declare that they have no conflicts of interest

Acknowledgments

We are dedicating this article to those who lost their livesbecause of COVID-19

References

[1] C Stein ldquoInadmissibility of the usual estimator for mean ofmultivariate normal distributionrdquo in Proceedings of the 0irdBerkley Symposium on Mathematical and Statistics Proba-bility J Neyman Ed vol 1 pp 197ndash206 Springer BerlinGermany 1956

[2] W F Massy ldquoPrincipal components regression in exploratorystatistical researchrdquo Journal of the American Statistical As-sociation vol 60 no 309 pp 234ndash256 1965

[3] A E Hoerl and R W Kennard ldquoRidge regression biasedestimation for nonorthogonal problemsrdquo Technometricsvol 12 no 1 pp 55ndash67 1970

[4] L S Mayer and T A Willke ldquoOn biased estimation in linearmodelsrdquo Technometrics vol 15 no 3 pp 497ndash508 1973

[5] B F Swindel ldquoGood ridge estimators based on prior infor-mationrdquo Communications in Statistics-0eory and Methodsvol 5 no 11 pp 1065ndash1075 1976

[6] K Liu ldquoA new class of biased estimate in linear regressionrdquoCommunication in Statistics- 0eory and Methods vol 22pp 393ndash402 1993

[7] F Akdeniz and S Kaccediliranlar ldquoOn the almost unbiasedgeneralized liu estimator and unbiased estimation of the biasand mserdquo Communications in Statistics-0eory and Methodsvol 24 no 7 pp 1789ndash1797 1995

[8] M R Ozkale and S Kaccediliranlar ldquo+e restricted and unre-stricted two-parameter estimatorsrdquo Communications in Sta-tistics-0eory and Methods vol 36 no 15 pp 2707ndash27252007

[9] S Sakallıoglu and S Kaccedilıranlar ldquoA new biased estimatorbased on ridge estimationrdquo Statistical Papers vol 49 no 4pp 669ndash689 2008

Table 11 +e results of regression coefficients and the corresponding MSE values

Coef 1113954α 1113954α(k) 1113954α(d)1113954dalt 1113954α(d)1113954dopt 1113954αKL(kHNM) 1113954αKL(kmin)

α0 minus 197127 minus 167613 minus 125762 minus 188410 minus 165855 minus 188782α1 00327 01419 02951 00648 01485 00636α2 04059 03576 02875 03914 03548 03922α3 02421 00709 minus 01696 01918 00606 01937MSE 173326 2130519 5828312 1660293 2211899 1660168kd mdash 00527 05282 09423 00258 00065

Table 12 Correlation matrix

X1 X2 X3

X1 1000 0210 0999X2 0210 1000 0208X3 0999 0208 1000

Scientifica 15

[10] H Yang and X Chang ldquoA new two-parameter estimator inlinear regressionrdquo Communications in Statistics-0eory andMethods vol 39 no 6 pp 923ndash934 2010

[11] M Roozbeh ldquoOptimal QR-based estimation in partially linearregression models with correlated errors using GCV crite-rionrdquo Computational Statistics amp Data Analysis vol 117pp 45ndash61 2018

[12] F Akdeniz and M Roozbeh ldquoGeneralized difference-basedweightedmixed almost unbiased ridge estimator in partially linearmodelsrdquo Statistical Papers vol 60 no 5 pp 1717ndash1739 2019

[13] A F Lukman K Ayinde S Binuomote and O A ClementldquoModified ridge-type estimator to combat multicollinearityapplication to chemical datardquo Journal of Chemometricsvol 33 no 5 p e3125 2019

[14] A F Lukman K Ayinde S K Sek and E Adewuyi ldquoAmodified new two-parameter estimator in a linear regressionmodelrdquo Modelling and Simulation in Engineering vol 2019Article ID 6342702 10 pages 2019

[15] A E Hoerl R W Kannard and K F Baldwin ldquoRidge re-gressionsome simulationsrdquo Communications in Statisticsvol 4 no 2 pp 105ndash123 1975

[16] G C McDonald and D I Galarneau ldquoA monte carlo eval-uation of some ridge-type estimatorsrdquo Journal of the Amer-ican Statistical Association vol 70 no 350 pp 407ndash416 1975

[17] J F Lawless and P Wang ldquoA simulation study of ridge andother regression estimatorsrdquo Communications in Statistics-0eory and Methods vol 5 no 4 pp 307ndash323 1976

[18] D W Wichern and G A Churchill ldquoA comparison of ridgeestimatorsrdquo Technometrics vol 20 no 3 pp 301ndash311 1978

[19] B M G Kibria ldquoPerformance of some new ridge regressionestimatorsrdquo Communications in Statistics-Simulation andComputation vol 32 no 1 pp 419ndash435 2003

[20] A F Lukman and K Ayinde ldquoReview and classifications ofthe ridge parameter estimation techniquesrdquoHacettepe Journalof Mathematics and Statistics vol 46 no 5 pp 953ndash967 2017

[21] A K M E Saleh M Arashi and B M G Kibria 0eory ofRidge Regression Estimation with Applications WileyHoboken NJ USA 2019

[22] K Liu ldquoUsing Liu-type estimator to combat collinearityrdquoCommunications in Statistics-0eory and Methods vol 32no 5 pp 1009ndash1020 2003

[23] K Alheety and B M G Kibria ldquoOn the Liu and almostunbiased Liu estimators in the presence of multicollinearitywith heteroscedastic or correlated errorsrdquo Surveys in Math-ematics and its Applications vol 4 pp 155ndash167 2009

[24] X-Q Liu ldquoImproved Liu estimator in a linear regressionmodelrdquo Journal of Statistical Planning and Inference vol 141no 1 pp 189ndash196 2011

[25] Y Li and H Yang ldquoA new Liu-type estimator in linear regressionmodelrdquo Statistical Papers vol 53 no 2 pp 427ndash437 2012

[26] B Kan O Alpu and B Yazıcı ldquoRobust ridge and robust Liuestimator for regression based on the LTS estimatorrdquo Journalof Applied Statistics vol 40 no 3 pp 644ndash655 2013

[27] R A Farghali ldquoGeneralized Liu-type estimator for linearregressionrdquo International Journal of Research and Reviews inApplied Sciences vol 38 no 1 pp 52ndash63 2019

[28] S G Wang M X Wu and Z Z Jia Matrix InequalitiesChinese Science Press Beijing China 2nd edition 2006

[29] R W Farebrother ldquoFurther results on the mean square errorof ridge regressionrdquo Journal of the Royal Statistical SocietySeries B (Methodological) vol 38 no 3 pp 248ndash250 1976

[30] G Trenkler and H Toutenburg ldquoMean squared error matrixcomparisons between biased estimators-an overview of recentresultsrdquo Statistical Papers vol 31 no 1 pp 165ndash179 1990

[31] B M G Kibria and S Banik ldquoSome ridge regression esti-mators and their performancesrdquo Journal of Modern AppliedStatistical Methods vol 15 no 1 pp 206ndash238 2016

[32] D G Gibbons ldquoA simulation study of some ridge estimatorsrdquoJournal of the American Statistical Association vol 76no 373 pp 131ndash139 1981

[33] J P Newhouse and S D Oman ldquoAn evaluation of ridgeestimators A report prepared for United States air forceproject RANDrdquo 1971

[34] H Woods H H Steinour and H R Starke ldquoEffect ofcomposition of Portland cement on heat evolved duringhardeningrdquo Industrial amp Engineering Chemistry vol 24no 11 pp 1207ndash1214 1932

[35] S Chatterjee and A S Hadi Regression Analysis by ExampleWiley Hoboken NJ USA 1977

[36] S Kaciranlar S Sakallioglu F Akdeniz G P H Styan andH J Werner ldquoA new biased estimator in linear regression anda detailed analysis of the widely-analysed dataset on portlandcementrdquo Sankhya 0e Indian Journal of Statistics Series Bvol 61 pp 443ndash459 1999

[37] S Chatterjee and A S Haadi Regression Analysis by ExampleWiley Hoboken NJ USA 2006

[38] E Malinvard Statistical Methods of Econometrics North-Holland Publishing Company Amsterdam Netherlands 3rdedition 1980

[39] D N Gujarati Basic Econometrics McGraw-Hill New YorkNY USA 1995

[40] A F Lukman K Ayinde and A S Ajiboye ldquoMonte Carlostudy of some classification-based ridge parameter estima-torsrdquo Journal of Modern Applied Statistical Methods vol 16no 1 pp 428ndash451 2017

16 Scientifica

Table 5 Estimated MSE when n 30 p 7 and ρ 070 and 080

n 30 07 08Sigma k d OLS Ridge Liu New ridge OLS Ridge Liu New ridge1 01 0838 0811 0651 0785 1239 1179 0859 1121

02 0786 0670 0737 1124 0895 101803 0763 0689 0694 1074 0933 092804 0741 0709 0654 1029 0973 085005 0720 0729 0618 0987 1014 078106 0701 0750 0586 0949 1056 072107 0682 0771 0556 0914 1100 066908 0665 0793 0529 0881 1145 062309 0649 0815 0505 0851 1191 05831 0633 0838 0484 0823 1239 0549

5 01 20955 20275 16063 19608 30981 29455 21084 2797502 19633 16568 18362 28060 22071 2531403 19026 17083 17208 26780 23086 2295104 18452 17607 16139 25602 24130 2084505 17908 18141 15147 24513 25201 1896306 17391 18685 14226 23506 26301 1727907 16901 19238 13369 22570 27429 1576708 16435 19801 12572 21699 28585 1440809 15990 20373 11829 20885 29769 131851 15567 20955 11137 20125 30981 12081

10 01 83821 81095 64205 78423 123923 117811 84259 11188702 78523 66233 73429 112224 88219 10122503 76091 68299 68804 107097 92291 9174904 73789 70403 64513 102375 96475 8330105 71608 72545 60530 98014 100770 7575006 69537 74725 56827 93973 105177 6898307 67569 76942 53382 90220 109696 6290808 65698 79197 50173 86725 114327 5744109 63915 81490 47182 83463 119069 525151 62215 83821 44392 80411 123923 48069

Table 6 Estimated MSE when n 30 p 7 and ρ 09 and 099

N 30 09 099Sigma k d OLS Ridge Liu New ridge OLS Ridge Liu New ridge1 01 252 227 129 203 2868 1120 226 445

02 206 139 166 682 355 41603 188 151 137 478 525 57804 173 163 116 362 736 75805 161 176 099 288 989 92506 150 190 085 237 1283 107507 141 204 075 201 1617 120708 132 220 068 174 1993 132409 125 235 062 154 2410 14271 118 252 057 138 2868 1519

5 01 6303 5658 3123 5057 71709 27885 5083 1080002 5127 3411 4103 16823 8411 971703 4682 3715 3361 11638 12757 1345404 4303 4035 2778 8652 18123 1768905 3977 4372 2314 6736 24507 2164706 3694 4725 1942 5418 31910 2518707 3445 5095 1643 4467 40332 2831608 3225 5481 1401 3754 49772 3108009 3028 5884 1206 3206 60231 335291 2852 6303 1048 2773 71709 35710

10 01 25214 22630 12475 20223 286835 111506 20239 4314802 20503 13628 16403 67243 33562 38784

8 Scientifica

32 Simulation Results and Discussion From Tables 1ndash8and Figures 1ndash6 it appears that as the values of σ increasethe MSE values also increase (Figure 3) while the sample sizeincreases as the MSE values decrease (Figure 4) Ridge Liuand proposed KL estimators uniformly dominate the ordinaryleast squares (OLS) estimator In general from these tables anincrease in the levels of multicollinearity and the number ofexplanatory variables increase the estimated MSE values of theestimators +e figures consistently show that the OLS esti-mator performs worst when there is multicollinearity FromFigures 1ndash6 and simulation Tables 1ndash8 it clearly indicated thatfor ρ 090 or less the proposed estimator uniformly

dominates the ridge regression estimator while Liu performedmuch better than both proposed and ridge estimators for smalld say 03 or lessWhen ρ 099 the ridge regression performsthe best for higher k while the proposed estimator performsthe best for say k (say 03 or less) When d k 05 andρ 099 both ridge and KL estimators outperform the Liuestimator None of the estimators uniformly dominates eachother However it appears that our proposed estimator KLperforms better in the wider space of d k in the parameterspace If we review all Tables 1ndash8 we observed that theconclusions about the performance of all estimators remainthe same for both p 3 and p 7

Table 6 Continued

N 30 09 099Sigma k d OLS Ridge Liu New ridge OLS Ridge Liu New ridge

03 18721 14846 13432 46491 50959 5370704 17205 16130 11091 34538 72431 7062605 15899 17479 9229 26865 97978 8644306 14763 18895 7737 21588 127600 10058607 13766 20376 6534 17777 161296 11308908 12882 21923 5562 14924 199068 12413409 12095 23535 4775 12725 240914 1339211 11389 25214 4138 10992 286835 142634

Table 7 Estimated MSE when n 100 p 7 and ρ 070 and 080

n 100 07 08Sigma k d OLS Ridge Liu New ridge OLS Ridge Liu New ridge1 01 0174 0173 0163 0171 0263 0259 0235 0255

02 0171 0164 0169 0255 0238 024903 0170 0165 0166 0252 0241 024304 0169 0166 0164 0249 0244 023705 0167 0168 0161 0246 0247 023206 0166 0169 0159 0243 0250 022707 0165 0170 0157 0240 0253 022208 0164 0171 0155 0238 0256 021809 0163 0173 0154 0235 0259 02141 0162 0174 0152 0233 0263 0210

5 01 4356 4320 4055 4284 6563 6474 5852 638602 4285 4087 4214 6388 5928 621603 4250 4120 4146 6304 6005 605304 4216 4153 4079 6222 6082 589505 4182 4187 4013 6143 6160 574406 4149 4220 3949 6066 6239 559807 4116 4254 3887 5991 6319 545708 4084 4288 3826 5917 6399 532209 4053 4322 3767 5846 6481 51911 4022 4356 3708 5777 6563 5066

10 01 17425 17281 16219 17138 26250 25896 23408 2554502 17140 16350 16858 25551 23713 2486603 17001 16482 16584 25216 24020 2421204 16864 16614 16316 24891 24330 2358205 16729 16748 16054 24573 24643 2297506 16597 16882 15797 24265 24959 2238907 16467 17016 15547 23964 25277 2182508 16339 17152 15301 23671 25599 2128009 16213 17288 15062 23385 25923 207551 16089 17425 14827 23107 26250 20247

Scientifica 9

Table 8 Estimated MSE when n 100 p 7 and ρ 090 and 099

n 100 09 099Sigma k d OLS Ridge Liu New ridge OLS Ridge Liu New ridge1 01 0546 0529 0431 0512 6389 4391 1624 2949

02 0513 0442 0482 3407 1934 183603 0498 0454 0456 2819 2298 145304 0485 0466 0432 2423 2718 134705 0472 0478 0411 2135 3192 135906 0460 0491 0392 1914 3721 142607 0449 0504 0375 1738 4306 151908 0439 0517 0360 1593 4945 162509 0429 0531 0346 1472 5640 17371 0420 0546 0334 1370 6389 1851

5 01 13640 13216 10676 12802 159732 109722 38895 7328402 12820 10979 12037 84915 47018 4450603 12448 11289 11336 69971 56467 3386504 12099 11605 10693 59823 67242 3014605 11770 11928 10102 52370 79343 2941706 11460 12257 9558 46597 92769 3009007 11168 12593 9056 41953 107521 3145508 10891 12935 8593 38114 123599 3317109 10628 13284 8165 34875 141003 350631 10379 13640 7768 32097 159732 37036

10 01 54558 52866 42699 51212 638928 438910 155399 29312102 51282 43914 48150 339663 187945 17787403 49796 45155 45344 279860 225785 13515104 48399 46422 42768 239236 268921 12012005 47084 47714 40397 209391 317351 11705306 45843 49032 38214 186265 371077 11959907 44670 50375 36198 167659 430097 12492208 43560 51744 34336 152274 494412 13165409 42508 53138 32612 139287 564022 1390941 41509 54558 31014 128149 638928 146866

OLSRidge

LiuKL

025

030

035

040

MSE

02 04 06 08 1000d = k

(a)

OLSRidge

LiuKL

02 04 06 08 1000d = k

20

25

30

35

MSE

(b)

Figure 1 Continued

10 Scientifica

4 Numerical Examples

To illustrate our theoretical results we consider two datasets(i) famous Portland cement data originally adopted byWoods et al [34] and (ii) French economy data from

Chatterjee and Hadi [35] and they are analyzed in thefollowing sections respectively

41 Example 1 Portland Data +ese data are widely knownas the Portland cement dataset It was originally adopted by

OLSRidge

LiuKL

025

035

045

055

MSE

02 04 06 08 1000d = k

(c)

OLSRidge

LiuKL

02 04 06 08 1000d = k

20

30

40

50

MSE

(d)

Figure 1 Estimated MSEs for n 30 Sigma 1 10 rho 070 080 and different values of k d (a) n 30 p 3 sigma 1 and rho 070(b) n 30 p 3 sigma 10 and rho 070 (c) n 30 p 3 sigma 1 and rho 080 (d) n 30 p 3 sigma 10 and rho 080

OLSRidge

LiuKL

04

06

08

10

12

MSE

02 04 06 08 1000d = k

(a)

OLSRidge

LiuKL

20

40

60

80

100M

SE

02 04 06 08 1000d = k

(b)

OLSRidge

LiuKL

02468

101214

MSE

02 04 06 08 1000d = k

(c)

OLSRidge

LiuKL

0200

600

1000

1400

MSE

02 04 06 08 1000d = k

(d)

Figure 2 Estimated MSEs for n 30 sigma 1 10 rho 090 099 and different values of k d (a) n 30 p 3 sigma 1 and rho 090(b) n 30 p 3 sigma 10 and rho 090 (c) n 30 p 3 sigma 1 and rho 099 (d) n 30 p 3 sigma 10 and rho 099

Scientifica 11

OLSRidge

LiuKL

0

10

20

30

MSE

2 4 6 8 100Sigma

(a)

OLSRidge

LiuKL

0

10

30

50

MSE

2 4 6 8 100Sigma

(b)

OLSRidge

LiuKL

0

40

80

120

MSE

2 4 6 8 100Sigma

(c)

OLSRidge

LiuKL

2 4 6 8 100Sigma

0

400

800

1200

MSE

(d)

Figure 3 EstimatedMSEs for n 30 d 05 and different values of rho and sigma (a) n 30 p 3 d 05 and rho 070 (b) n 30 p 3d 05 and rho 080 (c) n 30 p 3 d 05 and rho 090 (d) n 30 p 3 d 05 and rho 099

OLSRidge

LiuKL

50 70 9030n

010

020

030

040

MSE

(a)

OLSRidge

LiuKL

50 70 9030n

02

04

06

MSE

(b)

Figure 4 Continued

12 Scientifica

Woods et al [34] It has also been analyzed by the followingauthors Kaciranlar et al [36] Li and Yang [25] and recentlyby Lukman et al [13] +e regression model for these data isdefined as

yi β0 + β1X1 + β2X2 + β3X3 + β4X4 + εi (46)

where yi heat evolved after 180 days of curing measured incalories per gram of cement X1 tricalcium aluminateX2 tricalcium silicate X3 tetracalcium aluminoferriteand X4 β-dicalcium silicate +e correlation matrix of thepredictor variables is given in Table 9

OLSRidge

LiuKL

50 70 9030n

02

06

10

MSE

(c)

OLSRidge

LiuKL

50 70 9030n

0

2

4

6

8

12

MSE

(d)

Figure 4 Estimated MSEs for sigma 1 p 3 and different values of rho and sample size (a)p 3 sigma 1 d 05 and rho 070(b)p 3 sigma 1 d 05 and rho 080 (c)p 3 sigma 1 d 05 and rho 090 (d)p 3 sigma 1 d 05 and rho 099

OLSRidge

LiuKL

20

30

40

MSE

4 5 6 7 83p

(a)

OLSRidge

LiuKL

3

4

5

6

7M

SE

4 5 6 7 83p

(b)

6

8

10

14

MSE

OLSRidge

LiuKL

4 5 6 7 83p

(c)

OLSRidge

LiuKL

4 5 6 7 83p

0

50

100

150

MSE

(d)

Figure 5 Estimated MSEs for n 100 d 05 sigma 5 and different values of rho and p (a) n 100 sigma 5 d 05 and rho 070 (b)n 100 sigma 5 d 05 and rho 080 (c) n 100 sigma 5 d 05 and rho 090 (d) n 100 sigma 5 d 05 and rho 099

Scientifica 13

+e variance inflation factors are VIF1 = 3850VIF2 = 25442 VIF3 = 4687 and VIF4 = 28251 Eigen-values of XprimeX are λ1 44676206 λ2 5965422

λ3 809952 and λ4 105419 and the condition numberof XprimeX is approximately 424 +e VIFs the eigenvalues

and the condition number all indicate the presence ofsevere multicollinearity +e estimated parameters andMSE are presented in Table 10 It appears from Table 11that the proposed estimator performed the best in thesense of smaller MSE

OLSRidge

LiuKL

0

100

200

300

MSE

075 085 095065Rho

(a)

OLSRidge

LiuKL

0

200

400

600

800

MSE

075 085 095065Rho

(b)

OLSRidge

LiuKL

0

20

40

60

80

MSE

075 085 095065Rho

(c)

OLSRidge

LiuKL

0

50

100

150

MSE

075 085 095065Rho

(d)

Figure 6 Estimated MSEs for n 100 p 3 7 d 05 sigma 5 and different values of rho (a) n 30 p 3 sigma 5 and d 05 (b)n 30 p 7 sigma 5 and d 05 (c) n 100 p 3 sigma 5 and d 05 (d) n 100 p 7 sigma 5 and d 05

Table 9 Correlation matrix

X1 X2 X3 X4

X1 1000 0229 minus 0824 minus 0245X2 0229 1000 minus 0139 minus 0973X3 minus 0824 minus 0139 1000 0030X4 minus 0245 minus 0973 0030 1000

Table 10 +e results of regression coefficients and the corresponding MSE values

Coef 1113954α 1113954α(k) 1113954α(d)1113954dalt 1113954αKL(kHNM) 1113954αKL(kmin)

α0 624054 85870 276490 minus 197876 276068α1 15511 21046 19010 23965 19090α2 05102 10648 08701 13573 08688α3 01019 06681 04621 09666 04680α4 minus 01441 03996 02082 06862 02074MSE 491209 298983 2170963 7255603 217096kd mdash 00077 044195 000235 000047

14 Scientifica

42 Example 2 French Economy Data +e French economydata in Chatterjee and Hadi [37] are considered in this ex-ample It has been analyzed by Malinvard [38] and Liu [6]among others+e variables are imports domestic productionstock formation and domestic consumption All are measuredin milliards of French francs for the years 1949 through 1966

+e regression model for these data is defined as

yi β0 + β1X1 + β2X2 + β3X3 + εi (47)

where yi IMPORT X1 domestic production X2 stockformation and X3 domestic consumption +e correlationmatrix of the predicted variable is given in Table 12

+e variance inflation factors areVIF1 469688VIF2 1047 and VIF3 469338 +e ei-genvalues of the XprimeX matrix are λ1 161779 λ2 158 andλ3 4961 and the condition number is 32612 If we reviewthe above correlation matrix VIFs and condition number itcan be said that there is presence of severe multicollinearityexisting in the predictor variables

+e biasing parameter for the new estimator is defined in(41) and (42) +e biasing parameter for the ridge and Liuestimator is provided in (6) (8) and (9) respectively

We analyzed the data using the biasing parameters foreach of the estimators and presented the results in Tables 10and 11 It can be seen from Tables 10 and 11 that theproposed estimator performed the best in the sense ofsmaller MSE

5 Summary and Concluding Remarks

In this paper we introduced a new biased estimator toovercome the multicollinearity problem for the multiplelinear regression model and provided the estimation tech-nique of the biasing parameter A simulation study has beenconducted to compare the performance of the proposedestimator and Liu [6] and ridge regression estimators [3]Simulation results evidently show that the proposed esti-mator performed better than both Liu and ridge under somecondition on the shrinkage parameter Two sets of real-lifedata are analyzed to illustrate the benefits of using the newestimator in the context of a linear regression model +eproposed estimator is recommended for researchers in this

area Its application can be extended to other regressionmodels for example logistic regression Poisson ZIP andrelated models and those possibilities are under currentinvestigation [37 39 40]

Data Availability

Data will be made available on request

Conflicts of Interest

+e authors declare that they have no conflicts of interest

Acknowledgments

We are dedicating this article to those who lost their livesbecause of COVID-19

References

[1] C Stein ldquoInadmissibility of the usual estimator for mean ofmultivariate normal distributionrdquo in Proceedings of the 0irdBerkley Symposium on Mathematical and Statistics Proba-bility J Neyman Ed vol 1 pp 197ndash206 Springer BerlinGermany 1956

[2] W F Massy ldquoPrincipal components regression in exploratorystatistical researchrdquo Journal of the American Statistical As-sociation vol 60 no 309 pp 234ndash256 1965

[3] A E Hoerl and R W Kennard ldquoRidge regression biasedestimation for nonorthogonal problemsrdquo Technometricsvol 12 no 1 pp 55ndash67 1970

[4] L S Mayer and T A Willke ldquoOn biased estimation in linearmodelsrdquo Technometrics vol 15 no 3 pp 497ndash508 1973

[5] B F Swindel ldquoGood ridge estimators based on prior infor-mationrdquo Communications in Statistics-0eory and Methodsvol 5 no 11 pp 1065ndash1075 1976

[6] K Liu ldquoA new class of biased estimate in linear regressionrdquoCommunication in Statistics- 0eory and Methods vol 22pp 393ndash402 1993

[7] F Akdeniz and S Kaccediliranlar ldquoOn the almost unbiasedgeneralized liu estimator and unbiased estimation of the biasand mserdquo Communications in Statistics-0eory and Methodsvol 24 no 7 pp 1789ndash1797 1995

[8] M R Ozkale and S Kaccediliranlar ldquo+e restricted and unre-stricted two-parameter estimatorsrdquo Communications in Sta-tistics-0eory and Methods vol 36 no 15 pp 2707ndash27252007

[9] S Sakallıoglu and S Kaccedilıranlar ldquoA new biased estimatorbased on ridge estimationrdquo Statistical Papers vol 49 no 4pp 669ndash689 2008

Table 11 +e results of regression coefficients and the corresponding MSE values

Coef 1113954α 1113954α(k) 1113954α(d)1113954dalt 1113954α(d)1113954dopt 1113954αKL(kHNM) 1113954αKL(kmin)

α0 minus 197127 minus 167613 minus 125762 minus 188410 minus 165855 minus 188782α1 00327 01419 02951 00648 01485 00636α2 04059 03576 02875 03914 03548 03922α3 02421 00709 minus 01696 01918 00606 01937MSE 173326 2130519 5828312 1660293 2211899 1660168kd mdash 00527 05282 09423 00258 00065

Table 12 Correlation matrix

X1 X2 X3

X1 1000 0210 0999X2 0210 1000 0208X3 0999 0208 1000

Scientifica 15

[10] H Yang and X Chang ldquoA new two-parameter estimator inlinear regressionrdquo Communications in Statistics-0eory andMethods vol 39 no 6 pp 923ndash934 2010

[11] M Roozbeh ldquoOptimal QR-based estimation in partially linearregression models with correlated errors using GCV crite-rionrdquo Computational Statistics amp Data Analysis vol 117pp 45ndash61 2018

[12] F Akdeniz and M Roozbeh ldquoGeneralized difference-basedweightedmixed almost unbiased ridge estimator in partially linearmodelsrdquo Statistical Papers vol 60 no 5 pp 1717ndash1739 2019

[13] A F Lukman K Ayinde S Binuomote and O A ClementldquoModified ridge-type estimator to combat multicollinearityapplication to chemical datardquo Journal of Chemometricsvol 33 no 5 p e3125 2019

[14] A F Lukman K Ayinde S K Sek and E Adewuyi ldquoAmodified new two-parameter estimator in a linear regressionmodelrdquo Modelling and Simulation in Engineering vol 2019Article ID 6342702 10 pages 2019

[15] A E Hoerl R W Kannard and K F Baldwin ldquoRidge re-gressionsome simulationsrdquo Communications in Statisticsvol 4 no 2 pp 105ndash123 1975

[16] G C McDonald and D I Galarneau ldquoA monte carlo eval-uation of some ridge-type estimatorsrdquo Journal of the Amer-ican Statistical Association vol 70 no 350 pp 407ndash416 1975

[17] J F Lawless and P Wang ldquoA simulation study of ridge andother regression estimatorsrdquo Communications in Statistics-0eory and Methods vol 5 no 4 pp 307ndash323 1976

[18] D W Wichern and G A Churchill ldquoA comparison of ridgeestimatorsrdquo Technometrics vol 20 no 3 pp 301ndash311 1978

[19] B M G Kibria ldquoPerformance of some new ridge regressionestimatorsrdquo Communications in Statistics-Simulation andComputation vol 32 no 1 pp 419ndash435 2003

[20] A F Lukman and K Ayinde ldquoReview and classifications ofthe ridge parameter estimation techniquesrdquoHacettepe Journalof Mathematics and Statistics vol 46 no 5 pp 953ndash967 2017

[21] A K M E Saleh M Arashi and B M G Kibria 0eory ofRidge Regression Estimation with Applications WileyHoboken NJ USA 2019

[22] K Liu ldquoUsing Liu-type estimator to combat collinearityrdquoCommunications in Statistics-0eory and Methods vol 32no 5 pp 1009ndash1020 2003

[23] K Alheety and B M G Kibria ldquoOn the Liu and almostunbiased Liu estimators in the presence of multicollinearitywith heteroscedastic or correlated errorsrdquo Surveys in Math-ematics and its Applications vol 4 pp 155ndash167 2009

[24] X-Q Liu ldquoImproved Liu estimator in a linear regressionmodelrdquo Journal of Statistical Planning and Inference vol 141no 1 pp 189ndash196 2011

[25] Y Li and H Yang ldquoA new Liu-type estimator in linear regressionmodelrdquo Statistical Papers vol 53 no 2 pp 427ndash437 2012

[26] B Kan O Alpu and B Yazıcı ldquoRobust ridge and robust Liuestimator for regression based on the LTS estimatorrdquo Journalof Applied Statistics vol 40 no 3 pp 644ndash655 2013

[27] R A Farghali ldquoGeneralized Liu-type estimator for linearregressionrdquo International Journal of Research and Reviews inApplied Sciences vol 38 no 1 pp 52ndash63 2019

[28] S G Wang M X Wu and Z Z Jia Matrix InequalitiesChinese Science Press Beijing China 2nd edition 2006

[29] R W Farebrother ldquoFurther results on the mean square errorof ridge regressionrdquo Journal of the Royal Statistical SocietySeries B (Methodological) vol 38 no 3 pp 248ndash250 1976

[30] G Trenkler and H Toutenburg ldquoMean squared error matrixcomparisons between biased estimators-an overview of recentresultsrdquo Statistical Papers vol 31 no 1 pp 165ndash179 1990

[31] B M G Kibria and S Banik ldquoSome ridge regression esti-mators and their performancesrdquo Journal of Modern AppliedStatistical Methods vol 15 no 1 pp 206ndash238 2016

[32] D G Gibbons ldquoA simulation study of some ridge estimatorsrdquoJournal of the American Statistical Association vol 76no 373 pp 131ndash139 1981

[33] J P Newhouse and S D Oman ldquoAn evaluation of ridgeestimators A report prepared for United States air forceproject RANDrdquo 1971

[34] H Woods H H Steinour and H R Starke ldquoEffect ofcomposition of Portland cement on heat evolved duringhardeningrdquo Industrial amp Engineering Chemistry vol 24no 11 pp 1207ndash1214 1932

[35] S Chatterjee and A S Hadi Regression Analysis by ExampleWiley Hoboken NJ USA 1977

[36] S Kaciranlar S Sakallioglu F Akdeniz G P H Styan andH J Werner ldquoA new biased estimator in linear regression anda detailed analysis of the widely-analysed dataset on portlandcementrdquo Sankhya 0e Indian Journal of Statistics Series Bvol 61 pp 443ndash459 1999

[37] S Chatterjee and A S Haadi Regression Analysis by ExampleWiley Hoboken NJ USA 2006

[38] E Malinvard Statistical Methods of Econometrics North-Holland Publishing Company Amsterdam Netherlands 3rdedition 1980

[39] D N Gujarati Basic Econometrics McGraw-Hill New YorkNY USA 1995

[40] A F Lukman K Ayinde and A S Ajiboye ldquoMonte Carlostudy of some classification-based ridge parameter estima-torsrdquo Journal of Modern Applied Statistical Methods vol 16no 1 pp 428ndash451 2017

16 Scientifica

32 Simulation Results and Discussion From Tables 1ndash8and Figures 1ndash6 it appears that as the values of σ increasethe MSE values also increase (Figure 3) while the sample sizeincreases as the MSE values decrease (Figure 4) Ridge Liuand proposed KL estimators uniformly dominate the ordinaryleast squares (OLS) estimator In general from these tables anincrease in the levels of multicollinearity and the number ofexplanatory variables increase the estimated MSE values of theestimators +e figures consistently show that the OLS esti-mator performs worst when there is multicollinearity FromFigures 1ndash6 and simulation Tables 1ndash8 it clearly indicated thatfor ρ 090 or less the proposed estimator uniformly

dominates the ridge regression estimator while Liu performedmuch better than both proposed and ridge estimators for smalld say 03 or lessWhen ρ 099 the ridge regression performsthe best for higher k while the proposed estimator performsthe best for say k (say 03 or less) When d k 05 andρ 099 both ridge and KL estimators outperform the Liuestimator None of the estimators uniformly dominates eachother However it appears that our proposed estimator KLperforms better in the wider space of d k in the parameterspace If we review all Tables 1ndash8 we observed that theconclusions about the performance of all estimators remainthe same for both p 3 and p 7

Table 6 Continued

N 30 09 099Sigma k d OLS Ridge Liu New ridge OLS Ridge Liu New ridge

03 18721 14846 13432 46491 50959 5370704 17205 16130 11091 34538 72431 7062605 15899 17479 9229 26865 97978 8644306 14763 18895 7737 21588 127600 10058607 13766 20376 6534 17777 161296 11308908 12882 21923 5562 14924 199068 12413409 12095 23535 4775 12725 240914 1339211 11389 25214 4138 10992 286835 142634

Table 7 Estimated MSE when n 100 p 7 and ρ 070 and 080

n 100 07 08Sigma k d OLS Ridge Liu New ridge OLS Ridge Liu New ridge1 01 0174 0173 0163 0171 0263 0259 0235 0255

02 0171 0164 0169 0255 0238 024903 0170 0165 0166 0252 0241 024304 0169 0166 0164 0249 0244 023705 0167 0168 0161 0246 0247 023206 0166 0169 0159 0243 0250 022707 0165 0170 0157 0240 0253 022208 0164 0171 0155 0238 0256 021809 0163 0173 0154 0235 0259 02141 0162 0174 0152 0233 0263 0210

5 01 4356 4320 4055 4284 6563 6474 5852 638602 4285 4087 4214 6388 5928 621603 4250 4120 4146 6304 6005 605304 4216 4153 4079 6222 6082 589505 4182 4187 4013 6143 6160 574406 4149 4220 3949 6066 6239 559807 4116 4254 3887 5991 6319 545708 4084 4288 3826 5917 6399 532209 4053 4322 3767 5846 6481 51911 4022 4356 3708 5777 6563 5066

10 01 17425 17281 16219 17138 26250 25896 23408 2554502 17140 16350 16858 25551 23713 2486603 17001 16482 16584 25216 24020 2421204 16864 16614 16316 24891 24330 2358205 16729 16748 16054 24573 24643 2297506 16597 16882 15797 24265 24959 2238907 16467 17016 15547 23964 25277 2182508 16339 17152 15301 23671 25599 2128009 16213 17288 15062 23385 25923 207551 16089 17425 14827 23107 26250 20247

Scientifica 9

Table 8 Estimated MSE when n 100 p 7 and ρ 090 and 099

n 100 09 099Sigma k d OLS Ridge Liu New ridge OLS Ridge Liu New ridge1 01 0546 0529 0431 0512 6389 4391 1624 2949

02 0513 0442 0482 3407 1934 183603 0498 0454 0456 2819 2298 145304 0485 0466 0432 2423 2718 134705 0472 0478 0411 2135 3192 135906 0460 0491 0392 1914 3721 142607 0449 0504 0375 1738 4306 151908 0439 0517 0360 1593 4945 162509 0429 0531 0346 1472 5640 17371 0420 0546 0334 1370 6389 1851

5 01 13640 13216 10676 12802 159732 109722 38895 7328402 12820 10979 12037 84915 47018 4450603 12448 11289 11336 69971 56467 3386504 12099 11605 10693 59823 67242 3014605 11770 11928 10102 52370 79343 2941706 11460 12257 9558 46597 92769 3009007 11168 12593 9056 41953 107521 3145508 10891 12935 8593 38114 123599 3317109 10628 13284 8165 34875 141003 350631 10379 13640 7768 32097 159732 37036

10 01 54558 52866 42699 51212 638928 438910 155399 29312102 51282 43914 48150 339663 187945 17787403 49796 45155 45344 279860 225785 13515104 48399 46422 42768 239236 268921 12012005 47084 47714 40397 209391 317351 11705306 45843 49032 38214 186265 371077 11959907 44670 50375 36198 167659 430097 12492208 43560 51744 34336 152274 494412 13165409 42508 53138 32612 139287 564022 1390941 41509 54558 31014 128149 638928 146866

OLSRidge

LiuKL

025

030

035

040

MSE

02 04 06 08 1000d = k

(a)

OLSRidge

LiuKL

02 04 06 08 1000d = k

20

25

30

35

MSE

(b)

Figure 1 Continued

10 Scientifica

4 Numerical Examples

To illustrate our theoretical results we consider two datasets(i) famous Portland cement data originally adopted byWoods et al [34] and (ii) French economy data from

Chatterjee and Hadi [35] and they are analyzed in thefollowing sections respectively

41 Example 1 Portland Data +ese data are widely knownas the Portland cement dataset It was originally adopted by

OLSRidge

LiuKL

025

035

045

055

MSE

02 04 06 08 1000d = k

(c)

OLSRidge

LiuKL

02 04 06 08 1000d = k

20

30

40

50

MSE

(d)

Figure 1 Estimated MSEs for n 30 Sigma 1 10 rho 070 080 and different values of k d (a) n 30 p 3 sigma 1 and rho 070(b) n 30 p 3 sigma 10 and rho 070 (c) n 30 p 3 sigma 1 and rho 080 (d) n 30 p 3 sigma 10 and rho 080

OLSRidge

LiuKL

04

06

08

10

12

MSE

02 04 06 08 1000d = k

(a)

OLSRidge

LiuKL

20

40

60

80

100M

SE

02 04 06 08 1000d = k

(b)

OLSRidge

LiuKL

02468

101214

MSE

02 04 06 08 1000d = k

(c)

OLSRidge

LiuKL

0200

600

1000

1400

MSE

02 04 06 08 1000d = k

(d)

Figure 2 Estimated MSEs for n 30 sigma 1 10 rho 090 099 and different values of k d (a) n 30 p 3 sigma 1 and rho 090(b) n 30 p 3 sigma 10 and rho 090 (c) n 30 p 3 sigma 1 and rho 099 (d) n 30 p 3 sigma 10 and rho 099

Scientifica 11

OLSRidge

LiuKL

0

10

20

30

MSE

2 4 6 8 100Sigma

(a)

OLSRidge

LiuKL

0

10

30

50

MSE

2 4 6 8 100Sigma

(b)

OLSRidge

LiuKL

0

40

80

120

MSE

2 4 6 8 100Sigma

(c)

OLSRidge

LiuKL

2 4 6 8 100Sigma

0

400

800

1200

MSE

(d)

Figure 3 EstimatedMSEs for n 30 d 05 and different values of rho and sigma (a) n 30 p 3 d 05 and rho 070 (b) n 30 p 3d 05 and rho 080 (c) n 30 p 3 d 05 and rho 090 (d) n 30 p 3 d 05 and rho 099

OLSRidge

LiuKL

50 70 9030n

010

020

030

040

MSE

(a)

OLSRidge

LiuKL

50 70 9030n

02

04

06

MSE

(b)

Figure 4 Continued

12 Scientifica

Woods et al [34] It has also been analyzed by the followingauthors Kaciranlar et al [36] Li and Yang [25] and recentlyby Lukman et al [13] +e regression model for these data isdefined as

yi β0 + β1X1 + β2X2 + β3X3 + β4X4 + εi (46)

where yi heat evolved after 180 days of curing measured incalories per gram of cement X1 tricalcium aluminateX2 tricalcium silicate X3 tetracalcium aluminoferriteand X4 β-dicalcium silicate +e correlation matrix of thepredictor variables is given in Table 9

OLSRidge

LiuKL

50 70 9030n

02

06

10

MSE

(c)

OLSRidge

LiuKL

50 70 9030n

0

2

4

6

8

12

MSE

(d)

Figure 4 Estimated MSEs for sigma 1 p 3 and different values of rho and sample size (a)p 3 sigma 1 d 05 and rho 070(b)p 3 sigma 1 d 05 and rho 080 (c)p 3 sigma 1 d 05 and rho 090 (d)p 3 sigma 1 d 05 and rho 099

OLSRidge

LiuKL

20

30

40

MSE

4 5 6 7 83p

(a)

OLSRidge

LiuKL

3

4

5

6

7M

SE

4 5 6 7 83p

(b)

6

8

10

14

MSE

OLSRidge

LiuKL

4 5 6 7 83p

(c)

OLSRidge

LiuKL

4 5 6 7 83p

0

50

100

150

MSE

(d)

Figure 5 Estimated MSEs for n 100 d 05 sigma 5 and different values of rho and p (a) n 100 sigma 5 d 05 and rho 070 (b)n 100 sigma 5 d 05 and rho 080 (c) n 100 sigma 5 d 05 and rho 090 (d) n 100 sigma 5 d 05 and rho 099

Scientifica 13

+e variance inflation factors are VIF1 = 3850VIF2 = 25442 VIF3 = 4687 and VIF4 = 28251 Eigen-values of XprimeX are λ1 44676206 λ2 5965422

λ3 809952 and λ4 105419 and the condition numberof XprimeX is approximately 424 +e VIFs the eigenvalues

and the condition number all indicate the presence ofsevere multicollinearity +e estimated parameters andMSE are presented in Table 10 It appears from Table 11that the proposed estimator performed the best in thesense of smaller MSE

OLSRidge

LiuKL

0

100

200

300

MSE

075 085 095065Rho

(a)

OLSRidge

LiuKL

0

200

400

600

800

MSE

075 085 095065Rho

(b)

OLSRidge

LiuKL

0

20

40

60

80

MSE

075 085 095065Rho

(c)

OLSRidge

LiuKL

0

50

100

150

MSE

075 085 095065Rho

(d)

Figure 6 Estimated MSEs for n 100 p 3 7 d 05 sigma 5 and different values of rho (a) n 30 p 3 sigma 5 and d 05 (b)n 30 p 7 sigma 5 and d 05 (c) n 100 p 3 sigma 5 and d 05 (d) n 100 p 7 sigma 5 and d 05

Table 9 Correlation matrix

X1 X2 X3 X4

X1 1000 0229 minus 0824 minus 0245X2 0229 1000 minus 0139 minus 0973X3 minus 0824 minus 0139 1000 0030X4 minus 0245 minus 0973 0030 1000

Table 10 +e results of regression coefficients and the corresponding MSE values

Coef 1113954α 1113954α(k) 1113954α(d)1113954dalt 1113954αKL(kHNM) 1113954αKL(kmin)

α0 624054 85870 276490 minus 197876 276068α1 15511 21046 19010 23965 19090α2 05102 10648 08701 13573 08688α3 01019 06681 04621 09666 04680α4 minus 01441 03996 02082 06862 02074MSE 491209 298983 2170963 7255603 217096kd mdash 00077 044195 000235 000047

14 Scientifica

42 Example 2 French Economy Data +e French economydata in Chatterjee and Hadi [37] are considered in this ex-ample It has been analyzed by Malinvard [38] and Liu [6]among others+e variables are imports domestic productionstock formation and domestic consumption All are measuredin milliards of French francs for the years 1949 through 1966

+e regression model for these data is defined as

yi β0 + β1X1 + β2X2 + β3X3 + εi (47)

where yi IMPORT X1 domestic production X2 stockformation and X3 domestic consumption +e correlationmatrix of the predicted variable is given in Table 12

+e variance inflation factors areVIF1 469688VIF2 1047 and VIF3 469338 +e ei-genvalues of the XprimeX matrix are λ1 161779 λ2 158 andλ3 4961 and the condition number is 32612 If we reviewthe above correlation matrix VIFs and condition number itcan be said that there is presence of severe multicollinearityexisting in the predictor variables

+e biasing parameter for the new estimator is defined in(41) and (42) +e biasing parameter for the ridge and Liuestimator is provided in (6) (8) and (9) respectively

We analyzed the data using the biasing parameters foreach of the estimators and presented the results in Tables 10and 11 It can be seen from Tables 10 and 11 that theproposed estimator performed the best in the sense ofsmaller MSE

5 Summary and Concluding Remarks

In this paper we introduced a new biased estimator toovercome the multicollinearity problem for the multiplelinear regression model and provided the estimation tech-nique of the biasing parameter A simulation study has beenconducted to compare the performance of the proposedestimator and Liu [6] and ridge regression estimators [3]Simulation results evidently show that the proposed esti-mator performed better than both Liu and ridge under somecondition on the shrinkage parameter Two sets of real-lifedata are analyzed to illustrate the benefits of using the newestimator in the context of a linear regression model +eproposed estimator is recommended for researchers in this

area Its application can be extended to other regressionmodels for example logistic regression Poisson ZIP andrelated models and those possibilities are under currentinvestigation [37 39 40]

Data Availability

Data will be made available on request

Conflicts of Interest

+e authors declare that they have no conflicts of interest

Acknowledgments

We are dedicating this article to those who lost their livesbecause of COVID-19

References

[1] C Stein ldquoInadmissibility of the usual estimator for mean ofmultivariate normal distributionrdquo in Proceedings of the 0irdBerkley Symposium on Mathematical and Statistics Proba-bility J Neyman Ed vol 1 pp 197ndash206 Springer BerlinGermany 1956

[2] W F Massy ldquoPrincipal components regression in exploratorystatistical researchrdquo Journal of the American Statistical As-sociation vol 60 no 309 pp 234ndash256 1965

[3] A E Hoerl and R W Kennard ldquoRidge regression biasedestimation for nonorthogonal problemsrdquo Technometricsvol 12 no 1 pp 55ndash67 1970

[4] L S Mayer and T A Willke ldquoOn biased estimation in linearmodelsrdquo Technometrics vol 15 no 3 pp 497ndash508 1973

[5] B F Swindel ldquoGood ridge estimators based on prior infor-mationrdquo Communications in Statistics-0eory and Methodsvol 5 no 11 pp 1065ndash1075 1976

[6] K Liu ldquoA new class of biased estimate in linear regressionrdquoCommunication in Statistics- 0eory and Methods vol 22pp 393ndash402 1993

[7] F Akdeniz and S Kaccediliranlar ldquoOn the almost unbiasedgeneralized liu estimator and unbiased estimation of the biasand mserdquo Communications in Statistics-0eory and Methodsvol 24 no 7 pp 1789ndash1797 1995

[8] M R Ozkale and S Kaccediliranlar ldquo+e restricted and unre-stricted two-parameter estimatorsrdquo Communications in Sta-tistics-0eory and Methods vol 36 no 15 pp 2707ndash27252007

[9] S Sakallıoglu and S Kaccedilıranlar ldquoA new biased estimatorbased on ridge estimationrdquo Statistical Papers vol 49 no 4pp 669ndash689 2008

Table 11 +e results of regression coefficients and the corresponding MSE values

Coef 1113954α 1113954α(k) 1113954α(d)1113954dalt 1113954α(d)1113954dopt 1113954αKL(kHNM) 1113954αKL(kmin)

α0 minus 197127 minus 167613 minus 125762 minus 188410 minus 165855 minus 188782α1 00327 01419 02951 00648 01485 00636α2 04059 03576 02875 03914 03548 03922α3 02421 00709 minus 01696 01918 00606 01937MSE 173326 2130519 5828312 1660293 2211899 1660168kd mdash 00527 05282 09423 00258 00065

Table 12 Correlation matrix

X1 X2 X3

X1 1000 0210 0999X2 0210 1000 0208X3 0999 0208 1000

Scientifica 15

[10] H Yang and X Chang ldquoA new two-parameter estimator inlinear regressionrdquo Communications in Statistics-0eory andMethods vol 39 no 6 pp 923ndash934 2010

[11] M Roozbeh ldquoOptimal QR-based estimation in partially linearregression models with correlated errors using GCV crite-rionrdquo Computational Statistics amp Data Analysis vol 117pp 45ndash61 2018

[12] F Akdeniz and M Roozbeh ldquoGeneralized difference-basedweightedmixed almost unbiased ridge estimator in partially linearmodelsrdquo Statistical Papers vol 60 no 5 pp 1717ndash1739 2019

[13] A F Lukman K Ayinde S Binuomote and O A ClementldquoModified ridge-type estimator to combat multicollinearityapplication to chemical datardquo Journal of Chemometricsvol 33 no 5 p e3125 2019

[14] A F Lukman K Ayinde S K Sek and E Adewuyi ldquoAmodified new two-parameter estimator in a linear regressionmodelrdquo Modelling and Simulation in Engineering vol 2019Article ID 6342702 10 pages 2019

[15] A E Hoerl R W Kannard and K F Baldwin ldquoRidge re-gressionsome simulationsrdquo Communications in Statisticsvol 4 no 2 pp 105ndash123 1975

[16] G C McDonald and D I Galarneau ldquoA monte carlo eval-uation of some ridge-type estimatorsrdquo Journal of the Amer-ican Statistical Association vol 70 no 350 pp 407ndash416 1975

[17] J F Lawless and P Wang ldquoA simulation study of ridge andother regression estimatorsrdquo Communications in Statistics-0eory and Methods vol 5 no 4 pp 307ndash323 1976

[18] D W Wichern and G A Churchill ldquoA comparison of ridgeestimatorsrdquo Technometrics vol 20 no 3 pp 301ndash311 1978

[19] B M G Kibria ldquoPerformance of some new ridge regressionestimatorsrdquo Communications in Statistics-Simulation andComputation vol 32 no 1 pp 419ndash435 2003

[20] A F Lukman and K Ayinde ldquoReview and classifications ofthe ridge parameter estimation techniquesrdquoHacettepe Journalof Mathematics and Statistics vol 46 no 5 pp 953ndash967 2017

[21] A K M E Saleh M Arashi and B M G Kibria 0eory ofRidge Regression Estimation with Applications WileyHoboken NJ USA 2019

[22] K Liu ldquoUsing Liu-type estimator to combat collinearityrdquoCommunications in Statistics-0eory and Methods vol 32no 5 pp 1009ndash1020 2003

[23] K Alheety and B M G Kibria ldquoOn the Liu and almostunbiased Liu estimators in the presence of multicollinearitywith heteroscedastic or correlated errorsrdquo Surveys in Math-ematics and its Applications vol 4 pp 155ndash167 2009

[24] X-Q Liu ldquoImproved Liu estimator in a linear regressionmodelrdquo Journal of Statistical Planning and Inference vol 141no 1 pp 189ndash196 2011

[25] Y Li and H Yang ldquoA new Liu-type estimator in linear regressionmodelrdquo Statistical Papers vol 53 no 2 pp 427ndash437 2012

[26] B Kan O Alpu and B Yazıcı ldquoRobust ridge and robust Liuestimator for regression based on the LTS estimatorrdquo Journalof Applied Statistics vol 40 no 3 pp 644ndash655 2013

[27] R A Farghali ldquoGeneralized Liu-type estimator for linearregressionrdquo International Journal of Research and Reviews inApplied Sciences vol 38 no 1 pp 52ndash63 2019

[28] S G Wang M X Wu and Z Z Jia Matrix InequalitiesChinese Science Press Beijing China 2nd edition 2006

[29] R W Farebrother ldquoFurther results on the mean square errorof ridge regressionrdquo Journal of the Royal Statistical SocietySeries B (Methodological) vol 38 no 3 pp 248ndash250 1976

[30] G Trenkler and H Toutenburg ldquoMean squared error matrixcomparisons between biased estimators-an overview of recentresultsrdquo Statistical Papers vol 31 no 1 pp 165ndash179 1990

[31] B M G Kibria and S Banik ldquoSome ridge regression esti-mators and their performancesrdquo Journal of Modern AppliedStatistical Methods vol 15 no 1 pp 206ndash238 2016

[32] D G Gibbons ldquoA simulation study of some ridge estimatorsrdquoJournal of the American Statistical Association vol 76no 373 pp 131ndash139 1981

[33] J P Newhouse and S D Oman ldquoAn evaluation of ridgeestimators A report prepared for United States air forceproject RANDrdquo 1971

[34] H Woods H H Steinour and H R Starke ldquoEffect ofcomposition of Portland cement on heat evolved duringhardeningrdquo Industrial amp Engineering Chemistry vol 24no 11 pp 1207ndash1214 1932

[35] S Chatterjee and A S Hadi Regression Analysis by ExampleWiley Hoboken NJ USA 1977

[36] S Kaciranlar S Sakallioglu F Akdeniz G P H Styan andH J Werner ldquoA new biased estimator in linear regression anda detailed analysis of the widely-analysed dataset on portlandcementrdquo Sankhya 0e Indian Journal of Statistics Series Bvol 61 pp 443ndash459 1999

[37] S Chatterjee and A S Haadi Regression Analysis by ExampleWiley Hoboken NJ USA 2006

[38] E Malinvard Statistical Methods of Econometrics North-Holland Publishing Company Amsterdam Netherlands 3rdedition 1980

[39] D N Gujarati Basic Econometrics McGraw-Hill New YorkNY USA 1995

[40] A F Lukman K Ayinde and A S Ajiboye ldquoMonte Carlostudy of some classification-based ridge parameter estima-torsrdquo Journal of Modern Applied Statistical Methods vol 16no 1 pp 428ndash451 2017

16 Scientifica

Table 8 Estimated MSE when n 100 p 7 and ρ 090 and 099

n 100 09 099Sigma k d OLS Ridge Liu New ridge OLS Ridge Liu New ridge1 01 0546 0529 0431 0512 6389 4391 1624 2949

02 0513 0442 0482 3407 1934 183603 0498 0454 0456 2819 2298 145304 0485 0466 0432 2423 2718 134705 0472 0478 0411 2135 3192 135906 0460 0491 0392 1914 3721 142607 0449 0504 0375 1738 4306 151908 0439 0517 0360 1593 4945 162509 0429 0531 0346 1472 5640 17371 0420 0546 0334 1370 6389 1851

5 01 13640 13216 10676 12802 159732 109722 38895 7328402 12820 10979 12037 84915 47018 4450603 12448 11289 11336 69971 56467 3386504 12099 11605 10693 59823 67242 3014605 11770 11928 10102 52370 79343 2941706 11460 12257 9558 46597 92769 3009007 11168 12593 9056 41953 107521 3145508 10891 12935 8593 38114 123599 3317109 10628 13284 8165 34875 141003 350631 10379 13640 7768 32097 159732 37036

10 01 54558 52866 42699 51212 638928 438910 155399 29312102 51282 43914 48150 339663 187945 17787403 49796 45155 45344 279860 225785 13515104 48399 46422 42768 239236 268921 12012005 47084 47714 40397 209391 317351 11705306 45843 49032 38214 186265 371077 11959907 44670 50375 36198 167659 430097 12492208 43560 51744 34336 152274 494412 13165409 42508 53138 32612 139287 564022 1390941 41509 54558 31014 128149 638928 146866

OLSRidge

LiuKL

025

030

035

040

MSE

02 04 06 08 1000d = k

(a)

OLSRidge

LiuKL

02 04 06 08 1000d = k

20

25

30

35

MSE

(b)

Figure 1 Continued

10 Scientifica

4 Numerical Examples

To illustrate our theoretical results we consider two datasets(i) famous Portland cement data originally adopted byWoods et al [34] and (ii) French economy data from

Chatterjee and Hadi [35] and they are analyzed in thefollowing sections respectively

41 Example 1 Portland Data +ese data are widely knownas the Portland cement dataset It was originally adopted by

OLSRidge

LiuKL

025

035

045

055

MSE

02 04 06 08 1000d = k

(c)

OLSRidge

LiuKL

02 04 06 08 1000d = k

20

30

40

50

MSE

(d)

Figure 1 Estimated MSEs for n 30 Sigma 1 10 rho 070 080 and different values of k d (a) n 30 p 3 sigma 1 and rho 070(b) n 30 p 3 sigma 10 and rho 070 (c) n 30 p 3 sigma 1 and rho 080 (d) n 30 p 3 sigma 10 and rho 080

OLSRidge

LiuKL

04

06

08

10

12

MSE

02 04 06 08 1000d = k

(a)

OLSRidge

LiuKL

20

40

60

80

100M

SE

02 04 06 08 1000d = k

(b)

OLSRidge

LiuKL

02468

101214

MSE

02 04 06 08 1000d = k

(c)

OLSRidge

LiuKL

0200

600

1000

1400

MSE

02 04 06 08 1000d = k

(d)

Figure 2 Estimated MSEs for n 30 sigma 1 10 rho 090 099 and different values of k d (a) n 30 p 3 sigma 1 and rho 090(b) n 30 p 3 sigma 10 and rho 090 (c) n 30 p 3 sigma 1 and rho 099 (d) n 30 p 3 sigma 10 and rho 099

Scientifica 11

OLSRidge

LiuKL

0

10

20

30

MSE

2 4 6 8 100Sigma

(a)

OLSRidge

LiuKL

0

10

30

50

MSE

2 4 6 8 100Sigma

(b)

OLSRidge

LiuKL

0

40

80

120

MSE

2 4 6 8 100Sigma

(c)

OLSRidge

LiuKL

2 4 6 8 100Sigma

0

400

800

1200

MSE

(d)

Figure 3 EstimatedMSEs for n 30 d 05 and different values of rho and sigma (a) n 30 p 3 d 05 and rho 070 (b) n 30 p 3d 05 and rho 080 (c) n 30 p 3 d 05 and rho 090 (d) n 30 p 3 d 05 and rho 099

OLSRidge

LiuKL

50 70 9030n

010

020

030

040

MSE

(a)

OLSRidge

LiuKL

50 70 9030n

02

04

06

MSE

(b)

Figure 4 Continued

12 Scientifica

Woods et al [34] It has also been analyzed by the followingauthors Kaciranlar et al [36] Li and Yang [25] and recentlyby Lukman et al [13] +e regression model for these data isdefined as

yi β0 + β1X1 + β2X2 + β3X3 + β4X4 + εi (46)

where yi heat evolved after 180 days of curing measured incalories per gram of cement X1 tricalcium aluminateX2 tricalcium silicate X3 tetracalcium aluminoferriteand X4 β-dicalcium silicate +e correlation matrix of thepredictor variables is given in Table 9

OLSRidge

LiuKL

50 70 9030n

02

06

10

MSE

(c)

OLSRidge

LiuKL

50 70 9030n

0

2

4

6

8

12

MSE

(d)

Figure 4 Estimated MSEs for sigma 1 p 3 and different values of rho and sample size (a)p 3 sigma 1 d 05 and rho 070(b)p 3 sigma 1 d 05 and rho 080 (c)p 3 sigma 1 d 05 and rho 090 (d)p 3 sigma 1 d 05 and rho 099

OLSRidge

LiuKL

20

30

40

MSE

4 5 6 7 83p

(a)

OLSRidge

LiuKL

3

4

5

6

7M

SE

4 5 6 7 83p

(b)

6

8

10

14

MSE

OLSRidge

LiuKL

4 5 6 7 83p

(c)

OLSRidge

LiuKL

4 5 6 7 83p

0

50

100

150

MSE

(d)

Figure 5 Estimated MSEs for n 100 d 05 sigma 5 and different values of rho and p (a) n 100 sigma 5 d 05 and rho 070 (b)n 100 sigma 5 d 05 and rho 080 (c) n 100 sigma 5 d 05 and rho 090 (d) n 100 sigma 5 d 05 and rho 099

Scientifica 13

+e variance inflation factors are VIF1 = 3850VIF2 = 25442 VIF3 = 4687 and VIF4 = 28251 Eigen-values of XprimeX are λ1 44676206 λ2 5965422

λ3 809952 and λ4 105419 and the condition numberof XprimeX is approximately 424 +e VIFs the eigenvalues

and the condition number all indicate the presence ofsevere multicollinearity +e estimated parameters andMSE are presented in Table 10 It appears from Table 11that the proposed estimator performed the best in thesense of smaller MSE

OLSRidge

LiuKL

0

100

200

300

MSE

075 085 095065Rho

(a)

OLSRidge

LiuKL

0

200

400

600

800

MSE

075 085 095065Rho

(b)

OLSRidge

LiuKL

0

20

40

60

80

MSE

075 085 095065Rho

(c)

OLSRidge

LiuKL

0

50

100

150

MSE

075 085 095065Rho

(d)

Figure 6 Estimated MSEs for n 100 p 3 7 d 05 sigma 5 and different values of rho (a) n 30 p 3 sigma 5 and d 05 (b)n 30 p 7 sigma 5 and d 05 (c) n 100 p 3 sigma 5 and d 05 (d) n 100 p 7 sigma 5 and d 05

Table 9 Correlation matrix

X1 X2 X3 X4

X1 1000 0229 minus 0824 minus 0245X2 0229 1000 minus 0139 minus 0973X3 minus 0824 minus 0139 1000 0030X4 minus 0245 minus 0973 0030 1000

Table 10 +e results of regression coefficients and the corresponding MSE values

Coef 1113954α 1113954α(k) 1113954α(d)1113954dalt 1113954αKL(kHNM) 1113954αKL(kmin)

α0 624054 85870 276490 minus 197876 276068α1 15511 21046 19010 23965 19090α2 05102 10648 08701 13573 08688α3 01019 06681 04621 09666 04680α4 minus 01441 03996 02082 06862 02074MSE 491209 298983 2170963 7255603 217096kd mdash 00077 044195 000235 000047

14 Scientifica

42 Example 2 French Economy Data +e French economydata in Chatterjee and Hadi [37] are considered in this ex-ample It has been analyzed by Malinvard [38] and Liu [6]among others+e variables are imports domestic productionstock formation and domestic consumption All are measuredin milliards of French francs for the years 1949 through 1966

+e regression model for these data is defined as

yi β0 + β1X1 + β2X2 + β3X3 + εi (47)

where yi IMPORT X1 domestic production X2 stockformation and X3 domestic consumption +e correlationmatrix of the predicted variable is given in Table 12

+e variance inflation factors areVIF1 469688VIF2 1047 and VIF3 469338 +e ei-genvalues of the XprimeX matrix are λ1 161779 λ2 158 andλ3 4961 and the condition number is 32612 If we reviewthe above correlation matrix VIFs and condition number itcan be said that there is presence of severe multicollinearityexisting in the predictor variables

+e biasing parameter for the new estimator is defined in(41) and (42) +e biasing parameter for the ridge and Liuestimator is provided in (6) (8) and (9) respectively

We analyzed the data using the biasing parameters foreach of the estimators and presented the results in Tables 10and 11 It can be seen from Tables 10 and 11 that theproposed estimator performed the best in the sense ofsmaller MSE

5 Summary and Concluding Remarks

In this paper we introduced a new biased estimator toovercome the multicollinearity problem for the multiplelinear regression model and provided the estimation tech-nique of the biasing parameter A simulation study has beenconducted to compare the performance of the proposedestimator and Liu [6] and ridge regression estimators [3]Simulation results evidently show that the proposed esti-mator performed better than both Liu and ridge under somecondition on the shrinkage parameter Two sets of real-lifedata are analyzed to illustrate the benefits of using the newestimator in the context of a linear regression model +eproposed estimator is recommended for researchers in this

area Its application can be extended to other regressionmodels for example logistic regression Poisson ZIP andrelated models and those possibilities are under currentinvestigation [37 39 40]

Data Availability

Data will be made available on request

Conflicts of Interest

+e authors declare that they have no conflicts of interest

Acknowledgments

We are dedicating this article to those who lost their livesbecause of COVID-19

References

[1] C Stein ldquoInadmissibility of the usual estimator for mean ofmultivariate normal distributionrdquo in Proceedings of the 0irdBerkley Symposium on Mathematical and Statistics Proba-bility J Neyman Ed vol 1 pp 197ndash206 Springer BerlinGermany 1956

[2] W F Massy ldquoPrincipal components regression in exploratorystatistical researchrdquo Journal of the American Statistical As-sociation vol 60 no 309 pp 234ndash256 1965

[3] A E Hoerl and R W Kennard ldquoRidge regression biasedestimation for nonorthogonal problemsrdquo Technometricsvol 12 no 1 pp 55ndash67 1970

[4] L S Mayer and T A Willke ldquoOn biased estimation in linearmodelsrdquo Technometrics vol 15 no 3 pp 497ndash508 1973

[5] B F Swindel ldquoGood ridge estimators based on prior infor-mationrdquo Communications in Statistics-0eory and Methodsvol 5 no 11 pp 1065ndash1075 1976

[6] K Liu ldquoA new class of biased estimate in linear regressionrdquoCommunication in Statistics- 0eory and Methods vol 22pp 393ndash402 1993

[7] F Akdeniz and S Kaccediliranlar ldquoOn the almost unbiasedgeneralized liu estimator and unbiased estimation of the biasand mserdquo Communications in Statistics-0eory and Methodsvol 24 no 7 pp 1789ndash1797 1995

[8] M R Ozkale and S Kaccediliranlar ldquo+e restricted and unre-stricted two-parameter estimatorsrdquo Communications in Sta-tistics-0eory and Methods vol 36 no 15 pp 2707ndash27252007

[9] S Sakallıoglu and S Kaccedilıranlar ldquoA new biased estimatorbased on ridge estimationrdquo Statistical Papers vol 49 no 4pp 669ndash689 2008

Table 11 +e results of regression coefficients and the corresponding MSE values

Coef 1113954α 1113954α(k) 1113954α(d)1113954dalt 1113954α(d)1113954dopt 1113954αKL(kHNM) 1113954αKL(kmin)

α0 minus 197127 minus 167613 minus 125762 minus 188410 minus 165855 minus 188782α1 00327 01419 02951 00648 01485 00636α2 04059 03576 02875 03914 03548 03922α3 02421 00709 minus 01696 01918 00606 01937MSE 173326 2130519 5828312 1660293 2211899 1660168kd mdash 00527 05282 09423 00258 00065

Table 12 Correlation matrix

X1 X2 X3

X1 1000 0210 0999X2 0210 1000 0208X3 0999 0208 1000

Scientifica 15

[10] H Yang and X Chang ldquoA new two-parameter estimator inlinear regressionrdquo Communications in Statistics-0eory andMethods vol 39 no 6 pp 923ndash934 2010

[11] M Roozbeh ldquoOptimal QR-based estimation in partially linearregression models with correlated errors using GCV crite-rionrdquo Computational Statistics amp Data Analysis vol 117pp 45ndash61 2018

[12] F Akdeniz and M Roozbeh ldquoGeneralized difference-basedweightedmixed almost unbiased ridge estimator in partially linearmodelsrdquo Statistical Papers vol 60 no 5 pp 1717ndash1739 2019

[13] A F Lukman K Ayinde S Binuomote and O A ClementldquoModified ridge-type estimator to combat multicollinearityapplication to chemical datardquo Journal of Chemometricsvol 33 no 5 p e3125 2019

[14] A F Lukman K Ayinde S K Sek and E Adewuyi ldquoAmodified new two-parameter estimator in a linear regressionmodelrdquo Modelling and Simulation in Engineering vol 2019Article ID 6342702 10 pages 2019

[15] A E Hoerl R W Kannard and K F Baldwin ldquoRidge re-gressionsome simulationsrdquo Communications in Statisticsvol 4 no 2 pp 105ndash123 1975

[16] G C McDonald and D I Galarneau ldquoA monte carlo eval-uation of some ridge-type estimatorsrdquo Journal of the Amer-ican Statistical Association vol 70 no 350 pp 407ndash416 1975

[17] J F Lawless and P Wang ldquoA simulation study of ridge andother regression estimatorsrdquo Communications in Statistics-0eory and Methods vol 5 no 4 pp 307ndash323 1976

[18] D W Wichern and G A Churchill ldquoA comparison of ridgeestimatorsrdquo Technometrics vol 20 no 3 pp 301ndash311 1978

[19] B M G Kibria ldquoPerformance of some new ridge regressionestimatorsrdquo Communications in Statistics-Simulation andComputation vol 32 no 1 pp 419ndash435 2003

[20] A F Lukman and K Ayinde ldquoReview and classifications ofthe ridge parameter estimation techniquesrdquoHacettepe Journalof Mathematics and Statistics vol 46 no 5 pp 953ndash967 2017

[21] A K M E Saleh M Arashi and B M G Kibria 0eory ofRidge Regression Estimation with Applications WileyHoboken NJ USA 2019

[22] K Liu ldquoUsing Liu-type estimator to combat collinearityrdquoCommunications in Statistics-0eory and Methods vol 32no 5 pp 1009ndash1020 2003

[23] K Alheety and B M G Kibria ldquoOn the Liu and almostunbiased Liu estimators in the presence of multicollinearitywith heteroscedastic or correlated errorsrdquo Surveys in Math-ematics and its Applications vol 4 pp 155ndash167 2009

[24] X-Q Liu ldquoImproved Liu estimator in a linear regressionmodelrdquo Journal of Statistical Planning and Inference vol 141no 1 pp 189ndash196 2011

[25] Y Li and H Yang ldquoA new Liu-type estimator in linear regressionmodelrdquo Statistical Papers vol 53 no 2 pp 427ndash437 2012

[26] B Kan O Alpu and B Yazıcı ldquoRobust ridge and robust Liuestimator for regression based on the LTS estimatorrdquo Journalof Applied Statistics vol 40 no 3 pp 644ndash655 2013

[27] R A Farghali ldquoGeneralized Liu-type estimator for linearregressionrdquo International Journal of Research and Reviews inApplied Sciences vol 38 no 1 pp 52ndash63 2019

[28] S G Wang M X Wu and Z Z Jia Matrix InequalitiesChinese Science Press Beijing China 2nd edition 2006

[29] R W Farebrother ldquoFurther results on the mean square errorof ridge regressionrdquo Journal of the Royal Statistical SocietySeries B (Methodological) vol 38 no 3 pp 248ndash250 1976

[30] G Trenkler and H Toutenburg ldquoMean squared error matrixcomparisons between biased estimators-an overview of recentresultsrdquo Statistical Papers vol 31 no 1 pp 165ndash179 1990

[31] B M G Kibria and S Banik ldquoSome ridge regression esti-mators and their performancesrdquo Journal of Modern AppliedStatistical Methods vol 15 no 1 pp 206ndash238 2016

[32] D G Gibbons ldquoA simulation study of some ridge estimatorsrdquoJournal of the American Statistical Association vol 76no 373 pp 131ndash139 1981

[33] J P Newhouse and S D Oman ldquoAn evaluation of ridgeestimators A report prepared for United States air forceproject RANDrdquo 1971

[34] H Woods H H Steinour and H R Starke ldquoEffect ofcomposition of Portland cement on heat evolved duringhardeningrdquo Industrial amp Engineering Chemistry vol 24no 11 pp 1207ndash1214 1932

[35] S Chatterjee and A S Hadi Regression Analysis by ExampleWiley Hoboken NJ USA 1977

[36] S Kaciranlar S Sakallioglu F Akdeniz G P H Styan andH J Werner ldquoA new biased estimator in linear regression anda detailed analysis of the widely-analysed dataset on portlandcementrdquo Sankhya 0e Indian Journal of Statistics Series Bvol 61 pp 443ndash459 1999

[37] S Chatterjee and A S Haadi Regression Analysis by ExampleWiley Hoboken NJ USA 2006

[38] E Malinvard Statistical Methods of Econometrics North-Holland Publishing Company Amsterdam Netherlands 3rdedition 1980

[39] D N Gujarati Basic Econometrics McGraw-Hill New YorkNY USA 1995

[40] A F Lukman K Ayinde and A S Ajiboye ldquoMonte Carlostudy of some classification-based ridge parameter estima-torsrdquo Journal of Modern Applied Statistical Methods vol 16no 1 pp 428ndash451 2017

16 Scientifica

4 Numerical Examples

To illustrate our theoretical results we consider two datasets(i) famous Portland cement data originally adopted byWoods et al [34] and (ii) French economy data from

Chatterjee and Hadi [35] and they are analyzed in thefollowing sections respectively

41 Example 1 Portland Data +ese data are widely knownas the Portland cement dataset It was originally adopted by

OLSRidge

LiuKL

025

035

045

055

MSE

02 04 06 08 1000d = k

(c)

OLSRidge

LiuKL

02 04 06 08 1000d = k

20

30

40

50

MSE

(d)

Figure 1 Estimated MSEs for n 30 Sigma 1 10 rho 070 080 and different values of k d (a) n 30 p 3 sigma 1 and rho 070(b) n 30 p 3 sigma 10 and rho 070 (c) n 30 p 3 sigma 1 and rho 080 (d) n 30 p 3 sigma 10 and rho 080

OLSRidge

LiuKL

04

06

08

10

12

MSE

02 04 06 08 1000d = k

(a)

OLSRidge

LiuKL

20

40

60

80

100M

SE

02 04 06 08 1000d = k

(b)

OLSRidge

LiuKL

02468

101214

MSE

02 04 06 08 1000d = k

(c)

OLSRidge

LiuKL

0200

600

1000

1400

MSE

02 04 06 08 1000d = k

(d)

Figure 2 Estimated MSEs for n 30 sigma 1 10 rho 090 099 and different values of k d (a) n 30 p 3 sigma 1 and rho 090(b) n 30 p 3 sigma 10 and rho 090 (c) n 30 p 3 sigma 1 and rho 099 (d) n 30 p 3 sigma 10 and rho 099

Scientifica 11

OLSRidge

LiuKL

0

10

20

30

MSE

2 4 6 8 100Sigma

(a)

OLSRidge

LiuKL

0

10

30

50

MSE

2 4 6 8 100Sigma

(b)

OLSRidge

LiuKL

0

40

80

120

MSE

2 4 6 8 100Sigma

(c)

OLSRidge

LiuKL

2 4 6 8 100Sigma

0

400

800

1200

MSE

(d)

Figure 3 EstimatedMSEs for n 30 d 05 and different values of rho and sigma (a) n 30 p 3 d 05 and rho 070 (b) n 30 p 3d 05 and rho 080 (c) n 30 p 3 d 05 and rho 090 (d) n 30 p 3 d 05 and rho 099

OLSRidge

LiuKL

50 70 9030n

010

020

030

040

MSE

(a)

OLSRidge

LiuKL

50 70 9030n

02

04

06

MSE

(b)

Figure 4 Continued

12 Scientifica

Woods et al [34] It has also been analyzed by the followingauthors Kaciranlar et al [36] Li and Yang [25] and recentlyby Lukman et al [13] +e regression model for these data isdefined as

yi β0 + β1X1 + β2X2 + β3X3 + β4X4 + εi (46)

where yi heat evolved after 180 days of curing measured incalories per gram of cement X1 tricalcium aluminateX2 tricalcium silicate X3 tetracalcium aluminoferriteand X4 β-dicalcium silicate +e correlation matrix of thepredictor variables is given in Table 9

OLSRidge

LiuKL

50 70 9030n

02

06

10

MSE

(c)

OLSRidge

LiuKL

50 70 9030n

0

2

4

6

8

12

MSE

(d)

Figure 4 Estimated MSEs for sigma 1 p 3 and different values of rho and sample size (a)p 3 sigma 1 d 05 and rho 070(b)p 3 sigma 1 d 05 and rho 080 (c)p 3 sigma 1 d 05 and rho 090 (d)p 3 sigma 1 d 05 and rho 099

OLSRidge

LiuKL

20

30

40

MSE

4 5 6 7 83p

(a)

OLSRidge

LiuKL

3

4

5

6

7M

SE

4 5 6 7 83p

(b)

6

8

10

14

MSE

OLSRidge

LiuKL

4 5 6 7 83p

(c)

OLSRidge

LiuKL

4 5 6 7 83p

0

50

100

150

MSE

(d)

Figure 5 Estimated MSEs for n 100 d 05 sigma 5 and different values of rho and p (a) n 100 sigma 5 d 05 and rho 070 (b)n 100 sigma 5 d 05 and rho 080 (c) n 100 sigma 5 d 05 and rho 090 (d) n 100 sigma 5 d 05 and rho 099

Scientifica 13

+e variance inflation factors are VIF1 = 3850VIF2 = 25442 VIF3 = 4687 and VIF4 = 28251 Eigen-values of XprimeX are λ1 44676206 λ2 5965422

λ3 809952 and λ4 105419 and the condition numberof XprimeX is approximately 424 +e VIFs the eigenvalues

and the condition number all indicate the presence ofsevere multicollinearity +e estimated parameters andMSE are presented in Table 10 It appears from Table 11that the proposed estimator performed the best in thesense of smaller MSE

OLSRidge

LiuKL

0

100

200

300

MSE

075 085 095065Rho

(a)

OLSRidge

LiuKL

0

200

400

600

800

MSE

075 085 095065Rho

(b)

OLSRidge

LiuKL

0

20

40

60

80

MSE

075 085 095065Rho

(c)

OLSRidge

LiuKL

0

50

100

150

MSE

075 085 095065Rho

(d)

Figure 6 Estimated MSEs for n 100 p 3 7 d 05 sigma 5 and different values of rho (a) n 30 p 3 sigma 5 and d 05 (b)n 30 p 7 sigma 5 and d 05 (c) n 100 p 3 sigma 5 and d 05 (d) n 100 p 7 sigma 5 and d 05

Table 9 Correlation matrix

X1 X2 X3 X4

X1 1000 0229 minus 0824 minus 0245X2 0229 1000 minus 0139 minus 0973X3 minus 0824 minus 0139 1000 0030X4 minus 0245 minus 0973 0030 1000

Table 10 +e results of regression coefficients and the corresponding MSE values

Coef 1113954α 1113954α(k) 1113954α(d)1113954dalt 1113954αKL(kHNM) 1113954αKL(kmin)

α0 624054 85870 276490 minus 197876 276068α1 15511 21046 19010 23965 19090α2 05102 10648 08701 13573 08688α3 01019 06681 04621 09666 04680α4 minus 01441 03996 02082 06862 02074MSE 491209 298983 2170963 7255603 217096kd mdash 00077 044195 000235 000047

14 Scientifica

42 Example 2 French Economy Data +e French economydata in Chatterjee and Hadi [37] are considered in this ex-ample It has been analyzed by Malinvard [38] and Liu [6]among others+e variables are imports domestic productionstock formation and domestic consumption All are measuredin milliards of French francs for the years 1949 through 1966

+e regression model for these data is defined as

yi β0 + β1X1 + β2X2 + β3X3 + εi (47)

where yi IMPORT X1 domestic production X2 stockformation and X3 domestic consumption +e correlationmatrix of the predicted variable is given in Table 12

+e variance inflation factors areVIF1 469688VIF2 1047 and VIF3 469338 +e ei-genvalues of the XprimeX matrix are λ1 161779 λ2 158 andλ3 4961 and the condition number is 32612 If we reviewthe above correlation matrix VIFs and condition number itcan be said that there is presence of severe multicollinearityexisting in the predictor variables

+e biasing parameter for the new estimator is defined in(41) and (42) +e biasing parameter for the ridge and Liuestimator is provided in (6) (8) and (9) respectively

We analyzed the data using the biasing parameters foreach of the estimators and presented the results in Tables 10and 11 It can be seen from Tables 10 and 11 that theproposed estimator performed the best in the sense ofsmaller MSE

5 Summary and Concluding Remarks

In this paper we introduced a new biased estimator toovercome the multicollinearity problem for the multiplelinear regression model and provided the estimation tech-nique of the biasing parameter A simulation study has beenconducted to compare the performance of the proposedestimator and Liu [6] and ridge regression estimators [3]Simulation results evidently show that the proposed esti-mator performed better than both Liu and ridge under somecondition on the shrinkage parameter Two sets of real-lifedata are analyzed to illustrate the benefits of using the newestimator in the context of a linear regression model +eproposed estimator is recommended for researchers in this

area Its application can be extended to other regressionmodels for example logistic regression Poisson ZIP andrelated models and those possibilities are under currentinvestigation [37 39 40]

Data Availability

Data will be made available on request

Conflicts of Interest

+e authors declare that they have no conflicts of interest

Acknowledgments

We are dedicating this article to those who lost their livesbecause of COVID-19

References

[1] C Stein ldquoInadmissibility of the usual estimator for mean ofmultivariate normal distributionrdquo in Proceedings of the 0irdBerkley Symposium on Mathematical and Statistics Proba-bility J Neyman Ed vol 1 pp 197ndash206 Springer BerlinGermany 1956

[2] W F Massy ldquoPrincipal components regression in exploratorystatistical researchrdquo Journal of the American Statistical As-sociation vol 60 no 309 pp 234ndash256 1965

[3] A E Hoerl and R W Kennard ldquoRidge regression biasedestimation for nonorthogonal problemsrdquo Technometricsvol 12 no 1 pp 55ndash67 1970

[4] L S Mayer and T A Willke ldquoOn biased estimation in linearmodelsrdquo Technometrics vol 15 no 3 pp 497ndash508 1973

[5] B F Swindel ldquoGood ridge estimators based on prior infor-mationrdquo Communications in Statistics-0eory and Methodsvol 5 no 11 pp 1065ndash1075 1976

[6] K Liu ldquoA new class of biased estimate in linear regressionrdquoCommunication in Statistics- 0eory and Methods vol 22pp 393ndash402 1993

[7] F Akdeniz and S Kaccediliranlar ldquoOn the almost unbiasedgeneralized liu estimator and unbiased estimation of the biasand mserdquo Communications in Statistics-0eory and Methodsvol 24 no 7 pp 1789ndash1797 1995

[8] M R Ozkale and S Kaccediliranlar ldquo+e restricted and unre-stricted two-parameter estimatorsrdquo Communications in Sta-tistics-0eory and Methods vol 36 no 15 pp 2707ndash27252007

[9] S Sakallıoglu and S Kaccedilıranlar ldquoA new biased estimatorbased on ridge estimationrdquo Statistical Papers vol 49 no 4pp 669ndash689 2008

Table 11 +e results of regression coefficients and the corresponding MSE values

Coef 1113954α 1113954α(k) 1113954α(d)1113954dalt 1113954α(d)1113954dopt 1113954αKL(kHNM) 1113954αKL(kmin)

α0 minus 197127 minus 167613 minus 125762 minus 188410 minus 165855 minus 188782α1 00327 01419 02951 00648 01485 00636α2 04059 03576 02875 03914 03548 03922α3 02421 00709 minus 01696 01918 00606 01937MSE 173326 2130519 5828312 1660293 2211899 1660168kd mdash 00527 05282 09423 00258 00065

Table 12 Correlation matrix

X1 X2 X3

X1 1000 0210 0999X2 0210 1000 0208X3 0999 0208 1000

Scientifica 15

[10] H Yang and X Chang ldquoA new two-parameter estimator inlinear regressionrdquo Communications in Statistics-0eory andMethods vol 39 no 6 pp 923ndash934 2010

[11] M Roozbeh ldquoOptimal QR-based estimation in partially linearregression models with correlated errors using GCV crite-rionrdquo Computational Statistics amp Data Analysis vol 117pp 45ndash61 2018

[12] F Akdeniz and M Roozbeh ldquoGeneralized difference-basedweightedmixed almost unbiased ridge estimator in partially linearmodelsrdquo Statistical Papers vol 60 no 5 pp 1717ndash1739 2019

[13] A F Lukman K Ayinde S Binuomote and O A ClementldquoModified ridge-type estimator to combat multicollinearityapplication to chemical datardquo Journal of Chemometricsvol 33 no 5 p e3125 2019

[14] A F Lukman K Ayinde S K Sek and E Adewuyi ldquoAmodified new two-parameter estimator in a linear regressionmodelrdquo Modelling and Simulation in Engineering vol 2019Article ID 6342702 10 pages 2019

[15] A E Hoerl R W Kannard and K F Baldwin ldquoRidge re-gressionsome simulationsrdquo Communications in Statisticsvol 4 no 2 pp 105ndash123 1975

[16] G C McDonald and D I Galarneau ldquoA monte carlo eval-uation of some ridge-type estimatorsrdquo Journal of the Amer-ican Statistical Association vol 70 no 350 pp 407ndash416 1975

[17] J F Lawless and P Wang ldquoA simulation study of ridge andother regression estimatorsrdquo Communications in Statistics-0eory and Methods vol 5 no 4 pp 307ndash323 1976

[18] D W Wichern and G A Churchill ldquoA comparison of ridgeestimatorsrdquo Technometrics vol 20 no 3 pp 301ndash311 1978

[19] B M G Kibria ldquoPerformance of some new ridge regressionestimatorsrdquo Communications in Statistics-Simulation andComputation vol 32 no 1 pp 419ndash435 2003

[20] A F Lukman and K Ayinde ldquoReview and classifications ofthe ridge parameter estimation techniquesrdquoHacettepe Journalof Mathematics and Statistics vol 46 no 5 pp 953ndash967 2017

[21] A K M E Saleh M Arashi and B M G Kibria 0eory ofRidge Regression Estimation with Applications WileyHoboken NJ USA 2019

[22] K Liu ldquoUsing Liu-type estimator to combat collinearityrdquoCommunications in Statistics-0eory and Methods vol 32no 5 pp 1009ndash1020 2003

[23] K Alheety and B M G Kibria ldquoOn the Liu and almostunbiased Liu estimators in the presence of multicollinearitywith heteroscedastic or correlated errorsrdquo Surveys in Math-ematics and its Applications vol 4 pp 155ndash167 2009

[24] X-Q Liu ldquoImproved Liu estimator in a linear regressionmodelrdquo Journal of Statistical Planning and Inference vol 141no 1 pp 189ndash196 2011

[25] Y Li and H Yang ldquoA new Liu-type estimator in linear regressionmodelrdquo Statistical Papers vol 53 no 2 pp 427ndash437 2012

[26] B Kan O Alpu and B Yazıcı ldquoRobust ridge and robust Liuestimator for regression based on the LTS estimatorrdquo Journalof Applied Statistics vol 40 no 3 pp 644ndash655 2013

[27] R A Farghali ldquoGeneralized Liu-type estimator for linearregressionrdquo International Journal of Research and Reviews inApplied Sciences vol 38 no 1 pp 52ndash63 2019

[28] S G Wang M X Wu and Z Z Jia Matrix InequalitiesChinese Science Press Beijing China 2nd edition 2006

[29] R W Farebrother ldquoFurther results on the mean square errorof ridge regressionrdquo Journal of the Royal Statistical SocietySeries B (Methodological) vol 38 no 3 pp 248ndash250 1976

[30] G Trenkler and H Toutenburg ldquoMean squared error matrixcomparisons between biased estimators-an overview of recentresultsrdquo Statistical Papers vol 31 no 1 pp 165ndash179 1990

[31] B M G Kibria and S Banik ldquoSome ridge regression esti-mators and their performancesrdquo Journal of Modern AppliedStatistical Methods vol 15 no 1 pp 206ndash238 2016

[32] D G Gibbons ldquoA simulation study of some ridge estimatorsrdquoJournal of the American Statistical Association vol 76no 373 pp 131ndash139 1981

[33] J P Newhouse and S D Oman ldquoAn evaluation of ridgeestimators A report prepared for United States air forceproject RANDrdquo 1971

[34] H Woods H H Steinour and H R Starke ldquoEffect ofcomposition of Portland cement on heat evolved duringhardeningrdquo Industrial amp Engineering Chemistry vol 24no 11 pp 1207ndash1214 1932

[35] S Chatterjee and A S Hadi Regression Analysis by ExampleWiley Hoboken NJ USA 1977

[36] S Kaciranlar S Sakallioglu F Akdeniz G P H Styan andH J Werner ldquoA new biased estimator in linear regression anda detailed analysis of the widely-analysed dataset on portlandcementrdquo Sankhya 0e Indian Journal of Statistics Series Bvol 61 pp 443ndash459 1999

[37] S Chatterjee and A S Haadi Regression Analysis by ExampleWiley Hoboken NJ USA 2006

[38] E Malinvard Statistical Methods of Econometrics North-Holland Publishing Company Amsterdam Netherlands 3rdedition 1980

[39] D N Gujarati Basic Econometrics McGraw-Hill New YorkNY USA 1995

[40] A F Lukman K Ayinde and A S Ajiboye ldquoMonte Carlostudy of some classification-based ridge parameter estima-torsrdquo Journal of Modern Applied Statistical Methods vol 16no 1 pp 428ndash451 2017

16 Scientifica

OLSRidge

LiuKL

0

10

20

30

MSE

2 4 6 8 100Sigma

(a)

OLSRidge

LiuKL

0

10

30

50

MSE

2 4 6 8 100Sigma

(b)

OLSRidge

LiuKL

0

40

80

120

MSE

2 4 6 8 100Sigma

(c)

OLSRidge

LiuKL

2 4 6 8 100Sigma

0

400

800

1200

MSE

(d)

Figure 3 EstimatedMSEs for n 30 d 05 and different values of rho and sigma (a) n 30 p 3 d 05 and rho 070 (b) n 30 p 3d 05 and rho 080 (c) n 30 p 3 d 05 and rho 090 (d) n 30 p 3 d 05 and rho 099

OLSRidge

LiuKL

50 70 9030n

010

020

030

040

MSE

(a)

OLSRidge

LiuKL

50 70 9030n

02

04

06

MSE

(b)

Figure 4 Continued

12 Scientifica

Woods et al [34] It has also been analyzed by the followingauthors Kaciranlar et al [36] Li and Yang [25] and recentlyby Lukman et al [13] +e regression model for these data isdefined as

yi β0 + β1X1 + β2X2 + β3X3 + β4X4 + εi (46)

where yi heat evolved after 180 days of curing measured incalories per gram of cement X1 tricalcium aluminateX2 tricalcium silicate X3 tetracalcium aluminoferriteand X4 β-dicalcium silicate +e correlation matrix of thepredictor variables is given in Table 9

OLSRidge

LiuKL

50 70 9030n

02

06

10

MSE

(c)

OLSRidge

LiuKL

50 70 9030n

0

2

4

6

8

12

MSE

(d)

Figure 4 Estimated MSEs for sigma 1 p 3 and different values of rho and sample size (a)p 3 sigma 1 d 05 and rho 070(b)p 3 sigma 1 d 05 and rho 080 (c)p 3 sigma 1 d 05 and rho 090 (d)p 3 sigma 1 d 05 and rho 099

OLSRidge

LiuKL

20

30

40

MSE

4 5 6 7 83p

(a)

OLSRidge

LiuKL

3

4

5

6

7M

SE

4 5 6 7 83p

(b)

6

8

10

14

MSE

OLSRidge

LiuKL

4 5 6 7 83p

(c)

OLSRidge

LiuKL

4 5 6 7 83p

0

50

100

150

MSE

(d)

Figure 5 Estimated MSEs for n 100 d 05 sigma 5 and different values of rho and p (a) n 100 sigma 5 d 05 and rho 070 (b)n 100 sigma 5 d 05 and rho 080 (c) n 100 sigma 5 d 05 and rho 090 (d) n 100 sigma 5 d 05 and rho 099

Scientifica 13

+e variance inflation factors are VIF1 = 3850VIF2 = 25442 VIF3 = 4687 and VIF4 = 28251 Eigen-values of XprimeX are λ1 44676206 λ2 5965422

λ3 809952 and λ4 105419 and the condition numberof XprimeX is approximately 424 +e VIFs the eigenvalues

and the condition number all indicate the presence ofsevere multicollinearity +e estimated parameters andMSE are presented in Table 10 It appears from Table 11that the proposed estimator performed the best in thesense of smaller MSE

OLSRidge

LiuKL

0

100

200

300

MSE

075 085 095065Rho

(a)

OLSRidge

LiuKL

0

200

400

600

800

MSE

075 085 095065Rho

(b)

OLSRidge

LiuKL

0

20

40

60

80

MSE

075 085 095065Rho

(c)

OLSRidge

LiuKL

0

50

100

150

MSE

075 085 095065Rho

(d)

Figure 6 Estimated MSEs for n 100 p 3 7 d 05 sigma 5 and different values of rho (a) n 30 p 3 sigma 5 and d 05 (b)n 30 p 7 sigma 5 and d 05 (c) n 100 p 3 sigma 5 and d 05 (d) n 100 p 7 sigma 5 and d 05

Table 9 Correlation matrix

X1 X2 X3 X4

X1 1000 0229 minus 0824 minus 0245X2 0229 1000 minus 0139 minus 0973X3 minus 0824 minus 0139 1000 0030X4 minus 0245 minus 0973 0030 1000

Table 10 +e results of regression coefficients and the corresponding MSE values

Coef 1113954α 1113954α(k) 1113954α(d)1113954dalt 1113954αKL(kHNM) 1113954αKL(kmin)

α0 624054 85870 276490 minus 197876 276068α1 15511 21046 19010 23965 19090α2 05102 10648 08701 13573 08688α3 01019 06681 04621 09666 04680α4 minus 01441 03996 02082 06862 02074MSE 491209 298983 2170963 7255603 217096kd mdash 00077 044195 000235 000047

14 Scientifica

42 Example 2 French Economy Data +e French economydata in Chatterjee and Hadi [37] are considered in this ex-ample It has been analyzed by Malinvard [38] and Liu [6]among others+e variables are imports domestic productionstock formation and domestic consumption All are measuredin milliards of French francs for the years 1949 through 1966

+e regression model for these data is defined as

yi β0 + β1X1 + β2X2 + β3X3 + εi (47)

where yi IMPORT X1 domestic production X2 stockformation and X3 domestic consumption +e correlationmatrix of the predicted variable is given in Table 12

+e variance inflation factors areVIF1 469688VIF2 1047 and VIF3 469338 +e ei-genvalues of the XprimeX matrix are λ1 161779 λ2 158 andλ3 4961 and the condition number is 32612 If we reviewthe above correlation matrix VIFs and condition number itcan be said that there is presence of severe multicollinearityexisting in the predictor variables

+e biasing parameter for the new estimator is defined in(41) and (42) +e biasing parameter for the ridge and Liuestimator is provided in (6) (8) and (9) respectively

We analyzed the data using the biasing parameters foreach of the estimators and presented the results in Tables 10and 11 It can be seen from Tables 10 and 11 that theproposed estimator performed the best in the sense ofsmaller MSE

5 Summary and Concluding Remarks

In this paper we introduced a new biased estimator toovercome the multicollinearity problem for the multiplelinear regression model and provided the estimation tech-nique of the biasing parameter A simulation study has beenconducted to compare the performance of the proposedestimator and Liu [6] and ridge regression estimators [3]Simulation results evidently show that the proposed esti-mator performed better than both Liu and ridge under somecondition on the shrinkage parameter Two sets of real-lifedata are analyzed to illustrate the benefits of using the newestimator in the context of a linear regression model +eproposed estimator is recommended for researchers in this

area Its application can be extended to other regressionmodels for example logistic regression Poisson ZIP andrelated models and those possibilities are under currentinvestigation [37 39 40]

Data Availability

Data will be made available on request

Conflicts of Interest

+e authors declare that they have no conflicts of interest

Acknowledgments

We are dedicating this article to those who lost their livesbecause of COVID-19

References

[1] C Stein ldquoInadmissibility of the usual estimator for mean ofmultivariate normal distributionrdquo in Proceedings of the 0irdBerkley Symposium on Mathematical and Statistics Proba-bility J Neyman Ed vol 1 pp 197ndash206 Springer BerlinGermany 1956

[2] W F Massy ldquoPrincipal components regression in exploratorystatistical researchrdquo Journal of the American Statistical As-sociation vol 60 no 309 pp 234ndash256 1965

[3] A E Hoerl and R W Kennard ldquoRidge regression biasedestimation for nonorthogonal problemsrdquo Technometricsvol 12 no 1 pp 55ndash67 1970

[4] L S Mayer and T A Willke ldquoOn biased estimation in linearmodelsrdquo Technometrics vol 15 no 3 pp 497ndash508 1973

[5] B F Swindel ldquoGood ridge estimators based on prior infor-mationrdquo Communications in Statistics-0eory and Methodsvol 5 no 11 pp 1065ndash1075 1976

[6] K Liu ldquoA new class of biased estimate in linear regressionrdquoCommunication in Statistics- 0eory and Methods vol 22pp 393ndash402 1993

[7] F Akdeniz and S Kaccediliranlar ldquoOn the almost unbiasedgeneralized liu estimator and unbiased estimation of the biasand mserdquo Communications in Statistics-0eory and Methodsvol 24 no 7 pp 1789ndash1797 1995

[8] M R Ozkale and S Kaccediliranlar ldquo+e restricted and unre-stricted two-parameter estimatorsrdquo Communications in Sta-tistics-0eory and Methods vol 36 no 15 pp 2707ndash27252007

[9] S Sakallıoglu and S Kaccedilıranlar ldquoA new biased estimatorbased on ridge estimationrdquo Statistical Papers vol 49 no 4pp 669ndash689 2008

Table 11 +e results of regression coefficients and the corresponding MSE values

Coef 1113954α 1113954α(k) 1113954α(d)1113954dalt 1113954α(d)1113954dopt 1113954αKL(kHNM) 1113954αKL(kmin)

α0 minus 197127 minus 167613 minus 125762 minus 188410 minus 165855 minus 188782α1 00327 01419 02951 00648 01485 00636α2 04059 03576 02875 03914 03548 03922α3 02421 00709 minus 01696 01918 00606 01937MSE 173326 2130519 5828312 1660293 2211899 1660168kd mdash 00527 05282 09423 00258 00065

Table 12 Correlation matrix

X1 X2 X3

X1 1000 0210 0999X2 0210 1000 0208X3 0999 0208 1000

Scientifica 15

[10] H Yang and X Chang ldquoA new two-parameter estimator inlinear regressionrdquo Communications in Statistics-0eory andMethods vol 39 no 6 pp 923ndash934 2010

[11] M Roozbeh ldquoOptimal QR-based estimation in partially linearregression models with correlated errors using GCV crite-rionrdquo Computational Statistics amp Data Analysis vol 117pp 45ndash61 2018

[12] F Akdeniz and M Roozbeh ldquoGeneralized difference-basedweightedmixed almost unbiased ridge estimator in partially linearmodelsrdquo Statistical Papers vol 60 no 5 pp 1717ndash1739 2019

[13] A F Lukman K Ayinde S Binuomote and O A ClementldquoModified ridge-type estimator to combat multicollinearityapplication to chemical datardquo Journal of Chemometricsvol 33 no 5 p e3125 2019

[14] A F Lukman K Ayinde S K Sek and E Adewuyi ldquoAmodified new two-parameter estimator in a linear regressionmodelrdquo Modelling and Simulation in Engineering vol 2019Article ID 6342702 10 pages 2019

[15] A E Hoerl R W Kannard and K F Baldwin ldquoRidge re-gressionsome simulationsrdquo Communications in Statisticsvol 4 no 2 pp 105ndash123 1975

[16] G C McDonald and D I Galarneau ldquoA monte carlo eval-uation of some ridge-type estimatorsrdquo Journal of the Amer-ican Statistical Association vol 70 no 350 pp 407ndash416 1975

[17] J F Lawless and P Wang ldquoA simulation study of ridge andother regression estimatorsrdquo Communications in Statistics-0eory and Methods vol 5 no 4 pp 307ndash323 1976

[18] D W Wichern and G A Churchill ldquoA comparison of ridgeestimatorsrdquo Technometrics vol 20 no 3 pp 301ndash311 1978

[19] B M G Kibria ldquoPerformance of some new ridge regressionestimatorsrdquo Communications in Statistics-Simulation andComputation vol 32 no 1 pp 419ndash435 2003

[20] A F Lukman and K Ayinde ldquoReview and classifications ofthe ridge parameter estimation techniquesrdquoHacettepe Journalof Mathematics and Statistics vol 46 no 5 pp 953ndash967 2017

[21] A K M E Saleh M Arashi and B M G Kibria 0eory ofRidge Regression Estimation with Applications WileyHoboken NJ USA 2019

[22] K Liu ldquoUsing Liu-type estimator to combat collinearityrdquoCommunications in Statistics-0eory and Methods vol 32no 5 pp 1009ndash1020 2003

[23] K Alheety and B M G Kibria ldquoOn the Liu and almostunbiased Liu estimators in the presence of multicollinearitywith heteroscedastic or correlated errorsrdquo Surveys in Math-ematics and its Applications vol 4 pp 155ndash167 2009

[24] X-Q Liu ldquoImproved Liu estimator in a linear regressionmodelrdquo Journal of Statistical Planning and Inference vol 141no 1 pp 189ndash196 2011

[25] Y Li and H Yang ldquoA new Liu-type estimator in linear regressionmodelrdquo Statistical Papers vol 53 no 2 pp 427ndash437 2012

[26] B Kan O Alpu and B Yazıcı ldquoRobust ridge and robust Liuestimator for regression based on the LTS estimatorrdquo Journalof Applied Statistics vol 40 no 3 pp 644ndash655 2013

[27] R A Farghali ldquoGeneralized Liu-type estimator for linearregressionrdquo International Journal of Research and Reviews inApplied Sciences vol 38 no 1 pp 52ndash63 2019

[28] S G Wang M X Wu and Z Z Jia Matrix InequalitiesChinese Science Press Beijing China 2nd edition 2006

[29] R W Farebrother ldquoFurther results on the mean square errorof ridge regressionrdquo Journal of the Royal Statistical SocietySeries B (Methodological) vol 38 no 3 pp 248ndash250 1976

[30] G Trenkler and H Toutenburg ldquoMean squared error matrixcomparisons between biased estimators-an overview of recentresultsrdquo Statistical Papers vol 31 no 1 pp 165ndash179 1990

[31] B M G Kibria and S Banik ldquoSome ridge regression esti-mators and their performancesrdquo Journal of Modern AppliedStatistical Methods vol 15 no 1 pp 206ndash238 2016

[32] D G Gibbons ldquoA simulation study of some ridge estimatorsrdquoJournal of the American Statistical Association vol 76no 373 pp 131ndash139 1981

[33] J P Newhouse and S D Oman ldquoAn evaluation of ridgeestimators A report prepared for United States air forceproject RANDrdquo 1971

[34] H Woods H H Steinour and H R Starke ldquoEffect ofcomposition of Portland cement on heat evolved duringhardeningrdquo Industrial amp Engineering Chemistry vol 24no 11 pp 1207ndash1214 1932

[35] S Chatterjee and A S Hadi Regression Analysis by ExampleWiley Hoboken NJ USA 1977

[36] S Kaciranlar S Sakallioglu F Akdeniz G P H Styan andH J Werner ldquoA new biased estimator in linear regression anda detailed analysis of the widely-analysed dataset on portlandcementrdquo Sankhya 0e Indian Journal of Statistics Series Bvol 61 pp 443ndash459 1999

[37] S Chatterjee and A S Haadi Regression Analysis by ExampleWiley Hoboken NJ USA 2006

[38] E Malinvard Statistical Methods of Econometrics North-Holland Publishing Company Amsterdam Netherlands 3rdedition 1980

[39] D N Gujarati Basic Econometrics McGraw-Hill New YorkNY USA 1995

[40] A F Lukman K Ayinde and A S Ajiboye ldquoMonte Carlostudy of some classification-based ridge parameter estima-torsrdquo Journal of Modern Applied Statistical Methods vol 16no 1 pp 428ndash451 2017

16 Scientifica

Woods et al [34] It has also been analyzed by the followingauthors Kaciranlar et al [36] Li and Yang [25] and recentlyby Lukman et al [13] +e regression model for these data isdefined as

yi β0 + β1X1 + β2X2 + β3X3 + β4X4 + εi (46)

where yi heat evolved after 180 days of curing measured incalories per gram of cement X1 tricalcium aluminateX2 tricalcium silicate X3 tetracalcium aluminoferriteand X4 β-dicalcium silicate +e correlation matrix of thepredictor variables is given in Table 9

OLSRidge

LiuKL

50 70 9030n

02

06

10

MSE

(c)

OLSRidge

LiuKL

50 70 9030n

0

2

4

6

8

12

MSE

(d)

Figure 4 Estimated MSEs for sigma 1 p 3 and different values of rho and sample size (a)p 3 sigma 1 d 05 and rho 070(b)p 3 sigma 1 d 05 and rho 080 (c)p 3 sigma 1 d 05 and rho 090 (d)p 3 sigma 1 d 05 and rho 099

OLSRidge

LiuKL

20

30

40

MSE

4 5 6 7 83p

(a)

OLSRidge

LiuKL

3

4

5

6

7M

SE

4 5 6 7 83p

(b)

6

8

10

14

MSE

OLSRidge

LiuKL

4 5 6 7 83p

(c)

OLSRidge

LiuKL

4 5 6 7 83p

0

50

100

150

MSE

(d)

Figure 5 Estimated MSEs for n 100 d 05 sigma 5 and different values of rho and p (a) n 100 sigma 5 d 05 and rho 070 (b)n 100 sigma 5 d 05 and rho 080 (c) n 100 sigma 5 d 05 and rho 090 (d) n 100 sigma 5 d 05 and rho 099

Scientifica 13

+e variance inflation factors are VIF1 = 3850VIF2 = 25442 VIF3 = 4687 and VIF4 = 28251 Eigen-values of XprimeX are λ1 44676206 λ2 5965422

λ3 809952 and λ4 105419 and the condition numberof XprimeX is approximately 424 +e VIFs the eigenvalues

and the condition number all indicate the presence ofsevere multicollinearity +e estimated parameters andMSE are presented in Table 10 It appears from Table 11that the proposed estimator performed the best in thesense of smaller MSE

OLSRidge

LiuKL

0

100

200

300

MSE

075 085 095065Rho

(a)

OLSRidge

LiuKL

0

200

400

600

800

MSE

075 085 095065Rho

(b)

OLSRidge

LiuKL

0

20

40

60

80

MSE

075 085 095065Rho

(c)

OLSRidge

LiuKL

0

50

100

150

MSE

075 085 095065Rho

(d)

Figure 6 Estimated MSEs for n 100 p 3 7 d 05 sigma 5 and different values of rho (a) n 30 p 3 sigma 5 and d 05 (b)n 30 p 7 sigma 5 and d 05 (c) n 100 p 3 sigma 5 and d 05 (d) n 100 p 7 sigma 5 and d 05

Table 9 Correlation matrix

X1 X2 X3 X4

X1 1000 0229 minus 0824 minus 0245X2 0229 1000 minus 0139 minus 0973X3 minus 0824 minus 0139 1000 0030X4 minus 0245 minus 0973 0030 1000

Table 10 +e results of regression coefficients and the corresponding MSE values

Coef 1113954α 1113954α(k) 1113954α(d)1113954dalt 1113954αKL(kHNM) 1113954αKL(kmin)

α0 624054 85870 276490 minus 197876 276068α1 15511 21046 19010 23965 19090α2 05102 10648 08701 13573 08688α3 01019 06681 04621 09666 04680α4 minus 01441 03996 02082 06862 02074MSE 491209 298983 2170963 7255603 217096kd mdash 00077 044195 000235 000047

14 Scientifica

42 Example 2 French Economy Data +e French economydata in Chatterjee and Hadi [37] are considered in this ex-ample It has been analyzed by Malinvard [38] and Liu [6]among others+e variables are imports domestic productionstock formation and domestic consumption All are measuredin milliards of French francs for the years 1949 through 1966

+e regression model for these data is defined as

yi β0 + β1X1 + β2X2 + β3X3 + εi (47)

where yi IMPORT X1 domestic production X2 stockformation and X3 domestic consumption +e correlationmatrix of the predicted variable is given in Table 12

+e variance inflation factors areVIF1 469688VIF2 1047 and VIF3 469338 +e ei-genvalues of the XprimeX matrix are λ1 161779 λ2 158 andλ3 4961 and the condition number is 32612 If we reviewthe above correlation matrix VIFs and condition number itcan be said that there is presence of severe multicollinearityexisting in the predictor variables

+e biasing parameter for the new estimator is defined in(41) and (42) +e biasing parameter for the ridge and Liuestimator is provided in (6) (8) and (9) respectively

We analyzed the data using the biasing parameters foreach of the estimators and presented the results in Tables 10and 11 It can be seen from Tables 10 and 11 that theproposed estimator performed the best in the sense ofsmaller MSE

5 Summary and Concluding Remarks

In this paper we introduced a new biased estimator toovercome the multicollinearity problem for the multiplelinear regression model and provided the estimation tech-nique of the biasing parameter A simulation study has beenconducted to compare the performance of the proposedestimator and Liu [6] and ridge regression estimators [3]Simulation results evidently show that the proposed esti-mator performed better than both Liu and ridge under somecondition on the shrinkage parameter Two sets of real-lifedata are analyzed to illustrate the benefits of using the newestimator in the context of a linear regression model +eproposed estimator is recommended for researchers in this

area Its application can be extended to other regressionmodels for example logistic regression Poisson ZIP andrelated models and those possibilities are under currentinvestigation [37 39 40]

Data Availability

Data will be made available on request

Conflicts of Interest

+e authors declare that they have no conflicts of interest

Acknowledgments

We are dedicating this article to those who lost their livesbecause of COVID-19

References

[1] C Stein ldquoInadmissibility of the usual estimator for mean ofmultivariate normal distributionrdquo in Proceedings of the 0irdBerkley Symposium on Mathematical and Statistics Proba-bility J Neyman Ed vol 1 pp 197ndash206 Springer BerlinGermany 1956

[2] W F Massy ldquoPrincipal components regression in exploratorystatistical researchrdquo Journal of the American Statistical As-sociation vol 60 no 309 pp 234ndash256 1965

[3] A E Hoerl and R W Kennard ldquoRidge regression biasedestimation for nonorthogonal problemsrdquo Technometricsvol 12 no 1 pp 55ndash67 1970

[4] L S Mayer and T A Willke ldquoOn biased estimation in linearmodelsrdquo Technometrics vol 15 no 3 pp 497ndash508 1973

[5] B F Swindel ldquoGood ridge estimators based on prior infor-mationrdquo Communications in Statistics-0eory and Methodsvol 5 no 11 pp 1065ndash1075 1976

[6] K Liu ldquoA new class of biased estimate in linear regressionrdquoCommunication in Statistics- 0eory and Methods vol 22pp 393ndash402 1993

[7] F Akdeniz and S Kaccediliranlar ldquoOn the almost unbiasedgeneralized liu estimator and unbiased estimation of the biasand mserdquo Communications in Statistics-0eory and Methodsvol 24 no 7 pp 1789ndash1797 1995

[8] M R Ozkale and S Kaccediliranlar ldquo+e restricted and unre-stricted two-parameter estimatorsrdquo Communications in Sta-tistics-0eory and Methods vol 36 no 15 pp 2707ndash27252007

[9] S Sakallıoglu and S Kaccedilıranlar ldquoA new biased estimatorbased on ridge estimationrdquo Statistical Papers vol 49 no 4pp 669ndash689 2008

Table 11 +e results of regression coefficients and the corresponding MSE values

Coef 1113954α 1113954α(k) 1113954α(d)1113954dalt 1113954α(d)1113954dopt 1113954αKL(kHNM) 1113954αKL(kmin)

α0 minus 197127 minus 167613 minus 125762 minus 188410 minus 165855 minus 188782α1 00327 01419 02951 00648 01485 00636α2 04059 03576 02875 03914 03548 03922α3 02421 00709 minus 01696 01918 00606 01937MSE 173326 2130519 5828312 1660293 2211899 1660168kd mdash 00527 05282 09423 00258 00065

Table 12 Correlation matrix

X1 X2 X3

X1 1000 0210 0999X2 0210 1000 0208X3 0999 0208 1000

Scientifica 15

[10] H Yang and X Chang ldquoA new two-parameter estimator inlinear regressionrdquo Communications in Statistics-0eory andMethods vol 39 no 6 pp 923ndash934 2010

[11] M Roozbeh ldquoOptimal QR-based estimation in partially linearregression models with correlated errors using GCV crite-rionrdquo Computational Statistics amp Data Analysis vol 117pp 45ndash61 2018

[12] F Akdeniz and M Roozbeh ldquoGeneralized difference-basedweightedmixed almost unbiased ridge estimator in partially linearmodelsrdquo Statistical Papers vol 60 no 5 pp 1717ndash1739 2019

[13] A F Lukman K Ayinde S Binuomote and O A ClementldquoModified ridge-type estimator to combat multicollinearityapplication to chemical datardquo Journal of Chemometricsvol 33 no 5 p e3125 2019

[14] A F Lukman K Ayinde S K Sek and E Adewuyi ldquoAmodified new two-parameter estimator in a linear regressionmodelrdquo Modelling and Simulation in Engineering vol 2019Article ID 6342702 10 pages 2019

[15] A E Hoerl R W Kannard and K F Baldwin ldquoRidge re-gressionsome simulationsrdquo Communications in Statisticsvol 4 no 2 pp 105ndash123 1975

[16] G C McDonald and D I Galarneau ldquoA monte carlo eval-uation of some ridge-type estimatorsrdquo Journal of the Amer-ican Statistical Association vol 70 no 350 pp 407ndash416 1975

[17] J F Lawless and P Wang ldquoA simulation study of ridge andother regression estimatorsrdquo Communications in Statistics-0eory and Methods vol 5 no 4 pp 307ndash323 1976

[18] D W Wichern and G A Churchill ldquoA comparison of ridgeestimatorsrdquo Technometrics vol 20 no 3 pp 301ndash311 1978

[19] B M G Kibria ldquoPerformance of some new ridge regressionestimatorsrdquo Communications in Statistics-Simulation andComputation vol 32 no 1 pp 419ndash435 2003

[20] A F Lukman and K Ayinde ldquoReview and classifications ofthe ridge parameter estimation techniquesrdquoHacettepe Journalof Mathematics and Statistics vol 46 no 5 pp 953ndash967 2017

[21] A K M E Saleh M Arashi and B M G Kibria 0eory ofRidge Regression Estimation with Applications WileyHoboken NJ USA 2019

[22] K Liu ldquoUsing Liu-type estimator to combat collinearityrdquoCommunications in Statistics-0eory and Methods vol 32no 5 pp 1009ndash1020 2003

[23] K Alheety and B M G Kibria ldquoOn the Liu and almostunbiased Liu estimators in the presence of multicollinearitywith heteroscedastic or correlated errorsrdquo Surveys in Math-ematics and its Applications vol 4 pp 155ndash167 2009

[24] X-Q Liu ldquoImproved Liu estimator in a linear regressionmodelrdquo Journal of Statistical Planning and Inference vol 141no 1 pp 189ndash196 2011

[25] Y Li and H Yang ldquoA new Liu-type estimator in linear regressionmodelrdquo Statistical Papers vol 53 no 2 pp 427ndash437 2012

[26] B Kan O Alpu and B Yazıcı ldquoRobust ridge and robust Liuestimator for regression based on the LTS estimatorrdquo Journalof Applied Statistics vol 40 no 3 pp 644ndash655 2013

[27] R A Farghali ldquoGeneralized Liu-type estimator for linearregressionrdquo International Journal of Research and Reviews inApplied Sciences vol 38 no 1 pp 52ndash63 2019

[28] S G Wang M X Wu and Z Z Jia Matrix InequalitiesChinese Science Press Beijing China 2nd edition 2006

[29] R W Farebrother ldquoFurther results on the mean square errorof ridge regressionrdquo Journal of the Royal Statistical SocietySeries B (Methodological) vol 38 no 3 pp 248ndash250 1976

[30] G Trenkler and H Toutenburg ldquoMean squared error matrixcomparisons between biased estimators-an overview of recentresultsrdquo Statistical Papers vol 31 no 1 pp 165ndash179 1990

[31] B M G Kibria and S Banik ldquoSome ridge regression esti-mators and their performancesrdquo Journal of Modern AppliedStatistical Methods vol 15 no 1 pp 206ndash238 2016

[32] D G Gibbons ldquoA simulation study of some ridge estimatorsrdquoJournal of the American Statistical Association vol 76no 373 pp 131ndash139 1981

[33] J P Newhouse and S D Oman ldquoAn evaluation of ridgeestimators A report prepared for United States air forceproject RANDrdquo 1971

[34] H Woods H H Steinour and H R Starke ldquoEffect ofcomposition of Portland cement on heat evolved duringhardeningrdquo Industrial amp Engineering Chemistry vol 24no 11 pp 1207ndash1214 1932

[35] S Chatterjee and A S Hadi Regression Analysis by ExampleWiley Hoboken NJ USA 1977

[36] S Kaciranlar S Sakallioglu F Akdeniz G P H Styan andH J Werner ldquoA new biased estimator in linear regression anda detailed analysis of the widely-analysed dataset on portlandcementrdquo Sankhya 0e Indian Journal of Statistics Series Bvol 61 pp 443ndash459 1999

[37] S Chatterjee and A S Haadi Regression Analysis by ExampleWiley Hoboken NJ USA 2006

[38] E Malinvard Statistical Methods of Econometrics North-Holland Publishing Company Amsterdam Netherlands 3rdedition 1980

[39] D N Gujarati Basic Econometrics McGraw-Hill New YorkNY USA 1995

[40] A F Lukman K Ayinde and A S Ajiboye ldquoMonte Carlostudy of some classification-based ridge parameter estima-torsrdquo Journal of Modern Applied Statistical Methods vol 16no 1 pp 428ndash451 2017

16 Scientifica

+e variance inflation factors are VIF1 = 3850VIF2 = 25442 VIF3 = 4687 and VIF4 = 28251 Eigen-values of XprimeX are λ1 44676206 λ2 5965422

λ3 809952 and λ4 105419 and the condition numberof XprimeX is approximately 424 +e VIFs the eigenvalues

and the condition number all indicate the presence ofsevere multicollinearity +e estimated parameters andMSE are presented in Table 10 It appears from Table 11that the proposed estimator performed the best in thesense of smaller MSE

OLSRidge

LiuKL

0

100

200

300

MSE

075 085 095065Rho

(a)

OLSRidge

LiuKL

0

200

400

600

800

MSE

075 085 095065Rho

(b)

OLSRidge

LiuKL

0

20

40

60

80

MSE

075 085 095065Rho

(c)

OLSRidge

LiuKL

0

50

100

150

MSE

075 085 095065Rho

(d)

Figure 6 Estimated MSEs for n 100 p 3 7 d 05 sigma 5 and different values of rho (a) n 30 p 3 sigma 5 and d 05 (b)n 30 p 7 sigma 5 and d 05 (c) n 100 p 3 sigma 5 and d 05 (d) n 100 p 7 sigma 5 and d 05

Table 9 Correlation matrix

X1 X2 X3 X4

X1 1000 0229 minus 0824 minus 0245X2 0229 1000 minus 0139 minus 0973X3 minus 0824 minus 0139 1000 0030X4 minus 0245 minus 0973 0030 1000

Table 10 +e results of regression coefficients and the corresponding MSE values

Coef 1113954α 1113954α(k) 1113954α(d)1113954dalt 1113954αKL(kHNM) 1113954αKL(kmin)

α0 624054 85870 276490 minus 197876 276068α1 15511 21046 19010 23965 19090α2 05102 10648 08701 13573 08688α3 01019 06681 04621 09666 04680α4 minus 01441 03996 02082 06862 02074MSE 491209 298983 2170963 7255603 217096kd mdash 00077 044195 000235 000047

14 Scientifica

42 Example 2 French Economy Data +e French economydata in Chatterjee and Hadi [37] are considered in this ex-ample It has been analyzed by Malinvard [38] and Liu [6]among others+e variables are imports domestic productionstock formation and domestic consumption All are measuredin milliards of French francs for the years 1949 through 1966

+e regression model for these data is defined as

yi β0 + β1X1 + β2X2 + β3X3 + εi (47)

where yi IMPORT X1 domestic production X2 stockformation and X3 domestic consumption +e correlationmatrix of the predicted variable is given in Table 12

+e variance inflation factors areVIF1 469688VIF2 1047 and VIF3 469338 +e ei-genvalues of the XprimeX matrix are λ1 161779 λ2 158 andλ3 4961 and the condition number is 32612 If we reviewthe above correlation matrix VIFs and condition number itcan be said that there is presence of severe multicollinearityexisting in the predictor variables

+e biasing parameter for the new estimator is defined in(41) and (42) +e biasing parameter for the ridge and Liuestimator is provided in (6) (8) and (9) respectively

We analyzed the data using the biasing parameters foreach of the estimators and presented the results in Tables 10and 11 It can be seen from Tables 10 and 11 that theproposed estimator performed the best in the sense ofsmaller MSE

5 Summary and Concluding Remarks

In this paper we introduced a new biased estimator toovercome the multicollinearity problem for the multiplelinear regression model and provided the estimation tech-nique of the biasing parameter A simulation study has beenconducted to compare the performance of the proposedestimator and Liu [6] and ridge regression estimators [3]Simulation results evidently show that the proposed esti-mator performed better than both Liu and ridge under somecondition on the shrinkage parameter Two sets of real-lifedata are analyzed to illustrate the benefits of using the newestimator in the context of a linear regression model +eproposed estimator is recommended for researchers in this

area Its application can be extended to other regressionmodels for example logistic regression Poisson ZIP andrelated models and those possibilities are under currentinvestigation [37 39 40]

Data Availability

Data will be made available on request

Conflicts of Interest

+e authors declare that they have no conflicts of interest

Acknowledgments

We are dedicating this article to those who lost their livesbecause of COVID-19

References

[1] C Stein ldquoInadmissibility of the usual estimator for mean ofmultivariate normal distributionrdquo in Proceedings of the 0irdBerkley Symposium on Mathematical and Statistics Proba-bility J Neyman Ed vol 1 pp 197ndash206 Springer BerlinGermany 1956

[2] W F Massy ldquoPrincipal components regression in exploratorystatistical researchrdquo Journal of the American Statistical As-sociation vol 60 no 309 pp 234ndash256 1965

[3] A E Hoerl and R W Kennard ldquoRidge regression biasedestimation for nonorthogonal problemsrdquo Technometricsvol 12 no 1 pp 55ndash67 1970

[4] L S Mayer and T A Willke ldquoOn biased estimation in linearmodelsrdquo Technometrics vol 15 no 3 pp 497ndash508 1973

[5] B F Swindel ldquoGood ridge estimators based on prior infor-mationrdquo Communications in Statistics-0eory and Methodsvol 5 no 11 pp 1065ndash1075 1976

[6] K Liu ldquoA new class of biased estimate in linear regressionrdquoCommunication in Statistics- 0eory and Methods vol 22pp 393ndash402 1993

[7] F Akdeniz and S Kaccediliranlar ldquoOn the almost unbiasedgeneralized liu estimator and unbiased estimation of the biasand mserdquo Communications in Statistics-0eory and Methodsvol 24 no 7 pp 1789ndash1797 1995

[8] M R Ozkale and S Kaccediliranlar ldquo+e restricted and unre-stricted two-parameter estimatorsrdquo Communications in Sta-tistics-0eory and Methods vol 36 no 15 pp 2707ndash27252007

[9] S Sakallıoglu and S Kaccedilıranlar ldquoA new biased estimatorbased on ridge estimationrdquo Statistical Papers vol 49 no 4pp 669ndash689 2008

Table 11 +e results of regression coefficients and the corresponding MSE values

Coef 1113954α 1113954α(k) 1113954α(d)1113954dalt 1113954α(d)1113954dopt 1113954αKL(kHNM) 1113954αKL(kmin)

α0 minus 197127 minus 167613 minus 125762 minus 188410 minus 165855 minus 188782α1 00327 01419 02951 00648 01485 00636α2 04059 03576 02875 03914 03548 03922α3 02421 00709 minus 01696 01918 00606 01937MSE 173326 2130519 5828312 1660293 2211899 1660168kd mdash 00527 05282 09423 00258 00065

Table 12 Correlation matrix

X1 X2 X3

X1 1000 0210 0999X2 0210 1000 0208X3 0999 0208 1000

Scientifica 15

[10] H Yang and X Chang ldquoA new two-parameter estimator inlinear regressionrdquo Communications in Statistics-0eory andMethods vol 39 no 6 pp 923ndash934 2010

[11] M Roozbeh ldquoOptimal QR-based estimation in partially linearregression models with correlated errors using GCV crite-rionrdquo Computational Statistics amp Data Analysis vol 117pp 45ndash61 2018

[12] F Akdeniz and M Roozbeh ldquoGeneralized difference-basedweightedmixed almost unbiased ridge estimator in partially linearmodelsrdquo Statistical Papers vol 60 no 5 pp 1717ndash1739 2019

[13] A F Lukman K Ayinde S Binuomote and O A ClementldquoModified ridge-type estimator to combat multicollinearityapplication to chemical datardquo Journal of Chemometricsvol 33 no 5 p e3125 2019

[14] A F Lukman K Ayinde S K Sek and E Adewuyi ldquoAmodified new two-parameter estimator in a linear regressionmodelrdquo Modelling and Simulation in Engineering vol 2019Article ID 6342702 10 pages 2019

[15] A E Hoerl R W Kannard and K F Baldwin ldquoRidge re-gressionsome simulationsrdquo Communications in Statisticsvol 4 no 2 pp 105ndash123 1975

[16] G C McDonald and D I Galarneau ldquoA monte carlo eval-uation of some ridge-type estimatorsrdquo Journal of the Amer-ican Statistical Association vol 70 no 350 pp 407ndash416 1975

[17] J F Lawless and P Wang ldquoA simulation study of ridge andother regression estimatorsrdquo Communications in Statistics-0eory and Methods vol 5 no 4 pp 307ndash323 1976

[18] D W Wichern and G A Churchill ldquoA comparison of ridgeestimatorsrdquo Technometrics vol 20 no 3 pp 301ndash311 1978

[19] B M G Kibria ldquoPerformance of some new ridge regressionestimatorsrdquo Communications in Statistics-Simulation andComputation vol 32 no 1 pp 419ndash435 2003

[20] A F Lukman and K Ayinde ldquoReview and classifications ofthe ridge parameter estimation techniquesrdquoHacettepe Journalof Mathematics and Statistics vol 46 no 5 pp 953ndash967 2017

[21] A K M E Saleh M Arashi and B M G Kibria 0eory ofRidge Regression Estimation with Applications WileyHoboken NJ USA 2019

[22] K Liu ldquoUsing Liu-type estimator to combat collinearityrdquoCommunications in Statistics-0eory and Methods vol 32no 5 pp 1009ndash1020 2003

[23] K Alheety and B M G Kibria ldquoOn the Liu and almostunbiased Liu estimators in the presence of multicollinearitywith heteroscedastic or correlated errorsrdquo Surveys in Math-ematics and its Applications vol 4 pp 155ndash167 2009

[24] X-Q Liu ldquoImproved Liu estimator in a linear regressionmodelrdquo Journal of Statistical Planning and Inference vol 141no 1 pp 189ndash196 2011

[25] Y Li and H Yang ldquoA new Liu-type estimator in linear regressionmodelrdquo Statistical Papers vol 53 no 2 pp 427ndash437 2012

[26] B Kan O Alpu and B Yazıcı ldquoRobust ridge and robust Liuestimator for regression based on the LTS estimatorrdquo Journalof Applied Statistics vol 40 no 3 pp 644ndash655 2013

[27] R A Farghali ldquoGeneralized Liu-type estimator for linearregressionrdquo International Journal of Research and Reviews inApplied Sciences vol 38 no 1 pp 52ndash63 2019

[28] S G Wang M X Wu and Z Z Jia Matrix InequalitiesChinese Science Press Beijing China 2nd edition 2006

[29] R W Farebrother ldquoFurther results on the mean square errorof ridge regressionrdquo Journal of the Royal Statistical SocietySeries B (Methodological) vol 38 no 3 pp 248ndash250 1976

[30] G Trenkler and H Toutenburg ldquoMean squared error matrixcomparisons between biased estimators-an overview of recentresultsrdquo Statistical Papers vol 31 no 1 pp 165ndash179 1990

[31] B M G Kibria and S Banik ldquoSome ridge regression esti-mators and their performancesrdquo Journal of Modern AppliedStatistical Methods vol 15 no 1 pp 206ndash238 2016

[32] D G Gibbons ldquoA simulation study of some ridge estimatorsrdquoJournal of the American Statistical Association vol 76no 373 pp 131ndash139 1981

[33] J P Newhouse and S D Oman ldquoAn evaluation of ridgeestimators A report prepared for United States air forceproject RANDrdquo 1971

[34] H Woods H H Steinour and H R Starke ldquoEffect ofcomposition of Portland cement on heat evolved duringhardeningrdquo Industrial amp Engineering Chemistry vol 24no 11 pp 1207ndash1214 1932

[35] S Chatterjee and A S Hadi Regression Analysis by ExampleWiley Hoboken NJ USA 1977

[36] S Kaciranlar S Sakallioglu F Akdeniz G P H Styan andH J Werner ldquoA new biased estimator in linear regression anda detailed analysis of the widely-analysed dataset on portlandcementrdquo Sankhya 0e Indian Journal of Statistics Series Bvol 61 pp 443ndash459 1999

[37] S Chatterjee and A S Haadi Regression Analysis by ExampleWiley Hoboken NJ USA 2006

[38] E Malinvard Statistical Methods of Econometrics North-Holland Publishing Company Amsterdam Netherlands 3rdedition 1980

[39] D N Gujarati Basic Econometrics McGraw-Hill New YorkNY USA 1995

[40] A F Lukman K Ayinde and A S Ajiboye ldquoMonte Carlostudy of some classification-based ridge parameter estima-torsrdquo Journal of Modern Applied Statistical Methods vol 16no 1 pp 428ndash451 2017

16 Scientifica

42 Example 2 French Economy Data +e French economydata in Chatterjee and Hadi [37] are considered in this ex-ample It has been analyzed by Malinvard [38] and Liu [6]among others+e variables are imports domestic productionstock formation and domestic consumption All are measuredin milliards of French francs for the years 1949 through 1966

+e regression model for these data is defined as

yi β0 + β1X1 + β2X2 + β3X3 + εi (47)

where yi IMPORT X1 domestic production X2 stockformation and X3 domestic consumption +e correlationmatrix of the predicted variable is given in Table 12

+e variance inflation factors areVIF1 469688VIF2 1047 and VIF3 469338 +e ei-genvalues of the XprimeX matrix are λ1 161779 λ2 158 andλ3 4961 and the condition number is 32612 If we reviewthe above correlation matrix VIFs and condition number itcan be said that there is presence of severe multicollinearityexisting in the predictor variables

+e biasing parameter for the new estimator is defined in(41) and (42) +e biasing parameter for the ridge and Liuestimator is provided in (6) (8) and (9) respectively

We analyzed the data using the biasing parameters foreach of the estimators and presented the results in Tables 10and 11 It can be seen from Tables 10 and 11 that theproposed estimator performed the best in the sense ofsmaller MSE

5 Summary and Concluding Remarks

In this paper we introduced a new biased estimator toovercome the multicollinearity problem for the multiplelinear regression model and provided the estimation tech-nique of the biasing parameter A simulation study has beenconducted to compare the performance of the proposedestimator and Liu [6] and ridge regression estimators [3]Simulation results evidently show that the proposed esti-mator performed better than both Liu and ridge under somecondition on the shrinkage parameter Two sets of real-lifedata are analyzed to illustrate the benefits of using the newestimator in the context of a linear regression model +eproposed estimator is recommended for researchers in this

area Its application can be extended to other regressionmodels for example logistic regression Poisson ZIP andrelated models and those possibilities are under currentinvestigation [37 39 40]

Data Availability

Data will be made available on request

Conflicts of Interest

+e authors declare that they have no conflicts of interest

Acknowledgments

We are dedicating this article to those who lost their livesbecause of COVID-19

References

[1] C Stein ldquoInadmissibility of the usual estimator for mean ofmultivariate normal distributionrdquo in Proceedings of the 0irdBerkley Symposium on Mathematical and Statistics Proba-bility J Neyman Ed vol 1 pp 197ndash206 Springer BerlinGermany 1956

[2] W F Massy ldquoPrincipal components regression in exploratorystatistical researchrdquo Journal of the American Statistical As-sociation vol 60 no 309 pp 234ndash256 1965

[3] A E Hoerl and R W Kennard ldquoRidge regression biasedestimation for nonorthogonal problemsrdquo Technometricsvol 12 no 1 pp 55ndash67 1970

[4] L S Mayer and T A Willke ldquoOn biased estimation in linearmodelsrdquo Technometrics vol 15 no 3 pp 497ndash508 1973

[5] B F Swindel ldquoGood ridge estimators based on prior infor-mationrdquo Communications in Statistics-0eory and Methodsvol 5 no 11 pp 1065ndash1075 1976

[6] K Liu ldquoA new class of biased estimate in linear regressionrdquoCommunication in Statistics- 0eory and Methods vol 22pp 393ndash402 1993

[7] F Akdeniz and S Kaccediliranlar ldquoOn the almost unbiasedgeneralized liu estimator and unbiased estimation of the biasand mserdquo Communications in Statistics-0eory and Methodsvol 24 no 7 pp 1789ndash1797 1995

[8] M R Ozkale and S Kaccediliranlar ldquo+e restricted and unre-stricted two-parameter estimatorsrdquo Communications in Sta-tistics-0eory and Methods vol 36 no 15 pp 2707ndash27252007

[9] S Sakallıoglu and S Kaccedilıranlar ldquoA new biased estimatorbased on ridge estimationrdquo Statistical Papers vol 49 no 4pp 669ndash689 2008

Table 11 +e results of regression coefficients and the corresponding MSE values

Coef 1113954α 1113954α(k) 1113954α(d)1113954dalt 1113954α(d)1113954dopt 1113954αKL(kHNM) 1113954αKL(kmin)

α0 minus 197127 minus 167613 minus 125762 minus 188410 minus 165855 minus 188782α1 00327 01419 02951 00648 01485 00636α2 04059 03576 02875 03914 03548 03922α3 02421 00709 minus 01696 01918 00606 01937MSE 173326 2130519 5828312 1660293 2211899 1660168kd mdash 00527 05282 09423 00258 00065

Table 12 Correlation matrix

X1 X2 X3

X1 1000 0210 0999X2 0210 1000 0208X3 0999 0208 1000

Scientifica 15

[10] H Yang and X Chang ldquoA new two-parameter estimator inlinear regressionrdquo Communications in Statistics-0eory andMethods vol 39 no 6 pp 923ndash934 2010

[11] M Roozbeh ldquoOptimal QR-based estimation in partially linearregression models with correlated errors using GCV crite-rionrdquo Computational Statistics amp Data Analysis vol 117pp 45ndash61 2018

[12] F Akdeniz and M Roozbeh ldquoGeneralized difference-basedweightedmixed almost unbiased ridge estimator in partially linearmodelsrdquo Statistical Papers vol 60 no 5 pp 1717ndash1739 2019

[13] A F Lukman K Ayinde S Binuomote and O A ClementldquoModified ridge-type estimator to combat multicollinearityapplication to chemical datardquo Journal of Chemometricsvol 33 no 5 p e3125 2019

[14] A F Lukman K Ayinde S K Sek and E Adewuyi ldquoAmodified new two-parameter estimator in a linear regressionmodelrdquo Modelling and Simulation in Engineering vol 2019Article ID 6342702 10 pages 2019

[15] A E Hoerl R W Kannard and K F Baldwin ldquoRidge re-gressionsome simulationsrdquo Communications in Statisticsvol 4 no 2 pp 105ndash123 1975

[16] G C McDonald and D I Galarneau ldquoA monte carlo eval-uation of some ridge-type estimatorsrdquo Journal of the Amer-ican Statistical Association vol 70 no 350 pp 407ndash416 1975

[17] J F Lawless and P Wang ldquoA simulation study of ridge andother regression estimatorsrdquo Communications in Statistics-0eory and Methods vol 5 no 4 pp 307ndash323 1976

[18] D W Wichern and G A Churchill ldquoA comparison of ridgeestimatorsrdquo Technometrics vol 20 no 3 pp 301ndash311 1978

[19] B M G Kibria ldquoPerformance of some new ridge regressionestimatorsrdquo Communications in Statistics-Simulation andComputation vol 32 no 1 pp 419ndash435 2003

[20] A F Lukman and K Ayinde ldquoReview and classifications ofthe ridge parameter estimation techniquesrdquoHacettepe Journalof Mathematics and Statistics vol 46 no 5 pp 953ndash967 2017

[21] A K M E Saleh M Arashi and B M G Kibria 0eory ofRidge Regression Estimation with Applications WileyHoboken NJ USA 2019

[22] K Liu ldquoUsing Liu-type estimator to combat collinearityrdquoCommunications in Statistics-0eory and Methods vol 32no 5 pp 1009ndash1020 2003

[23] K Alheety and B M G Kibria ldquoOn the Liu and almostunbiased Liu estimators in the presence of multicollinearitywith heteroscedastic or correlated errorsrdquo Surveys in Math-ematics and its Applications vol 4 pp 155ndash167 2009

[24] X-Q Liu ldquoImproved Liu estimator in a linear regressionmodelrdquo Journal of Statistical Planning and Inference vol 141no 1 pp 189ndash196 2011

[25] Y Li and H Yang ldquoA new Liu-type estimator in linear regressionmodelrdquo Statistical Papers vol 53 no 2 pp 427ndash437 2012

[26] B Kan O Alpu and B Yazıcı ldquoRobust ridge and robust Liuestimator for regression based on the LTS estimatorrdquo Journalof Applied Statistics vol 40 no 3 pp 644ndash655 2013

[27] R A Farghali ldquoGeneralized Liu-type estimator for linearregressionrdquo International Journal of Research and Reviews inApplied Sciences vol 38 no 1 pp 52ndash63 2019

[28] S G Wang M X Wu and Z Z Jia Matrix InequalitiesChinese Science Press Beijing China 2nd edition 2006

[29] R W Farebrother ldquoFurther results on the mean square errorof ridge regressionrdquo Journal of the Royal Statistical SocietySeries B (Methodological) vol 38 no 3 pp 248ndash250 1976

[30] G Trenkler and H Toutenburg ldquoMean squared error matrixcomparisons between biased estimators-an overview of recentresultsrdquo Statistical Papers vol 31 no 1 pp 165ndash179 1990

[31] B M G Kibria and S Banik ldquoSome ridge regression esti-mators and their performancesrdquo Journal of Modern AppliedStatistical Methods vol 15 no 1 pp 206ndash238 2016

[32] D G Gibbons ldquoA simulation study of some ridge estimatorsrdquoJournal of the American Statistical Association vol 76no 373 pp 131ndash139 1981

[33] J P Newhouse and S D Oman ldquoAn evaluation of ridgeestimators A report prepared for United States air forceproject RANDrdquo 1971

[34] H Woods H H Steinour and H R Starke ldquoEffect ofcomposition of Portland cement on heat evolved duringhardeningrdquo Industrial amp Engineering Chemistry vol 24no 11 pp 1207ndash1214 1932

[35] S Chatterjee and A S Hadi Regression Analysis by ExampleWiley Hoboken NJ USA 1977

[36] S Kaciranlar S Sakallioglu F Akdeniz G P H Styan andH J Werner ldquoA new biased estimator in linear regression anda detailed analysis of the widely-analysed dataset on portlandcementrdquo Sankhya 0e Indian Journal of Statistics Series Bvol 61 pp 443ndash459 1999

[37] S Chatterjee and A S Haadi Regression Analysis by ExampleWiley Hoboken NJ USA 2006

[38] E Malinvard Statistical Methods of Econometrics North-Holland Publishing Company Amsterdam Netherlands 3rdedition 1980

[39] D N Gujarati Basic Econometrics McGraw-Hill New YorkNY USA 1995

[40] A F Lukman K Ayinde and A S Ajiboye ldquoMonte Carlostudy of some classification-based ridge parameter estima-torsrdquo Journal of Modern Applied Statistical Methods vol 16no 1 pp 428ndash451 2017

16 Scientifica

[10] H Yang and X Chang ldquoA new two-parameter estimator inlinear regressionrdquo Communications in Statistics-0eory andMethods vol 39 no 6 pp 923ndash934 2010

[11] M Roozbeh ldquoOptimal QR-based estimation in partially linearregression models with correlated errors using GCV crite-rionrdquo Computational Statistics amp Data Analysis vol 117pp 45ndash61 2018

[12] F Akdeniz and M Roozbeh ldquoGeneralized difference-basedweightedmixed almost unbiased ridge estimator in partially linearmodelsrdquo Statistical Papers vol 60 no 5 pp 1717ndash1739 2019

[13] A F Lukman K Ayinde S Binuomote and O A ClementldquoModified ridge-type estimator to combat multicollinearityapplication to chemical datardquo Journal of Chemometricsvol 33 no 5 p e3125 2019

[14] A F Lukman K Ayinde S K Sek and E Adewuyi ldquoAmodified new two-parameter estimator in a linear regressionmodelrdquo Modelling and Simulation in Engineering vol 2019Article ID 6342702 10 pages 2019

[15] A E Hoerl R W Kannard and K F Baldwin ldquoRidge re-gressionsome simulationsrdquo Communications in Statisticsvol 4 no 2 pp 105ndash123 1975

[16] G C McDonald and D I Galarneau ldquoA monte carlo eval-uation of some ridge-type estimatorsrdquo Journal of the Amer-ican Statistical Association vol 70 no 350 pp 407ndash416 1975

[17] J F Lawless and P Wang ldquoA simulation study of ridge andother regression estimatorsrdquo Communications in Statistics-0eory and Methods vol 5 no 4 pp 307ndash323 1976

[18] D W Wichern and G A Churchill ldquoA comparison of ridgeestimatorsrdquo Technometrics vol 20 no 3 pp 301ndash311 1978

[19] B M G Kibria ldquoPerformance of some new ridge regressionestimatorsrdquo Communications in Statistics-Simulation andComputation vol 32 no 1 pp 419ndash435 2003

[20] A F Lukman and K Ayinde ldquoReview and classifications ofthe ridge parameter estimation techniquesrdquoHacettepe Journalof Mathematics and Statistics vol 46 no 5 pp 953ndash967 2017

[21] A K M E Saleh M Arashi and B M G Kibria 0eory ofRidge Regression Estimation with Applications WileyHoboken NJ USA 2019

[22] K Liu ldquoUsing Liu-type estimator to combat collinearityrdquoCommunications in Statistics-0eory and Methods vol 32no 5 pp 1009ndash1020 2003

[23] K Alheety and B M G Kibria ldquoOn the Liu and almostunbiased Liu estimators in the presence of multicollinearitywith heteroscedastic or correlated errorsrdquo Surveys in Math-ematics and its Applications vol 4 pp 155ndash167 2009

[24] X-Q Liu ldquoImproved Liu estimator in a linear regressionmodelrdquo Journal of Statistical Planning and Inference vol 141no 1 pp 189ndash196 2011

[25] Y Li and H Yang ldquoA new Liu-type estimator in linear regressionmodelrdquo Statistical Papers vol 53 no 2 pp 427ndash437 2012

[26] B Kan O Alpu and B Yazıcı ldquoRobust ridge and robust Liuestimator for regression based on the LTS estimatorrdquo Journalof Applied Statistics vol 40 no 3 pp 644ndash655 2013

[27] R A Farghali ldquoGeneralized Liu-type estimator for linearregressionrdquo International Journal of Research and Reviews inApplied Sciences vol 38 no 1 pp 52ndash63 2019

[28] S G Wang M X Wu and Z Z Jia Matrix InequalitiesChinese Science Press Beijing China 2nd edition 2006

[29] R W Farebrother ldquoFurther results on the mean square errorof ridge regressionrdquo Journal of the Royal Statistical SocietySeries B (Methodological) vol 38 no 3 pp 248ndash250 1976

[30] G Trenkler and H Toutenburg ldquoMean squared error matrixcomparisons between biased estimators-an overview of recentresultsrdquo Statistical Papers vol 31 no 1 pp 165ndash179 1990

[31] B M G Kibria and S Banik ldquoSome ridge regression esti-mators and their performancesrdquo Journal of Modern AppliedStatistical Methods vol 15 no 1 pp 206ndash238 2016

[32] D G Gibbons ldquoA simulation study of some ridge estimatorsrdquoJournal of the American Statistical Association vol 76no 373 pp 131ndash139 1981

[33] J P Newhouse and S D Oman ldquoAn evaluation of ridgeestimators A report prepared for United States air forceproject RANDrdquo 1971

[34] H Woods H H Steinour and H R Starke ldquoEffect ofcomposition of Portland cement on heat evolved duringhardeningrdquo Industrial amp Engineering Chemistry vol 24no 11 pp 1207ndash1214 1932

[35] S Chatterjee and A S Hadi Regression Analysis by ExampleWiley Hoboken NJ USA 1977

[36] S Kaciranlar S Sakallioglu F Akdeniz G P H Styan andH J Werner ldquoA new biased estimator in linear regression anda detailed analysis of the widely-analysed dataset on portlandcementrdquo Sankhya 0e Indian Journal of Statistics Series Bvol 61 pp 443ndash459 1999

[37] S Chatterjee and A S Haadi Regression Analysis by ExampleWiley Hoboken NJ USA 2006

[38] E Malinvard Statistical Methods of Econometrics North-Holland Publishing Company Amsterdam Netherlands 3rdedition 1980

[39] D N Gujarati Basic Econometrics McGraw-Hill New YorkNY USA 1995

[40] A F Lukman K Ayinde and A S Ajiboye ldquoMonte Carlostudy of some classification-based ridge parameter estima-torsrdquo Journal of Modern Applied Statistical Methods vol 16no 1 pp 428ndash451 2017

16 Scientifica