Received 18 May 2015; accepted 23 June 2015; published 29 June 2015

ABSTRACT

In this paper we compare recently developed preliminary test estimator called Preliminary Test Stochastic Restricted Liu Estimator (PTSRLE) with Ordinary Least Square Estimator (OLSE) and Mixed Estimator (ME) in the Mean Square Error Matrix (MSEM) sense for the two cases in which the stochastic restrictions are correct and not correct. Finally a numerical example and a Monte Carlo simulation study are done to illustrate the theoretical findings.

Keywords:

Multicollinearity, Stochastic Restrictions, Ordinary Least Square Estimator, Mixed Estimator, Preliminary Test Estimator, Mean Square Error Matrix

1. Introduction

To overcome the multicollinearity problem arises in the Ordinary Least Squares Estimation (OLSE) procedure, different methods have been proposed in the literature. One of the most important estimation methods is to consider biased estimators, such as the Ridge Estimator (RE) by Hoerl and Kennard [1] , the Liu Estimator (LE) by Liu [2] , and the Almost Unbiased Liu Estimator (AULE) by Akdeniz and Kaçiranlar [3] . Alternative method to solve the multicollinearity problem is to consider parameter estimation with some restrictions on the unknown parameters, which may be exact or stochastic. When the stochastic restrictions are available in addition to sample model, Theil and Goldberger [4] introduced the Mixed Estimator (ME). Replacing OLSE by ME in the Liu Estimator, the Stochastic Restricted Liu Estimator (SRLE) has been proposed by Hubert and Wijekoon [5] . When different estimators are available, the preliminary test estimation procedure is adopted to select a suitable estimator, and it can also be used as another estimator with combining properties of both estimators. The preliminary test approach was first proposed by Bancroft [6] , and then has been studied by many researchers, such as Judge and Bock [7] , Wijekoon [8] and Arumairajan and Wijekoon [9] . By combining OLSE and ME, the Ordinary Stochastic Preliminary Test Estimator (OSPE) was proposed by Wijekoon [8] . Recently, Arumairajan and Wijekoon [9] introduced the Preliminary Test Stochastic Restricted Liu Estimator (PTSRLE) by combining the Stochastic Restricted Liu Estimator and Liu Estimator. In their study, they compared PTSRLE with SRLE by using the Mean Square Error Matrix (MSEM) and Scalar Mean Square Error (SMSE) criterions.

In this research we further compare the mean square error matrix of PTSRLE with OLSE and ME. The rest of the paper is organized as follows. The model specification and estimation are given in section 2. In section 3, the mean square error matrix comparisons between PTSRLE with OLSE and ME are performed. A numerical example and a Monte Carlo simulation are used to illustrate the theoretical findings in section 4, and in section 5 we state the conclusions.

2. Model Specification and Estimation

First we consider the multiple linear regression model

, (1)

where y is an n × 1 observable random vector, X is an n × p known design matrix of rank p, β is a p × 1 vector of unknown parameters and ε is an n × 1 vector of disturbances.

In addition to sample model (1), let us be given some prior information about β in the form of a set of m independent stochastic linear restrictions as follows;

(2)

where r is an m × 1 stochastic known vector R is a m × p of full row rank with known elements, is non zero m × 1 unknown vector and υ is an m × 1 random vector of disturbances and is assumed to be known and positive definite. Further it is assumed that υ is stochastically independent of ε. i.e..

The Ordinary Least Squares Estimator (OLSE) for model (1) and the Mixed Estimator (ME) (Theil and Goldberger [4] ) due to a stochastic prior restriction (2) are given by

(3)

and

(4)

respectively, where.

The expectation vector, and the mean square error matrix of are given as

(5)

and

(6)

respectively.

The expectation vector, dispersion matrix, and the mean square error matrix of are given as

, (7)

(8)

and

(9)

respectively, where, , and. Note that when the stochastic restrictions are correct then, and consequently the Mixed Estimator becomes an unbiased estimator.

The Liu Estimator (Liu, [2] ) is given as

. (10)

Replacing OLSE by ME in the Liu Estimator, Hubert and Wijekoon [5] introduced the Stochastic Restricted Liu Estimator (SRLE), and is given by

(11)

when different estimators are available for the same parameter vector in the linear regression model one must solve the problem of their comparison. Usually as a simultaneous measure of covariance and bias, the mean square error matrix of is used, and is defined by

, (12)

where is the dispersion matrix, and denotes the bias vector. We recall that the Scalar

Mean Square Error of .

For two given estimators and, the estimator is said to be superior to under the MSEM criterion if and only if

. (13)

Let us now turn to the question of the statistical evaluation of the compatibility of sample and stochastic information. The classical procedures is to test the hypothesis

(14)

under linear model (1) and stochastic prior information (2).

The Ordinary Stochastic Preliminary Test Estimator (OSPE) of (Wijekoon [8] ) is defined as

(15)

Further, we can write (15) as

, (16)

where,

(17)

which has a non-central distribution under, with non-centrality parameter

(18)

and and are indicator functions which take the value one if falls in the sub-

scripted interval, and zero otherwise. is the upper α-level critical value from the central F distribution.

The expectation vector, dispersion matrix, and the mean square error matrix of are derived by Wijekoon [8] , and given by

(19)

(20)

an

(21)

respectively, where,.

Recently, Arumairajan and Wijekoon [9] proposed the Preliminary Test Stochastic Restricted Liu Estimator (PTSRLE) by combining the Liu Estimator and Stochastic Restricted Liu Estimator, and is given by

(22)

Note that the PTSRLE can be rewritten as follows:

(23)

By using Equations (19), (20) and (21), Arumairajan and Wijekoon [9] derived the expectation vector, bias vector, dispersion matrix and mean square error matrix of PTSRLE as follows:

, (24)

, (25)

(26)

and

(27)

respectively, where.

3. Mean Square Error Matrix Comparisons

In this section we compare the PTSRLE with OLSE and ME in the mean square error matrix sense for the two cases in which the stochastic restrictions are correct and not correct.

3.1. Comparison between the PTSRLE and OLSE

The mean square error matrix difference between OLSE and PTSRLE can be written as

(28)

where,

with, and.

Now the following theorem can be stated

Theorem 1:

1) When the stochastic restrictions are true (i.e.), the PTSRLE is superior to OLSE in the mean square error matrix sense if and only if, where, , and is the value of when.

2) When the stochastic restrictions are not true (i.e.), and if the maximum eigenvalue of is less than one, the PTSRLE is superior to OLSE if and only if.

Proof:

1) If stochastic restrictions are correct then, and the equation (28) reduced to

.

To apply lemma 2 (Appendix), we have to show that

.

Since for we have

Hence is positive definite matrix for all. According to lemma 1 (Appendix), is positive definite matrix since is positive definite matrix and is nonnegative definite matrix. By applying lemma 2 (Appendix), if and only if. Therefore the PTSRLE is superior to OLSE in the mean square error matrix sense when stochastic restrictions are correct if and only if.

2) If stochastic restrictions are not correct then, and consequently with respect to the MSE matrix criterion is superior to if and only if. To apply lemma 2 (Appendix) in order to prove that, one required condition is that.

We have already proved that. Also is positive definite since, and. Now according to lemma 3 (Appendix),

if and only if, where is the maxi-

mum eigenvalue of. Now according to lemma 2, if and only if. This completes the proof.

3.2. Comparison between the PTSRLE and ME

The mean square error matrix difference between ME and PTSRLE is

(29)

where with, , and.

Now we can give the following theorem.

Theorem 2:

1) When the stochastic restrictions are true (i.e.), and if the maximum eigen value of is less than one, the PTSRLE is superior to ME in the mean square error matrix sense if and only if where with and.

2) When the stochastic restrictions are not true (i.e.), and if the maximum eigen value of is less than one, the PTSRLE is superior to ME in the mean square error matrix sense if and only if.

Proof:

1) When stochastic restrictions are true then, and hence, which is clearly a nonnegative matrix. We have already proved that is positive definite matrix. Now according to lemma 3, if and only if the maximum eigen values of is less than one. Now according to lemma 2, the PTSRLE is superior to ME if and only if, and if.

2) When stochastic restrictions are not true (i.e.), if and only if

. To prove that lemma 4 (Appendix) can be used. To use this lemma first we prove that C is positive definite. According to lemma 3, C is a positive definite matrix if and only if the maximum eigenvalue of is less than one. By applying lemma 4, we can prove that if and only if. This completes the proof.

According to theorem 1 and 2 it is clear that PTSRLE is superior to OLSE and ME under certain conditions.

4. Numerical Illustrations

In this section the comparison of PTSRLE with OLSE and ME are demonstrated using a numerical example, and a simulation study.

4.1. Numerical Example

To illustrate our theoretical result, we consider the data set on Total National Research and Development Expenditures as a percent of Gross National product due to Gruber [10] . This data set is used by Akdeniz and Erol [11] , Li and Yang [12] and Wu and Yang [13] to verify the theoretical results. Data shows Total National Research and Development Expenditures as a Percent of Gross National Product by Country: 1972-1986. It represents the relationship between the dependent variable y the percentage spent by the United States and the four other independent variables and. The variable represents the percent spent by France, that spent by West Germany, that spent by Japan, and that spent by the former Soviet Union. The data set is given below:

The four column of the 10 × 4 matrix X comprise the data on and respectively, and y is the response variable. From this data, we obtain the following results:

1) The eigen values of: 302.9626, 0.7283, 0.0447, 0.0345.

2) The OLS estimator of.

3) The OLS estimator of.

4) The condition number of: 8781.53.

The condition number implies the existence of multicollinearity in the data set. We consider the following stochastic restrictions (Li and Yang, [12] )

.

Further the significance level is taken as α = 0.05. Figure 1 is drawn by using the SMSE obtained by using Equations (6), (9) and (27).

Based on Figure 1, we can say that the SMSE of PTSRLE is larger than the SMSE of both ME and OLSE when d is small. But the PTSRLE has the smallest SMSE than both ME and OLSE for. Moreover the ME is always superior to OLSE.

4.2. Monte Carlo Simulation

To illustrate the behavior of our proposed estimators, we perform the Monte Carlo Simulation study by considering different levels of multicollinearity. Following McDonald and Galarneau [14] we generate explanatory variables as follows:

where is an independent standard normal pseudo random number, and is specified so that the theoretical correlation between any two explanatory variables is given by. A dependent variable is generated by using the equation.

where is a normal pseudo random number with mean zero and variance. Newhouse and Oman [15] have noted that if the MSE is a function of and β, and if the explanatory variables are fixed, then subject to the constraint, the MSE is minimized when β is the normalized eigenvector corresponding to the largest eigenvalue of the matrix. In this study we choose the normalized eigenvector corresponding to the largest eigenvalue of as the coefficient vector β, n = 100, p = 4 and. Four different sets of correlations are considered by selecting the values as γ = 0.9, 0.99, 0.999 and 0.9999, and the significance level is taken as α = 0.05. The condition numbers for γ = 0.9, 0.99, 0.999 and 0.9999 are 19.49, 223.67, 2297.89 and 23137.88 respectively. Figures 2-5 are drawn by using the SMSE obtained by using Equation (6), (9) and (27).

According to Figure 2, the PTSRLE has the smallest SMSE than ME except d is small. However the OLSE has the smallest SMSE than ME and PTSRLE. From Figure 3, the SMSE of PTSRLE is larger than the SMSE of both OLSE and ME when d is small. From Figure 4 and Figure 5, we can notice that most of the cases, the PTSRLE has the smallest SMSE than ME and OLSE.

5. Conclusion

In this paper we have shown that the Preliminary Test Stochastic Restricted Liu Estimator is superior to Mixed Estimator and Ordinary Least Square Estimator in the mean square error matrix sense under certain conditions.

From the simulation study and the numerical illustration we notice that the PTSRLE has the smallest SMSE than ME and OLSE when multicollinearity among the predictor variables is large.

Acknowledgements

We thank the Postgraduate Institute of Science, University of Peradeniya, Sri Lanka for providing all facilities to do this research.

Cite this paper

SivarajahArumairajan,PushpakanthieWijekoon,11, (2015) More on the Preliminary Test Stochastic Restricted Liu Estimator in Linear Regression Model. Open Journal of Statistics,05,340-349. doi: 10.4236/ojs.2015.54035

References

Appendix

Lemma 1: (Rao and Touterburg, [16] )

Let A and B be (n × n) matrices such that A > 0 and. Then.

Lemma 2: (Farebrother, [17] )

Let A > 0 be an (n × n) matrix, b an (n × 1) vector. Then if and only if.

Lemma 3: (Wang et al., [18] )

Let n × n matrices M > 0, N > 0 (or), then M > N if and only if. where is the largest eigenvalue of the matrix.

Lemma 4: (Trenkler and Toutenburg, [19] )

Let and be two linear estimator of. Suppose that is positive definite then is nonnegative definite if and only if, where denotes the bias vector of.