^{1}

^{*}

^{2}

^{3}

Parameter estimation by maximizing the marginal likelihood function in generalized linear mixed models (GLMMs) is highly challenging because it may involve analytically intractable high-dimensional integrals. In this paper, we propose to use Quasi-Monte Carlo (QMC) approximation through implementing Newton-Raphson algorithm to address the estimation issue in GLMMs. The random effects release to be correlated and joint mean-covariance modelling is considered. We demonstrate the usefulness of the proposed QMC-based method in approximating high-dimensional integrals and estimating the parameters in GLMMs through simulation studies. For illustration, the proposed method is used to analyze the infamous salamander mating binary data, of which the marginalized likelihood involves six 20-dimensional integrals that are analytically intractable, showing that it works well in practices.

McCullagh and Nelder (1989) [

Generalized linear mixed models (GLMMs) are very useful for non-Gaussian correlated or clustered data and widely applied in many areas including epidemiology, ecological, and clinical trials. Parameters estimation in GLMMs is, however, very challenging, and correlated binary data is even more difficult to analyze than continuous data. This is because the marginalized likelihood function, obtained by integrating out multivariate random effects from the joint density function of the responses and random effects, is in general analytically intractable. For example, the likelihood function for the pooled salamander mating data described above involves six 20-dimensional integrals which are analytically intractable. In the literature, the marginalized likelihood function was considered by two typical methods: a) analytical Laplace approximation to integrals: This is represented by the work of Breslow and Clayton (1993) [

However, these assumptions may not always hold due to the likely correlation of random effects, for instance. At least, the independence assumption of random effects should be testable. Misspecification of random effects covariance structure may yield inefficient estimates of parameters and can cause the problem of biased estimates of variance components, making statistical inferences for GLMMs unreliable. In this paper, we release the assumption of independence for random effects and aim to develop a new methodology which can model the covariance structures of random effects in GLMMs. Covariance modeling strategy based on a modified Cholesky decomposition is studied within the framework of GLMMs. The paper is organized as follows. GLMMs and the marginalized Quasi-likelihood are briefly reviewed in Section 2. The QMC approximation, the simplest good point set and analytical form of the MLEs for GLMMs are discussed in Section 3. Covariance modeling strategy through a modified Cholesky decomposition is described in Section 4. In Section 5, the salamander data set is analyzed as an example for illustration of the use of the new method. Simulation studies with the same design protocol as the salamander data are conducted in Section 6. Discussions on the related issues and further studies are presented in Section 7.

Let

The GLMMs are of the form

where

The conditional mean and variances, given random effects b, are given by

When the linear predictor

Note q-dimensional random effects b are assumed to be normally distributed with mean 0 and positive definite variance-covariance matrix

Quasi-likelihood function of the fixed effects

where

so that

is the conditional log quasi-likelihood of

It is challenging to calculate the maximum likelihood estimates (MLEs),

In this section, Quasi-Monte Carlo (QMC) approach is used to approximate high dimensional integrals. The traditional Monte Carlo (MC) method is outlined first as follow. Let

Assume K independent random samples

According to the strong law of large number, the approximation

Since the rate of convergence is independent of q, the MC approximation seems to be very appealing. However, the MC algorithm has some serious drawbacks (see, e.g., Niederreiter, 1992 [

Quasi-Monte Carlo (QMC) sequences are a deterministic alternative to Monte Carlo sequences (Niderreiter, 1992). In contrast to MC sequences, QMC sequences have a property that they are uniformly distributed on the unit cube. Thus, QMC approximation has the same equation as (8) but the MC random samples are replaced by deterministic samples that are uniformly distributed on the domain. Uniformity of sequences is measured by means of discrepancy, and for this reason QMC sequences are also called low-discrepancy sequences. Note the Koksma-Hlawka inequality

where

that the absolute integration error has an order

chosen such that their start discrepancy is close to

In the QMC approach, there are many good algorithms for generating such integration nodes. One set of such integration nodes is called good point set and discussed below.

Denote a good point by

percube. The set

where

Suppose

where

example, it can be taken as the Cholesky decomposition of

Let

the inverse function of the link function

where

Note that the weight

In general, the score equations in (12)-(13) have no analytical solutions for

Misspecification of covariance structures of random effects may cause problems of inefficient estimates of parameters in GLMMs. In some circumstances, it may lead to biased estimates of the parameters. Hence, in this paper we propose to model the covariance structure of random effects in GLMMs, rather than specify a structure to the covariance matrix

where

Thus,

where

novation variances (IVs), i.e.,

In a spirit of Pourahmadi (1999) [

where

Assume that

where

Note that each experiment involves 40 salamanders so that the log-likelihood function for each experiment involves one 40-dimensional integral. It can be further reduced to the sum of two 20-dimensional integrals due to the block design of the two closed groups, see Karim and Zeger (1992) [

We now apply the proposed covariance modeling method to the salamander mating data. Note

where

where

The model for the innovation variances is

where

where

The pooled data are now analyzed using the proposed QMC approach. The MLEs of the fixed effects and variance components in the model are calculated. Specifically, we implement the simplest QMC points, square root sequences, to approximate the 20-dimensional analytically intractable integrals. A set of 20-dimensional points on the unit cube

Note that the covariance modeling for

We code our programme in FORTRAN and run on a PC Pentium (R) 4 PC (CPU 3.20 GHz). We choose the PQL estimate as the starting value of

10,000 | 0.91 | −2.94 | −0.62 | 3.71 | 0.00045 | 0.00062 | 0.0044 | 0.0031 | 0.19 | −0.022 | −0.024 | 0.00021 | −207.21 |

(0.38) | (0.39) | (0.33) | (0.12) | (0.0024) | (0.0021) | (0.0056) | (0.0067) | (0.046) | (0.051) | (0.025) | (0.0046) | ||

20,000 | 0.77 | −2.87 | −0.59 | 3.58 | 0.00087 | 0.0019 | 0.0052 | 0.0039 | 0.064 | 0.0081 | −0.0024 | −0.00010 | −207.43 |

(0.34) | (0.52) | (0.50) | (0.52) | (0.0056) | (0.0055) | (0.0089) | (0.0087) | (0.034) | (0.044) | (0.042) | (0.0018) | ||

30,000 | 1.24 | −2.88 | −1.00 | 3.71 | 0.00030 | 0.0035 | −0.00082 | −0.0063 | 0.43 | 0.0050 | 0.00047 | 0.00054 | −205.95 |

(0.15) | (0.37) | (0.22) | (0.57) | (0.0002) | (0.0000) | (0.0000) | (0.0000) | (0.0019) | (0.026) | (0.023) | (0.0027) | ||

40,000 | 1.20 | −2.84 | −0.96 | 3.64 | −0.00032 | 0.00092 | −0.00091 | 0.0027 | 0.39 | 0.0031 | 0.0032 | 0.00012 | −206.34 |

(0.32) | (0.48) | (0.25) | (0.58) | (0.00026) | (0.0000) | (0.0000) | (0.0000) | (0.031) | (0.025) | (0.031) | (0.0033) | ||

50,000 | 1.20 | −2.82 | −1.00 | 3.70 | 0.00021 | 0.00093 | 0.000042 | 0.0014 | 0.43 | 0.00012 | 0.0029 | 0.00095 | −206.46 |

(0.31) | (0.46) | (0.27) | (0.54) | (0.0000) | (0.00024) | (0.0000) | (0.0000) | (0.0022) | (0.024) | (0.019) | (0.0000) | ||

60,000 | 1.16 | −2.84 | −0.97 | 3.68 | 0.00023 | 0.00089 | −0.000053 | 0.00067 | 0.41 | −0.00029 | 0.0017 | −0.00036 | −206.63 |

(0.33) | (0.37) | (0.30) | (0.53) | (0.00017) | (0.0000) | (0.00045) | (0.00045) | (0.016) | (0.040) | (0.032) | (0.00064) | ||

70,000 | 1.22 | −2.85 | −0.95 | 3.72 | 0.00011 | 0.00057 | −0.00021 | −0.0023 | 0.52 | −0.00034 | 0.0010 | −0.00087 | −206.75 |

(0.30) | (0.49) | (0.18) | (0.57) | (0.00022) | (0.0000) | (0.00023) | (0.00054) | (0.035) | (0.055) | (0.025) | (0.00026) | ||

80,000 | 1.21 | −2.91 | −0.93 | 3.73 | −0.00091 | −0.0034 | 0.0045 | −0.0032 | 0.51 | 0.00091 | 0.00024 | 0.00020 | −206.25 |

(0.40) | (0.41) | (0.15) | (0.49) | (0.00087) | (0.00095) | (0.00066) | (0.0037) | (0.023) | (0.026) | (0.014) | (0.00034) | ||

90,000 | 1.16 | −2.86 | −0.87 | 3.71 | −0.00034 | −0.0035 | 0.0054 | −0.0086 | 0.50 | −0.0027 | −0.0023 | 0.00041 | −206.94 |

(0.23) | (0.45) | (0.22) | (0.60) | (0.0013) | (0.0037) | (0.0034) | (0.0000) | (0.00031) | (0.029) | (0.026) | (0.00091) | ||

100,000 | 1.25 | −2.88 | −0.91 | 3.68 | −0.00042 | −0.0022 | 0.0031 | −0.0032 | 0.45 | −0.0022 | −0.00020 | 0.00034 | −206.98 |

(0.41) | (0.52) | (0.17) | (0.63) | (0.0014) | (0.00058) | (0.00010) | (0.0063) | (0.018) | (0.025) | (0.020) | (0.00062) |

log-likelihood

It is noted that in the autoregressive coefficients model, the estimates of

In other words, T is an identity matrix and D is a diagonal matrix with an identical element on diagonals, implying

By comparing the numerical results of the QMC estimation approach with the literature work, we find that they are quite close to those made by Gibbs sampling method. But the QMC method has much light computational loads and easy to use for practitioners. This is because Gibbs sampling method involves multiple draws of random samples and needs more experience, for instance, in specifying prior distributions of parameters, etc., whereas the QMC approximation method does not need such specifications. Also, Gibbs sampling may be very slow in obtaining the parameter estimates. On the other hand, the PQL approach gives very biased estimates of the variance components for modeling clustered binary data, as reported by many authors including Breslow and Clayton (1993) [

In this section, we carry out simulation studies to assess the performance of the proposed QMC estimation method in GLMMs. In the simulation studies we consider the logistic model (19) and use the same protocol of design as the real salamander mating experiment, resulting in 360 correlated binary observations. We run 100 simulations and calculate the average of the parameter estimates over the 100 simulations. The log-likelihood function for each simulated data set involves six 20-dimensional integrals that are analytically intractable.

We generate

From

We have studied the performance of using QMC approach to calculate the MLEs of the fixed effects and variance components in GLMMs with correlated random effects. We proposed to use Newton-Raphson algorithm to calculate the parameter estimates. The marginalized log-likelihood function that is in general analytically intractable is approximated well through the use of the simplest QMC points, i.e., square root sequences. We also addressed the issue of covariance modelling for random effects covariance matrix through a modified Cholesky decomposition. As a result, pre-specification of covariance structures for random effects is not necessary and misspecification of covariance structures is thus avoided. The score function and observed information matrix are calculated and expressed in analytically closed forms, so that the algorithm can be implemented straightforwardly. The performance of the proposed method was assessed through real salamander mating data analysis and simulation studies. Even though the marginalized log-likelihood function in the numerical analysis involves six 20-dimensional analytically intractable integrals, the QMC square root sequence approximation performs very well.

True | 1.20 | −2.80 | −1.00 | 3.60 | −1.20 | 0.60 | −0.10 | 0.40 |

QMC-gp | 1.16 | −2.82 | −0.93 | 3.65 | −1.21 | 0.55 | −0.12 | 0.41 |

StD | 0.04 | 0.12 | 0.08 | 0.17 | 0.000094 | 0.0027 | 0.0020 | 0.016 |

True | −2.00 | −1.00 | 0.50 | |||||

QMC-gp | −1.99 | −1.03 | 0.51 | |||||

StD | 0.0028 | 0.0017 | 0.0017 |

In general, a practical issue for the use of the QMC points is the choice of the number of integration nodes. Pan and Thompson (2007) [

We thank the editors and the reviewers for their comments. This research is funded by the National Social Science Foundation No.12CGL077, National Science Foundation granted No.71201029, No.71303045 and No.11561071. This support is greatly appreciated.

Yin Chen,Yu Fei,Jianxin Pan, (2015) Quasi-Monte Carlo Estimation in Generalized Linear Mixed Model with Correlated Random Effects. Open Access Library Journal,02,1-16. doi: 10.4236/oalib.1102002

The second-order derivatives of log-likelihood function is given by

The first and second derivative of

trix

Every element in

We can put the j^{th} column of C into a ^{th} matrix. For the j^{th} matrix,

have the relationship

The above relationship is a one-to-one correspondence between i and

are all lower triangular matrices. Denote

The first derivative can be calculated as

where

The second derivative can be calculated as

because of

where