ABSTRACT

F distribution is one of the most frequently used distributions in statistics. For example, it is used for testing: equality of variances of two independent normal distributions, equality of means in the one-way ANOVA setting, overall significance of a normal linear regression model, and so on. In this paper, a simple chi-square approximation for the cumulative distribution of the F- distribution is obtained via an adjusted log-likelihood ratio statistic. This new approximation exhibits remarkable accuracy even when the degrees of freedom of the F distribution are small.

Keywords:

Bartlett Correction, Homoscedasticity, Likelihood Ratio Statistic, One-Way ANOVA

1. Introduction

F distribution is one of the most frequently used distributions in statistics. It arises in many practical situations. For example, the test statistic for testing equality of variances of two independently distributed normal distributions is distributed as an F distribution. Another example is the test statistic for testing equality of means of k independent normal distributions with homogeneous variance is also distributed as an F distribution.

Johnson and Kotz [1] give a comprehensive review on the approximations to the cumulative distribution function (cdf) of the F distribution. Li and Martin [2] propose a shrinking factor approximation method and approximate the cdf of the F distribution by the cdf of the ${\chi }^2$ distribution. On the other hand, considering testing equality of variances of two independent normal distributions, Wong [3] derives the modified signed log-likelihood ratio statistic. As a result, a normal approximation for the cdf of the F distribution is obtained. The approximation by Wong [3] has a theoretical order of convergence $O\left(n^{-3/2}\right)$ .

In this paper, we consider the problem of testing equality of means of k independent normal distributions with homogeneous variance. Rather than the standard one-way ANOVA approach, we derive an adjusted log-likelihood ratio statistic, which is asymptotically distributed as ${\chi }^2$ distribution such that the mean of this adjusted log-likelihood ratio statistic is exactly the same as the mean of the ${\chi }^2$ distribution. As a result, a very accurate new ${\chi }^2$ approximation for the cdf of the F distribution is obtained.

2. Bartlett Corrected Log-Likelihood Ratio Statistic

Let $\left(Y_1\mathrm{,}\cdots \mathrm{,}{Y}_n\right)$ be identical independently distributed random variables with joint log-likelihood function $\mathcal{l}\left(\theta \right)$ , where $\theta$ is a p-dimensional vector parameter. A frequently used asymptotic method for testing the hypothesis

$H_0\mathrm{<mo>\; :\; </mo>}\psi \left(\theta \right)={\psi }_0\text{vs}{H}_a\mathrm{<mo>\; :\; </mo>}\psi \left(\theta \right)\ne {\psi }_0\mathrm{,}$ (1)

is based on the asymptotic distribution of the log-likelihood ratio statistic. In particular, the log-likelihood ratio statistic is defined as

$W=2\left\{\mathcal{l}\left(\widehat{\theta }\right)-\mathcal{l}\left(\tilde{\theta }\right)\right\}$

where $\widehat{\theta }$ is the unconstrained maximum likelihood estimator of $\theta$ , which is obtained by maximizing the log-likelihood function with respect to $\theta$ , and $\tilde{\theta }$ is the constrained maximum likelihood estimator of $\theta$ , which is obtained by maximizing the log-likelihood function with respect to $\theta$ subject to the constraint that $\psi \left(\theta \right)={\psi }_0$ . Generally, this constrained maximum likelihood estimator of $\theta$ can be obtained by the Lagrange multiplier method. With the regularity conditions stated in Cox and Hinkley [4] , it is well-known that W is asymptotically distributed as ${\chi }_r^2$ distribution, where r is the degrees of freedom, which is the difference in the number of unconstrained parameters being estimated and the number of constrained parameters being estimated. Hence, the observed level of significance for testing the hypothesis in (1) is $P\left({\chi }_r^2>w\right)$ , where w is the observed value of the log-likelihood ratio statistic W. Note that Cox and Hinkley [4] show that this method of obtaining the observed level of significance has order of convergence of only $O\left(n^{-1/2}\right)$ .

There exists many different ways of improving the accuracy of the convergence of the log-likelihood ratio statistic. Barndorff-Nielsen and Cox [5] and Brazzale et al. [6] give detail review of some higher order asymptotic methods and their applications. Recently, Davison et al. [7] derive a directional test for a vector parameter of interest for the linear exponential families. The method is quite complicated, both in terms of theories and computations.

In this paper, we propose a statistic, which is very similar to the Bartlett corrected log-ikelihood ratio statistic. Bartlett [8] [9] show that the expected value of W can be expressed as

$E\left(W\right)=r\left(1+\frac{b}{n}+O\left(n^{-2}\right)\right),$

where b is known as the Bartlett factor. Since $E\left(W\right)$ does not equal to the mean of the ${\chi }_r^2$ distribution, Bartlett [8] [9] propose to adjust the log-like- lihood ratio statistic by

$W^{*}=\frac{W}{1+\frac{b}{n}}$

such that $E\left(W^{*}\right)=r$ with rate of convergence $O\left(n^{-2}\right)$ . Lawley [10] shows that in fact all cumulants of $W^{\mathrm{<mo>\; *\; </mo>}}$ agree with those of a ${\chi }_r^2$ distribution to the same order. Lawley’s proof is very complicated. Barndorff-Nielsen and Cox [11] discuss a much simpler derivation based on the saddlepoint approximation. However, the Bartlett factor, b, in general, is very difficult to obtain. This limited the use of the Bartlett corrected log-likelihood ratio statistic in applied statistic.

In this paper, we propose to adjust the log-likelihood ratio statistic W such that the adjusted log-likelihood ratio statistic has exactly the same mean as the ${\chi }_r^2$ distribution. In other words, let

$W^{†}=\frac{W}{E\left(W\right)/r}.$ (2)

$W^{†}$ is asymptotically distributed as ${\chi }_r^2$ distribution. Thus, the observed level of significance for testing the hypothesis in (1) is $P\left({\chi }_r^2>w^{†}\right)$ , where $w^{†}$ is the observed value of $W^{†}$ . Note that his adjusted log-likelihood ratio statistic is just a modified version of the Bartlett corrected log-likelihood ratio statistic.

In the next section, the proposed adjusted log-likelihood ratio statistic for testing the equality of means of k homoscedastic normally distributed populations is derived. By comparing to the standard F-test in the one-way ANOVA approach, an approximation of the cdf of the F distribution is obtained.

3. Main Result

Let $X_{ij}$ be independent normally distributed random variables with mean ${\mu }_i$ and a common variance ${\sigma }^2$ , where $i=1,\cdots ,k$ and $j=1,\cdots ,n_i$ . Our aim is to test

$H_0:{\mu }_1=\cdots ={\mu }_k=\mu \text{vs}{H}_a\mathrm{<mo>\; :\; </mo>}\text{the}\text{means}\text{are}\text{not}\text{all}\text{the}\text{same}\mathrm{.}$ (3)

From the one-way ANOVA approach, we have the following sum of squares:

$\begin{array}{l}SST=SSTr+SSE\\ \iff {\displaystyle \sum _{i=1}^k}{\displaystyle \sum _{j=1}^{n_i}}{\left(X_{ij}-\overline{X}\right)}^2={\displaystyle \sum _{i=1}^k}{n}_i{\left({\overline{X}}_i-\overline{X}\right)}^2+{\displaystyle \sum _{i=1}^k}{\displaystyle \sum _{j=1}^{n_i}}{\left(X_{ij}-{\overline{X}}_i\right)}^2,\end{array}$

and the degrees of freedom are

$dfTr=k-1,dfE={\displaystyle \sum _{i=1}^k}{n}_i-k.$

For testing the hypothesis in (3), the F-test is used. Denote the test statistic as

$F^{*}=\frac{SSTr/dfTr}{SSE/dfE}.$ (4)

It is well-known that $F^{\mathrm{<mo>\; *\; </mo>}}$ is distributed as the F distribution with degrees of freedom $\left(dfTr\mathrm{,}dfE\right)$ . Hence, the observed level of significance for testing the hypothesis in (3) is $P\left(F_{dfTr,dfE}>f^{*}\right)$ with $f^{\mathrm{<mo>\; *\; </mo>}}$ being the observed value of $F^{\mathrm{<mo>\; *\; </mo>}}$ .

From the likelihood analysis point of view, let $\theta ={\left({\mu }_1,\cdots ,{\mu }_k,{\sigma }^2\right)}^{\prime }$ , and the log-likelihood function can be written as

$\mathcal{l}\left(\theta \right)=\mathcal{l}\left({\mu }_1,\cdots ,{\mu }_k,{\sigma }^2\right)={\displaystyle \sum _{i=1}^k}\left[-\frac{n_i}{2}\log {\sigma }^2-\frac{1}{2{\sigma }^2}{\displaystyle \sum _{j=1}^{n_i}}{\left(X_{ij}-{\mu }_i\right)}^2\right].$

It can be shown that the unconstrained maximum likelihood estimator is $\widehat{\theta }={\left({\widehat{\mu }}_1,\cdots ,{\widehat{\mu }}_k,{\widehat{\sigma }}^2\right)}^{\prime }$ , where

${\widehat{\mu }}_1={\overline{X}}_1,\cdots ,{\widehat{\mu }}_k={\overline{X}}_k,{\widehat{\sigma }}^2=\frac{SSE}{n_1+\cdots +n_k}.$

Therefore

$\mathcal{l}\left(\widehat{\theta }\right)=-\frac{n_1+\cdots +n_k}{2}\log {\widehat{\sigma }}^2-\frac{n_1+\cdots +n_k}{2}.$

When the null hypothesis in (3) is true, the log-likelihood function can be written as

$\mathcal{l}\left(\mu ,\cdots ,\mu ,{\sigma }^2\right)={\displaystyle \sum _{i=1}^k}\left[-\frac{n_i}{2}\log {\sigma }^2-\frac{1}{2{\sigma }^2}{\displaystyle \sum _{j=1}^{n_i}}{\left(X_{ij}-\mu \right)}^2\right],$

and the constrained maximum likelihood estimator is $\tilde{\theta }={\left(\tilde{\mu },\cdots ,\tilde{\mu },{\tilde{\sigma }}^2\right)}^{\prime }$ , where

$\tilde{\mu }=\overline{X},{\tilde{\sigma }}^2=\frac{SSTr}{n_1+\cdots +n_k}.$

Thus, we have

$\mathcal{l}\left(\tilde{\theta }\right)=\mathcal{l}\left(\tilde{\mu },\cdots ,\tilde{\mu },{\tilde{\sigma }}^2\right)=-\frac{n_1+\cdots +n_k}{2}\log {\tilde{\sigma }}^2-\frac{n_1+\cdots +n_k}{2}.$

Therefore, the log-likelihood ratio statistic is

$W=\left(n_1+\cdots +n_k\right)\log \frac{SSTr}{SSE}=\left(dfTr+dfE+1\right)\log \left(1+\frac{dfTr}{dfE}{F}^{*}\right),$

and W is asymptotically distributed as ${\chi }^2$ distribution with $dfTr$ degrees of freedom.

Our proposed method required to obtain $E\left(W\right)$ . Since $F^{\mathrm{<mo>\; *\; </mo>}}$ is distributed as F distribution with $\left(dfTr\mathrm{,}dfE\right)$ degrees of freedom,

$E\left(W\right)={\displaystyle {\int }_0^{\infty }}\left(dfTr+dfE+1\right)\log \left(1+\frac{dfTr}{dfE}y\right)g\left(y;dfTr,dfE\right)\text{d}y$ (5)

where $g\left(y\mathrm{<mo>\; ;\; </mo>}dfTr\mathrm{,}dfE\right)$ is the probability density function of the F distribution with degrees of freedom $\left(dfTr\mathrm{,}dfE\right)$ . Therefore, the observed level of significance for testing the hypothesis in (3) based on the proposed adjusted loglikelihood ratio statistic is

$P\left({\chi }_{dfTr}^2>\frac{\left(dfTr+dfE+1\right)\log \left(1+\frac{dfTr}{dfE}{f}^{*}\right)}{E\left(W\right)/dfTr}\right)$

where $E\left(W\right)$ is defined in (5) and $f^{\mathrm{<mo>\; *\; </mo>}}$ is the observed value of the test statistic given in (4).

By re-indexing the above approximation, let X be distributed as the $F_{u\mathrm{,}v}$ distribution, where $\left(u\mathrm{,}v\right)$ are the corresponding degrees of freedom. Then the cdf of X is $P\left(F_{u\mathrm{,}v}\le x\right)$ for $x>0$ . Hence, the log-likelihood ratio statistic is

$W=\left(u+v+1\right)\log \left(1+\frac{u}{v}X\right).$

Since W is asymptotically distributed as ${\chi }_u^2$ distribution, we have

$P\left(F_{u,v}\le x\right)\approx P\left({\chi }_u^2\le \left(u+v+1\right)\log \left(1+\frac{u}{v}x\right)\right).$

However, this approximation has order of convergence $O\left(n^{-1/2}\right)$ only.

The proposed approach gives

$W^{†}=\frac{W}{E\left(W\right)/u}=\frac{W}{b\left(u,v\right)}$

where

$\begin{array}{c}b\left(u,v\right)=\frac{E\left[\left(u+v+1\right)\log \left(1+\frac{u}{v}X\right)\right]}{u}\\ =\frac{{\displaystyle {\int }_0^{\infty }}\left(u+v+1\right)\log \left(1+\frac{u}{v}x\right)g\left(x;u,v\right)\text{d}x}{u}.\end{array}$

As a result,

$P\left(F_{u,v}\le x\right)\approx P\left({\chi }_u^2\le \frac{\left(u+v+1\right)\log \left(1+\frac{u}{v}x\right)}{b\left(u,v\right)}\right).$

Note that $b\left(u,v\right)$ does not have a closed form solution but it can be obtained numerically by software like R, Maple and Matlab. Table 1 records some values of $b\left(u,v\right)$ for $u\le v$ . Moreover,

$\underset{v\to \infty }{\lim }b\left(u,v\right)=1\text{and}\underset{u\to \infty }{\lim }b\left(u,v\right)=\infty .$

Hence, the proposed approximation will be problematic when u is large. Never- theless, the $F_{u\mathrm{,}v}$ distribution has the inverse property:

$P\left(F_{u\mathrm{,}v}\le x\right)=1-P\left(F_{v\mathrm{,}u}\le 1/x\right)$

that can be applied to obliviate this problem. Thus, the proposed approximation is:

$P\left(F_{u,v}\le x\right)=\{\begin{array}{l}P\left({\chi }_u^2\le \frac{\left(u+v+1\right)\log \left(1+\frac{u}{v}x\right)}{b\left(u,v\right)}\right)\text{if}u\le v\\ 1-P\left({\chi }_v^2\le \frac{\left(v+u+1\right)\log \left(1+\frac{v}{u}\frac{1}{x}\right)}{b\left(u,v\right)}\right)\text{if}u>v\end{array}$ (6)

4. Numerical comparisons

Wong [3] gives a simple and accurate normal approximation to the cdf of the $F_{u\mathrm{,}v}$ distribution, which has order of convergence $O\left(n^{-3/2}\right)$ . It takes the form

$P\left(F_{u,v}\le x\right)=\Phi \left(r-\frac{1}{r}\log \frac{r}{q}\right)$

where $\Phi (\cdot )$ is the cdf of the standard normal distribution,

$r=\mathrm{sgn}\left(x-1\right){\left\{\left(u+v\right)\log \frac{ux+v}{u+v}-u\log x\right\}}^{1/2}$

$q=\frac{x-1}{ux+v}{\left\{\frac{uv\left(u+v\right)}{2}\right\}}^{1/2}$

It is of interest to compare the proposed method, to the approximation by Wong [3] .

Figures 1(a)-8(a) are the plots of the cumulative distribution functions for

the $F_{u\mathrm{,}v}$ distribution for various u and v obtained by the exact method, the

approximation by Wong [3] , and the proposed method. The difference between the two approximated cumulative distribution functions and the exact cumu- lative distribution function are barely noticeable. To explore the accuracy of the two approximations, we examine the relative error, which is defined as

$\text{relative}\text{error}=\frac{\text{approximation}-\text{exact}}{\text{exact}}.$

Figures 1(b)-8(b) are the plots of the corresponding relative errors. It is clear that the proposed method generally outperformed the approximation by Wong [3] in all cases.

5. Conclusion

In this paper, a simple chi-square approximation to the cumulative distribution function of the F-distribution is obtained via an adjusted log-likelihood ratio statistic. Simulation studies illustrated that the new approximation outperformed the higher-order asymptotic method discussed in Wong (2008), regardless of how show the degrees of freedom are.

Cite this paper

Jiang, L. and Wong, A. (2018) A Chi-Square Approximation for the F Distribution. Open Journal of Statistics, 8, 146-158. https://doi.org/10.4236/ojs.2018.81010

References