^{1}

^{*}

^{1}

^{*}

In this paper two kernel density estimators are introduced and investigated. In order to reduce bias, we intuitively subtract an estimated bias term from ordinary kernel density estimator. The second proposed density estimator is a geometric extrapolation of the first bias reduced estimator. Theoretical properties such as bias, variance and mean squared error are investigated for both estimators. To observe their finite sample performance, a Monte Carlo simulation study based on small to moderately large samples is presented.

Many efforts have been devoted to investigating the optimal performance of kernel density estimator since it has been the most widely used nonparametric method in the last decades. Suppose we use

There have been numerous literatures that discuss approaches to improving the performance of kernel estimators, while reducing the bias has been the most commonly considered one. Article [

asymptotic convergence rate

metric extrapolation of nonnegative kernels, while [

Although the variance reduction method is not as approachable as the bias reduction method, there still have been a lot of scholars working on it. Article [

Many of above mentioned bias reduction methods result in complex kernel density estimators. In this paper, we introduce a novel but intuitive and feasible bias reduced kernel density estimator. In Section 2, we present the bias reduced estimator and investigate its asymptotic bias, variance and MSE. A second estimator is proposed and studied in Section 3 as a geometric extrapolation of the bias reduced kernel. To examine the finite sample performance of both estimators, a simulation study is carried out in Section 4. Finally some remarks are given in Section 5.

Kernel density estimator was first introduced in [

To make the estimator meaningful, the kernel function is usually required to satisfy conditions

and

Then from (2.2) and (2.3) we have

We can easily see that the optimized bandwidth is

In order to reduce the bias of ordinary kernel density estimator, we can intuitively subtract the leading bias

term

One could use any type of estimation of the bias term. We could simply replace f with the kernel estimator f_{n} since it is readily available. As a result, our proposed estimator is

From the way of construction, this new estimator should be able to reduce the bias and thus the MSE. To see whether this is the case or not, we next calculate the bias and the variance of

1)

2)

3)

Theorem 2.1. Under 1), 2) and 3),

and

Consequently,

and the optimal MSE is of the order

Proof. By Taylor expansion we have

Thus we have

On the other hand,

Note that (2.7) gives

Finally (2.9) and (2.10) together with (2.3) gives (2.6). ,

Remark 2.1. From Theorem 2.1 we can see that if K is symmetric, i.e.

case, the optimal MSE is further reduced to

Remark 2.2. From the definition of

where

sult

Geometric extrapolation was introduced in kernel density estimation by [

Suppose the kernel function K above is symmetric so that all the odd moments of K are zero. Article [

Note that

Instead of using the ordinary kernel estimator, we propose to use the bias reduced kernel estimator, presented in Section 2, in the construction of geometric extrapolated kernel (3.1). Denote the bias reduced kernel estimator with two bandwidths h and 2h as

Now the geometric extrapolated kernel estimator with bias reduction is proposed as

Since the bias reduced kernel estimator has improved bias and MSE over the ordinary kernel estimator, especially when K is symmetric, we expect that with geometric extrapolation it will achieve further improvement.

Theorem 3.1. Under 1), 2) and 3),

and

Consequently,

and the optimal MSE is of the order

Proof. We calculate

Let

where

Taking logarithm of (3.3) gives

Here we want to construct a geometric extrapolated kernel estimator of the form

The solution to above equation system is

and a series expansion for exponential function gives

We rewrite

where U and V are both of order

and then

Since

,

Remark 3.1. Article [

optimal MSE of the order

Remark 3.2. When K is symmetric, we propose another estimator

This estimator reduces the bias to

In this section, we carry out a simulation study designed to demonstrate the finite sample performance of the proposed bias reduced kernel estimator (BRK)

Without loss of generality, we suppose f is the standard normal density. We randomly select 1000 independent samples of size n = 20, 50, 100 or 200. We choose arbitrarily the points x = 0, 0.5, 1, 1.5, 2, 2.5 and 3 at which the kernel estimators are calculated and compared. Since the properties of kernel estimators do not depend much on which particular kernel is used, we choose the standard normal as the kernel function K without loss of generality. For the bandwidth h, we use the optimal one for each individual kernel estimator. In another word, since here K is symmetric, by Remarks 2.1 and 3.2, we choose

and

where

lation results are presented in Tables 1-7.

From Tables 1-7 we can see that BRK consistently has smaller bias and MSE than OK except for x = 1. This is simply due to the fact that

. Bias, variance and MSE of different kernel density estimators evaluated at x = 0

Sample size | Kernel estimator | Bias | Variance | MSE |
---|---|---|---|---|

20 | OK | −0.051923 | 0.003405 | 0.006101 |

BRK | −0.028022 | 0.003849 | 0.004644 | |

GEOK | −0.036122 | 0.003106 | 0.004411 | |

GEBRK | −0.027881 | 0.003158 | 0.003935 | |

50 | OK | −0.036192 | 0.002127 | 0.003437 |

BRK | −0.017092 | 0.002105 | 0.002397 | |

GEOK | −0.025861 | 0.001686 | 0.002355 | |

GEBRK | −0.018000 | 0.001620 | 0.001944 | |

100 | OK | −0.028514 | 0.001245 | 0.002058 |

BRK | −0.012703 | 0.001132 | 0.001294 | |

GEOK | −0.021273 | 0.000885 | 0.001338 | |

GEBRK | −0.014114 | 0.000829 | 0.001028 | |

200 | OK | −0.022334 | 0.000815 | 0.001314 |

BRK | −0.009988 | 0.000691 | 0.000791 | |

GEOK | −0.017786 | 0.000531 | 0.000847 | |

GEBRK | −0.011795 | 0.000491 | 0.000630 |

. Bias, variance and MSE of different kernel density estimators evaluated at x = 0.5

Sample size | Kernel estimator | Bias | Variance | MSE |
---|---|---|---|---|

20 | OK | −0.035504 | 0.003537 | 0.004798 |

BRK | −0.015563 | 0.004579 | 0.004821 | |

GEOK | −0.021755 | 0.003326 | 0.003800 | |

GEBRK | −0.015037 | 0.003978 | 0.004204 | |

50 | OK | −0.022483 | 0.002055 | 0.002560 |

BRK | −0.007345 | 0.002321 | 0.002375 | |

GEOK | −0.013747 | 0.001689 | 0.001878 | |

GEBRK | −0.007874 | 0.001904 | 0.001966 | |

100 | OK | −0.017031 | 0.001308 | 0.001598 |

BRK | −0.005084 | 0.001334 | 0.001360 | |

GEOK | −0.011120 | 0.000978 | 0.001102 | |

GEBRK | −0.006062 | 0.001048 | 0.001085 | |

200 | OK | −0.013480 | 0.000816 | 0.000998 |

BRK | −0.003789 | 0.000793 | 0.000807 | |

GEOK | −0.009041 | 0.000569 | 0.000650 | |

GEBRK | −0.004779 | 0.000605 | 0.000628 |

. Bias, variance and MSE of different kernel density estimators evaluated at x = 1

Sample size | Kernel estimator | Bias | Variance | MSE |
---|---|---|---|---|

20 | OK | −0.003146 | 0.003591 | 0.003601 |

BRK | 0.006564 | 0.005192 | 0.005236 | |

GEOK | 0.007186 | 0.003470 | 0.003522 | |

GEBRK | 0.008243 | 0.004727 | 0.004795 | |

50 | OK | −0.000313 | 0.002001 | 0.002001 |

BRK | 0.006093 | 0.002522 | 0.002560 | |

GEOK | 0.007797 | 0.001668 | 0.001729 | |

GEBRK | 0.007741 | 0.002177 | 0.002237 | |

100 | OK | −0.000609 | 0.001197 | 0.001197 |

BRK | 0.004084 | 0.001369 | 0.001385 | |

GEOK | 0.006424 | 0.000926 | 0.000967 | |

GEBRK | −0.058670 | 0.001150 | 0.001184 | |

200 | OK | −0.000124 | 0.000661 | 0.000661 |

BRK | −0.003466 | 0.000698 | 0.000710 | |

GEOK | 0.005702 | 0.000487 | 0.000520 | |

GEBRK | 0.005187 | 0.000581 | 0.000608 |

. Bias, variance and MSE of different kernel density estimators evaluated at x = 1.5

Sample size | Kernel estimator | Bias | Variance | MSE |
---|---|---|---|---|

20 | OK | 0.019396 | 0.002676 | 0.003052 |

BRK | 0.017134 | 0.003727 | 0.004021 | |

GEOK | 0.025892 | 0.002580 | 0.003250 | |

GEBRK | 0.019406 | 0.003468 | 0.003845 | |

50 | OK | 0.013847 | 0.001334 | 0.001525 |

BRK | 0.010788 | 0.001646 | 0.001763 | |

GEOK | 0.020440 | 0.001163 | 0.001581 | |

GEBRK | 0.013526 | 0.001486 | 0.001669 | |

100 | OK | 0.020478 | 0.000747 | 0.000857 |

BRK | 0.007408 | 0.000848 | 0.000903 | |

GEOK | 0.016685 | 0.000601 | 0.000879 | |

GEBRK | 0.010161 | 0.000745 | 0.000848 | |

200 | OK | 0.008525 | 0.000441 | 0.000514 |

BRK | 0.005869 | 0.000460 | 0.000494 | |

GEOK | 0.014117 | 0.000324 | 0.000523 | |

GEBRK | 0.008425 | 0.000389 | 0.000460 |

. Bias, variance and MSE of different kernel density estimators evaluated at x = 2

Sample size | Kernel estimator | Bias | Variance | MSE |
---|---|---|---|---|

20 | OK | 0.020056 | 0.001439 | 0.001862 |

BRK | 0.012293 | 0.001680 | 0.001831 | |

GEOK | 0.026015 | 0.001280 | 0.001957 | |

GEBRK | 0.014823 | 0.001515 | 0.001735 | |

50 | OK | 0.015395 | 0.001068 | 0.000912 |

BRK | 0.006896 | 0.000758 | 0.000806 | |

GEOK | 0.019464 | 0.001280 | 0.001957 | |

GEBRK | 0.008646 | 0.000670 | 0.000745 | |

100 | OK | 0.012046 | 0.000390 | 0.000536 |

BRK | 0.004979 | 0.000412 | 0.000437 | |

GEOK | 0.015909 | 0.000297 | 0.000550 | |

GEBRK | 0.006350 | 0.000349 | 0.000390 | |

200 | OK | 0.009334 | 0.000230 | 0.000317 |

BRK | 0.003604 | 0.000227 | 0.000240 | |

GEOK | 0.013159 | 0.000164 | 0.000337 | |

GEBRK | 0.004859 | 0.000189 | 0.000213 |

. Bias, variance and MSE of different kernel density estimators evaluated at x = 2.5

Sample size | Kernel estimator | Bias | Variance | MSE |
---|---|---|---|---|

20 | OK | 0.016515 | 0.000460 | 0.000732 |

BRK | 0.008864 | 0.000469 | 0.001831 | |

GEOK | 0.014181 | 0.000533 | 0.000734 | |

GEBRK | 0.010294 | 0.000522 | 0.000633 | |

50 | OK | 0.011925 | 0.000206 | 0.000348 |

BRK | 0.002861 | 0.000215 | 0.000224 | |

GEOK | 0.009428 | 0.000260 | 0.000348 | |

GEBRK | 0.003742 | 0.000239 | 0.000253 | |

100 | OK | 0.009607 | 0.000106 | 0.000199 |

BRK | 0.001155 | 0.000118 | 0.000119 | |

GEOK | 0.007231 | 0.000140 | 0.000193 | |

GEBRK | 0.001456 | 0.000134 | 0.000135 | |

200 | OK | 0.007813 | 0.000057 | 0.000118 |

BRK | 0.000254 | 0.000066 | 0.000066 | |

GEOK | 0.005655 | 0.000082 | 0.000115 | |

GEBRK | 0.000566 | 0.000078 | 0.000079 |

. Bias, variance and MSE of different kernel density estimators evaluated at x = 3

Sample size | Kernel estimator | Bias | Variance | MSE |
---|---|---|---|---|

20 | OK | 0.006430 | 0.000166 | 0.000207 |

BRK | 0.011847 | 0.000197 | 0.000338 | |

GEOK | 0.007738 | 0.000132 | 0.000192 | |

GEBRK | 0.009690 | 0.000163 | 0.000256 | |

50 | OK | 0.004081 | 0.000076 | 0.000093 |

BRK | 0.005342 | 0.000067 | 0.000096 | |

GEOK | 0.005354 | 0.000055 | 0.000084 | |

GEBRK | 0.003942 | 0.000055 | 0.000070 | |

100 | OK | 0.002957 | 0.000038 | 0.000047 |

BRK | 0.002107 | 0.000030 | 0.000035 | |

GEOK | 0.004131 | 0.000026 | 0.000043 | |

GEBRK | 0.001226 | 0.000024 | 0.000026 | |

200 | OK | 0.002289 | 0.000021 | 0.000027 |

BRK | 0.000900 | 0.000016 | 0.000017 | |

GEOK | 0.003296 | 0.000014 | 0.000025 | |

GEBRK | 0.000157 | 0.000013 | 0.000013 |

conclusion cannot be generalized. When the two estimators with geometric extrapolation are compared, GEBRK generally has smaller bias and MSE than GEOK, especially when sample size is large. When BRK and GEBRK are compared, GEBRK tends to have smaller variance and MSE but larger bias than BRK. In terms of bias, BRK and GEBRK perform much better than OK and GEOK while BRK and GEBRK are very competitive. Geometric extrapolation reduces the variance and MSE in general, i.e. GEOK and GEBRK perform better than OK and BRK in terms of variance and MSE. When MSE is concerned, GEBRK performs best and then GEOK. These observations are somehow different at point x = 1 due to the fact that

In this paper, we first propose a very intuitive and feasible kernel density estimator which reduces the bias and MSE significantly compared with the ordinary kernel density estimator. Secondly, we construct a geometric extrapolation of the bias reduced kernel estimator which further improves the convergence rates of both bias and MSE. Our simulation study shows that for finite sample size both estimators perform competitively well and better than the ordinary kernel estimator and its geometric extrapolation.

For the bias reduced kernel density estimator presented in Section 2, we may find that part of the curve is under zero, especially at the tails. Taking standard normal density as an example, at point x = 4 the estimator may give a negative value. Apparently, this is unreasonable. Though in Remark 2.2 we suggest a modified version of the estimator, further work is necessary to deal with this problem.

The authors acknowledge with gratitude the support of this research by Discovery Grants from National Sciences and Engineering Research Council (NSERC) of Canada, and would like to thank the anonymous referees for their constructive comments.