### Journal Menu >>

 Applied Mathematics, 2011, 2, 1258-1262 doi:10.4236/am.2011.210175 Published Online October 2011 (http://www.SciRP.org/journal/am) Copyright © 2011 SciRes. AM Convergence Rates of Density Estimation in Besov Spaces Huiying Wang Department of Ap pl i e d M athematics, Beijing University of Technology, Beijing, Ch ina E-mail: b200806005@emails.bjut.edu.cn Received July 29, 201 1; revised August 23, 2011; accepted August 30, 2011 Abstract The optimality of a density estimation on Besov spaces ,srqBR for the risk was established by Donoho, Johnstone, Kerkyacharian and Picard (“Density estimation by wavelet thresholding,” The Annals of Statistics, Vol. 24, No. 2, 1996, pp. 508-539). To show the lower bound of optimal rates of conver-gencepL,,snrqRBp, they use Korostelev and Assouad lemmas. However, the conditions of those two lemmas are difficult to be verified. This paper aims to give another proof for that bound by using Fano’s Lemma, which looks a little simpler. In addition, our method can be used in many other statistical models for lower bounds of estimations. Keywords: Optimal Rate of Convergence, Density Estimation, Besov Spaces, Wavelets 1. Introduction Wavelet analysis has many applications, one of which is to estimate an unknown density function based on inde-pendent and identically distributed (i.i.d.) random sam-ples. Let be a probability measurable space and 1(,,)P,,nXX be i.i.d. random variables with an un-known density function f. We use to denote the expectation of a random variable X. The sequence EX,:inf supnnnpffVRVpE ff is called optimal rate of convergence on the functional class V for the pL risk. Here, nf is an arbitrary estimator of f with n i.i.d. ran-dom samples. Kerkyacharian and Picard [1] study n when V is a Besov space with matched case. Donoho, Johnstone, Kerkyacharian and Picard [2] con-sider unmatched cases. In fact, they show the optimal convergence rates for Besov class ,pRVrL,srqB and pL risk 1/ 1/21/1,21ln, ,21,,.21sr psrsnrq sspnnr sRB ppnrs (1.1) To show the lower bou nd of (1.1), author s of [2,3] use Korostelev and Assouad lemmas. However, the condi-tions of those two lemmas are difficult to be verified. In this small paper, we give another proof for the lower bound of (1.1) by using Fano’s lemma [4]. It should be pointed out that Fano’s lemma can be used to a variety of statistical models, see [5-7]. As usual, 1pLRp denotes the classical Lebes-gue space on the real line R. In particular, 2LRR stands for the Hilbert space, which consists of all square inte-grable functions. As a subspace of p, the Sobolev space with an integer exponent k means L:,,0,1,,1mkppWRffLRmkp . The corresponding norm :.kpkWppfff Moreover, the Besov space , [3] , spqBR (1 pq, sn and (0,1]) can be defined by  2,,2 ,2nsnjjpqppjZBR fWRfl q with the associated norm ,2():2,2snpq pqjnjpBW lZff f , where 2,:sup2 2.phtpftfxhfxh fx In general, it can be shown that compactly supported and n times differentiable functions belong to ,spqBR for H. Y. WANG 1259Where p and q are density functions of P Q respec-tively. Lemma 1.2. (Fano’s Lemma, [4]) Let be pr, 0sn and 1, . pqThe Besov space can be discretized by the sequence norm of wavelet coefficients. Many useful wavelets are generated by scaling functions. More precisely, if  is a scaling function with  22 ,kkxhxk then122k:1kkxhxk defines a wave-let [3]. Clearly, when  is compactly supported and continuous, the corresponding wavelet  has the same properties. An orthonormal wavelet basis of 2LR is generated from dilation and translation of a scaling func-tion and its corresponding wavelet, i.e. 00022,:2 2,:2 2.jjjj0jkjkjjkZxxkxxk Although wavelet basis are constructed for 2LR, most of them constitute unconditional bases for pLR. A scaling function  is called t regular, if  has con-tinuous derivatives of order t and its corresponding wavelet  has vanishing moments of order t, i.e. kxxd0, 0,1,,1x kt. The following lemma [3] plays important roles in this paper. Lemma 1.1. Let  be a compactly supported, t regu-lar orthonormal scaling function with the corresponding wavelet  and0st. IfpfLR, 00kk :,,sf:,jkjk and 1, dfpq, then the following two conditions are equivalent: 1) ,spqfBR; 2) 112002js pjppjqsd. Furthermore, ,112002spqjs pjBppjqfsd. Before introducing Fano’s Lemma, we need the nota-tion of Kullback-Leilber distance [4]. Let P and Q with P being absolutely continuous with respect to Q (denoted by ). Then the Kullback-Leilber distance is de-fined by PQ 0,:ln dpqpx(,, )kPobability measurable spaces and kA, 0,1, ,km. If kA,KPQp xxqx vAkv, then with cA standing for for the complement of A and ,kvv01:infmvm kKPP, m 11,exp 3.2mm e0sup minckkkmPA By Lemma 1.1 and 1.2, we can show the following re-sult: Theorem 1.1. Let ,,srqfBRL with, 1rq, 1p and 1sr . If nf is an estimator of f with n i.i.d. random samples, then ,1/ 1/21/1 21,lnsupmax, ,srqsr pssr snpfB RLnEf fn  n where  ,,,,:, srqssrqrq BBRL fBRfL f and xy means has compact support}; The notation with a constant C. rk 1.1. Note that xCy Rema 1/ 1/21/1ln 1/ 1/21/121 lnmax ,srp pssrnn srsrsnnn  for 21prs and for 21prs 1/ 1/1/1 21 21max ,srp2lnsssrssnnn . Then theorem 1.1 is a reformulate of the loweund in (1.1). By using the idea of reference [5], we show this theorem in the next two sections. irstly, we prove nr bo 2. Proof of Theorem 1.1 F1/ 1/21/1ln .s,,supsrqrpsrnpnf fn One need constructfB RLE such that ,skr kg,qgBRL and 1/ 1//1.21lnsupsrprnkpkn Let snEf g ono be a compactly supported, regular and orthrmal scaling function, tt s be the corresponding wavlet with suppe0,l, lN. Here and after, Copyright © 2011 SciRes. AM 1260 dgers. ThH. Y. WANG N enotes the set of positive inteen there ex-ists a compactly supported density function g (i.e. 0gx and g) satisfying  ,srqd1x xgxBR and 00, 0.lgx c ,2jlll l. ThenLet elem,2,,21j:0,jents in the number of jenoteed by [ is 2j5], one defi nes 1, dd by1. #2jj 2 1/Motivat 1/:2jsr and ja 2jkjjkklgx gxxIk:, ja:1 if 2with 2Ijkljkl, else 2:0jklI. Obvi-ously, 2jlgg, gx xd1 and 1/s rkgx  for large implies that k020c jj, whichg his a density function e assumptions of for each k. By t, the wavelet  is com-pported and pactly susrqBt times differentiable. Therefore,  Rts and ,.skrq,gBRBec ause1/2 1/21js rja , ,srqjjkBaC and so is ,srqkBg Hence, due to Lemma 1.1. ,skrq,gxBRL. Clearly, 21/ 1/2:jkk kjjkplppjsprjpgggga   (2.1) For 2jjkk l due to :21/2 1/jsjar. Fur- thermore, :2jknkpAfg satisfies kkAA . Recall that 1. By Lemmfor kk #2ja 1.2, j 213supmin,2 expjkc jgk. Here and, 2jke afternPA nfP thstands for the peasure correspond-robability moe density function ing t2n1:nfxfxfnn xfx. It is easy to see that 0kggPP from the constructions of kg. Since nf is an estimator of density with n i.i.d. ra ndom samp le s, 2ikjjjnncnkg nkgkppEfgPfgP A . 22Then, 2supsup 213min,2 exp.22kjk jjjncnk gkpkjjEfgP Ae (2.2) Next, one shows1202jjcna 12112 102,:ln dnnnnn nnfffxKPPf xxfx, 11nnjfx 1jfx and 212nnjjfxfx. Then  11112 11212,lnd,inn iiiifx: Recall that .nKPPfxx nKPPfxNote that  11112 12,:ln dfxPP fxxfxln 1uuK and for en 0 Thu.   1121 211212212,lnd1dd.nn fxKP Pnfxxfxfxnfx xfxnfxfx fxx Hence, 22:inf 2,2,.jkvk jljjjnn jnngg ggvkv kKP PKPP  Moreover,  1222djjjkkngxgxgx .x (2.3) ng to the definition of Accordikg, supp 0,kggl and 0gxc on 0,l. Thus,  12kgxgxg x 2211212002d0jjkj jkjxxc axca by the orthonormality of dx ca. Then (2.3) reduces to jk1202.jjcna (2.4) ke Ta1212ln/1 112222ljs rjna nn.srn. Thenjnn Now, one can choose such that lnC n 0C2jnaand 041Csr c2. Therefore, 41/ 21sr cn 111002222 lnjcCjjjneenanan ) reduces to d (2.2jEfsupknk jpg C n . Thethe desired follows from 1/21/jspp ) rj by (2.1and 121/12lnsrjnn. ,21,supsrqssnpRLEf fnNow, we prove fB. Our proof depends on another lemma [4]. Copyright © 2011 SciRes. AM H. Y. WANG 1261ov-Gilbert) Let Lemma 2.1. (Varsham1:,,m , 0,1i. Then there exists a sub-set 0,,M of /8m with 00, ,0 such that 2M and 10.8mijmkkkijM  construct It is sufficient to0,1, ,igiM such that ,,isrqgBRL and 21.isnpn (2.5) supiEf gAs proove, letsved ab  be a compactly supported, tt s regular on,and orthormal scaling functi on be uppthe corresponding wavelet with sN0,l, l. s sAssume,,rqgBR1/2 , :j:L and ne :ja0,,2,ll [0, ]0|0lgc1l and . Defi2js ,2jjjkjkkgxgxa x with 0,1 2jjkk (note that 0gg). Since 0,1k, one knows that 2jrjkk  nd a 1/1/2 1/21jjrrrkkaBy Lemma 1.1,.js ,sjrqkjkBjkaC, and so is ,srqBg. Hence ,s,rqgBRL. at the supports ofNote th jk for jk are mutu-ally disjoint. Then 0020jsjjkgxc ac  for big j. This with1  ddgxx gxx implies that g is a density 20,1function for each j1, there exists . Ac-cording to Lemma 2.01,,,M such that 322jM and 32.li jk (2.6) jkk Because suppjksupp jk for jkk, one knows that 12li jpppl ijk kkpjpjkpppsp jlikkkpgg a  This with (2.6) and ,lk 0,1ik leads to 32li pjpg and 2ppspg1/82:li psjjppgg . (2.7)Clearly, the sets 0,1,,2iijnAfgi M  satisfy liAA for . Then Fano’s Lemma yiilelds 101supmin,expM3.2iincgMiMPA e (2.8) On the other hand, it follows 1220jMcf of (2.4). Take jna from the similar arguments to the proo1212jsn. Then choose a c0 suhat 2(21)21sjjnaa . Hence, one can onstant ch tC12 1440022222e2e1jjjjjMcna cCMe .Therefore, (2.8) reduces to 0sup iincgiMPA C0 and 0supinpiMEfg0sup .22jiM Ciijjngn pPfg This with (2.7) and 1212jsn yield (2.5). 3. The author Huiying Wang is grateful to the referees for their valuable comments and thanks her advisoProfes-sor Youming Liu, for his helpful guidance. Th work is supported by the National Natural Scien ce Foundation of China (No. 10871012) and Natural Science Foundation of Beijing (No. 1082003) . ] G. Kerkyacharian and D. Picard, “Density Estimation in Acknowledgements r, is4. References [1 Besov Spaces,” Statistics & Probability Letters, Vol. 13, No. 1, 1992, pp. 15-24. doi:10.1016/0167-7152(92)90231-S [2] D. L. Donoho, I. M. Johnstone, G. Kerkyacharian and D. Picard, “Density Estimation by Wavelet Thresholding,” The Annals of Statistics, Vol. 24, No. 2, 1996, pp. 508- 539. doi:10.1214/aos/1032894451 Kerkyacharian, D. Picard and A. B. Tsy-lets, Approximation and Statistical Appli-cations,” Springer-Verlag, New York, 1997. mir Zaiats, Springer rk, 2009. [3] W. Härdle, G.bakov, “Wave[4] A. B. Tsybakov, “Introduction to Nonparametric Estima-tion,” (English) Revised and Extended from the 2004 French Original, Translated by VladiSeries in Statistics, Springer, New Yo[5] P. Baldi, G. Kerkyacharian, D. Marinucci and D. Picard, “Adaptive Density Estimation for Directional Data Using Needlets,” The Annals of Statistics, Vol. 37, No. 6A, Copyright © 2011 SciRes. AM H. Y. WANG Copyright © 2011 SciRes. AM 1262 9-AOS6822009, pp. 3362-3395. doi:10.1214/0 05.010[6] C. Christophe, “Regression with Random Design: A Minimax Study,” Statistics & Probability Letters, Vol. 77,No. 1, 2007, pp. 40-53. doi:10.1016/j.spl.2006. [7] A. B. Tsybakov, “Optimal Rates of Aggregation,” COLT/ Kernel 2003 Lecture Notes in Artificial Intelligence 2777, Springer, Heidelberg, 2003, pp. 303-313.