Likelihood Ratio and Strong Limit Theorems for the Discrete Random Variable

doi:10.4236/ojdm.2012.24034

Paper Menu >>

Journal Menu >>

Open Journal of Discrete Mathematics, 2012, 2, 169-172

http://dx.doi.org/10.4236/ojdm.2012.24034 Published Online October 2012 (http://www.SciRP.org/journal/ojdm)

Likelihood Ratio and Strong Limit Theorems for the

Discrete Random Variable

Wenhan Li1, Wei Wang1, Zhiqiang Liu2

1College of Mathematics and Physics, Shijiazhuang University of Economics, Shijiazhuang, China

2Science College, Beijing University of Civil Engineering and Architecture, Beijing, China

Email: lwh88@163.com, zhiqliu@163.com

Received June 26, 2012; revised July 26, 2012; accepted August 26, 2012

ABSTRACT

This in virtue of the notion of likelihood ratio and the tool of moment generating function, the limit properties of the

sequences of random discrete random variables are studied, and a class of strong deviation theorems which represented

by inequalities between random variables and their expectation are obtained. As a result, we obtain some strong devia-

tion theorems for Poisson distribution and binomial distribution.

Keywords: Likelihood Ratio; Strong Limit Theorem; Moment Generating Function

1. Introduction

The analytic technique was proposed by Liu [1] to study

the strong deviation theorems represented by inequalities,

many comprehensive works may be found in Liu [2] and

references therein. In references, Liu [3] discussed the

strong deviation theorems for discrete random variables

by using the generating function method. Liu [4] ob-

tained some strong limit theorems for a multivariate

function sequence of discrete random variables by using

the concept of the conditional moment generating func-

tion and the concept of measure differentiation on a net.

Yang and Yang [5] established a strong limit theorem of

the Dubins-Freedman type for arbitrary stochastic se-

quences. Li, Chen and Zhang [6] and Li, Chen and Feng

[7] studied the strong limit theorems of arbitrary de-

pendent continuous random variables by using the ana-

lytic technique and the Laplace transform approach, and

further extended the strong deviation theorems to the

differential entropy for arbitrary dependent continuous

information sources in more general settings. Yang [8]

further studied the limit properties for Markov chains in-

dexed by a homogeneous tree through the analytic tech-

nique.

The purpose of this paper is to establish a kind of

strong limit theorems represented by inequalities with

random bounds, and to extend the analytic technique

proposed by Liu [9]. In the proof, the approach of apply-

ing the tool the notions of likelihood ratio and of moment

generating function to the study of strong limit theorem

of the sequences of random discrete variables is proposed.

As a result, we obtain some strong deviation theorems

for Poisson distribution and binomial distribution.

Let







 be a probability space, and let







 which is defined in



P  be a se-

quence of random variables taking values in





2,1,S with the joint distribution



 

11 1

,,,, ,

nn nn

PXxXxf xx

xS kn







(1)

Let





kk kkk

PXxpxxS k



 (2)

In order to indicate the deviation between









and a sequence of independent random variables with the

joint distribution

 

nn k







x

(3)

we first introduce the following definition.

Definition 1. Let





1Xn



 be a sequence of ran-

dom variables with the joint distribution (1), and





πnn



 be defined by (3), let



 

π,

log

XX X













 (4)

The random variable

 

limsup n





 (5)

is called the limit logarithmic likelihood ratio, relative to

the product of marginal distribution of (3), of ,

Xn

W. H. LI ET AL.

170

where is the natural logarithm,

log



is the sample

point, and k

stands for







Although





is not a proper metric between prob-

ability measures, we nevertheless think of it as a measure

of “dissimilarity” between their joint distribution

1nn



xx and the product nn





x of their

marginals. Obviously,







if and only if



x 



n











and it will be shown in (21) that , a.s. in any

case. Hence,





can be used as a random measure

of the deviation between the true joint distribution

1nn



xx

and the distribution (3) of independent

type. In view of the above discussion of the limit loga-

rithmic likelihood ratio, it is natural to think of







a measure how far (the random deviation of) n

from being independent, how dependent they are. The

smaller





is, the smaller the deviation is.

Definition 2. Let n be a sequence of random

variables taking values in S, and ,

be the marginal distribution of n





1





12k n 



x



x , let mo-

ment generating funct ion and expect at i on as follows:

 

Mt i









m ip











(6)

and

EX (7)

In this paper, we assume that there exists , such

that

00t







00 12

tttt 







k



2. Main Results

Theorem 1. Let n be a sequence of random

variables taking values in S,



Xn



i







be given as above, and





t is defined in







,

let













P 1D (8)

Then



Xm





xt







t



1a s

n









lim

n inf



 (9)

where



sup 0xt



 (10)











lim inf

Mt xt





 







m

 (11)

and then





0lim0 0









 (12)

Proof. For arbitrary





ttt



 , let









eit

kk k

tip iMt iS



 (13)

Then



gti







(14)

Let





 



11 1

exp

kkkk k

nn n

kkk

kk k

qtx x

txp xMt

tx pxMt



 

 







 





 





 



(15)

By (14), it is easy to see that is an n

multivariate distribution function, let



qtx x n







nnn

qtX X

tt fX X













 (16)

By [10], we have that







 is a nonnegative su-

permartingale by Doob’s martingale convergence theo-

rem,









converges a.s. an integrable random vari-

able T. Hence there exists



tF such that









1tPA



So we have





lim n

ntt At









  (17)

This implies that

 

lim supln0

ntt At



   (18)

By (4), (15) and (18), we obtain

 



lim suplnln0

kkn

nkk

Mtt Xr



 













 

(19)

Let 0t



, we obtain

 

lim infln00



  A (20)

that is







 



0A (21)

Let 00tt



 . From (6) and (19), we have

 



lim inflim supln

tX Mtr



 









 (22)

By the property of the inferior limit





 

lim inf

lim inflim inf

nn nn

ab d

acbcd



 







W. H. LI ET AL. 171

and dividing the two sides of (22) by t, we obtain



 

lim inf

rt At



 













 (23)

Let be the set of rational numbers

let

1nk



Q



and

in the interval



00t











  then



1PA





. By

(23), we have



 

lim inf

n

lim inf

Mt r

nt t

AtQ





 



















 (24)

By (11), (21) and (24), we have



lim inf

Xm tr

AtQ











 



 (25)

Because is a continuous function with respect

to on th





e interval





00t and

By (11) and (21), for every , take

suc

 (2

By (26), w



limxt



 





x.



0AA











 h that

 

 





lim n

 

 

 6)

e have



lim inf

kk n

Xm t







A



 









By (26) and (27), we have





(27)



0AA





lim inf

Xm r



 





 (28)

Since , (9) holds by (28). By Jensen

Inequality, we

 (29)

hav





01PA A



obtain



ee e

tE X

tX tm

Mt E

By (11) and (29), wee



00 (30)

txtt





then









 

(31)

By L’hospital rule, we have



lim lnk

lim











 (32)

By (10) and (32), we have









By (10) and (11), we have

(33)



 

Mxx









m

x

 (34)



It is easy to see that





lim 0

xxx











we have

(35)

By (31), (34) and (35),





lim 0





 (36)

Then (12) follows from (34) and (36).

of the Theorem 1,

then

Theorem 2. Under the conditions





lim supkk

nXm r



 





1a s



 (37)

1k

where













inf 0

txt t





  (38)

 

lim sup

nkk

txm xt



 















(39)

and then





0lim0 0







 

The proof of Theorem 2 is similar to that of Theorem 1

is omitted here.

Corollary 1. Let

(40)

and hence







 are independ

variables, then

ent random



lim

nXm





0a s

n



(41)

e that (41) can be obtained by (9) and

(37).

Corollary 2. Under the conditions of th

and Theorem 2, if

n

It is easy to se

e Theorem 1







 are independent and

id eters m and p , then

entically binomial distribution random variables with

param



 

lim s

 





01,as





 



(42)

W. H. LI ET AL.

172



lim inf,

0,a s



 



 

 (43)

Proof. Here, we only proof (42) and (43) is omitted.

The moment generating function of binomial distri-

bution is defined by. Using inequa-

lity and



Mt qp



ln1 0tt t 



ee 1tt



, we have



 

ln Mt

lnee 1

pmp x





mp t

mp xmpmptxmp



 



Hence (44) follows directly from (46).

Corollary 3. Under the conditions of the Theorem

and Theorem 2, if are independen

identically distribution andom variables with

parameters

 

(44)

mp mp



Xn

of Poisson rt and



, then



 

lim supkk

 



mp t









 

(45)







lim inf

0,a s



 



 

 (46)

Proof. The moment generating function of distribution

of Poisson is defined by Using ine-

quality we have







,



ee1

ttt,



tttt











(45) follows directly from (47). At the sa

time, by using inequality we have



 (47)

Hence me



e10

ttt,







ln t

tt t







 .

3. Acknowledgements

The authors would like to thank the National Science

41172299) and the Funding

[1] W. Liu, “Relative Entropy Densities and a Class of Limit

Theorems of Sandom Variables,”

Annals of Prob990, pp. 829-839.

Funding of China (Grant No.

Project Education Department Science Project of Hebei

Province (Grant No. Z2010297).

REFERENC

equence of M-Valued R

ability, Vol. 18, No. 2, 1

doi:10.1214/aop/1176990860

[2] W. Liu, “Strong Deviation Theorems and Analytic Method,”

Science Press, Beijing, 2003.

[3] W. Liu, “A Kind of Strong Deviation Theorems for the

Sequence of Nonnegative Integer-Valued Random Vari-

ables,” Statistics & Probability Letters, Vol. 32, No. 4,

s, Vol. 61, No. 1, 2003, pp. 41-50.

1997, pp. 269-276.

[4] W. Liu, “Some Limit Properties of the Multivariate Func-

tion Sequences of Discrete Random Variables,” Statistics

& Probability Letter

doi:10.1016/S0167-7152(02)00304-8

[5] W. G. Yang and X. Yang, “A Note on Strong Limit Theo-

rems for Arbitrary Stochastic Sequences,” Statistics &

Probability Letters, Vol. 78, No. 14, 200

8, pp. 2018-2023.

doi:10.1016/j.spl.2008.01.084

[6] G. R. Li, S. Chen and J. H. Zhang, “A Class of Random

Deviation Theorems and the Approach of Laplace Trans-

form,” Statistics & Probability Letters, Vol. 79, No. 2,

2009, pp. 202-210. doi:10.1016/j.spl.2008.07.048

[7] G. R. Li, S. Chen and S. Y. Feng, “A Strong Limit Theo-

rem for Functions of Continuous Random Variables and

an Extension of the Shannon-McMillan Theorem,” Jour-

nal of Applied Mathematics, Vol. 2008, 2008, pp. 1-10.

doi:10.1155/2008/639145

[8] W. G. Yang, “Some Limit Properties for Markov Chains

Indexed by a Homogeneous Tree,” Statistics & Probabil-

ity Letters, Vol. 65, No. 3, 2

003, pp. 241-250.

doi:10.1016/j.spl.2003.04.001

[9] W. Liu, “Strong Deviation Theorems and Analytic Method

(in Chinese),” Science Press, Beijing, 2003.

[10] J. L. Doob, “Stochastic Processes,” John Wiley & Sons,

New York, 1953.