Open Journal of Statistics, 2012, 2, 383-388
http://dx.doi.org/10.4236/ojs.2012.24046 Published Online October 2012 (http://www.SciRP.org/journal/ojs)
Edgeworth Approximation of a Finite Sample Distribution
for an AR(1) Model with Measurement Error
Shuichi Nagata
Department of Mathematical Sciences, Kwansei Gakuin University, Sanda, Japan
Received July 25, 2012; revised August 27, 2012; accepted September 9, 2012
ABSTRACT
In this paper, we consider the finite sample property of the ordinary least squares (OLS) estimator for an AR(1) model
with measurement error. We present the Edgeworth approximation for a finite distribution of OLS up to O(12 T). We
introduce an instrumental variable estimator that is consistent in the presence of measurement error. Finally, a simula-
tion study is conducted to assess the theoretical results and to compare the finite sample performances of these estima-
tors.
Keywords: Edgeworth Expansion; OLS; Measurement Error; Instrumental Variables Estimator
1. Introduction
The Ordinary Least Squares (OLS) estimator for the AR(1)
model is one of the most general estimators in economet-
rics, and a number of studies considering the properties
of the OLS estimator under certain conditions have been
conducted by several authors. For example, it is well
known that the OLS estimator for the AR(1) model has a
non-negligible bias when the sample size T is not large.
This problem is known as the small sample problem ([1,
2]).
Another problem of the OLS estimator is that the ob-
servation data are sometimes contaminated by noise,
which also affects the estimation result. In this case, the
OLS estimator in the AR(1) model is not consistent. This
problem is commonly known as the measurement error
problem. Following [3] that summarized the early results
on this topic, numerous articles have been published on
this topic. For example, with respect to time series analy-
sis, some estimators in an AR model with measurement
errors in [4] and statistical a test for the existence of
noise is proposed in [5].
In this paper, we deal with these two important prob-
lems simultaneously. In particular, we consider the OLS
estimation when the sample size T is not large, and when
an measurement error is present but ignored. To evaluate
the effect of a small sample size and ignoring measure-
ment error, we derive finite sample properties of the OLS
estimator with noise using the Edgeworth expansion,
which is a traditional technique in econometrics to ap-
proximate a finite sample distribution. For example, the
OLS estimator was studied in [6] for pure AR(1), in [7]
for AR(1) with an unknown mean, and in [8] for AR(1)
with exogenous variables. Following these studies, we
apply the algorithm proposed in [9] to calculate the
Edgeworth coefficients.
In our setting, some problems are the result of noise,
which make calculation difficult. First, if data are af-
fected by noise, it is difficult to obtain variables that are
related to the autocovariance function of the observation
process. To obtain these variables, we use the result in
[10], which shows that an AR(1) process together with
noise can be represented by an ARMA(1, 1) process.
Second, the OLS estimator is inconsistent with noise, and
it is impossible to apply the formula in [9] in this case;
hence we use a corrected error function that follows [8]
and [11] to avoid this problem.
In the simulation section, we also consider a instru-
mental variable (IV) estimation, which is the consistent
estimation in our setting. We compare the finite sample
performances of the OLS estimator with those of our
proposed IV estimator using simulations.
This paper is organized as follows. In the next section,
we provide our setting and the main result for the Edge-
worth approximation of the OLS estimator up to O(12
T).
In Section 3, we examine a Monte Carlo simulation and
graphical comparison. Finally, Section 4 concludes this
paper.
2. Setup and Main Results
We consider a following measurement error model given
by
C
opyright © 2012 SciRes. OJS

S. NAGATA 384


2
2
.0,,
...0,,
u
dN
iidN



1
,~..
,~
ttt t
tttt
yxuuii
xx 


 (1)
where only yt is observable, xt is a stationary AR(1)
process with 1
 
0,1,,tT
and ut is the measurement error or
noise. For a given sample period , the OLS
estimate can be written as follows:
1
2
ˆ,
yCy
yC y




0,, T
(2)
where
y
yy,
12
1
000
2
10
1000 01
2
10
00 200
1
00 0
2
CC

















  

00
00
.
010
00












 
2
The result of this paper relies on the following well
known result given in [10]. If xt is an AR(1) process with
AR parameter β, and ut is white noise with constant
variance


, then yt follows an ARMA(1, 1) process,
which is given by the following equations:

11,
tt
Ly L  

2
~...0,,
t
iidN

 
2
(3)
where L is the lag operator. The parameters


and γ
(the MA parameter) can be related to β, 2


, and 2


as
follows:
2
114
,,
2
q
q


22
u





where
2
22 2
u
u
1
q



 
.
Then, we obtain the following theorem.
Theorem 1. The finite sample distribution of the OLS
estimator up to O(12
T) is given by

 



2
6,
Q Qw 
1
4
32 61
26 1
ˆ
3
Td
pw
iw P
Iw PPP
TPPP










where
 

12
2
12πexp2 ,iww I
 
d
w
wit t




1,, 6
i
Pi

.
12
42
612
PPP  
 
11Prr
ω, and Q are as follows:
,

  
2
212Prr

,


5244334323
3
232 43
22
44
Prrrr r
rr rrr
,

  


 
4233222422
432232Prrrrrr

,

  
343323 324
5
7552443 65
63549 43105 4
5263 74 85 76
65677485 6
783
36263832
61624122616
203212384
862444
3220262432
823,
Prrrrr
rrrr rr
rrrrr
rr rrr
rrrrr
rr
 

 

,



  
 

 


32 4552
6
224233
6524343
635436 354
10 36
3439
449
2643
Pr rrrr
rrrr
rr rr
rrrrr
 
 
 


 
 
  

322
5252413241
612QPPPPPPPP PP   
,
.
Proof. The proof is given in Appendix.
Here, we also examine the IV estimator, which is de-
fined as follows:
2
2
1
2
T
ii
i
T
ii
i
yy
yy







. (4)
The IV estimator is consistent in the presence of the
noise. It is easy to show that the asymptotic variance of
the IV estimator is


22
1


21


. When there is no noise,
the asymptotic variance of the OLS estimator is 1 – β2.
Therefore, the OLS estimator is more efficient than the
IV estimator in the absence of noise.
3. Simulation and Graphical Comparison
In this section, we examine the finite sample property of
the OLS estimator, and evaluate the approximate distri-
bution generated in Section 2 by Monte Carlo simulation.
Data were simulated from Equation (1) with

.
Therefore, the noise-to-signal ratio 22 2



  
throughout this section. In addition to the OLS estimator,
we also compute the IV estimator defined in the previous
section. The number of replications was 20,000. We
computed the mean (Mean) and the root mean squared
error (RMSE) for each estimator. The simulation results
are summarized in Tables 1-3.
From Tables 1-3, we confirm that the OLS estimator
is biased. As expected, the smaller the sample size and

S. NAGATA 385
Table 3. Simulation results (ρ = 0.7). Table 1. Simulation results (ρ = 0.2).
β = 0.4 β = 0.8
Mean RMSE Mean RMSE
T OLS IV OLS IV OLS IV OLSIV
20 0.31 0.91 0.24 68.73 0.68 0.71 0.231.68
40 0.33 0.58 0.17 45.50 0.71 0.75 0.160.22
100 0.34 0.20 0.12 23.74 0.73 0.78 0.110.10
500 0.34 0.40 0.07 0.13 0.74 0.80 0.070.04
800 0.34 0.40 0.07 0.10 0.74 0.80 0.060.03
β = 0.4 β = 0.8
Mean RMSE Mean RMSE
TOLSIVOLSIV OLS IV OLSIV
200.230.620.28109.62 0.56 0.59 0.32 12.6
400.240.990.2272.55 0.60 0.77 0.262.89
1000.250.370.184.39 0.62 0.78 0.20 0.12
5000.250.400.160.18 0.64 0.80 0.17 0.05
8000.250.400.150.14 0.64 0.80 0.17 0.04
the larger the noise variance, the larger will be the bias.
On the other hand, as the IV estimator is a consistent
estimator, IV may converge to the true value. The simu-
lation results are consistent with this hypothesis. How-
ever, in small samples such as for T = 20 and 40, the
RMSE of the IV estimator is rather large, as seen in all
tables.
Table 2. Simulation results (ρ = 0.4).
β = 0.4 β = 0.8
Mean RMSE Mean RMSE
T OLS IV OLS IV OLS IV OLSIV
20 0.27 -0.06 0.25 48.96 0.63 0.69 0.27 3.20
40 0.29 4.23 0.19 540.2 0.66 0.75 0.20 0.29
100 0.29 0.36 0.15 2.53 0.68 0.78 0.150.10
500 0.30 0.40 0.11 0.15 0.70 0.80 0.110.04
800 0.30 0.40 0.11 0.11 0.70 0.80 0.110.03
From the simulation results for the RMSE in Tables 2
and 3, we find that the IV estimator is more efficient than
the OLS estimator when T ≥ 800 (β = 0.4) and T ≥ 100 (β
= 0.8). Therefore, we can conclude that, if the sample
size is not large (T = 20, 40), or both β and ρ are small as
in Table 1 (β = 0.4 and ρ = 0.2), then the OLS estimator
is better than IV in terms of the RMSE.
Next, we compare the exact cdf with the asymptotic
normal distribution. Figure 1 depicts the approximate
distributions of the OLS and the IV with β = 0.4, T = 20,
and various values of ρ. Figure 1 indicates that the OLS
Figure 1. Exact distributions of OLS and IV.

S. NAGATA 386
values have a downward bias. The IV exhibits good be-
havior in the central region of the distribution; however,
its distribution is fatter in the tails compared to the nor-
mal distribution.
Finally, we evaluate the approximate distributions ob-
tained in Section 2, and compare them with the exact cdf
and asymptotic normal distributions. To enable a com-
parison of the shapes of the distributions, the asymptotic
bias of the OLS estimator is corrected hereinafter. Fig-
ure 2 shows the approximate distributions of the OLS
estimator with T = 20, ρ = 0.2, and three different values
of β. From this figure, we can observe the same result as
those obtained in [6]. Figure 3 depicts the approximate
distributions of the OLS with ρ = 0.7, where the other pa-
Figure 2. Exact and approximate distributions of OLS.
0.8
Figure 3. Exact and approximate distributions of OLS.

S. NAGATA 387
rameter values are the same as those for Figure 2. We
note that the shapes of the distributions are almost the
same, even if the noise ratio ρ is changed. From these
figures, the noise variance has only a small effect on the
shape of the OLS distribution.
4. Discussion
In this paper, we considered finite sample properties of
the OLS estimator for the AR(1) model with measure-
ment error. Using the formula in [9], we obtained the
Edgeworth expansion for finite sample distributions of
the OLS estimator up to O(12 T).
In the simulation work, we have compared naive OLS
estimator with the IV estimator which is a consistent es-
timator in the presence of noise. We can confirm that,
even if the measurement errors is exist, the OLS estima-
tor is more efficient than the IV estimator when the sam-
ple size is small such as T = 20 and 40. If the noise-
to-signal ratio is not so small (ρ ≥ 0.4), the IV estimator
is more efficient than the OLS estimator when T ≥ 800 (β
= 0.4) or T ≥ 100 (β = 0.8). From the graph of the nor-
malized OLS distributions, we find similar properties to
those of the distributions, which correspond to the no
noise situation examined by [6]. This result implies that
measurement error mainly distorts the OLS distributions
for mean and variance; hence we can separately deal with
the two problems of small sample size and observation
errors.
Recently, the differenced-AR(1) estimator was dis-
cussed in [12,13]. Even if the sample size T is relatively
small, this estimator has a small bias. To obtain the finite
sample distribution and to examine the robustness of
such estimators with respect to observation errors, we
can apply the technique of this paper, and this will be
dealt with in a future study.
5. Acknowledgements
I am grateful to Professor Koichi Maekawa for his guid-
ance on this topic and his valuable comments on this
paper. I am also grateful to Professor Yasuyoshi Tokutsu
for his valuable comments.
REFERENCES
[1] F. H. C. Marriott and J. A. Pope, “Bias in the Estimation
of Autocorrelations,” Biometrika, Vol. 41, No. 3-4, 1954,
pp. 390-402. doi:10.2307/2332719
[2] M. G. Kendall, “Note on the Bias in the Estimation of
Autocorrelation,” Biometrika, Vol. 41, No. 3-4, 1954, pp.
403-404. doi:10.2307/2332720
[3] W. A. Fuller, “Measurement Error Models,” John Wiley,
New York, 1987. doi:10.1002/9780470316665
[4] J. Staudenmayer and P. Buonaccorsi, “Measurement Er-
ror in Linear Autoregressive Model,” Journal of the Ameri-
can Statistical Association, Vol. 100, No. 471. 2005, pp.
841-851. doi:10.1198/016214504000001871
[5] K. Tanaka, “A Unified Approach to the Measurement
Error Problem in Time Series Models,” Econometric The-
ory, Vol. 18, No. 2, 2002, pp. 278-296.
doi: 10.1017/S026646660218203X
[6] P. C. B. Phillips, “Approximations to Some Sample Dis-
tributions Associated with a First Order Stochastic Dif-
ference Equation,” Econometrica, Vol. 45, No. 2, 1977,
pp. 463-485. doi.org/10.2307/1911222
[7] K. Tanaka, “Asymptotic Expansions Associated with the
AR(1) Model with Unknown Mean,” Econometrica, Vol.
51, No. 4, 1983, pp. 1221-1231. doi:10.2307/1912060
[8] K. Maekawa “An Approximation to the Distribution of
the Least Squares Estimator in an Autoregressive Model
with Exogenous Variables,” Econometrica, Vol. 51, No.1,
1983, pp. 229-238. doi:10.2307/1912258
[9] J. D. Sargan, “Econometric Estimators and the Edgeworth
Expansions,” Econometrica, Vol. 44, No. 3, 1976, pp.
421-448. doi:10.2307/1913972
[10] C. W. J. Granger and M. J. Morris, “Time Series Model-
ing and Interpretation,” Journal of the Royal Statistical
Society A, Vol. 139, No. 2, 1976, pp. 246-257.
doi:10.2307/2345178
[11] K. Maekawa, “Edgeworth Expansion for the OLS Esti-
mator in a Time Series Regression Model,” Econometric
Theory, Vol. 1, No. 2, 1985, pp. 223-239.
doi:10.1017/S0266466600011154
[12] K. Hayakawa, “A Note on Bias in First-Differenced AR(1)
Models,” Economics Bulletin, Vol. 3 No. 27, 2006, pp. 1-
10.
[13] P. C. B. Phillips and C. Han, “Gaussian Inference in
AR(1) Times Series with or without Unit Root,” Econo-
metric Theory, Vol. 24, No. 3, 2008, pp. 631-650.
doi:10.1017/S0266466608080262

S. NAGATA 388
Appendix
Proof of Theorem 1
The OLS estimator for β is given by Equation (2). Intro-
ducing


ii
, we can write the derivation of
the estimation as follows:
EyCy

 




2
2
2
.
uT 11 2
22
ˆyC yyCy
yC y
 


 


 


In addition, we introduce iii
and
. Then, we have the error function as;
qyCy T 



12
,qqq


2
qq
12
22
ˆ.
u
qT

 
   
In order to develop the Edgeworth expansion, we de-
fine a modified function e(q):

222
2 2
uuu
T
12
22
ˆTq q
eq qT


 

 


.
It is easy to obtain the cumulant generating function of
12
Tq is
 
1122
IC C 

 



,
11 22
12
log det
2
,
T
TT
 
 
 

where θj = itj and 12




2
1
34
,,
,,
.
. The matrices I and Σ are
the identity matrix and the covariance matrix of y, re-
spectively.
j
kjkjkljkl
j jkkjk jk
aajj
ee eee
ee
e
 
 

 


Edgeworth expansion requires the partial derivatives
of e(q) and φ(θ) up to the third order. In the current paper
these derivatives are denoted as ej, ejk, ψjk and ψjkl, which
are all evaluated at the origin. Using the tensor summa-
tion convention, Edgeworth coefficients of Sargan’s
formula are obtained by these derivatives as follows:
Although we only show Edgeworth coefficients re-
lated to approximations of up to O(12 T), the original
formula of Sargan can approximate up to O(T), see [6].
After many calculations, we finally obtain the Edge-
worth coefficients. To save space, we show only the re-
sults as follows:



22
15 324114
13
74
63
2
2
261
41
432 42
22
21 42
,,
4,,
iP PPPPPPP
P
PT
PP
PP
PP






 


where Pi and Q are polynomials defined in Section 2.
The approximation of the OLS estimator up to O(12 T) is
derived in [6] as follows.


2
02 ,
xx x
pTeqxIicc
 


 
 

 
 


(5)
where c0 and c2 are
33 41 1
02
33
3
1
6 6
22 2
ii
cc
TT T

 



 


.
Using these results, we obtain the following equations.


 
1461 1
02
261 26
3
1
32 2
6
33
PPPPQPQ
cc
PPP TPPP T 


Substituting these results into Equation (5), we obtain
Theorem 1.