^{1}

^{*}

^{1}

Theoretical results related to properties of a regularized recursive algorithm for estimation of a high dimensional vector of parameters are presented and proved. The recursive character of the procedure is proposed to overcome the difficulties with high dimension of the observation vector in computation of a statistical regularized estimator. As to deal with high dimension of the vector of unknown parameters, the regularization is introduced by specifying a priori non-negative covariance structure for the vector of estimated parameters. Numerical example with Monte-Carlo simulation for a low-dimensional system as well as the state/parameter estimation in a very high dimensional oceanic model is presented to demonstrate the efficiency of the proposed approach.

In [

where

It is assumed

where

gers, the covariance matrix of the joint vector

No particular assumption is made regarding the probability density function of

In [

where

As in practice all the matrices

hence the resulting estimate

As shown in [

In this paper we are interested in obtaining a simple recursive algorithm for computation of

This problem is very important for many practical applications. As example, consider data assimilation problems in meteorology and oceanography [

First consider the model (1) and assume that

and

Suppose that the system (6) is compatible, i.e. there exists

The problem is to obtain a simple recursive procedure to compute a solution of the system (6) when the dimension of

To find a solution to Equation (6), let us introduce the following system of recursive equations

Mention that the system is compatible if

Theorem 1. Suppose the system (6) is compatible. Then for any finite

In order to prove Theorem 1 we need

Lemma 1. The following equalities hold

Proof. By induction. We have for

since

Let the statement be true for some

Substituting

Lemma 2. The following equalities hold

Proof. By induction. We have for

Let the statement be true for some

However from Lemma 1, as

Proof of Theorem 1.

Theorem follows from Lemma 2 since under the conditions of Theorem, from Equation (9) for

Corollary 1. Suppose the rows of

Proof. By induction. The fact that for

Suppose that

its subspace

independent elements. We conclude that it is impossible

Suppose now Corollary is true for

we have

By the same way as proved for

Comment 1. By verifying the rank of

Using the result (8) in Lemma 1, it is easy to see that:

Corollary 2. Suppose

Corollary 3. Suppose

Corollary 3 follows from the fact that when

Comment 2. Equation (7c) for

In the next section we shall show that the fact

Theorem 1 says only that Equations (7a)-(7c) give a solution to (1). We are going now to study the question on whether a solution of Equations (7a)-(7c) is optimal, and if yes, in what sense? Return to Equations (1)-(3) and assume that

where

and one can prove also that the mean square error for

If we apply Equation (3.14.1) in [

For simplicity, let

In [

We prove now a more strong result saying that all the estimates

Theorem 2. Consider the algorithm (7). Suppose

Proof. For

Suppose the statement is true for some

Really as

Theorem 2 says that by specifying

Comment 3

1) If

2) Theorem 2 says that there is a possibility to regularize the estimate when the number of observations is less than the number of estimated parameters by choosing

Suppose

Theorem 3. Suppose

Proof. Introduce for the system (6),

and the class of all estimates

The condition for unbiasedness of

or

from which follows

It means that all the estimate in Equation (14) is unbiased.

Consider the minimization problem

We have

Taking the derivative of

from which follows one of the solutions

If now instead of Equation (13) we consider the system

and repeat the same proof, one can show that the unbiased minimum variance estimate for

Using the properties of the pseudo-inverse (Theorem 3.8 [

Thus for a very small

On the other hand, applying Lemma 1 in [

Letting

The algorithm (18a)-(18d) thus yields an unbiased minimal variance (UMV) estimates for

In the field of data assimilation in meteorology and oceanography usually the state vector

Let us consider the eigen-decomposition for

In (19) the columns of

Theorem 4

Consider two algorithms of the type (7) subject to two matrices

where the columns of

Proof

Write the representation of

where

Let

In what follows we introduce the notation:

The samples

There are the following different cases

1)

2)

a) For samples belonging to

b) For samples belonging to

Thus in the mean sense

3) Consider two initializations

a)

i)

ii)

b)

i) of minimal variance for

ii) of minimal variance for

iii) not of minimal variance for

Thus in the mean sense

Theorem 5

Consider the algorithm (7) subject to

Then this algorithm can be rewritten in the form

It is seen that in the algorithm (24a)-(24d), the estimate

Mention that the version (24a)-(24d) is very closed to that studied in [

Consider the system (1) subject to the covariance

Here we assume that the 1st and 3rd components of

Numerical computation of eigendecomposition of

The algorithm (24a)-(24d) is applied subject to three covariance matrices

In

There are 100 samples simulating the true

The curve

As seen from

The results produced by ALG (1) and ALG (4) show also that when the priori information is insufficiently rich, the algorithm naturally produces the estimates of poor quality. In such situation, simple applying orthogonal projection can yield a better result. For the present example, the reason is that using the 2nd mode

In this section we will show importance of the regularization factor in the form of the priori covariance

The numerical model used in this experiment is the MICOM (Miami Isopycnic Coordinate Ocean Model) which is exactly as that presented in [

The assimilation experiment consists of using the SSH to correct the model solution, which is initialized by some arbitrarily chosen state resulting from the control run.

The different filters are implemented to estimate the oceanic circulation. It is well known that determining the filter gain is one of the most important tasks in the design of a filter. As for the considered problem it is impossible to apply the standard Kalman filter [

According to Corollary 4.1 in [

where

We remark that all the gain coefficients in two filters are of identical sign but the elements of

The vertical gain coefficients for the CHF are taken from [

In

For the velocity estimates, the same tendency is observed as seen from

errors. These results prove that the choice of ECM as a regularization factor on the basis of members of the En(PEF) allows to much better approach the true system state compared to that based on the samples taken from En(EnOI) or to that constructed on the basis of the physical consideration as in the CHF. The reason is that the members of En(PEF) by construction [

We have presented some properties of an efficient recursive procedure for computation of a statistical regularized estimator for the optimal linear estimator in a linear model with arbitrary non-negative covariance structure. The main objective of this paper is to obtain an algorithm which allows overcoming the difficulties concerned with high dimensions of the observation vector as well as that of the estimated vector of parameters. As it was seen, the recursive nature of the proposed algorithm allows dealing with high dimension of the observation vector. By initialization of the associated matrix equation by a low rank approximation covariance which accounts for only first leading components of the eigenvalue decomposition of the priori covariance matrix, the proposed algorithm permits to reduce greatly the number of estimated parameters in the algorithm. The efficiency of the proposed recursive procedure has been demonstrated by numerical experiments, with the systems of small and very high dimension.