^{1}

^{*}

^{2}

^{*}

In this study, by starting from Maximum entropy (MaxEnt) distribution of time series, we introduce a measure that quantifies information worth of a set of autocovariances. The information worth of autocovariences is measured in terms of entropy difference of MaxEnt distributions subject to different autocovariance sets due to the fact that the information discrepancy between two distributions is measured in terms of their entropy difference in MaxEnt modeling. However, MinMaxEnt distributions (models) are obtained on the basis of MaxEnt distributions dependent on parameters according to autocovariances for time series. This distribution is the one which has minimum entropy and maximum information out of all MaxEnt distributions for family of time series constructed by considering one or several values as parameters. Furthermore, it is shown that as the number of autocovariances increases, the entropy of approximating distribution goes on decreasing. In addition, it is proved that information worth of each model defined on the basis of MinMaxEnt modeling about stationary time series is equal to sum of all possible information increments corresponding to each model with respect to preceding model starting with first model in the sequence of models. The fulfillment of obtained results is demonstrated on an example by using a program written in Matlab.

In many instances, the type of data available for modeling and that used for optimization is a set of observations measured over time of system variable(s) of interest [_{max}. It can be shown that as the number of constraints generated by autocovariances increases, value of H_{max} decreases. In this investigation, firstly MaxEnt distribution for stationary time series subject to constraints generated by autocovariances set

In this section, MaxEnt distributions according to different number of autocovariances are considered and it is proved that the entropy values of these distributions constitute a monotonically decreasing sequence when the number of autocovariances increases. Moreover it is shown that the information generated by autocovariances set is expressed as sum of information worth of each autocovariance taken separately.

Theorem 1. Let

Proof. The Shannon entropy measure subject to constraints generated by autocovariances

If we denote by _{k}, due to the fact that the information discrepancy between two distributions is measured in terms of their entropy difference in MaxEnt modeling, then

Furthermore, if information worth generated by autocovariances set

Remark 1. The information

From (3) by virtue of formula (2) follows

consequently

In this section, according to different number of autocovariances MaxEnt distributions dependent on parameters are considered and it is proved that at each value of parameter, these distributions and their entropies possess the same properties as in section 2.

Theorem 2. Let

Between entropy values

In other words, entropy values of MaxEnt distributions dependent on

Proof. According to Theorem 1, entropy values

Information worth _{k} dependent on

Then, information worth generated by autocovariances set

Remark 2. The information

From (7) by virtue of formula (6) follows

In this section, MinMaxEnt distributions (models) are obtained on the basis of MaxEnt distributions dependent on parameters and it is shown that as the number of autocovariances k goes on increasing, the entropy of approximating distribution (model) goes on decreasing. Furthermore, it is proved that information worth of each model defined on the basis of MinMaxEnt modeling about stationary time series is equal to the sum of all possible information increments corresponding to each model with respect to preceding model starting with first model in the sequence of models.

Theorem 3. Let

Then, between entropy values of MinMaxEnt distributions the inequalities

are satisfied.

Proof. According to Theorem 2 for any

On the other hand,

From inequality (11) by taken into account (12) and (13), the inequality

is got. If this process is consecutively repeated, then it is easy to get to the inequalities (10). Theorem 3 is proved.

Remark 3. By using Theorem 3, it is possible to obtain information worth of MinMaxEnt distributions with the different number of autocovariances.

By using Theorem 3, it is possible to obtain information worth of MinMaxEnt distributions with the different number of autocovariances. However, in order to simplify the description of results, we introduce the following symbols. Let

and

From (15) and (16),

where

Theorem 4. Information worth _{m} defined on the basis of MinMaxEnt modelling about stationary time series

Proof. By using the new notations

Equation (10) shows that as the number of autocovariances k increases, the entropy of approximating distribution (model) goes on decreasing but it never goes below the entropy of probability distribution satisfying the same conditions as MinMaxEnt distribution. According to (15) and (17)

or

According to (18) in (19),

The developed MinMaxEnt models

and the data set is given in

estimations with autocovariances is 0.2564 and it is lower than

Ent estimations with

Furthermore, in _{max} decreases.

t | t | ||||||
---|---|---|---|---|---|---|---|

1 | −7.6164 | −3.7049 | −4.5863 | 26 | −2.5809 | −0.4340 | −2.2861 |

2 | −7.9251 | −5.9152 | −8.2637 | 27 | −1.8546 | 0.0249 | −1.8094 |

3 | −2.3466 | −3.1335 | −1.5912 | 28 | 4.7113 | 2.7239 | 4.5856 |

4 | −1.0788 | −2.6884 | −1.1953 | 29 | 5.2406 | 2.9464 | 5.1481 |

5 | −6.3050 | −4.4728 | −6.1961 | 30 | −1.0943 | 0.4262 | −0.8107 |

6 | −7.7206 | −4.9192 | −7.7193 | 31 | −2.4052 | −0.1378 | −2.3785 |

7 | −2.2376 | −2.6242 | −2.2308 | 32 | 4.0709 | 3.0661 | 4.3309 |

8 | 0.33865 | −1.6810 | −0.0090 | 33 | 7.9505 | 4.9433 | 7.7333 |

9 | −4.5611 | −3.2248 | −4.3121 | 34 | 3.5777 | 3.7644 | 3.5249 |

10 | −7.3435 | −4.7417 | −7.6510 | 35 | 0.8252 | 3.1348 | 0.8623 |

11 | −3.3723 | −2.9111 | −3.2169 | 36 | −2.4052 | −0.1378 | −2.3785 |

12 | 0.13548 | −1.8088 | −0.0447 | 37 | 4.0709 | 3.0661 | 4.3309 |

13 | −3.7786 | −3.4259 | −3.7174 | 38 | 7.9505 | 4.9433 | 7.7333 |

14 | −8.2637 | −5.3028 | −8.1113 | 39 | 3.5777 | 3.7644 | 3.5249 |

15 | −5.2458 | −4.0749 | −4.8305 | 40 | 0.8252 | 3.1348 | 0.8623 |

16 | −0.2230 | −2.2286 | −0.1069 | 41 | 11.292 | 8.1259 | 11.159 |

17 | −2.1272 | −2.2977 | −1.8858 | 42 | 7.5889 | 6.6536 | 7.3807 |

18 | −5.4257 | −2.6645 | −5.2509 | 43 | 3.3139 | 5.2987 | 3.5224 |

19 | −1.0920 | −0.1997 | −1.3106 | 44 | 6.5842 | 6.0319 | 6.2192 |

20 | 5.5526 | 3.1233 | 5.2295 | 45 | 10.412 | 7.5539 | 10.267 |

21 | 4.5110 | 3.1064 | 3.9525 | 46 | 7.2051 | 6.1065 | 7.1059 |

22 | −0.8572 | 1.2503 | −0.9899 | 47 | 2.0869 | 3.6081 | 2.1044 |

23 | 0.0716 | 1.1921 | −0.2413 | 48 | 3.1468 | 3.5739 | 3.1619 |

24 | 4.7447 | 2.7488 | 4.8959 | 49 | 7.1153 | 5.0748 | 7.1544 |

25 | 3.4163 | 1.6973 | 3.3217 | 50 | 5.9239 | 4.4184 | 4.9659 |

171.94 | 155.74 | 153.80 | 117.37 | 16.20 | 1.94 | 36.43 | 54.57 |

In this study, the following results are established.

・ MaxEnt distributions according to different number of autocovariances are considered and it is proved that the entropy values of these distributions constitute a monotonically decreasing sequence when the number of autocovariances increases. Moreover it is shown that the information generated by autocovariances set is expressed as sum of information worth of each autocovariance taken separately.

・ According to different number of autocovariances, MaxEnt distributions dependent on parameters are considered and it is proved that at each value of parameter these distributions and their entropies possess the same properties as the MaxEnt distributions.

・ MinMaxEnt distributions (models) are obtained on the basis of MaxEnt distributions dependent on parameters and it is shown that as the number of autocovariances k goes on increasing, the entropy of approximating distribution (model) goes on decreasing. Furthermore, it is proved that information worth of each model defined on the basis of MinMaxEnt modeling about stationary time series is equal to the sum of all possible information increments corresponding to each model with respect to preceding model starting with first model in the sequence of models.

・ Information worth of autocovariances in time series and values generating MinMaxEnt distributions can be applied in solving many problems. One of the mentioned problems is the problem of estimation of missing value in time series. It is proved that the value generating MinMaxEnt distribution independence on position represents the best estimation of the missing value in the sense of information worth.

・ The fulfillment of the obtained results is demonstrated on an example by using a program written in Matlab.

We thank the Editor and the referee for their comments. This support is greatly appreciated.