ABSTRACT

This paper studies a class of continuous-time two person zero-sum stochastic differential games characterized by linear Itô’s differential equation with state-dependent noise and Markovian parameter jumps. Under the assumption of stochastic stabilizability, necessary and sufficient condition for the existence of the optimal control strategies is presented by means of a system of coupled algebraic Riccati equations via using the stochastic optimal control theory. Furthermore, the stochastic H_∞ control problem for stochastic systems with Markovian jumps is discussed as an immediate application, and meanwhile, an illustrative example is presented.

1. Introduction

The stochastic control problems governed by Itô’s differential equation have become a popular research topic in past decades. Recently, stochastic H_∞ control problem with state and control—dependent noise was considered [1,2]. It has attracted much attention and has been widely applied to various fields. Particularly, the stochastic H₂/H_∞ control with state-dependent noise has been addressed [3,4]. Recently, linear quadratic differential games and their applications have been widely investigated in many literatures, and examples of differential games in economics and management science can be found e.g. in [5-9]. These results are mainly based on the deterministic systems. However, to the best of our knowledge, few results have been obtained for stochastic differential games with Markovian jumps.

In this paper, the stochastic zero-sum games for linear quadratic systems governed by Itô’s differential equations with state-dependent noise and Markovian jumps are addressed, Such class of systems has important applications in engineering practice since they can be used to represent random failure processes in manufacturing systems, electric power systems and so on, see [10-18]. In particular, stability and robust stabilization for such perturbed systems were investigated extensively in [13, 15,17]. A bounded real lemma for Markovian jump stochastic systems was derived in [13]. [11] studied the optimal filtering problem for such systems, while [12,14,16] addressed the issue of linear quadratic regulator. The goal of this paper is to develop the differential game theory for stochastic Itô systems with Markovian jumps, a necessary and sufficient condition is developed for the existence of optimal control strategies in terms of a coupled algebraic Riccati equations (AREs), which can be viewed as an extension of the existing results of [19]. In the end, stochastic H_∞ control problem with Markovian jumps is given as our theoretical applications and an illustrative example is presented.

For convenience, we will make use of the following notations in this paper:

A^T: transpose of a matrix or vector A; A⁻¹: inverse of a matrix or vector A; A > 0 (A ≥ 0): positive definite (positive semidefinite) symmetric matrix A; χ_A: indicator function of a set A;: space of all with A (i) being n × m matrix,;;: space of all n × n symmetric matrices;: space of all with A(i) being n × n symmetric matrix,;   means for;: space of all n-dimensional real vectors with usual 2-norm | • |.

2. Definitions and Preliminaries

Throughout this paper, let be a given filtered probability space where there exists a standard one dimensional Wiener process, and a right continuous homogeneous Markov chain with state space. We assume that is independent of and has the following transition probability:

(1)

where for and. F_t stands for the smallest σ-algebra generated by process, r_s, , i.e..

By denote the space of all measureable functions, which is F_t -measurable for every, and,. Obviously,is a Hilbert space with the inner product

Consider the following linear stochastic controlled system with Markovian jumps

(2)

where and are the state and control input, respectively. The coefficients and with A(i), B(i), C(i), , being constant matrices.

It is well known that for any and, there exists a unique solution of (1) with initial condition,. Next, we first introduce the definition of stochastic stabilizability which is an essential assumption in this paper.

Definition 1. System (2) or (A, B, C) is called stochastic stabilizable (in mean-square sense), if there exists a feedback control withbeing constant matrices, such that for any initial state, , the closed-loop system

is asymptotically mean-square stable, i.e.

Now we give two lemmas which are important in our subsequent analysis. For system (2), by applying Itô’s formula to, we immediately obtain the following result.

Lemma 1. Suppose is given, then for system (2) with initial condition, we have (see Equation (3) below)

Lemma 2 [4]. For system (2), (A, B, C) is stochastic stabilizable if and only if (iff) the following Lyapunov-type equation:

(4)

has a unique positive semidefinite solution .

3. Problem Formulation

Fix. Let be the set of the -valued, square integrable processes adapted with the σ-field generated by, r_t, , respectively. Associated with each is a quadratic cost functional:

(5)

where, , , , , represents the expectation of the enclosed random variable, is the solution to the following linear stochastic differential equation with statedependent noise and Markovian parameter jumps

(6)

In (5) and (6), , etc. whenever. Now we consider the following zero-sum differential game problem.

Problem 1. Given a system described by (6), find, such that

(3)

or equivalently,

That is, there are two players for the differential game. Player 1 chooses control to minimize the objective J, while Player 2 chooses control to maximize J. Now we introduce a new type of coupled algebraic Riccati equations associated with the problem 1.

Definition 2. The following system of algebraic equations

(7)

with

is called a system of coupled algebraic Riccati equations (AREs).

In the next section, we will give our main results of this paper.

4. Main Results

In this section, we will show that the solvability of the AREs (7) is sufficient and necessary for the existence of the optimal control strategies of problem 1.

Theorem 1. Suppose is stochastic stabilizable, problem 1 has a pair of solutions with respect to the initial , where and are the following feedback strategies

respectively, iff the AREs (7) admits a solution. In this case

1)

2)

Proof. Sufficiency: Let

be a solution of the AREs (7). According to lemma 1, we have

By a series of simple computation together with (7), the cost function can be expressed as following

Thus, is minimized by the control strategies and with the optimal value being.

Necessity: Let

be the optimal control strategies to problem 1. Implement and in (6), then

where

According to lemma 2 and the stochastic optimal control theory, we can easily obtain the conclusion that the AREs (7) admit a solution .

So this completes the proof of Theorem 1.

Remark 1. It is interesting to see the specialization of our results in the deterministic case (i.e. for). The corresponding AREs are

(8)

which can be viewed as an extended results of [12].

Remark 2. From Theorem 1 we can see that the derivation of the optimal control strategies for this type of differential games is transformed into deriving the solutions to coupled algebraic Riccati Equations (7), this conclusion be coincident with the results presented in [4], etc.

Remark 3. For the coupled algebraic Riccati Equations (7) may be solved by a standard numerical integration such as LMI method [20], or iterative algorithm similar with the algorithm presented in [21].

5. Application to Stochastic H_∞ Control

Now, we apply the above developed theory to solve some problems related to stochastic H_∞ control. Firstly, we statement the stochastic H_∞ control problem with Markovian jumps, then, we demonstrate the usefulness of the above developed theory in the study of stochastic H_∞ control.

Consider the following controlled system:

(9)

with the cost functional

(10)

where is a right continuous Markov process on a given probability space and the state space and the transition probability described by (1); here is a standard one dimensional Wiener processes. In (9) and (10), is the state vector, is the input control and is the vector of the exogenous disturbances.

The following definition is parallel with the definition 2 presented in [22].

Definition 3. Given disturbance attenuation level γ > 0, the state feedback strategy is said to be an H_∞ control for system (9), if for, , , we have

1) stabilizes system (9) internally, i.e. when, , the state trajectory of (9) with any initial value satisfies

2) with

where for all.

Generally speaking, the H_∞ control problem described by (9) and (10) is to find a control such that for arbitrary exogenous disturbances. As stated in [23], if we view and in the stochastic H_∞ control problem as two control strategies of players P₁ and P₂ from the viewpoint of game theory, the H_∞ control problem can be converted into solving a stochastic game problem, while is in fact the saddle point of this game, e.g.

According to Theorem 1 discussed in Section 4, the following results can be obtained straightly:

Theorem 2. For system described by (9), the stochastic H_∞ control admits a pair of solutions with, , iff the following AREs

(11)

with

has a solution, where

.

In this case, is an H_∞ control for system (9), and is the corresponding worst case disturbance.

Illustrative example: Consider system (9) with the coefficients as follows:

Set, solving (11) via using the algorithm proposed in [21], we have

Therefore, the H_∞ control is given by while; while.

Remark 4. Although we restrict ourselves to single noise stochastic systems throughout the paper, our main theorem still hold for multiple multiplicative noise case. For example, if we replace (6) with

(12)

where, , being independent, one dimensional Wiener processes, then Theorem 1 still holds with AREs (7) replaced by

(13)

with

6. Conclusion

This paper has investigated the linear quadratic zero-sum stochastic differential games with state-dependent noise and Markovian jump parameters in infinite-time horizon, sufficient and necessary conditions for the existence of the optimal control strategies have been obtained, which are expressed in a system of coupled algebraic Riccati equations. The results obtained in this paper extend the existing results of [19]. Throughout this paper, we only have focused on the zero-sum LQ differential games for stochastic systems, while we believe that nonzero-sum LQ differential games still have essential applications, and further studies on such kind of case should be continued.

7. Acknowledgements

This work was supported by the National Natural Science Foundation of China under Grant No. 71171061, the Natural Science Foundation of Guangdong Province under Grant No. S2011010004970.

REFERENCES